The University of Waterloo Libraries will be performing maintenance on UWSpace tomorrow, November 5th, 2025, from 10 am – 6 pm EST.
UWSpace will be offline for all UW community members during this time. Please avoid submitting items to UWSpace until November 7th, 2025.

A Machine Learning Approach for RDP-based Lateral Movement Detection

Loading...
Thumbnail Image

Authors

Bai, Zhenyu

Advisor

Raouf, Boutaba

Journal Title

Journal ISSN

Volume Title

Publisher

University of Waterloo

Abstract

Detecting cyber threats has been an on-going research endeavor. In this era, advanced persistent threats (APTs) can incur significant costs for organizations and businesses. The ultimate goal of cybersecurity is to thwart attackers from achieving their malicious intent, whether it is credential stealing, infrastructure takeover, or program sabotage. Every cyberattack goes through several stages before its termination. Lateral movement (LM) is one of those stages that is of particular importance. Remote Desktop Protocol (RDP) is a method used in LM to successfully authenticate to an unauthorized host that leaves footprints on both host and network logs. In this thesis, we propose to detect evidence of LM using an anomaly-based approach that leverages Windows RDP event logs. We explore different feature sets extracted from these logs and evaluate various supervised and unsupervised machine learning (ML) techniques for classifying RDP sessions with high precision and recall. We also compare the performance of our proposed approach to a state-of-the-art approach and demonstrate that our ML model outperforms in classifying RDP sessions in Windows event logs. In addition, we demonstrate that our model is robust against certain types of adversarial attacks.

Description

LC Subject Headings

Citation