Autonomous Articulated Vehicle Control via Deep Reinforcement Learning and High-Dimensional Perception
Loading...
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Waterloo
Abstract
Autonomous control of articulated tractor-trailer vehicles is difficult because the passive hitch adds a revolute degree of freedom whose reverse dynamics are open-loop unstable, driving the trailer toward the jackknife regime. This thesis develops and evaluates a learning-based controller for this setting, targeting the confined low-speed maneuvers typical of distribution yard and loading-dock operations, and carries it from simulation onto physical hardware.
A Twin Delayed Deep Deterministic Policy Gradient (TD3) agent is trained in a custom Gymnasium simulation of a shunt-truck tractor-trailer. Controlled ablations isolate the learning algorithm, reward formulation, and observation modality. TD3 under a multiplicative reward that jointly penalizes tracking error and hitch articulation produces the highest and most consistent reverse completion while also reducing tracking and articulation errors. A compact engineered state representation is more reliable than higher-dimensional LiDAR or bird’s-eye-view (BEV) image observations under an otherwise identical training recipe. Training a policy directly on the raw image fails, but controlled experiments show that the image contains sufficient task-relevant information and identify the off-policy actor-critic training process as the primary source of failure. Geometry distillation addresses this failure by training a supervised encoder to recover the vehicle’s path geometry from the BEV image and freezing that encoder during policy optimization. The resulting policy restores reliable from-scratch learning to near engineered-state performance in both forward and reverse lane following. The same approach extends to obstacle navigation, where the image-derived representation provides both an avoidance path and a measure of layout difficulty, allowing the learned controller to follow feasible detours and stop when a layout should not be attempted.
The geometry-encoded BEV policies are deployed on a physical articulated platform, an AgileX Hunter SE with a scale trailer. Deployment uses a stripped-down Autoware stack whose planning and control layers are replaced by an RL bridge that reproduces the observation and action interfaces used during training. Four formal lane-following trials, two forward and two reverse, were conducted and all reached the goal under policy control. Forward tracking transferred with low error, while reverse tracking exhibited a sustained, steering-rate-limited articulation oscillation that was absent from simulation. The hardware results therefore demonstrate physical transfer of the learned controller while identifying reverse articulation dynamics as the dominant remaining sim-to-real limitation.