Q-learning for outbound container stacking at container terminals

Abstract

The efficient stacking of outbound containers presents a significant challenge within container terminal operations. It's crucial to minimize the antic-ipated need for rehandling, as this directly impacts yard productivity and overall terminal efficiency. To address this challenge, we introduce a novel approach based on reinforcement learning. Our method employs Q-learning, incorporating Monte Carlo techniques to identify optimal storage locations by maximizing re-ward values. Furthermore, we've developed effective strategies for determining storage placements through extensive training iterations. Through numerical ex-perimentation using real-world container terminal data, we've compared our model with existing algorithms. Numerical results highlight the robustness of our approach in navigating uncertain operational environments, its ability to support real-time decision-making, and its effectiveness in minimizing rehandling re-quirements.