A Digital Twin-driven Deep Reinforcement Learning Approach for Smart Workshop Scheduling

Abstract

Workshop scheduling is a fundamental challenge in industrial engineering, requiring sophisticated strategies to manage increasing production complexity. This study proposes an integrated framework that combines Digital Twin (DT) technology with Deep Reinforcement Learning (DRL) to achieve autonomous and adaptive scheduling. While DT technology enables high-fidelity, real-time synchronization between physical shop floors and virtual models, DRL provides the intelligent decision-making framework necessary for dynamic optimization. By mapping physical assets into a synchronized digital environment, the DT provides a risk-free simulation bed for the training and validation of DRL agents. This approach facilitates real-time feedback loops, allowing the system to respond to stochastic disturbances while maximizing resource utilization and minimizing production costs. In this research, DRL algorithms are deployed to optimize multi-objective scheduling tasks, leveraging the DT to provide a low-cost, high-efficiency environment for iterative learning. The results demonstrate that this synergy effectively addresses complex, non-linear decision-making problems that traditional heuristic methods often fail to resolve. This study utilizes an improved DQN algorithm (with dual networks and prioritized experience replay) to address practical manufacturing challenges like machine breakdowns and urgent order insertions. By leveraging the Digital Twin for real-time synchronization, the model adaptively re-schedules resources, achieving a 14.6% reduction in makespan and a 12.3% improvement in resource utilization compared to traditional rules. The results confirm the system's resilience, maintaining an 86.5% equipment utilization rate under stochastic disturbances. Ultimately, the integration of DT and DRL provides a robust foundation for the realization of smart manufacturing and resilient production systems, offering a scalable solution for modern industrial environments.