A

In recent years, swarm robotics has emerged as a transformative paradigm for accomplishing complex tasks through the coordinated operation of large numbers of simple, low-cost robots [1], [2], [3]. Unlike traditional single-robot systems that rely on sophisticated sensing, computation, and actuation capabilities concentrated in individual units, swarm robotics distributes functionality across many agents, offering significant advantages in robustness (graceful degradation under failures), flexibility (adaptation to diverse tasks), and scalability (performance improvement with additional robots) [4]. These characteristics make swarm robotics particularly attractive for applications in warehouse automation [5], search and rescue operations [6], environmental monitoring [7], and precision agriculture, where tasks can be naturally decomposed and parallelized across multiple agents.

However, as swarm size grows, target congestion, where multiple robots simultaneously converge on shared goal locations, becomes a critical bottleneck that can degrade throughput by up to 70% and induce system-wide deadlock [8], [9], [10]. This problem is distinct from general collision avoidance: it is spatially concentrated at predictable locations, temporally dynamic with stochastic arrival patterns, and exhibits non-linear degradation with sharp phase transitions beyond critical density thresholds. Existing approaches, centralized planning, reactive avoidance, spatial partitioning, and learning-based methods, each address only a subset of these characteristics, leaving a significant gap in scalable, proactive congestion prevention.

The resulting framework is theoretically grounded and empirically validated, rather than provably optimal.

Our proposed framework is inspired by human traffic management systems, such as traffic lights, flow metering, and lane control, as well as transportation theory, including queuing models and flow dynamics. These concepts inform our method for regulating robot entry into congested regions while maintaining smooth bidirectional flows. Specifically, we address key limitations in existing approaches as follows. (i) Flow regulation is implemented through an RFID-based entry and exit control module, which manages robot admission based on real-time density measurements rather than relying on post-hoc collision avoidance. (ii) Predictive coordination is achieved using a DQN that learns anticipatory policies, such as proactively slowing before arrival when target occupancy approaches capacity, by optimizing cumulative discounted rewards over a planning horizon. (iii) Entry and exit balance is maintained through zone geometry design, which features separated entry and exit sectors and distributed exit angles. (iv) Scalability is ensured by the CTDE architecture: training utilizes the global state, while execution is fully decentralized with constant per-robot computation and linear communication complexity. (v) Automatic parameter tuning replaces manual threshold selection, as the DQN jointly optimizes admission thresholds, detour triggers, and speed-adjustment parameters through reward maximization. (vi) Theoretical grounding is established through formal DTMDP convergence analysis under tabular assumptions (Theorem 1, Remark 1), with explicit acknowledgment of the gap to practical DQN and supporting empirical convergence evidence.

The central insight of this framework is the modeling of target congestion as a DTMDP, in which system states represent robot distributions across behavioral modes (waiting, navigating, following, detouring, agitated) and target-area density. Concurrently, actions regulate admission policies and behavioral triggers via an RFID-based coordination module. By employing DQN to learn high-quality policies, the system adaptively balances competing objectives: maximizing throughput, minimizing waiting times, reducing energy consumption, and preventing deadlocks. Fig. 1 presents an overview of the complete system architecture.

The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 introduces the DQL framework for multi-robot anti-congestion navigation. Section 4 presents a comprehensive simulation-based evaluation. Section 5 discusses the theoretical contributions and practical implications. Finally, Section 6 concludes the paper and outlines directions for future research.

Related work

Multi-robot congestion control occupies a central position among several interrelated research domains. This section first formally characterizes the congestion problem and differentiates it from traditional collision avoidance. Subsequently, it reviews four principal paradigms addressing this challenge: centralized navigation planning, reactive collision avoidance, spatial partitioning strategies, and learning-based adaptive methods. For each paradigm, methodological strengths and inherent

Proposed deep Q-learning framework for multi-robot anti-congestion navigation

This section presents our DQL framework that integrates RFID-based coordination with adaptive reinforcement learning to solve target congestion in large-scale robot swarms. The framework consists of three integrated components: (1) a rigorous DTMDP formulation that captures robot behaviors and congestion dynamics, (2) an RFID-based information process module for real-time state estimation and command broadcasting, and (3) DQN learning for high-quality policy discovery with RCRs that translate

Simulation evaluation

To validate the proposed DQN-based anti-congestion navigation framework, we conducted comprehensive simulations.

DQN training analysis

Fig. 5 illustrates the complete training dynamics of our DQN-based framework over 1000 episodes. The four subplots demonstrate successful convergence across multiple metrics.

The training progression reveals four distinct learning phases. Subplot (a) shows episode rewards converging from highly negative values (exploration phase, episodes 0-200) to consistently positive rewards (stable policy phase, episodes 800-1000). The 50 episode moving average smooths short-term variance and clearly

Conclusion

This paper presented a novel Deep Q-Learning based framework for multi-robot anti-congestion navigation that explicitly targets flow regulation at shared destinations rather than reactive collision avoidance. By integrating learning-based decision making with structured, rule-constrained coordination, the proposed approach addresses the dual challenges of scalability and safety in large-scale robot swarms.

The primary contributions of this work are fourfold. First, we introduced a CTDE

CRediT authorship contribution statement

Ning Wang: Writing – original draft, Methodology, Formal analysis, Data curation, Conceptualization. Hamid R. Parsaei: Writing – review & editing, Supervision, Conceptualization. Yali Ren: Writing – review & editing, Validation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.