Overview

The system employs a Central Nervous System (LLM) to process textual inputs and issue commands to a scheduler, which optimizes task allocation and monitors robot states. The scheduler coordinates unit robots that autonomously perform tasks such as SLAM for mapping and localization, stereo depth sensing, and instance segmentation to generate a semantic map, which updates the map interpreter to inform the scheduler of environmental changes. The robots handle task-specific navigation, manipulation, and exploration, enabling dynamic and coordinated swarm navigation.

Autonomous mapping of enviornment

Autonomous Goal assignment and planning in swarm robots

Installations

ROS2-Humble & Gazebo Installation

1. To install ROS2-humble, open terminal and run the following commands

sudo apt update
sudo apt upgrade

2. Set locale

locale  # check for UTF-8

sudo apt update && sudo apt install locales
sudo locale-gen en_US en_US.UTF-8
sudo update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
export LANG=en_US.UTF-8

locale  # verify settings

3. Setup Sources

sudo apt install software-properties-common
sudo add-apt-repository universe

4. Add ROS 2 GPG key

sudo apt update && sudo apt install curl -y
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg

5. Add repository to sources

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

6. Install ROS 2 packages

sudo apt update
sudo apt upgrade

sudo apt install ros-humble-desktop
sudo apt install ros-dev-tools

7. Gazebo install

sudo apt install ros-humble-gazebo-ros
sudo apt install ros-humble-gazebo-ros-packages

8. Enviornment setup

echo "source /opt/ros/humble/setup.bash" >> ~/.bashrc

Extraction

1. Install unzip utility

sudo apt-get install unzip

2. Unzip the file in home directory

unzip ~/Downloads/Mobile-Swarm-Navigation.zip -d ~

Building workspace

1. Install requirements

pip install -r requirements.txt

(Note: In case of numpy version mismatch run pip install numpy==1.21.5)

• Nav2 Install

sudo apt install ros-humble-navigation2
sudo apt install ros-humble-nav2-*

• SLAM Install

sudo apt install ros-humble-slam-toolbox

2. Set Gazebo Paths

• Export Turtlebot3 Model paths

echo "export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:$HOME/Mobile-Swarm-Navigation/Team_53_ws/auto_explore/models" >>  ~/.bashrc
echo "export TURTLEBOT3_MODEL=waffle" >> ~/.bashrc

• Export World Models paths

echo "export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:~/Mobile-Swarm-Navigation/Team_53_ws/aws-robomaker-hospital-world/models:$GAZEBO_MODEL_PATH" >> ~/.bashrc
echo "export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:~/Mobile-Swarm-Navigation/Team_53_ws/aws-robomaker-hospital-world/fuel_models:$GAZEBO_MODEL_PATH" >> ~/.bashrc  
echo "export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:~/Mobile-Swarm-Navigation/Team_53_ws/aws-robomaker-small-warehouse-world/models:$GAZEBO_MODEL_PATH" >> ~/.bashrc
echo "export GAZEBO_MODEL_PATH=$GAZEBO_MODEL_PATH:~/Mobile-Swarm-Navigation/Team_53_ws/turtlebot3_multi_robot/models:$GAZEBO_MODEL_PATH" >> ~/.bashrc

• Export Plugins paths

echo "export GAZEBO_PLUGIN_PATH=$GAZEBO_PLUGIN_PATH:~/Mobile-Swarm-Navigation/Team_53_ws/plugins" >> ~/.bashrc

3. Install dependencies

• Note: The following installation may take some time (>5 mins) depending on the system.

cd ~/Mobile-Swarm-Navigation/Team_53_ws/aws-robomaker-hospital-world/
rosdep install --from-paths . --ignore-src -r -y
chmod +x setup.sh
./setup.sh
cd ../../
colcon build
echo "source ~/Mobile-Swarm-Navigation/Team_53_ws/install/setup.bash" >> ~/.bashrc

Deployment

1. Launch auto_explore package(manual):

• It can be run through the the chatbot also by giving it a prompt to explore the environment.

cd ~/Mobile-Swarm-Navigation/Team53_ws/
ros2 launch auto_explore auto_map.launch.py

2. To save the map:

ros2 run nav2_map_server map_saver_cli -f ~/auto_explore/maps/map_new 

3. Running the chatbot:

python3 ~/Mobile-Swarm-Navigation/AI_ML_models/chatbot/app.py

4. Start Semantic mapping:

python3 ~/Mobile-Swarm-Navigation/Team_53_ws/semantic_mapper/semantic_mapper/semantic_mapper_launch.py --bot_name=<bot_i>   #Do this for each bot on which you want semantic mapping to happen 
#<bot_i> is the bot name where i can be from 0 to num_bots

5. Run the interpreter and Scheduler:

ros2 run interpreter interpreter
python3 ~/Mobile-Swarm-Navigation/Team_53_ws/scheduler/launch/scheduler_launch.py --num_bots= <number_of_robots>

6. Start the simulation:

python3 ~/Mobile-Swarm-Navigation/Team_53_ws/turtlebot3_multi_robot/launch/spawn_bots.py --num_bots=<number-of-robots> --x_pose =<space-separated-array> --y_pose =<space-separated-array>

7. Give task commands via chatbot

Directory Structure

$HOME
└── Mobile-Swarm-Navigation
    ├── AI_ML_models
    │   ├── chatbot
    │   ├── Instance_Segmentation_Model_YOLOv8-seg
    │   └── README.md
    ├── assets
    ├── README.md
    ├── requirements.txt
    ├── Videos
    ├── Mobile-Swarm-Navigation.pdf
    └── Team_53_ws
        ├── auto_explore
        ├── aws-robomaker-hospital-world
        ├── aws-robomaker-small-warehouse-world-ros2
        ├── explore_lite
        ├── frame_changer
        ├── interpreter
        ├── plugins
        ├── scheduler
        ├── semantic_mapper
        ├── turtlebot3
        └── turtlebot3_multi_robot

How does it work

The nervous system manages robot activities by assigning states—idle, exploration, or occupied—based on inputs from the Central Nervous System (LLM). Initially, one robot begins in exploration mode to map the environment, while others remain idle until instructed. Tasks are dynamically assigned to the nearest available robot, prioritizing efficiency while ensuring ongoing tasks are not disrupted. The swarm control system enables real-time data sharing among robots, allowing them to collaboratively build and update a global semantic map detailing the environment’s layout and key landmarks. Robots in occupied mode utilize this shared map to efficiently complete tasks, avoiding redundant efforts.

For Further Details refer to individual package readme files

Demonstration

• Videos: drive_link
• Presentation: canva_link

Resources

1. Turtlebot3 Model

• https://github.com/ROBOTIS-GIT/turtlebot3.git

2. AWS RoboMaker Hospital gazebo world

• https://github.com/aws-robotics/aws-robomaker-hospital-world.git

3. AWS RoboMaker Small Warehouse gazebo world

• https://github.com/aws-robotics/aws-robomaker-small-warehouse-world.git

4. Nav2 Framework

• https://github.com/ros-navigation/navigation2.git

5. SLAM Toolbox

• https://github.com/SteveMacenski/slam_toolbox.git

6. M-Explore ROS2

• https://github.com/robo-friends/m-explore-ros2.git

7. Turtlebot3 Multi Robot

• https://github.com/arshadlab/turtlebot3_multi_robot.git

Future Work

The system can feature Dynamic Map Merging, where robots collaboratively create a unified map, enabling faster mapping while dynamically updating it as environments change. A Robotic Arm can be integrated with the central system to autonomously retrieve coordinates from the semantic map and perform tasks independently. Additionally, Swarm Map Exploration allows robots to collaboratively navigate and explore unknown environments, ensuring efficient coverage and adaptability to dynamic conditions.

Notes

This project was built and tested on a system with following specifications:

• Intel Core i7-12650H Processor
• 16GB RAM
• Nvidia RTX 4060 Laptop GPU - 8GB VRAM
• NVIDIA Driver 535.183.01
• NVIDIA CUDA Toolkit 12.2
• OS: Ubuntu 22.04 LTS
• ROS2 Humble
• Gazebo Classic
• Python 3.10.12