RAI

RAI is a flexible AI agent framework to develop and deploy Embodied AI features for your robots.

📚 Visit robotecai.github.io/rai for the latest documentation, setup guide and tutorials. 📚

🎯 Overview

Category	Description	Features
🤖 Multi-Agent Systems	Empowering robotics with advanced AI capabilities	• Seamlessly integrate Gen AI capabilities into your robots • Enable sophisticated agent-based architectures
🔄 Robot Intelligence	Enhancing robotic systems with smart features	• Add natural human-robot interaction capabilities • Bring flexible problem-solving to your existing stack • Provide ready-to-use AI features out of the box
🌟 Multi-Modal Interaction	Supporting diverse interaction capabilities	• Handle diverse data types natively • Enable rich sensory integration • Process multiple input/output modalities simultaneously

RAI framework

rai core: Core functionality for multi-agent system, human-robot interaction and multi-modalities.
rai whoami: Tool to extract and synthesize robot embodiment information from a structured directory of documentation, images, and URDFs.
rai_asr: Speech-to-text models and tools.
rai_tts: Text-to-speech models and tools.
rai_sim: Package for connecting RAI to simulation environments.
rai_bench: Benchmarking suite for RAI. Test agents, models, tools, simulators, etc.
rai_perception: Object detection tools based on open-set models and machine learning techniques.
rai_nomad: Integration with NoMaD for navigation.
rai_finetune: Finetune LLMs on your embodied data.

Getting started

See Quick setup guide.

Simulation demos

Try RAI yourself with these demos:

Application	Robot	Description	Docs Link
Mission and obstacle reasoning in orchards	Autonomous tractor	In a beautiful scene of a virtual orchard, RAI goes beyond obstacle detection to analyze best course of action for a given unexpected situation.	link
Manipulation tasks with natural language	Robot Arm (Franka Panda)	Complete flexible manipulation tasks thanks to RAI and Grounded SAM 2	link
Autonomous mobile robot demo	Husarion ROSbot XL	Demonstrate RAI's interaction with an autonomous mobile robot platform for navigation and control	link
Agentic mobile manipulator	RB-KAIROS	A comprehensive demo with on-board execution	link

Community

Embodied AI Community Group

RAI is one of the main projects in focus of the Embodied AI Community Group. If you would like to join the next meeting, look for it in the ROS Community Calendar.

Publicity

A talk about RAI at ROSCon 2024.

RAI Q&A

Please take a look at Q&A.

Developer Resources

See our documentation for a deeper dive into RAI, including instructions on creating a configuration specifically for your robot.

Contributing

You are welcome to contribute to RAI! Please see our Contribution Guide.

Citation

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@misc{rachwał2025raiflexibleagentframework,
      title={RAI: Flexible Agent Framework for Embodied AI},
      author={Kajetan Rachwał and Maciej Majek and Bartłomiej Boczek and Kacper Dąbrowski and Paweł Liberadzki and Adam Dąbrowski and Maria Ganzha},
      year={2025},
      eprint={2505.07532},
      archivePrefix={arXiv},
      primaryClass={cs.MA},
      url={https://arxiv.org/abs/2505.07532},
}

rai

RAI