Centralized motion planning algorithms struggle to scale in multi-arm manipulation, where inter-arm coupling and tight workspaces amplify complexity. We introduce MR-STORM, a distributed sampling-based MPC framework that leverages massively parallel GPU sampling for arm control. Our approach integrates inter-arm collision avoidance via plan-sharing and a dynamic priority scheme to maintain task efficiency. We show empirically that MR-STORM outperforms existing baselines in complex simulation environments and that it transfers effectively to physical hardware.
🔊 Recommended to watch with sound.
STORM Decentralized
CuRobo
MR STORM no prioritization (tau=0)
⭐ MR STORM
#REQUIREMENTS: ubuntu >= 20.04, git and gitlfs (see curobo docs to see why they are needed but don't install them from there but from the repo readme instructions (in code repo or from th[...]
(It's recommended to install the packages will now described inside a conda environment. We also installed isaac sim in that conda env. Supported Isaac sim versions are 4.5 and 5.0.
Before running conda create please checkout https://docs.isaacsim.omniverse.nvidia.com/5.0.0/installation/install_python.html and https://docs.isaacsim.omniverse.nvidia.com/4.5.0/install[...]
Then take isaac sim conda installation instructions (isaac sim 4.5 with python 3.10 and ldd 2.34+ or 5.0 with python 3.11 and ldd 2.35+ depending on your machine and follow the instructio[...]
Note that you can install isaac sim differently if you want for some reason (docker, from source etc) but make sure you install all packages similarly to the instructions will now describ[...]
pick a name for your environment (%env%)
- create a new env using: 'conda create env -b %env%' (and then 'conda activate %env%') # make sure python is 3.10 or 3.11 depending on the mentioned above!
- then install isaac sim using "pip" installation instructions the isaac sim env inside the conda env https://docs.isaacsim.omniverse.nvidia.com/latest/installation/install_python.htm[...]
(pip is the way worked best to me and the rest of process follows, but any other isaac sim installation should work,
look for the instructions start with: conda create -n env_isaacsim python=3.11 # use %env% conda activate env_isaacsim # use %env% and keep from there with their tutoial)
- If installing isaac sim 4.5 (python 3.10), run:
conda activate %env%
pip install isaacsim[all]==4.5.0 --extra-index-url https://pypi.nvidia.com
pip install isaacsim[extscache]==4.5.0 --extra-index-url https://pypi.nvidia.com
- If installing isaac sim 5.0 (python 3.11), run:
conda activate %env%
pip install isaacsim[all,extscache]==5.0.0 --extra-index-url https://pypi.nvidia.com
- clone our repo: Git clone https://github.com/RoboWorkshop/rl_for_curobo.git
- cd rl_for_curobo/curobo # currently rl_for_curobo is a legacy name for our mpc-multi arm project (sorry for the inconvenience, this will be changed soon to "mpc-multi-arm" or similar in[...]
- install curobo in your conda env:
- pip install -e . --no-build-isolation
- if fails, try: SETUPTOOLS_SCM_PRETEND_VERSION_FOR_NVIDIA_CUROBO=0.0.0+local pip install -e . --no-build-isolation
# more instructions and examples here: https://curobo.org/get_started/1_install_instructions.html
# note that you don't need to clone the repository like they require, because we already have it in our repository (don't do that: git clone https://github.com/NVlabs/curobo.git)
- cd .. # back to rl_for_curobo
pip install . # install rl_for_curobo module in yourconda env
# should see some log and then at the end "Successfully installed rl_for_curobo-0.1.0"
- git lfs install # if not yet done
- git lfs pull # very important so that all big files like robots meshes will be pulled properly
- run hello world example # todo