cuda OutOfMemory:

#6
by nikita200 - opened

I am training the Groot model on a dual RTX 5080 setup with 32 GB VRAM per GPU, but I’m getting a CUDA out-of-memory error. I previously trained the same model on a single RTX 4080 (12 GB VRAM) without issues. Now, even though I’m using a machine with a higher configuration, I’m still running into this problem.
Screenshot from 2025-12-09 12-42-21

here is the command I am using for the training:
accelerate launch --multi_gpu --num_processes=2 $(which lerobot-train) --output_dir=outputs/train/groot_training1_latest --save_checkpoint=true --batch_size=4 --steps=25000 --policy.push_to_hub=true --policy.type=groot --policy.repo_id=hf_user/groot_training_latest --policy.tune_diffusion_model=false --dataset.repo_id=hf_user/lerobot-record-test --wandb.enable=true --wandb.disable_artifact=true --job_name=groot_training_latest

Sign up or log in to comment