Skip to content

feat: Enable torch.compile for DiffusionPolicy inference#2486

Merged
imstevenpmwork merged 3 commits intohuggingface:mainfrom
jashshah999:feat/diffusion-torch-compile
Feb 24, 2026
Merged

feat: Enable torch.compile for DiffusionPolicy inference#2486
imstevenpmwork merged 3 commits intohuggingface:mainfrom
jashshah999:feat/diffusion-torch-compile

Conversation

@jashshah999
Copy link
Copy Markdown
Contributor

@jashshah999 jashshah999 commented Nov 20, 2025

What this does

(⚡️ Performance) Adds torch.compile support to DiffusionPolicy to optimize inference latency. This allows users to optionally enable PyTorch 2.0's compilation feature for the U-Net component, which is the computational bottleneck during diffusion inference.

Changes:

  • Added compile_model: bool = False and compile_mode: str = "reduce-overhead" configuration flags to DiffusionConfig
  • Applied torch.compile to the U-Net in DiffusionModel.__init__ when compile_model=True
  • Defaults maintain backward compatibility (compilation disabled by default)

How it was tested

  • Created benchmark script to measure inference latency with and without compilation
  • Tested on CUDA with 50 iterations after proper warmup
  • Verified backward compatibility (default behavior unchanged)
  • Confirmed code compiles without errors

Performance Results:

  • Baseline (no compile): 1796.57 ms average latency
  • With torch.compile: 1468.84 ms average latency
  • Speedup: 1.22x (18.2% improvement)

How to checkout & try? (for the reviewer)

from lerobot.policies.diffusion.configuration_diffusion import DiffusionConfig
from lerobot.configs.types import FeatureType, PolicyFeature

# Enable compilation
cfg = DiffusionConfig(
    input_features={
        "observation.state": PolicyFeature(type=FeatureType.STATE, shape=(14,)),
        "observation.image.camera": PolicyFeature(type=FeatureType.VISUAL, shape=(3, 96, 96)),
    },
    output_features={
        "action": PolicyFeature(type=FeatureType.ACTION, shape=(2,)),
    },
    compile_model=True,  # Enable torch.compile
    compile_mode="reduce-overhead",  # Optional: "default", "reduce-overhead", or "max-autotune"
)

policy = DiffusionPolicy(cfg)
# Use policy as normal - first inference will be slower due to compilation overhead
# Subsequent inferences benefit from the optimization

@imstevenpmwork imstevenpmwork self-assigned this Feb 24, 2026
@imstevenpmwork imstevenpmwork self-requested a review February 24, 2026 14:42
@github-actions github-actions Bot added the policies Items related to robot policies label Feb 24, 2026
Copy link
Copy Markdown
Collaborator

@imstevenpmwork imstevenpmwork left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for this contribution !

@imstevenpmwork imstevenpmwork merged commit dac1efd into huggingface:main Feb 24, 2026
8 checks passed
pkooij pushed a commit that referenced this pull request Feb 27, 2026
Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
xianglunkai pushed a commit to xianglunkai/lerobot that referenced this pull request Feb 28, 2026
…#2486)

Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
sorelferris pushed a commit to sorelferris/lerobot that referenced this pull request Mar 1, 2026
…#2486)

Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
lu391see pushed a commit to lu391see/lerobot_tactile that referenced this pull request Mar 24, 2026
…#2486)

Co-authored-by: Steven Palma <imstevenpmwork@ieee.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

policies Items related to robot policies

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants