Daft Launches daft-physical-ai: Open-Source Library for Robot Video Training Data Pipeline
Key Takeaways
- ▸daft-physical-ai abstracts away data preprocessing bottlenecks that consume engineering time on physical AI projects—hand tracking annotation and reward scoring are available as drop-in operations
- ▸The library supports flexible hand-tracking backends (MediaPipe for CPU, WiLoR for GPU) that return identical schemas, allowing downstream pipeline steps to remain unchanged when switching methods
- ▸Integration with LeRobot datasets and Robometer reward models provides end-to-end support for the common physical AI training workflow: read video, annotate, score, filter, train
Summary
Daft Labs has released daft-physical-ai, a new open-source Python library that automates the critical data processing step between raw robot video recordings and training-ready datasets for physical AI models. The library builds on top of Daft's existing data framework, providing lazy, batched, and distributed operations that handle annotation and preprocessing tasks that typically consume significant engineering effort on robotics projects.
The initial release includes two core use cases: hand tracking (supporting both MediaPipe for CPU-based 2D tracking and WiLoR for GPU-accelerated 3D MANO keypoints) and reward scoring via Robometer-4B integration. The library integrates natively with LeRobot datasets from Hugging Face and allows data processing pipelines to filter, annotate, and score episodes before they reach model training.
According to the announcement, Daft recently achieved 15× performance improvements in reading LeRobot datasets, and daft-physical-ai is positioned to help teams skip building custom data plumbing infrastructure. Developers can install the library via pip and layer it into existing data pipelines, with parallelized processing handled transparently by Daft's execution engine.
- Lazy evaluation and distributed batching ensure efficient processing of large robot video datasets; quality gates for filtering problematic episodes can be implemented as single-line Daft expressions
Editorial Opinion
daft-physical-ai addresses a real pain point in physical AI development: the unglamorous but essential work of turning raw footage into training signals. By packaging hand tracking and reward scoring as composable operations, Daft removes friction from the data pipeline—exactly where robotics teams lose time today. The tight integration with LeRobot and explicit support for swapping hand-tracking backends show thoughtful API design. If adoption follows, this could become the default scaffolding for robotics ML pipelines.
