CADRE: Dynamic Catching via Implicit Contact Descriptors and Task-Appropriate Recovery Affordances

Abstract

Real-world dexterous manipulation often encounters unexpected errors and disturbances, which can lead to catastrophic failures, such as dropping the manipulated object. To address this challenge, we focus on the problem of catching a falling object while it remains within grasping range and, importantly, resetting the system to a configuration favorable for resuming the primary manipulation task. We propose Contact-Aware Dynamic Recovery (CADRE), a reinforcement learning framework that incorporates a Neural Descriptor Field (NDF)-inspired module to extract implicit contact features. Building on these contact features, we introduce an Implicit Recovery Affordance function to encourage recovery to task-appropriate states. Compared to methods that rely solely on object pose or point cloud input, NDFs can directly reason about finger-object correspondence and better establish a recovery target for RL training. Our experiments show that incorporating contact features improves training efficiency, enhances convergence performance for RL training, and ultimately leads to more successful recoveries. Additionally, we demonstrate that CADRE can generalize zero-shot to unseen objects with different geometries.

Simulation

Ours: CADRE

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Baseline: PointCloud

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Baseline: DexPoint

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Baseline: CADRE-EBM

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Ablation: CADRE-No NDF

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Ablation: CADRE-No IRA

Screwdriver In-Distribution
Screwdriver Out-of-Distribution
Socket In-Distribution
Socket Out-of-Distribution

Hardware Experiment

CADRE

0.25x
1x

DexPoint

0.25x
1x

CADRE - No IRA

0.25x
1x

Subsequent Screwdriver Turning

Ours: CADRE

Final Catching State - CADRE
Final Catching State
Subsequent Screwdriver Turning - CADRE
Subsequent Screwdriver Turning

Baseline: PointCloud

Final Catching State - PointCloud
Final Catching State
Subsequent Screwdriver Turning - PointCloud
Subsequent Screwdriver Turning

Baseline: DexPoint

Final Catching State - CADRE
Final Catching State
Subsequent Screwdriver Turning - CADRE
Subsequent Screwdriver Turning

Baseline: CADRE-EBM

Final Catching State - DexPoint
Final Catching State
Subsequent Turning - DexPoint
Subsequent Screwdriver Turning

Ablation: CADRE - No NDF

Final Catching State - Pose
Final Catching State
Subsequent Screwdriver Turning - Pose
Subsequent Screwdriver Turning

Ablation: CADRE - No IRA

Final Catching State - Pose
Final Catching State
Subsequent Screwdriver Turning - Pose
Subsequent Screwdriver Turning

Subsequent Hole-on-Peg

Ours: CADRE

Final Catching State - CADRE
Final Catching State
Subsequent Screwdriver Turning - CADRE
Subsequent Hole-on-Peg

Baseline: PointCloud

Final Catching State - PointCloud
Final Catching State
Subsequent Screwdriver Turning - PointCloud
Subsequent Hole-on-Peg

Baseline: DexPoint

Final Catching State - CADRE
Final Catching State
Subsequent Screwdriver Turning - CADRE
Subsequent Hole-on-Peg

Baseline: CADRE-EBM

Final Catching State - DexPoint
Final Catching State
Subsequent Turning - DexPoint
Subsequent Hole-on-Peg

Ablation: CADRE - No NDF

Final Catching State - Pose
Final Catching State
Subsequent Screwdriver Turning - Pose
Subsequent Hole-on-Peg

Ablation: CADRE - No IRA

Final Catching State - Pose
Final Catching State
Subsequent Screwdriver Turning - Pose
Subsequent Hole-on-Peg