SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

Aman Arora1 Ricard Marsal I Castan1 Matteo El-Hariry1 Miguel Olivares-Mendez1

1SnT — Interdisciplinary Centre for Security, Reliability and Trust

University of Luxembourg, Luxembourg

Conference on Robot Learning (CoRL) 2026

ICE2THRUST, funded by the European Union
Paper Code
Recorded policy rollouts · Floating Platform (3-DoF)

Abstract

Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation.

We identify a structural failure mode of this method class on terminal-navigation tasks reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion.

We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this “feasibility collapse”. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms—a 6U CubeSat and a floating platform testbed for zero-shot sim-to-real transfer—supports the generality of these findings.

Feasibility collapse in termination-based CRL

Termination-based constrained RL can fail on terminal-navigation tasks when the goal lies inside a constraint-active corridor. CaT may then learn to hover safely outside the goal because entering suppresses its survival-weighted return, producing high constraint compliance without task completion. We derive the condition under which this hover policy becomes preferred. SCoCaT fixes the problem with an auxiliary success critic trained independently of CaT terminations, preserving the learning signal inside the goal corridor while keeping the original constraint mechanism.

CaT stalls outside the goal while SCoCaT enters it Two cases show CaT stopping outside the constraint-active goal region and SCoCaT continuing into the feasible goal pose. Every arrow points from left to right. CaT constraint-active region safe hover high compliance · no completion SCoCaT constraint-active region feasible goal pose high compliance · task completion

Results

We evaluate SCoCaT on a planar floating platform and a 6U CubeSat against unconstrained PPO, CaT, and gradient-projection baselines.

Radar plots comparing task completion and constraint compliance for PPO, CaT, and SCoCaT on the floating platform and 6U CubeSat
Task success and constraint compliance. Each axis is oriented so that larger area is better. CaT is compliant but fails to complete the task; SCoCaT recovers task completion while retaining high compliance.
Constraint compliance versus goal-corridor occupancy for PPO, CaT, and SCoCaT configurations
Safety–performance trade-off. SCoCaT configurations occupy the high-compliance, high-occupancy region on the floating platform.
Fraction of episodes inside the docking corridor over time for PPO, CaT, and SCoCaT
Corridor occupancy. SCoCaT enters and remains in the goal corridor through the task-completion window.

Zero-shot sim-to-real transfer

Top-down sequence of an air-bearing floating platform approaching its docking target during a successful SCoCaT hardware run
Air-bearing floating platform testbed. A successful SCoCaT approach is shown at several points in time. SCoCaT succeeded in three of four hardware runs and reached a final distance of 1.0 ± 0.9 cm.

BibTeX

@inproceedings{arora2026scocat,
  title={SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking},
  author={Arora, Aman and Marsal I Castan, Ricard and El-Hariry, Matteo and Olivares-Mendez, Miguel},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2026},
}
BibTeX copied