Open-source Anomaly Detection System Brings Trustworthy AI to Satellite Operations
Key Takeaways
- ▸Calibrated confidence scores transform arbitrary anomaly scores into interpretable, distribution-free probabilities that operators can act on with confidence
- ▸Attribution layer identifies exactly which sensor channels caused each detection, enabling triage to start at the right plot instead of mass manual investigation
- ▸LLM-generated operator briefs are audited by dual guardrails that block any claim unsupported by the evidence, with 100% accuracy on real and planted test anomalies
Summary
A new open-source project introduces a trustworthy anomaly detection system for satellite telemetry that operators can actually act on. Built on ESA-ADB (the European Space Agency's official anomaly detection benchmark using real mission data), the system improves upon traditional binary anomaly flags by adding three critical layers: calibrated confidence scores that quantify uncertainty, attribution analysis identifying which sensor channels caused the detection, and LLM-generated operator briefs whose every claim is audited against the data.
The project's core contribution is not the detector itself—which deliberately reproduces ESA's published baseline (F0.5 = 0.949) using commodity hardware—but rather the trustworthy wrapper layers that sit on top. A mission operator receives data from hundreds of sensors and cannot manually monitor them all. Fixed thresholds miss contextual and collective anomalies, but a bare 0/1 flag from a model that cannot justify its verdict becomes an ignored alert. The system transforms this dynamic by making every detection auditable and actionable.
The implementation consists of four modular components: anomaly detection via Isolation Forest, uncertainty quantification through conformal p-values, attribution via perturbation/ablation analysis, and a grounded alert layer powered by an LLM judge with dual guardrails. Testing shows zero false alarms across 976,182 normal windows at 99% precision, with attribution correctly identifying expert-annotated channels 120 out of 120 times. The entire system is available as open-source with an interactive web demo (runnable in two minutes with no dependencies) and reproducible notebooks on GitHub.
- Entire system runs on commodity hardware (MacBook, no GPU or cluster required) and is reproducible end-to-end with published notebooks
- Modular architecture makes the detector itself replaceable—modules 2-4 (uncertainty, attribution, alert) operate only on detector outputs, never internals
Editorial Opinion
This project exemplifies how trustworthy AI requires more than better models—it requires auditable reasoning. Satellite operators ignore alerts they can't interrogate, making a detector's confidence and justification as important as its accuracy. By pairing a standard baseline detector with rigorous uncertainty quantification, causal attribution, and grounded report generation, the system transforms anomaly detection from a black-box binary decision into an investigation tool operators can actually rely on. Open-sourcing this work sets a template for how safety-critical AI systems should expose their reasoning to domain experts.
