ToTra: Open-Source LLM Gateway Brings GDPR and EU AI Act Compliance to Any LLM
Key Takeaways
- ▸Zero-code integration: Drop-in replacement for direct LLM API calls via single config change, compatible with OpenAI SDK, LangChain, curl, and other tools
- ▸Enterprise compliance out-of-the-box: GDPR workflows, EU AI Act checklist, and immutable hash-chained audit logs for every request
- ▸Data privacy at the edge: PII scanning and redaction across 18 languages before sensitive data reaches any LLM provider
Summary
ToTra is a new open-source AI gateway and governance platform written in Go that sits between applications and LLM providers like OpenAI, Anthropic, and Gemini. Developers can route requests through ToTra with a single-line configuration change, gaining instant access to quota enforcement, PII blocking, cost tracking, and compliance features without modifying application code.
The platform addresses critical enterprise requirements by offering per-user and per-team budget caps, edge-based PII scanning across 18 languages, real-time cost tracking with chargeback reports, and compliance workflows including GDPR data-subject handling and an EU AI Act checklist. All data processing happens on self-hosted infrastructure with zero external dependencies.
ToTra is designed for minimal operational overhead, written in Go to ensure sub-2ms latency overhead and running as a native binary without Python runtime requirements. The project is fully compatible with the OpenAI API specification, allowing teams to swap their API endpoint without rewriting application code.
- Cost control and visibility: Per-user/team hard budget caps, real-time anomaly detection, and chargeback-ready reporting
- Self-hosted and open: Full data sovereignty with no external dependencies, deployable via Docker Compose in 5 minutes
Editorial Opinion
ToTra addresses a critical gap in enterprise LLM adoption by decoupling governance from the LLM provider itself. As regulatory pressure increases around data residency and AI compliance, a self-hosted gateway approach could become essential infrastructure for organizations serving regulated industries. The zero-code integration design is particularly clever—allowing existing applications to gain compliance features without engineering rewrites significantly lowers the barrier to adoption, though teams will need to evaluate the operational overhead of maintaining additional infrastructure.
