AET: New Transpiler Compresses Source Code for LLMs, Reducing Token Usage by 30-55%

Summary

A new open-source transpiler called AET has been released that compresses source code in Go, Java, and Python into an AI-native format optimized for large language models. The tool strips syntactic elements designed for human readability—such as public static void main declarations, curly braces around single-statement blocks, and redundant type annotations—while preserving full semantic fidelity and round-trip compilability. Benchmarks show token savings ranging from 30-55% across various code types, with boilerplate-heavy code (like Java DTOs and Spring Boot controllers) seeing the most significant compression. The transpiler works by parsing source code into a language-specific abstract syntax tree (AST), lowering it to a shared intermediate representation (IR), and then compressing it into the compact AET format, with a fully deterministic and reversible process that allows code to be compiled back to valid source.

Editorial Opinion

AET represents a pragmatic optimization for an acute problem in the LLM era: the rising cost of token-based pricing for code processing. By intelligently stripping human-oriented syntactic ceremony, it enables more efficient use of context windows and reduces inference costs without sacrificing semantic accuracy. This tool exemplifies how AI-native tooling can bridge the gap between human and machine preferences in code representation, though widespread adoption will depend on integration into popular AI development workflows.