Talos: Student-Built FPGA Accelerator Rethinks CNN Inference from Silicon Up
Key Takeaways
- ▸Talos is an FPGA-based CNN accelerator built entirely in SystemVerilog with deterministic, cycle-accurate control, eliminating runtime and scheduler overhead
- ▸The project required two intensive weeks of development, highlighting the fundamental challenges of hardware engineering including nanosecond-precision timing and physics-constrained debugging
- ▸The design prioritizes inference-specific optimization through fixed-point arithmetic, streaming memory pipelines, and purpose-built control logic rather than general-purpose flexibility
Summary
Krish Chhajer and Luthira Abeykoon have released Talos, a custom FPGA-based hardware accelerator designed specifically for convolutional neural network inference. Unlike traditional deep learning frameworks built for flexibility, Talos strips away runtime overhead, schedulers, and operating system layers to achieve deterministic, cycle-accurate control over every calculation. Implemented entirely in SystemVerilog, the accelerator represents a ground-up rethinking of how deep learning inference should work at the circuit level, prioritizing efficiency over generality.
The project was completed in an intensive two-week development period, during which the team confronted the fundamental differences between hardware and software engineering. Hardware development requires navigating physical constraints of silicon, fixed logic elements, on-chip memory limitations, and nanosecond-precision timing requirements—where a signal missing a timing window by half a nanosecond can cause system failure. The team spent hours analyzing waveforms to catch single-bit errors, a level of granularity rarely encountered in software development.
Talos's first inference pipeline implements a straightforward CNN architecture: a single convolutional layer processing 28×28 grayscale images with 4 kernels of 3×3 size, followed by ReLU activation, 2×2 MaxPool with stride 2, flattening, and a fully connected layer mapping to 10 output classes. The design philosophy centers on four core principles: determinism through fixed operation paths, minimal latency via cycle-level scheduling, efficient memory usage through streaming pipelines that avoid storing full intermediate feature maps, and hardware-optimized fixed-point arithmetic that eliminates general-purpose overhead.
The accelerator directly challenges the design assumptions underlying frameworks like PyTorch, which prioritize training flexibility at the cost of inference overhead. By making the entire pipeline deterministic in hardware and removing anything that isn't core mathematical computation, Talos demonstrates an alternative approach to deep learning deployment focused exclusively on production inference efficiency.
- The architecture implements a basic CNN (conv → ReLU → MaxPool → FC) for digit classification, demonstrating the approach on a constrained but complete inference pipeline
- Talos represents a philosophical challenge to mainstream frameworks like PyTorch, arguing that production inference requires fundamentally different architectural assumptions than training
Editorial Opinion
Talos is a reminder that the future of AI deployment may not lie in ever-larger general-purpose frameworks, but in specialized hardware that ruthlessly eliminates everything except the essential computation. While the implemented model is modest—a simple digit classifier—the design philosophy is profound: when you control the silicon, determinism and efficiency become possible in ways software can never achieve. This project also highlights a growing gap in AI education, where most practitioners never touch the hardware layer that ultimately executes their models, leaving performance and efficiency as abstract concerns rather than physical realities governed by nanosecond timing constraints.
