Ventaglio: New Sparse Tensor Hardware Accelerator Speeds Up AI Inference on Vector Processors

Summary

Researchers have published a new hardware architecture called Ventaglio that dramatically accelerates sparse tensor contractions on vector processors, addressing a critical bottleneck in AI inference optimization. The design introduces runtime-configurable sparse execution units coupled with RISC-V Vector (RVV) ISA extensions to enable efficient indexed gather-accumulate-scatter operations that leverage both weight pruning and activation sparsification in Transformer models.

When benchmarked on a pruned LLaMA-3-8B model with 40-60% dual sparsity (combining weight and activation sparsity), Ventaglio achieves 2.40-5.25x speedup during prefill operations and 2.06-3.16x speedup during autoregressive decoding compared to dense baselines. The implementation uses only 3.1% additional area overhead in a 12nm FinFET process and demonstrates 6.9-7.4x improvements over optimized RVV baselines. This research addresses a practical gap in vector processor support for sparse operations, which were previously handled inefficiently through software index decoding.

• Research demonstrates feasibility of native hardware support for sparsity exploitation on RISC-V vector architectures

Editorial Opinion

Ventaglio represents a meaningful step toward closing the efficiency gap between theoretical and actual performance for sparse AI inference. As weight pruning and activation sparsification become standard optimization techniques in production deployments, native hardware support for these patterns becomes increasingly valuable. The research is particularly timely given the emergence of cost-conscious inference as a major competitive pressure in the AI industry.