Onde Swift: New Open-Source LLM Inference Engine for Apple Silicon Devices
Key Takeaways
- ▸Onde Inference releases Swift package for on-device LLM inference on Apple platforms (iOS 15.0+, macOS 11.0+, tvOS 15.0+)
- ▸Package leverages Metal GPU backends for optimized performance on Apple silicon hardware
- ▸Pre-configured with Qwen 2.5 models (1.5B and 3B variants) and supports streaming, conversation, and one-shot inference modes
Summary
Onde Inference has released a Swift package for on-device LLM inference optimized for Apple silicon, enabling developers to run large language models directly on iOS, macOS, and tvOS without relying on cloud services. The package, generated from a Rust crate, supports Metal GPU acceleration and comes with pre-configured models including Qwen 2.5 1.5B for mobile devices and Qwen 2.5 3B for macOS. The library offers multiple inference modes including streaming responses, one-shot generation, and conversation management, making it accessible for developers to integrate local LLM capabilities into their Apple applications. Installation is straightforward through Xcode's package manager, with comprehensive support for App Store compliance and sandboxed environments.
- Open-source tool enables developers to build privacy-preserving AI features without cloud dependencies
Editorial Opinion
The release of Onde Swift represents a meaningful step toward democratizing on-device AI inference for Apple ecosystem developers. By providing a well-documented, easy-to-integrate solution with sensible defaults, Onde lowers the barrier for developers to build privacy-respecting AI features without cloud infrastructure costs. This could accelerate adoption of local LLM inference across iOS apps, particularly for use cases where data privacy and offline functionality are critical.
