FastFlow, a lightweight runtime designed for AMD Ryzen™ AI systems, has released version 1.0.13, marking a significant milestone by adding support for AMD GPUs. This update integrates HIP EP v0.5.3 through the new oga-hipep backend, enabling developers to run AI models on both NPUs and GPUs under a unified command-line interface (CLI) and local API.
Unified Execution Across NPU and GPU
With the introduction of the oga-hipep backend, FastFlow now supports high-performance GPU execution alongside its existing NPU capabilities. Developers can seamlessly switch between hardware flows while maintaining consistent behavior for chat templates, tokenization, sampling, and generation. The model tag determines the backend, allowing for flexibility in deployment without rewriting code.
Performance Benchmarks
HIP EP, an open-source ONNX Runtime execution provider optimized for AMD GPUs, powers the new GPU flow. Benchmarks on a Ryzen™ AI Max (Strix Halo) system demonstrate significant performance improvements over llama.cpp (Vulkan) at a 16K token context length. The time to first token (TTFT) is reduced by approximately 43%, while decode throughput (TPS) is 7% higher. Specific results include:
- TTFT at 16K: HIP EP build (11/10/2026) achieves 0.57 (normalized to llama.cpp at 1.00).
- Decode TPS at 16K: HIP EP build achieves 1.07 (normalized to llama.cpp at 1.00).
Additional benchmarks for specific models, such as Qwen2.5 and GPT-OSS, show decode speeds ranging from 26.10 tokens/s to 85.96 tokens/s, depending on the model and input type.
Streamlined Developer Experience
FastFlow v1.0.13 simplifies the setup process with just three commands: flm pull, flm run, and flm serve. These commands enable developers to download models, run terminal-based chats, and expose models through an OpenAI-compatible API. The OGA catalog currently includes 13 models, such as Qwen 3.5 and Gemma 4, with support for text and image inputs.
The first OGA model load automatically installs the HIP EP runtime if not already present. However, a supported NPU on the host system is still required, even when running models on the GPU.
Implications for Local AI Development
FastFlow’s expansion to AMD GPUs broadens its appeal to developers seeking high-performance AI solutions on Ryzen AI hardware. By unifying NPU and GPU execution under a single umbrella, FastFlow enhances flexibility and reduces complexity for local AI model deployment. This release also underscores AMD’s commitment to advancing AI capabilities across its hardware ecosystem.


