Accelerating vision-language models with LFM2.5-VL-DSpark
2026-09-27 · Hugging Face
Accelerating Vision-Language Models with LFM2.5-VL-DSpark
Liquid AI has announced the release of an experimental DSpark draft model for its vision-language model (VLM), LFM2.5-VL-3B. Similar to their recently released text LFM2.5-DSpark drafters, this model introduces a speculative decoding path that trades a minimal increase in memory footprint for significant inference speedups without altering the output quality.
Key Benefits
- Faster Inference: Achieves decode speedups of up to 3.13x on-device and 2.66x on an H100 GPU, with end-to-end gains reaching 2.62x and 2.27x respectively.
- Small Memory Cost: The drafter adds only 280M parameters, representing an 8.9% increase over the 3B target model.
- Day-One Support: Includes LFM-compatible DSpark integrations for llama.cpp, MLX-VLM, and SGLang.
How Speculative Decoding Works for VLMs
The vision drafter utilizes the same architecture as the text LFM2.5-DSpark drafters. It captures the target model's hidden states at a fixed set of tapped layers and conditions on them to draft a block of k candidate tokens. Image patches and text tokens are projected into a shared representation before reaching these layers, allowing the drafter to operate on hidden-state vectors of identical dimensionality regardless of the input modality. Consequently, the inference algorithm remains unchanged from the text models.
Training and Architecture
The team followed the DSpark recipe using a mixture of vision-language SFT data, weighted toward expected workloads. Based on ablations across 3, 4, and 5 layers, the final draft model is a simplified attention-only drafter with 4 layers and a block size of 9. Training ran for 10 epochs on the final mixture, measuring acceptance after each epoch, which improved with additional tokens before hitting diminishing returns. For inference, a block size of 8 or 9 is recommended depending on the hardware.
The resulting drafter contains approximately 280M parameters, broken down as follows:
- Decoder stack (4 layers): 193.0M
- Hidden-state projection: 21.0M
- Markov head: 65.5M
- Norms + confidence head: 6.4k
Inference Speedup on CPU and GPU
The DSpark draft model ships with day-one support for llama.cpp, MLX-VLM, and SGLang. Performance was measured across both on-device and GPU configurations using a DSpark block size of 8. Evaluations followed the MMSpec benchmark across six diverse vision-based tasks: general VQA, text VQA, image captioning, chart VQA, complex reasoning, and multi-turn conversation.
- On-Device Inference: Using MLX on an M5 Max, decoding ran 2.30x to 3.13x faster by task, with end-to-end latency improving by 1.56x to 2.62x. Using llama.cpp on an M3 Ultra, decoding improved by 1.57x to 2.14x and end-to-end by 1.30x to 1.77x.
- GPU Inference: On an H100, the drafter delivered 1.64x to 2.66x faster decoding, with end-to-end improvements of 1.64x to 2.27x.
Limitations of Speculation for Vision Workloads
In LLMs, prefill is mostly compute-bound, with costs growing subquadratically with prompt length. VLMs add complexity because images pass through a vision encoder, and the language backbone processes hundreds of visual tokens alongside the text prompt. Edge devices have significantly less compute than datacenter GPUs, meaning prefill consumes a larger portion of end-to-end latency.
Speculative decoding accelerates only the decode phase, not vision encoding or prefill. When those unaccelerated stages take up much of the wall time, even a large decode speedup yields only modest end-to-end gains. This is a direct application of Amdahl's Law, where overall speedup is capped by the unaccelerated portion of the workload.
How to Use LFM2.5-VL-DSpark
Users can run the DSpark draft models via SGLang, llama.cpp, or MLX-VLM by specifying the draft model path and speculative parameters in the launch commands. Speculative decoding is exact; the target model verifies every proposed token, ensuring greedy output matches the target alone.
Get Started
The vision DSpark draft model is available on Hugging Face in Safetensors and GGUF formats. With LFM2.5, Liquid AI continues to deliver on its vision of AI that runs anywhere, offering open-weight models, fast day-one support, and a complete architecture family for diverse use cases.