ComfyUI LLMs 2× Faster: Enhance Prompts, Describe Images, Write Stories at Double Speed

By Mirrorize ·

ComfyUI LLMs 2× Faster: Enhance Prompts, Describe Images, Write Stories at Double Speed

ComfyUI just rewrote the core loop that handles LLM text generation, and the results are so clean that creators working locally don't need to change anything—just update the software. Qwen3.5 9B now hits 219 tokens per second on an RTX 5090 (up from 106), and larger models like Qwen3.8 27B went from 55 to 122 tok/s when describing a 1536×1536 image.

The optimization focused specifically on the Qwen3.x family, rewriting what the team calls "the hot path" of text generation to reduce overhead. The changes landed in ComfyUI v0.45.0 and include shared INT8 KV caches, deferred linear-attention updates, and reused rotary data.

What this means for you

If you run ComfyUI workflows that use LLMs for prompt enhancement, image description, or story generation, those steps now complete in half the time. A prompt enhancer that used to add 20 seconds to your workflow might now add 10.

[Qwen3.8 27B on an RTX 5090, describing a high-resolution image while writing a story: 137 tok/s with speculative decoding enabled.]

The Qwen3.8 27B speedup is especially relevant for creators working with large language models on consumer cards. An RTX 5090 seeing 122 tok/s from describing a 1536×1536 image means the model stays useful for batch workflows where image analysis is the bottleneck.

How it works

The optimization specifically targets bottlenecks in how the model generates tokens one after another. By smartly caching attention weights and deferring expensive operations until they're actually needed, the engine spends less time moving data and more time computing. The team also implemented speculative decoding, which lets the system guess-and-verify tokens faster—a technique that adapts to the text being written, not just a static setting.

What's actually good

The speed gains show up on both NVIDIA and AMD cards running Windows or Linux. You don't install anything new or fiddle with quantizations. An RTX 5080 went from 91 to 129 tok/s on Qwen3.5 9B—that's a real, measurable win on mid-range hardware. The speculative decoder now adapts to the text length, so longer prompts don't suffer from fixed overhead anymore.

The trade-off: this is only for Qwen3.x models. If you're using other LLMs in ComfyUI, you won't see the speed boost yet.

How to try it

Update ComfyUI to v0.45.0 or later. Your existing Qwen3 nodes will automatically run at the new speeds. No model weights to re-download, no settings to tweak. If you're already using Qwen3 models for prompt enhancers or image description, the next time you run a workflow, you'll see the faster token generation in the terminal output.

Sources




More from the Learning Center

Make AI art · AI video · AI music on Mirrorize, free to start.