A developer on Reddit, u/iamMess, put Qwen3.8-27B on a single RTX 3090 and measured 82 tokens per second.
The card has 936.2 GB/s of memory bandwidth and 24GB of VRAM, which sounds like a hard ceiling for a 27B model.
Every generated token makes the GPU pull the model weights back out of memory, so the compute units spend most of their time waiting on that fetch instead of calculating.
He attacked the waiting instead of buying around it.
With speculative decoding, a small drafter proposes the next few tokens and the main model verifies all of them in one pass, keeping the correct ones and discarding the rest.
The output is unchanged, and accuracy on his math benchmark held at 96.5 percent.
The gains arrived in stages.
Tuning Qwen3.8-27B's built-in multi-token prediction lifted single-user speed to about 114 tokens per second.
Adding DFlash2, which proposes seven tokens at a time, plus a lookup drafter that reuses text already in the conversation, pushed it to 382 tokens per second.
That is a 4.6x gain on one card.
The GPU was the same and the model was the same.
Only the inference engine changed.
For a team deciding where to serve a 27B model, that changes the math on the bill.
An RTX 3090 at $0.15/hr with an engine you can tune carries real serving work, and renting it keeps the tuning effort in your hands when the hardware underneath changes.
BHK Cloud runs RTX 3090s at $0.15/hr.
Buy Now Pay Later keeps the first month at zero upfront, with payment after one month.
https://ai.bhkcloud.com/?utm_source=linkedin&utm_medium=social&utm_campaign=bhk_social_2026w42&utm_content=evening What tokens per second is your current model serving at, and was the last jump a hardware upgrade or an inference optimization?
Originally posted on linkedin_personal