Whisper picks the wrong GPU, and tells you it is accelerated
A laptop with a discrete GPU usually has two: the integrated one in the CPU package, and the one you bought the laptop for. Vulkan enumerates both. whisper.cpp defaults to device 0, and device 0 is the integrated one.
Nothing warns you. The log says GPU acceleration is on, the model loads, transcription runs. It is simply slower than it should be — and on a laptop-class integrated GPU, “slower” turns out to mean slower than the audio you are transcribing, which quietly makes live transcription impossible while looking configured.
The numbers
Same machine, same model (medium.en), same benchmark, encode time per 30-second chunk:
| device | encode |
|---|---|
| integrated (default) | 16 340 ms |
| discrete | 122 ms |
133×. The default is not a small tax; it is a different capability. At 122 ms per 30 seconds of audio there is enough headroom to transcribe live with the largest English model. At 16 seconds per 30 seconds of audio, the transcript falls permanently behind the meeting.
Selecting the right device
whisper-cli takes -dev N. The environment variable GGML_VK_VISIBLE_DEVICES=1 works for every binary in the family — which matters, because whisper-bench has no -dev flag at all. That asymmetry is its own trap: benchmark and CLI can silently disagree about which hardware they used, so a benchmark number does not describe the run you actually care about.
Any launcher — a streaming process, a server, a recording script — has to set this explicitly. There is no configuration that makes the default correct.
The general shape
The interesting part is not the flag. It is that “GPU acceleration: enabled” was true, and useless. A status line reported the category of thing that was happening, not which thing, and the difference between the two was two orders of magnitude.
Whenever a system reports capability rather than identity — accelerated, cached, connected, secure — the report cannot distinguish the good case from the bad one. Only a measurement can. This cost one command to check and would have cost a whole feature to not check.
Related: A transcript is not evidence · Reading wrap