Dictation using Voxtype

Hold Meta+V anywhere, speak, release. The words appear at the cursor of whatever window has focus. English, Chinese, and Japanese can be mixed inside one sentence. Nothing leaves the machine.

This page documents how that pipeline is put together on my laptop (X11 + QTile, Intel i5-10300H, NVIDIA GTX 1650 Ti, PipeWire) and how to reproduce it.

What actually runs

The flow: hold the chord → record 16 kHz mono from the PipeWire default source → release → transcribe → paste at the cursor. A single recording is capped at 60 s. voxtype status prints idle, recording, or transcribing.

Why a source build

Two things forced local patches, and neither is a packaging accident:

• QTile cannot bind a held chord. Its keybinds are X grabs, so a binding fires once on press, and hold-to-record is not expressible in the config. The daemon therefore listens on evdev directly and owns the chord itself.
• A chord owned by evdev also reaches the focused application, and Chromium inserts a literal v into whatever text box is open. The exclusive-capture patch is what stops that.

The rest is upstream 1.0.1 with three open PRs attached: the xdotool output driver, the clipboard fallback on the paste path, and the fix for a chord whose modifier is released first (which used to leave the recording running until the 60 s cap). What runs here is that tree, on the fork’s custom branch:

The last one is mine and is not upstreamed; the other three are waiting in review. All four are on the fork’s custom branch, which is the tree this machine runs: iacore/voxtype.

An older 0.7.5 install is still in /usr/lib/voxtype, with /usr/bin/voxtype pointing at it; that variant was hand-patched at the time to add xdotool, which is why pacman -Qkk voxtype reports a checksum mismatch. Nothing uses any of it now.

Reproducing

1. Packages

# build toolchain
sudo pacman -S rustup alsa-lib clang cmake pkgconf patchelf

# runtime
sudo pacman -S xdotool xclip ydotool vulkan-icd-loader

# on-screen visualiser
sudo pacman -S gtk4 gtk4-layer-shell

# evdev hotkeys and exclusive capture
sudo usermod -aG input "$USER"

Log out and back in. The input group matters twice: /dev/input/event* for the hotkey listener, and /dev/uinput (mode 0660, group input) for the keyboard mirrors that exclusive capture creates.

2. Source tree with the four patches

git clone -b custom https://github.com/iacore/voxtype
cd voxtype

custom is upstream 320a737 (the merge of PR #746) plus the four patches above, and two follow-ups that only touch printed help text, so it is the tree that runs here.

Three of those patches are also open against upstream dev, if the goal is to get them merged rather than to run them — the fork branch can be rebuilt from public refs:

git clone https://github.com/peteonrails/voxtype
cd voxtype && git checkout -b custom 320a737
git fetch origin pull/559/head:pr559 pull/588/head:pr588
git fetch https://github.com/iacore/voxtype fix/hotkey-release-without-modifier:release-and-grab
git merge pr559 pr588 release-and-grab

That merge applies cleanly and produces the same code; it lags the fork branch only by two cosmetic commits — the xdotool row in voxtype setup check and the driver order printed by voxtype config show.

3. ONNX Runtime

The ONNX engines (SenseVoice, Paraformer, Dolphin, Omnilingual) link against ONNX Runtime. Here it is linked dynamically against a local prefix, which keeps the binary small and lets the runtime be replaced without a rebuild:

mkdir -p ~/.local/lib/onnxruntime
curl -L https://github.com/microsoft/onnxruntime/releases/download/v1.24.4/onnxruntime-linux-x64-1.24.4.tgz \
  | tar -xz --strip-components=1 -C ~/.local/lib/onnxruntime

4. Build

export ORT_LIB_PATH=$HOME/.local/lib/onnxruntime/lib
export ORT_PREFER_DYNAMIC_LINK=1

cargo build --release \
  --features "sensevoice,paraformer,dolphin,omnilingual,ml-diarization,gpu-vulkan" \
  --bin voxtype

# bake the runtime location into the binary (the build only records -L)
patchelf --set-rpath "$HOME/.local/lib/onnxruntime/lib" target/release/voxtype

install -Dm755 target/release/voxtype ~/.local/lib/voxtype/voxtype

gpu-vulkan is what gives Whisper the GTX 1650 Ti; SenseVoice and the other ONNX engines run on the CPU. voxtype info variants prints the feature set that ended up in the binary.

The waveform overlay is a separate build, and the daemon spawns it as a sibling of its own executable, so both files have to sit next to voxtype:

cargo build --release --features osd-gtk4 --bin voxtype-osd --bin voxtype-osd-gtk4
install -Dm755 target/release/voxtype-osd target/release/voxtype-osd-gtk4 \
  ~/.local/lib/voxtype/

Before switching feature sets, run cargo clean. Upstream warns about this loudly, and rightly: with stale artifacts the binary still compiles, still reports the right version, and silently runs without GPU or ONNX support.

5. Models

voxtype setup --download --model sensevoice-small --activate

That lands in ~/.local/share/voxtype/models/: sensevoice-small/ (model.int8.onnx, 239 MB, plus tokens.txt) and, if the Whisper engine is kept around, ggml-small.bin (466 MB) via --model small. voxtype info models lists what each engine has installed.

6. Configuration

~/.config/voxtype/config.toml. Everything omitted is a default; the file that ships with a fresh install is a commented-out copy of every option, and voxtype config schema lists the keys that the CLI can reach directly.

state_file = "auto"          # $XDG_RUNTIME_DIR/voxtype/state, for `voxtype status`
engine = "sensevoice"

[hotkey]
key = "EVTEST_47"            # kernel keycode 47 = V (`evtest` prints "Event code 47 (KEY_V)")
modifiers = ["LEFTMETA"]     # so the chord is Meta+V
grab = true                  # exclusive capture; the chord stops leaking to applications

[audio]
device = "default"
sample_rate = 16000
max_duration_secs = 60

[output]
mode = "paste"               # clipboard + synthetic Ctrl+V
fallback_to_clipboard = true
wait_for_modifier_release = true   # send the paste after Meta comes up, not while it is held
modifier_release_timeout_ms = 2000

[sensevoice]
model = "sensevoice-small"
language = "auto"            # per-utterance identification: en, zh, ja, ko, yue
use_itn = true               # punctuation and written-out numbers
threads = 8
on_demand_loading = false    # keep the 239 MB model resident

[whisper]                    # only used when engine = "whisper"
model = "small"
language = ["en", "zh"]      # constrained auto-detect: helps on short utterances
threads = 8
flash_attention = true

Three of these earn their keep:

• grab = true (exclusive capture). Every keyboard is held with EVIOCGRAB and its events are re-emitted through a uinput virtual keyboard that voxtype owns, minus the hotkey chord. Without it Meta+V reaches the focused application as well, and Chromium inserts a stray v into whatever text box is open. In xinput list this shows up as one voxtype key mirror per grabbed keyboard — five here.
• wait_for_modifier_release: a paste fired while Meta is still down arrives as Super+Ctrl+V, which nothing binds, so the text is silently lost.
• on_demand_loading = false: the alternative would unload an idle model and pay roughly two seconds on the first dictation after every pause.

7. Autostart

This machine has no systemd; the user session runs under dinit --user. ~/.config/dinit.d/voxtype is the whole service:

type = process
# Custom build: upstream v1.0.1 + PRs #559 (xdotool output driver), #588, #589,
# plus the SenseVoice/ONNX engines. NOT the /usr/bin/voxtype pacman binary.
command = /home/user/.local/lib/voxtype/voxtype
env-file = xserver.env

with ~/.config/dinit.d/xserver.env holding DISPLAY=:0, since xdotool needs a display but the daemon can start before X is up. Enabling it drops the symlink into boot.d:

dinitctl enable voxtype     # once
dinitctl start voxtype
dinitctl restart voxtype    # after every rebuild

QTile starts xserver.target from its hooks, and that target has waits-for.d = xserver.d, so the empty marker file ~/.config/dinit.d/xserver.d/voxtype makes the X session start the daemon as well.

8. Verify

voxtype setup check                       # dependency and output-chain report
voxtype status                            # idle
xinput list | grep -c 'voxtype key mirror'   # one per grabbed keyboard

voxtype record toggle                     # for scripts and keybindings; the chord does the same

Then hold Meta+V, say something, and let go.

Gotchas

• Exclusive capture is per keyboard and fails soft. If /dev/uinput is not writable, or another program (keyd, kmonad, interception-tools) already holds a keyboard, that keyboard is skipped: dictation still works, but the chord reaches applications again. Startup logs one line, Exclusive capture active: N of M keyboard(s) grabbed.
• While it is on, every keystroke in the session is routed through voxtype’s virtual keyboards. A broken build therefore means broken typing, which is why there is a copy of the previous binary in ~/.local/lib/voxtype-backup/.
• The chord is grabbed at the kernel level, so do not also bind it in QTile.
• If Meta is still held two seconds after the key is released, the paste is skipped and the text only reaches the clipboard.
• The pacman package stays installed for its /usr/lib/voxtype variants, but its sidecars are 0.7.5 and must not be mixed with the 1.0.1 daemon.
• When the three open PRs land, only the exclusive-capture patch is left to carry. PR #589 currently bundles it with the release-order fix, so upstream may split it out during review.