Just my blogroll - BlogFlock

Queuing multiple transcriptions with whisper.el speech recognition - Sacha Chua

2026-01-23T19:50:37.000Z

I want to be able to talk out loud and have the ideas go into Emacs. I can do this in a number of different ways:

I briefly demonstrated a step-by-step approach with natrys/whisper.el with a single file. I press a keyboard shortcut to start the recording, another shortcut to stop the recording, and it transcribes it in the background. But the way whisper.el is set up is that if I press the keyboard shortcut to start recording again it will offer to interrupt the transcription process, which is not what I want. I want to just keep talking and have it process results as things come in.
I'm also experimenting with Google Chrome's web speech API to do continuous speech recognition, which I can get into Emacs using a web socket.
What I've just figured out is how to layer a semi-continuous interface for speech recognition on top of whisper.el so that while it's processing in the background, I can just press a keyboard shortcut (I'm using numpad 9 to call my-whisper-continue) to stop the previous recording, queue it for processing, and start the next recording. If I use this keyboard shortcut to separate my thoughts, then Whisper has a much easier time making sense of the whole sentence or paragraph or whatever, instead of trying to use the sliding 30 second context window that many streaming approaches to speech recognition try to use.

Question: Did you fix the keyboard delay you've got while speech catches what you're saying?

Sometimes, when the speed recognition kicks in, my computer gets busy. When my computer gets really busy, it doesn't process my keystrokes in the right order, which is very annoying because then I have to delete the previous word and retype it. I haven't sorted that out yet, but it seems like I probably have to lower the priority on different processes. On the plus side, as I mentioned, if I dictate things instead of typing them, then I don't run into that problem at all.

Also, other notes on delays: The continuous speech recognition via Google Chrome shows up fairly quickly, but it's not very precise, and it doesn't have punctuation. Even if there's a little bit of a delay, as long as I press the my-whisper-continue shortcut after each thought, then I can get that text into my Emacs buffer using the nicer transcription from my selected model. There is going to be a bit of a delay for that one because it gets processed at the end of the thought. Also, I need to start thinking in complete sentences instead of just adding one cause after the other as my brain goes on all of these tangents. I think it's pretty promising. There's the continuous speech recognition via Google Chrome if I don't mind the lower accuracy and lack of punctuation, and I can still get the pretty version on the other side.

Why talk out loud? I liked the Bookclub Tapas presentation that Maddie Sullivan did at EmacsConf 2025. Talking out loud helps me be a lot more verbose about what I'm saying, compared to typing things out or even like having to switch to my notes or interrupting my screen with an Org capture buffer. Of course I want to clean that up for putting into a blog post, but given that my life still sometimes has random interruptions from a kiddo who must have my attention at that very minute, having that kind of record that I can at least try to reread afterwards to reconstruct what I was thinking about sounds like it might be helpful.

Still, making sense out loud is hard. I'm not actually used to talking to people that much now. This is probably a good reason for me to experiment with streaming more. Then I get the practice in talking out loud, there are backup recordings, and people can ask questions when things are unclear.

Of course, sometimes the text doesn't quite make sense because of the speech recognition errors. I can usually figure it out from the context. I save the audio as well so that I can go back and listen to it again if I really need to.

Anyway, here's the code for sending the current recording to whisper in the background and starting another recording. It assumes a lot about how things are set up. For example, I'm only testing this with a local speaches server instead of whisper.cpp. You might need to look at my other speech related configuration blog posts and sections in order to make sense of it.

Code for queuing whisper.el requests to a local server

(defvar my-whisper--queue nil)
(defun my-whisper-continue (&optional arg)
  "Send what we've got so far for transcription and then continue recording.
Call with \\[universal-argument] to signal that we can stop."
  (interactive "P")
  (setq whisper--marker (point-marker) whisper--point-buffer (current-buffer))
  (when (process-live-p whisper--recording-process)
    (add-to-list
     'my-whisper--queue
     (list :file whisper--temp-file
           :buffer
           (format "*result: %s*" (file-name-base whisper--temp-file)))
     t)
    ;; Remove the sentinel; handle results ourselves
    (set-process-sentinel whisper--recording-process
                          (lambda (process event)
                            (my-whisper-process-queue)))
    (interrupt-process whisper--recording-process))
  (unless arg
    (run-hooks 'whisper-before-transcription-hook)
    (whisper--record-audio)))

(defun my-whisper-discard ()
 "Ignore the previous recording."
  (interactive)
  (when (process-live-p whisper--recording-process)
    ;; Remove the sentinel; handle results ourselves
    (set-process-sentinel whisper--recording-process
                          (lambda (process event)
                            (when (file-exists-p whisper--temp-file)
                              (delete-file whisper--temp-file))
                            (my-whisper-process-queue)))
    (interrupt-process whisper--recording-process)))

(defun my-whisper-discard-and-continue ()
 "Ignore the previous recording and continue."
  (interactive)
  (if (process-live-p whisper--recording-process)
      (progn
        ;; Remove the sentinel; handle results ourselves
        (set-process-sentinel whisper--recording-process
                              (lambda (process event)
                                (my-whisper-process-queue)
                                (my-whisper-continue)))
        (interrupt-process whisper--recording-process))
    (my-whisper-continue)))

(defun my-whisper-done ()
  (interactive)
  (when (process-live-p whisper--recording-process)
    (add-to-list
     'my-whisper--queue
     (list :file whisper--temp-file
           :buffer
           (format "*result: %s*" (file-name-base whisper--temp-file)))
     t)
    ;; Remove the sentinel; handle results ourselves
    (set-process-sentinel whisper--recording-process
                          (lambda (process event)
                            (my-whisper-process-queue)))
    (interrupt-process whisper--recording-process)))

(defun my-whisper-process-queue-result ()
  "Process the first part of the queue that already has results."
  (while (plist-get (car my-whisper--queue) :results)
    (let ((o (pop my-whisper--queue)))
      (unless my-whisper-target-markers
        (setq whisper--marker (point-marker)
              whisper--point-buffer (current-buffer)))
      (with-current-buffer (plist-get o :buffer)
        (erase-buffer)
        (insert (plist-get o :results))
        (goto-char (point-min))
        (run-hook-wrapped
         'whisper-after-transcription-hook
         (lambda (f)
           (with-current-buffer (get-buffer (plist-get o :buffer))
             (save-excursion
               (funcall f)))
           nil))))))

(defun my-whisper-process-queue ()
  (let (o)
    (while (setq o (seq-find (lambda (o) (and (plist-get o :file)
                                              (not (plist-get o :process))
                                              (not (plist-get o :results))))
                             my-whisper--queue))
      (let* ((headers (list "Content-Type: multipart/form-data"))
             (params (list (concat "file=@"
                                   (plist-get o :file))
                           "temperature=0.0"
                           "temperature_inc=0.2"
                           "response_format=json"
                           (concat "model=" whisper-model)
                           (concat "language=" whisper-language)))
             (url (format my-whisper-url-format whisper-server-host whisper-server-port))
             (command `("curl" "-s"
                        ,url
                        ,@(mapcan (lambda (h) (list "-H" h)) headers)
                        ,@(mapcan (lambda (p) (list "-F" p)) params))))
        (with-current-buffer (get-buffer-create (plist-get o :buffer))
          (erase-buffer))
        (plist-put
         o :process
         (make-process
          :name "whisper-curl"
          :command command
          :buffer (plist-get o :buffer)
          :coding 'utf-8
          :sentinel
          (lambda (process event)
            (with-current-buffer (process-buffer process)
              (let ((current my-whisper--queue-item))
                (when (and (get-buffer (plist-get current :buffer))
                           (string-equal "finished\n" event))
                  (with-current-buffer (plist-get current :buffer)
                    (goto-char (point-min))
                    (plist-put current :results
                               (or
                                (condition-case nil
                                    (gethash "text" (json-parse-buffer))
                                  (error ""))
                                "(error)"))))))
            (my-whisper-process-queue-result))))
        (plist-put o :command (string-join command " "))
        (with-current-buffer (process-buffer (plist-get o :process))
          (setq-local my-whisper--queue-item o))))))
(defvar-local my-whisper--queue-item nil)

(defun my-whisper-reprocess-queue ()
  (interactive)
  (setq whisper--marker (point-marker) whisper--point-buffer (current-buffer))
  (mapc (lambda (o)
          (when (process-live-p (plist-get o :process))
            (kill-process (plist-get o :process)))
          (when (get-buffer (plist-get o :buffer))
            (kill-buffer (plist-get o :buffer)))
          (plist-put o :process nil)
          (plist-put o :results nil))
        my-whisper--queue)
  (my-whisper-process-queue))

(defun my-whisper-clear-queue ()
  (interactive)
  (mapc (lambda (o)
          (when (process-live-p (plist-get o :process))
            (kill-process (plist-get o :process)))
          (when (get-buffer (plist-get o :buffer))
            (kill-buffer (plist-get o :buffer)))
          (plist-put o :process nil)
          (plist-put o :results nil))
        my-whisper--queue)
  (setq my-whisper--queue nil))

(keymap-global-set "<kp-9>" #'my-whisper-continue)
(keymap-global-set "<kp-8>" #'my-whisper-discard-and-continue)
(keymap-global-set "C-<kp-9>" #'my-whisper-done)

This is part of my Emacs configuration.

View org source for this post

You can comment on Mastodon or e-mail me at sacha@sachachua.com.

Arc Raiders devs reveal plans for "new large Arc", a fresh map, and two more Expeditions this spring - Rock Paper Shotgun Latest Articles Feed

2026-01-23T17:41:41.000Z

Har(c)ken to me, defenders of Speranza! Embark have just announced their plans for ARC Raiders updates over the first four months of 2026. The extraction shooter’s next season is called Escalation, and its offerings range from new maps and enemies to new Raider decks, changes to existing maps, and more Expeditions. The headline attraction is surely the “new large Arc” coming in April. I wonder how big Arcs can get. Perhaps an Arc the size of a map, like Cronos in God Of War 3? But, you know, metal?

46 US lawmakers express "serious concerns" about the Saudi-led EA buyout's impact on game developers - Rock Paper Shotgun Latest Articles Feed

2026-01-23T17:08:37.000Z

The Saudi-led acquisition of Electronic Arts is meeting with more friction in the US House of Congress. 46 lawmakers have signed a letter calling on the Federal Trade Commission to investigate the debt-financed $55 billion deal for signs that it could be damaging for workers and anti-competitive to the point of breaking US anti-trust law.

Fable, Forza Horizon 6 and more from Xbox Developer Direct 2026 - GamingOnLinux Latest Articles

2026-01-23T16:55:22.000Z

Microsoft recently did their latest Xbox Developer_Direct event with the 2026 edition giving us a new look at multiple games and new release dates.

Read the full article on GamingOnLinux.

Steam Machine and Steam Deck support will be coming to Nexus Mods' Windows-only mod manager at some point - Rock Paper Shotgun Latest Articles Feed

2026-01-23T16:45:53.000Z

Ah, cool. The world's foremost hosts of unofficial Skyrim additions you'd rather not have to explain to an elderly relative, Nexus Mods, have committed to bringing SteamOS support to their mod manager Vortex at some point down the line. That'll mean you can use it with the Steam Machine and Steam Deck, if you fancy a modded playthrough on the go or via a smaller package.

AMD confirm the Ryzen 7 9850X3D launch date and pricing - GamingOnLinux Latest Articles

2026-01-23T16:32:59.000Z

AMD's powerful new gaming processor the Ryzen 7 9850X3D now actually has an official release date and pricing.

Read the full article on GamingOnLinux.

Valve's in-development game Deadlock just got a massive upgrade - GamingOnLinux Latest Articles

2026-01-23T16:03:54.000Z

Deadlock is an online-only third-person MOBA currently in development by Valve, and they just released a massive upgrade for the game.

Read the full article on GamingOnLinux.

Casual EWW - Irreal

2026-01-23T15:58:34.000Z

You’ve all heard me moan many times about my failure to bring my last remaining significant application—Web browsing—into Emacs. There are some solutions but none of them come close to being sufficient. If you want full featured browsing, you’re mostly restricted to Safari, Firefox, one of the niche browsers, or the privacy violating Chrome and whatever Microsoft’s browser de jour is called.

Still, there is EWW, which some people find some sufficient for a subset of browsing tasks. Charles Choi is one of those people so naturally he’s made using EWW easier by adding Casual EWW to his Casual Suite.

You can read his announcement for the details but, as usual, it’s mainly a set of transient menus that make discovery and remembering obscure commands easier. As I’ve said before, even if you need only some of the apps, it’s worthwhile installing the Casual Suite.

Get the Just Cause Complete Collection in a new Humble Bundle - GamingOnLinux Latest Articles

2026-01-23T15:31:32.000Z

The Just Cause Complete Collection Humble Bundle launched with a whole lot of action that you can grab for a decent price.

Read the full article on GamingOnLinux.

Become a literal galaxy brain in this ambitious space strategy sim about an AI seeking a new home for humanity - Rock Paper Shotgun Latest Articles Feed

2026-01-23T15:31:22.000Z

Sine Fine is a “hard sci-fi” space exploration game from the avid stargazers at Vindemiatrix Collective, a developer based across western Europe. It is certainly hard to get your head around, but also, very promising. The premise is that you’re a lonely, functionally immortal AI, seeking a new home for some meagre frozen embryos in the wake of humanity’s extinction. Labouring across eons, you’ll send out probes to nearby systems and build outposts and communication networks.

Cyberpunk 2077 VR modder hit by alleged second DMCA strike, "forced to temporarily pause" Patreon - Rock Paper Shotgun Latest Articles Feed

2026-01-23T15:30:07.000Z

After having the Cyberpunk 2077 bit of their R.E.A.L VR mod/framework taken down via a DMCA from CD Projekt, modder Luke Ross claims to have had another stike filed against their work. This time, the strike has come from Ghostrunner publishers 505 Games, according to a post from Ross in which they say they've been "forced to temporarily pause" their Patreon page.

While Ross' Cyberpunk VR mod being behind a paywall was at the centre of CD Projekt's decision to file a takedown, there appears to have been no indication so far as to whether the same is true of the alleged strike from 505.

[$] Filesystem medley: EROFS, NTFS, and XFS - LWN.net

2026-01-23T15:27:39.000Z

Filesystems seem to be one of those many areas where the problems are well understood, but there is always somebody working toward a better solution. As a result, filesystem development in the Linux kernel continues at a fast pace even after all these years. In recent news, the EROFS filesystem is on the path to gain a useful page-cache-sharing feature, there is a new NTFS implementation on the horizon, and XFS may be about to get an infrastructure for self healing.

GPD claim the WIN 5 is getting an official Bazzite Linux adaptation but the Bazzite team say otherwise - GamingOnLinux Latest Articles

2026-01-23T15:16:25.000Z

GPD are being weird again - making a claim that the GPD WIN 5 is getting an "official" adaptation with Bazzite Linux but the Bazzite team say otherwise.

Read the full article on GamingOnLinux.

Vulkan 1.4.340 released with new extension to improve DirectX performance on Linux - GamingOnLinux Latest Articles

2026-01-23T14:51:07.000Z

We're seeing excitement build up from the Vulkan API release 1.4.340, with a new extension that should eventually improve DirectX performance on Linux.

Read the full article on GamingOnLinux.

Nexus Mods say they are committed to supporting Vortex on SteamOS Linux - GamingOnLinux Latest Articles

2026-01-23T14:28:05.000Z

After cancelling the newer Nexus Mods app that had Linux support to refocus on the older Vortex, the Nexus Mods team now say Vortex will support SteamOS.

Read the full article on GamingOnLinux.

Ubisoft implementing cost-reduction restructuring, cancelling various games and closing studios - GamingOnLinux Latest Articles

2026-01-23T14:18:34.000Z

Ubisoft are implementing various cost-cutting measures and have announced they've cancelled 6 games, while also closing studios in a big shift.

Read the full article on GamingOnLinux.

GNU Guix 1.5.0 released - LWN.net

2026-01-23T14:14:25.000Z

Version 1.5.0 of the GNU Guix package manager and the Guix System have been released. Notable improvements include the ability to run the Guix daemon without root privileges, support for 64-bit RISC-V, and experimental support for the GNU Hurd kernel.

The release comes with ISO-9660 installation images, virtual machine images, and with tarballs to install the package manager on top of your GNU/Linux distro, either from source or from binaries—check out the download page. Guix users can update by running guix pull.

It's been 3 years since the previous release. That's a lot of time, reflecting both the fact that, as a rolling release, users continuously get new features and update by running guix pull; but it also shows a lack of processes, something that we had to address before another release could be made.

During that time, Guix received about 71,338 commits by 744 people, which include many new features.

LWN last looked at Guix in February 2024.

Two new stable kernels for Friday - LWN.net

2026-01-23T14:05:12.000Z

Greg Kroah-Hartman has released the 6.18.7 and 6.12.67 stable kernels. As always, each contains important fixes throughout the tree. Users are advised to upgrade.

Valve tweak accessibility categories and release new Steam update with controller improvements and new Beta UI - GamingOnLinux Latest Articles

2026-01-23T14:00:41.000Z

Valve have given their accessibility categories on Steam for discovering games a small overhaul, along with a big new Steam client update for everyone.

Read the full article on GamingOnLinux.

Security updates for Friday - LWN.net

2026-01-23T13:59:30.000Z

Security updates have been issued by AlmaLinux (kernel), Debian (bind9, chromium, osslsigncode, and python-urllib3), Fedora (freerdp, ghostscript, hcloud, rclone, rust-rkyv0.7, rust-rkyv_derive0.7, and vsftpd), Mageia (avahi and harfbuzz), SUSE (alloy, avahi, busybox, cargo-c, corepack22, corepack24, curl, docker, dpdk, exiv2-0_26, ffmpeg-4, firefox, glib2, go1.24, go1.25, gpg2, haproxy, kernel, kernel-firmware, keylime, libpng16, librsvg, libsodium, libsoup, libsoup2, libtasn1, log4j, net-snmp, open-vm-tools, openldap2_5, ovmf, pgadmin4, php7, podman, python-filelock, python-marshmallow, python-pyasn1, python-tornado, python-urllib3, python-virtualenv, python3, python311-pyasn1, python311-weasyprint, rust1.91, rust1.92, util-linux, webkit2gtk3, and wireshark), and Ubuntu (libxml2 and pyasn1).