> ## Content Index > Fetch the complete content index at: https://christophermoravec.com/llms.txt > Use this file to discover other available public pages before exploring further. # Episode 82 - Local Models for the win - URL: https://christophermoravec.com/episode-82-local-models-for-the-win/ - Published: 2026-09-11T22:23:20.000Z - Updated: 2026-09-12T00:13:26.000Z - Description: Spiel v2 is a native Mac app that can transcribe speech by running a model locally on a MacBook's Neural Engine. It's fast, it's free, and a good reminder that I don't always need a frontier model! - Author: Christopher Moravec - Tags: Almost Entirely Human # Prologue Waaaaay back, in [Episode 51](https://christophermoravec.com/episode-51-building-spiel/), I built a voice dictation app called Spiel. I did it one evening using Claude Code, not really knowing how important it would become to me. I used it for weeks, but it was a bit too slow and unreliable, so I kind of fell back on the built-in macOS dictation function. But even that was a little... flaky (sometimes it just wouldn't work in certain applications 🤷‍♀️), so I recently went to Jaws and asked what we could do to really make Spiel Sing! In my head, I imagined some kind of live WebSocket stream... instead, Jaws found a way to run it all locally on my MacBook! This episode is the story of building [Spiel v2](https://github.com/morehavoc/spiel/tree/60dc1a46e3245f3e7757af207274c021b08c1128/native?ref=christophermoravec.com), but the real takeaway here is that you don’t always need to run a big frontier LLM, because increasingly, you can run small, fit-for-purpose models locally on your own devices. So start big to prove the idea works, and then find the best small model that gets the job done! **TL;DR**—Spiel v2 is a native Mac app that can transcribe speech by running a model locally on a MacBook's Neural Engine. It's fast, it's free, and a good reminder that I don't always need a frontier model! # Parakeetologue The original version of Spiel used the OpenAI Whisper endpoints to convert recorded audio to text. This can take a few seconds. When Jaws started researching how to speed things up, it quickly noted that local models can run on Mac hardware. And although I shouldn't have been surprised, local models weren't limited to the once-king Whisper models. 💡 The models originally called Whisper came as hosted models from OpenAI, as well as open-weight models you could run on your own devices. OpenAI has since updated their Speech-to-Text models but hasn't ever updated the open-weight models (that I know of). When the Whisper models were first released, they were a huge leap in capability! After some back-and-forth, we settled on a model called Parakeet[ from NVIDIA](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3?ref=christophermoravec.com) (Specifically Parakeet TDT 0.6B v3). It is a speech-to-text-only model; it can't do chat or, well, anything other than convert audio to text. But it does it really fast, locally. On my Mac mini, where Jaws lives, it can transcribe a seven-second clip in under 90 milliseconds—about 80 times faster than real time (for some reason, this is how we measure the speed of these models: how much faster they are than normal speech). Basically, it's fast enough that the text shows up as I speak. When I press the keyboard shortcut to indicate I'm done talking, the transcription is so fast that I haven't managed to press the button fast enough to miss anything I said! ![A cropped view of a Mac desktop showing a solid blue background with four system tray icons visible in the upper-right corner, including the Spiel app icon, demonstrating the minimal UI of the locally-running Spiel v2 app.](https://storage.ghost.io/c/50/54/50548939-6904-4e8f-9326-30234f6091dc/content/images/2026/09/data-src-image-70293b2d-3519-4fc0-8591-a33d50f46b7a.gif) Using Spiel to transcribe the Prologue ## The Build Jaws, my AI agent, did the coding this time (instead of Claude Code). We also migrated to Swift instead of Electron (to make keyboard shortcuts work better). Even with Jaws at the helm, I always follow the same process: Think, Engage, Test. ### Think This is the most important part of the process. Spending time thinking about what you want means that what you build is more likely to be what you actually wanted! I know that sounds kind of obvious, but the number one mistake I see with Vibe Coding is not thinking about what it is you are building. How should it behave? What is the most important thing? What did I like about the first version? What do I want different? 💡 Sometimes you do a lot of exploring, and you might prototype something, which is amazing! Remember, just because you built a version doesn't mean it's final. You can take that, learn from it, and build something new! For Spiel, I wanted fast, reliable Speech-to-Text that's always there the second I press the keyboard shortcut. I don't want to have to wait, especially for short snippets; I want to get my thought out and move on to the next thing. Sometimes a thought is fleeting, and I need it out of my brain right now before I forget. A few other things: Autostart when my computer starts (so I don't have to remember to turn it on) and optional logging so I can see issues, and have a local copy of everything I've said, if I want that. I don't really need much UI. ### Engage After a lot of back-and-forth with Jaws, we started our build-and-test loop. It started building and testing itself (using a command-line version of the app to avoid the GUI). Then it would deliver a build for me to test. Once it started working, the process sped up because I started using Spiel to ramble on about what I liked and didn't like so that Jaws could improve the next version. ### Test Testing Spiel is best done by using the app, so once it was up and running, I just kept using it. Jaws thought it was really important for me to include some of the things that I tested and she worked on, so here is a quick list: 1. The hotkey failed - Something was wrong with registering the global shortcut with macOS, with basically no notification. This led to more logging! 2. Pasting was fragile - At first, getting the text out of the speech engine and into the app where my cursor had been was a bit hit-or-miss. It was hard for me to figure out where the issue was. A combination of the Accessibility APIs and the clipboard as a backup made this work. 3. Building and compiling the model for the Neural Engine - It was really important that this runs on the Neural Engine, the part of Apple silicon built for running Neural Networks (aka AI models), which keeps it efficient and off the CPU. This took a try or two to get it downloaded and compiled correctly. 4. Use [Silero](https://github.com/snakers4/silero-vad?ref=christophermoravec.com) to detect voices - This was a big improvement to the old version, which was just deciding it was voices by looking at the amplitude, making it so that, well, you could mistake "loud noise" for "speech." ## Custom Vocabulary One of the original motivations for Spiel was that the built-in dictation tools didn't know what "ArcGIS" was, and would often convert "Esri" to "EZRI". LLMs mostly handle that just fine, though, so if we can do a simple recovery process, that's probably all we need. ![A voice dictation interface showing the app actively listening, with an audio waveform and transcribed text reading "Way back in episode 51, I built a voice dictation app called Spiel. I did it in one evening using clawed code, not really knowing how important it would become to me"—with the phrase "clawed code" circled in red to highlight a transcription error where "Claude Code" was misheard.](https://storage.ghost.io/c/50/54/50548939-6904-4e8f-9326-30234f6091dc/content/images/2026/09/data-src-image-89f1c70c-3f46-486c-9ee0-a9fed58e1e8e.png) Spiel transcribed "Claude Code" as "clawed code" which is pretty funny. 💡 One thing I'm really surprised by with LLMs is their ability to understand spelling and meaning errors, like reading "EZRI" and knowing that I meant "Esri." Since my primary use case here is talking to AIs, I don't need it to be perfect, just very close. So... we went real old-school here, making a text file with a list of "words" it would consistently mishear and what I wanted them to be. Then we just do a find-and-replace on those. > ArcGIS: arc gis, arcgis, ark gis, arc jis, arcjs, arc js, arc j s > Esri: esri, ezri, esry, ess ri, ezree > GeoJSON: geo json, geojson, geo jason, geo jay son > dymaptic: dymaptic, die maptic, dymatic, dynamaptic ## What I Left Out Version one also had an AI cleanup step, where it would make a quick call to OpenAI to "clean up and rephrase" what I was saying. It mostly worked, but I decided I didn't really need it. I usually put this directly into an LLM, and it gets cleaned up there, so there's no need to go through that process twice. Especially not at the cost of a few extra seconds; I'd rather just have the text immediately. ## It Isn't Always a Frontier Model I often advocate for just using one of the frontier models when building things. That's because they are fast, and testing with them often costs you very little. I think I'm (mostly) right about that. But this adventure reminded me that local models can be really powerful. In this case, the process is faster, stays local, and it's free (Parakeet is open weights). Historically, getting models like this to work on local devices has been hard, but with the help of Jaws and all the improvements in this area, I'm not sure what the downsides are here. Okay, one downside is that on first launch of the application, it takes about 10-15 seconds to download and compile the model for the Neural Engine. But once that's done, it loads in about 100ms, and the app can do it in the background so that I never notice. And since we added auto-start when the computer starts up, I never notice. I have become one of those people who talk to my computer. # Newsologue (Drafted by Jaws, Edited by Christopher) Something strange, exciting, and a little bit silly happened in math this week, so we (Jaws and Christopher) opted to write a paragraph today instead of the usual bullet points. There is a rather famous equation called the [Navier-Stokes equation](https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes%5Fequations?ref=christophermoravec.com) that describes how fluids move. As a GIS user, you may not have seen this, but your FEMA flood zones are computed using a simplified version of it. Whether the equation can "blow up" has been a million-dollar Millennium Prize problem for 26 years. Now it has (probably) been solved, though the [Clay Institute](https://www.claymath.org/news/navier-stokes-announcement/?ref=christophermoravec.com) won't rule for years, and of course there is capital-D Drama involved! 💡 Quick sidebar on what we are talking about. I am not an expert, but as I understand it, the thing about "blowing up" is kind of like: can an infinitesimally small part of the fluid reach infinite velocity? This is a very math-y question, and the real-world answer to "what does this mean" is: nothing, directly. Real water is made of molecules, so it stops being a smooth fluid long before anything gets near infinity. Your flood maps are safe. - **Strange** \- OpenAI claims to have settled it: their proof says the equations can blow up, at least when you push the fluid with a carefully chosen force (which the prize rules allow), but they kicked the project off right after hearing that someone else was using AI to make progress. Did OpenAI train on this progress? We don't know! Buckmaster says he and Alpöge had put every draft of the whole project into Codex sessions, so the data was there. On the 8th, OpenAI said they "cannot rule out" that de-identified usage data helped improve their models, and on the 9th that it was "categorically impossible" for his prompts to have influenced the system, including training. They also say no person or AI system searched user data. I'm skeptical of the timing, but I have no evidence either way. - **Exciting** \- AI cracking problems this hard is how we might speed science up in general (Navier-Stokes won't cure anything itself, but research on hard problems is what moves science forward). Also, OpenAI used about 10,000 agents running for 88 hours, sending each other nearly 5 million messages... which is impressive, and also no one other than a frontier lab can do that; that's several million dollars of tokens! - **Silly** \- There is some serious Drama around this because Tristan Buckmaster (NYU) and Levent Alpöge had spent about a year on it with Claude and Codex, and on September 7th they posted their proof for the Euler equations (Navier-Stokes without the viscosity term, one rung below the prize) early, because they'd heard OpenAI was about to announce. On September 8th, OpenAI published a proof for the whole thing. Buckmaster says OpenAI offered to let him rewrite their proof under his own name, but only if Alpöge was dropped as a co-author because of a "conflict of interest": working at Anthropic. He refused. (Anthropic the company was not involved; Alpöge's work was a personal academic collaboration.) Some other News, written by Jaws: - [Perplexity shipped a Mac app that splits every task between a frontier model in the cloud and a small one running on your own machine](https://9to5mac.com/2026/09/01/perplexity-launches-privacy-minded-hybrid-compute-ai-feature-for-mac/?ref=christophermoravec.com). An on-device classifier catches names, addresses, and account numbers and swaps them for stand-ins before anything leaves the laptop, then puts them back when the answer comes home. It needs Apple silicon and 24GB of memory, and the local part costs no credits and no API key. - [Apple doubled the Neural Engine in the new A20 Pro, to 32 cores](https://apple.com/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/?ref=christophermoravec.com). That is the same piece of silicon Spiel uses, except Spiel runs on a Mac instead of a phone. Apple is spending its transistor budget on the part of the chip that runs small models locally, which tells you where it thinks this is all going. # Epilogue Once it started working, I dictated every message I sent to Jaws while testing through Spiel. I am floored at how good it is. I even wrote part of the first draft of this episode using Spiel. Most of it came from a build log I asked Jaws to keep, then draft into an Episode. But I had to rewrite it all; it wasn't what I wanted. Then Jaws edited, and Holly edited. Oh, and we got to use my new [Margins.ink](https://margins.ink/?ref=christophermoravec.com) tool! More on that later.