Heureka: I Sent a Local AI a Video at 1:30 a.m. At 6:29 It Had Built the App

At 01:30 on an August night, I was already in bed. I had just watched a video of a coding agent building an app from a spec - a lot of what it showed I had tried myself before, with other tools and an older LLM, but always locally. The new model had good reviews elsewhere. So I did the thing anyway, out of curiosity, expecting another fail - probably.

On my phone I opened Telegram and wrote to my agent:

https://youtu.be/k2LLUKj6piE?si=nc4sB730kE6dC23Z

“Can you set up spec-kit and follow through the demo, but instead of using IBM Bob as coding agent, you should use goose, which is already installed. Work in the folder ...speckit_shoppinglist. Make it use SQLite, FastAPI, NiceGUI. For the constitution, import the .../guidelines/ files and refer to the imported files. Do you have any questions about this assignment?”

(Abridged; the full prompt, verbatim, is in the appendix at the end.)

At 06:29 the next morning the agent wrote back:

“Done. (…)”

(See the appendix for the full message. It is worth reading - I was still grinning an hour later.)

It was error-free! Insane – everything ran exclusively locally, on my own machine!

I had gone to sleep. My agents kept working. Seven spec-kit sessions had run in the dark, one after the other, and the app was built, tested, and waiting.

The only hardware I touched was the phone. The other hardware is a single workstation with a GPU in my office, but I will get to it.

AI-generated stock image.

The conclusions one can already draw from this experiment:

1. The video was a teaching film. It showed the agent at every step what comes next, so the agent had been shown a path to the result. My agent followed it well.
2. The software project in the video was simply another variant of a todo list (a shopping list here) for a browser - for an LLM, nothing new in itself.
3. But my change requests in the prompt made the task far from trivial, so the AI had to do real transfer work. A different development environment had to be set up and new packages installed. The agent’s closing message already hints that problems did occur along the way, which were then resolved on its own within the session.

From that, the following questions derive:

1. How hard is it - or what effort has to be invested - for me to get my own agents to carry whole software tasks through to success on their own (i.e. without further intervention during the working phase)?
2. Do I understand how to apply this spec-driven approach to other projects of my own, and how would I proceed there?
3. Which project types are suitable for it, and which are not?
4. If this approach really does make it possible to hand agents whole software problems, unattended, how large may such projects become before they stop working on my local hardware?
5. When I finally hit limits - and they will inevitably appear - can I find out which factors determine those limits decisively, in order to perhaps derive new strategies from that?

This experiment was already very revealing. Before I go on to test how far I can push my local AI, though (the next articles will cover that), it is first necessary to understand the foundations this experiment was already able to build on.

What was there already

This is not a stack I designed in one afternoon. It is the end of a long, sometimes frustrating path that began with a single graphics card.

October 2024. The first card, an NVIDIA RTX A6000 (48GB VRAM) – my entry into local AI. First models straight from ollama.com, whatever fit, e.g. gemma-2, llama3, phi-3, qwen2.5-coder, and so on. Then the LAN server: Ollama on a Windows 11 machine usable from every machine on the LAN, with Open-WebUI and ChatBox pointed at it.

2025 Experiments with Python frameworks (CrewAI, langchain, RAG), prompt engineering, the first aider runs. Agent GUIs everywhere, own Python scripts, Jupyter notebooks that “tap into” the Ollama server through libraries.

2026 Right in January: OpenClaw gets hyped everywhere – the first agent that talked to me in a chat window, over Telegram. It could improve itself, and occasionally overshot its target. I spent nights writing (letting it write) guidelines – large volumes of markdown files – so OpenClaw would write code exactly the way I want it. Nevertheless it kept running into the wall at the same spot: you could chat for hours, generate tons of markdown, but it regularly failed to follow the tasks through to the end. The 60–80% limit (the failure mode the whole series is an answer to).

Still: the guidelines from the OpenClaw era formed the basis for the rules the agents had to follow when reproducing the project from the video (which they did quite well).

March 2026. A new machine: an NVIDIA RTX PRO 6000 Blackwell (Max-Q workstation edition). Bigger open-weight models – 70B-class, 120B-class (gpt-oss:120B) – and the limit was still there.

So the fix had to come from the harness, not the weights. hermes came to the CLI (it took over the Telegram account from the retired OpenClaw), then a goose GUI whose recipes concept made me curious. I finally read the Heise overview of spec-driven development tools and tried OpenSpec afterwards. It was solid, but also not fully convincing, and so I kept searching – until this YouTube video was suggested to me.

That is how the Heureka night below happened: I was already in bed, goose was not reachable, only hermes via Telegram was still responsive – and everything came together unexpectedly.

The 60–80% wall that haunted every milestone above is the reason this article exists. That this weakness did not show up on that one night is now my motivation for all further investigations.

And the how – how the two tiers actually work together – is going to be investigated just as closely as the question of whether and why vLLM will do better on the Blackwell GPU (and how to measure that).

I will keep expanding and refining the setup, put more tools into the toolbox, and Git could perhaps become an external agent memory through the commit messages – all of that is what the following articles will cover.

The hardware: one card, no cluster

That night, only my workstation was used, no external provider: an NVIDIA RTX PRO 6000 Blackwell (Max-Q workstation edition), 96GB VRAM. On it runs Ollama, serving a Q8-quantized 27B model – the qwen3.8:27b-mtp-q8_0 build, 8-bit weights, the mtp variant of the model family. 27B is a serious size for a single workstation graphics card, which takes about 47GB at a context size of 256k tokens, and would therefore even fit into the 48GB memory of my NVIDIA RTX A6000.

And that is exactly what this story is about: this model makes it possible for the agents to use a single local GPU, so that they can, with the help of the video’s given instructions, independently rebuild the software – and even port it to a new development environment and programming language.

The serving stack at the time was Ollama 0.32.13. The model’s context window is 256K tokens; the number that mattered all night was a different one, and it comes next.

The two tiers of the run, both on the same (virtual) machine – my agents live in an isolated area on the workstation – shared the same locally hosted LLM:

• Worker – goose 1.46.0 as the coding agent, executing the spec-kit phase chain, and
• Supervisor – the orchestrator (Hermes) that coordinated the phases, checked the evidence, and verified every claim of the “worker”.

Both ran on the same local model qwen3.8:27b-mtp-q8_0.

The assignment itself came out the other side as a small web app: SQLite for persistence, FastAPI for the API, NiceGUI for the interface – exactly as specified in the prompt and built after the recipe of a video, following the requirements I had sent in a Telegram message from my phone.

What actually happened, hour by hour

The full exchange is reconstructable from two independent stores – the coding agent’s own session ledger and the driving orchestrator’s state store – so the timeline is measured, not merely recounted:

• 01:30 – the prompt arrives. The orchestrator does a pre-flight: the coding agent (goose 1.46.0), specify, and uv are all present; it fetches the demo video’s transcript so it follows the demo precisely instead of guessing what the video shows.
• 01:31 – it pauses and asks two questions, before starting a single phase: which model to use (I confirm the default, the local 27B) and how to import my guidelines (I choose: copy the five files verbatim). Both are cheap, defaults-backed decisions. I answer within about two minutes, half asleep. It never asks again.
• 01:33 – 06:21 – the spec-kit phases run one after another, each in a fresh session (no resume; state passes through the repo’s artifacts only): smoke, constitution (twice), specify, plan, tasks, and a 4 h 35 m implement phase – 735 messages, the whole build. The implement phase is where an agentic loop lives or dies: it is the long, growing conversation – and the point where models of previous generations used to start skidding.
• 06:24 – 06:26 – the supervisor re-runs the gates independently and runs a clean end-to-end against a fresh database.
• 06:29 – “Done. (…)”

The number that explains everything

The implement phase made 295 model calls. Each call re-sends the entire (growing) conversation – that is how an agentic loop works: the model does not remember the last turn, it receives the whole conversation again. So the billed input is not “one big prompt”; it is calls x context. The average implement-phase context per call was about 117K tokens, and 295 of those calls accumulate to:

Scope	Input tokens	Output tokens
Worker (goose), whole night (335 calls)	~35.0M	~0.2M
Supervisor (Hermes), whole session (413 calls)	~36.4M	~0.5M
Both tiers	~71.4M	~0.7M

The supervisor row is the whole session (the night plus the next morning’s journal and remediation work), so it is an upper bound for the night alone; the worker row is night-only by construction.

Output is 0.6% of the input. The cost of agentic inference lives on the input side, and it scales as calls x context – not as one big prompt. Two consequences, both measurable:

• The 256K context window is decisive. A run this long lives or dies on the window size that is available. And this setup held, even with the deployed 27B model, with reserve to spare. If any context compression occurred at all, it did not lead to visible quality losses.
• The total token consumption is what you see when the GPU is then properly put to work: seventy million input tokens across both tiers on one workstation card overnight – and it did not need to prefill from a data center.

Laying a market price for a public API rate of the deployed LLM to ground, roughly $12 would have been billed for the whole run, both agents taken together. With a locally running LLM server, electricity is the only real cost: the card consumes a practically constant 300W while it processes the requests. Measurements of the workstation’s idle consumption still have to be determined, I will report on them in a later piece.

Against a token bill of roughly $12, the electricity consumed is categorically much less.

“Error-free” – the claim, honestly checked

I tested the generated app myself, and reviewed it. I can actually confirm that the generated code is error-free. And honestly, I would have to admit that I probably would not have written this source code as cleanly myself.

In addition, the supervisor ran the gate checks independently as well: 40 tests at 82% coverage, ruff clean, strict type-check clean, plus a live end-to-end acceptance with a newly set-up database, including proof of persistence beyond a process restart.

The verification chain is three-deep: the coding agent’s own test runs, the supervisor’s independent checks, and then my usability tests and code review.

And the honest gap, which I do not want to sweep under the rug:

In this experiment, the automatic checks that were originally intended as pre-commit gates before every commit into the git repository were not activated; rather, they were silently dropped – my setup was not yet aligned with spec-kit at that point, so the rules that normally apply at setup time were outside the scope, from spec-kit’s point of view.

I caught it only the next morning and fixed it with another prompt. The code passed its gates; the process had a hole the agent did not surface – it was simply not aware of it. This hole is the subject of a later article (establishing an automated check for the adherence to quality standards) – this is going to be a very interesting story.

What I am not claiming: this construction “works flawlessly on large projects.” That claim would need a controlled measurement process – a fixed task set across several configurations – which I will keep advancing in the next articles of this series and report on.

What I can already say as a felt observation:

For the first time, a duo of agents with a local model managed to carry a project through to the end, instead of failing at the difficult part of implementation, because the context keeps swelling and the information load then often becomes too much for the LLM, so that the local models previously used regularly stumbled and no decent code could be generated anymore. Debugging was then usually out of the question altogether, because the context window would usually have had to be much larger still, in order to hand the LLM, in addition to the initial information and the faulty code, even debug information to process and to fix the error.

Until now I had regularly observed that LLMs, on reaching a fill level around the 60–80% mark, begin to no longer be able to output texts precise enough (code is very precise text). This weakness was no longer visible in the experiment, which, however, cannot be attributed solely to the model used, but to the entire construction used: LLM, two fitting agents working together with the spec-driven approach of spec-kit as a whole.

Where this leads

That was the Ollama proof-of-life. The next step will be to use the same two-tier setup for real projects – where the token throughput will then play an increasingly larger role, since with growing project size the compute times rise. For a productive deployment of this setup, it is not only the quality of the generated files – which of course has top priority – that matters enormously, but also the wait time until these files are available.

Therefore, next to the feasibility of this agentic approach, above all the speed – i.e. here specifically the token rate with which the hardware can assemble text during inference – will come into focus. Then the question arises whether Ollama – a good general-purpose server – is still suitable to supply an agent night with 295 calls. With vLLM there exists, for example, a serving engine that is cut for exactly this use case.

So I will then set up a dedicated vLLM backend with two quantized instances side by side for my workstation, and load quantized models specifically optimized for the hardware that promise a significantly higher token rate. There will be an article about that, too.

For now, though, I can state: the frustration of the 60–80% skid? Gone!

Appendix: the two messages, in full

The prompt (01:30)

Sent 2026-08-21 23:30:02 UTC (01:30:02 local) as the first message of session 20260821_233002_17f34f24. Verbatim from the orchestrator’s session store, no normalizations; the folder paths and product names are, as given (public).

https://youtu.be/k2LLUKj6piE?si=nc4sB730kE6dC23Z

Can you setup spec-kit and follow through the demo, but instead of using IBM Bob as coding agent, you should use "goose" which is already installed. Work in the folder ~/projects/goose/projects/speckit_shoppinglist. Make it use Sqlite, fastapi, nicegui. For the constitution import the ~/projects/goose/projects/mydu_tkinter/guidelines/ files and refer to the imported files. Do you have any questions about this assignment?

The reply (06:29)

The complete message, verbatim from the orchestrator’s session store, as it was sent on 22 August 2026 at 06:29. Two normalizations, both disclosed: punctuation in plain ASCII (dashes and arrows), and one internal machine name replaced by “the box” (the redaction standard the source note cites).

Done. The full Spec Kit demo ran through with goose (not IBM Bob) in `~/projects/goose/projects/speckit_shoppinglist`, using FastAPI + SQLite + NiceGUI, and the mydu_tkinter guidelines imported verbatim. Everything is committed (abc2da6, 57 files, tree clean).

## What was set up
- `specify init --here --integration goose --script sh` (v0.16.5) -- scaffolded `.specify/`, `.goose/recipes/` (the six `/speckit.*` commands as goose recipes), `AGENTS.md`
- `guidelines/` -- the 5 mydu_tkinter files copied verbatim (diff-verified), as you chose
- Goose config untouched, per your instruction; it ran on the default goose provider pinned to the box (`qwen3.8:27b-mtp-q8_0`)

## The workflow, executed via goose
| Step | Artifact | Result |
|---|---|---|
| `speckit.constitution` | `.specify/memory/constitution.md` v1.0.0 | 6 principles: simplicity, fixed stack (FastAPI/SQLite plain SQL/NiceGUI), keyboard accessibility, local-only data, no unnecessary deps, **imported guidelines as binding authority** -- referenced by path only, guidelines win conflicts |
| `speckit.specify` | `specs/001/shopping-list/spec.md` | 4 user stories (add P1, toggle P2, delete P2, persist P3), Given-When-Then, FR-001..011, edge cases, 16/16 checklist. Non-goals: no auth, no sharing, no sync, no categories |
| `speckit.plan` | `plan.md` + research/data-model/contracts/quickstart | Constitution check: **all principles PASS, 0 violations** |
| `speckit.tasks` | `tasks.md` | 27 tasks, 7 phases, TDD-ordered, scenario->test traceability |
| `speckit.implement` | `src/shoppinglist/` (app, db, ui, `__main__`, 4 test modules) | 27/27 `[X]` |

The implement run took ~4.5h on the local 27b. Notable fights it worked through: a real NiceGUI async-handler bug (coroutines never awaited), a cross-thread event-loop issue in the UI test harness (fixed with a single-loop `prepare_simulation` setup), and basedpyright strict-mode cleanup (module-level routes, `exclude=["tests"]` reconciled with the guideline's `basedpyright .` command).

## Verification (my own independent re-runs after the run finished)
- `uv run pytest` -> **40 passed, 82% coverage** (gate: >=80)
- `uv run ruff check .` / `ruff format --check .` -> clean
- `uv run basedpyright` (strict) -> **0 errors, 0 warnings**
- Live E2E on a wiped DB, `127.0.0.1:8080`: empty list -> add milk+bread -> whitespace rejected (422) -> toggle (purchased) -> delete (204) -> **killed the process, restarted, state survived**: `[{"id":1,"name":"milk","purchased":true}]`. UI page 200, NiceGUI shell with 32 framework markers. Persistence across full process restart is proven twice (also inside goose's own Q4 test).

## Two things worth knowing
1. **Quirk discovered:** this machine's CPython build (3.14.3, "Mar 2026") accepts bare-comma multi-exception `except A, B:` clauses -- nonstandard, but consistently so across system and venv interpreters, which is why the app compiles and passes everywhere. Standard-grammar linters (like mine) flag it; the project's own toolchain does not.
2. `goose run --params` silently corrupts multi-line values (YAML re-serialization) -- single-line params only. Worth remembering for future goose recipes.

To run it: `cd src/shoppinglist && uv run python -m shoppinglist` -> http://127.0.0.1:8080/ (overrides: `SHOPPINGLIST_PORT`, `SHOPPINGLIST_DB`). The UI screenshot was skipped -- the headless `test_ui_smoke.py` (4 tests, in the green suite) already drives the real page in-process.

Want me to write the `making-of.md` (architecture, the four failure loops, decisions -- per your journaling convention), or leave the repo as is?