When the Fast Dumb Approach Beats the Smart One

My final year project was a platform that takes a video and produces useful things from it. Summaries, timestamped answers to questions, highlight reels, even a generated podcast discussing the content.

The highlight reel feature is where I learned something that has stuck with me.

Two approaches, both bad

The problem is straightforward to state. Given a long transcript with timestamps, pick the segments worth watching and produce a short reel with titles.

The obvious approach is to ask a model. Hand it the transcript, ask for the best moments and titles. This produces good titles. The segment boundaries are unreliable, it is slow on long transcripts, and it will occasionally invent a timestamp that does not exist in the source, which then breaks the clipping step downstream.

The other approach is statistical. Score segments on word frequency, structural position, changes in speech density, and pick the peaks. This is fast, deterministic, and the boundaries are always real because they come from the transcript. The titles are terrible. You get things like "Discussion Of The Main Point" because there is nothing generating language, only ranking existing text.

I spent a while trying to make each one do the whole job. Better prompting to fix the boundaries. Better heuristics to fix the titles. Both were uphill.

Splitting by what each is actually good at

The version that worked stopped treating it as one problem.

The statistical pass picks the segments. It is fast, it never hallucinates a timestamp, and picking high signal regions of a transcript turns out to be a job that simple scoring does perfectly well.

Then, with the boundaries already fixed and validated against the source, the model writes titles and descriptions for those specific segments. That is a language task, which is what it is good at, and it cannot damage anything structural because the structure was decided before it was called.

Faster than the model doing everything, and better titles than the heuristics doing everything.

The mental model

The framing I took from this is that "should I use an LLM for this" is usually the wrong question. Most tasks are several tasks in a coat, and the parts have different shapes.

Anything where correctness is checkable and structural, timestamps, ranges, IDs, anything that has to line up with a source of truth, is usually better handled by code you can verify. Anything requiring language, judgement or summary is where the model earns its place.

You get the second one to run inside constraints the first one has already established. Then a bad generation produces a mediocre title, not a broken output.

Where else this shows up

Once you start looking, this pattern is everywhere in AI systems.

Deduplicating documents before a model sees them is the same idea. Validating that a generated citation actually exists is the same idea. Constraining a model to a fixed set of tool calls rather than free text is the same idea.

In each case something deterministic defines the boundaries and something generative works inside them.

The version of me that started that project assumed better AI engineering meant handing more of the problem to the model. It is mostly the opposite. It means being precise about the specific thing you need the model for, and building enough structure around it that the parts which must be correct do not depend on it.