An LLM cannot watch a video.
It's worth being precise about this, because the interfaces hide it. When you hand a model a video, nothing in it is watching. Somewhere upstream the file is decoded, a small number of frames are pulled out, and those frames plus a transcript are what the model actually sees. The video was converted into something readable before the reasoning started.
That conversion is the entire problem. Do it carelessly and the model reasons confidently about a version of the ad that lost the thing that made it work. Do it well and a fifteen second video becomes a paragraph a model can compare against ten thousand others.
We do that conversion for every live ad we collect. This is what it involves.
Taking the ad apart.
An ad is not one artifact. It's four, and the meaning is distributed across them. A hook lands in the first second of video, the offer sits in the overlay text, the claim is in the copy, and the real promise is on the page after the click. Read any one of them alone and you have a fragment.
Decode, detect scene changes, sample frames where something actually changes, then describe the action rather than the pixels.
Transcribe the audio, OCR the on-screen overlays, parse the ad copy and headline into claim, offer, and format.
Follow the click, render the page, capture the promise, the funnel, and where it diverges from the ad.
Frame sampling is where most naive pipelines quietly fail. Sampling at a fixed interval gives you three near-identical frames of a talking head and misses the cut where the product appears. Sampling on scene boundaries costs more up front and captures the ad's actual structure: hook, demonstration, proof, call to action.
Putting it back together.
Now you have a transcript, a set of frame descriptions, OCR fragments, copy, and a page. Concatenating them is the obvious move and the wrong one. You get a pile of observations with no idea which mattered.
The reassembly step turns those signals into one account of what the ad does. Not "a woman appears, then text appears, then a bottle appears," but the hook it opens with, the claim it makes, the offer it lands, and the format it uses to get there. That's the level at which ads are comparable to each other, and comparison is the point.
It also has to be consistent. Two advertisers running the same free-trial offer need to come out described the same way, or nothing downstream can group them. Every attribute resolves into a controlled vocabulary rather than whatever adjective the model reached for that run. Consistency is what makes the data queryable instead of merely readable.
The token budget.
Raw creative is enormous. Frames, transcripts, and a full landing page for a single ad will fill a serious fraction of a context window, and a question about a market touches thousands of ads. Push raw creative into a model and you can afford to ask about one advertiser, once.
So the pipeline is also a compression problem. Keep what supports reasoning, drop what only supports rendering. Colors, fonts, and exact framing go. The hook, the claim, the offer, the format, the funnel, and the entity behind it stay.
The test is simple. If a model can answer the same questions from the processed version that it could from the original, the compression was lossless where it counted.
Losing meaning at scale.
Everything above works fine on one ad. Scale is where the failure modes live, and most of them are silent.
Drift is the first. Run the same creative through two model versions and you get two vocabularies, so ads processed in March stop being comparable to ads processed in August. The fix is boring and mandatory: a fixed schema, a pinned vocabulary, and reprocessing when either changes.
Variants are the second. Advertisers ship the same ad twenty ways: different thumbnails, different first three seconds, different length cuts. Treat them as twenty ads and your market data inflates. Collapse them too aggressively and you erase the test the advertiser was running, which is often the most interesting signal on the page.
Freshness is the third and never ends. Campaigns rotate weekly. A library processed last month describes ads nobody is running. Processing has to be incremental and continuous, or the whole corpus decays into history.
What a model finally sees.
At the end of it, an ad is a compact, structured record: the advertiser it belongs to, resolved to a real company and graded on spend, traffic, and sales from D to A+; the creative broken into hook, claim, offer, and format; the copy and headlines; and the page behind the click.
Small enough that a model can hold a whole category at once. Structured enough that it can filter, group, and compare. Faithful enough that the conclusions it reaches are about the ads and not about our summary of them.
That's the job. We render the ads into tokens. Your AI does the thinking.