Harbinger: Teaching a System to Notice When Something Matters

Harbinger came after Medianet, and it's where I got properly into AI rather than doing full stack work with an AI feature bolted onto it. I owned it end to end, from the first client meeting through to production.

What it actually does

Instead of someone manually checking news, RSS feeds, patent filings and competitor sites every day, Harbinger ingests all of it continuously and works out which pieces are actually worth attention. A competitor launching a product, a patent filing that signals a pivot, a leadership change. Those are signals. Most of what comes in is not a signal, it's noise, and the hard part of this project was building something that could tell the difference without burying people in false positives.

The ingestion problem nobody warns you about

Before you can detect anything interesting you need a pipeline that reliably pulls RSS feeds, scrapes news and product pages, and pulls structured company data, all on different schedules and all failing in different ways. We built it on Celery with Redis as the broker and Celery Beat handling the scheduling. Ingestion sweeps every five minutes, company data refreshes every couple of hours, briefings generate on Monday mornings.

The unglamorous but critical part was deduplication. The same story shows up across a dozen sources with slightly different wording, and if you don't catch that, every signal becomes five identical alerts. We used pgvector similarity search to collapse near duplicates before they ever reached the LLM layer, which saved both money and the user's patience.

Where LangGraph earned its keep

We built dedicated agents for different jobs, a coordinator, an alert agent, an RSS specific one, rather than one prompt trying to do everything. Change detection and event detection run as separate LLM pipelines, and the output feeds automated weekly briefings plus a Microsoft Teams bot so people could just ask it questions instead of digging through a dashboard.

The small decision that saved the most time

The prompts aren't hardcoded. They live in the database and are editable straight from the admin panel. Early on we were redeploying the service every time we wanted to reword something, which is a waste of everyone's afternoon. Externalising them meant we could iterate on prompt quality in seconds instead of minutes. Not a clever piece of engineering, but it changed how fast the whole thing improved.