The week-two audit: duplicates, citations, quiet days, missing sources.
A prompt and a scheduled job make a convincing briefing fast. Four checks show what it misses once a week of editions piles up.
Day nine of a briefing you built yourself
Picture a hypothetical briefing someone set up on a Sunday afternoon: a prompt, a model with web search, and a scheduled job that runs at 6 a.m. The first week's editions are sharp. Five stories, clean summaries, a link under each one. For a weekend project, it feels almost too easy.
By the second week, small things start to itch. Tuesday's lead story is the same wire report as story three, rewritten by a different outlet. One link opens to a page that mentions the company but not the number the summary attributes to it. And Saturday, a day the field was dead quiet, still produced five stories, two of them from last month.
None of this shows up on day one. That's the problem.
The week-two gap
A prompt plus a scheduled job gets you a fluent briefing almost immediately. Fluency is what the model does best, so the first editions read as finished work. Accuracy, deduplication and honest silence are slower properties. You only see them once you have enough editions to compare against each other and against the sources.
We call this the week-two gap: the distance between how good a DIY briefing reads and how well it holds up when you check it. A single edition can't reveal it. A week of saved editions usually can.
So before you trust the thing for work, run a short audit on purpose. Four checks.
Why a better prompt only gets you partway
The DIY guides circulating right now are useful, and builders who follow them get a better briefing than builders who wing it. The common recipe is sensible: pick a model that can search, write a prompt that names the field and the output format, require a source link for every item, and put the whole thing on a schedule.
But a prompt is an instruction to the same system whose output you're trying to check. Asking a model to avoid duplicates gives it no reliable way to tell that two articles share one wire origin. Asking it to cite sources produces links, and nothing in the loop confirms each link supports the sentence above it. A prompt that asks for five stories gets five stories, every day, whether or not five things happened.
The weaknesses sit in the parts a prompt can't see. The audit checks those parts directly.
What research on AI news answers keeps finding
Two large studies of AI assistants answering news questions point at the same weak spots the audit targets. In research coordinated by the European Broadcasting Union and led by the BBC, published in October 2025, journalists from public service media reviewed assistant answers across many countries and languages and found significant problems in a large share of them, with sourcing the most common issue. Earlier, in March 2025, the Tow Center for Digital Journalism at Columbia tested eight AI search tools on identifying the source of news excerpts and found them wrong more often than right, usually without signaling any doubt.
The EBU/BBC News Integrity in AI Assistants study found 45% of all AI answers had at least one significant issue and 31% showed serious sourcing problems. (source)
Neither study tested a homemade briefing pipeline. The mechanism is the same, though: a model retrieving news, summarizing it and attaching sources. If assistants built by large teams stumble on attribution this often, a weekend build deserves at least the same scrutiny.
- 45% of AI assistant answers about news had at least one significant issue (EBU/BBC 2025)
- 31% of those answers had serious sourcing problems (EBU/BBC 2025)
- 60%+ of queries answered incorrectly across eight AI search tools (Tow Center 2025)
How to check an AI news briefing: four checks on a hypothetical week
Save a week of editions first, then run each check across all of them using the worksheet below. To show what it catches, we'll walk a hypothetical: an imagined daily briefing for someone following medical-device regulation. The stories and results are illustrative, not drawn from a real build.
Check one is duplicates. Group stories by the original event and the original reporter. In our hypothetical week, one wire report on a device recall appears Monday under one regional outlet's byline and again Wednesday under another's, summarized as two separate developments. That's one story.
Check two is citations. Open every link and mark it: supports the specific claim, mentions the topic only, or dead. The hypothetical turns up a summary stating a clearance date the linked article never gives. The link is real. The claim isn't in it.
Check three is quiet days. Find the slowest day and read what the briefing said. Ours filled its five slots anyway, padding with a month-old guidance document and an opinion column restated as news.
Check four is absent sources. List every domain cited across the week, then list what a careful reporter on the beat would use: the regulator's own announcements, the trade press, court filings. In the hypothetical, the regulator's site never appears once. That gap is a blind spot you'd inherit.
| Check | How to run it | Mark each item as | What a failure tells you |
|---|---|---|---|
| 1. Duplicate origins | Group a week of stories by original event and byline; trace rewrites back to the wire report or press release | Unique / syndicated copy | The briefing can't tell a story from its echoes, so volume overstates what happened |
| 2. Citation support | Open every link and find the sentence that supports the summary's specific claim | Supports / topic only / dead link | Links are decoration; specific claims still need checking by hand |
| 3. Quiet-day behavior | Read the edition from the slowest day; check each story's date and whether it's news | Honest quiet / padded | A fixed story count is forcing output, and stale or minor items fill the slots |
| 4. Absent sources | List cited domains for the week; compare against the sources a careful reporter on the beat would use | Present / never cited | The briefing's view of the field has holes you'll inherit |
The week-two audit worksheet. Run it on at least a week of saved editions.
Save the editions before you audit
If your setup only emails its output or overwrites a file each morning, start archiving now. Checks one and three can't be run on a single edition. They need a week of history to compare.
What changes once you've run it
If the week-two gap is real, the question for a DIY briefing shifts from whether it reads well to which of the four checks it passes.
Some fixes are cheap. Letting the prompt say there's no significant news today usually helps with quiet-day padding. Others are hard. Merging syndicated copies means comparing articles against each other, which a single prompt run doesn't do on its own, and citation checking needs a second pass that actually opens each source and reads it.
Be honest about which checks you've automated and which you're quietly trusting. A briefing that fails check two is fine for a skim. It's a liability for anything you'd quote in a meeting or put in a newsletter. And check four never fully goes away for any briefing, because absence is invisible from inside the output.
Where a checked briefing fits
Three of the four checks describe work Niche does before a story reaches a briefing. Syndicated copies of one origin are merged into a single story. Citations that don't hold up are dropped, and the supporting sources stay attached. On a day with no new stories, the briefing page records a quiet day and the email is skipped. Source checking is no guarantee that every claim is true, and we'd still recommend running check four on any briefing you rely on, ours included.
You don't have to throw out an agent workflow you've built. An assistant can read Niche's public briefings free over MCP, so checked stories and their sources can feed the notes, summaries or drafts you and your assistant already write. Custom scans and story angles require Pro. The integrations page walks through the setup.
Frequently asked questions
How do I check an AI-generated news summary?
Open every cited link and find the sentence that supports the summary's specific claim, such as a date, number or quote. Mark each one as supporting the claim, mentioning the topic only, or dead. A link that loads can still fail to back up what the summary says.
Why does my AI daily briefing repeat the same story?
Wire reports and press releases get republished and rewritten by many outlets, and a model searching the web often treats each copy as a separate development. Group stories by original event and byline to find the duplicates.
Can I fix these problems with a better prompt?
Partly. Letting the prompt report a quiet day usually helps with padding. Merging syndicated copies and verifying citations need comparison across articles and a second pass that reads each source, which a single prompt run doesn't do on its own.
How long should I run a DIY briefing before trusting it for work?
Save at least a week of editions and run the four-check audit on them. Duplicate and quiet-day problems only show up when you can compare editions against each other.
Can my AI assistant read Niche briefings?
Yes. An assistant can read Niche's public briefings free over MCP, with the same stories and source links as the site. Custom scans and story angles require Pro. The reader and their assistant do the writing.