Practical guide · Source verification
How to verify citations in an AI answer
Treat every AI-supplied citation as an unverified lead. Open the source, match its identity and metadata, find the passage that supports the nearby claim, and check whether the source is current and appropriate for that claim. A working link proves only that a page exists.
Published October 18, 2026 · Reviewed October 18, 2026 · 12 minute read
The short answer
Split the answer into checkable claims. For each claim, open its citation and record four results: the source exists, the title and author match, the source directly supports the claim, and its date, status, population, and evidence type fit your use. Mark any failed or uncertain check. Remove unsupported claims instead of asking the same model to reassure you.
Why a polished reference can still be wrong
A model can produce a plausible title, author list, journal, and DOI without retrieving a real record. It can also cite a real source with incorrect metadata or attach a real source to a claim that the source never makes. Walters and Wilder checked 636 references in 84 generated literature reviews. In that 2023 experiment, 55% of GPT-3.5 references and 18% of GPT-4 references were fabricated. Their Scientific Reports paper describes that fixed model and prompt set, so its percentages should not be presented as current rates for every AI tool.
Search-enabled products reduce one failure path because they retrieve records, but retrieval does not prove claim support. OpenAI exposes cited URLs and the larger set of consulted sources through its web search API. Anthropic can return exact cited passages from supplied documents, and its hallucination guidance recommends retracting a claim when no supporting quote can be found. You still need to judge whether the passage supports the wording you plan to use.
Separate four checks
-
1. Existence: can you retrieve the claimed source?
Open the cited URL. For a DOI, resolve it through
https://doi.org/. For a scholarly title without a DOI, search the exact title in a bibliographic index and then search the first author with distinctive title words. Crossref's REST API documentation explains how to retrieve a DOI record and search publisher-deposited metadata. No result means “not verified,” because books, reports, new preprints, and some journals may not appear in one index. -
2. Identity: does the record match the citation?
Compare title, authors, publication, date, volume, pages or article number, and DOI. Do not accept a DOI merely because it resolves. It may lead to a different paper. When Crossref lacks the item, try a suitable domain database or an open scholarly index. The OpenAlex works API covers articles, books, chapters, datasets, dissertations, and preprints, and it accepts persistent identifiers for direct lookup.
-
3. Support: does the source justify this exact claim?
Find the sentence, table, figure, method, or result that supports the claim. Record the page, section, or quoted passage. Check qualifiers: sample, geography, date range, comparison group, uncertainty, and whether the result shows association or causation. If the source cites another work for the point, follow that reference and cite the original when you can read it. A source can be real and relevant to the topic while offering no support for the sentence beside it.
-
4. Suitability: should you rely on it here?
Check publication status, corrections, retractions, conflicts, method, recency, and source type. A vendor benchmark can document the vendor's result, but it does not become an independent comparison. A preprint can support a description of preliminary work, but label it as a preprint. COPE's discussion of AI and fake papers keeps responsibility for fact-checking and citation verification with the human author.
Worked example: an impressive percentage with a real paper
Suppose an AI answer says, “ChatGPT fabricates 55% of citations,” and cites Walters and Wilder. The DOI resolves, the title and authors match, and the paper reports 55% for one set of GPT-3.5 outputs. The same study reports 18% for GPT-4. The source exists and the metadata passes, but the broad claim fails the support check because it removes the model version and experimental setting.
A supported rewrite would say: “In Walters and Wilder's 2023 experiment, 55% of references generated by GPT-3.5 and 18% generated by GPT-4 were fabricated across 84 short literature reviews.” That sentence keeps the sample, model versions, date, and measured outcome. It does not claim that the result estimates the current performance of all search-enabled assistants.
Choose the right verification path
| Source type | Start here | Then verify |
|---|---|---|
| Journal article or proceedings paper | DOI resolver, Crossref, publisher | Metadata, passage, study design, correction or retraction status |
| Preprint | Named repository record | Version, date, authors, withdrawal status, peer-review status |
| Official product behavior | Current first-party docs or changelog | Exact version, plan, region, date, and whether the behavior was tested |
| News or public event | Original filing, statement, transcript, or data release | Event date, publication date, quotation context, later correction |
| Vendor benchmark | Vendor report and disclosed method | Dataset, baseline, sample size, exclusions, reproducibility, independent evidence |
Verify a long answer without losing the claim trail
- Copy factual claims into a table with one row per claim. Keep the original wording.
- Add the cited URL, source type, publication date, and the AI tool that supplied it.
- Run fast existence and identity checks in batches, but keep “not found” separate from “fabricated.” A timeout or incomplete index proves neither.
- Read every source used for an important conclusion and paste the supporting passage or record its location.
- Mark each row supported, contradicted, uncertain, or out of scope. Rewrite or remove failed claims.
- Ask a second person to inspect high-impact claims when the answer informs health, law, finance, safety, policy, or publication.
Automation can resolve links and compare metadata, but a script cannot reliably decide whether a nuanced source supports your exact wording. Keep that judgment visible and reviewable.
Limits and common mistakes
- Asking the original model “Are you sure?” does not create independent evidence.
- A working URL and matching title do not prove claim support.
- A missing DOI does not prove fabrication. Many legitimate source types do not use one.
- A search snippet can omit the qualifier that changes the meaning. Open the source.
- Several articles may repeat one unsupported claim. Trace it to the earliest inspectable evidence.
- Recent sources can be weaker than older primary research, while old product documentation can describe behavior that no longer exists.
- Source count does not measure answer quality. One direct primary source can support a claim better than ten topical links.
Verification checklist
- □ Every factual claim has a source or is clearly labelled as interpretation.
- □ Each cited source opens, and its canonical record matches the citation.
- □ The supporting passage, table, or result has been read.
- □ The wording preserves population, date, uncertainty, and evidence type.
- □ Corrections, retractions, versions, and current product behavior have been checked.
- □ Vendor and creator claims are labelled rather than presented as independent findings.
- □ Unverified claims are removed or marked uncertain.
- □ High-impact conclusions receive independent human review.