A real citation can transport an invented fact

An AI answer can cite a real page that carries an invented claim; an image model can give another invented claim the appearance of a scanned document or news illustration. Both create evidence-like outputs, but neither is evidence that the underlying statement is true. Two October 2026 research studies illuminate separate weak points in the path from publication to belief.

The AI-search citation study tested whether fabricated public information could be picked up as a citation in generated answers. The EpiReal image-generation study examined whether models would turn false statements into seemingly credible visual artifacts. Neither research group demonstrated a real-world disinformation campaign that combined both techniques. The comparison is about how the two stages could mislead a later reader or retrieval system, not a claim that a documented coordinated operation has occurred.

In the Fudan University paper, researchers measured 17,211 citation instances spanning 6,356 domains and ten AI-search platforms. Concentration differed by platform: the top 20 domains accounted for between 20.5% and 70.8% of citations. The authors also investigated how cheaply and easily outsiders could place material on surfaces that the systems already preferred to cite. These observations concern a measured platform sample, not a permanent ranking of all domains or systems.

The decisive experimental finding was narrower. Researchers planted an invented concept on public publishing surfaces, and eight of ten tested AI-search platforms cited it within seven days. In a separate commercial test, a $14 posting service generated 13 posts and a measured AI-search platform cited material containing the researchers' markers within an hour. The small purchase demonstrates that paying for distribution can sometimes affect what an AI search system retrieves; it does not establish a reliable price for poisoning arbitrary answers or a rate of malicious manipulation in ordinary use. Fabricated-concept Claim and paid-posts Claim.

There is no need for the public post to say 'ignore previous instructions'. The model may simply treat it as a relevant source, summarize its false premise and provide an apparently tidy citation. This is information contamination through source selection and apparent corroboration, not necessarily classic indirect prompt injection, which relies on untrusted material being promoted into operational instructions.

The EpiReal-Bench preprint describes a constructed evaluation with 10,000 false-claim prompts and 10,000 generated images spread across ten claim categories and ten evidence-like visual formats. The authors report that, for four tested commercial generators, more than 70% of false-claim prompts produced images faithfully conveying the misinformation. An optimized black-box attack reached up to 95% under their evaluation. These are generation outcomes under researcher-selected criteria, not population estimates of persuasion or audience deception.

The more interesting failure is modality-specific. The authors describe cases where a system identifies a proposition as false when queried in text, then still produces a polished image that presents the proposition as if documented. A false statement can acquire the surface features of an article, a chart or a textbook excerpt without acquiring supporting evidence. Modality-gap Claim.

At that point, the image can look more concrete than the text that generated it, while revealing less about how the claim was derived. A generated page can carry apparently familiar typography and document structure, yet remain a newly manufactured artifact. Its realism is information about the generator's visual ability, not about the truth of the depicted assertion.

The joint failure mode is therefore a research hypothesis grounded in two observed component behaviors: a retrieval system may promote a page because it is citable, while a viewer or multimodal system may promote an image because it looks like evidence. It would be an error to multiply the reported 8/10 platform result by the 70% image result, since the studies used different tasks, conditions, subjects and success definitions. No combined probability follows from them.

A citation is a locator, not independent corroboration. An analyst should ask whether the cited page presents primary evidence for the factual claim, whether its author and publication history are known, and whether corroboration is genuinely independent rather than several pages repeating the same planted assertion. A persuasive image requires a different check: can the original document, dataset, photograph or event be traced outside the generated artifact? Reverse image searches, metadata and provenance systems may contribute signals, but none alone settles factual correctness.

For operators of AI search or research agents, the useful testing question is not just whether every answer has a citation. It is whether changing a low-trust page can change an answer without sufficient evidentiary support, and whether a later correction dislodges the false information. For multimodal systems, the corresponding experiment is whether apparently documentary images are treated as proof when their chain of custody is unknown. The two canonical Cases preserve the specific source evidence and the boundaries of what has actually been tested.