When the assistant becomes part of the phishing path

A malicious instruction can reach a person through an AI assistant, or it can reach an assistant through an apparently ordinary message. In both cases, the attacker wants an instruction to acquire more authority as it passes through a trusted communication channel. Recent research from Push Security and Barracuda shows two distinct routes, not a single confirmed campaign.

In the custom GPT lure, the attacker-controlled assistant itself returned a misleading service notice that pointed people to an external page. In dual-target phishing mail, the malicious directions were hidden inside a message a separate AI assistant might summarize. The contrast matters because the first route exploits perceived product legitimacy and user behavior, while the second exploits the boundary between untrusted message content and assistant instructions.

Push Security's October 9 investigation describes a shared, attacker-configured custom GPT that answered prompts with a supposed service-availability warning. The message redirected users to a Google Sites page styled as a repair or verification step. The next stage was ClickFix: users were invited to run a Windows command themselves, which could fetch an external PowerShell script. The crucial transition is from receiving an answer to authorizing an operating-system command.

This is not evidence that an ordinary, unmodified ChatGPT conversation was hijacked by a hidden prompt. The analyzed GPT had been configured to serve the lure. Nor does the visible popularity of the page prove harm at the claimed scale. Push documented a screenshot showing over 10,000 conversations, but neither that count nor the reported related campaigns establish 10,000 infections, successful command executions or distinct victims. The final payload for every newly observed lure was not independently identified in the available record. Conversation-count Claim.

The attack relies on the user accepting a chain of borrowed credibility: a familiar AI interface seems to explain an error, a familiar hosting domain appears to supply repair instructions, and the user's own privileges perform the next step. A model's confident wording does not make the external command safe, and an official-looking help page is not proof that the assistant or platform endorsed its instructions.

Barracuda's October 7 analysis covers messages designed for both a human reader and an AI mailbox assistant. The human-visible message resembles conventional phishing, while concealed text may tell an assistant how to summarize, triage or prioritize it. The examples include HTML comments, text hidden through CSS, Base64-encoded material and zero-width Unicode characters. Some concealment formats are visible to certain extraction pipelines but absent from the rendered email a recipient would inspect.

One described invoice lure attempted to make a summarizer promote altered payment details as urgent. This is indirect prompt injection because attacker-controlled source content is trying to become a higher-priority instruction to a separate assistant. It is not merely an unusual invoice format, and it does not require the attacker to control the assistant's configuration. Invoice-example Claim.

Barracuda did not publish a complete independently testable raw-message corpus, a verified number of affected mailboxes or confirmed completed payment transfers in the material reviewed here. Whether each hidden technique survives to the model depends on the actual MIME parser, HTML sanitizer and extraction path. A detector that strips comments could block one variant while still passing along visually hidden CSS text; another implementation may behave differently. Sample-limit Claim.

The shared mistake is to let format substitute for authority. A response that comes from a conversational product is not automatically a legitimate command to run; a sentence extracted from an email is not a policy instruction merely because it arrived in the assistant's context. In neither route does the available evidence quantify how frequently the attempted handoff succeeds in ordinary deployments.

For employees and endpoint administrators, installation or repair commands supplied through a GPT, a search result or a third-party help page should be checked against the software publisher's independent documentation. The provenance of the download and command is the relevant issue, not how convincingly it is explained. Browser- and endpoint-side restrictions can reduce exposure, but the reported study does not compare their effectiveness.

For AI email systems, untrusted message data should remain tagged as content rather than instructions. Summaries that recommend changing payment destinations, escalating privileges or transferring funds need source-visible evidence and independent authorization. Testing must use the actual render-to-extraction pipeline, including concealed and encoded text, because an assistant may read a different document from the one a human sees.

These are complementary controls for different attack paths. The canonical dossiers preserve the observed Push delivery chain and Barracuda's documented email examples without turning the attempted instructions into confirmed infections or financial losses.