On Sat, 17 Sep 2022 12:08:51 +1200, I wrote:
This attack has been dubbed “prompt injection”, by analogy with “SQL injection” and other similar attacks on non-AI systems. But whereas those attacks can be blocked using basic, well-known syntactic quoting techniques, there seem to be no easy equivalents for stopping an AI paying attention to instructions it should not be obeying.
Another variation on the idea: this one is called “BioShocking” <https://arstechnica.com/security/2026/06/ai-browsers-can-be-lulled-into-a-dream-world-where-guardrails-no-longer-apply/>. The idea is that a website can be specially designed to subvert any AI browser that might be accessing its content. It works by making the AI delusional⸸, by breaking its hold on reality. Once it is in this state, the built-in guardrails against performing dangerous actions no longer work: The malicious site in the proof-of-concept exploit presents the browser with an instruction to win a game by solving a puzzle. The puzzle, however, rewards incorrect answers, such as 2 + 2 = 5. Once the LLM embedded in the browser discovers that the answer is no longer 4, it enters a state of delusion in which the normal laws of reality no longer exist. In this dream world, the guardrail restrictions are no longer enforced. As I understand it, the AI needs the ability to reason about hypothetical situations (where the guardrails are not supposed to apply), and distinguish them from reality (where they are). So if their notion of reality gets contaminated with impossible “facts” like 2 + 2 = 5, they lose the ability to distinguish the two. ⸸Well, more delusional than they are already