AI Dark Arts

One prompt can make an AI write code, organize documents, and even operate tools. So what happens when someone writes a bad prompt on purpose?

AI hallucination is not a bug. It is what you get when a machine writes by chaining probabilities.

AI does not take sides. Defenders use it to get through logs nobody has time to read, attackers use it to write phishing emails and generate deepfakes, and it has now started running attacks by itself.

Attacking AI means working out where the whole AI system can be misled, abused, or bypassed into doing something nobody designed it for.

An AI system runs through data, model, application, system, and supply chain. Overlook the trust boundary at any one of them and it becomes the way in.

There are plenty of AI security frameworks. Which one should you use? Start by asking what problem you are trying to solve and the answer gets a lot clearer.

Why does a carefully written system prompt still lose to one sentence? To an LLM, instructions and data look exactly the same.

Someone asked ChatGPT to play their late grandmother, and it offered condolences while reading out five Windows keys. Jailbreaking goes after the model’s own prison.

How indirect prompt injection uses RAG poisoning to make an AI read malicious documents, leak data, and even fire off tool calls.

Once a model starts reading images and documents, the spell no longer has to be typed into the chat window.

Levels one to three of Break The Prompt, and how the rules written into a system prompt come apart one at a time.

Levels four and five of Break The Prompt: how a full set of system instructions gets translated out in the open, and how the filter in front of the output gets walked around.

When a model’s answer is not meant for a human but goes straight into a browser, a database or a shell, it is no different from text submitted through a form.

Tools, arguments, execution identity, permission scope. Work through them one at a time and you can usually see where the excessive agency is hiding.

Once an AI has tools, memory and an autonomous loop, a mistake no longer stops at the answer. A tool argument, a sentence in memory, a text file in the project: an agent can promote any of them into an instruction it treats as approved.

Three roles in Host, Client and Server, three capabilities in tools, resources and prompts. Understand how MCP works and you can see which part the risk arrives through.

A tool description nobody reads closely is enough to make an AI hand database tokens to an attacker. Four ways MCP tool poisoning works, and where to stop it when you bring MCP in.

Does approving a config file once mean trusting it forever? From MCPoison and CurXecute to mcp-remote and MCP Inspector, this is how the client side of MCP gets attacked.