What is prompt engineering?

What prompt engineering actually involves

The phrase suggests magic words, but the work is closer to testing software. You decide what a good output looks like, collect a set of inputs that represent real use, and measure how often the model gets them right. Then you change the prompt and measure again. Without the measuring step, prompt changes are guesses that happen to look good on the example you tried.

Both major vendors frame it this way. Anthropic's guide assumes three things before any technique: clear success criteria, a way to test against them, and a first draft to improve. OpenAI describes the goal as getting a non-deterministic model to produce output that consistently meets your requirements, and recommends building evaluations to measure prompt changes rather than eyeballing them. Anthropic adds a caveat worth keeping: some problems that look like prompt problems — latency and cost in particular — are often solved more easily by choosing a different model.

The techniques that matter

• Say exactly what you want. The task, the audience, the format, the length, what to do when information is missing. Most failures are instructions the writer thought were obvious.
• Show examples. A few input-and-output pairs teach format and tone faster than description. OpenAI advises covering a varied range of inputs so the model does not copy one example's quirks.
• Give the prompt structure. Headers or XML-style tags separate instructions from reference material; OpenAI suggests putting long context near the end.
• Use the roles. Standing rules belong in the system or developer message; the user message carries the specific input.
• Leave room to reason. On hard tasks, letting the model think before answering — or raising its reasoning effort — improves accuracy at the cost of extra output tokens.
• Split big jobs. A chain of small, checkable prompts is easier to debug than one prompt that does everything.

What a prompt costs

A prompt is input, and input is billed on every call. A 4,000-token set of instructions and examples sent with 100,000 requests a month is 400 million input tokens — before any user has typed a word. That makes prompt length a budget line, not a style choice. On GPT-6 Sol, fresh input is $2 per million and output $10, so both what you send and what you ask the model to write back carry a price.

Two habits keep it under control. First, trim: every example and paragraph should earn its place on the test set. Second, order the prompt so the part that never changes comes first — OpenAI recommends exactly this, because a repeated prefix can be served from the cache at $0.2 per million instead of the full rate. Asking for shorter answers helps on the other side of the bill: output is usually the more expensive token. The worked example prices the same prompt three ways.

Prompting reasoning models

Reasoning models change the advice. OpenAI's guidance is that they work best when given the goal and the constraints, and need less step-by-step direction than conventional models — closer to briefing a senior colleague than scripting a junior one. Detailed instructions on how to think can get in the way of the model's own planning.

The cost lever moves too. With a reasoning model, the expensive part is often the hidden reasoning tokens, billed as output. Setting the effort level to match the task is part of prompt engineering now: a classification job rarely needs the reasoning budget of a hard proof.

When prompting is the wrong lever

• The model lacks the knowledge. No wording supplies facts the model never saw; retrieve them instead with RAG.
• The model lacks the capability. If the test set shows a ceiling no prompt breaks, a stronger model is cheaper than more prompt iterations.
• The format must be exact at huge volume. A long prompt full of examples, repeated millions of times, can cost more than fine-tuning a model that needs a short prompt — where fine-tuning is still offered.
• The problem is latency or price. A smaller or faster model may solve it outright, as Anthropic notes.

Prompts in production

Treat prompts as code. OpenAI now recommends keeping them in the application's source, with typed inputs, code review and tests, and is deprecating its stored prompt objects in favour of that. Pin the application to a specific model snapshot so behaviour does not shift under you, and rerun the test set before switching versions — a prompt tuned for one model can quietly degrade on its successor, which matters every time a vendor schedules a model retirement.