Shorten a prompt without losing the task

Shortening a prompt is an edit, not a compression trick. You are trying to remove text that does not change the answer, and to keep text that does. A summarizer applied to your own instructions will happily delete "do not invent a tracking number" because that sentence looks small. Do not hand the prompt to a model and ask it to make itself shorter. Read it, with a count beside you, and delete things you can name.

Print the prompt with the token estimate next to each block. System rules. Examples. Retrieved text. The user turn. The counter is enough for the blocks that are English instructions. Sort the blocks by size. The largest block is the first candidate, not the adverbs. Teams waste a day on "utilize" versus "use" while a 900-token example sits below the fold. Look at the bottom of the template. Souvenirs accumulate there: an example from an incident in March, a paragraph someone added before a demo, a second statement of the JSON keys in prose because the schema "might not be clear."

Delete in this order. First, exact duplicates. If the policy appears in the system text and again in the user template, delete one. Second, examples that no longer match a failure you can reproduce. Rerun the case the example was added for. If the case passes without it, the example is a souvenir. Third, tone and origin stories: how the brand speaks, how the company started, a reminder to be helpful. Replace a tone paragraph with one line if the product actually needs a tone. Fourth, filler phrases, which the filler note lists. They are last because they are small. Doing them first feels productive and does not change the bill.

Keep these even when they look verbose. The verb. A prompt with no verb has nothing to shorten; it has something to add. The constraint that names a real incident, including the object: tracking numbers, refund amounts, medical claims, whatever your log showed. The output shape, in one place. One example that still changes the result on your regression tickets. A fence around untrusted text. Shortening that deletes the fence to save six tokens is a bad trade. Put the fence back and take the tokens from the origin story.

A before and after from a shipping route, compressed here to the point:

Before (the shape, not every word):
You are a helpful, warm, brand-aligned assistant for a company that has cared about parcels since 2014.
Please carefully answer the user.
Policy: do not invent a carrier or a date.
Here is the policy again in simpler words: never make up a carrier or a delivery date.
Example: [a 400-token transcript from a resolved chat]
Example: [a second transcript that makes the same point]
User: [the ticket]

After:
You answer shipping questions.
Do not invent a carrier or a date. If the record does not say, say you do not have it.
Respond with two short paragraphs and no preamble.
Record and ticket below, each fenced.

The after version can be longer than the cartoon above once you fence a real record, and it should still be hundreds of tokens smaller than a pair of transcripts plus a doubled policy. Count your real before and after. Then run the same twenty tickets. You are looking for a regression: a ticket the long prompt handled and the short one misses. If you find one, the missing piece is usually a single sentence hiding in the transcript, not the transcript's bulk. Lift that sentence into the constraints and leave the transcript out. You have shortened the prompt and kept the task. You have also made the task legible, which the transcript was not.

Beware of "shorter" prompts that are harder. A stack of abbreviations the model has never seen is not a saving if it increases retries. "NICT" to mean "do not invent a carrier or a tracking number" will be interpreted inconsistently, and you will spend the saved tokens on failures. Use plain words. Plain words are cheap. Clever codes are a second language you now have to maintain. The point of the edit is to remove text that does no work, not to encode the work in a private cipher.

Retrieved documents are part of the prompt and they are where shortening actually changes a budget. If you concatenate five pages and the answer uses one paragraph, you are not going to fix that by editing adjectives in the system text. Ask the retriever for the paragraph, or ask a cheaper pass to extract the paragraph before the expensive model sees it. That second pass has its own tokens. Add them up. A 200-token extract plus a short instruction often beats a 4,000-token paste even after you pay for the extract. Sometimes it does not, because the extract drops the caveat the answer needed. Check the caveat explicitly. "Include any exception in the policy, not only the general rule" is a sentence worth adding to the extract step.

Keep a diff. The shortened prompt should be reviewable as a deletion, with a line in the description: what you removed, the token delta, and the twenty-ticket result. "Cleanup" is not a description. "Remove the 2014 brand story and the duplicated policy, −640 tokens, 20 of 20 tickets still match" is a description. The next person who wants to add the brand story back has to argue with a result, not with a vibe. This is also how you stop the prompt growing a hundred tokens a month. Growth is the default. Shortening is a decision you write down.

If the short prompt and the long prompt tie on your twenty tickets, ship the short one. Ties go to the version you can read in one screen. A prompt you can read is a prompt you will notice is wrong. A prompt of 3,000 tokens of mixed examples is a place for a wrong rule to hide. The failure note lists the hides: a constraint in the middle of a story, a sample that looks like the assignment, two formats at once. Shortening is how those become visible. Do it with a counter and a test set, not with a hope that fewer words are always smarter.