# Evaluation

> Testing a prompt on a fixed set of inputs and scoring the outputs against acceptance criteria, so improvements can be measured instead of guessed.

Source: Promptea — https://www.promptea.me/en/glossary/evaluation · Language: English · Last updated: 2026-11-09

## What it means

- Definition: prompt evaluation means running the same prompt over a small, fixed set of test inputs — including awkward edge cases — and grading the outputs with a rubric.
- Why it matters: one good answer proves little. Without a fixed test set, every edit is a coin flip and a change that fixes one case can quietly break another.
- How to apply it: write 5–20 inputs that cover typical and difficult cases, define what a correct output looks like (format, facts, tone), score each criterion (for example 0–2) and rerun after every prompt change.
- Common mistake: tuning the prompt against the same three examples you keep staring at. Keep a few test inputs aside that you never tune on.

## Example

```text
Create an evaluation set for this task: [task description]
Return:
1) A rubric with 5 criteria, each scored 0–2, with what each score means.
2) 8 test inputs: 3 typical, 3 edge cases, 2 adversarial.
3) For each input, what a perfect output must contain.
```

[Open in Promptea](https://www.promptea.me/en/analyzer?prompt=Create+an+evaluation+set+for+this+task%3A+%5Btask+description%5D%0AReturn%3A%0A1%29+A+rubric+with+5+criteria%2C+each+scored+0%E2%80%932%2C+with+what+each+score+means.%0A2%29+8+test+inputs%3A+3+typical%2C+3+edge+cases%2C+2+adversarial.%0A3%29+For+each+input%2C+what+a+perfect+output+must+contain.&purpose=text&target=gpt)

## Frequently asked questions

### How many test cases do I need?

Five to twenty is enough to start. Quality matters more than count: include the cases that have failed before.

### Is a prompt score the same as an evaluation?

No. Promptea's 0–100 score rates how well specified the prompt itself is, using fixed rules. An evaluation judges the outputs a prompt produces on your own test inputs. They complement each other.

## Related

- [How to test one prompt across ChatGPT, Claude, Gemini, Grok and DeepSeek](https://www.promptea.me/en/guides/test-prompt-across-models)
- [How to score and improve your prompt quality before sending it](https://www.promptea.me/en/guides/prompt-quality-scoring)
- [How to debug a prompt that gives bad output](https://www.promptea.me/en/guides/prompt-debugging)
- [Hallucinations](https://www.promptea.me/en/glossary/hallucinations)
- [Constraints](https://www.promptea.me/en/glossary/constraints)