# How to write prompts for AI with image and file inputs (multimodal)

> Multimodal prompts combine text instructions with images, PDFs, or screenshots. Learn how to direct AI attention to what matters and extract structured output reliably.

Source: Promptea — https://www.promptea.me/en/guides/multimodal-prompts · Language: English · Last updated: 2026-11-14

## What multimodal AI models can and cannot do

- They can: describe image content, read text in images (OCR-like), extract data from tables and forms, analyze charts and diagrams, compare multiple images, and answer questions grounded in what they see.
- They cannot: reliably read very small text in low-resolution images, count objects precisely in dense scenes, or guarantee pixel-level accuracy on complex diagrams.
- Quality of output depends heavily on image resolution and clarity. Blurry or low-contrast images produce vague descriptions.
- For PDFs: some APIs accept PDFs directly; others require converting pages to images first. Check your model's documentation.

## How to direct AI attention in images

- Name what you want analyzed: 'Focus on the bar chart in the top-right corner' is clearer than 'analyze this image'.
- Describe what the image contains if the model might not recognize the context: 'This is a screenshot of a warehouse management dashboard showing inventory levels.'
- For documents and forms, specify which fields matter: 'Extract only the vendor name, invoice number, date, and total amount. Ignore line items.'
- Use spatial anchors: 'the table in the second column', 'the highlighted row', 'the legend at the bottom'.
- If the image has multiple elements, tell the model what to prioritize first and what to skip.

## Common multimodal prompt mistakes

- No image description: asking 'what does this show?' without context forces the model to guess the purpose. Tell it what kind of document or image you're sending.
- Expecting precision on messy inputs: a photo of a handwritten note taken at an angle in dim lighting will produce uncertain results — scan or crop before prompting.
- No output format specified: without one, the model might describe the image in prose when you needed JSON, or give you a table when you needed a summary.
- Assuming it reads all text: models process images as visual patterns. Very small text, rotated text, or text on complex backgrounds may be missed or misread.

## Templates

### Visual data extraction from an image or screenshot

```text
I'm sending you [a screenshot of a dashboard / a photo of a form / a chart image].

Context: [describe what the image contains and where it comes from — e.g. "This is a weekly sales report screenshot from our internal reporting tool."]

Extract the following fields:
- [Field 1]: [what it looks like or where it appears]
- [Field 2]: [what it looks like or where it appears]
- [Field 3]: [what it looks like or where it appears]

Return ONLY JSON:
{
  "field_1": "value or null",
  "field_2": "value or null",
  "field_3": "value or null",
  "extraction_notes": "any uncertainty or ambiguity noted"
}

Rules:
- Use null for any field you cannot read clearly.
- Do not guess or infer values not visible in the image.
- Add a note in extraction_notes if any field was difficult to read.
```

[Open in Promptea](https://www.promptea.me/en/analyzer?prompt=I%27m+sending+you+%5Ba+screenshot+of+a+dashboard+%2F+a+photo+of+a+form+%2F+a+chart+image%5D.%0A%0AContext%3A+%5Bdescribe+what+the+image+contains+and+where+it+comes+from+%E2%80%94+e.g.+%22This+is+a+weekly+sales+report+screenshot+from+our+internal+reporting+tool.%22%5D%0A%0AExtract+the+following+fields%3A%0A-+%5BField+1%5D%3A+%5Bwhat+it+looks+like+or+where+it+appears%5D%0A-+%5BField+2%5D%3A+%5Bwhat+it+looks+like+or+where+it+appears%5D%0A-+%5BField+3%5D%3A+%5Bwhat+it+looks+like+or+where+it+appears%5D%0A%0AReturn+ONLY+JSON%3A%0A%7B%0A++%22field_1%22%3A+%22value+or+null%22%2C%0A++%22field_2%22%3A+%22value+or+null%22%2C%0A++%22field_3%22%3A+%22value+or+null%22%2C%0A++%22extraction_notes%22%3A+%22any+uncertainty+or+ambiguity+noted%22%0A%7D%0A%0ARules%3A%0A-+Use+null+for+any+field+you+cannot+read+clearly.%0A-+Do+not+guess+or+infer+values+not+visible+in+the+image.%0A-+Add+a+note+in+extraction_notes+if+any+field+was+difficult+to+read.&purpose=data&target=gpt)

### Document Q&A with source citation

```text
I'm sending you [a PDF / a set of document images / a scanned contract].

Context: [briefly describe what the document is — e.g. "This is a vendor contract for software licensing, approximately 12 pages."]

Answer the following questions based ONLY on the content visible in the document:
1. [Your question 1]
2. [Your question 2]
3. [Your question 3]

For each answer:
- Quote the exact sentence or section that supports your answer.
- If the document does not contain the answer, say "Not found in the document" — do not infer.
- If the answer is ambiguous, explain why and quote the relevant passage.

Format:
**Q1:** [restate the question]
**Answer:** [your answer]
**Source:** "[exact quote from document]" (Page X / Section Y if identifiable)
```

[Open in Promptea](https://www.promptea.me/en/analyzer?prompt=I%27m+sending+you+%5Ba+PDF+%2F+a+set+of+document+images+%2F+a+scanned+contract%5D.%0A%0AContext%3A+%5Bbriefly+describe+what+the+document+is+%E2%80%94+e.g.+%22This+is+a+vendor+contract+for+software+licensing%2C+approximately+12+pages.%22%5D%0A%0AAnswer+the+following+questions+based+ONLY+on+the+content+visible+in+the+document%3A%0A1.+%5BYour+question+1%5D%0A2.+%5BYour+question+2%5D%0A3.+%5BYour+question+3%5D%0A%0AFor+each+answer%3A%0A-+Quote+the+exact+sentence+or+section+that+supports+your+answer.%0A-+If+the+document+does+not+contain+the+answer%2C+say+%22Not+found+in+the+document%22+%E2%80%94+do+not+infer.%0A-+If+the+answer+is+ambiguous%2C+explain+why+and+quote+the+relevant+passage.%0A%0AFormat%3A%0A**Q1%3A**+%5Brestate+the+question%5D%0A**Answer%3A**+%5Byour+answer%5D%0A**Source%3A**+%22%5Bexact+quote+from+document%5D%22+%28Page+X+%2F+Section+Y+if+identifiable%29&purpose=text&target=claude)

## Frequently asked questions

### Which AI models support image inputs?

Most current models do. In Promptea's model registry (checked against provider docs in September and October 2026), image input is listed for GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna; Claude Opus 5.5, Fable 5.1, Sonnet 5.5 and Haiku 5.5 (which also read PDFs); the Gemini 3.x Flash models (which also take audio, video and PDF); Grok 4.7 and Grok 4.3; DeepSeek Flash; and Kimi K3 (also video). DeepSeek V4 Pro and Perplexity's Agent API presets are text-only there. Support can differ between the chat app and the API and changes over time, so confirm in the provider's current documentation.

### How do I extract a table from a PDF using AI?

The most reliable approach: convert the PDF page containing the table to a high-resolution image (300 DPI or higher), then send it with a prompt that specifies the table location, the columns you need, and the output format (JSON or Markdown table). Ask the model to flag cells it cannot read clearly. For native-digital PDFs where text is selectable, copy-paste the table text directly into your prompt instead — text extraction is more reliable than visual OCR.
