Translate text from a screenshot

When to use it

Use region capture when there is no selection to read — or when the selection is unreliable. The common cases:

• Screenshots and images pasted into Slack, Jira, Linear, or a document.
• Screen shares and video calls, where the text exists only as pixels.
• Scanned PDFs and photographed documents.
• Native error dialogs and installer windows that cannot be selected.
• Design files in Figma or Sketch, and text inside a canvas or game UI.
• Apps with custom-drawn text that expose nothing to Accessibility.

Capture a region

1. Press ⌥S. The screen dims and the pointer becomes a crosshair.
2. Drag a rectangle around the text. Include whole lines; partial lines are harder to reconstruct.
3. The panel opens with the recognized text already in the source field.
4. Pick an action — Translate, Polish, Reply, or one of your own — and run it.

The first time you use it, macOS asks for Screen Recording permission. If you deny it, region capture will not work until you enable it in Privacy & Security.

What happens on-device

Recognition is handled by Apple Vision, the same framework behind Live Text. It runs locally: the image is not uploaded to Typeme Cloud or to your model provider. Only the resulting text is sent when an action runs.

The capture is also written to your daily log as a context record, the same way a selection is. That is what lets Typeme remember a ticket you looked at yesterday. The record contains the recognized text, the app, and the window title — not the image, unless screenshot recording is turned on.

Fix the text before running

OCR is good, not perfect. The panel's source field is editable, so correct the obvious errors before running an action:

• Join lines that OCR split mid-sentence.
• Fix 0/O, 1/l, and broken punctuation.
• Remove UI labels that came along with the paragraph (“Copy”, “Reply”, timestamps).
• Add a short line of context if the fragment is ambiguous — for example, “from the billing settings page”.

Then press Return to run the action on the corrected text.

Accuracy tips

• Crop tightly. Extra background, borders, and icons reduce accuracy.
• Prefer dark text on a light background. Inverted or low-contrast text is harder; if the app has a light theme, switch for the capture.
• Capture at the largest size available. Zoom the document or image before capturing rather than cropping a tiny region.
• Include the whole sentence. OCR is per-line; a sentence split across two lines still needs both lines in the rectangle.
• Check names and numbers. These are the most consequential OCR errors and the easiest to miss.

Screenshots and privacy

Screenshot recording is off by default. With it off, ⌥S performs OCR and keeps no image — not even the region you selected.

If you turn recording on under Settings › Advanced:

• Only apps on your allowlist are captured. The list starts empty, which means nothing is captured.
• Images are stored in shots/YYYY-MM-DD/ inside your context folder, named with the time and app.
• They are pruned on the same schedule as the daily log, which you control in Settings › Advanced.

OCR itself never leaves the Mac. What leaves is the recognized text, and only when you run an action — see How Typeme remembers for the exact payload.

Questions