# Designing apps with decisions

Read this before you build something on Bridgit that has to decide: whether to interrupt the owner, whether a sound
matters, whether the dog is on the couch. Bridgit gives you two decision models. You send a state (text, JSON, or
media) and typed questions; you get back typed answers, each with a probability or a confidence. No prose to parse,
so "act only when at least 80% sure" is one rule.

- **Jev** (TypeSafe, `jev-1.14.0`): text and JSON, and up to 2 small images. Best at reasoning about text.
- **Clef-omni** (Cloudflare): photos and screenshots, sound clips and short videos, plus text. The only one that hears
  and watches.

Use a chat model instead for anything open-ended: summaries, replies, explanations.

## Pick the route

| You want to | Call | Model | Media | What runs |
|---|---|---|---|---|
| Decide once about text, and have the phone act | `POST /v1/judgment` on the phone | Jev | none | The phone runs the matching notice, speak or log actions as you |
| Be called back when the phone's readings or camera match | `POST /v1/watches` on the phone | Clef (default with a photo), Jev (default without), or `none` (local only, free) | sensors and a camera photo | The phone checks every 30 s or more, skips the paid call while the picture is unchanged, and calls your `callback_url` |
| Decide about text or a photo from anywhere | `POST {base_url}/v1/jev` with your team key | Jev | up to 2 images | Matching actions come back as a list for you to run |
| Decide about photos, screenshots, sound or video | `POST {base_url}/v1/clef` with your team key | Clef-omni | 4 images, 4 audio clips, 2 videos | Matching actions come back as a list for you to run |

Get `base_url` and your team key (`bgk_...`) from `GET /v1/feedback/team-key` on the phone. Self-registered keys
can't use Jev or Clef. Both routes take the same `actions` rules as `/v1/judgment` (bridgit-actions/1).

## Which model for which input

| Input | Use | Why |
|---|---|---|
| Text, JSON, sensor readings | Jev | Clearly more confident on text. On the battery example below, Jev was 93% sure; Clef-omni 66%. |
| A screenshot or photo, one decision now | Clef-omni, or Jev with an image | Both labelled 5 of 5 phone screenshots right. Clef-omni takes bigger images; Jev's image input is still marked experimental by TypeSafe. |
| A sound (alarm, doorbell, glass, a voice) | Clef-omni | Jev can't hear. Clef-omni told a spoken delivery message from an alarm beep (88% and 85% confident). |
| A video or screen recording | Clef-omni | Jev can't watch. Clef-omni followed a 9 s recording across three screens. |
| Hard reasoning (several facts, arithmetic, rules) | Jev | Clef (27B) scores far lower on reasoning benchmarks (GPQA Diamond 48 vs 78); Clef-omni is smaller. |

When a decision needs both (a photo and a rule about the owner's calendar), ask Clef-omni what's in the photo,
then put its answer into the state you send Jev.

## Write questions that work

This example goes to `POST {base_url}/v1/clef` (the `images` field is Clef's; Jev uses `assets`).

```json
{
  "state": {"screen": "see image", "time": "14:05", "focus_mode": "work"},
  "images": ["data:image/jpeg;base64,..."],
  "questions": {
    "activity": {"type": "choice", "instructions": "What is the owner doing on the phone?",
      "criteria": {"call": "on a voice or video call", "signing": "signing or paying for something",
                   "game": "playing a game", "reading": "reading or browsing", "other": "none of these"}},
    "busy": {"type": "noul", "instructions": "Would a notification right now interrupt something important?"}
  },
  "actions": [
    {"id": "hold", "when": {"answer": "busy", "min": 0.7}, "do": {"type": "log", "message": "Held the card: owner is busy."}}
  ]
}
```

- **Typed, not open.** `noul` gives the probability of yes (0 to 1). `choice` picks one option and gives a
  probability for each plus a `confidence`. Clef also has `score` (a rating on a scale of 2 to 10 levels), but
  action rules don't match scores yet.
- **Describe every option** in `criteria`, and always include a way out (`other`, `nothing`, `none of these`).
  Without one, the model has to pick something.
- **One thing per question.** Ask "is there a person?" and "is the person carrying a box?" separately, not together.
- **Point at the data.** In Jev, refer to state fields in backticks (`` `screen` ``) and to images by their asset
  reference. In Clef, images, audio and video come before the state in the order you send them; say "the first image",
  "the clip".
- **Put context in the state**, not in the question: the owner's preferences, the time, what "urgent" means for them.
- **Up to 50 questions** per Jev call and 64 per Clef call. Ask several questions about one state in one call: they share
  the state's cost (you pay for input only, so each extra question adds just its own text).

## Pick thresholds

- **Confidence is not probability.** For a `choice`, `confidence` is lower than the winning probability: a 67%
  likely answer is only about 34% confident, and on the battery example Clef-omni said yes at 91% probability
  but 66% confidence. Use `min_confidence` 0.8 for choices you act on, or test `min` on a `noul` probability.
- **Start at 0.8** for anything the owner sees or hears; use 0.9 for interruptions at night. Use `log` actions to
  record the near-misses (0.5 to 0.8) and look at them before lowering a threshold.
- **Media answers are less certain.** Sound and video answers came back at 81% to 88% confidence in tests: they
  pass 0.8, not 0.9.
- **Test with real samples.** Before shipping, run 10 to 20 real captures (with known answers) through your
  questions and check where the right and wrong answers fall.
- **Even 95% is wrong one time in twenty.** Show, suggest, or log; leave anything costly or irreversible to the owner.
  If the model fails or times out, nothing runs: design for "no answer".

## Prepare media

| Capture | Phone call | Then |
|---|---|---|
| Screen | `POST /v1/screen/capture?max_edge_px=1024&quality=70` | Clef: send as `data:image/jpeg;base64,...` (Jev: also fine, 1,568 px at most) |
| Photo | `POST /v1/camera/snapshot {"max_width_px": 640}` | 640 px keeps Jev images under its 300 KB limit and cuts Clef tokens |
| Video | `POST /v1/camera/video {"duration_seconds": 10, "resolution": "720p"}` | Clef only. The phone records 1 to 30 s (40 MB over HTTP, 6 MB over MCP); Clef takes up to 16 MB per video and reads 2 frames a second. Silent unless you add `"audio": true` (needs the microphone grant) |
| A loud sound | the `sound.heard` event, then `GET /v1/sounds/clip/{id}` | Clef only. Up to 10 s, sent as `data:audio/...` |
| A wake word | `GET /v1/listen/clip/{id}` | Clef only. Classify the request straight from audio |

- **Clef:** base64 data URLs or `{content_type, base64}`, never links. Up to 4 images (PNG, JPEG or WebP, 4 MiB each, 8 MiB
  together), 4 audio clips (8 MiB each), 2 videos (16 MiB each); audio and video together 16 MiB; the whole body 24 MiB. Too big gives 413 `payload_too_large` before anything is billed.
- **Jev images:** `assets` map, PNG or JPEG, 280 to 1,568 px per side, 300 KB each, 2 per call. Rotate photos upright
  first: Jev ignores EXIF rotation.
- **Smaller is cheaper.** A 640 px photo is about 1,550 Jev tokens; a screenshot about 600 to 760 Clef tokens; a
  second of video about 250 to 300.

## Cost

You pay for input only; answers are free. Jev is $0.042 and Clef-omni $0.15 per million input tokens.

| One call sends | Tokens | Per call | Per day, every 10 s | Per month, every 10 s | Per month, every minute | Per month, every 5 min |
|---|---|---|---|---|---|---|
| Jev, a text question | ~200 | $0.000008 | $0.07 | $2.18 | $0.36 | $0.07 |
| Jev, a 640 px photo | ~1,550 | $0.000065 | $0.56 | $16.87 | $2.81 | $0.56 |
| Clef-omni, a screenshot or photo | ~700 | $0.000105 | $0.91 | $27.22 | $4.54 | $0.91 |
| Clef-omni, a 6 s sound clip | ~250 | $0.000037 | $0.32 | $9.72 | $1.62 | $0.32 |
| Clef-omni, a 10 s video | ~3,000 | $0.00045 | $3.89 | $116.64 | $19.44 | $3.89 |

Each reply has `usage.input_tokens` and `cost_usd`, so measure your own calls rather than trusting this table.
Only check as often as the decision needs: trigger on an event (a loud sound, a screen change, motion) instead of
polling, and check video only after a cheaper photo check says something is happening.

## Limits

- **Per phone or team key:** 400 Jev and Clef calls an hour together (60 on a phone the team
  couldn't verify, which includes test builds), and $1.50 a month (calendar month, UTC). Over either: 429
  `rate_limited` or `spend_cap`. One call every 10 s (360 an hour) fits (for Clef, only until the free allowance below runs out); video every 10 s does not.
- **All of Bridgit:** $5 a day and $50 a month across every caller. Plan for other agents sharing it.
  At $1.50 a month, paying for a photo every 10 s would last about 2.5 days on Jev: skip unchanged pictures
  (watches do it with `local`) and check on events instead of a timer.
- **Watches:** every 30 s at most, and 240 paid checks an hour across the whole phone (50 on a phone the team couldn't
  verify). Checks answered on the phone (unchanged picture, `model: none`) are free and don't count.
- **Clef's free allowance:** about 970 screenshots or 220 ten-second clips a day for all of Bridgit. Past that,
  calls fail with 429 `clef_quota` until the next day unless the team has Cloudflare's paid plan on.
- **Time:** Jev gives up after 10 s, Clef after 30 s (both come back as an error, and no action runs). Typical
  answers take 0.2 to 0.7 s; a 20 s 720p video took about 4 s.

## App patterns

- **Don't interrupt.** Before a card, capture the screen and ask Clef whether the owner is on a call, paying, or
  signing. Hold the card on a confident yes.
- **Sound alerts.** On `sound.heard`, send the clip to Clef: smoke alarm, glass breaking, a baby crying, the doorbell,
  or nothing. Speak or show a card only on a confident hit.
- **Camera watch.** `POST /v1/watches` with `"data": ["camera"]` and a yes/no ("is a package at the door?"): the phone
  takes a 640 px photo each reading and asks Clef only when the picture changed (`local` sets the free change check,
  `max_paid_per_hour` caps the bill, `usage` shows free vs paid). `"model": "none"` just calls you back when the
  picture changes, free. On a yes, record a 10 s clip and ask Clef to confirm before telling the owner.
- **Gallery sorting.** For each new photo in `GET /v1/gallery/recent`: receipt, document, screenshot, or personal photo? Offer to file it.
- **Sensor watch.** "Call me back when the owner starts cycling": `POST /v1/watches` with location and motion, Jev.
- **Triage text.** Bug reports, messages, notifications: crash, wrong result, or request? How likely does it affect
  others? Jev, through `/v1/judgment`, with a `speak` action for the urgent ones.

## Errors

| Error | Means | Do |
|---|---|---|
| 400 `invalid_parameter` | Bad state, questions, actions, or too many media items | Fix the request; nothing was billed |
| 413 `payload_too_large` | Body or a media item too big | Shrink, trim or send fewer |
| 422 `jev_rejected` / `clef_rejected` | The model refused the input (its words are included) | Usually the questions' shape, or a clip too long |
| 429 `rate_limited` / `spend_cap` / `clef_quota` | Over a limit above | Slow down or try tomorrow; don't retry in a loop |
| 502, 503, 504 | The model is down, busy or slow | Retry once after a few seconds; nothing ran |

Model details: [Jev](https://docs.typesafe.ai) and Cloudflare's
[Clef-omni](https://developers.cloudflare.com/workers-ai/models/clef-omni/). The full request and reply for
`/v1/judgment` and `/v1/watches` are in the [guide](https://bridgit.cc/learn/guide.md); an overview for people is at [https://bridgit.cc/decisions/](https://bridgit.cc/decisions/).