Extraction: text, numbers and integers

Some decisions need a value rather than a label: who issued an invoice, what it totals, how many which invoice number it carries. Text, Number and Integer questions ask the model to write the value, typed and checked against its bounds, together with confidence: its probability that the value is correct. That confidence is calibrated from your feedback, exactly like a label answer’s.

from curva import Curva, Choice, Integer, Number, Text

curva = Curva()
d = curva.decide(
    {"invoice": "ACME Inc. / Invoice 2291 / 3 x Widget @ 4.10 / Shipping 2.00 / Total due 14.30 EUR"},
    {
        "vendor": Text("Who issued the invoice?", max_length=100),
        "total": Number("Total amount due", min=0),
        "invoice_no": Integer("Invoice number", min=1),
        "po_number": Text("Purchase order number", max_length=40, nullable=True),
        "category": Choice("Expense category?", ["hardware", "software", "services"]),
    },
)
d["total"].value, d["total"].confidence    # 14.3, 0.94
d["po_number"].value                       # None: the invoice has no PO number

Label and extraction questions go into one model call. Add images=["invoice.png"] with a vision model to read the document itself (see Images).

The questions

Type	Settings	`value`
`text`	`max_length`: 1–2000 characters, default 200	a string
`number`	`min`, `max` (inclusive, optional)	a number
`integer`	`min`, `max` (whole numbers, optional)	an integer

Each also takes nullable, min_confidence, examples (the label is the value), when and rules (the answer is the value). Over HTTP:

{
  "state": {"receipt": "Corner Cafe, 2 coffees, total 7.40"},
  "questions": {
    "merchant": {"type": "text", "instructions": "Merchant name", "max_length": 80},
    "total": {"type": "number", "instructions": "Total paid", "min": 0, "min_confidence": 0.9},
    "tip": {"type": "number", "instructions": "Tip amount", "min": 0, "nullable": true}
  }
}

"answers": {
  "merchant": {"value": "Corner Cafe", "confidence": 0.97, "calibrated": false},
  "total": {"value": 7.4, "confidence": 0.95, "abstain": false, "calibrated": false},
  "tip": {"value": null, "confidence": 0.88, "calibrated": false}
}

Null means “not in the state”

With nullable: true, null is a real answer: the model says the state has no such value, with a confidence like any other answer. Without it, the model must give a value. Use nullable for fields that are often missing (a PO number, a discount, a due date on a receipt) so the model isn’t pushed into inventing one.

A reply that doesn’t fit the question (a string for a number, a total below min, text over max_length, null when not nullable, a confidence outside 0–1) is not an answer: it comes back as value: null with confidence: 0, so min_confidence marks it abstain: true.

Calibrated confidence

Send the true value as feedback. Curva compares it with the value it answered (text ignoring case and extra whitespace, numbers within a relative 1e-6) and records right or wrong. From 30 labels for the question, confidence is recalibrated (Platt scaling) against how often the values were actually right, whenever that makes it more accurate:

curva.feedback(d.id, "vendor", "ACME Inc.")   # the true value
curva.feedback(d.id, "po_number", None)       # it really had none

Then min_confidence becomes a dependable cut-off: automate the values above it and send the rest to a person.

Modes and plans

Verbal only. A value can’t be read from one token’s logprobs, so mode: auto switches to verbal for requests with an extraction question, and mode: logprobs gets 422.
Debias. Extraction questions are not reordered. If the two calls return different values, the answer is the original-order value at the lower of the two confidences.
Council. The majority value wins (a tie goes to the more confident side). agreement is the share of members that gave it, and confidence is their mean confidence × agreement.
Cascade. A value below escalate_below confidence goes to the next model.
Explain skips extraction questions. Drift reports confidence only (mix is null).

Documentation for everything Tarkova makes.