Every kind of answer at once

`annotate`: fill out a form for every record.

form.json, a file the examples read

{
  "version": 1,
  "questions": {
    "steps": {
      "decide": "Does the report give steps to reproduce?"
    },
    "area": {
      "choose": "Which part of the app is this?",
      "options": [
        "export",
        "login",
        "billing"
      ]
    },
    "impact": {
      "score": "How much does this block the user?",
      "levels": [
        "None.",
        "Slows them.",
        "Blocks work."
      ]
    }
  }
}

One JSON document goes in. The same document comes back with three answers added: steps, area, and impact. The nested report rides through unchanged.

cat <<'EOF' |
{
  "id": "B-7",
  "report": {
    "page": "/login",
    "body": "Steps: click Log in. Nobody gets in."
  }
}
EOF
thinkthen annotate form.json \
  --field /report/body |
jq .

Output

{
  "id": "B-7",
  "report": {
    "page": "/login",
    "body": "Steps: click Log in. Nobody gets in."
  },
  "steps": true,
  "area": "login",
  "impact": 1.98
}

exit 0

You give it a saved set of questions and your JSON. You get back the same JSON with one field added per question. Nested fields ride through unchanged.

Read the answer

Each question carries its own threshold. One question set can mix cuts and bands. A question the backend could not answer is marked failed and counted. It never turns into null.

Exit code	What it means
0	every question was answered
6	the run finished with failed questions
2	usage or input error
4	the backend failed or refused, as it does for evidence over the size limit
5	the question set could not be read
70	a defect in the tool

Send the not-sure answers to a person.

Call it from your language

The Bash example opens this page.

import assert from "node:assert/strict";
import * as tt from "thinkthen";

const reports = [
  "Steps: click Export. It is very slow.",
  "Steps: click Log in. Nobody gets in.",
  "The Pay button on billing is too blue.",
];
const triage = (await tt.annotate(
  "form.json",
  reports,
)).value;
assert.deepEqual(triage, [
  { steps: true, area: "export", impact: 1.04 },
  { steps: true, area: "login", impact: 1.98 },
  { steps: false, area: "billing", impact: 0.09 },
]);

require "thinkthen"

form = ThinkThen.set(
  steps: {
    decide: "Does the report give steps to reproduce?"
  },
  area: {
    choose: "Which part of the app is this?",
    options: ["export", "login", "billing"]
  },
  impact: {
    score: "How much does this block the user?",
    levels: ["None.", "Slows them.", "Blocks work."]
  }
)
reports = [
  "Steps: click Export. It is very slow.",
  "Steps: click Log in. Nobody gets in.",
  "The Pay button on billing is too blue."
]
triage = ThinkThen.annotate(form, reports).value
raise unless triage == [
  { steps: true, area: "export", impact: 1.04 },
  { steps: true, area: "login", impact: 1.98 },
  { steps: false, area: "billing", impact: 0.09 }
]

library(thinkthen)

reports <- data.frame(body = c(
  "Steps: click Export. It is very slow.",
  "Steps: click Log in. Nobody gets in.",
  "The Pay button on billing is too blue."
))
triage <- tt_annotate(
  "form.json", reports, on = "body"
)$value
stopifnot(identical(triage$steps, c(TRUE, TRUE, FALSE)))
stopifnot(identical(
  triage$area,
  c("export", "login", "billing")
))
stopifnot(identical(triage$impact, c(1.04, 1.98, 0.09)))

use thinkthen::{Annotated, Engine, QuestionSet};

let tt = Engine::from_env()?;
let set = QuestionSet::load("form.json")?;
let reports = [
    "Steps: click Export. It is very slow.",
    "Steps: click Log in. Nobody gets in.",
    "The Pay button on billing is too blue.",
];
let areas = ["export", "login", "billing"];
let triage = tt.annotate(&set, reports);
for (form, want) in triage.zip(areas) {
    let area = form?.values()[1].value().clone();
    assert_eq!(area, Annotated::Choice(Some(want.into())));
}

Put this code inside fn main() -> Result<(), Box<dyn std::error::Error>> and end it with Ok(()). main returns a Result, so ? compiles.

#include <assert.h>
#include <json-c/json.h>
#include <thinkthen.h>

thinkthen_engine *tt = thinkthen_engine_new();
assert(tt);

const char *annotate =
    "{\"annotate\": {\"version\": 1, \"questions\": {"
    "\"steps\": {\"decide\": "
    "\"Does the report give steps to reproduce?\"}, "
    "\"area\": {\"choose\": "
    "\"Which part of the app is this?\", "
    "\"options\": [\"export\", \"login\", \"billing\"]}, "
    "\"impact\": {\"score\": "
    "\"How much does this block the user?\", "
    "\"levels\": [\"None.\", \"Slows them.\", "
    "\"Blocks work.\"]}}}, "
    "\"records\": ["
    "\"CSV export fails every time. Steps: open a report,"
    "\\nclick Export, pick CSV. My month-end numbers "
    "are stuck."
    "\\n\", "
    "\"Steps: open the login page, enter a password, "
    "press Enter. The page spins and nobody can "
    "sign in.\", "
    "\"The Pay button on the billing page is a slightly "
    "different blue. No steps, I just noticed it.\"]}";
const char *expected =
    "[{\"steps\":true,\"area\":\"export\","
    "\"impact\":1.99},"
    "{\"steps\":true,\"area\":\"login\","
    "\"impact\":2.0},"
    "{\"steps\":false,\"area\":\"billing\","
    "\"impact\":0.01}]";
char *triage_call = thinkthen_call(tt, annotate);
assert(triage_call);
enum json_tokener_error parse_error;
struct json_object *result =
    json_tokener_parse_verbose(triage_call, &parse_error);
assert(parse_error == json_tokener_success);
assert(result && json_object_get_type(result)
    == json_type_object);
struct json_object *value;
json_bool has_value = json_object_object_get_ex(
    result, "value", &value);
assert(has_value);
struct json_object *facts;
json_bool has_facts = json_object_object_get_ex(
    result, "facts", &facts);
assert(has_facts);
assert(facts && json_object_get_type(facts)
    == json_type_object);
struct json_object *wanted =
    json_tokener_parse_verbose(expected, &parse_error);
assert(parse_error == json_tokener_success);
assert(json_object_equal(value, wanted));
if (wanted) json_object_put(wanted);
json_object_put(result);
thinkthen_free_string(triage_call);
thinkthen_engine_free(tt);

Keep the #include lines on top. Put the rest inside int main(void) and end it with return 0;.

LOAD './thinkthen.duckdb_extension';

SELECT thinkthen_annotate('@form.json', body) AS triage
FROM (VALUES
    ('Steps: click Export. It is very slow.'),
    ('Steps: click Log in. Nobody gets in.'),
    ('The Pay button on billing is too blue.')
) t(body);

What DuckDB printed

triage
{"steps":true,"area":"export","impact":1.04}
{"steps":true,"area":"login","impact":1.98}
{"steps":false,"area":"billing","impact":0.09}

.load ./thinkthen

WITH t(body) AS (VALUES
    ('Steps: click Export. It is very slow.'),
    ('Steps: click Log in. Nobody gets in.'),
    ('The Pay button on billing is too blue.'))
SELECT thinkthen_annotate('@form.json', body) AS triage
FROM t;

What SQLite printed

{"steps":true,"area":"export","impact":1.04}
{"steps":true,"area":"login","impact":1.98}
{"steps":false,"area":"billing","impact":0.09}

SELECT thinkthen_annotate('@form.json', body) AS triage
FROM (VALUES
    ('Steps: click Export. It is very slow.'),
    ('Steps: click Log in. Nobody gets in.'),
    ('The Pay button on billing is too blue.')
) AS t(body);

What PostgreSQL printed

                       triage                        
-----------------------------------------------------
 {"area": "export", "steps": true, "impact": 1.04}
 {"area": "login", "steps": true, "impact": 1.98}
 {"area": "billing", "steps": false, "impact": 0.09}
(3 rows)

Arguments, options, and more examples · annotate's edge cases

Jobs that use `annotate`

Triage a support inboxfor support teams

Watch annotate answer questions about Beatles songs: annotate answers a question set.

On GitHub: github.com/botassembly/thinkthen

annotate: fill out a form for every record.

Read the answer

Call it from your language

Jobs that use annotate

`annotate`: fill out a form for every record.

Jobs that use `annotate`