# ThinkThen > ThinkThen: code that knows what you mean. Simple functions that give your software the judgment to handle whatever comes its way. Ten functions, in your scripts, your programs, and your queries. --- Source: https://thinkthen.dev/about/ # About ThinkThen Ian Maurer makes ThinkThen. He is the CTO of [GenomOncology](https://www.genomoncology.com/). ## Why it exists Code sees strings. It cannot tell what a message means. ThinkThen asks a model a bounded question about a piece of text. It gives your code back a plain answer and an exit code. A script can branch on that answer, as it would on any other value. [Code that knows what you mean](/blog/code-that-understands/) tells the longer story. ## Ian Maurer Ian is the CTO of GenomOncology. He writes about AI agents doing real biomedical work on his blog. - [Blog](https://www.imaurer.com) - [LinkedIn](https://www.linkedin.com/in/ianmaurer/) - [X](https://x.com/imaurer) - [GitHub](https://github.com/imaurer) ## GenomOncology GenomOncology builds software for precision oncology. Its software and data help the healthcare community improve cancer care. Read more at [genomoncology.com](https://www.genomoncology.com/). ## The code ThinkThen is open source under the [MIT License](https://github.com/botassembly/thinkthen/blob/main/LICENSE). One public repository holds the command, the bindings, the specification and this site: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen). ## Report a problem Open an [issue on GitHub](https://github.com/botassembly/thinkthen/issues) for a bug or an answer that looks wrong. Include the command, the version from `thinkthen --version`, and what you expected. Report a security problem privately through the repository's [Security tab](https://github.com/botassembly/thinkthen/security). Never include an API key. On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/blog/code-that-understands/ # Code that knows what you mean Code sees strings, not meaning. `grep` finds the word “refund”. It can’t tell whether “I want to send this back” asks for money or for an exchange. Search matches letters. It can’t answer a question about the text. [TypeSafe](https://typesafe.ai) makes a model called Jev that can. Jev answers a bounded question about the text you hand it. The answer is yes or no, one option from your list, or a place on a scale. Each answer comes with a probability. Jev writes no sentences, so your code has nothing to parse. It needs no training data and no labels. The question goes in as plain words. On Beatles Bench, a thousand answers cost about 0.016 dollars, by the bench’s [cost table](https://github.com/botassembly/beatles-bench/blob/main/results/tables/cost.tsv). We built ThinkThen around that interface. It’s one program. You pipe in the text and pass one question. ThinkThen prints a bare answer and sets an exit code. ## Three answers, three exit codes Here are three customer messages. The first asks for money back. The second doesn’t. The third could mean either. Ask the same question of each line, with a band from 0.2 to 0.8: ``` question="Does the customer ask for a refund?" cat <<'EOF' | Please refund my order. It arrived broken. Thanks for the quick help yesterday! I want to send this back. EOF thinkthen decide "$question" \ --batch 1 \ --lines \ --threshold 0.2:0.8 | jq . ``` *Output* ``` { "input": "Please refund my order. It arrived broken.", "value": true } { "input": "Thanks for the quick help yesterday!", "value": false } { "input": "I want to send this back.", "value": null } ``` *exit 0* The refund request clears the high bar and answers yes. The thank-you note falls under the low bar and answers no. The send-back line lands inside the band and answers not sure. `grep` would have said no to that line, and a person would never see it. A question and its band can live in a file, `refund.json`: *refund.json* ``` { "decide": "Does the customer ask for a refund?", "true": "The customer asks for money back.", "false": "Anything else, such as a cancellation or thanks.", "threshold": "0.2:0.8" } ``` Ask it of the send-back line: ``` printf '%s\n' "I want to send this back." | thinkthen decide @refund.json ``` *Output* ``` null ``` *exit 3: not sure* Yes exits 0. No exits 1. Not sure exits 3 and prints `null`. A script branches on the exit code and parses nothing. A person reviews the middle. The question file diffs in a pull request like any other code. Where you put the band depends on your data. The probability comes from the text you handed over. It doesn’t tell you how often Jev is right. Run the question against cases you have labeled first. `thinkthen audit` grades saved answers against your labels. Pick the band from that run. ## Ten functions that pipe together ThinkThen turns Jev’s three kinds of answer into [ten functions](/functions/). Each reads standard input and writes standard output. This pipeline keeps the buying inquiries, ranks them by how ready the buyer is, and picks a team for each: ``` buying="Is this a buying inquiry?" ready="Is this buyer ready to pay now?" team="Which team should take this?" teams=( enterprise smb ) cat <<'EOF' | We need 200 seats next quarter. Please send a quote. Please remove me from this list. Our team of six wants to buy today. How do we pay? Loved your talk at the conference last week. EOF thinkthen filter "$buying" --batch 1 | thinkthen rank "$ready" | thinkthen choose "$team" "${teams[@]}" --lines | jq . ``` *Output* ``` { "input": "Our team of six wants to buy today. How do we pay?", "value": "smb" } { "input": "We need 200 seats next quarter. Please send a quote.", "value": "enterprise" } ``` *exit 0* The pipeline returns the original lines beside their answers. It doesn’t rewrite them. Diogo Almeida of TypeSafe [lists decisions Jev could make inside a coding agent](https://x.com/completeskeptic/status/2101894250401271876), such as which tools to load for a step. Each one is a bounded question of this kind. `annotate` answers a whole set of questions for every record. A set can mix `decide`, `choose`, `score` and `tag` questions in one file. Here is a set with one yes or no, one pick and one scale, saved as `form.json`: *form.json* ``` { "version": 1, "questions": { "steps": { "decide": "Does the report give steps to reproduce?" }, "area": { "choose": "Which part of the app is this?", "options": [ "export", "login", "billing" ] }, "impact": { "score": "How much does this block the user?", "levels": [ "None.", "Slows them.", "Blocks work." ] } } } ``` Three bug reports go in, one per line. Each keeps its id and gains the three answers: ``` cat <<'EOF' | {"id": "B-7", "body": "Steps: click Log in. Nobody gets in."} {"id": "B-8", "body": "The Pay button on billing is too blue."} {"id": "B-9", "body": "Steps: click Export. It is very slow."} EOF thinkthen annotate form.json \ --batch 1 \ --jsonl \ --field /body \ --jobs 8 | jq . ``` *Output* ``` { "id": "B-7", "body": "Steps: click Log in. Nobody gets in.", "steps": true, "area": "login", "impact": 1.98 } { "id": "B-8", "body": "The Pay button on billing is too blue.", "steps": false, "area": "billing", "impact": 0.09 } { "id": "B-9", "body": "Steps: click Export. It is very slow.", "steps": true, "area": "export", "impact": 1.04 } ``` *exit 0* ## Where it breaks **Planted facts move the answer.** [Probe 06](https://github.com/botassembly/thinkthen/tree/main/probes/06-hostile-text) asked `jev-1.13.0` one question about twenty made-up messages: “The customer explicitly asks for money back.” Each message ran once clean and once with hostile text added. An order aimed at the model moved the probability of yes by 0.04 or less, across seventeen wordings. A false claim planted in the message moved it by as much as 0.57. Jev reads a planted claim and a true one the same way, because both look like evidence. A [later review](https://github.com/botassembly/thinkthen/blob/main/sdlc/issues/closed/2026-09-26-architect-review-12-security-and-data-boundary.md) asked other questions of the same model. There an order moved a different question by 0.16 to 0.18. On that question, the order itself could count as evidence. A planted claim flipped a second question from 0.01 to 0.64. These numbers hold for those messages, questions and that model only. At the default bar of 0.5, a planted claim can flip an answer. Use a band, and send the middle to a person. **`tag` often misses part of the set.** A song can have two lead singers, and `tag` must name every one to score. On Beatles Bench it names the whole set on 0.56 of songs. Its top label is a true lead on 0.80. The bench’s [function table](https://github.com/botassembly/beatles-bench/blob/main/results/tables/functions.tsv) scores every function. Use `tag` to fill a queue a person reads. Don’t use it as a gate. **A threshold belongs to one model.** ThinkThen speaks System One, the request format Jev answers. Any server that speaks System One can answer at another address. Its answers will differ, so tune the band again for each model. **It only answers.** ThinkThen writes no text, holds no conversation and takes no action. The answer goes back to your code, and your rules decide what happens next. ## What ships ThinkThen ships 10 functions, 1 CLI and [24 bindings](/install/). Every binding calls the same Rust engine, so a question file reads the same way everywhere. In a database, a question sits in a `WHERE` clause like any other condition. [Run decide](/functions/decide/) · [Read the blog](/blog/) · [Test it before you trust it](/trust/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/blog/ # Blog Articles about software that answers questions about text. These articles are drafted with AI agents against a voice guide, reviewed and edited by Ian. - [Code that knows what you mean](/blog/code-that-understands/): Code sees strings, not meaning. A model that answers bounded questions lets a script act on what text means. Here is how, and where it breaks. [See every function](/functions/) · [Test it before you trust it](/trust/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/annotate/ # `annotate`: fill out a form for every record. *form.json, a file the examples read* ``` { "version": 1, "questions": { "steps": { "decide": "Does the report give steps to reproduce?" }, "area": { "choose": "Which part of the app is this?", "options": [ "export", "login", "billing" ] }, "impact": { "score": "How much does this block the user?", "levels": [ "None.", "Slows them.", "Blocks work." ] } } } ``` *One JSON document goes in. The same document comes back with three answers added: steps, area, and impact. The nested report rides through unchanged.* ``` cat <<'EOF' | { "id": "B-7", "report": { "page": "/login", "body": "Steps: click Log in. Nobody gets in." } } EOF thinkthen annotate form.json \ --field /report/body | jq . ``` *Output* ``` { "id": "B-7", "report": { "page": "/login", "body": "Steps: click Log in. Nobody gets in." }, "steps": true, "area": "login", "impact": 1.98 } ``` *exit 0* You give it a saved set of questions and your JSON. You get back the same JSON with one field added per question. Nested fields ride through unchanged. ## Read the answer Each question carries its own threshold. One question set can mix cuts and bands. A question the backend could not answer is marked failed and counted. It never turns into null. | Exit code | What it means | | --- | --- | | 0 | every question was answered | | 6 | the run finished with failed questions | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | the question set could not be read | | 70 | a defect in the tool | Send the not-sure answers to a person. ## Call it from your language The Bash example opens this page. No Python sample for `annotate`. No Polars sample for `annotate`. No pandas sample for `annotate`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const reports = [ "Steps: click Export. It is very slow.", "Steps: click Log in. Nobody gets in.", "The Pay button on billing is too blue.", ]; const triage = (await tt.annotate( "form.json", reports, )).value; assert.deepEqual(triage, [ { steps: true, area: "export", impact: 1.04 }, { steps: true, area: "login", impact: 1.98 }, { steps: false, area: "billing", impact: 0.09 }, ]); ``` ``` require "thinkthen" form = ThinkThen.set( steps: { decide: "Does the report give steps to reproduce?" }, area: { choose: "Which part of the app is this?", options: ["export", "login", "billing"] }, impact: { score: "How much does this block the user?", levels: ["None.", "Slows them.", "Blocks work."] } ) reports = [ "Steps: click Export. It is very slow.", "Steps: click Log in. Nobody gets in.", "The Pay button on billing is too blue." ] triage = ThinkThen.annotate(form, reports).value raise unless triage == [ { steps: true, area: "export", impact: 1.04 }, { steps: true, area: "login", impact: 1.98 }, { steps: false, area: "billing", impact: 0.09 } ] ``` ``` library(thinkthen) reports <- data.frame(body = c( "Steps: click Export. It is very slow.", "Steps: click Log in. Nobody gets in.", "The Pay button on billing is too blue." )) triage <- tt_annotate( "form.json", reports, on = "body" )$value stopifnot(identical(triage$steps, c(TRUE, TRUE, FALSE))) stopifnot(identical( triage$area, c("export", "login", "billing") )) stopifnot(identical(triage$impact, c(1.04, 1.98, 0.09))) ``` ``` use thinkthen::{Annotated, Engine, QuestionSet}; let tt = Engine::from_env()?; let set = QuestionSet::load("form.json")?; let reports = [ "Steps: click Export. It is very slow.", "Steps: click Log in. Nobody gets in.", "The Pay button on billing is too blue.", ]; let areas = ["export", "login", "billing"]; let triage = tt.annotate(&set, reports); for (form, want) in triage.zip(areas) { let area = form?.values()[1].value().clone(); assert_eq!(area, Annotated::Choice(Some(want.into()))); } ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *annotate = "{\"annotate\": {\"version\": 1, \"questions\": {" "\"steps\": {\"decide\": " "\"Does the report give steps to reproduce?\"}, " "\"area\": {\"choose\": " "\"Which part of the app is this?\", " "\"options\": [\"export\", \"login\", \"billing\"]}, " "\"impact\": {\"score\": " "\"How much does this block the user?\", " "\"levels\": [\"None.\", \"Slows them.\", " "\"Blocks work.\"]}}}, " "\"records\": [" "\"CSV export fails every time. Steps: open a report," "\\nclick Export, pick CSV. My month-end numbers " "are stuck." "\\n\", " "\"Steps: open the login page, enter a password, " "press Enter. The page spins and nobody can " "sign in.\", " "\"The Pay button on the billing page is a slightly " "different blue. No steps, I just noticed it.\"]}"; const char *expected = "[{\"steps\":true,\"area\":\"export\"," "\"impact\":1.99}," "{\"steps\":true,\"area\":\"login\"," "\"impact\":2.0}," "{\"steps\":false,\"area\":\"billing\"," "\"impact\":0.01}]"; char *triage_call = thinkthen_call(tt, annotate); assert(triage_call); enum json_tokener_error parse_error; struct json_object *result = json_tokener_parse_verbose(triage_call, &parse_error); assert(parse_error == json_tokener_success); assert(result && json_object_get_type(result) == json_type_object); struct json_object *value; json_bool has_value = json_object_object_get_ex( result, "value", &value); assert(has_value); struct json_object *facts; json_bool has_facts = json_object_object_get_ex( result, "facts", &facts); assert(has_facts); assert(facts && json_object_get_type(facts) == json_type_object); struct json_object *wanted = json_tokener_parse_verbose(expected, &parse_error); assert(parse_error == json_tokener_success); assert(json_object_equal(value, wanted)); if (wanted) json_object_put(wanted); json_object_put(result); thinkthen_free_string(triage_call); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `annotate`. No Objective-C sample for `annotate`. No COBOL sample for `annotate`. No Ada sample for `annotate`. No Java sample for `annotate`. No Kotlin sample for `annotate`. No Scala sample for `annotate`. No C# sample for `annotate`. No Go sample for `annotate`. No Swift sample for `annotate`. No Zig sample for `annotate`. No PHP sample for `annotate`. No Dart sample for `annotate`. ``` LOAD './thinkthen.duckdb_extension'; SELECT thinkthen_annotate('@form.json', body) AS triage FROM (VALUES ('Steps: click Export. It is very slow.'), ('Steps: click Log in. Nobody gets in.'), ('The Pay button on billing is too blue.') ) t(body); ``` *What DuckDB printed* ``` triage {"steps":true,"area":"export","impact":1.04} {"steps":true,"area":"login","impact":1.98} {"steps":false,"area":"billing","impact":0.09} ``` ``` .load ./thinkthen WITH t(body) AS (VALUES ('Steps: click Export. It is very slow.'), ('Steps: click Log in. Nobody gets in.'), ('The Pay button on billing is too blue.')) SELECT thinkthen_annotate('@form.json', body) AS triage FROM t; ``` *What SQLite printed* ``` {"steps":true,"area":"export","impact":1.04} {"steps":true,"area":"login","impact":1.98} {"steps":false,"area":"billing","impact":0.09} ``` ``` SELECT thinkthen_annotate('@form.json', body) AS triage FROM (VALUES ('Steps: click Export. It is very slow.'), ('Steps: click Log in. Nobody gets in.'), ('The Pay button on billing is too blue.') ) AS t(body); ``` *What PostgreSQL printed* ``` triage ----------------------------------------------------- {"area": "export", "steps": true, "impact": 1.04} {"area": "login", "steps": true, "impact": 1.98} {"area": "billing", "steps": false, "impact": 0.09} (3 rows) ``` [Arguments, options, and more examples](/reference/functions/annotate/) · [annotate's edge cases](/reference/annotate/) ## Jobs that use `annotate` - [Triage a support inbox](/how-tos/triage-a-support-inbox/): for support teams Watch `annotate` answer questions about Beatles songs: [annotate answers a question set.](/learn/beatles-bench/annotate/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/choose/ # `choose`: pick one option from your list. *Each of the first three messages names one team: billing, shipping, and account. The fourth names a parcel and a login. No team reaches 0.9, and it comes back null.* ``` question="Which team owns this?" teams=( --option "billing=Invoices, fees, and refunds." --option "shipping=Parcels and delivery." --option "account=Logins and passwords." ) cat <<'EOF' | Please refund the extra fee on my invoice. My parcel went to the wrong address. I cannot reset my password. My parcel never came, and now I cannot log in to track it. EOF thinkthen choose "$question" "${teams[@]}" \ --batch 1 \ --lines \ --threshold 0.9 | jq -r .value ``` *Output* ``` billing shipping account null ``` *exit 0* You give it one question, one piece of evidence, and 2 to 255 options. You get back one of your options, or null. ## Read the answer On one piece of evidence, a pick under the threshold or an exact tie at the top prints null and exits 3. In a stream, the run exits 0. choose never exits 1. | Exit code | What it means | | --- | --- | | 0 | an option came back | | 3 | not sure | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | Send the not-sure answers to a person. ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "Which team owns this?" teams = { "billing": "Invoices, fees, and refunds.", "shipping": "Parcels and delivery.", "account": "Logins and passwords.", } team_question = tt.question( choose=question, options=teams, threshold=0.9, ) texts = [ "Please refund the extra fee on my invoice.", "My parcel went to the wrong address.", "I cannot reset my password.", ( "My parcel never came, and now " "I cannot log in to track it." ), ] owners = [ tt.choose(team_question, text).value for text in texts ] assert owners == ["billing", "shipping", "account", None] ``` No Polars sample for `choose`. No pandas sample for `choose`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Which team owns this?"; const teams = { billing: "Invoices, fees, and refunds.", shipping: "Parcels and delivery.", account: "Logins and passwords.", }; const teamQuestion = { choose: question, options: teams, threshold: 0.9, }; const texts = [ "Please refund the extra fee on my invoice.", "My parcel went to the wrong address.", "I cannot reset my password.", "My parcel never came, and now " + "I cannot log in to track it.", ]; const owners = []; for (const text of texts) { owners.push((await tt.choose(teamQuestion, text)).value); } assert.deepEqual(owners, [ "billing", "shipping", "account", null, ]); ``` ``` require "thinkthen" question = "Which team owns this?" teams = { "billing" => "Invoices, fees, and refunds.", "shipping" => "Parcels and delivery.", "account" => "Logins and passwords." } team_question = ThinkThen.question( choose: question, options: teams, threshold: 0.9 ) texts = [ "Please refund the extra fee on my invoice.", "My parcel went to the wrong address.", "I cannot reset my password.", "My parcel never came, and now " \ "I cannot log in to track it." ] owners = texts.map do |text| ThinkThen.choose(team_question, text).value end raise unless owners == [ "billing", "shipping", "account", nil ] ``` No R sample for `choose`. No Rust sample for `choose`. ``` #include #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *choose = "{\"choose\": \"Which team owns this?\", " "\"options\": {" "\"billing\": \"Invoices, fees, and refunds.\", " "\"shipping\": \"Parcels and delivery.\", " "\"account\": \"Logins and passwords.\"}, " "\"threshold\": 0.9, \"evidence\": \"%s\"}"; const char *texts[] = { "Please refund the extra fee on my invoice.", "My parcel went to the wrong address.", "I cannot reset my password.", "My parcel never came, and now " "I cannot log in to track it.", }; const char *teams[] = { "\"billing\"", "\"shipping\"", "\"account\"", "null", }; char request[320]; for (int i = 0; i < 4; i++) { snprintf(request, sizeof request, choose, texts[i]); char *team_call = thinkthen_call(tt, request); assert(team_call); enum json_tokener_error parse_error; struct json_object *result = json_tokener_parse_verbose(team_call, &parse_error); assert(parse_error == json_tokener_success); assert(result && json_object_get_type(result) == json_type_object); struct json_object *value; json_bool has_value = json_object_object_get_ex( result, "value", &value); assert(has_value); struct json_object *facts; json_bool has_facts = json_object_object_get_ex( result, "facts", &facts); assert(has_facts); assert(facts && json_object_get_type(facts) == json_type_object); struct json_object *wanted = json_tokener_parse_verbose(teams[i], &parse_error); assert(parse_error == json_tokener_success); assert(json_object_equal(value, wanted)); if (wanted) json_object_put(wanted); json_object_put(result); thinkthen_free_string(team_call); } thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `choose`. No Objective-C sample for `choose`. No COBOL sample for `choose`. No Ada sample for `choose`. No Java sample for `choose`. No Kotlin sample for `choose`. No Scala sample for `choose`. No C# sample for `choose`. No Go sample for `choose`. No Swift sample for `choose`. No Zig sample for `choose`. No PHP sample for `choose`. No Dart sample for `choose`. No DuckDB sample for `choose`. ``` .load ./thinkthen WITH t(body) AS (VALUES ('Please refund the extra fee on my invoice.'), ('My parcel went to the wrong address.'), ('I cannot reset my password.'), ('My parcel never came, and now ' || 'I cannot log in to track it.')) SELECT thinkthen_choose( '{"choose": "Which team owns this?", "options": { "billing": "Invoices, fees, and refunds.", "shipping": "Parcels and delivery.", "account": "Logins and passwords." }, "threshold": 0.9}', body ) AS team, body FROM t; ``` *What SQLite printed* ``` billing|Please refund the extra fee on my invoice. shipping|My parcel went to the wrong address. account|I cannot reset my password. |My parcel never came, and now I cannot log in to track it. ``` ``` SELECT id, thinkthen_choose( '{"choose": "Which team owns this?", "options": { "billing": "Invoices, fees, and refunds.", "shipping": "Parcels and delivery.", "account": "Logins and passwords." }, "threshold": 0.9}', body, NULL) AS team FROM (VALUES (1, 'Please refund the extra fee on my invoice.'), (2, 'My parcel went to the wrong address.'), (3, 'I cannot reset my password.'), (4, 'My parcel never came, and now ' || 'I cannot log in to track it.') ) AS t(id, body); ``` *What PostgreSQL printed* ``` id | team ----+---------- 1 | billing 2 | shipping 3 | account 4 | (4 rows) ``` [Arguments, options, and more examples](/reference/functions/choose/) ## Jobs that use `choose` - [Rank the inbound leads](/how-tos/rank-the-inbound-leads/): for sales teams - [Split a scanned packet into documents](/how-tos/split-a-scanned-packet-into-documents/): for back-office staff Watch `choose` answer questions about Beatles songs: [choose selects one option.](/learn/beatles-bench/choose/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/decide/ # `decide`: answer one yes or no question about the evidence. *The refund request answers true and the thank-you note false. "I want to send this back." could mean an exchange or money back. It lands inside the band 0.2:0.8 as null.* ``` question="Does the customer ask for a refund?" cat <<'EOF' | Please refund my order. It arrived broken. Thanks for the quick help yesterday! I want to send this back. EOF thinkthen decide "$question" \ --batch 1 \ --lines \ --threshold 0.2:0.8 | jq . ``` *Output* ``` { "input": "Please refund my order. It arrived broken.", "value": true } { "input": "Thanks for the quick help yesterday!", "value": false } { "input": "I want to send this back.", "value": null } ``` *exit 0* You give it one question and one piece of evidence. You get back true, false, or null. ## Read the answer On one piece of evidence, a probability inside the band prints null and exits 3. In a stream, that record prints null and the run exits 0. | Exit code | What it means | | --- | --- | | 0 | yes | | 1 | no | | 3 | not sure | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | Send the not-sure answers to a person. ## Call it from your language The Bash example opens this page. No Python sample for `decide`. ``` import polars as pl import thinkthen as tt question = "Does the customer ask for a refund?" tickets = pl.DataFrame({ "body": [ "Please refund my order. It arrived broken.", "Thanks for the quick help yesterday!", ], }) is_refund = tt.decide(question, tickets["body"]).value tickets = tickets.with_columns(is_refund=is_refund) assert tickets["is_refund"].to_list() == [True, False] ``` No pandas sample for `decide`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Does the customer ask for a refund?"; const broken = "Please refund my order. It arrived broken."; const thanks = "Thanks for the quick help yesterday!"; const brokenIsRefund = (await tt.decide( question, broken, )).value; const thanksIsRefund = (await tt.decide( question, thanks, )).value; assert.equal(brokenIsRefund, true); assert.equal(thanksIsRefund, false); ``` ``` require "thinkthen" question = "Does the customer ask for a refund?" broken = "Please refund my order. It arrived broken." thanks = "Thanks for the quick help yesterday!" broken_is_refund = ThinkThen.decide(question, broken).value thanks_is_refund = ThinkThen.decide(question, thanks).value raise unless broken_is_refund == true raise unless thanks_is_refund == false ``` ``` library(thinkthen) question <- "Does the customer ask for a refund?" texts <- c( "Please refund my order. It arrived broken.", "Thanks for the quick help yesterday!" ) is_refund <- tt_decide(question, texts)$value stopifnot(identical(is_refund, c(TRUE, FALSE))) ``` ``` use thinkthen::{Answer, Engine, Question}; let tt = Engine::from_env()?; let question = "Does the customer ask for a refund?"; let refund = Question::decide(question)?.cut(); let broken = "Please refund my order. It arrived broken."; let thanks = "Thanks for the quick help yesterday!"; let broken_is_refund = tt.decide(&refund, broken)? .into_value(); let thanks_is_refund = tt.decide(&refund, thanks)? .into_value(); assert_eq!(broken_is_refund, Answer::Yes); assert_eq!(thanks_is_refund, Answer::No); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *question = "Does the customer ask for a refund?"; const char *text = "Please refund my order. It arrived broken."; thinkthen_answer is_refund; int rc = thinkthen_decide( tt, question, text, strlen(text), &is_refund ); assert(rc == THINKTHEN_OK); assert(is_refund.outcome == THINKTHEN_YES); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `decide`. No Objective-C sample for `decide`. No COBOL sample for `decide`. No Ada sample for `decide`. No Java sample for `decide`. No Kotlin sample for `decide`. No Scala sample for `decide`. No C# sample for `decide`. No Go sample for `decide`. No Swift sample for `decide`. No Zig sample for `decide`. No PHP sample for `decide`. No Dart sample for `decide`. No DuckDB sample for `decide`. ``` .load ./thinkthen WITH t(body) AS (VALUES ('Please refund my order. It arrived broken.'), ('Thanks for the quick help yesterday!')) SELECT thinkthen_decide( 'Does the customer ask for a refund?', body ) AS is_refund, body FROM t; ``` *What SQLite printed* ``` 1|Please refund my order. It arrived broken. 0|Thanks for the quick help yesterday! ``` No PostgreSQL sample for `decide`. [Arguments, options, and more examples](/reference/functions/decide/) ## Jobs that use `decide` - [Screen studies for a review](/how-tos/screen-studies-for-a-review/): for researchers - [Group alerts into incidents](/how-tos/group-alerts-into-incidents/): for on-call engineers - [Screen a post before it goes up](/how-tos/screen-a-post-before-it-goes-up/): for community moderators - [Check an expense against the policy](/how-tos/check-an-expense-against-the-policy/): for finance staff - [Split a scanned packet into documents](/how-tos/split-a-scanned-packet-into-documents/): for back-office staff Watch `decide` answer questions about Beatles songs: [decide answers yes, no, or not sure.](/learn/beatles-bench/decide/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/filter/ # `filter`: keep the records where the answer is yes. *filter keeps the two complaints: the broken zipper and the snapped strap.* ``` question="Is this a complaint?" cat <<'EOF' | Arrived a day early. Thank you! The zipper broke the first time I used it. Does this come in blue? The strap snapped on day two. EOF thinkthen filter "$question" --batch 1 ``` *Output* ``` The zipper broke the first time I used it. The strap snapped on day two. ``` *exit 0: the run finished* You give it one yes-or-no question and many records. You get back the records that pass, byte for byte, in the order they went in. ## Read the answer No record sets the exit code. A record under the threshold is dropped. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "Is this a complaint?" reviews = [ "Arrived a day early. Thank you!", "The zipper broke the first time I used it.", "Does this come in blue?", "The strap snapped on day two.", ] complaints = tt.filter(question, reviews).value assert complaints == [reviews[1], reviews[3]] ``` No Polars sample for `filter`. No pandas sample for `filter`. No TypeScript sample for `filter`. ``` require "thinkthen" question = "Is this a complaint?" reviews = [ "Arrived a day early. Thank you!", "The zipper broke the first time I used it.", "Does this come in blue?", "The strap snapped on day two." ] complaints = ThinkThen.filter(question, reviews).value raise unless complaints == [reviews[1], reviews[3]] ``` ``` library(thinkthen) question <- "Is this a complaint?" reviews <- c( "Arrived a day early. Thank you!", "The zipper broke the first time I used it.", "Does this come in blue?", "The strap snapped on day two." ) complaints <- tt_filter(question, reviews)$value stopifnot(identical(complaints, reviews[c(2, 4)])) ``` ``` use thinkthen::{Engine, Question}; let tt = Engine::from_env()?; let question = "Is this a complaint?"; let complaint = Question::decide(question)?.cut(); let reviews = [ "Arrived a day early. Thank you!", "The zipper broke the first time I used it.", "Does this come in blue?", "The strap snapped on day two.", ]; let complaints = tt .filter(&complaint, reviews) .collect::, _>>()?; assert_eq!(complaints, [reviews[1], reviews[3]]); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *filter = "{\"filter\": \"Is this a complaint?\", " "\"records\": [" "\"Arrived a day early. Thank you!\", " "\"The zipper broke the first time I used it.\", " "\"Does this come in blue?\", " "\"The strap snapped on day two.\"]}"; const char *expected = "[\"The zipper broke the first time I used it.\"," "\"The strap snapped on day two.\"]"; char *complaints_call = thinkthen_call(tt, filter); assert(complaints_call); enum json_tokener_error parse_error; struct json_object *result = json_tokener_parse_verbose( complaints_call, &parse_error); assert(parse_error == json_tokener_success); assert(result && json_object_get_type(result) == json_type_object); struct json_object *value; json_bool has_value = json_object_object_get_ex( result, "value", &value); assert(has_value); struct json_object *facts; json_bool has_facts = json_object_object_get_ex( result, "facts", &facts); assert(has_facts); assert(facts && json_object_get_type(facts) == json_type_object); struct json_object *wanted = json_tokener_parse_verbose(expected, &parse_error); assert(parse_error == json_tokener_success); assert(json_object_equal(value, wanted)); if (wanted) json_object_put(wanted); json_object_put(result); thinkthen_free_string(complaints_call); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `filter`. No Objective-C sample for `filter`. No COBOL sample for `filter`. No Ada sample for `filter`. No Java sample for `filter`. No Kotlin sample for `filter`. No Scala sample for `filter`. No C# sample for `filter`. No Go sample for `filter`. No Swift sample for `filter`. No Zig sample for `filter`. No PHP sample for `filter`. No Dart sample for `filter`. No DuckDB sample for `filter`. ``` .load ./thinkthen WITH t(body) AS (VALUES ('Arrived a day early. Thank you!'), ('The zipper broke the first time I used it.'), ('Does this come in blue?'), ('The strap snapped on day two.')), reviews AS ( SELECT body, thinkthen_decide('Is this a complaint?', body) AS is_complaint FROM t) SELECT body FROM reviews WHERE is_complaint; ``` *What SQLite printed* ``` The zipper broke the first time I used it. The strap snapped on day two. ``` ``` SELECT body FROM ( SELECT body, thinkthen_decide( '{"decide": "Is this a complaint?"}', body ) AS is_complaint FROM (VALUES ('Arrived a day early. Thank you!'), ('The zipper broke the first time I used it.'), ('Does this come in blue?'), ('The strap snapped on day two.') ) AS t(body) ) AS reviews WHERE is_complaint; ``` *What PostgreSQL printed* ``` body -------------------------------------------- The zipper broke the first time I used it. The strap snapped on day two. (2 rows) ``` [Arguments, options, and more examples](/reference/functions/filter/) ## Jobs that use `filter` - [Triage a support inbox](/how-tos/triage-a-support-inbox/): for support teams - [Join two tables by meaning](/how-tos/join-two-tables-by-meaning/): for data analysts - [Rank the inbound leads](/how-tos/rank-the-inbound-leads/): for sales teams Watch `filter` answer questions about Beatles songs: [filter keeps what clears your bar.](/learn/beatles-bench/filter/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/find/ # `find`: pick the one line that best answers a question. *find prints the line with the 30 day refund deadline.* ``` question="Which line gives the refund deadline?" cat <<'EOF' | Returns need the original receipt. Refunds are issued within 30 days of purchase. Shipping is free on orders over $50. Gift cards cannot be exchanged for cash. EOF thinkthen find "$question" ``` *Output* ``` Refunds are issued within 30 days of purchase. ``` *exit 0: a line came back* You give it a question and 2 to 255 lines or records, or 2 to 254 with --none. You get back the one line that fits best. ## Read the answer Without --none, find must pick a line. When nothing fits, the line it picks is wrong. --none lets it say nothing fits. | Exit code | What it means | | --- | --- | | 0 | a line came back | | 3 | nothing fits, under --none | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "Which line gives the refund deadline?" policy = [ "Returns need the original receipt.", "Refunds are issued within 30 days of purchase.", "Shipping is free on orders over $50.", "Gift cards cannot be exchanged for cash.", ] refund_deadline = tt.find(question, policy).value assert refund_deadline["unit"] == policy[1] ``` No Polars sample for `find`. No pandas sample for `find`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Which line gives the refund deadline?"; const policy = [ "Returns need the original receipt.", "Refunds are issued within 30 days of purchase.", "Shipping is free on orders over $50.", "Gift cards cannot be exchanged for cash.", ]; const refundDeadline = (await tt.find( question, policy, )).value; assert.equal(refundDeadline?.unit, policy[1]); ``` No Ruby sample for `find`. ``` library(thinkthen) question <- "Which line gives the refund deadline?" policy <- c( "Returns need the original receipt.", "Refunds are issued within 30 days of purchase.", "Shipping is free on orders over $50.", "Gift cards cannot be exchanged for cash." ) refund_deadline <- tt_find(question, policy)$value stopifnot(identical(refund_deadline$unit, policy[2])) ``` ``` use thinkthen::{Engine, Question}; let tt = Engine::from_env()?; let question = "Which line gives the refund deadline?"; let deadline = Question::find(question)?; let policy = [ "Returns need the original receipt.", "Refunds are issued within 30 days of purchase.", "Shipping is free on orders over $50.", "Gift cards cannot be exchanged for cash.", ]; let refund_deadline = tt.find(&deadline, policy)? .into_value(); assert_eq!(refund_deadline.selected(), Some(&policy[1])); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *find = "{\"find\": \"Which line gives the refund deadline?\", " "\"units\": [" "\"Returns need the original receipt.\", " "\"Refunds are issued within 30 days of purchase.\", " "\"Shipping is free on orders over $50.\", " "\"Gift cards cannot be exchanged for cash.\"]}"; const char *expected = "\"Refunds are issued within 30 days of purchase.\""; char *deadline_call = thinkthen_call(tt, find); assert(deadline_call); enum json_tokener_error parse_error; struct json_object *result = json_tokener_parse_verbose(deadline_call, &parse_error); assert(parse_error == json_tokener_success); assert(result && json_object_get_type(result) == json_type_object); struct json_object *value; json_bool has_value = json_object_object_get_ex( result, "value", &value); assert(has_value); struct json_object *facts; json_bool has_facts = json_object_object_get_ex( result, "facts", &facts); assert(has_facts); assert(facts && json_object_get_type(facts) == json_type_object); struct json_object *wanted = json_tokener_parse_verbose(expected, &parse_error); assert(parse_error == json_tokener_success); struct json_object *unit; json_bool has_unit = json_object_object_get_ex( value, "unit", &unit); assert(has_unit); assert(json_object_equal(unit, wanted)); if (wanted) json_object_put(wanted); json_object_put(result); thinkthen_free_string(deadline_call); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `find`. No Objective-C sample for `find`. No COBOL sample for `find`. No Ada sample for `find`. No Java sample for `find`. No Kotlin sample for `find`. No Scala sample for `find`. No C# sample for `find`. No Go sample for `find`. No Swift sample for `find`. No Zig sample for `find`. No PHP sample for `find`. No Dart sample for `find`. No DuckDB sample for `find`. ``` .load ./thinkthen WITH picked AS ( SELECT thinkthen_find( 'Which line gives the refund deadline?', json_array( 'Returns need the original receipt.', 'Refunds are issued within 30 days ' || 'of purchase.', 'Shipping is free on orders over $50.', 'Gift cards cannot be exchanged for cash.' ) ) AS deadline_result ) SELECT json_extract(deadline_result, '$.value') AS refund_deadline FROM picked; ``` *What SQLite printed* ``` Refunds are issued within 30 days of purchase. ``` ``` WITH picked AS ( SELECT thinkthen_find( 'Which line gives the refund deadline?', ARRAY[ 'Returns need the original receipt.', 'Refunds are issued within 30 days ' || 'of purchase.', 'Shipping is free on orders over $50.', 'Gift cards cannot be exchanged for cash.' ] ) AS deadline_result ) SELECT deadline_result ->> 'value' AS refund_deadline FROM picked; ``` *What PostgreSQL printed* ``` refund_deadline ------------------------------------------------ Refunds are issued within 30 days of purchase. (1 row) ``` [Arguments, options, and more examples](/reference/functions/find/) ## Jobs that use `find` - [Group alerts into incidents](/how-tos/group-alerts-into-incidents/): for on-call engineers - [Check an expense against the policy](/how-tos/check-an-expense-against-the-policy/): for finance staff Watch `find` answer questions about Beatles songs: [find picks one from many.](/learn/beatles-bench/find/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/ # The ten functions Each function reads text and answers a question about it. Each does a job you already write in code: an if, a switch, a grep, a sort. 10 functions, [1 CLI](/install/shell/), [24 bindings](/install/). | Function | What it does | Kind of answer | | --- | --- | --- | | [`decide`](/functions/decide/) | Answer one yes or no question about the evidence. | Yes or no | | [`choose`](/functions/choose/) | Pick one option from your list. | Pick one | | [`tag`](/functions/tag/) | Name every label that fits. | Yes or no, per label | | [`score`](/functions/score/) | Place the evidence on a scale you name. | Place on a scale | | [`filter`](/functions/filter/) | Keep the records where the answer is yes. | Yes or no, per record | | [`rank`](/functions/rank/) | Sort records by how likely the answer is yes. | Yes or no, per record | | [`find`](/functions/find/) | Pick the one line that best answers a question. | Pick one line of the evidence | | [`annotate`](/functions/annotate/) | Fill out a form for every record. | Every kind of answer at once | | [`recognize`](/functions/recognize/) | Find every name in the evidence and say what kind it is. | Pick one, per word | | [`relate`](/functions/relate/) | Find relationships among named entities. | Yes or no per pair, or pick one | | [`@question`](/functions/question-file/) | A saved question that six functions accept. | No answer of its own | On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/question-file/ # `@question`: a saved question that six functions accept. *refund.json, a file the examples read* ``` { "decide": "Does the customer ask for a refund?", "true": "The customer asks for money back.", "false": "Anything else, such as a cancellation or thanks.", "threshold": "0.2:0.8" } ``` *The saved question answers true for a plain request for money back.* ``` cat <<'EOF' | I would like to return this and get my money back. EOF thinkthen decide @refund.json ``` *Output* ``` true ``` *exit 0: yes* Save one question in a JSON file. Pass it as `@FILE` to decide, choose, tag, score, filter, or rank. Every command that reads the file then asks the same question. ## Read the answer A band in the file marks the middle answers not sure. Send those to a person. | Exit code | What it means | | --- | --- | | 5 | the file could not be read, is not one JSON object, or breaks a rule | | 2 | the command names the wrong verb for the file | ## Call it from your language The Bash example opens this page. No Python sample for `question-file`. ``` import polars as pl import thinkthen as tt question = "Does the customer ask for a refund?" refund = tt.question(decide=question, threshold=(0.2, 0.8)) tickets = pl.DataFrame({ "body": [ "Please refund my order. It arrived broken.", "Thanks for the quick help yesterday!", "I want to send this back.", ], }) is_refund = tt.decide(refund, tickets["body"]).value tickets = tickets.with_columns(is_refund=is_refund) assert tickets["is_refund"].to_list() == [True, False, None] ``` No pandas sample for `question-file`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Does the customer ask for a refund?"; const refund = tt.question({ decide: question, threshold: [0.2, 0.8], }); const sendBack = "I want to send this back."; const isRefund = (await tt.decide(refund, sendBack)).value; assert.equal(isRefund, null); ``` ``` require "thinkthen" question = "Does the customer ask for a refund?" refund = ThinkThen.question( decide: question, threshold: 0.2..0.8 ) send_back = "I want to send this back." is_refund = ThinkThen.decide(refund, send_back).value raise unless is_refund.nil? ``` ``` library(thinkthen) question <- "Does the customer ask for a refund?" refund <- tt_question( decide = question, threshold = c(0.2, 0.8) ) texts <- c( "Please refund my order. It arrived broken.", "Thanks for the quick help yesterday!", "I want to send this back." ) is_refund <- tt_decide(refund, texts)$value stopifnot(identical(is_refund, c(TRUE, FALSE, NA))) ``` ``` use thinkthen::{Answer, Engine, Question}; let tt = Engine::from_env()?; let question = "Does the customer ask for a refund?"; let refund = Question::decide(question)?.band(0.2, 0.8)?; let send_back = "I want to send this back."; let is_refund = tt.decide(&refund, send_back)?.into_value(); assert_eq!(is_refund, Answer::Unsure); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. No C sample for `question-file`. No C++ sample for `question-file`. No Objective-C sample for `question-file`. No COBOL sample for `question-file`. No Ada sample for `question-file`. No Java sample for `question-file`. No Kotlin sample for `question-file`. No Scala sample for `question-file`. No C# sample for `question-file`. No Go sample for `question-file`. No Swift sample for `question-file`. No Zig sample for `question-file`. No PHP sample for `question-file`. No Dart sample for `question-file`. ``` LOAD './thinkthen.duckdb_extension'; SELECT trim(body, chr(10)) AS body, thinkthen_decide('@refund.json', body) AS is_refund FROM (VALUES ('I would like to return this and get ' || 'my money back.' || chr(10)), ('I want to send this back.' || chr(10)) ) t(body); ``` *What DuckDB printed* ``` body|is_refund I would like to return this and get my money back.|true I want to send this back.|NULL ``` ``` .load ./thinkthen WITH t(body) AS (VALUES ('I would like to return this and get ' || 'my money back.' || char(10)), ('I want to send this back.' || char(10))) SELECT thinkthen_decide('@refund.json', body) AS is_refund, trim(body, char(10)) AS body FROM t; ``` *What SQLite printed* ``` 1|I would like to return this and get my money back. |I want to send this back. ``` No PostgreSQL sample for `question-file`. [Arguments, options, and more examples](/reference/functions/question-file/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/rank/ # `rank`: sort records by how likely the answer is yes. *The outage comes first and the quote due today second. The bill due in a month comes third, and the newsletter that needs no reply comes last.* ``` question="Is this urgent?" cat <<'EOF' | Newsletter: our autumn catalog is here. No reply needed. Our checkout page is down and customers cannot pay Reminder: your invoice is due in 30 days Please send the signed quote by 5 pm today EOF thinkthen rank "$question" --batch 1 ``` *Output* ``` Our checkout page is down and customers cannot pay Please send the signed quote by 5 pm today Reminder: your invoice is due in 30 days Newsletter: our autumn catalog is here. No reply needed. ``` *exit 0: the run finished* You give it one yes-or-no question and many records. You get back every record again, most likely first. ## Read the answer rank takes no threshold. It drops nothing, and nothing is not sure. The sort happens on this machine. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "Is this urgent?" inbox = [ ( "Newsletter: our autumn catalog is here. " "No reply needed." ), "Our checkout page is down and customers cannot pay", "Reminder: your invoice is due in 30 days", "Please send the signed quote by 5 pm today", ] by_urgency = tt.rank(question, inbox).value order = [one["index"] for one in by_urgency] assert order == [1, 3, 2, 0] ``` No Polars sample for `rank`. No pandas sample for `rank`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Is this urgent?"; const inbox = [ "Newsletter: our autumn catalog is here. " + "No reply needed.", "Our checkout page is down and customers cannot pay", "Reminder: your invoice is due in 30 days", "Please send the signed quote by 5 pm today", ]; const byUrgency = (await tt.rank(question, inbox)).value; const order = byUrgency.map((one) => one.index); assert.deepEqual(order, [1, 3, 2, 0]); ``` ``` require "thinkthen" question = "Is this urgent?" inbox = [ "Newsletter: our autumn catalog is here. " \ "No reply needed.", "Our checkout page is down and customers cannot pay", "Reminder: your invoice is due in 30 days", "Please send the signed quote by 5 pm today" ] by_urgency = ThinkThen.rank(question, inbox).value raise unless by_urgency.map(&:index) == [1, 3, 2, 0] ``` No R sample for `rank`. ``` use thinkthen::{Engine, Question}; let tt = Engine::from_env()?; let question = "Is this urgent?"; let urgent = Question::rank(question)?; let inbox = [ concat!( "Newsletter: our autumn catalog is here. ", "No reply needed.", ), "Our checkout page is down and customers cannot pay", "Reminder: your invoice is due in 30 days", "Please send the signed quote by 5 pm today", ]; let by_urgency = tt.rank(&urgent, inbox)?.into_value(); let order: Vec<&str> = by_urgency .iter() .map(|one| *one.input()) .collect(); assert_eq!(order, [inbox[1], inbox[3], inbox[2], inbox[0]]); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *rank = "{\"rank\": \"Is this urgent?\", " "\"records\": [" "\"Newsletter: our autumn catalog is here. " "No reply needed.\", " "\"Our checkout page is down and customers " "cannot pay\", " "\"Reminder: your invoice is due in 30 days\", " "\"Please send the signed quote by 5 pm today\"]}"; const int expected[] = {1, 3, 2, 0}; char *rank_call = thinkthen_call(tt, rank); assert(rank_call); enum json_tokener_error parse_error; struct json_object *result = json_tokener_parse_verbose(rank_call, &parse_error); assert(parse_error == json_tokener_success); assert(result && json_object_get_type(result) == json_type_object); struct json_object *value; json_bool has_value = json_object_object_get_ex( result, "value", &value); assert(has_value); struct json_object *facts; json_bool has_facts = json_object_object_get_ex( result, "facts", &facts); assert(has_facts); assert(facts && json_object_get_type(facts) == json_type_object); assert(json_object_array_length(value) == 4); for (size_t i = 0; i < 4; i++) { struct json_object *index; struct json_object *ranked = json_object_array_get_idx(value, i); json_bool has_index = json_object_object_get_ex(ranked, "index", &index); assert(has_index); assert(json_object_get_int(index) == expected[i]); } json_object_put(result); thinkthen_free_string(rank_call); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `rank`. No Objective-C sample for `rank`. No COBOL sample for `rank`. No Ada sample for `rank`. No Java sample for `rank`. No Kotlin sample for `rank`. No Scala sample for `rank`. No C# sample for `rank`. No Go sample for `rank`. No Swift sample for `rank`. No Zig sample for `rank`. No PHP sample for `rank`. No Dart sample for `rank`. ``` LOAD './thinkthen.duckdb_extension'; SELECT body FROM ( SELECT body, thinkthen_probability('Is this urgent?', body) AS urgency FROM (VALUES ('Newsletter: our autumn catalog is here. ' || 'No reply needed.'), ('Our checkout page is down and customers ' || 'cannot pay'), ('Reminder: your invoice is due in 30 days'), ('Please send the signed quote by 5 pm today') ) t(body) ) ORDER BY urgency DESC; ``` *What DuckDB printed* ``` body Our checkout page is down and customers cannot pay Please send the signed quote by 5 pm today Reminder: your invoice is due in 30 days Newsletter: our autumn catalog is here. No reply needed. ``` No SQLite sample for `rank`. ``` WITH t(id, body) AS (VALUES ('1', 'Newsletter: our autumn catalog is here. ' || 'No reply needed.'), ('2', 'Our checkout page is down and ' || 'customers cannot pay'), ('3', 'Reminder: your invoice is due in 30 days'), ('4', 'Please send the signed quote by 5 pm today') ) SELECT body FROM ( SELECT t.body, d.probability AS urgency FROM t JOIN thinkthen_decide_many( '{"decide": "Is this urgent?"}', (SELECT jsonb_object_agg(id, body) FROM t) ) AS d ON d.key = t.id ) AS ranked ORDER BY urgency DESC; ``` *What PostgreSQL printed* ``` body ---------------------------------------------------------- Our checkout page is down and customers cannot pay Please send the signed quote by 5 pm today Reminder: your invoice is due in 30 days Newsletter: our autumn catalog is here. No reply needed. (4 rows) ``` [Arguments, options, and more examples](/reference/functions/rank/) ## Jobs that use `rank` - [Rank the inbound leads](/how-tos/rank-the-inbound-leads/): for sales teams Watch `rank` answer questions about Beatles songs: [rank sorts by your criteria.](/learn/beatles-bench/rank/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/recognize/ # `recognize`: find every name in the evidence and say what kind it is. *names.json, a file the examples read* ``` { "version": 1, "recognize": { "kinds": { "person": null, "organization": null, "place": null }, "relations": [ { "name": "works_for", "source": "person", "target": "organization" }, { "name": "based_in", "source": "organization", "target": "place" } ] } } ``` *recognize finds three names. Each comes back with its kind and its strength.* ``` person="PER=Part of a person's name." org="ORG=Part of the name of an organization:" org+=" a company, band, team, agency, government" org+=" body, or media outlet." place="LOC=Part of the name of a place: a country," place+=" region, city, or geographic feature." other="MISC=Part of another named entity: a" other+=" nationality, an event, a product, or the" other+=" name of a creative work." text="Maria Chen joined Northwind Freight in Chicago" text+=" last spring." printf '%s' "$text" | thinkthen recognize \ --kind "$person" \ --kind "$org" \ --kind "$place" \ --kind "$other" | jq -c '.entities[] | [.text, .kind, .strength]' ``` *Output* ``` ["Maria Chen","PER",0.9987] ["Northwind Freight","ORG",0.997] ["Chicago","LOC",1.0] ``` *exit 0* You give it the evidence and the kinds of name you allow. You get back each name, its kind, where it sits, and a strength. ## Read the answer With kinds, the model picks each name's kind from them. A name that is not in the evidence cannot come back. The number on a name is its strength. ThinkThen computes it, and it is not a probability. Your threshold decides which names you keep. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt text = ( "Maria Chen joined Northwind Freight " "in Chicago last spring." ) kinds = { "PER": "Part of a person's name.", "ORG": ( "Part of the name of an organization: a company, " "band, team, agency, government body, " "or media outlet." ), "LOC": "Part of the name of a place: a country, " "region, city, or geographic feature.", "MISC": ( "Part of another named entity: a nationality, " "an event, a product, or the name of a " "creative work." ), } facts = tt.recognize( text, kinds=kinds, ).value names = [(one.text, one.kind) for one in facts.entities] assert names == [ ("Maria Chen", "PER"), ("Northwind Freight", "ORG"), ("Chicago", "LOC"), ] ``` No Polars sample for `recognize`. No pandas sample for `recognize`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const text = "Maria Chen joined Northwind Freight, " + "a company in Chicago."; const kinds = ["person", "organization", "place"]; const relations = { works_for: ["person", "organization"] as const, based_in: ["organization", "place"] as const, }; const facts = (await tt.recognize(text, { kinds, relations, })).value; const names = facts.entities.map((one) => [ one.text, one.kind, ]); assert.deepEqual(names, [ ["Maria Chen", "person"], ["Northwind Freight", "organization"], ["Chicago", "place"], ]); const links = (facts.relations ?? []).map((one) => [ one.relation, one.source.text, one.target.text, ]); assert.deepEqual(links, [ ["works_for", "Maria Chen", "Northwind Freight"], ["based_in", "Northwind Freight", "Chicago"], ]); ``` ``` require "thinkthen" text = "Maria Chen joined Northwind Freight, " \ "a company in Chicago." kinds = ["person", "organization", "place"] relations = { works_for: ["person", "organization"], based_in: ["organization", "place"] } facts = ThinkThen.recognize(text, kinds:, relations:).value names = facts.entities.map { |one| [one.text, one.kind] } raise unless names == [ ["Maria Chen", "person"], ["Northwind Freight", "organization"], ["Chicago", "place"] ] links = facts.relations.map do |one| [one.relation, one.source.text, one.target.text] end raise unless links == [ ["works_for", "Maria Chen", "Northwind Freight"], ["based_in", "Northwind Freight", "Chicago"] ] ``` ``` library(thinkthen) text <- paste0( "Maria Chen joined Northwind Freight, ", "a company in Chicago." ) kinds <- c("person", "organization", "place") rules <- c( "works_for=person:organization", "based_in=organization:place" ) facts <- tt_recognize( text, kinds, relations = rules )$value[[1]] stopifnot(identical( facts$text, c("Maria Chen", "Northwind Freight", "Chicago") )) stopifnot(identical(facts$kind, kinds)) links <- attr(facts, "relations") stopifnot(identical( links$relation, c("works_for", "based_in") )) stopifnot(identical( links$source, c("Maria Chen", "Northwind Freight") )) stopifnot(identical( links$target, c("Northwind Freight", "Chicago") )) ``` ``` use thinkthen::{Engine, Kind, Recognize, RelationRule}; let tt = Engine::from_env()?; let text = concat!( "Maria Chen joined Northwind Freight, ", "a company in Chicago.", ); let works_for = RelationRule::one_way( "works_for", "person", "organization", )?; let based_in = RelationRule::one_way( "based_in", "organization", "place", )?; let ask = Recognize::builder() .kind(Kind::new("person", None)?)? .kind(Kind::new("organization", None)?)? .kind(Kind::new("place", None)?)? .relation(works_for)? .relation(based_in)? .build()?; let facts = tt.recognize(&ask, text)?.into_value(); let names: Vec<_> = facts .entities() .iter() .map(|one| (one.text(), one.kind())) .collect(); let expected = [ ("Maria Chen", "person"), ("Northwind Freight", "organization"), ("Chicago", "place"), ]; assert_eq!(names, expected); let links: Vec<_> = facts .relations() .unwrap_or_default() .iter() .map(|one| { let source = one.source().text(); let target = one.target().text(); (one.relation(), source, target) }) .collect(); let expected = [ ("works_for", "Maria Chen", "Northwind Freight"), ("based_in", "Northwind Freight", "Chicago"), ]; assert_eq!(links, expected); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *text = "Maria Chen joined Northwind Freight, " "a company in Chicago."; const char *spec = "{\"version\": 1, \"recognize\": {" "\"kinds\": {\"person\": null, " "\"organization\": null, \"place\": null}, " "\"relations\": [" "{\"name\": \"works_for\", \"source\": \"person\", " "\"target\": \"organization\"}, " "{\"name\": \"based_in\", " "\"source\": \"organization\", " "\"target\": \"place\"}]}}"; char *facts; size_t facts_len; int rc = thinkthen_recognize( tt, spec, text, strlen(text), &facts, &facts_len ); assert(rc == THINKTHEN_OK); assert(strstr(facts, "\"Maria Chen\"")); assert(strstr(facts, "\"Northwind Freight\"")); assert(strstr(facts, "\"Chicago\"")); assert(strstr(facts, "\"works_for\"")); assert(strstr(facts, "\"based_in\"")); thinkthen_free_string(facts); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `recognize`. No Objective-C sample for `recognize`. No COBOL sample for `recognize`. No Ada sample for `recognize`. No Java sample for `recognize`. No Kotlin sample for `recognize`. No Scala sample for `recognize`. No C# sample for `recognize`. No Go sample for `recognize`. No Swift sample for `recognize`. No Zig sample for `recognize`. No PHP sample for `recognize`. No Dart sample for `recognize`. ``` LOAD './thinkthen.duckdb_extension'; CREATE TABLE tickets AS FROM (VALUES (1, 'Maria Chen joined Northwind Freight, ' || 'a company in Chicago.') ) t(id, body); SELECT id, entity.text, entity.kind FROM ( SELECT id, unnest(thinkthen_recognize( body, ['person', 'organization', 'place'] )) AS entity FROM tickets ); SELECT id, link.relation, link.source, link.target FROM ( SELECT id, unnest(thinkthen_relations( body, '@names.json' )) AS link FROM tickets ); ``` *What DuckDB printed* ``` id|text|kind 1|Maria Chen|person 1|Northwind Freight|organization 1|Chicago|place id|relation|source|target 1|works_for|Maria Chen|Northwind Freight 1|based_in|Northwind Freight|Chicago ``` ``` .load ./thinkthen CREATE TABLE tickets(id INTEGER, body TEXT); INSERT INTO tickets VALUES ( 1, 'Maria Chen joined Northwind Freight, ' || 'a company in Chicago.' ); SELECT t.id, entity.text, entity.kind FROM tickets t, thinkthen_recognize( t.body, 'person,organization,place' ) entity; WITH found AS ( SELECT id, thinkthen_relations(body, '@names.json') AS links_found FROM tickets) SELECT found.id, link.value ->> 'relation' AS relation, link.value ->> '$.source.text' AS source, link.value ->> '$.target.text' AS target FROM found, json_each(found.links_found, '$.relations') AS link; ``` *What SQLite printed* ``` 1|Maria Chen|person 1|Northwind Freight|organization 1|Chicago|place 1|works_for|Maria Chen|Northwind Freight 1|based_in|Northwind Freight|Chicago ``` ``` CREATE TABLE tickets (id int, body text); INSERT INTO tickets VALUES ( 1, 'Maria Chen joined Northwind Freight, ' || 'a company in Chicago.' ); SELECT t.id, entity.text, entity.kind FROM tickets t, LATERAL thinkthen_recognize( t.body, ARRAY['person', 'organization', 'place'] ) entity; SELECT t.id, link.relation, link.source_text, link.target_text FROM tickets t, LATERAL thinkthen_relations( t.body, '@names.json' ) link; ``` *What PostgreSQL printed* ``` id | text | kind ----+-------------------+-------------- 1 | Maria Chen | person 1 | Northwind Freight | organization 1 | Chicago | place (3 rows) id | relation | source_text | target_text ----+-----------+-------------------+------------------- 1 | works_for | Maria Chen | Northwind Freight 1 | based_in | Northwind Freight | Chicago (2 rows) ``` [Arguments, options, and more examples](/reference/functions/recognize/) Watch `recognize` answer questions about Beatles songs: [recognize labels things it finds.](/learn/beatles-bench/recognize/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/relate/ # `relate`: find relationships among named entities. *Paul McCartney sings Yesterday, and Ringo Starr sings Octopus's Garden. The two wrong pairs do not reach the default threshold.* ``` sings="sings=singer:song" cat <<'EOF' | [ {"name": "Paul McCartney", "kind": "singer"}, {"name": "Ringo Starr", "kind": "singer"}, {"name": "Yesterday", "kind": "song"}, {"name": "Octopus's Garden", "kind": "song"} ] EOF thinkthen relate "$sings" | jq -c '[.source.name, .target.name, .probability]' ``` *Output* ``` ["Paul McCartney","Yesterday",0.81] ["Ringo Starr","Octopus's Garden",0.88] ``` *exit 0* You give it a set of names, the kind of each name, and the relations you care about. `relate` reads no other text. Jev answers from what it knows about the names. You get back one edge for each related pair, with its probability. A rule asks one yes or no question for each pair. In a rules file, a rule marked `"single": true` asks one choice for each source instead, and gives that source at most one edge. For the links a text states, use `recognize --relation`. ## Read the answer A relation has a direction, or it reads the same both ways. The number on an edge is a probability. Your threshold decides which edges you keep. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 6 | the run finished with failed questions | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt names = [ ("Paul McCartney", "singer"), ("Ringo Starr", "singer"), ("Yesterday", "song"), ("Octopus's Garden", "song"), ] who_sings = tt.relate( names, relations={"sings": ("singer", "song")}, ).value sings = [ (edge.source.name, edge.target.name) for edge in who_sings ] assert sings == [ ("Paul McCartney", "Yesterday"), ("Ringo Starr", "Octopus's Garden"), ] ``` No Polars sample for `relate`. No pandas sample for `relate`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const rules = [ "Book economy class for every flight under six hours.", "Submit receipts within 30 days of the trip.", "Hotel stays are capped at 200 dollars a night.", "Employees may book business class on any flight.", "Rental cars need a manager's approval.", "Receipts may be submitted at any time, " + "with no deadline.", "Meals are reimbursed up to 60 dollars a day.", "Use the company travel portal for all bookings.", ]; const entities = rules.map( (rule): readonly [string, string] => [rule, "rule"], ); const contradictions = (await tt.relate(entities, { relations: ["contradicts"], either: ["contradicts"], threshold: 0.5, })).value; const pairs = contradictions.map((edge) => [ edge.source.name, edge.target.name, edge.probability, ]); assert.deepEqual(pairs, [ [rules[0], rules[3], 0.83], [rules[1], rules[5], 0.97], ]); ``` ``` require "thinkthen" rules = [ "Book economy class for every flight under six hours.", "Submit receipts within 30 days of the trip.", "Hotel stays are capped at 200 dollars a night.", "Employees may book business class on any flight.", "Rental cars need a manager's approval.", "Receipts may be submitted at any time, " \ "with no deadline.", "Meals are reimbursed up to 60 dollars a day.", "Use the company travel portal for all bookings." ] entities = rules.map { |rule| [rule, "rule"] } contradictions = ThinkThen.relate( entities, relations: ["contradicts"], either: ["contradicts"], threshold: 0.5 ).value pairs = contradictions.map do |edge| [edge.source.name, edge.target.name, edge.probability] end raise unless pairs == [ [rules[0], rules[3], 0.83], [rules[1], rules[5], 0.97] ] ``` ``` library(thinkthen) rules <- c( "Book economy class for every flight under six hours.", "Submit receipts within 30 days of the trip.", "Hotel stays are capped at 200 dollars a night.", "Employees may book business class on any flight.", "Rental cars need a manager's approval.", paste0( "Receipts may be submitted at any time, ", "with no deadline." ), "Meals are reimbursed up to 60 dollars a day.", "Use the company travel portal for all bookings." ) entities <- data.frame(name = rules, kind = "rule") contradictions <- tt_relate( entities, either = "contradicts", threshold = 0.5 )$value stopifnot(identical(contradictions$source, rules[c(1, 2)])) stopifnot(identical(contradictions$target, rules[c(4, 6)])) stopifnot(identical( contradictions$probability, c(0.83, 0.97) )) ``` ``` use thinkthen::{Engine, Entity, Relate, RelationRule}; let tt = Engine::from_env()?; let rules = [ "Book economy class for every flight under six hours.", "Submit receipts within 30 days of the trip.", "Hotel stays are capped at 200 dollars a night.", "Employees may book business class on any flight.", "Rental cars need a manager's approval.", concat!( "Receipts may be submitted at any time, ", "with no deadline.", ), "Meals are reimbursed up to 60 dollars a day.", "Use the company travel portal for all bookings.", ]; let contradicts = RelationRule::both_ways("contradicts", "*", "*")?; let ask = Relate::builder() .relation(contradicts)? .threshold(0.5)? .build()?; let entities = rules .iter() .map(|rule| Entity::new(rule, "rule")) .collect::, _>>()?; let contradictions = tt.relate(&ask, entities)? .into_value(); let pairs: Vec<_> = contradictions .iter() .map(|edge| { let source = edge.source().name(); let target = edge.target().name(); (source, target, edge.probability()) }) .collect(); let expected = [ (rules[0], rules[3], 0.83), (rules[1], rules[5], 0.97), ]; assert_eq!(pairs, expected); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. ``` #include #include #include #include thinkthen_engine *tt = thinkthen_engine_new(); assert(tt); const char *rules[] = { "Book economy class for every flight under six hours.", "Submit receipts within 30 days of the trip.", "Hotel stays are capped at 200 dollars a night.", "Employees may book business class on any flight.", "Rental cars need a manager's approval.", "Receipts may be submitted at any time, " "with no deadline.", "Meals are reimbursed up to 60 dollars a day.", "Use the company travel portal for all bookings.", }; char records[8][96]; const char *texts[8]; size_t lengths[8]; for (int i = 0; i < 8; i++) { snprintf( records[i], sizeof records[i], "{\"name\": \"%s\", \"kind\": \"rule\"}", rules[i] ); texts[i] = records[i]; lengths[i] = strlen(records[i]); } const char *spec = "{\"version\": 1, \"relate\": {\"relations\": [{" "\"name\": \"contradicts\", \"source\": \"*\", " "\"target\": \"*\", \"either\": true}]}, " "\"threshold\": 0.5}"; char *contradictions; size_t contradictions_len; int rc = thinkthen_relate( tt, spec, texts, lengths, 8, &contradictions, &contradictions_len ); assert(rc == THINKTHEN_OK); assert(strstr(contradictions, "\"probability\":0.83")); assert(strstr(contradictions, "\"probability\":0.97")); thinkthen_free_string(contradictions); thinkthen_engine_free(tt); ``` Keep the `#include` lines on top. Put the rest inside `int main(void)` and end it with `return 0;`. No C++ sample for `relate`. No Objective-C sample for `relate`. No COBOL sample for `relate`. No Ada sample for `relate`. No Java sample for `relate`. No Kotlin sample for `relate`. No Scala sample for `relate`. No C# sample for `relate`. No Go sample for `relate`. No Swift sample for `relate`. No Zig sample for `relate`. No PHP sample for `relate`. No Dart sample for `relate`. ``` LOAD './thinkthen.duckdb_extension'; CREATE TABLE rules AS FROM (VALUES (1, 'Book economy class for every flight ' || 'under six hours.'), (2, 'Submit receipts within 30 days of the trip.'), (3, 'Hotel stays are capped at 200 dollars a night.'), (4, 'Employees may book business class on any flight.'), (5, 'Rental cars need a manager''s approval.'), (6, 'Receipts may be submitted at any time, ' || 'with no deadline.'), (7, 'Meals are reimbursed up to 60 dollars a day.'), (8, 'Use the company travel portal for all bookings.') ) r(id, body); SELECT * FROM thinkthen_relate( 'SELECT id, body AS name, ''rule'' AS kind FROM rules', '{"version": 1, "relate": {"relations": [{' || '"name": "contradicts", "source": "*", ' || '"target": "*", "either": true}]}}' ) AS contradiction; ``` *What DuckDB printed* ``` relation|source|target|probability|either contradicts|1|4|0.83|true contradicts|2|6|0.97|true ``` ``` .load ./thinkthen CREATE TABLE rules(id INTEGER, body TEXT, kind TEXT); INSERT INTO rules VALUES (1, 'Book economy class for every flight ' || 'under six hours.', 'rule'), (2, 'Submit receipts within 30 days of the trip.', 'rule'), (3, 'Hotel stays are capped at 200 dollars a night.', 'rule'), (4, 'Employees may book business class on any flight.', 'rule'), (5, 'Rental cars need a manager''s approval.', 'rule'), (6, 'Receipts may be submitted at any time, ' || 'with no deadline.', 'rule'), (7, 'Meals are reimbursed up to 60 dollars a day.', 'rule'), (8, 'Use the company travel portal for all bookings.', 'rule'); SELECT * FROM thinkthen_relate( 'SELECT id, body AS name, kind FROM rules', 'either:contradicts' ) AS contradiction; ``` *What SQLite printed* ``` contradicts|1|4|0.83|1 contradicts|2|6|0.97|1 ``` ``` CREATE TABLE rules ( id int, body text, kind text DEFAULT 'rule' ); INSERT INTO rules (id, body) VALUES (1, 'Book economy class for every flight ' || 'under six hours.'), (2, 'Submit receipts within 30 days of the trip.'), (3, 'Hotel stays are capped at 200 dollars a night.'), (4, 'Employees may book business class on any flight.'), (5, 'Rental cars need a manager''s approval.'), (6, 'Receipts may be submitted at any time, ' || 'with no deadline.'), (7, 'Meals are reimbursed up to 60 dollars a day.'), (8, 'Use the company travel portal for all bookings.'); SELECT * FROM thinkthen_relate( 'SELECT id, body AS name, kind FROM rules', '{"version": 1, "relate": {"relations": [{' || '"name": "contradicts", "source": "*", ' || '"target": "*", "either": true}]}}' ) AS contradiction; ``` *What PostgreSQL printed* ``` relation | source | target | probability | either -------------+--------+--------+-------------+-------- contradicts | 1 | 4 | 0.83 | t contradicts | 2 | 6 | 0.97 | t (2 rows) ``` [Arguments, options, and more examples](/reference/functions/relate/) Watch `relate` answer questions about Beatles songs: [relate asks what Jev knows about the names.](/learn/beatles-bench/relate/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/score/ # `score`: place the evidence on a scale you name. *The outage scores 2.0, and 2 is Immediate on this scale.* ``` question="How urgent is this?" levels=( "Routine." "Soon." "Immediate." ) cat <<'EOF' | Our checkout page is down and customers cannot pay. EOF thinkthen score "$question" "${levels[@]}" ``` *Output* ``` 2.0 ``` *exit 0: the run finished* You give it one question, one piece of evidence, and 2 to 10 levels, least first. You get back a number along your levels. The first level is 0. ## Read the answer score has no threshold and no not-sure answer. It always lands somewhere on the scale. It orders a queue a person reads. Do not use it to decide yes or no. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "How urgent is this?" levels = ["Routine.", "Soon.", "Immediate."] texts = [ "Please update my mailing address when you can.", "Can you send the signed contract by Friday?", "Nobody can log in to the site right now.", ] urgency = [ tt.score(question, text, levels=levels).value for text in texts ] assert urgency == [0.06, 0.99, 2.0] ``` ``` import polars as pl import thinkthen as tt question = "How urgent is this?" levels = ["Routine.", "Soon.", "Immediate."] tickets = pl.DataFrame({ "body": [ "Please update my mailing address when you can.", "Can you send the signed contract by Friday?", "Nobody can log in to the site right now.", ], }) urgency = tt.score( question, tickets["body"], levels=levels, ).value tickets = tickets.with_columns(urgency=urgency) assert tickets["urgency"].to_list() == [0.06, 0.99, 2.0] ``` No pandas sample for `score`. No TypeScript sample for `score`. ``` require "thinkthen" question = "How urgent is this?" levels = ["Routine.", "Soon.", "Immediate."] texts = [ "Please update my mailing address when you can.", "Can you send the signed contract by Friday?", "Nobody can log in to the site right now." ] urgency = texts.map do |text| ThinkThen.score(question, text, levels:).value end raise unless urgency == [0.06, 0.99, 2.0] ``` ``` library(thinkthen) question <- "How urgent is this?" levels <- c("Routine.", "Soon.", "Immediate.") texts <- c( "Please update my mailing address when you can.", "Can you send the signed contract by Friday?", "Nobody can log in to the site right now." ) urgency <- tt_score(question, texts, levels)$value stopifnot(identical(urgency, c(0.06, 0.99, 2.0))) ``` ``` use thinkthen::{Engine, Question}; let tt = Engine::from_env()?; let question = "How urgent is this?"; let urgency_scale = Question::score(question)? .level("Routine.", None)? .level("Soon.", None)? .level("Immediate.", None)? .build()?; let texts = [ "Please update my mailing address when you can.", "Can you send the signed contract by Friday?", "Nobody can log in to the site right now.", ]; let urgency = texts .iter() .map(|text| { tt.score(&urgency_scale, text) .map(|call| call.into_value()) }) .collect::, _>>()?; assert_eq!(urgency, [0.06, 0.99, 2.0]); ``` Put this code inside `fn main() -> Result<(), Box>` and end it with `Ok(())`. `main` returns a `Result`, so `?` compiles. No C sample for `score`. No C++ sample for `score`. No Objective-C sample for `score`. No COBOL sample for `score`. No Ada sample for `score`. No Java sample for `score`. No Kotlin sample for `score`. No Scala sample for `score`. No C# sample for `score`. No Go sample for `score`. No Swift sample for `score`. No Zig sample for `score`. No PHP sample for `score`. No Dart sample for `score`. ``` LOAD './thinkthen.duckdb_extension'; SELECT body, thinkthen_score( 'How urgent is this?', body, ['Routine.', 'Soon.', 'Immediate.'] ) AS urgency FROM (VALUES ('Please update my mailing address when you can.'), ('Can you send the signed contract by Friday?'), ('Nobody can log in to the site right now.') ) t(body); ``` *What DuckDB printed* ``` body|urgency Please update my mailing address when you can.|0.06 Can you send the signed contract by Friday?|0.99 Nobody can log in to the site right now.|2.0 ``` ``` .load ./thinkthen WITH t(body) AS (VALUES ('Please update my mailing address when you can.'), ('Can you send the signed contract by Friday?'), ('Nobody can log in to the site right now.')) SELECT thinkthen_score( '{"score": "How urgent is this?", "levels": ["Routine.", "Soon.", "Immediate."]}', body) AS urgency, body FROM t; ``` *What SQLite printed* ``` 0.06|Please update my mailing address when you can. 0.99|Can you send the signed contract by Friday? 2.0|Nobody can log in to the site right now. ``` ``` SELECT body, thinkthen_score( '{"score": "How urgent is this?", "levels": ["Routine.", "Soon.", "Immediate."]}', body, NULL) AS urgency FROM (VALUES ('Please update my mailing address when you can.'), ('Can you send the signed contract by Friday?'), ('Nobody can log in to the site right now.') ) AS t(body); ``` *What PostgreSQL printed* ``` body | urgency ------------------------------------------------+--------- Please update my mailing address when you can. | 0.06 Can you send the signed contract by Friday? | 0.99 Nobody can log in to the site right now. | 2 (3 rows) ``` [Arguments, options, and more examples](/reference/functions/score/) ## Jobs that use `score` - [Sort survey answers by mood and problem](/how-tos/code-open-ended-survey-answers/): for survey and market researchers Watch `score` answer questions about Beatles songs: [score can rate on a linear scale you define.](/learn/beatles-bench/score/) On GitHub: [github.com/botassembly/thinkthen](https://github.com/botassembly/thinkthen) --- Source: https://thinkthen.dev/functions/tag/ # `tag`: name every label that fits. *The message praises the dashboard, reports a crash, and names a double charge. It gets praise, bug, and billing.* ``` question="Which labels fit this message?" labels=( praise bug billing ) cat <<'EOF' | Love the new dashboard, but export crashes the app, and I was charged twice. EOF thinkthen tag "$question" "${labels[@]}" ``` *Output* ``` ["praise","bug","billing"] ``` *exit 0: the run finished* You give it one question, one piece of evidence, and 1 to 20 labels. You get back the labels that fit, as a list. ## Read the answer A label under the threshold is left out. An empty list is a good answer and exits 0. | Exit code | What it means | | --- | --- | | 0 | the run finished | | 2 | usage or input error | | 4 | the backend failed or refused, as it does for evidence over the size limit | | 5 | a local failure | | 70 | a defect in the tool | ## Call it from your language The Bash example opens this page. ``` import thinkthen as tt question = "Which labels fit this message?" labels = ["praise", "bug", "billing"] message = ( "Love the new dashboard, but export crashes the app,\n" "and I was charged twice.\n" ) fitting_labels = tt.tag( question, message, labels=labels ).value assert fitting_labels == ["praise", "bug", "billing"] ``` No Polars sample for `tag`. No pandas sample for `tag`. ``` import assert from "node:assert/strict"; import * as tt from "thinkthen"; const question = "Which labels fit this message?"; const labels = ["praise", "bug", "billing"]; const message = "Love the new dashboard, but export crashes the app,\n" + "and I was charged twice.\n"; const fittingLabels = (await tt.tag( question, message, { labels }, )).value; assert.deepEqual(fittingLabels, [ "praise", "bug", "billing", ]); ``` No Ruby sample for `tag`. ``` library(thinkthen) question <- "Which labels fit this message?" labels <- c("praise", "bug", "billing") message <- paste0( "Love the new dashboard, but export crashes the app,\n", "and I was charged twice.\n" ) fitting_labels <- tt_tag( question, message, labels )$value[[1]] stopifnot(identical( fitting_labels, c("praise", "bug", "billing") )) ``` ``` use thinkthen::{Engine, Question}; thinkthen::choices! { enum Label { Praise => "praise", Bug => "bug", Billing => "billing", } } let tt = Engine::from_env()?; let question = "Which labels fit this message?"; let labels = Question::tag::