Settings
This page lists every setting ThinkThen takes, with its default, the values it allows, and how to set it on each surface.
The page comes from the settings table in the product's specification.
What counts as a setting
A setting is a value a caller chooses that changes how the tool reads, asks, answers, stores, or reports. Every flag in the command's help is one. The question text, the records, and help itself are not.
Which value wins
The typed value wins, then the environment, then the question file, then the configuration file, then the built-in default. A setting skips the tiers it has no home in.
Only a per-call value counts as typed. That is a command flag, or an argument on one library or SQL call, such as a question's threshold or a call's deadline. An engine-level library setting, such as EngineBuilder::throttle or Python's Engine(throttle=), sits in the environment tier. So does a SQL session setting, such as DuckDB's SET thinkthen_throttle or SQLite's thinkthen_configure(json). Each is set once for a process or a session, as a variable is.
The three SQL extensions offer thinkthen_try_details. It returns a row's detailed result, or a JSON failure for a row whose failure lets the query go on to later rows. Cancellation and deadlines still raise errors. SQLite's thinkthen_budget_ms(n) sets one time budget for the connection across ThinkThen calls: -1 clears it, 0 means it is spent, and a positive number of milliseconds bounds the calls that follow until it is reset. PostgreSQL bounds its query with its own statement_timeout and each call with thinkthen.deadline_ms. DuckDB's SET thinkthen_query_budget_ms bounds a whole query on Linux x86-64, Linux ARM64, Apple Silicon and Intel macOS.
The orders on record follow the rule:
- The address:
--url, thenTHINKTHEN_BASE_URL, then the configuration file'surl, then the built-in address. - The backend and the address, together: typed
--backendand--url, thenEngineBuilder::backendandEngineBuilder::base_url, thenTHINKTHEN_BACKENDandTHINKTHEN_BASE_URL, then the configuration file'sbackendandurl. The first tier that names either decides. For these two settings the engine setting sits above the environment variables, so the environment tier splits in two. On the named path the backend supplies the key variables and, after--model, the question file'smodel, and the engine'smodel, the model. - The model:
--model, then the question file'smodel, then the engine'smodel, then the configuration file'smodel, thenjev-1.13.0. For example, a saved file with"model":"saved-1"selectssaved-1over a configuration file with"model":"configured-1";--model typed-1selectstyped-1over both. The question-file page shows the complete example. - The answer cache:
--cache DIRselects that folder and--no-cacheturns caching off for the command. Otherwise a nonblankTHINKTHEN_CACHEnames a folder and enables caching, even when the configuration file says"cache": false. Without a named folder, the configuration file's booleancacheturns the default answer cache folder on (true, omitted, ornull) or off (false). The configuration file never names a cache folder. - The batch:
--batch, thenTHINKTHEN_BATCH, then the question file'sbatch, thenmax. - A threshold-bearing question file names its tuned batch setting through
batch, or 1 when it has nobatch. The setting still resolves by the four tiers above; another running setting warns instead of changing it. - The request size:
--max-request-bytes, thenTHINKTHEN_MAX_REQUEST_BYTES, then 96,000 bytes.
--profile FILE loads a backend profile and has no environment or file tier. The question file's profile is calibration identity, a separate setting with its own row.
Cache forms differ by surface. EngineBuilder::from_env() reads the configuration and THINKTHEN_CACHE; explicit Rust cache_at(folder) and no_cache() override that choice, while a bare Engine::builder() does not read the configuration. Python accepts a folder, True for the default folder, or False for off; TypeScript and C accept a folder or false; Ruby accepts a folder or false (nil keeps the default); R accepts a folder or FALSE (NULL keeps the default). DuckDB's session setting names a folder. PostgreSQL accepts an absolute folder or off; RESET keeps THINKTHEN_CACHE, or no cache when it is unset. SQLite's thinkthen_configure takes "cache" as a folder, or false for off. These library and SQL settings do not change the configuration file's boolean type.
The configuration file is thinkthen.config/1 under the platform configuration home. The tool reads it and never writes it. The file names the address, the model, and the backends with the names of their key variables, so whoever can write it decides where the key and the evidence go. It may also set requests_per_minute on any backend, built-in or added, as the Requests a minute row says. On Unix, the command warns when another user owns the file or any user may write it. It still reads the file, and prints thinkthen: the configuration file is writable by another user; it decides where the key and evidence go on standard error before it runs. The libraries read the file the same way and print nothing. A file that draws this warning may not hold backends, and such a file is refused with exit 5. A file owned by the current user that only its group may write, as the usual 002 umask leaves it, gets no warning. Keep it writable by its owner alone. The cache folder and the configuration file follow XDG_CACHE_HOME, XDG_CONFIG_HOME and HOME. The usage totals follow XDG_STATE_HOME and HOME, as recording.md states. The cache folder never moves them.
question="Does the customer ask for a refund?"
export THINKTHEN_BASE_URL="http://localhost:8080/v1"
printf '%s\n' "I want to send this back." |
thinkthen decide "$question" --plan |
head -1 |
jq .url"http://localhost:8080/v1/systemone"
question="Does the customer ask for a refund?"
export THINKTHEN_BASE_URL="http://localhost:8080/v1"
printf '%s\n' "I want to send this back." |
thinkthen decide "$question" \
--url https://api.typesafe.ai/v1 \
--plan |
head -1 |
jq .url"https://api.typesafe.ai/v1/systemone"
Every setting
Each setting below lists only the surfaces it reaches, with the spelling each one uses.
"In the question JSON" means the key of the question file, given inline as JSON or as a file. The Polars and pandas columns take the Python or Rust library's spelling, so they have no column of their own. Library usage results report retries beside requests_sent. retries counts the requests sent again after a retried status, so it never exceeds requests_sent. SQL usage output has no retries.
- Threshold
- Relation threshold
- What true and false mean
- Options
- Labels
- Levels
- Kinds
- Recognize text limit
- Relation rules
- Kind pointer
- None option
- Top
- Input file
- Framing
- Evidence pointer
- Run facts
- Quiet and raw
- Plan preview
- Portable call settings
- Address
- Key
- Private TLS roots
- Backend
- Named backends
- Backend key
- Model
- Backend profile
- Calibration identity
- Timeout
- Retries
- Throttle
- Requests a minute
- Batch
- Context
- Request size
- Request limit
- Process request total
- Estimated input admission total
- Caller prices for estimated cost
- Deadline and cancel
- Relate query limits
- File directory
- Answer cache
- Recording
- Prune target
- Prune preview
- Unused entry report
- Convert quoted form
- Prune selectors
- Audit grouping and keys
- Audit rescore rule
- Audit split seed
- Audit agreement target
- Audit optimize measure
- Audit write
- Audit output destination
- Audit case evidence
- Audit and diff output
- Diff comparison
- Status format
Threshold
One inclusive cut on the quantity each verb names in threshold.md, or a decide band LOW:HIGH. recognize cuts printed name strength, which is not itself a probability.
- Default
- 0.5 on
decide,tag,filter,recognizeandrelate; none onchoose,score,rankandfind - Allowed values
- A cut above 0 and at most 1; a
decideband has 0 ≤ LOW < HIGH ≤ 1;choosemay take an optional cut;score,rank, andfindrefuse one - Command flag
--threshold- Question-file key
threshold- Rust
cut,cut_at,bandon the question builder;thresholdon recognize and relate- Python
threshold=onquestion,recognize,relate- TypeScript
thresholdin the question- Ruby
threshold:onrecognize,relate;thresholdin the question- R
threshold =on the verbs- C
thresholdin the question JSON- DuckDB
thresholdin the question JSON- PostgreSQL
thresholdin the question JSON- SQLite
thresholdin the question JSON
Relation threshold
The cut at or above which recognize keeps a relation edge.
- Default
- 0.5
- Allowed values
- Above 0 and at most 1
- Command flag
--relation-threshold- Question-file key
relation_thresholdin a recognize question file- Rust
relation_thresholdon the recognize builder- Python
relation_threshold=- TypeScript
relationThreshold- Ruby
relation_threshold:- R
relation_threshold =- C
relation_thresholdin the question JSON- DuckDB
relation_thresholdin the relations file- PostgreSQL
relation_thresholdin the relations file- SQLite
relation_thresholdin the relations file
What true and false mean
Text that says what a yes and a no mean, sent with a decide question.
- Default
- No text
- Allowed values
- Text that is not blank
- Command flag
--true,--false- Question-file key
true,false- Rust
yes,noon the question builder- Python
true_=,false_=onquestion- Ruby
true,falsein the question- R
true,falsein the question- C
true,falsein the question JSON- DuckDB
true,falsein the question JSON- PostgreSQL
true,falsein the question JSON- SQLite
true,falsein the question JSON
Options
The labels choose picks from, each with an optional description. --options POINTER takes them from each JSON record.
- Default
- None
- Allowed values
- 2 to 255 distinct labels
- Command flag
--option,--options- Question-file key
options- Rust
optionon the question builder- Python
options=onquestionorchoose- TypeScript
optionsin the question- Ruby
optionsin the question- R
optionsin the question- C
optionsin the question JSON- DuckDB
- the
optionslist argument - PostgreSQL
- the
options text[]argument - SQLite
optionsin the question JSON
Labels
The labels tag may name, each with an optional description.
- Default
- None
- Allowed values
- 1 to 20 distinct labels
- Command flag
--label- Question-file key
labels- Rust
labelon the question builder- Python
labels=onquestionortag- TypeScript
labelsin the question- Ruby
labelsin the question- R
labelsin the question- C
labelsin the question JSON- DuckDB
- the
labelslist argument - PostgreSQL
- the
labels text[]argument - SQLite
labelsin the question JSON
Levels
The scale score places a text on, lowest first.
- Default
- None
- Allowed values
- 2 to 10 distinct levels
- Question-file key
levels- Rust
levelon the question builder- Python
levels=onquestionorscore- TypeScript
levelsin the question- Ruby
levelsin the question- R
levelsin the question- C
levelsin the question JSON- DuckDB
- the
levelslist argument - PostgreSQL
- the
levels text[]argument - SQLite
levelsin the question JSON
Kinds
The kinds of name recognize looks for, each with an optional description.
- Default
- None, and every name has the kind
ENTITY - Allowed values
- 0 to 20 kind words, or
KIND=DESCRIPTION; notnone of these,ENTITYorANYin any ASCII case - Command flag
--kind- Question-file key
recognize.kinds- Rust
- the recognize builder's kinds
- Python
kinds=- TypeScript
kinds- Ruby
kinds:- R
kinds =- C
kindsin the question JSON- DuckDB
- the
kindsargument ofthinkthen_recognize - PostgreSQL
- the kinds argument of
thinkthen_recognize - SQLite
- the kinds argument of
thinkthen_recognize
Recognize text limit
The largest text recognize takes, in UTF-8 bytes. A longer text exits 2 before any request.
- Default
- 600,000
- Allowed values
- 1 to 2^53 - 1
- Command flag
--max-text-bytes
Relation rules
Named, directed relations between kinds that recognize and relate ask about. --either makes every inline rule hold both ways. The either field defaults to false.
- Default
- None
- Allowed values
NAME=SOURCE:TARGET, or bareNAMEfor any kinds;*orANYfor any kind- Command flag
--relation,--either- Question-file key
recognize.relations,relate.relations; each rule carries its own either member- Rust
- the recognize and relate builders' relations
- Python
relations=,either=- TypeScript
relations,either- Ruby
relations:- R
relations =- C
relationsin the question JSON- DuckDB
- the relations file of
thinkthen_relations; the rules ofthinkthen_relate - PostgreSQL
- the relations file of
thinkthen_relations - SQLite
- the relations file of
thinkthen_relations
Kind pointer
The JSON pointer to each entity's kind in relate's records.
- Default
/kind- Allowed values
- An RFC 6901 pointer
- Command flag
--kind-field- Question-file key
relate.fields- DuckDB
- the
kindcolumn of the relate query
None option
Lets find answer that no unit fits.
- Default
- Off
- Allowed values
- On or off
- Command flag
--none- DuckDB
{"none":true}inthinkthen_findsettings; omitted is false- PostgreSQL
{"none":true}inthinkthen_findsettings; omitted is false- SQLite
{"none":true}inthinkthen_findsettings; omitted is false
Top
How many of the most likely records rank prints.
- Default
- Every record
- Allowed values
- A whole number of 1 or more
- Command flag
--top- Python
top=onrank- DuckDB
LIMIT- PostgreSQL
LIMIT- SQLite
LIMIT
Input file
Reads the records or the document from a file in place of standard input.
- Default
- Standard input
- Allowed values
- A readable file
- Command flag
--input
Framing
How the input splits into records: text lines, JSON lines, or a table with a header row.
- Default
- One document; lines on
filter,rankandfind; JSON lines when a pointer is set - Allowed values
- One of the four, or none
- Command flag
--lines,--jsonl,--csv,--tsv- Python
- a list or a frame column
- TypeScript
- a list
- Ruby
- a list
- R
- a vector
- C
- the
_manycalls - DuckDB
- one row per record
- PostgreSQL
- one row per record, a decide record array, or one ordered
text[]set for find;array_agg(unit ORDER BY ordinal)preserves positions and duplicates - SQLite
- one row per record
Evidence pointer
Sends only the part of each record a JSON pointer names. Several pointers send an object of the parts.
- Default
- The whole record
- Allowed values
- RFC 6901 pointers
- Command flag
--field- Question-file key
on- Python
on=onrecognizeandannotate
Run facts
Prints a final thinkthen.run/1 line on standard error with counts and, on failure, a stable stop cause.
- Default
- Off
- Allowed values
- On or off
- Command flag
--factson asking verbs- Rust
- Rust
Call::facts, terminalBatch::facts, and startedError::facts; accessors, not a run line - Python
Call.value,Call.factsand immutableCall.detailson every verb; a started failure carries its facts and details, and a stop by Ctrl-C, the caller's token or a signal handler'sSystemExitcarriescompletiononce the worker has started; it gives the final facts once the worker ends- TypeScript
Call.value, finalCall.factsand ordered immutableCall.detailson each asking verb; started failures retain facts and details, and an early signal rejection carriescompletion.wait()for the same worker- C
- The typed
*_with_factscalls return each call's facts as JSON on decide, decide-many, recognize and relate, and the typed calls without that suffix return the bare value. The JSON door returnsvalueandfacts. After a started call fails,thinkthen_error_facts_jsonreads its facts, and the caller does not free them. No run line.
Quiet and raw
--quiet prints nothing and leaves the answer in the exit code. --raw prints a choose label without quotation marks.
- Default
- Off
- Allowed values
- On or off; one document only for
--quiet - Command flag
--quiet,--raw
Plan preview
Validates the whole input, prints the first prepared request and a second line with whole-input records, planned requests, the exact bytes of the request bodies, and a low and a high estimate of input tokens; sends nothing. An optional configured key is checked for an exact backend-address collision.
- Default
- Off
- Allowed values
- On or off
- Command flag
--plan- Rust
Engine::plan; feature-gatedPolarsEngine::plan_series- TypeScript
tt.plan(),engine.plan()- Ruby
ThinkThen.plan,Engine#plan
Portable call settings
One JSON object of settings for one call. It takes the question-file keys, plus context, batch, deadline_ms and, on find, none. A repeated or unknown key, a key the verb does not take, or a key the question also sets is a usage error.
- Default
- Absent (
{}) - Allowed values
- A
thinkthen.settings/1JSON object holding each key at most once - Rust
Settings::parse, thencheckfor the verb andconflictswith the question- Python
- the question and call keywords follow its rules
- R
- the question and call keywords follow its rules
- C
settingsinthinkthen_plan_json- DuckDB
- the call settings JSON argument
- PostgreSQL
- the call settings JSON argument
- SQLite
- the call settings JSON argument
Address
Where requests go: the backend's base address.
- Default
https://api.typesafe.ai/v1- Allowed values
httpswith a host, orhttponly onlocalhost,127.0.0.1, or[::1]; no user name, query, or fragment. backends.md- Command flag
--url- Environment variable
THINKTHEN_BASE_URL- Configuration file
url- Rust
EngineBuilder::base_url- Python
Engine(base_url=)- TypeScript
new Engine({baseUrl})- Ruby
Engine.new(base_url:)- R
tt_engine(base_url =)- C
"base_url"
Key
The credential sent with each live request, only to the address named. A remote live call needs a key; the three named loopback hosts send without an Authorization header when it is absent.
- Default
- None
- Allowed values
- Text that is not blank
- Environment variable
THINKTHEN_API_KEY- Rust
EngineBuilder::api_keyorTHINKTHEN_API_KEY- Python
THINKTHEN_API_KEY- TypeScript
THINKTHEN_API_KEY- Ruby
THINKTHEN_API_KEY- R
THINKTHEN_API_KEY- C
THINKTHEN_API_KEY- DuckDB
THINKTHEN_API_KEYfrom the host environment- PostgreSQL
THINKTHEN_API_KEYfrom the server environment;thinkthen.api_keyis refused- SQLite
THINKTHEN_API_KEYfrom the host environment
Private TLS roots
Trusts the certificates in a named PEM file in place of the bundled public root certificates, for this engine. Certificate and host name checks still run. An explicit file is validated at engine construction, including replay-only calls. backends.md.
- Default
- Bundled Mozilla roots
- Allowed values
- Absolute local PEM file of at most 2 MiB with 1 to 256 certificate blocks
- Environment variable
THINKTHEN_CA_BUNDLE- Rust
EngineBuilder::ca_bundle- Python
- process environment only; no per-instance option
- TypeScript
- process environment only; no per-instance option
- Ruby
- process environment only; no per-instance option
- R
- process environment only; no per-instance option
- C
- process environment only; no per-instance option
- DuckDB
- process environment only; no SQL setting
- PostgreSQL
- process environment only; no SQL setting
- SQLite
- process environment only; no SQL setting
Backend
A named backend: a base with its own key variables and model.
- Default
- None: the unnamed path, with
THINKTHEN_API_KEY - Allowed values
liquid,ollama,typesafe, or a name the configuration file adds: 1 to 32 lowercase letters, digits, and hyphens. backends.md- Command flag
--backend- Environment variable
THINKTHEN_BACKEND- Configuration file
backend- Rust
EngineBuilder::backend
Named backends
Backends the configuration file adds beside the built-ins.
- Default
- None
- Allowed values
- An object mapping a backend name to
url,key_env, andmodel, and optionallyrequests_per_minute; a built-in's name maps torequests_per_minutealone. Never a key. backends.md - Configuration file
backends- Rust
- read by
EngineBuilder::from_env
Backend key
The key a named backend sends, read from its own variables; the first nonblank one wins. ollama at its loopback base needs none.
- Default
- None
- Allowed values
- Text that is not blank
- Environment variable
TYPESAFE_API_KEYfortypesafe;LIQUIDAI_API_KEY, thenLIQUID_API_KEYforliquid;OLLAMA_API_KEYforollama; the entry'skey_envfor a configured backend- Configuration file
key_envin abackendsentry names the variable, never the key- Rust
EngineBuilder::api_key, or the variableEngineBuilder::from_envcaptured
Model
The model named in each request. The default is a pinned version.
- Default
jev-1.13.0- Allowed values
- A name that is not blank and holds no control character or white space but a plain space. Space around it is dropped
- Command flag
--model- Configuration file
model- Question-file key
model- Rust
EngineBuilder::model,modelon the question builder- Python
Engine(model=),model=onquestion- TypeScript
new Engine({model}),modelin the question- Ruby
Engine.new(model:),modelin the question- R
tt_engine(model =),model =on the verbs- C
"model",modelin the question JSON- DuckDB
modelin the question JSON;SET thinkthen_model- PostgreSQL
thinkthen.model- SQLite
"model"inthinkthen_configureJSON
Backend profile
Loads enforceable backend limits and a calibration name from a file or inline SQL JSON. It never picks an address, model, key or cache. Its max_request_bytes lowers the request size and never raises it.
- Default
- None
- Allowed values
- A readable profile file, or version-one JSON text in SQL
- Command flag
--profile- Rust
EngineBuilder::profile,EngineBuilder::profile_json- Python
Engine(profile=)- TypeScript
new Engine({profile})- Ruby
Engine.new(profile:)- R
tt_engine(profile =)- C
"profile"- DuckDB
SET thinkthen_profilewith JSON text- PostgreSQL
thinkthen.profilewith JSON text- SQLite
"profile"inthinkthen_configureJSON
Calibration identity
Names the backend a question's threshold was tuned on, so a run elsewhere warns.
- Default
- Absent
- Allowed values
- Lowercase letters, digits, hyphens and underscores
- Question-file key
profile- C
profilein the question JSON- DuckDB
profilein the question JSON- PostgreSQL
profilein the question JSON- SQLite
profilein the question JSON
Timeout
Bounds one attempt from connect to last byte, and each retry wait. A server's retry header asking a longer wait than the lesser of this and 60 seconds fails the request at once.
- Default
- 30 seconds
- Allowed values
- Command: 1 to 86400 whole seconds; Rust: nonzero
Durationof at most 86400 seconds; other libraries: 1 to 86400 whole seconds. A library refuses a larger value witha timeout is at most 86400 seconds - Command flag
--timeout- Rust
EngineBuilder::timeout- Python
Engine(timeout=)- TypeScript
new Engine({timeoutSeconds})- Ruby
Engine.new(timeout:)- R
tt_engine(timeout =)- C
"timeout"- DuckDB
SET thinkthen_timeoutin seconds- PostgreSQL
thinkthen.timeoutin seconds- SQLite
"timeout"inthinkthen_configureJSON, in seconds
Retries
How many times a retried status is sent again. A transport failure is never sent again.
- Default
- 3
- Allowed values
- A whole number of 0 or more
- Command flag
--max-retries- Rust
EngineBuilder::max_retries- Python
Engine(max_retries=)- TypeScript
new Engine({maxRetries})- Ruby
Engine.new(max_retries:)- R
tt_engine(max_retries =)- C
"max_retries"- DuckDB
SET thinkthen_max_retries- PostgreSQL
thinkthen.max_retries- SQLite
"max_retries"inthinkthen_configureJSON
Throttle
The most requests in flight at once in one process, including retries. A request waiting to retry after a busy or failing backend status holds no slot. A request already sent when its call stops keeps its place until it ends, within its attempt timeout (the Timeout row), so a later call can wait that long for a place. On the command it acts in record mode, on annotate, and on relate, whose relations and split requests run at once.
- Default
- 4
- Allowed values
- 1 to 32
- Command flag
--jobs- Rust
EngineBuilder::throttle- Python
Engine(throttle=)- TypeScript
new Engine({throttle})- Ruby
Engine.new(throttle:)- R
tt_engine(throttle =)- C
"throttle"- DuckDB
SET thinkthen_throttle- PostgreSQL
thinkthen.throttle- SQLite
"throttle"inthinkthen_configureJSON
Requests a minute
The most HTTP attempts one process starts each minute to one posting address, retries included. The limit holds within one process: every engine and thread in the process shares it, and separate processes each get the full rate, so N processes can send N times it. Each PostgreSQL backend process paces alone, so N connections can send N times the rate. The variable outranks the configuration file's requests_per_minute on the selected backend's backends entry. A built-in's entry may hold only requests_per_minute. With no backend named, a run whose posting address is a built-in's own base takes that built-in's rate, so a rate on typesafe paces the default address; an added entry's rate applies only when it is named. A paced wait holds no throttle place. A cancel ends it, and a slot past the deadline fails at once. Stored and replayed answers never wait.
- Default
- None: no address is paced unless a rate is set
- Allowed values
- A whole number from 1 to 60,000
- Environment variable
THINKTHEN_REQUESTS_PER_MINUTE- Configuration file
requests_per_minutein abackendsentry- Rust
EngineBuilder::from_envreads the variable, and the file's rate for the backend selected by any tier- Python
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - TypeScript
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - Ruby
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - R
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - C
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - DuckDB
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - PostgreSQL
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to - SQLite
- reads the variable, and the file's rate for the backend the file's
backendorTHINKTHEN_BACKENDselects, or for the built-in whose base a run naming no backend posts to
Batch
The most records of a stream one request carries on decide, filter, rank, choose, tag, score and annotate. max fills each request to the backend's limits, and 1 asks one record a request, as before batching. Records that share one request can affect each other's answers. A top-level decide, choose, tag or score question file, or one top-level annotate question set, holds batch. Nested annotate questions cannot hold it. An empty THINKTHEN_BATCH counts as unset. THINKTHEN_BATCH is ignored on unsupported verbs. A file or ambient batch is ignored on one document; typed --batch there is a usage error. The Rust library and the SQL _many calls batch eligible many-record calls.
- Default
max- Allowed values
maxor a whole number of 1 or more- Command flag
--batch- Environment variable
THINKTHEN_BATCH- Question-file key
batch- Rust
EngineBuilder::batch,CallOptions::batch, parsed question-filebatch, andTHINKTHEN_BATCHthroughEngineBuilder::from_env; call > engine/environment > file > max- Python
Engine(batch=)andbatch=on eligible list, Series and frame calls; call > engine/environment > question file > max- TypeScript
new Engine({batch})andbatchondecide_many,choose_many,score_many,tag_many,filter,rankandannotate; call > engine/environment > question file > max- Ruby
Engine.new(batch:)andbatch:on eligible many-record calls; call > engine/environment > question file > max- R
tt_engine(batch =)overEngineBuilder::from_env; per-callbatch =on decide, choose, score, tag, filter, rank and annotate; call > engine/environment > question file > max- C
thinkthen_engine_new_with"batch"orTHINKTHEN_BATCH; eligible C JSON"call":{"batch":N}outranks the engine; typed many calls use the engine default- DuckDB
SET thinkthen_batchon_manycalls, orbatchin the call settings JSON; session >THINKTHEN_BATCH> question file >max- PostgreSQL
SET thinkthen.batchon_manycalls, orbatchin the call settings JSON; session >THINKTHEN_BATCH> question file >max;1keeps singleton wire bytes; ordinary scalar calls still run per SQL row- SQLite
"batch"inthinkthen_configureJSON, orbatchin the call settings JSON; explicit SQL value >THINKTHEN_BATCH> question file >max;1keeps legacy singleton cache identity
Context
Sends one reference text as shared evidence for each eligible record batch on decide, filter, rank, choose, tag and score. The SQL surfaces take it in the call settings JSON. The text bytes set meta.context_sha256 and change the request digest, not the question digest. The command reads a file; Rust borrows text. records.md.
- Default
- None
- Allowed values
- CLI: a readable, nonblank UTF-8 file; Rust: nonblank borrowed text; SQL: nonblank text in the settings JSON
- Command flag
--context FILE- Rust
CallOptions::context(&str)on eligible many-record calls; single-document, find, annotate, recognize and relate refuse it- Python
context=with nonblank text on eligible many-record text and decide-column calls; one document, find, annotate, recognize and relate refuse it- TypeScript
contextwith nonblank text on eligible many-record calls; scalar, find, annotate, recognize and relate refuse it- Ruby
context:nonblank text on eligible many-record calls; scalar, find, annotate, recognize and relate refuse it- R
- literal nonblank
context =on decide, choose, score, tag, filter and rank; annotate, find, recognize, relate and scalar details do not take it - C
- C JSON
"call":{"context":"..."}on eligible text-record arrays; refused on scalar and unsupported routes - DuckDB
contextin the call settings JSON on eligible judgments; NULL settings mean absent- PostgreSQL
contextin the call settings JSON on eligible judgments; SQLNULLsettings mean absent- SQLite
contextin the call settings JSON on eligible judgments; NULL settings mean absent
Request size
The most bytes a batched record request or relation request holds before the tool closes or splits it. A single record or question still goes alone.
- Default
- 96,000 bytes at every address
- Allowed values
- A whole number of at least 1. A blank
THINKTHEN_MAX_REQUEST_BYTEScounts as unset - Command flag
--max-request-bytesondecide,filter,rank,choose,tag,score,annotate,relate,recognize- Environment variable
THINKTHEN_MAX_REQUEST_BYTESon those verbs- Rust
EngineBuilder::max_request_bytes- Python
Engine(max_request_bytes=)- TypeScript
new Engine({maxRequestBytes})- Ruby
Engine.new(max_request_bytes:)- R
tt_engine(max_request_bytes =)- C
"max_request_bytes"- DuckDB
SET thinkthen_max_request_byteson the four C++ target packages- PostgreSQL
thinkthen.max_request_bytes- SQLite
"max_request_bytes"inthinkthen_configureJSON
Request limit
Refuses a call over more records than this.
- Default
- No limit
- Allowed values
- A whole number of 1 or more
- Rust
EngineBuilder::max_requests- Python
Engine(max_requests=)- TypeScript
new Engine({maxRequests})- Ruby
Engine.new(max_requests:)- R
tt_engine(max_requests =)- C
"max_requests"- DuckDB
SET thinkthen_max_requests- PostgreSQL
thinkthen.max_requests- SQLite
"max_requests"inthinkthen_configureJSON
Process request total
Refuses a send once the process has attempted this many sends. Every engine in the process adds to one count, and each engine checks the count against its own limit. Changing or removing one engine's limit does not reset the count.
- Default
- No limit
- Allowed values
- CLI, Rust and C: 0 or more; DuckDB and PostgreSQL: 0 or more; SQLite: 1 or more
- Command flag
--max-requests-totalon asking verbs- Rust
EngineBuilder::max_requests_total- TypeScript
new Engine({maxRequestsTotal})- Ruby
Engine.new(max_requests_total:)- C
"max_requests_total"inthinkthen_engine_new_withJSON- DuckDB
SET thinkthen_max_requests_total- PostgreSQL
thinkthen.max_requests_total- SQLite
"max_requests_total"inthinkthen_configureJSON
Estimated input admission total
Refuses a send once the estimated input tokens of the process would pass this total. Just before each live request goes out, its body counts 0.908 tokens a byte, rounded up. Every engine in the process adds to one total, an engine with no limit included, and each send is checked against its own engine's limit. The estimate bounds what is sent. It is not the backend's token count, and it leaves out output tokens and cost. An explicit flag, setter or C key outranks the variable. The command flag cannot clear a limit the variable sets; unset the variable for no limit.
- Default
- No limit
- Allowed values
- A whole number of 0 or more, or unset
- Command flag
--max-estimated-input-tokens-totalon asking verbs- Environment variable
THINKTHEN_MAX_ESTIMATED_INPUT_TOKENS_TOTAL- Rust
EngineBuilder::max_estimated_input_tokens_total;EngineBuilder::from_envreads the variable- Python
- reads the variable
- TypeScript
- reads the variable
- Ruby
- reads the variable
- R
- reads the variable
- C
"max_estimated_input_tokens_total"inthinkthen_engine_new_withJSON; reads the variable- DuckDB
- reads the variable
- PostgreSQL
- reads the variable
- SQLite
- reads the variable
Caller prices for estimated cost
Price complete reported input and output usage once per call or command; this is no admission limit or provider bill.
- Default
- Off
- Allowed values
- Two decimal strings from 0 through 1000000 with at most six fractional digits; both required
- Configuration file
usd_per_million_input,usd_per_million_outputtogether- Rust
EngineBuilder::prices_usd_per_million- C
- Both keys in
thinkthen_engine_new_withJSON
Deadline and cancel
Bounds a whole eager call in time, or stops it from another thread; a Rust Polars expression starts the deadline again for each chunk of rows it evaluates.
- Default
- No deadline, spelled -1
- Allowed values
- -1 for none, 0 for spent, or a positive time: seconds in Python and R; milliseconds in TypeScript, Ruby, C, and SQL; a Duration or integer milliseconds in Rust
- Rust
CallOptions::deadline_after(Duration), canonicaldeadline_ms(i64)and cancel;deadline_seconds(f64)anddeadline_millis(i64)also work- Python
deadline=,token=- TypeScript
deadlineMs,signal- Ruby
deadline_ms:,cancel:- R
deadline =- C
deadline_ms,THINKTHEN_NO_DEADLINE, a cancel token- DuckDB
deadline_msin the call settings JSON- PostgreSQL
thinkthen.deadline_ms, ordeadline_msin the call settings JSON for one call- SQLite
deadline_msin the call settings JSON
Relate query limits
Bound thinkthen_relate's query in seconds, and refuse a plan that holds too many rows.
- Default
- 60 seconds and 1,000,000 rows
- Allowed values
- 0 or more seconds, 0 for none; a whole number of rows
- DuckDB
SET thinkthen_relate_seconds,SET thinkthen_relate_holding_rows
File directory
The folder PostgreSQL may read question and relations files from; a relative @ name resolves inside it.
- Default
- Empty
- Allowed values
- A folder path
- PostgreSQL
thinkthen.file_directory
Answer cache
Where complete answers are saved and read back. A repeat on a pinned model normally costs no request. A question to jev-latest always asks the backend again and replaces the saved answer, and --refresh-cache does the same for another model. Each entry holds the judged text. --cache DIR keeps the answer cache in DIR. A folder that cannot be written refuses before the call's first send.
- Default
- On, in the default answer cache folder. SQL extensions: off unless
THINKTHEN_CACHEor the SQL setting names a folder - Allowed values
- Configuration
cache:trueorfalse(nullacts as omitted); named-path settings: a folder; off where the surface lists an off form. See Precedence for each surface's forms - Command flag
--cache,--no-cache,--refresh-cache- Environment variable
THINKTHEN_CACHE- Configuration file
cache- Rust
default_cache,cache_at,no_cache;shared_hostdrops the platform folder- Python
Engine(cache=)- TypeScript
new Engine({cache})- Ruby
Engine.new(cache:)- R
tt_engine(cache =)- C
"cache"- DuckDB
SET thinkthen_cachewithoffto disable- PostgreSQL
thinkthen.cachewithoffto disable- SQLite
"cache"inthinkthen_configureJSON: a folder, orfalsefor off
Recording
--record DIR writes each exchange into DIR. --replay DIR answers from DIR alone, with no connection.
- Default
- Off
- Allowed values
- A folder
- Command flag
--record,--replay- Rust
EngineBuilder::record,EngineBuilder::replay- Python
Engine(record=),Engine(replay=)- TypeScript
new Engine({record}),new Engine({replay})- Ruby
Engine.new(record:),Engine.new(replay:)- R
tt_engine(record =),tt_engine(replay =)- C
"record","replay"- DuckDB
SET thinkthen_record,SET thinkthen_replay- PostgreSQL
thinkthen.record,thinkthen.replay- SQLite
"record","replay"inthinkthen_configureJSON
Prune target
The allocated bytes cache prune trims a cache folder to.
- Default
- 100,000,000 bytes
- Allowed values
- A whole number of bytes above 0
- Command flag
--max-size- Configuration file
cache_bytes
Prune preview
Shows the answers prune would remove from thinkthen.sqlite without changing the named folder. Without --dry-run, prune removes them. recording.md.
- Default
- Off
- Allowed values
- On or off
- Command flag
cache prune DIR --dry-run
Unused entry report
Reports stored question keys absent from a complete caller-supplied key list, without removing or creating folder state.
- Default
- Off
- Allowed values
- An existing explicit folder and a UTF-8 file with one lowercase 64-character question key per nonblank line
- Command flag
cache unused DIR --used KEYS
Convert quoted form
Also writes each single-record old exchange in the quoted form, with origin quoted, when cache convert builds thinkthen.jsonl.
- Default
- Off
- Allowed values
- On or off
- Command flag
cache convert DIR --quote
Prune selectors
Remove stored answers older than a duration, or answered by another model, before the size trim.
- Default
- None
- Allowed values
- A duration such as
30d; a model name some reply in the folder names, or any name on an empty folder - Command flag
--older-than,--answered-by-other-than
Audit grouping and keys
How audit groups answers, finds each record's id, and matches a recognize name.
- Default
--by question,--id /id,--match strict- Allowed values
question,verbor a pointer; a pointer;strictoroverlap- Command flag
--by,--id,--match
Audit rescore rule
A rule for rescoring saved answers from their probabilities.
- Default
- As run
- Allowed values
- A cut or band
- Command flag
--threshold
Audit split seed
Seeds the audit split and bootstrap.
- Default
- 0
- Allowed values
- An unsigned 64-bit integer
- Command flag
--seed
Audit agreement target
The agreement a suggested choose cut must reach.
- Default
- 0.9
- Allowed values
- A number from 0 to 1
- Command flag
--target
Audit optimize measure
The measure a yes/no suggested cut maximizes.
- Default
accuracy- Allowed values
accuracy,precision,recall, orf1- Command flag
--optimize
Audit write
Writes a steady suggested bar to the named question file.
- Default
- Off
- Allowed values
- A valid question file or question set path
- Command flag
--write
Audit output destination
With --write QUESTIONS, creates a tuned copy while keeping QUESTIONS unchanged; an existing output is refused.
- Default
- Off
- Allowed values
- A new file path
- Command flag
--write-to
Audit case evidence
Prints one saved case as each JSONL row after the same audit validation; aggregate output remains the default.
- Default
- Off
- Allowed values
- On or off
- Command flag
--cases
Audit and diff output
Adds the coverage curve or a pooled line, or prints a table for a person in place of JSON lines.
- Default
- Off
- Allowed values
- On or off
- Command flag
--curve,--pooled,--table
Diff comparison
What diff compares: an answer key, a cut for each side, the record id, and how names match.
- Default
- No key, no rescore,
--id /id,--match strict - Allowed values
- A JSON lines file or
-; a cut or band; a pointer;strictoroverlap - Command flag
--key,--threshold,--compare-threshold,--id,--match
Status format
Prints status as JSON in place of text.
- Default
- Text
- Allowed values
- On or off
- Command flag
--json
No setting matches.
Configuration · Backends · Reference
On GitHub: github.com/botassembly/thinkthen