filter
filter asks one yes-or-no question about every line of its input and prints the lines that answer yes, exactly as they came in. It drops the lines that answer no, and, when you set a low and a high bar, the lines it is not sure about.
thinkthen filter \ 'Does the customer request a refund?' messages.txtOutput
Please refund my order.
I was charged twice for one invoice.
Please refund my order.Where is my parcel?I was charged twice for one invoice.The refund request and the double charge answer yes, so filter prints them. The parcel question answers no, so filter drops it.
Not sure
Section titled “Not sure”Set a low bar and a high bar. filter keeps a line at or above the high bar, and drops it below the low bar or in between, where it is not sure.
thinkthen filter --low 0.2 --high 0.8 \ 'Does the customer request a refund?' messages.txtOutput
Please refund my order.
Only the refund request is kept. The double charge falls between the bars, so filter is not sure and drops it. Send not sure to a person. Never treat it as a no.
See why
Section titled “See why”See why
--json prints one line of details for every input line, kept or not. This jq keeps three of them.
thinkthen filter --low 0.2 --high 0.8 --json \ 'Does the customer request a refund?' messages.txt | jq -c '{item, status, kept}'Output
{"item":"Please refund my order.","status":"answered","kept":true}
{"item":"Where is my parcel?","status":"answered","kept":false}
{"item":"I was charged twice for one invoice.","status":"unsure","kept":false}
status says whether filter answered or was not sure, and kept says whether it printed the line. JSON output lists every field, including probability, the chance of yes.
Move the bar
Section titled “Move the bar”Move the bar
With one bar, --threshold, filter keeps a line at or above it and drops the rest. Raise the bar to 0.8 and the double charge, at 0.7, is dropped.
thinkthen filter --threshold 0.8 \ 'Does the customer request a refund?' messages.txtOutput
Please refund my order.
The default bar is 0.5. Lower it to keep more lines. Raise it to keep only the clear ones.
Use the answer in a script
Section titled “Use the answer in a script”Use the answer in a script
--check turns the answer into the exit code: 0 when filter keeps at least one line.
if thinkthen filter --check \ 'Does the customer request a refund?' \ messages.txt > /dev/nullthen echo 'refund requests are waiting'else status=$? case "$status" in 1|3) ;; *) exit "$status" ;; esacfiOutput
refund requests are waiting
| Answer | Exit code with --check |
|---|---|
| at least one line kept | 0 |
| none kept, none not sure | 1 |
| none kept, at least one not sure | 3 |
Without --check, filter exits 0 whenever it answers. Exit codes lists every code.
Keep the yes rows of a CSV file
Section titled “Keep the yes rows of a CSV file”Keep the yes rows of a CSV file
id,texta1,Please refund my order.a2,Where is my parcel?a3,I was charged twice for one invoice.thinkthen filter 'Does the customer request a refund?' \ --from csv --column text --into is_refund < messages.csvOutput
id,text,is_refund
a1,Please refund my order.,TRUE
a3,I was charged twice for one invoice.,TRUE
--column names the column to read, and --into names the new answer column. filter writes only the rows it keeps. With --in-place, a row that fails leaves your file as it was: filter exits 6 and says how many rows failed, so no row is lost. File types covers TSV, Excel and other files.
Options
Section titled “Options”Options
| Option | What it does |
|---|---|
<QUESTION> | Question to ask |
[FILE]... | Input files; default: standard input |
--name <NAME> | Optional execution identity |
--text <TEXT> | Judge this text instead of reading input |
--threshold <THRESHOLD> | Required probability |
--low <LOW> | Low probability bar |
--high <HIGH> | High probability bar |
--check [=<CHECK>] | Return the verdict as an exit code |
--true <TEXT> | Meaning of a true answer |
--false <TEXT> | Meaning of a false answer |
--column <COLUMN> | Field column; repeat; replaces a saved pointer |
--details [=<DETAILS>] | Add status, probability, error, detail and source columns |
--in-place [=<IN-PLACE>] | Replace source with backup |
--into <INTO> | Answer field for JSON Lines; column for tables |
--output <OUTPUT> | Destination path |
--sheet <SHEET> | Worksheet |
If filter can't answer a line, it prints an error and exits with code 2, 4, 5 or 6. Troubleshooting lists each error and its fix.
In code: Python · TypeScript · Go · Java · Kotlin · C# · R · Ruby · Rust · Swift · Zig · C · C++ · DuckDB · SQLite · PostgreSQL
Next: tag gives a text every label that fits.