---
title: "The zot command line"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{The zot command line}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

The `zot` command line operates this package without writing R. It parses
arguments, calls one exported function and maps the result and the package's
condition classes to standard output and an exit code. Every rule it applies
belongs to the package, so the same operation is available from R.

The command is a separate component: it lives in the product repository under
`cli/` and is installed from an authorized checkout together with this
package. Installing the package from an archive does not install it, and this
article describes the interface rather than shipping it.

## The three lanes

| Lane | Verbs | Effect on a library |
| --- | --- | --- |
| Read | `doctor`, `libraries`, `census`, `collections`, `items`, `get`, `local items\|query\|records\|files`, `identity twins\|match` | None |
| Build | `clean name\|title\|payload`, `payload template\|check\|crossref`, `plan retitle\|refile\|move-create\|move-trash\|attach`, `preview` | None; plan and preview verbs write local batch files |
| Mutation | `approve`, `execute`, `verify` | Only `execute` writes, and only under an approved preview |

`zot --help` lists them with one line each, and `zot VERB --help` prints that
verb's flags. Use the full two-word name for a subverb, for example
`zot plan retitle --help`. `--json` prints an operation result as one document
on standard output; diagnostics go to standard error. Help and version output
remain text.

## Local paths and result files

The CLI has no `--root` flag and does not discover a project directory.
Relative input and output file paths resolve from the **working directory
of the command**, including `--sql-file`, `--records`, `--candidates`,
`--items`, `--payload`, `--plan`, `--preview`, `--approval`, `--ledger` and
`--out`. Selecting or moving a JSON file does not change that base. Absolute
paths keep their meaning. `--library` selects a Zotero library, not a local
directory. A `plan attach` row stores each file's `path` as given, so a
relative path is resolved again from the working directory of `execute`.

For `local` verbs, `--db` selects a SQLite file; the default is
`~/Zotero/zotero.sqlite`. Explicit relative database paths also use the
working directory. `doctor` probes this default and has no `--db` override.

Only the five `plan` verbs, `preview` and `approve` accept `--out`. They
require a new destination in an existing directory, then write one JSON
batch document there. The writer uses `<destination>.tmp` beside the
destination and renames it into place. That companion path is part of the
write; use a separate destination for each concurrent batch writer.

`execute --ledger` is different: it creates or updates the ledger at the
exact path supplied, with a sibling `.tmp` during each replacement. The
parent directory must already exist. An existing ledger is reusable for the
same preview digest; a different digest is refused. `plan move-trash` and
`verify` only read their `--ledger` file. No command recursively clears a
project output directory.

All verbs also report a result on stdout. With `--json`, that result is an
envelope containing `ok`, `verb`, `data` and `meta`, or an `error` on an
exception. It is **not a batch file** and is not automatically accepted as
an input record array. `--items` for `plan retitle` and `--records` for
`identity twins`, the move plans and `plan attach` take JSON arrays of records
with the fields described below; `identity match` takes arrays of candidate and
reference records. Prepare those arrays from the result data before passing
them back. Payload inputs take the payload object itself. Shell redirection
such as `> result.json` is controlled by the shell and has none of the
command's new-`--out` checks.

## Inputs, results and effects by verb

The table covers the 27 verbs in the command's own verb table. Optional selectors
and limits are listed by each verb's help. Except for the batch and ledger
destinations shown here, results go to stdout; the command does not save a
second report automatically.

| Verb | Input or source | Result and destination |
| --- | --- | --- |
| `doctor` | Installed package, environment, one Web API read when credentials are available, and the local database. | Availability table. Opens SQLite read-only. |
| `libraries` | Configured user's accessible libraries via the Web API. | Personal/group identifiers, names and library versions. |
| `census` | `--library` and optional repeated `--collection NAME=KEY`; API reads. | Top-level, Trash and named-collection counts. |
| `collections` | `--library`; API reads. | Collection identifiers, hierarchy paths, item counts and versions. |
| `items` | `--library`, repeated `--key` or listing selectors/filters. No selector lists the library. | Items or versions, with listing metadata or missing keys. A listing returns one page by default; `--all` requires `--max`. |
| `get` | Positional `/SUBPATH`, `--library`, optional repeated `--query NAME=VALUE`. | GET status, headers and body under the library's API prefix. This path is not a file system path. |
| `local items`, `local records` | SQLite `--db`, optionally `--library`; without a library selector these read all local libraries. | Non-trashed items or bibliographic identity records. No database copy or update. |
| `local query` | Exactly one of `--sql` or `--sql-file`, optional positional `--param` values, SQLite `--db`. | Query rows through a read-only connection; supply a read-only SQL statement. |
| `local files` | SQLite `--db`, optional repeated `--key` and `--md5`. | Attachment metadata and existing local paths. Imported files resolve under `storage/<key>/` beside the selected database; supported absolute linked paths remain as stored. It does not download, move or copy attachments. |
| `identity twins` | `--records`: array with item keys, titles and identity evidence. | Duplicate-candidate pairs and their identity verdicts. |
| `identity match` | `--candidates` and `--records`: arrays of API records or identity objects. | Candidate/reference pairs and evidence verdicts. |
| `clean name`, `clean title` | Positional text. | Cleaned text. A filename argument is treated as text; no file is renamed. |
| `clean payload` | `--payload`: JSON payload object. | Cleaned payload on stdout; the input file is preserved. |
| `payload template` | `--type`; Zotero template API reads. | Allowed fields, creator types and empty template. |
| `payload check` | `--payload`: JSON object; template API reads. | Validation result or a refusal with the problems found; no item is created. |
| `payload crossref` | Positional DOI; Crossref lookup, without Zotero credentials. | Sanitized create payload on stdout; no item is created. |
| `plan retitle` | `--items`: array with `itemKey`, `library`, `title`. | Title patches using the fixed title cleaner, saved as a plan at new `--out`; also reports the plan on stdout. |
| `plan refile` | `--library`, repeated `--key`, `--add-to`; reads current API memberships. | Collection-membership patches at new `--out`, skipping absent items and memberships already present. |
| `plan move-create` | `--records`: array with `itemKey`, `hasFile`, `payload`; `--to-library` and `--collection`. | Create plan at new `--out`. Records carrying files are refused; this does not transfer attachments. |
| `plan move-trash` | Original `--records` in the same order as the create plan, its `--ledger`, and `--from-library`. | Trash plan at new `--out` for originals whose corresponding `create-<row>` ledger entry is `done`. No remote Trash action occurs here. |
| `plan attach` | `--records`: array with `path` and `parentKey` (`null` for a standalone attachment) and optional `filename`, `contentType`, `charset`; `--library`; reads the children of each parent. | Attach plan at new `--out`: one row per file with its MD5, size, cleaned filename and content type, omitting files whose MD5 is already attached to the same parent. A missing or empty file is refused. No item is created and no byte is uploaded here. |
| `preview` | `--plan` batch file and `--describe`. | Validated preview with digest, counts and rows at new `--out`; stdout reports its summary. No API request. |
| `approve` | `--preview`, the displayed `--digest`, and `--notes`. | Approval record at new `--out`; stdout reports digest, notes and time. It records authorization supplied by the caller; it does not authenticate the person granting approval or execute the batch. |
| `execute` | `--preview`, matching `--approval`, and `--ledger`; configured API access. | Applies approved patch/create/attach/Trash rows remotely, updating the local ledger before and after each attempt. An attach row creates the attachment item and uploads its file to Zotero storage, which can consume quota. Stdout reports counts, ledger path and unfinished rows. |
| `verify` | `--preview`, `--ledger`, optional `--full`; API reads. | Counts and read-back verdicts on stdout. Does not rewrite the ledger or modify Zotero. Default verification samples up to ten completed patch rows; `--full` checks every completed row, including create, attach and Trash; an attach row is confirmed by its MD5, filename, content type and parent. |

The plan, preview and approval files use the package's `kind`-tagged JSON
batch format. A new ledger contains `schema = 2`, `digest` and `items`; it is
maintained by execution. Existing items use `library:key` identities, while
creates and attachments keep `create-<row>`. The separate `--out` files let the owner inspect each stage
before authorizing execution. Changing an output path and rerunning a plan
or preview rebuilds it from its current inputs.

## Exit codes

| Code | Meaning |
| --- | --- |
| 0 | Success. Zero results, or a 404 from `get`, is a verdict rather than an error |
| 1 | Operational failure: an exhausted transport, an unavailable plane |
| 2 | Usage: an unknown verb or flag, a missing value, an input file that is not there |
| 3 | Refusal-class condition: configuration, plan, approval, bound or unresolved-ledger error; see the effect limits below |
| 4 | Incomplete: a batch whose rows are not all done, or a verification that did not confirm |

An exit code classifies the result; it does not by itself establish whether
remote effects occurred. Validation of credentials, plan and approval before
execution can refuse without a write. However, a failure to rename the
ledger after an API write is also a plan-class error mapped to code 3. The
phrase "refused without effects" in older help is therefore not a blanket
guarantee. Current help states that codes do not establish write effects.
A previous `attempt` can also mean an unresolved remote outcome.
Inspect the reported cause, ledger and current library state before resuming.

Code 4 means incomplete execution or verification, not proof of a successful
remote write: `verify` performs reads only, and an execution can record failed
rows. A JSON envelope with `ok: true` means the runner returned a result;
`doctor`, `execute` and `verify` can return that envelope with a nonzero exit
code. Check the exit code and the result's statuses together.

## From a plan to a verified batch

The mutation lane is the guarded flow of `vignette("guarded-workflow")` with
each step in its own process, so the owner can read and approve the exact
rows between them:

```sh
zot plan retitle --items items.json --out plan.json
zot preview --plan plan.json --describe "Normalize reviewed titles" --out preview.json
# show preview.json to the owner, then retype the digest it reports
zot approve --preview preview.json --digest <sha256> \
            --notes "owner approved the displayed batch" --out approval.json
zot execute --preview preview.json --approval approval.json --ledger ledger.json
zot verify --preview preview.json --ledger ledger.json --full
```

`approve` reads the preview under the digest given on the command line: a
preview whose rows changed since it was displayed cannot be approved by
mistake. `execute` recalculates that digest again before its first request,
resumes an interrupted run through the same ledger path, and stops on a row
whose outcome is unknown. `verify` re-reads what the ledger recorded.

The batch is not a transaction. Execution records `attempt` before each
row, then its result; completed rows are skipped on resume. A retained
`attempt` stops execution for reconciliation. A create saves its returned key
as `created` before reading it back; resuming that entry only confirms the key,
without another POST. An attach row saves the new attachment key as `created`
before uploading; a failed upload leaves it `created` with its error, and
resuming repeats the upload on that key without creating a second item. A
file whose MD5 changed since approval is refused before any request and
recorded as `failed`. Other unfinished statuses are attempted again. A failed
read-back can follow a successful remote write,
so an unfinished row is not evidence that nothing happened. Earlier rows are
not rolled back when a later row fails. Verification does not repair the
ledger or authorize a retry.

Legacy ledgers without a schema keep their original key-only format when each
id belongs to exactly one plan row. A repeated key across libraries makes a
legacy ledger ambiguous and is refused without migration. Legacy failed
creates require reconciliation because the old executor could have lost their
returned key. Verification counts absent plan rows as pending and lists them
in `unfinished`; they cannot yield exit 0.

A POST server failure or a 200 response without an interpretable row result
has an unknown effect: execution records `attempt` and reusing the ledger
cannot repeat the POST. The transport retries POST only for explicit rate
limiting (429); GET and conditional PATCH retain their bounded server retries.

The files are the package's own batch documents, so R can read and write the
same ones:

```{r}
library(data.table)
library(zot)

Items <- data.table(
  itemKey = c("A", "B"),
  library = "user",
  title = c("Reviewed book (1).pdf", "Existing title")
)
Plan <- ztRetitle(Items, transform = ztCleanTitle)

PlanFile <- tempfile(fileext = ".json")
zotWriteBatch(Plan, PlanFile)
Preview <- zotPreview(zotReadBatch(PlanFile), describe = "Normalize reviewed titles")
Preview$digest
```

A preview read back from disk recalculates its digest, so an edited file is
refused instead of executed under the earlier approval:

```{r, error = TRUE}
PreviewFile <- tempfile(fileext = ".json")
zotWriteBatch(Preview, PreviewFile)

Document <- jsonlite::read_json(PreviewFile, simplifyVector = FALSE)
Document$rows[[1]]$payload$title <- "Something else"
jsonlite::write_json(Document, PreviewFile, auto_unbox = TRUE, null = "null")

zotReadBatch(PreviewFile)
```

```{r, include = FALSE}
unlink(c(PlanFile, PreviewFile))
```

## Credentials and planes

The command reads `ZOTERO_LIBRARY_ID` and `ZOTERO_API_KEY` from its process
environment, which R fills from `~/.Renviron`; there is no flag for the key,
because command lines are visible to other processes. `zot doctor` reports
the package, the credentials, the Web API and the local database, naming the
missing variable or path instead of guessing. Local reads are an advisory
plane: the Web API remains the truth a write is verified against.