Core concepts

The entire mental model is three nouns and three verbs: Stores hold Documents, you search them. Create / ingest / search. Everything else is convenience.

Store

A named, updatable collection of document encodings. Cheap to create — use one per corpus or tenant. No schema.

• Addressable by name or id everywhere: /v1/stores/contracts/search works, and so does the id st_8c2hd0. Nobody should have to look up an id to run a query.
• Store names are 1–64 chars, lowercase alphanumerics plus - and _.

Document

A whole unit of text with a caller-chosen id and optional metadata. Pass whole documents — the engine handles length natively via hierarchical attention.

• No chunking. Ever. Any API that asks you to chunk is a design failure.
• Upsert semantics on id: re-ingesting the same id replaces the encoding in place — no reindex, no tombstones.
• Immediately queryable: the moment ingest returns, the document is searchable. There is no status to poll.

Search

A natural-language query answered by SKIM's multi-hop reasoning loop — not a single lookup. Ask real questions; the engine follows reasoning chains, not keywords. Search returns a SearchResult.

The SearchResult shape

This is identical in the API, both SDKs, and the CLI's --json output. Field names are snake_case in the API and Python; camelCase in TypeScript.

{
  "passages": [
    {
      "doc_id": "supplier_a",       // which document this passage came from
      "text": "…Supplier A payment terms are net-90 as amended in §4.2…",
      "score": 0.92,                // relevance of this passage, 0..1
      "hop": 1,                        // 1-based; absent = ranked candidate (see below)                     // which hop of the loop retrieved it (1-based)
      "metadata": { "source": "supplier_a.pdf" }
    }
  ],
  "confidence": 0.87,               // sufficiency head: "did I find enough to answer?"
  "hops_used": 2,                   // engine adapts effort; may be < max_hops
  "trace": [                        // present when return_trace = true
    { "hop": 1, "doc_id": "supplier_a", "why": "matches query" },
    { "hop": 2, "doc_id": "policy_2026", "why": "conflict found, given hop 1" }
  ],
  "latency_ms": 4.2,                // engine-only, measured server-side
  "engine": "skim-v1"               // backend that served this query
}

In TypeScript the same object is passages[].docId, hopsUsed, latencyMs, and trace[].docId — camelCased.

Field-by-field

Passage

TraceStep

Confidence, hops, and the trace

These three fields are the product — not debug output. They're the proof that Rekall reasons.

confidence — a signal you branch on

confidence is the output of a learned sufficiency head: "did I find enough to answer?" It is a first-class control-flow input. The canonical pattern — take the 4 ms path most of the time, escalate to an LLM only when the question is hard:

const r = await store.search(q);
const context = r.confidence >= 0.6
  ? r.passages                       // 4 ms path — most queries end here
  : await expensiveAgenticSearch(q); // rare fallback

r = store.search(q)
context = (
    r.passages                        # 4 ms path — most queries end here
    if r.confidence >= 0.6
    else expensive_agentic_search(q)  # rare fallback
)

hops_used — effort adapts to the question

The engine caps the loop at max_hops (default 6) but usually stops much earlier. A lookup question ends in one hop; a contradiction across two documents takes two; evidence-aggregation may take more. hops_used shows you how hard the question was — you never have to tell the engine.

trace — why it found that

With return_trace: true (near-zero cost), each hop reports the document it picked and why, given what it had already read. This is explainability that similarity search structurally cannot offer. Walk a full example in The reasoning trace.

Next steps