Skip to navigation
File SearchFile Types

Documents

Captain indexes PDF and Word files as chunks that keep their layout role and page range. This page covers the processing tiers, how a document is chunked, what a result contains, and the limits.

This is useful for

  • Contracts and policies, where a clause needs a page number and a highlight box to cite
  • Scanned invoices, forms and correspondence at volume, where OCR text has to be searchable
  • Reports and decks, where headings, tables and charts should be returned as separate blocks
  • Any file a reader will open at the right page rather than read in a snippet

Processing tiers

Every document job sets processing_type to basic, advanced or auto.

TierWhat it doesCredits
basicLayout-aware parse of the text layer, with OCR on scanned or image regions. Use for clean, born-digital files and high volume.1 per page
advancedEverything in basic, plus a vision pass over the text (handwriting, faded print, unusual characters), table reconstruction that keeps merged cells and nested headers, and figure handling that describes images and extracts chart values when they can be verified.2.5 per page
autoCaptain inspects each file before parsing and uses advanced for files that need it and basic for the rest. Use for mixed collections where only some files have pictures, tables or scans.1 or 2.5 per page, by the tier each file used

How auto picks a tier

Captain reads the structure of each PDF without parsing it. A file is parsed with advanced when any page has one of these:

  • A photo or drawing that covers at least 4% of the page. Logos, signatures and icons are smaller than that and do not count. An image that repeats on most pages, such as a letterhead, does not count either.
  • A table drawn with ruled lines, with at least two columns and two rows of content. Empty form grids do not count.
  • A scanned page: an image covering most of the page with almost no text layer.

Every other file is parsed with basic. A file that cannot be inspected, such as a password-protected PDF, is parsed with advanced. In a PDF, charts drawn as shapes rather than an image are not detected, so index those files with advanced if you need their values.

Word files (.docx) are inspected the same way, from the document itself. A picture, chart or diagram that covers at least 4% of the page, or a table with visible borders, sends the file to advanced. Pictures in headers and footers, such as logos, do not count, and neither do tables without borders, such as signature blocks. Legacy .doc files are parsed with advanced.

Each file is billed at the tier it used. The job’s billing.credits reports used_basic and used_advanced separately.

Parse Document (POST /v3/parse/documents) takes the same auto value and reports the pages each tier used in usage.pages_basic and usage.pages_advanced.

Word files are converted to PDF before either tier runs, so .docx and .doc are handled exactly like PDFs from that point, page ranges included.

How a document is chunked

1

Layout roles

Each block of the page is labelled with a role: body, table, heading, page_header, page_footer, footnote or figure. Blocks with different roles are never packed into the same passage. This keeps running headers and footers out of body text, so a query for “confidential” does not match the stamp at the top of every page. Use exclude_chunk_types on a query to drop roles you do not want in results.

2

Sections and passages

The document is split into sections of up to 40,000 tokens. Each section gets a generated summary of two or three sentences and 8 to 15 tags. Each section is then tiled into passages of about 250 tokens, and the passages are what the search matches. A result contains the passage and its section’s summary, so a hit on one clause also says which part of the document it belongs to. Passages do not overlap.

3

Context across section boundaries

When a document spans several sections, the next section starts with a short summary of the previous one. A clause that crosses the boundary is readable from either side.

4

Tables and figures

Tables are stored as text inside the chunk (HTML tables in advanced), so cell values are searchable. Figures are described in words. Chart values are stored only when the extraction can be verified against the image; otherwise the figure is kept with extraction: "unavailable" and no values.

5

Page ranges and regions

Every result carries location.page_start and location.page_end. Add include.regions to get the blocks behind a result, each with page, a normalised bbox (top, left, width, height from 0 to 1), type and confidence. To show a page, call GET /v2/collections/{collection}/documents/{document_id}/pages/{page_number}.

Limits and failures

Documents of 75 pages or more are parsed in parallel, 75 pages at a time. There is no page cap. The only size ceiling is the one on the route the file arrived by (see File Size Limits). A document that cannot be parsed fails on its own: the error is recorded on that file in the job and the rest of the batch continues. Both tiers run OCR; language detection is automatic.

With mask_pii on, Captain masks the chunk text, the block regions and any figure crops. If masking fails, the document is not indexed. The report at GET /v2/jobs/{job_id}/pii lists what was found, with block-level boxes. See PII Masking.

Examples

Goal: return the termination clause from a set of contracts indexed with advanced, with the box to draw on the page.

request
{
  "query": "termination for convenience notice period",
  "limit": 5,
  "include": {
    "regions": true,
    "document": true
  },
  "exclude_chunk_types": [
    "page_header",
    "page_footer"
  ]
}
result (one item)
{
  "text": "Either party may terminate this Agreement for convenience on ninety (90) days written notice...",
  "chunk_type": "body",
  "document": {
    "id": "…",
    "name": "MSA-2026-Acme.pdf"
  },
  "location": {
    "page_start": 14,
    "page_end": 14
  },
  "regions": [
    {
      "type": "body",
      "page": 14,
      "bbox": {
        "top": 0.42,
        "left": 0.12,
        "width": 0.76,
        "height": 0.09
      },
      "confidence": 0.97
    }
  ]
}

The clause is one body passage with its own page range. regions[0].bbox is the rectangle to highlight.

Goal: make invoice numbers and totals searchable across thousands of scans at the lowest cost. Index with basic; OCR runs on the scanned regions at 1 credit per page. Move a supplier to advanced when its invoices have handwritten annotations or dense line-item tables.

request
{
  "query": "invoice INV-88213 total due",
  "limit": 3,
  "filter": {
    "supplier": {
      "$eq": "northwind"
    }
  }
}

The OCR text is chunked like any other text. supplier comes from the job’s custom_metadata, which is copied onto every passage.

Goal: find the revenue chart on slide 9 by what it shows, and keep slide numbers out of results.

request
{
  "query": "revenue by region chart",
  "limit": 5,
  "exclude_chunk_types": [
    "page_footer"
  ]
}
result (one item)
{
  "chunk_type": "figure",
  "text": "Bar chart of FY26 revenue by region: EMEA 41%, Americas 38%, APAC 21% ...",
  "location": {
    "page_start": 9,
    "page_end": 9
  }
}

The figure is described in words and labelled figure. Slide numbers are page_footer blocks, so excluding that role removes them.

See Supported File Types for the extension table.