Bulk OCR: How to Process Thousands of Documents Without Losing Track

To OCR thousands of documents quickly, put each file in a queue, process many pages in parallel, give every job a unique ID so a failed job can be retried without creating duplicates, and send low-confidence results to a person instead of trusting them. At about 1.1 seconds per page, 100,000 pages take roughly 3 hours with 10 requests running at once. The rest of this guide shows the math and the parts that break first.

Speed is the easy half of bulk OCR. The harder half is what happens when 400 files out of 50,000 are corrupt, password protected, upside down or in a language nobody planned for, and you need to know which ones, without rerunning the whole batch. A pipeline that processes fast but loses track of failures is worse than a slow one that does not.

How long will my batch take?

Use this formula: hours = total pages x seconds per page / parallel requests / 3,600. Our benchmark measured about 1.1 seconds per page for DocsAPI, including the network round trip. Treat that as a planning number and time a sample of your own files, since page complexity changes it.

Total pages	1 request at a time	10 in parallel	50 in parallel
10,000	3.1 hours	18 minutes	4 minutes
100,000	30.6 hours	3.1 hours	37 minutes
1,000,000	12.7 days	30.6 hours	6.1 hours

Two cautions on these numbers. First, they assume pages are independent, so a 40-page statement counts as 40 pages. Second, the parallel columns assume your plan allows that many simultaneous requests. Ask what concurrency and rate limits apply to you before you plan a deadline around them. Local engines have a different ceiling: in our benchmark Tesseract took about 1.8 seconds per page on a mid-tier laptop CPU, and PaddleOCR 0.4 seconds with a GPU, so the same batch is limited by your own hardware.

What will it cost?

DocsAPI pricing starts at $0.02 per page, with volume discounts above 100,000 pages a month. Table extraction and classification are included in that price. At list price that is $2,000 for 100,000 pages and $20,000 for 1,000,000 pages, before any volume discount. Access starts with a demo, then a free 30-day trial for qualified lenders. Current terms are on the pricing page.

Cost per page is not the whole bill. Add the people cost of reviewing exceptions: if 5 percent of pages need a person for two minutes, 100,000 pages is 5,000 reviews, about 167 hours. Cutting the review rate by one point saves more than a small price difference between vendors. The accounts payable OCR guide has a fill-in cost model you can copy for your own numbers.

The six parts of a pipeline that survives bad files

1. A queue. Files go onto a queue first and workers pull from it. Do not loop over a folder in one script, because one crash loses your place.
2. A job ID per file. Use a hash of the file contents. If the same file arrives twice, you can see it and skip it, and a retry updates the same record instead of creating a second one.
3. Retries with backoff. Network errors and rate limits are normal at volume. Retry a few times with growing delays, then move the job to a failed list.
4. A dead-letter list. Files that keep failing go here with the error attached. Corrupt, encrypted and empty files end up here, and a person looks at them once instead of the pipeline stalling.
5. A confidence gate. Fields below your threshold go to review. See how to measure OCR accuracy for choosing a threshold from a real sample instead of a guess.
6. Counters. Track files received, processed, failed and in review. At any moment, received should equal the other three added together. If it does not, files are going missing.

A parallel worker sketch

This shows the shape of the loop. extract is a placeholder for whichever OCR call you use, so swap in your own.

import hashlib
from concurrent.futures import ThreadPoolExecutor, as_completed

def job_id(path):
    with open(path, "rb") as f:
        return hashlib.sha256(f.read()).hexdigest()

def run(path):
    jid = job_id(path)
    for attempt in range(4):
        try:
            return jid, extract(path)          # your OCR call
        except Exception as err:
            last = err
    return jid, {"failed": str(last)}

done, failed = {}, {}
with ThreadPoolExecutor(max_workers=10) as pool:
    futures = [pool.submit(run, p) for p in paths]
    for fut in as_completed(futures):
        jid, result = fut.result()
        (failed if "failed" in result else done)[jid] = result

In production add a delay between retries, persist done and failed to a database so a restart resumes where it stopped, and cap the worker count at what your plan allows.

What breaks first at volume

• Mixed document types. A folder of 50,000 files is rarely one type. Classify first, then route, as covered in document classification software.
• Duplicates. The same invoice arrives by email and by upload. Hashing catches exact copies. Near-duplicates need field-level matching, covered in duplicate invoice detection.
• Bad scans hiding in the average. A 96 percent average can hide a batch of faded receipts at 60 percent. Track accuracy by document type, not overall. Our benchmark shows the gap: phone-photographed receipts scored 58% to 82% across engines, against 97-99% on clean typed text.
• Sensitive data sitting in your queue. Know how long documents are kept. DocsAPI does not train on customer documents, and keeps documents and extracted data only for a short, configurable retention period before deleting them.

Run a pilot before the full batch

Pull a random sample of 200 to 500 files from the real batch, not the cleanest ones. Run them through the full pipeline, including failures and review. Measure time per page, failure rate, review rate and cost, then scale the figures up with the table above. Our guide to parallel-run sample size explains how big a sample you need to trust the result.

For a comparison of which engine suits which document type before you commit to a large run, see our 1,700-document benchmark. DocsAPI processes about 4 million pages a month, and if your volume is in that range, raise concurrency, retention and support expectations in the demo.