Split, then extract

Long reports mix sections that need different fields. Instead of one large schema, use Split to find each section’s pages, then run a small schema on only those pages.

What you need

The Python SDK and an API key, set up as in Cookbooks.
A long document. This recipe uses the 10-K Annual Report sample, which has 12 pages.

Build it

Parse the report

This code starts an async parse and polls it until it ends. wait works for any job: pass it the status call of that job’s endpoint, such as client.extract.get.

import os
import time

from pulse import Pulse

client = Pulse(api_key=os.environ["PULSE_API_KEY"])
REPORT = "https://platform.runpulse.com/api/examples/0514fc05-8b0a-4a3b-9b9b-18ac89fc04e5/pdf"


def wait(get, job_id):
    """Poll a job's status until it ends, backing off from 2 to 15 seconds."""
    delay = 2.0
    while (status := get(job_id)).status in ("queued", "processing"):
        time.sleep(delay)
        delay = min(delay * 1.5, 15)
    return status


parse_id = client.parse.run(file_url=REPORT, config={"model": "ultra-2"}, async_=True).parse_id
parsed = wait(client.parse.get, parse_id)
if parsed.status != "completed":
    raise RuntimeError(f"Parse {parse_id} ended {parsed.status}: {parsed.error}")

Find each section's pages

Name each section as a topic, with a description of what’s on its pages. This call waits for the split to finish, then reads each topic’s page numbers.

topics = [
    {"name": "Schedules", "description": "Financial statement schedules, exhibits and supporting tables"},
    {"name": "Signatures", "description": "Signature blocks, officers, titles and signing dates"},
    {"name": "Certifications", "description": "Officer certifications and compliance attestations"},
]
split_id = client.split.run(parse_id=parse_id, config={"topics": topics}).split_id
pages = {topic.name: topic.pages for topic in client.split.get_result(split_id).topics}
print(pages)

Every topic comes back, in the order you sent it. A topic’s pages is empty when the report has no such section.

Extract each section

Write a small schema for each topic. This code starts one async extract per topic that has pages, each reading only that topic’s pages through page_range.

strings = {"type": "array", "items": {"type": "string"}}
signatory = {
    "type": "object",
    "properties": {"name": {"type": "string"}, "title": {"type": "string"}, "date": {"type": "string"}},
}
schemas = {
    "Schedules": {"schedule_names": strings, "exhibit_numbers": strings},
    "Signatures": {"signatories": {"type": "array", "items": signatory}},
    "Certifications": {"certifying_officers": strings},
}

extract_ids = {}
for name, fields in schemas.items():
    if not pages[name]:
        continue  # this report has no such section
    extract_ids[name] = client.extract.run(
        parse_id=parse_id,
        config={
            "input_schema": {"type": "object", "properties": fields},
            "page_range": ",".join(str(page) for page in pages[name]),
        },
        async_=True,
    ).extract_id

Each extract only reads the parse, so you can start them all before you wait for any of them.

Collect the values

This code waits for each extract and reads its result.

results = {}
for name, extract_id in extract_ids.items():
    status = wait(client.extract.get, extract_id)
    if status.status == "completed":
        results[name] = client.extract.get_result(extract_id).extraction.enriched
    else:
        print(f"{name}: extract {extract_id} ended {status.status}: {status.error}")

print(results)

Every value carries its citation, the parse elements it was read from, and a confidence score. For the values alone, add "format": ["raw"] to the extract’s config and read extraction.raw.

How it works

Every step reads the same parse. Split decides which of its pages each schema sees: The calls in order: The recipe never reads the parse’s own result, so it also works where your organization deletes a result the first time it’s read. The split is billed for every page of the parse, whatever the number of topics, while each extract is billed only for the pages in its page_range. See Pricing.

Tips

Name topics that are easy to describe and rarely overlap. A page can belong to more than one topic.
Check each topic’s pages on a few reports before you trust its values.
A pipeline can run the parse and the split in one call. The extracts still start after it, because they need the split’s pages.
Once the topics and schemas work across several reports, store them as saved configs.

​What you need

​Build it

​How it works

​Tips

​Next steps

Split

Extract

What you need

Build it

How it works

Tips

Next steps