Can an AI find the right number in an annual report?

The problem

Finance teams, auditors and analysts spend a lot of time finding one specific number in a long filing. Retrieval-augmented generation (RAG) promises to help: the system fetches the relevant passages and a language model answers from them. But the answer can only be as good as what gets retrieved.

Before anything is retrieved, every document has to be cut into chunks. That choice is often left at a library default, even though financial documents mix narrative text, dense tables and footnotes that behave very differently when split.

Research question

How does the effect of chunk size and splitting strategy on retrieval quality vary across different types of financial document contexts?

Data

• FinanceBench: questions about public company filings, with the evidence passages marked.
• FinQA: numerical reasoning questions over earnings reports, mixing text and tables.
• TAT-QA: hybrid questions where the answer depends on both a table and the text around it.

Approach

I hold the rest of the pipeline fixed and vary only how documents are chunked: different chunk sizes and different splitting strategies. For each setup I measure whether the passages containing the answer are actually retrieved, using numeric retrieval metrics such as recall@k rather than token-overlap scores, which can reward answers that look similar without being correct.

def recall_at_k(retrieved_ids, gold_ids, k=5):
            top_k = set(retrieved_ids[:k])
            return len(top_k & set(gold_ids)) / len(gold_ids)

Results

Why it matters for companies

Any firm putting a document assistant in front of its finance team makes this chunking decision, usually without testing it. If the right choice depends on the document type, a one-size-fits-all setup quietly loses accuracy exactly where the numbers matter most.