Agent Benchmarking: Evaluations on Due Diligence Data

Benchmarking

Benchmarking allows us to automatically and manually compare new agent outputs against human-verified ground truths. This process is conducted within a Retrieval-Augmented Generation (RAG) setting, where the agent’s output is evaluated based on its ability to accurately retrieve and respond with relevant information. If the response differs significantly from the ground truth, the system flags it as a potential quality risk. Final decisions are still guided by a human reviewer.

Dataset

This benchmarking evaluation is based on proprietary financial services data gathered from a comprehensive due diligence exercise conducted by a global investment management firm.

The dataset includes structured content from investment strategy documents, valuation policies, fund presentations, and internal assessment tools. These materials were prepared for institutional and professional investors, covering private equity operations, secondary transactions, and portfolio management practices.

This evaluation is based on eight documents collected as part of a real-world due diligence process conducted by an investment firm. These materials reflect the kind of content typically reviewed by institutional investors and investment teams when evaluating private market opportunities. The documents used include:

1. Fund Due Diligence Questionnaire
   A detailed overview of a private equity fund, including its strategy, structure, performance history, risk management, and ESG integration.

2. Fund Snapshot Summary
   A short-form fund presentation covering key terms, return targets, geographic focus, and investment themes.

3. Comprehensive Investor Questionnaire
   A deep-dive document covering governance, team structure, operational infrastructure, succession planning, and other key firm-level insights.

4. Valuation Policy Document
   A policy manual outlining the methodologies and governance framework used to value private equity investments, aligned with regulatory requirements.

5. Private Equity Strategy Overview
   A document describing the investment philosophy, risk management approach, and long-term positioning of a private equity platform.

6. AI and Technology Integration Brief
   A presentation focused on the use of AI and data-driven methods to improve sourcing, due diligence, and portfolio management in private equity.

7. Organizational Overview
   A team chart and experience matrix providing visibility into the composition and capabilities of the investment team.

8. Performance and Allocation Report
   A performance snapshot including target metrics such as IRR, deployment timelines, portfolio construction, and diversification strategy.

These documents form the basis for the benchmarking comparisons. Each configuration is tested using the same set of prompts derived from a consistent dataset to ensure fair and repeatable comparisons.