12,400 questions over real enterprise documents with gold passages, hard negatives and unanswerable cases.
Teams building retrieval-augmented systems usually evaluate on academic question-answering sets that look nothing like their own documents. This benchmark is built from publicly available enterprise material — policies, product manuals, annual reports and tender documents — chunked into 480,000 passages.
Every question carries its gold passages, a set of deliberately hard negatives that look relevant but are not, and a difficulty label. Around 15% of questions are unanswerable from the corpus, so you can measure whether your system refuses instead of inventing an answer.
| Field | Type | Description |
|---|---|---|
question_id |
string | Unique identifier |
question |
string | Natural-language question |
question_type |
string | factoid, multi_hop, numeric, procedural |
answer |
string | Reference answer (empty when unanswerable) |
gold_passage_ids |
array | Passages that support the answer |
hard_negative_ids |
array | Similar passages that do not answer it |
is_unanswerable |
boolean | Answer is not in the corpus |
document_id / document_type |
string | Source document and its class |
language |
string | en, de or fr |
difficulty |
string | easy, medium or hard |
annotator_agreement |
decimal | Agreement score, 0–1 |
{
"question_id": "Q-004412",
"question": "How many days of notice are required to terminate the maintenance contract?",
"question_type": "factoid",
"answer": "30 days written notice",
"gold_passage_ids": [
"P-118822"
],
"hard_negative_ids": [
"P-118820",
"P-119004"
],
"is_unanswerable": false,
"document_type": "contract",
"language": "en",
"difficulty": "medium",
"annotator_agreement": 0.96
}
{
"question_id": "Q-004413",
"question": "What was the CEO's bonus in 2019?",
"question_type": "numeric",
"answer": "",
"gold_passage_ids": [],
"hard_negative_ids": [
"P-220110"
],
"is_unanswerable": true,
"document_type": "annual_report",
"language": "en",
"difficulty": "hard",
"annotator_agreement": 0.91
}
Illustrative sample showing the structure of the data. The full dataset is delivered after purchase.
Daily snapshots of 180+ hosted and open models: prices, context windows, limits and benchmark scores.
60K synthetic dialogues in six languages with function-call traces, tool schemas and realistic failure cases.
Half-hourly stock, delivery-time and price snapshots from 900 dark stores in 14 cities.
Tell us what you are trying to predict, automate or understand. We will reply within one business day with an approach, a timeline and a quote.