Methodology

How FIBRE scores a model

Updated Oct 8, 2026 · 202 tasks · 14 models

FIBRE tests whether a model can do fixed-income work: read a document, get the numbers right, and, on the research track, find the evidence itself. Every task has a rubric of specific claims. A panel of 3 judge models votes on each claim, the majority decides, and the claims that pass add up to the task’s score.

1. Tasks

There are two tracks, and they do not share tasks. The question-answering track (152 tasks) gives the model a written brief (a filing extract, a term sheet, a rate snapshot) and asks for one answer. Everything needed is in the prompt. The agentic-research track (50 tasks) gives only a mandate pinned to a date or a named filing, and the model has to find its own evidence.

Table 1. Tasks by category and track.
Category	Covers	QA	Research
Credit Analysis	Valuation, covenants, relative value, spread decomposition	62	17
Monitoring	Surveillance triggers, early-warning signals, false alarms	20	13
Structured Products	CLOs, ABS, MBS, cash-flow waterfalls	18	5
Research Intelligence	Thesis extraction, EDGAR deep-dives, trade construction	10	9
Discovery & Prediction	Trajectory forecasting, event-driven signals	2	2
Total		152	50

2. Running the models

Every model gets the same prompt, with no per-model tuning. On the research track it can search filings and the web, read documents and run calculations, and has to deliver its answer within a fixed number of steps. A research task a model never answers counts as zero.

3. Grading

Each rubric criterion is a specific claim a good answer should make, such as “identifies discount margin as the appropriate spread measure”. A panel of 3 judge models from different labs (opus-5, gpt-5.6-sol, deepseek-v4p1-flash) checks the answer against every claim and quotes the passage it relied on. A claim counts when the majority of judges agree. Judges never see how many points a claim is worth.

4. Scoring

A task’s score is

S=T + B − PC

where C is the points available from core criteria, T the core points earned, B any bonus points, and P points lost to penalties for specific mistakes. A few must-pass criteria set the score to zero when missed. A task scoring 80 or above counts as correct, and the overall score is the average across all tasks. Figure 1 shows the whole path for one real task.

Credit Analysis · Spread Analysis

Bond E is a SOFR-linked floating-rate note priced at 99.50. Analyze its spread characteristics: determine the appropriate market-implied spread measure, explain what the price discount reveals about the market's current credit assessment versus issuance conditions, assess how sensitive your spread estimate is to the assumed path of SOFR, and compare Bond E's effective spread to the Z-spreads on the fixed-rate bonds (D and F) for relative value purposes — accounting for the fundamentally different rate risk profiles of floating vs fixed-rate bonds.

Criterion	Pts	Highestgpt-5.5100.0 PASS221s	Lowestminimax-m3-16.7 FAIL207s
coreAutonomously identifies discount margin (DM) as the appropriate spread measure for a floating-rate note (not G-spread, Z-spread, or OAS). Computes DM by solving for the constant spread over SOFR that equates the present value of projected cash flows to the market price of 99.50. Result in the range 148-165bp.	3	+3	0
coreComputes the DM premium over the quoted margin: DM (~150-160bp) minus QM (125bp) = approximately 25-35bp. Explains this premium means the market now requires more spread than at issuance, reflecting deteriorated credit perception, wider sector conditions, or both. Acceptable range for premium: 23-40bp.	3	+3	0
coreAnalyzes DM sensitivity to SOFR: explains that for a short-dated FRN, DM is largely insensitive to the assumed SOFR level because both projected coupons and discount rates shift together. A ±100bp SOFR change shifts DM by only ~1-3bp. This rate-insensitivity is a key feature of DM — it isolates credit risk from rate risk, unlike Z-spread on fixed-rate bonds.	3	+3	0
coreCompares Bond E's effective spread (DM ~155bp) to the fixed-rate bonds' Z-spreads, while explicitly noting the comparison is not apples-to-apples due to different rate risk profiles. Bond E has near-zero rate duration (~0.25yr to next reset) vs Bond D's ~4.7yr and Bond F's ~6yr. Makes a relative value observation accounting for both spread level and rate exposure.	3	+3	0
penaltyDoes not report the quoted margin of 125bp as the bond's current spread. The quoted margin is a contractual feature set at issuance; the DM (~155bp) reflects the market's CURRENT required spread. For a bond trading below par, DM > QM.	2	0	−2
Score		(12 + 0 − 0) / 12100.0	(0 + 0 − 2) / 12-16.7

Read gpt-5.5’s answer (excerpt)

Read minimax-m3’s answer (excerpt)

Figure 1. One task, with the highest- and lowest-scoring model on it. Each dot is one judge’s vote; a criterion counts when the majority agrees. Click a criterion to see what the judges quoted.

5. What the scores look like

For most models the bulk of task scores falls between 60 and 100. The gap between models is mostly about how many tasks sit just below the pass mark, not about outright failures. Scores below zero are rare and come only from triggered penalties. Figure 2 shows the distribution for any model on either track.

30–20

420–40

340–60

2760–80

2880–99

76100

11>100

Figure 2. Task scores for opus-5 on the question-answering track. The shaded range is at or above the pass mark of 80. 115 of 152 tasks (76%) clear it; track score 88.8.

6. Cost and latency

Cost and latency are reported beside the score and never folded into it: whether a few extra seconds are worth a few points depends on how the model is used. Cost is per task at each provider’s list API prices.