Gemma 4 31B vs Mistral Medium 3.5

Same documents, same prompt: where each model is right, what it really costs and how fast it answers.

At a glance

Task · documents	English (US)	Czech	German	Spanish	French
Invoice extraction	97.6% / 97.8%	96.1% / 93.5%	94.9% / 97.4%	95.4% / 97.0%	93.6% / 98.2%
Email triage	96.5% / 94.9%	97.1% / 94.5%	95.1% / 92.7%	—	—
Personal data detection	99.9% / 99.7%	99.8% / 99.8%	99.9% / 100.0%	—	—
Contract clauses	99.6% / 98.1%	100.0% / 100.0%	94.4% / 97.1%	—	—

Correct fields: Gemma 4 31B / Mistral Medium 3.5; the better one in bold. Click a cell to see the details.

Invoice extraction · English (US) (150 documents)

Mistral Medium 3.5 is more accurate by 0.1 pp. Gemma 4 31B is 26.1× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	97.6%	97.8%
Error-free documents	81%	79%
Per 1,000 documents	$0.17	$4.37
To fix / 1,000	187	207
Speed (median)	8.9 sec	2.3 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Supplier	99.3%	100.0%
Supplier ID (EIN, IČO…)	98.0%	100.0%
Invoice number	100.0%	100.0%
Payment reference	99.3%	94.7%
Invoice date	98.7%	98.7%
Due date	98.7%	99.3%
Net by tax rate	95.3%	88.0%
Tax by rate	98.7%	98.7%
Total	100.0%	100.0%
Currency	100.0%	100.0%
Bank account	86.0%	96.0%

Documents where they differ

Harmon Logistics LLC · INV256017 · Scan · Gemma 4 31B 8/11, Mistral Medium 3.5 11/11
Austin Landscaping Inc. · INV-17836 · Scan · Gemma 4 31B 9/11, Mistral Medium 3.5 11/11
Miranda-Vega Group LLC · 4959 · PDF text · Gemma 4 31B 11/11, Mistral Medium 3.5 10/11
Summit Services Inc. · INV-15230 · Scan · Gemma 4 31B 10/11, Mistral Medium 3.5 11/11
Raleigh Engineering Inc. · INV253511 · Scan · Gemma 4 31B 11/11, Mistral Medium 3.5 10/11
Jennifer Moore · 5113 · Scan · Gemma 4 31B 10/11, Mistral Medium 3.5 11/11
Mason-Nelson Group LLC · 2025-3952 · Photo · Gemma 4 31B 10/11, Mistral Medium 3.5 11/11
Sarah Villanueva · 2025-7876 · Clean image · Gemma 4 31B 10/11, Mistral Medium 3.5 11/11

Invoice extraction · Czech (150 documents)

Gemma 4 31B is more accurate by 2.6 pp. Gemma 4 31B is 22.1× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	96.1%	93.5%
Error-free documents	67%	63%
Per 1,000 documents	$0.21	$4.69
To fix / 1,000	333	367
Speed (median)	10.4 sec	2.6 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Supplier	96.7%	90.0%
Supplier ID (EIN, IČO…)	93.3%	87.3%
VAT ID	96.7%	85.3%
Invoice number	100.0%	99.3%
Payment reference	96.0%	95.3%
Invoice date	96.0%	88.7%
Tax point	95.3%	86.7%
Due date	94.0%	90.7%
Net by tax rate	98.0%	99.3%
Tax by rate	97.3%	98.7%
Total	99.3%	100.0%
Currency	100.0%	100.0%
Bank account	86.7%	94.0%

Documents where they differ

Pavlík a Zeman s.r.o. · 25500457 · Clean image · Gemma 4 31B 13/13, Mistral Medium 3.5 9/13
Hrubý s.r.o. · DB2500960 · Clean image · Gemma 4 31B 13/13, Mistral Medium 3.5 9/13
Bednář spol. s r.o. · 20250610 · Clean image · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Slavěna Pospíšilová · 26200409 · Scan · Gemma 4 31B 12/13, Mistral Medium 3.5 10/13
Stavsklad s.r.o. · 2025/1111 · Scan · Gemma 4 31B 12/13, Mistral Medium 3.5 10/13
Ekolog s.r.o. · FV2600132 · Scan · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Sedláček s.r.o. · 2025/0774 · Scan · Gemma 4 31B 13/13, Mistral Medium 3.5 12/13
Hruška a Šťastný s.r.o. · 2025/0324 · Clean image · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13

Invoice extraction · German (150 documents)

Mistral Medium 3.5 is more accurate by 2.5 pp. Gemma 4 31B is 23.7× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	94.9%	97.4%
Error-free documents	64%	75%
Per 1,000 documents	$0.20	$4.72
To fix / 1,000	360	253
Speed (median)	7.7 sec	2.5 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Supplier	98.7%	97.3%
Supplier ID (EIN, IČO…)	99.3%	98.7%
VAT ID	94.0%	96.0%
Invoice number	99.3%	100.0%
Payment reference	98.7%	92.7%
Invoice date	87.3%	98.0%
Tax point	92.0%	90.7%
Due date	90.7%	97.3%
Net by tax rate	94.7%	98.0%
Tax by rate	97.3%	100.0%
Total	99.3%	100.0%
Currency	99.3%	100.0%
Bank account	82.7%	97.3%

Documents where they differ

Wesermetall GmbH · RE2501375 · Scan · Gemma 4 31B 9/13, Mistral Medium 3.5 13/13
Spedition Müller GmbH · 2026-1446 · Scan · Gemma 4 31B 8/13, Mistral Medium 3.5 12/13
Zimmermann & Schulz GmbH · RE11274 · Scan · Gemma 4 31B 10/13, Mistral Medium 3.5 13/13
Meier GmbH · 2025-0264 · Photo · Gemma 4 31B 13/13, Mistral Medium 3.5 10/13
Bauunternehmung Schröder GmbH · 2026-0622 · Clean image · Gemma 4 31B 13/13, Mistral Medium 3.5 10/13
Schmidt & Bauer GmbH & Co. KG · RE-2025-01397 · Scan · Gemma 4 31B 10/13, Mistral Medium 3.5 13/13
Dachdeckerei Fischer GmbH · 2025/0014 · Scan · Gemma 4 31B 10/13, Mistral Medium 3.5 12/13
Christoph Haas – Hausmeisterservice · 2025-1432 · Clean image · Gemma 4 31B 13/13, Mistral Medium 3.5 11/13

Invoice extraction · Spanish (87 documents)

Mistral Medium 3.5 is more accurate by 1.6 pp. Gemma 4 31B is 23.3× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	95.4%	97.0%
Error-free documents	54%	67%
Per 1,000 documents	$0.20	$4.75
To fix / 1,000	460	330
Speed (median)	8.7 sec	2.5 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Supplier	95.4%	99.0%
Supplier ID (EIN, IČO…)	93.1%	100.0%
VAT ID	98.9%	94.0%
Invoice number	100.0%	100.0%
Payment reference	95.4%	93.0%
Invoice date	98.9%	100.0%
Tax point	100.0%	93.0%
Due date	97.7%	97.0%
Net by tax rate	81.6%	85.0%
Tax by rate	100.0%	100.0%
Total	100.0%	100.0%
Currency	100.0%	100.0%
Bank account	79.3%	100.0%

Documents where they differ

Coloma Servicios Informáticos S.L.U. · PRO-2025-0066 · Scan · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Talavera y Aller Asociados, S.L. · 2025/0610 · Scan · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Distribuciones Mayo Santana, S.L.U. · A-2025-0563 · Clean image · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Construcciones del Norte S.L.U. · PF25/0020 · Clean image · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Centro de Formación del Sur S.L.U. · F2025-1424 · Scan · Gemma 4 31B 12/13, Mistral Medium 3.5 11/13
Arranz e Hijos S.L. · F2026-0940 · Clean image · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Goñi Logística S.L. · R-2025/1460 · Scan · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Centro de Formación Mediterránea S.L. · 2026/0876 · Scan · Gemma 4 31B 13/13, Mistral Medium 3.5 12/13

Invoice extraction · French (87 documents)

Mistral Medium 3.5 is more accurate by 4.6 pp. Gemma 4 31B is 21.7× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	93.6%	98.2%
Error-free documents	51%	81%
Per 1,000 documents	$0.22	$4.81
To fix / 1,000	494	190
Speed (median)	7.2 sec	2.6 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Supplier	96.6%	99.0%
Supplier ID (EIN, IČO…)	77.0%	99.0%
VAT ID	95.4%	98.0%
Invoice number	97.7%	100.0%
Payment reference	96.6%	94.0%
Invoice date	96.6%	100.0%
Tax point	94.3%	100.0%
Due date	96.6%	100.0%
Net by tax rate	93.1%	95.0%
Tax by rate	100.0%	98.0%
Total	100.0%	99.0%
Currency	98.9%	96.0%
Bank account	73.6%	98.0%

Documents where they differ

Lefèvre & Fils SARL · FAC-2026-00198 · Photo · Gemma 4 31B 8/13, Mistral Medium 3.5 13/13
SA Établissements Gauthier · F2025-1451 · Photo · Gemma 4 31B 10/13, Mistral Medium 3.5 13/13
Guérin & Fils EURL · 2026-05-688 · Clean image · Gemma 4 31B 10/13, Mistral Medium 3.5 12/13
Établissements Bonnet EURL · AV-10599 · Scan · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
SARL Garcia Informatique · F2025-0111 · Clean image · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Bernard et Associés SASU · FAC-2025-01115 · Scan · Gemma 4 31B 11/13, Mistral Medium 3.5 13/13
Carrosserie Gauthier SASU · F-11079 · Scan · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Établissements Bertrand SASU · F2025-0066 · Scan · Gemma 4 31B 13/13, Mistral Medium 3.5 12/13

Email triage · English (US) (150 documents)

Gemma 4 31B is more accurate by 1.6 pp. Gemma 4 31B is 12.2× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	96.5%	94.9%
Error-free documents	72%	57%
Per 1,000 documents	$0.12	$1.47
To fix / 1,000	280	427
Speed (median)	2.2 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Category	100.0%	100.0%
Priority	85.3%	63.3%
Sentiment	83.3%	90.7%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Cody Baldwin · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Allison Floyd · Email · Gemma 4 31B 7/9, Mistral Medium 3.5 9/9
Danielle Little · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Russell Bates · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Kyle Schwartz · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
James Arnold · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Sheila Evans · Email · Gemma 4 31B 8/9, Mistral Medium 3.5 9/9
Jason Davis · Email · Gemma 4 31B 8/9, Mistral Medium 3.5 9/9

Email triage · Czech (150 documents)

Gemma 4 31B is more accurate by 2.6 pp. Gemma 4 31B is 12.3× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	97.1%	94.5%
Error-free documents	75%	55%
Per 1,000 documents	$0.16	$1.95
To fix / 1,000	253	447
Speed (median)	2.5 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Category	100.0%	100.0%
Priority	85.3%	63.3%
Sentiment	88.7%	87.3%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Simona Kolářová · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Milan Kašpar · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Eduard Stejskal · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Alexandr Kubíček · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Lýdie Tůmová · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Zuzana Brožová · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Petra Šťastná · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Adam Vaněk · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9

Email triage · German (150 documents)

Gemma 4 31B is more accurate by 2.4 pp. Gemma 4 31B is 11.9× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	95.1%	92.7%
Error-free documents	62%	48%
Per 1,000 documents	$0.15	$1.77
To fix / 1,000	380	520
Speed (median)	3.6 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Category	100.0%	100.0%
Priority	78.0%	56.7%
Sentiment	78.0%	78.0%
Customer name	100.0%	100.0%
Order number	100.0%	100.0%
Invoice number	100.0%	100.0%
Amount	100.0%	100.0%
Currency	100.0%	100.0%
Deadline	100.0%	100.0%

Documents where they differ

Irmela Warmer · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Sibille Lehmann · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Augusta Lange · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 7/9
Rolf-Peter Metz · Email · Gemma 4 31B 8/9, Mistral Medium 3.5 7/9
Friedbert Hoffmann · Email · Gemma 4 31B 8/9, Mistral Medium 3.5 9/9
Swantje Noack · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Rosemarie Steuer · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9
Raymund Weiß · Email · Gemma 4 31B 9/9, Mistral Medium 3.5 8/9

Personal data detection · English (US) (150 documents)

Gemma 4 31B is more accurate by 0.2 pp. Gemma 4 31B is 13.6× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	99.9%	99.7%
Error-free documents	99%	98%
Per 1,000 documents	$0.08	$1.15
To fix / 1,000	7	20
Speed (median)	2.4 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Names	100.0%	99.3%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	99.3%	98.7%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Margaret Mills · Text · Gemma 4 31B 7/7, Mistral Medium 3.5 6/7

Personal data detection · Czech (150 documents)

Both are equally accurate. Gemma 4 31B is 11.8× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	99.8%	99.8%
Error-free documents	99%	99%
Per 1,000 documents	$0.12	$1.40
To fix / 1,000	13	13
Speed (median)	2.9 sec	0.8 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Names	100.0%	98.7%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	98.7%	100.0%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Zikmund Sedláček · Text · Gemma 4 31B 7/7, Mistral Medium 3.5 6/7

Personal data detection · German (150 documents)

Mistral Medium 3.5 is more accurate by 0.1 pp. Gemma 4 31B is 13.6× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	99.9%	100.0%
Error-free documents	99%	100%
Per 1,000 documents	$0.10	$1.38
To fix / 1,000	7	0
Speed (median)	1.9 sec	0.9 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Names	100.0%	100.0%
Email addresses	100.0%	100.0%
Phone numbers	100.0%	100.0%
Addresses	99.3%	100.0%
Dates of birth	100.0%	100.0%
National IDs	100.0%	100.0%
Bank accounts	100.0%	100.0%

Documents where they differ

Gunnar Zorbach · Text · Gemma 4 31B 6/7, Mistral Medium 3.5 7/7

Contract clauses · English (US) (150 documents)

Gemma 4 31B is more accurate by 1.5 pp. Gemma 4 31B is 13× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	99.6%	98.1%
Error-free documents	97%	87%
Per 1,000 documents	$0.18	$2.35
To fix / 1,000	27	127
Speed (median)	2.5 sec	1.1 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Provider	98.0%	87.3%
Provider ID	100.0%	100.0%
Client	98.0%	88.7%
Client ID	100.0%	99.3%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	99.3%	100.0%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Page Printing LLC · Raleigh Printing Inc. · Contract text · Gemma 4 31B 13/13, Mistral Medium 3.5 11/13
Frontier Solutions Inc. · San Diego Office Products Co. · Contract text · Gemma 4 31B 13/13, Mistral Medium 3.5 11/13
Boston Supply LLC · Summit Systems Corp. · Contract text · Gemma 4 31B 13/13, Mistral Medium 3.5 11/13
Blue Ridge Services Co. · Foster Janitorial Services Corp. · Contract text · Gemma 4 31B 13/13, Mistral Medium 3.5 12/13

Contract clauses · Czech (150 documents)

Both are equally accurate. Gemma 4 31B is 12× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	100.0%	100.0%
Error-free documents	99%	100%
Per 1,000 documents	$0.25	$3.06
To fix / 1,000	7	0
Speed (median)	6.0 sec	1.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Provider	100.0%	100.0%
Provider ID	100.0%	100.0%
Client	100.0%	100.0%
Client ID	100.0%	100.0%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	99.3%	100.0%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	100.0%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Elektrotrade s.r.o. · Konečný a Dostál spol. s r.o. · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13

Contract clauses · German (150 documents)

Mistral Medium 3.5 is more accurate by 2.7 pp. Gemma 4 31B is 12.5× cheaper.

Metric	Gemma 4 31B	Mistral Medium 3.5
Correct fields	94.4%	97.1%
Error-free documents	27%	75%
Per 1,000 documents	$0.22	$2.76
To fix / 1,000	727	247
Speed (median)	2.6 sec	1.2 sec
Valid answers	100.0%	100.0%

Accuracy per field

Field	Gemma 4 31B	Mistral Medium 3.5
Provider	100.0%	100.0%
Provider ID	100.0%	82.0%
Client	100.0%	100.0%
Client ID	100.0%	80.7%
Contract type	100.0%	100.0%
Price (net)	100.0%	100.0%
Currency	100.0%	100.0%
Billing period	100.0%	100.0%
Effective date	100.0%	100.0%
End date	100.0%	100.0%
Notice period (months)	100.0%	100.0%
Late-payment penalty (% per day)	27.3%	100.0%
Governing law	100.0%	100.0%

Documents where they differ

Richter & Scholz e.K. · Köhler AG · Contract text · Gemma 4 31B 13/13, Mistral Medium 3.5 11/13
Hahn GmbH · Bauer & Klein GmbH · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Wesermetall e.K. · Weserform GmbH · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 11/13
Haustechnik Braun UG (haftungsbeschränkt) · Nordbau GmbH · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 11/13
Hansawerk GmbH · Schreinerei Weber AG · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 11/13
Winkler & Beck e.K. · Meyer & Weber e.K. · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Kraus Consulting GmbH · Lange Consulting GmbH & Co. KG · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13
Hansametall GmbH & Co. KG · Roth & Schröder UG (haftungsbeschränkt) · Contract text · Gemma 4 31B 12/13, Mistral Medium 3.5 13/13