ModelVerdict

Comparison

Mistral Medium 3.5 vs Mistral Small 4

Same documents, same prompt: where each model is right, what it really costs and how fast it answers.

At a glance

Task · documentsEnglish (US)CzechGermanSpanishFrench
Invoice extraction97.8% / 95.6%93.5% / 93.6%97.4% / 94.6%97.0% / 91.9%98.2% / 94.2%
Email triage94.9% / 89.7%94.5% / 87.7%92.7% / 87.1%——
Personal data detection99.7% / 99.4%99.8% / 98.8%100.0% / 99.1%——
Contract clauses98.1% / 91.5%100.0% / 99.0%97.1% / 94.0%——

Correct fields: Mistral Medium 3.5 / Mistral Small 4; the better one in bold. Click a cell to see the details.

Invoice extraction · English (US) (150 documents)

Mistral Medium 3.5 is more accurate by 2.2 pp. Mistral Small 4 is 10.7× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields97.8%95.6%
Error-free documents79%58%
Per 1,000 documents$4.37$0.41
To fix / 1,000207420
Speed (median)2.3 sec1.8 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Supplier100.0%100.0%
Supplier ID (EIN, IČO…)100.0%99.3%
Invoice number100.0%100.0%
Payment reference94.7%87.3%
Invoice date98.7%97.3%
Due date99.3%98.7%
Net by tax rate88.0%72.0%
Tax by rate98.7%99.3%
Total100.0%100.0%
Currency100.0%99.3%
Bank account96.0%98.0%

Documents where they differ

Invoice extraction · Czech (150 documents)

Mistral Small 4 is more accurate by 0.2 pp. Mistral Small 4 is 10.7× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields93.5%93.6%
Error-free documents63%56%
Per 1,000 documents$4.69$0.44
To fix / 1,000367440
Speed (median)2.6 sec2.0 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Supplier90.0%91.3%
Supplier ID (EIN, IČO…)87.3%92.7%
VAT ID85.3%94.0%
Invoice number99.3%100.0%
Payment reference95.3%97.3%
Invoice date88.7%98.0%
Tax point86.7%76.0%
Due date90.7%97.3%
Net by tax rate99.3%90.0%
Tax by rate98.7%94.7%
Total100.0%98.7%
Currency100.0%97.3%
Bank account94.0%90.0%

Documents where they differ

Invoice extraction · German (150 documents)

Mistral Medium 3.5 is more accurate by 2.8 pp. Mistral Small 4 is 10.8× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields97.4%94.6%
Error-free documents75%50%
Per 1,000 documents$4.72$0.44
To fix / 1,000253500
Speed (median)2.5 sec2.0 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Supplier97.3%98.6%
Supplier ID (EIN, IČO…)98.7%80.0%
VAT ID96.0%92.9%
Invoice number100.0%100.0%
Payment reference92.7%87.1%
Invoice date98.0%100.0%
Tax point90.7%87.9%
Due date97.3%98.6%
Net by tax rate98.0%94.3%
Tax by rate100.0%99.3%
Total100.0%100.0%
Currency100.0%100.0%
Bank account97.3%91.4%

Documents where they differ

Invoice extraction · Spanish (100 documents)

Mistral Medium 3.5 is more accurate by 5.1 pp. Mistral Small 4 is 10.7× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields97.0%91.9%
Error-free documents67%31%
Per 1,000 documents$4.75$0.44
To fix / 1,000330691
Speed (median)2.5 sec2.2 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Supplier99.0%97.6%
Supplier ID (EIN, IČO…)100.0%98.8%
VAT ID94.0%53.6%
Invoice number100.0%100.0%
Payment reference93.0%88.1%
Invoice date100.0%100.0%
Tax point93.0%69.1%
Due date97.0%100.0%
Net by tax rate85.0%96.4%
Tax by rate100.0%97.6%
Total100.0%98.8%
Currency100.0%100.0%
Bank account100.0%95.2%

Documents where they differ

Invoice extraction · French (100 documents)

Mistral Medium 3.5 is more accurate by 3.9 pp. Mistral Small 4 is 10.8× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields98.2%94.2%
Error-free documents81%50%
Per 1,000 documents$4.81$0.45
To fix / 1,000190500
Speed (median)2.6 sec2.1 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Supplier99.0%96.1%
Supplier ID (EIN, IČO…)99.0%93.4%
VAT ID98.0%93.4%
Invoice number100.0%100.0%
Payment reference94.0%85.5%
Invoice date100.0%100.0%
Tax point100.0%92.1%
Due date100.0%100.0%
Net by tax rate95.0%94.7%
Tax by rate98.0%100.0%
Total99.0%100.0%
Currency96.0%94.7%
Bank account98.0%75.0%

Documents where they differ

Email triage · English (US) (150 documents)

Mistral Medium 3.5 is more accurate by 5.2 pp. Mistral Small 4 is 15.5× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields94.9%89.7%
Error-free documents57%37%
Per 1,000 documents$1.47$0.09
To fix / 1,000427627
Speed (median)0.8 sec0.8 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Category100.0%100.0%
Priority63.3%61.3%
Sentiment90.7%71.3%
Customer name100.0%100.0%
Order number100.0%100.0%
Invoice number100.0%100.0%
Amount100.0%99.3%
Currency100.0%99.3%
Deadline100.0%76.0%

Documents where they differ

  • Robert Lewis · Email · Mistral Medium 3.5 9/9, Mistral Small 4 6/9
  • Omar Cabrera · Email · Mistral Medium 3.5 9/9, Mistral Small 4 7/9
  • Jennifer Chapman · Email · Mistral Medium 3.5 9/9, Mistral Small 4 7/9
  • Leah Douglas · Email · Mistral Medium 3.5 9/9, Mistral Small 4 7/9
  • Russell Bates · Email · Mistral Medium 3.5 8/9, Mistral Small 4 7/9
  • Kyle Schwartz · Email · Mistral Medium 3.5 8/9, Mistral Small 4 7/9
  • James Arnold · Email · Mistral Medium 3.5 8/9, Mistral Small 4 9/9
  • Gene Everett · Email · Mistral Medium 3.5 8/9, Mistral Small 4 9/9

Email triage · Czech (150 documents)

Mistral Medium 3.5 is more accurate by 6.8 pp. Mistral Small 4 is 16.5× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields94.5%87.7%
Error-free documents55%32%
Per 1,000 documents$1.95$0.12
To fix / 1,000447685
Speed (median)0.8 sec0.8 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Category100.0%100.0%
Priority63.3%52.4%
Sentiment87.3%72.5%
Customer name100.0%98.7%
Order number100.0%100.0%
Invoice number100.0%100.0%
Amount100.0%98.0%
Currency100.0%98.0%
Deadline100.0%69.8%

Documents where they differ

Email triage · German (150 documents)

Mistral Medium 3.5 is more accurate by 5.6 pp. Mistral Small 4 is 16.6× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields92.7%87.1%
Error-free documents48%29%
Per 1,000 documents$1.77$0.11
To fix / 1,000520706
Speed (median)0.8 sec0.8 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Category100.0%100.0%
Priority56.7%55.2%
Sentiment78.0%65.0%
Customer name100.0%100.0%
Order number100.0%100.0%
Invoice number100.0%100.0%
Amount100.0%96.5%
Currency100.0%96.5%
Deadline100.0%70.6%

Documents where they differ

Personal data detection · English (US) (150 documents)

Mistral Medium 3.5 is more accurate by 0.3 pp. Mistral Small 4 is 14.9× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields99.7%99.4%
Error-free documents98%96%
Per 1,000 documents$1.15$0.08
To fix / 1,0002040
Speed (median)0.8 sec0.8 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Names99.3%97.3%
Email addresses100.0%100.0%
Phone numbers100.0%100.0%
Addresses98.7%98.7%
Dates of birth100.0%100.0%
National IDs100.0%100.0%
Bank accounts100.0%100.0%

Documents where they differ

Personal data detection · Czech (150 documents)

Mistral Medium 3.5 is more accurate by 1.1 pp. Mistral Small 4 is 15.1× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields99.8%98.8%
Error-free documents99%92%
Per 1,000 documents$1.40$0.09
To fix / 1,0001381
Speed (median)0.8 sec0.9 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Names98.7%94.0%
Email addresses100.0%100.0%
Phone numbers100.0%100.0%
Addresses100.0%98.7%
Dates of birth100.0%98.7%
National IDs100.0%100.0%
Bank accounts100.0%100.0%

Documents where they differ

Personal data detection · German (150 documents)

Mistral Medium 3.5 is more accurate by 0.9 pp. Mistral Small 4 is 15.4× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields100.0%99.1%
Error-free documents100%94%
Per 1,000 documents$1.38$0.09
To fix / 1,000065
Speed (median)0.9 sec0.9 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Names100.0%95.0%
Email addresses100.0%100.0%
Phone numbers100.0%100.0%
Addresses100.0%99.3%
Dates of birth100.0%100.0%
National IDs100.0%100.0%
Bank accounts100.0%99.3%

Documents where they differ

  • Rosi Karz · Text · Mistral Medium 3.5 7/7, Mistral Small 4 6/7

Contract clauses · English (US) (150 documents)

Mistral Medium 3.5 is more accurate by 6.6 pp. Mistral Small 4 is 13.8× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields98.1%91.5%
Error-free documents87%43%
Per 1,000 documents$2.35$0.17
To fix / 1,000127567
Speed (median)1.1 sec1.1 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Provider87.3%43.3%
Provider ID100.0%100.0%
Client88.7%46.7%
Client ID99.3%100.0%
Contract type100.0%100.0%
Price (net)100.0%100.0%
Currency100.0%100.0%
Billing period100.0%100.0%
Effective date100.0%100.0%
End date100.0%100.0%
Notice period (months)100.0%100.0%
Late-payment penalty (% per day)100.0%100.0%
Governing law100.0%100.0%

Documents where they differ

Contract clauses · Czech (150 documents)

Mistral Medium 3.5 is more accurate by 1 pp. Mistral Small 4 is 13.5× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields100.0%99.0%
Error-free documents100%92%
Per 1,000 documents$3.06$0.23
To fix / 1,000080
Speed (median)1.2 sec1.2 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Provider100.0%95.3%
Provider ID100.0%100.0%
Client100.0%95.3%
Client ID100.0%100.0%
Contract type100.0%100.0%
Price (net)100.0%100.0%
Currency100.0%100.0%
Billing period100.0%100.0%
Effective date100.0%96.7%
End date100.0%100.0%
Notice period (months)100.0%100.0%
Late-payment penalty (% per day)100.0%100.0%
Governing law100.0%100.0%

Documents where they differ

Contract clauses · German (150 documents)

Mistral Medium 3.5 is more accurate by 3.1 pp. Mistral Small 4 is 13.4× cheaper.

MetricMistral Medium 3.5Mistral Small 4
Correct fields97.1%94.0%
Error-free documents75%45%
Per 1,000 documents$2.76$0.21
To fix / 1,000247553
Speed (median)1.2 sec1.2 sec
Valid answers100.0%100.0%

Accuracy per field

FieldMistral Medium 3.5Mistral Small 4
Provider100.0%98.6%
Provider ID82.0%62.4%
Client100.0%98.6%
Client ID80.7%63.1%
Contract type100.0%100.0%
Price (net)100.0%100.0%
Currency100.0%100.0%
Billing period100.0%100.0%
Effective date100.0%99.3%
End date100.0%100.0%
Notice period (months)100.0%100.0%
Late-payment penalty (% per day)100.0%100.0%
Governing law100.0%100.0%

Documents where they differ