ModelVerdict

DeepSeek · open weights

DeepSeek V4.1 Flash

How DeepSeek V4.1 Flash by DeepSeek handles real business documents: accuracy, true cost, speed and where it goes wrong.

List price: $0.02 per 1M input tokens, $0.60 per 1M output tokens · deepseek/deepseek-v4.1-flash

Results

Task · documentsRankCorrect fieldsError-free documentsPer 1,000 documentsTo fix / 1,000Speed (median)Bar
Contract clauses · English (US)2/10100.0%100%$0.3203.9 secmeets the bar
Contract clauses · Czech2/10100.0%100%$0.4204.3 secmeets the bar
Contract clauses · German7/1097.2%77%$0.512284.6 secmeets the bar
Email triage · English (US)4/1097.6%79%$0.232133.8 secmeets the bar
Email triage · Czech6/1097.4%77%$0.302274.2 secmeets the bar
Email triage · German6/1096.3%68%$0.303244.2 secmeets the bar
Invoice extraction · English (US)5/1099.4%93%$0.52675.7 secmeets the bar
Invoice extraction · Czech3/1099.9%99%$0.50135.0 secmeets the bar
Invoice extraction · German3/1099.5%94%$0.57606.4 secmeets the bar
Invoice extraction · Spanish3/1098.3%77%$0.712285.8 secmeets the bar
Invoice extraction · French3/1099.4%92%$0.65815.7 secmeets the bar
Personal data detection · English (US)6/1099.6%97%$0.29273.8 secmeets the bar
Personal data detection · Czech3/1099.7%98%$0.31204.2 secmeets the bar
Personal data detection · German3/1099.8%99%$0.31133.7 secmeets the bar

Where it goes wrong

The fields it gets wrong most often and the kinds of errors, from all documents of the benchmark.

Contract clauses · English (US)

No errors on this task.

Contract clauses · Czech

No errors on this task.

Contract clauses · German

Weakest fields

  • Provider ID81.4%
  • Client ID82.8%

Kinds of errors (number of fields)

  • invented value52

Email triage · English (US)

Weakest fields

  • Sentiment82.0%
  • Priority96.7%

Kinds of errors (number of fields)

  • wrong value32

Email triage · Czech

Weakest fields

  • Sentiment78.0%
  • Priority98.7%

Kinds of errors (number of fields)

  • wrong value35

Email triage · German

Weakest fields

  • Sentiment69.7%
  • Priority97.2%

Kinds of errors (number of fields)

  • wrong value48

Invoice extraction · English (US)

Weakest fields

  • Payment reference96.7%
  • Bank account97.3%
  • Currency99.3%

Kinds of errors (number of fields)

  • wrong value5
  • invented value5

Invoice extraction · Czech

Weakest fields

  • Supplier99.3%
  • Bank account99.3%

Kinds of errors (number of fields)

  • wrong value2

Invoice extraction · German

Weakest fields

  • Bank account98.0%
  • Supplier ID (EIN, IČO…)98.7%
  • Supplier99.3%
  • VAT ID99.3%

Kinds of errors (number of fields)

  • wrong value8
  • swapped dates1

Invoice extraction · Spanish

Weakest fields

  • Net by tax rate88.6%
  • Supplier93.7%
  • Payment reference97.5%
  • Tax point98.7%

Kinds of errors (number of fields)

  • missing field9
  • invented value3
  • lost diacritics3
  • wrong value3

Invoice extraction · French

Weakest fields

  • Bank account94.6%
  • VAT ID98.7%
  • Payment reference98.7%

Kinds of errors (number of fields)

  • wrong value5
  • invented value1

Personal data detection · English (US)

Weakest fields

  • Names98.7%
  • Addresses98.7%

Kinds of errors (number of fields)

  • missing field2
  • invented value1
  • wrong value1

Personal data detection · Czech

Weakest fields

  • Addresses98.0%

Kinds of errors (number of fields)

  • missing field3

Personal data detection · German

Weakest fields

  • Addresses98.7%

Kinds of errors (number of fields)

  • missing field2

Language tax

How many more tokens this model needs for the same text than in English.

Language tax of all models →

Compliance

DeepSeek's own API: no DPA in its terms, data stored in China. ZDR via third-party hosts on OpenRouter; self-hosting in the EU possible.

How we verify compliance →

Use it now

The exact request of the benchmark (same prompt, JSON schema, temperature 0), ready to call DeepSeek V4.1 Flash via OpenRouter.

Contract clauses · English (US)

Python · httpx · OpenRouter
import json
import os

import httpx

PROMPT = "Extract the key terms of the contract below and return only JSON with these keys:\n\nprovider_name         the party that delivers (provider, seller, landlord, licensor, contractor),\n                      exactly as written, including the legal form\nprovider_id           that party's company ID (EIN), if stated\nclient_name           the party that pays (client, buyer, tenant, licensee, customer)\nclient_id             that party's company ID (EIN), if stated\ncontract_type         one of: services, supply, lease, license, maintenance\nprice                 the agreed price excluding VAT / sales tax (number)\ncurrency              ISO 4217 code (USD, EUR, CZK, …)\nprice_period          one of: one_time, monthly, yearly\neffective_date        the date the contract takes effect (YYYY-MM-DD)\nend_date              the date a fixed term ends (YYYY-MM-DD); null if the term is indefinite\nnotice_period_months  the termination notice period in months (number); null if there is none\npenalty_pct_per_day   the late-payment penalty in percent of the overdue amount per day\n                      (e.g. 0.05 for 0.05 %); null if there is none\ngoverning_law         ISO 3166 country code of the governing law; US states as US-XX (e.g. US-NY)\n\nRules:\n- Resolve relative dates (\"on the date of signature\", \"the first day of the following month\")\n  against the signing date.\n- The signing date is not the effective date unless the contract says so.\n- Deposits, advance payments and fixed penalties for other breaches are not the price or the\n  late-payment penalty.\n- If a value is not in the contract, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"provider_name\":{\"type\":[\"string\",\"null\"]},\"provider_id\":{\"type\":[\"string\",\"null\"]},\"client_name\":{\"type\":[\"string\",\"null\"]},\"client_id\":{\"type\":[\"string\",\"null\"]},\"contract_type\":{\"type\":[\"string\",\"null\"],\"enum\":[\"services\",\"supply\",\"lease\",\"license\",\"maintenance\",null]},\"price\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"price_period\":{\"type\":[\"string\",\"null\"],\"enum\":[\"one_time\",\"monthly\",\"yearly\",null]},\"effective_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"end_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"notice_period_months\":{\"type\":[\"integer\",\"null\"]},\"penalty_pct_per_day\":{\"type\":[\"number\",\"null\"]},\"governing_law\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"provider_name\",\"provider_id\",\"client_name\",\"client_id\",\"contract_type\",\"price\",\"currency\",\"price_period\",\"effective_date\",\"end_date\",\"notice_period_months\",\"penalty_pct_per_day\",\"governing_law\"],\"additionalProperties\":false}")


def run(document_text: str) -> dict:
    """Text of the document in, the extracted fields as a dict out."""
    response = httpx.post(
        "https://openrouter.ai/api/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={
            "model": "deepseek/deepseek-v4.1-flash",
            "temperature": 0,
            "messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
            # For a scan or photo, send [{"type": "text", "text": PROMPT},
            # {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
            "response_format": {
                "type": "json_schema",
                "json_schema": {"name": "contracts", "strict": False, "schema": SCHEMA},
            },
        },
        timeout=120,
    )
    response.raise_for_status()
    return json.loads(response.json()["choices"][0]["message"]["content"])

Email triage · English (US)

Python · httpx · OpenRouter
import json
import os

import httpx

PROMPT = "You triage the support inbox. Read the email below and return only JSON with these keys:\n\ncategory        one of: billing (invoices, payments, charges), delivery (where is the order, shipping,\n                address change), complaint (damaged, wrong or defective goods, warranty claims),\n                technical (login, app or system errors, integrations), cancellation (cancel an order,\n                contract or subscription), sales (quotes, prices, interest before buying)\npriority        one of: urgent, high, normal, low (rules below)\nsentiment       one of: positive, neutral, negative (the customer's tone)\ncustomer_name   full name of the customer who wrote the request\norder_number    the customer's order number, if mentioned\ninvoice_number  invoice number, if mentioned\namount          the amount of money the request is about (number)\ncurrency        ISO 4217 code of that amount (USD, EUR, CZK, …)\ndeadline        the date by which the customer wants a resolution or answer (YYYY-MM-DD)\n\nPriority rules:\n- urgent: the customer cannot work right now, or the deadline is at most 1 day after the email date\n- high: the deadline is at most 7 days after the email date, or the customer threatens to cancel\n  or escalate (lawyer, ending the business)\n- low: the customer says explicitly that it is not urgent\n- normal: everything else\n\nRules:\n- A thread: judge the newest message; order numbers, invoice numbers and amounts may come from the\n  quoted earlier messages. A forwarded email: the customer is the original sender, not the\n  colleague who forwarded it.\n- Resolve relative deadlines (\"by Friday\", \"by tomorrow\", \"by the end of the month\") against the\n  date of the message that states them.\n- Support ticket numbers, tax IDs and phone numbers are not order or invoice numbers.\n- If a value is not in the email, return null. Do not guess.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"category\":{\"type\":[\"string\",\"null\"],\"enum\":[\"billing\",\"delivery\",\"complaint\",\"technical\",\"cancellation\",\"sales\",null]},\"priority\":{\"type\":[\"string\",\"null\"],\"enum\":[\"urgent\",\"high\",\"normal\",\"low\",null]},\"sentiment\":{\"type\":[\"string\",\"null\"],\"enum\":[\"positive\",\"neutral\",\"negative\",null]},\"customer_name\":{\"type\":[\"string\",\"null\"]},\"order_number\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"amount\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"deadline\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"}},\"required\":[\"category\",\"priority\",\"sentiment\",\"customer_name\",\"order_number\",\"invoice_number\",\"amount\",\"currency\",\"deadline\"],\"additionalProperties\":false}")


def run(document_text: str) -> dict:
    """Text of the document in, the extracted fields as a dict out."""
    response = httpx.post(
        "https://openrouter.ai/api/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={
            "model": "deepseek/deepseek-v4.1-flash",
            "temperature": 0,
            "messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
            # For a scan or photo, send [{"type": "text", "text": PROMPT},
            # {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
            "response_format": {
                "type": "json_schema",
                "json_schema": {"name": "emails", "strict": False, "schema": SCHEMA},
            },
        },
        timeout=120,
    )
    response.raise_for_status()
    return json.loads(response.json()["choices"][0]["message"]["content"])

Invoice extraction · English (US)

Python · httpx · OpenRouter
import json
import os

import httpx

PROMPT = "Extract the following data from the attached document and return only JSON with these keys:\n\nsupplier_name      supplier's name (exactly as printed, including the legal form)\nsupplier_id        supplier's tax ID (EIN), if printed\ninvoice_number     document number\npayment_reference  payment reference to quote with the payment\nissue_date         invoice date (YYYY-MM-DD)\ndue_date           due date (YYYY-MM-DD)\nnet_by_rate        amount per sales tax rate, e.g. {\"8.25\": 1845.00}; non-taxable amounts under \"0\"\nvat_by_rate        sales tax per rate, e.g. {\"8.25\": 152.21}\ntotal              total due (number, negative for credit notes)\ncurrency           ISO 4217 currency code (USD, GBP, EUR, …)\nbank_account       supplier's bank details as \"routing-number account-number\", or IBAN\n\nRules:\n- Return the supplier's data only, never the customer's.\n- If a value is not on the document, return null. Do not guess.\n- If the document has no tax breakdown, return net_by_rate and vat_by_rate as null.\n- Amounts as numbers with a decimal point, no thousands separators and no currency symbol.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"supplier_name\":{\"type\":[\"string\",\"null\"]},\"supplier_id\":{\"type\":[\"string\",\"null\"]},\"invoice_number\":{\"type\":[\"string\",\"null\"]},\"payment_reference\":{\"type\":[\"string\",\"null\"]},\"issue_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"due_date\":{\"type\":[\"string\",\"null\"],\"description\":\"YYYY-MM-DD\"},\"net_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"vat_by_rate\":{\"type\":[\"object\",\"null\"],\"additionalProperties\":{\"type\":\"number\"}},\"total\":{\"type\":[\"number\",\"null\"]},\"currency\":{\"type\":[\"string\",\"null\"]},\"bank_account\":{\"type\":[\"string\",\"null\"]}},\"required\":[\"supplier_name\",\"supplier_id\",\"invoice_number\",\"payment_reference\",\"issue_date\",\"due_date\",\"net_by_rate\",\"vat_by_rate\",\"total\",\"currency\",\"bank_account\"],\"additionalProperties\":false}")


def run(document_text: str) -> dict:
    """Text of the document in, the extracted fields as a dict out."""
    response = httpx.post(
        "https://openrouter.ai/api/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={
            "model": "deepseek/deepseek-v4.1-flash",
            "temperature": 0,
            "messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
            # For a scan or photo, send [{"type": "text", "text": PROMPT},
            # {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
            "response_format": {
                "type": "json_schema",
                "json_schema": {"name": "invoices", "strict": False, "schema": SCHEMA},
            },
        },
        timeout=120,
    )
    response.raise_for_status()
    return json.loads(response.json()["choices"][0]["message"]["content"])

Personal data detection · English (US)

Python · httpx · OpenRouter
import json
import os

import httpx

PROMPT = "Find all personal data of natural persons in the text below (for anonymisation) and return only\nJSON with these keys, each a list of strings (an empty list when there is none):\n\nperson_names       full names of people, each person once, as written in the text\nemail_addresses    email addresses of people\nphone_numbers      phone numbers of people, as written\npostal_addresses   home or delivery addresses of people, as written (street, city, ZIP)\nbirth_dates        dates of birth (YYYY-MM-DD)\nnational_ids       national identification numbers (e.g. SSN, birth numbers)\nbank_accounts      bank account numbers of people, as written\n\nRules:\n- Only data of natural persons. Company names, company registration or tax numbers (EIN, IČO,\n  VAT), generic company addresses such as info@ or support@, order and ticket numbers are not\n  personal data: leave them out.\n- Dates that are not dates of birth (meetings, deliveries, start dates) are not personal data.\n- Do not invent or complete values; copy them from the text.\n"
SCHEMA = json.loads("{\"type\":\"object\",\"properties\":{\"person_names\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"email_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"phone_numbers\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"postal_addresses\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"birth_dates\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"national_ids\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}},\"bank_accounts\":{\"type\":[\"array\",\"null\"],\"items\":{\"type\":\"string\"}}},\"required\":[\"person_names\",\"email_addresses\",\"phone_numbers\",\"postal_addresses\",\"birth_dates\",\"national_ids\",\"bank_accounts\"],\"additionalProperties\":false}")


def run(document_text: str) -> dict:
    """Text of the document in, the extracted fields as a dict out."""
    response = httpx.post(
        "https://openrouter.ai/api/v1/chat/completions",
        headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"},
        json={
            "model": "deepseek/deepseek-v4.1-flash",
            "temperature": 0,
            "messages": [{"role": "user", "content": f"{PROMPT}\n\n---\n{document_text}"}],
            # For a scan or photo, send [{"type": "text", "text": PROMPT},
            # {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}] instead.
            "response_format": {
                "type": "json_schema",
                "json_schema": {"name": "pii", "strict": False, "schema": SCHEMA},
            },
        },
        timeout=120,
    )
    response.raise_for_status()
    return json.loads(response.json()["choices"][0]["message"]["content"])

Badge

Embed the current result on your site; the badge updates with every run.

<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/contracts-en.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/contracts-cs.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/contracts-de.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/emails-en.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/emails-cs.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/emails-de.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/invoices-en.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/invoices-cs.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/invoices-de.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/invoices-es.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/invoices-fr.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/pii-en.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/pii-cs.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>
<a href="https://modelverdict.ai/en/models/deepseek-v4-1-flash/"><img src="https://modelverdict.ai/badge/deepseek-v4-1-flash/pii-de.svg" alt="DeepSeek V4.1 Flash on ModelVerdict"></a>

Compare with another model