# Evaluation Engineer, Applied AI at Mistral AI

- Company: Mistral AI
- Status: Open
- Workplace: On-site
- Location: Paris
- Level: Mid-level
- Discipline: AI & ML
- Employment: Full-time
- Posted: 2026-10-19
- Skills: AI
- Apply: https://jobs.ashbyhq.com/mistral.ai/5e6f3efd-36a5-48ab-8643-843edaeee578

## Description

ABOUT MISTRAL

Mistral provides full-stack AI solutions: from frontier models to developer tools, applications, and compute. We partner with enterprises tackling the hardest problems across high-stakes industries like finance, manufacturing, defense, healthcare, and the public sector, co-creating customized AI systems that they can run on their terms.

We are a dynamic, collaborative team passionate about AI and its potential to transform society. Our diverse workforce thrives in competitive environments and is committed to driving innovation. Our teams are distributed between Europe, North America, Asia and the Middle East. We are creative, low-ego and team-spirited.




THE ROLE

This is a chance to build the frontier LLM evals that shape the next generations of flagship models.

Could a model design a computer chip, or find the backdoor an attacker left in a network? Public benchmarks can't tell us, as they're generally not a good measure of real work.

As an Evaluation Engineer, you'll own evals from start to finish. You'll run workshops with customers, industry experts and our customer-facing teams to understand their most important industry use cases (e.g., an engineer updating a part in CAD software), and help decide which are worth building first. You'll also look beyond today's use cases and design moonshot evals for work no model can do yet, so Mistral gets there first. Then you'll turn each one into a benchmark, often one where an agent has to do the work inside real software. You'll source the data, decide how to score it, and run it on every new Mistral model a

More Mistral AI roles: https://deviantjobs.com/companies/mistral-ai