# Member of Technical Staff, Evals at Profound

- Company: Profound
- Status: Open
- Workplace: On-site
- Location: New York, New York
- Level: Staff
- Discipline: AI & ML
- Employment: Full-time
- Posted: 2026-10-29
- Skills: Data
- Apply: https://jobs.ashbyhq.com/profound/d5a9a79a-8abd-40bd-98d0-d4f7f41d3095

## Description

Profound is the fastest-growing AI marketing platform, used by 15% of the Fortune 500 including Comcast, Estée Lauder, and Walmart, and innovators like Ramp, MongoDB, and Figma. It is where marketing teams do the work: research, briefs, content, campaigns, reporting, all in one place, alongside agents that carry the load with them.  

Built on 2B+ real user prompts and growing, Profound’s data shows you what people ask AI and what the models say back. Profound's AI Marketer joins that data with deep context about our customers to find the opportunities worth acting on, and then puts agents to work: automating reporting, drafting content, auditing pages for readability, running competitive research. It measures what works and gets better every day.

As an Evals Engineer, you will own how Profound knows its AI is right. At Profound, measurement is the product: when a Fortune 500 CMO sees their brand’s visibility drop in ChatGPT, they act on that number. No public benchmark measures whether a model attributed a citation correctly or ran a competitive analysis well, so we build our own. You’ll decide what gets measured, why, and how results shape what we ship. Every product runs through the evals you design, so your impact compounds across all of them.

What you’ll do

 - Build Profound’s eval system: golden datasets, scorers and LLM judges, programmatic verifiers, regression alerts, and dashboards

 - Write evals for every pipeline behind the product: brand extraction, visibility scoring, sentiment, FactCheck, and citation classification

 - Define what a great, mediocre, and 

More Profound roles: https://deviantjobs.com/companies/profound