T
True Talent Solutions Partners

Full-Stack Engineer, Agent Evaluation & Reliability

San Francisco

Client Roles

ABOUT THE ROLE

Join an AI/ML engineering team building infrastructure for agent evaluation and continuous improvement. This backend-leaning full-stack role combines foundational systems work with end-to-end product delivery, helping make agent workflows more reliable for enterprise customers.

WHAT YOU'LL DO

  • Build backend systems that power agent evaluation and continuous-improvement workflows.
  • Design data models for evaluation datasets, experiment runs, agent traces, knowledge versions, findings, and customer-defined quality criteria.
  • Build and maintain enterprise integrations, including authentication, OAuth, webhooks, incremental syncs, backfills, rate-limit handling, retries, and monitoring.
  • Own features from technical design and implementation through deployment, monitoring, customer feedback, and iteration.
  • Work with customers to understand their systems, diagnose failures, and turn specific needs into reusable product capabilities.
  • Take features through the full stack while contributing to architecture and product decisions.

WHAT WE'RE LOOKING FOR

  • Two to four years of experience in a full-stack or backend engineering role, building and shipping production systems.
  • Production experience with TypeScript and Node.js or Bun.
  • Hands-on machine learning model development experience, in addition to software engineering experience.
  • Experience at a fast-paced startup, from early stage through Series D.
  • Production experience designing PostgreSQL schemas and working with queues or worker systems.
  • A track record of owning projec
Read the rest on jobs.ashbyhq.com