T
True Talent Solutions Partners

Research Scientist

San Francisco·$150K–200K

Python · Machine Learning · Language Models

ABOUT THE ROLE

Join a research and engineering team developing benchmarks and evaluation methods for advanced language models. Your work will help AI labs and enterprise teams assess model capabilities and make better-informed decisions about production use.

WHAT YOU'LL DO

  • Evaluate newly released AI models across a range of use cases.
  • Build benchmarks from the ground up, including dataset development and labeler coordination.
  • Develop scalable and accurate automated evaluation methods for generated text.
  • Work with engineering teams to implement and scale evaluation systems.
  • Collaborate with AI labs and enterprise teams to translate evaluation needs into research projects.

WHAT WE'RE LOOKING FOR

  • At least 3 years of full-time AI/ML research experience, particularly in benchmarking, evaluation, or language models.
  • A track record of published research, ideally in NLP or benchmarking, and hands-on experience creating benchmarks or evaluation methods.
  • Strong Python skills and experience building clean, maintainable software; Django, Flask, or similar Python HTTP server experience is useful.
  • Experience with PyTorch or TensorFlow, language modeling, LLM infrastructure, and relevant deep learning techniques.
  • Strong computer science fundamentals and the ability to turn research findings into practical systems.
  • Experience collaborating through team development workflows, Git, code review, and feedback.
  • A degree in Computer Science, Machine Learning, or a related field; a Master's or PhD is preferred. Candidates with a Bachelor's degr
Read the rest on jobs.ashbyhq.com