Machine Learning · Python · TensorFlow
ABOUT THE ROLE
Join a small AI infrastructure team building tools and systems that improve the performance and energy efficiency of machine learning inference. You will work across software, models, and heterogeneous hardware, helping deliver fast, reliable inference in production.
WHAT YOU'LL DO
- Ship day-zero support for new open-source models, tuning them for latency and throughput.
- Optimize inference serving systems, including batching, KV cache, speculative decoding, and quantization.
- Write and tune kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
- Design, deploy, and operate heterogeneous clusters across hardware vendors.
- Run production inference across mixed hardware fleets, improving reliability, observability, and cost per token.
- Work with customers to understand their needs and help solve infrastructure challenges.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of experience in backend, infrastructure, or systems engineering, including experience at scale or in an early-stage environment.
- Strong understanding of computer architecture, memory hierarchy, and operating system internals beyond application-level development.
- Low-level or systems engineering experience, with familiarity in CUDA, Triton, HIP, or other GPU programming tools.
- Experience operating infrastructure at scale, such as clusters, data centers, or large GPU fleets, is valuable.
- Knowledge of machine learning inference optimization, including quantization, batching, KV cache, or speculative decoding, is beneficial.
- Strong C