model inference
ABOUT THE ROLE
Join a small AI infrastructure team building tools to improve the performance and efficiency of model inference across diverse hardware. This hands-on role spans low-level systems engineering, production operations, and customer collaboration.
WHAT YOU'LL DO
- Bring new open-source models into production with support tuned for latency and throughput.
- Improve the inference serving stack, including batching, KV caching, speculative decoding, and quantization.
- Write and tune GPU kernels using CUDA, HIP, and Triton across a range of accelerator platforms.
- Design, deploy, and operate heterogeneous clusters across infrastructure vendors.
- Operate production inference systems, improving reliability, observability, and cost efficiency at scale.
- Work with customers to understand performance needs and solve technical problems.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of experience in backend, infrastructure, systems, or related engineering, with strong low-level engineering skills.
- Strong understanding of computer architecture, memory hierarchy, and operating system internals.
- Experience with CUDA, HIP, Triton, or other GPU programming, and familiarity with Python and accelerator platforms such as TPU or AWS Trainium.
- Background deploying or operating large-scale infrastructure, clusters, or GPU fleets is valuable.
- Familiarity with machine learning inference optimization techniques such as quantization, batching, KV caching, or speculative decoding is valuable.
- Strong computer science fundamentals, a record of solv