AI · inference
ABOUT THE ROLE
Join a small AI infrastructure team working to improve the performance and efficiency of large language model inference. You will build and operate systems that serve open-source models across a heterogeneous mix of hardware.
WHAT YOU'LL DO
- Ship support for new open-source models, tuning them for latency and throughput.
- Optimize inference serving systems, including batching, KV cache, speculative decoding, and quantization.
- Write and tune kernels using CUDA, HIP, and Triton for a range of accelerator architectures.
- Design, deploy, and operate heterogeneous clusters across hardware and cloud vendors.
- Run production inference systems, improving reliability, observability, and cost efficiency.
- Work with customers to understand technical needs and solve infrastructure problems.
WHAT WE'RE LOOKING FOR
- Typically 1 to 6 years of experience in backend, infrastructure, or systems engineering, including experience at scale or ownership of a complex system end to end.
- Strong understanding of computer architecture, memory hierarchy, and operating system internals beyond application-level development.
- Experience with low-level or systems engineering, alongside any full-stack or web development work.
- Familiarity with CUDA, Triton, HIP, or other GPU programming, and with ML inference optimization techniques such as quantization, batching, KV cache, or speculative decoding.
- Experience deploying or operating large-scale infrastructure, clusters, data centers, or GPU fleets is valuable.
- Strong computer science fundamentals a