Senior Solution Architect, Applied AI
About the role
WEKA is hiring a tech lead for the AMG AI inference team. The role owns LLM serving from the NVMe token warehouse and GPUDirect Storage paths through the vLLM and LMCache stack, and leads a small group of engineers.
What you'll do
- Own AMG inference infrastructure from the NVMe token warehouse and GDS paths through the vLLM and LMCache serving stack.
- Lead a squad of 3 engineers on high-throughput, low-latency LLM serving.
- Stay hands-on in Python, C++, CUDA, vLLM, NIXL/Dynamo, and Kubernetes.
- Work on disaggregated prefill/decode, off-HBM KV caching, RDMA transport, and multi-tier GPU memory.
- Mentor engineers in 1:1s and sprint reviews, and benchmark tools such as SGLang, TRT-LLM, and NVIDIA Dynamo.
Requirements
- 8+ years of professional software engineering, with proven experience leading engineers and owning complex production systems — ideally in AI/ML infrastructure or high-performance computing.
- Hands-on expertise with LLM serving systems — KV cache reuse, disaggregated prefill/decode, continuous batching, and multi-tier GPU memory hierarchies (HBM to NVMe).
- Strong familiarity with vLLM, LMCache, NIXL/NVIDIA Dynamo, or similar frameworks.
- Strong Python and C++ skills (Rust a plus), with CUDA, GPU memory management, and high-performance I/O including GPUDirect Storage (GDS), RDMA, and NVMe data paths.
- deploying and scaling GPU workloads on Kubernetes, including RDMA networking and bare-metal GPU clusters (H100/A100).
- Demonstrated ability to mentor and develop engineers.
- USA Residents Only: The Total
About WekaIO
WEKA's AMG team builds high-throughput, low-latency LLM inference systems, from the NVMe token warehouse and GDS data paths to the vLLM and LMCache serving stack.
AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.