Software Engineer - AI Performance
About the role
Build AI benchmarking and agent-evaluation infrastructure for DevRev's Computer platform — profiling, telemetry, and performance tuning of model-serving systems. Remote in the Philippines. For a backend/infra engineer ready to move into LLM evaluation; DevRev trains you on the AI side.
What you'll do
- Build and scale benchmarking infrastructure for evaluating AI systems in enterprise settings
- Design agent evaluation pipelines that measure reasoning, accuracy, alignment, and user outcomes
- Instrument model-serving systems (vLLM, Triton, Ray, or Kubernetes-based deployments)
- Develop data and telemetry pipelines that capture model performance, latency, and quality
- Contribute to DevRev's open benchmarking frameworks for intelligent SaaS
Requirements
- 3–10+ years of experience as a Software Engineer in backend, systems, or infrastructure roles
- Strong foundation in Python or similar backend languages (Go, Java, C#, or Node.js)
- Hands-on experience building APIs and cloud-native architectures (AWS, GCP, or Azure)
- Understanding of metrics, profiling, or observability tools (Grafana, Prometheus, ELK, or similar)
- Curiosity and commitment to learn AI benchmarking, LLM evaluation, and performance frameworks
About DevRev
DevRev builds Computer, an AI teammate that unifies enterprise data, tools, and workflows into an AI-ready platform with agentic actions.
AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.