AI QA Specialist
About the role
Dry Ground AI is hiring an AI QA Specialist to test how its production agents behave: accuracy, consistency, safety, speed, and cost. Applicants must live in Brazil, Colombia, or the Philippines.
What you'll do
- Design test plans for AI agents, chatbots and multi-step workflows, including tool use, memory, hand-offs, edge cases and failure paths.
- Build and maintain evaluation sets and regression suites for agent responses, including reference answers, scoring rubrics and LLM-as-judge checks.
- Test for hallucinations, prompt injection, data leaks and unsafe or off-brand responses.
- Measure latency, response quality and cost, and run load and capacity tests before launches.
- Add AI checks to the CI pipeline and investigate failures through logs, traces and APIs.
Requirements
- 3 or more years in QA or test engineering, with hands-on test automation.
- Experience testing AI or LLM-powered products: chatbots, agents, RAG systems or generative features.
- Must live in Brazil, Colombia or the Philippines.
About Dry Ground AI
Dry Ground AI helps mid-market companies become AI native. It designs, builds, and runs custom software, AI agents, and automation workflows in production for U.S. clients.
AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.