MCP AI Software Evaluation Engineer

About the role

OpenTrain AI is hiring an MCP AI Software Evaluation Engineer to evaluate AI coding agents on realistic software engineering tasks and build MCP-based reinforcement learning environments. This is a flexible, worldwide remote contractor role through OpenTrain's AI-training marketplace, paying $60–$120/hour for about 15 hours per week. Prior AI-training experience is not required — real-world software engineering skill is the main qualification.

What you'll do

• Design reproducible RL environments and deterministic verification methods for software-engineering evaluations.
• Assess AI agents' bug fixes, feature development, refactoring, and performance-optimization work.
• Evaluate how effectively AI agents use Model Context Protocol (MCP) tools and servers.
• Document evidence-based technical judgments to help improve model behavior.
• Flexible schedule, approximately 15 hours per week.

Requirements

• Proficiency in at least one of C++, Python, Java, Go, TypeScript, or Rust.
• Strong understanding of algorithms and data structures.
• Demonstrated experience debugging, refactoring, and optimizing software for performance.
• Familiarity with MCP tools and agent interaction with MCP servers is valuable, not required.
• Strong written communication and attention to detail.
• Prior AI-training experience is not required.

About OpenTrain AI

OpenTrain AI is a talent marketplace for AI training and data-labeling work, connecting freelance contributors with RLHF, evaluation, annotation, and red-teaming projects from AI companies, and aggregating listings from other data-labeling platforms into one job feed.

Visit OpenTrain AI→

AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.