AI Annotation / Intent Verification Specialist
About the role
We are looking for an AI Annotation Specialist to support the evaluation and improvement of a conversational AI system used to automate hotel front desk calls. This role focuses on listening to real guest calls, validating AI intent classification and responses, and helping establish high quality ground truth data that feeds model accuracy, product decisions, and future capability expansion. You will work closely with a product and AI team to ensure annotations are consistent, reliable, and actionable.
What you'll do
- Listen to and annotate recorded hotel phone calls using internal annotation tools
- Validate first level and second level intents predicted by the AI system
- Verify the correctness and relevance of AI responses against hotel knowledge bases and documented policies
- Flag incorrect answers, missing information, or failure patterns
- Identify and propose new intents or sub-intents based on real guest conversations
- Follow detailed annotation Standard Operating Procedures to ensure consistency
- Log results and insights in shared spreadsheets or internal dashboards
- Provide weekly summaries highlighting accuracy, volume, and recurring error patterns
Requirements
- Strong English listening comprehension and attention to detail
- Prior experience in data labeling, annotation, quality assurance, or content review
- Comfort working with transcripts, audio recordings, and structured guidelines
- Proficiency with Google Sheets or similar spreadsheet tools
- Ability to follow detailed processes and apply consistent judgment across large volumes of work
- Reliable internet connection and ability to work full time hours
About Somehere
We recruit, vet, and place top 1% remote talent for up to 80% less than US equivalents.
AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.