Data Annotation Specialist, Generalist

About the role

We are on a mission to build machines that understand the world and make them safely accessible to all. Data quality is foundational to this process. Machines (or Large Language Models, to be exact) learn in similar ways to humans, by way of feedback. By labelling, ranking, auditing, and correcting model output, you will improve Large Language Models' performance for iterations to come, thus having a lasting impact on Cohere's technology. We are hiring Generalist professionals with broad backgrounds that span multiple consumer-facing or personal domains. This is a judgment-driven role, not passive data entry. You will review, assess, and provide structured feedback across a broad and evolving range of tasks, evaluating, stress-testing, and improving our models on English-language data spanning multiple modalities (text, image, and structured formats such as JSON, CSV/TSV, and Markdown). This is a great opportunity for professionals with strong analytical skills to contribute to high-impact annotation projects. Please Note: This is a part-time independent contractor position available within Canada only. We seek candidates who can commit to 16 hours per week at a CAD $30/hour contract rate. This role is BYOD 💻 (Bring Your Own Device, laptop). This position is remote.

What you'll do

• Evaluate and rank model outputs with written justifications; adversarially test models to surface failures and document reproducible bugs; author prompts/responses for training datasets; build and apply grading rubrics; label and correct multimodal (text/image/structured) data; participate in calibration checks to keep annotations consistent; adapt to new task types; report on model performance trends to cross-functional teams.

Requirements

• 1+ years of experience in AI data annotation, LLM evaluation, content moderation, research, or a related analytical role, with exposure to quality assurance, and/or preference ranking.
• Experience applying detailed guidelines to complex and often ambiguous content, with strong contextual and sociocultural judgment, sensitivity to nuance, tone, and register, and the ability to reason well in cases where there is no single correct answer.
• Comfort with ambiguity: a willingness to flag unclear or uncovered edge cases rather than guess, and to work productively on novel, experimental tasks whose definitions are still evolving.
• A sharp, curious eye for inconsistencies, subtle errors, and model failure modes, including the instinct to probe models adversarially and surface where they break. Familiarity with how large language models behave, such as hallucination, sycophancy, and instruction-following gaps, is a plus.
• Excellent command of written English and strong reading comprehension, with the ability to clearly justify your evaluations, including why an output is correct or incorrect, high-quality or low-quality, and to write clear prompts and exemplars that explain your reasoning. Bonus points if you are fluent in another language!
• Strong attention to detail and commitment to accuracy, with the ability to maintain consistency across high-volume and monotonous tasks.
• Comfort working with annotation platforms and structured formats such as JSON, CSV/TSV, Markdown, XML, and YAML.
• Strong execution in a remote environment, including good time management, comfort using new tools, and the ability to work independently in a global, asynchronous team.

About Cohere

Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us!

Visit Cohere→

AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.