AI Semantic Relevance Rater - Remote

Requirements:

• Fluent in English, with clear written communication and comfort with natural, everyday conversation.
• Strong attention to detail and patience when reviewing or replaying audio and video, or zooming in on images.
• Able to think quickly and speak naturally through realistic scenarios.
• Experience writing concise, evidence-based explanations or reviews.
• Able to make a clear choice when both options have flaws.
• A bachelors degree or higher is a plus.
• To sign up, provide a valid ID and mobile phone from your country of residence, an up-to-date resume, and a LinkedIn profile showing your education and work experience.
• Applicants must be from Australia, Canada, Ireland, New Zealand, the United Kingdom, or the United States, and be authorized to work in their country of residence.

Responsibilities:

• Hold live, scenario-based conversations in English with AI voice models and rate their naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and assess how well they follow the given prompts.
• Closely inspect outputs for issues such as distorted anatomy, flickering, unnatural movement, text errors, or mismatched audio and video.
• Score outputs for prompt adherence, quality, and realism, and write specific explanations in English based on what you observe or hear.
• Label first-person videos of people or robots performing hands-on tasks by dividing clips into actions and writing precise captions, correcting AI-generated drafts where needed.
• Evaluate AI-generated speech for naturalness, pronunciation, and similarity to a target voice; some assignments may include recording a short spoken explanation.

Technologies:

• AI
• Mobile

More

Outlier is an AI training platform owned and operated by Scale AI. Our experts contribute to projects that generate training data and evaluate AI model performance, helping improve how advanced systems reason, learn, and perform in real-world settings. This is remote freelance work on the Outlier.ai platform, with no minimum hours and a self-selected schedule. Project work can pay up to USD $15 per hour, with potential additional Missions incentives; rates vary by expertise, assessment results, location, project needs, and task type. Payments are processed weekly on Tuesdays through PayPal, AirTM, or ACH. Contributors receive free access to leading LLMs valued at over $200 per month and can collaborate with experts working on pre-release models. We describe this as a 1099 independent-contractor opportunity, not employment; internships and sponsorship are not offered. Applicants with questions about their individual tax or immigration circumstances should consult an appropriate advisor.

last updated 44 week of 2026