AI Response Evaluator (Remote)

Requirements:

• Fluent in English, with strong written communication skills and comfort speaking naturally in everyday conversations
• Strong attention to detail and patience when reviewing audio and visual content, including replaying or zooming in to investigate potential issues
• Able to think on your feet and speak naturally about realistic situations
• Experience writing concise, evidence-based explanations or reviews
• Able to make a clear choice when both options have flaws
• A bachelors degree or higher is a plus
• To sign up, youll need a valid ID and mobile phone from your country of residence, an up-to-date resume, and a LinkedIn profile showing your education and work experience
• Applicants must be authorized to work in their country of residence; international students may be able to sign up depending on their visa
• We are currently accepting applicants from Australia, Canada, Ireland, New Zealand, the United Kingdom, and the United States

Responsibilities:

• Hold live, scenario-based conversations in English with AI voice models, then rate how natural, useful, and clear the exchanges are
• Compare AI-generated images and videos, including edited media, and assess which better follows the prompt or source material
• Inspect media closely to identify issues such as distorted anatomy, flickering, unnatural motion, text errors, or audio that does not match the video
• Rate outputs for qualities such as prompt adherence, quality, and realism, and write specific English explanations supported by what you observed
• Label first-person videos of people or robots performing hands-on tasks by separating clips into actions and writing precise captions, reviewing and correcting AI-generated drafts
• Evaluate AI-generated speech for naturalness, pronunciation, and similarity to a target voice; some tasks involve recording a brief spoken explanation

Technologies:

• AI
• Mobile

More

Outlier is an AI training platform owned and operated by Scale AI. Our experts contribute to projects that generate training data and evaluate AI model performance, helping improve how advanced systems reason, learn, and perform in real-world settings. This is a remote freelance independent-contractor opportunity on the Outlier.ai platform, not an employee role; we do not offer internships, sponsorship, or employment. Work is flexible, with no minimum hours, and you set your own schedule. Project work can pay up to USD $15 per hour, with rates varying by expertise, skills assessment, location, project needs, and other factors; lower rates may apply during onboarding or overtime phases. Contributors may also receive project incentives through Missions, and payments are processed weekly on Tuesdays for the previous Tuesday-to-Monday pay period via PayPal, AirTM, or ACH transfer. We provide free access to leading LLMs valued at more than $200 per month and opportunities to collaborate with experts on pre-release models. Outlier has more than 900,000 experts across 52 countries and has paid over $600 million to experts. To get started, create an Outlier account, upload your resume and valid ID, and complete tasks.

last updated 44 week of 2026