Remote AI Quality Analyst (Thai)

About the Company

Based in San Francisco, California, Turing is a research accelerator for frontier AI labs and a partner for enterprises deploying advanced AI systems. Turing supports frontier research with high-quality data, training pipelines, and AI researchers, and helps enterprises move AI from proof of concept to production.

About the Role

As an AI Quality Analyst you will evaluate a personalization feature for Gemini, assessing how the model uses information from past Gemini conversations, Gmail, Google Search, and YouTube activity to make responses more relevant and helpful. The role combines creative prompt design with analytical evaluation of personalized model outputs.

Responsibilities

• Design and execute multi-turn conversational prompts (typically 1–5 turns) that require the AI to use your personal information and experiences.
• Evaluate model responses based on intent from the starting prompt and check appropriate application of personalization.
• Analyze responses for Grounding issues, poor inferences, hallucinations, and integration quality (avoid overnarrating).
• Stack-rank two model responses side-by-side (SxS) to determine which is more helpful, easy to use, and enjoyable.
• Write clear, defensible rationales for comparisons, explicitly referencing turn numbers and specific issues or positives.
• Extract and verify debug info to confirm chat summaries and data sources were used properly.
• Maintain strict data hygiene by deleting evaluation conversations to avoid polluting future chat history.

Qualifications

• Thai proficiency: ability to read and write Thai at a high degree of competency (Thai is the focus language for this project).
• Willingness to use your primary personal Google account (not a testing account) and enable personal data sources for genuine assessment.
• Schedule flexibility and full-time availability in your local time zone; the team operates 24 hours globally.
• Commitment: at least 4 hours per day and minimum 30 hours per week with 4 hours of overlap with PST; options: 30 hrs/week or 40 hrs/week.
• Exceptional analytical thinking and strong evaluation acumen for personalization concepts, including identifying incorrect personalization and poor inferences.
• Experience designing creative, multi-turn starting prompts (prompt engineering) and experience in data annotation, AI quality evaluation, or content moderation is strongly preferred.
• Excellent written communication: clear, concise, structured rationales and detailed annotations; ability to provide constructive feedback.
• Self-motivated, able to work independently in a remote setting; desktop/laptop with a good internet connection required.
• Education: BS/BA degree or equivalent experience in a relevant field (e.g., Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical fields).

Offer Details

• Engagement type: Contractor.
• Engagement length: 3 months.
• After applying you will receive an email with a login link to access the portal and complete your profile.