Annotation Specialist | $11.5/hr Remote

About the Role

We are seeking an LLM AI Quality Analyst (Personalization) for a short-term contract to evaluate a personalization feature for Gemini. This is a remote, part-time position requiring 30-40 hours per week over a 2-month period, compensated at $11.5/hr.

Responsibilities

• Evaluate a personalization feature for Gemini
• Design and execute multi-turn conversational prompts (1-5 turns) using personal context
• Assess how effectively the model uses past conversations and personal data to generate relevant responses
• Evaluate model responses for grounding, helpfulness, and appropriate personalization
• Identify flawed inferences, reasoning gaps, and hallucinations in AI responses
• Assess integration quality to ensure personalization feels natural and not over-narrated
• Perform side-by-side (SxS) evaluations and stack-rank model responses based on quality and relevance
• Write clear rationales referencing specific conversation turns
• Extract and verify debug information to confirm correct use of summaries and data sources
• Maintain strict data hygiene by deleting evaluation conversations after completion

Qualifications

• Strong Thai proficiency (reading and writing)
• Experience in data annotation, AI quality evaluation, content moderation, or related roles
• Strong analytical thinking and ability to evaluate nuanced AI outputs
• Experience with creative prompt engineering and multi-turn conversational design
• Understanding of personalization concepts and AI response evaluation
• High attention to detail for side-by-side comparisons
• Excellent written communication and structured feedback documentation skills
• BS/BA degree or equivalent experience in a relevant field
• Willingness to use a primary personal Google account and enable personal data sources for assessment
• Ability to work independently in a remote environment
• Desktop or laptop with a stable internet connection

Additional Information

Type: Short-Term Contract (2 months)
Compensation: $11.5/hr
Location: Remote (Global)
Commitment: Part-time (30-40 hrs/week)