Search Evaluation Specialist (Bilingual Mandarin & English)

About the team 

Our TikTok Search Operation team plays a key role in improving TikTok user search experience by investigating user feedback, evaluating search result, analyzing data, etc. Join our team at the intersection of human language and advanced AI. 

This role is perfect for analytical thinkers with a background in Linguistics, Cognitive Science, or a related field, who are passionate about defining and measuring high-quality user experience for millions of search users. You will serve as the expert human voice, leveraging your deep understanding of language structure, user intent, and cultural context to directly influence the quality of our search products and the evolution of Large Language Models (LLMs). 

Responsibilities 

- Serve as the Subject Matter Expert on User Intent and Language Nuance to collaborate with Product and R&D teams, informing search display strategies and driving improvements in user engagement and satisfaction. 

- Conduct Deep Linguistic Investigations into search query patterns, ambiguity, and information-seeking behavior to establish the foundational insights for innovative display features. 

- Design and Execute Rigorous Qualitative and Quantitative Evaluation Frameworks to precisely measure the quality, relevance, and semantic accuracy of new features against user expectations. 

- Analyze User Feedback and Performance Data through a Linguistic Lens to develop actionable insights that inform strategic improvements and optimizations in search ranking and presentation. 

-Define and Standardize Search Quality: Collaborate with technical and product teams to establish gold-standard evaluation criteria and Standard Operating Procedures (SOPs) rooted in linguistic principles. 

- Apply Linguistic Expertise to LLM Strategy: Leverage your understanding of semantics, pragmatics, and corpus linguistics to enhance LLM performance, specifically concerning search relevance, ranking, and user experience.

Minimum Qualifications 

- Bilingual proficiency in Chinese and English is required to collaborate with global partners, including participating in meetings that are conducted in Chinese as well as corresponding in chat or emails. 

- Bachelor's degree or above. 

- Min. 1-3 years of experience in data analysis and drawing actionable insights. Internship experience, academic research, or teaching assistant experience is acceptable. 

Preferred Qualifications 

- A background in Linguistics, Computational Linguistics, Cognitive Science, or a closely related field. 

- Demonstrated understanding of linguistic principles (e.g., semantics, pragmatics, morphology, or sociolinguistics) and how they apply to modern search technology.