5 Best Search Evaluator Jobs at Appen & TELUS AI (2024)

Search engine evaluators are remote-based, independent contractors that provide feedback on the performance of web search engines. It is a beginner-friendly, work from home job that requires no experience other than good analytical and web research skills.

Although search evaluation projects/jobs are designed by search engine companies like Google and Bing, the hiring and management process is conducted by third-party crowdsourcing companies. The two most popular ones are Appen and TELUS International AI (formerly Lionbridge AI).

The following is a list of the main search engine evaluator jobs that are available through these companies.

Search Engine Evaluator Job at Appen

Project Shasta

Shasta aims to improve the content, user relevance, and geographic accuracy of the service provided by Apple Maps. Shasta guidelines are extensive and this is generally considered a search evaluator job of above-average difficulty.

This is the same job known as Online Map Quality Analyst at TELUS International AI.

Search Engine Evaluator Jobs at TELUS International AI

Personalized Internet Assessor / Rater

A Personalized Internet Assessor provides feedback on the services of Google Web Search, particularly on the quality and relevance of search results. Guidelines for this project are nearly 200 pages long and require intense study and attention to detail.

At Appen, this job was known as Project Yukon*.

(*) as of March 19, 2024, Project Yukon has been terminated due to Google ending its contract with Appen. For similar opportunities, consider companies like OneForma, DataForce, and Welocalize.

Online Map Quality Analyst

An Online Map Quality Analyst rate the accuracy and relevance of search results from Apple Maps. The exam, which is broken down into two parts, is considered difficult and candidates are typically given two attempts to pass.

At Appen, this job is known as Project Shasta.

Personalized Internet Ads Assessor

A Personalized Internet Ads Assessor reviews online advertisements from Google Ads, particularly in terms of their quality and relevance attributes for the user's search query. Candidates need to go through many pages of guidelines and exam parts, which makes the qualification process quite extensive and overwhelming to some.

At Appen, this job was known as Project Arrow*.

(*) as of March 19, 2024, Project Arrow has been terminated due to Google ending its contract with Appen. For similar opportunities, consider companies like OneForma, DataForce, and Welocalize.

Web Content Assessor

A Web Content Assessor at TELUS International AI assigns relevance ratings for search results returned by Microsoft Bing. Basically, the tasks involve assessing how well the landing page satisfies the user's expectations and intent for the search. Candidates must complete a number of practice tasks for training before they are allowed to take the actual exam.

What makes these the top search engine evaluator jobs at Appen and TELUS AI?

The reason why these jobs were selected as the main search engine evaluator jobs at these companies are as follows:

They are popular projects that have been around for years
They are related to search engine companies and services that dominate the internet market
They are not so easy to get into, but once you are approved, you can expect a relatively steady flow of work for years (as long as you maintain high quality standards)
They are available in many different countries

Frequently Asked Questions about Search Engine Evaluator Jobs

How to apply for search engine evaluator jobs?

At Appen, go to Appen Connect and sign up for the Internet Analyst position. Once your application is approved, go ahead and look for a search engine evaluation project on the dashboard. Please note that Appen jobs are always presented in the form of projects. Therefore, instead of looking for “search engine evaluator” on the projects’ page, you must search for the name of the search engine evaluation project, such as Shasta, for example. Check out this blog post for more details on working from home with Appen,

At TELUS International AI, you must look for search engine evaluation job titles on its AI Community careers page. Examples of such jobs are Personalized Internet Assessor (Google Web Search), Online Map Quality Analyst (Apple Maps), and Personalized Internet Ads Assessor (Google Ads). Please note that the title of the job may slightly vary or change from time to time. For example, one of the jobs currently listed is titled [Language] speaking Search Engine Evaluators, where [Language] represents a local language, such as Japanese, Chinese, French, etc. Probably, this job title refers to the Bing Web Search project, which used to be known as Web Content Assessor.

How much do search engine evaluator jobs pay?

The hourly pay rate ranges from $3 to $20, depending on the job, company, and country where the remote agent is located. The expected workload is between 10 and 40 hours per week depending on the rater location and seasonality of tasks. Therefore, you can expect to earn anywhere between $120 and $3,200 per month on average.

How to prepare for the search engine evaluator exam?

For all search engine evaluator jobs, it is important that the applicant take their time to read the project guidelines thoroughly before taking the exam, ideally twice. The guidelines are sent to successful applicants about a week before the qualification exam is scheduled by the company's management team. You can find more details about the application process here.

For Personalized Internet Assessor (TELUS AI), you can get some extra help by taking the courses that are available at the Search Evaluator Academy. Although we cannot guarantee that they will make you pass the test, many applicants report that these courses make them more comfortable with the nuances of web search evaluation. In turn, that can help them feel more confident during the test.

More details about the Internet Assessor exam preparation can be found here.