Web Content Assessor at Lionbridge: Work Online for Bing

Search engine companies rely on an army of human evaluators to check whether their algorithms are performing as expected. The ratings assigned by these evaluators do not directly impact how search results are ranked, but they function as invaluable feedback for improving search engines' over time.

The Web Content Assessor role at Lionbridge is one of these search engine evaluation jobs that you can do from home.

Job Description

A Web Content Assessor assigns ratings to search results, indicating for each result how well it satisfies the population of possible users who may have issued the query. This role at Lionbridge is related to the web search evaluation program of Microsoft Bing.

Simply put, it is Bing's equivalent to the search engine evaluator role at Google that is known by the name of personalized internet assessor at Lionbridge, among others.

To perform the web content assessment, raters are expected to follow a five-step process:

Understand the LP and Rating Quality and Credibility (QC)
Understand the User Intents for the Search
Evaluate User Expectations
LP Satisfaction Rating and Explanation
Final Rating Process and Annotation

In the guidelines used to train evaluators on how to do the job, Bing refers to the rating tasks as HIT, which stands for Human Inference Task. The platform where the tasks are executed is known as HITapp, which is part of a rating system called UHRS (Universal Human Relevance System).

Web Content Assessors help Bing on its goal to deliver search results that lead users to the most relevant and authoritative content in response to their search queries.

Web Content Assessor Salary

The salary of a web content assessor at Lionbridge ranges from $7 / hour to $20 / hour, depending on the country where the rater is located. In the US, the pay rate may also vary by state.

Web Content Assessors earn by the hour and are paid monthly, regardless of their location.

Job Requirements

To work as a web content assessor, you must:

Be comfortable using the Internet and working from home without much supervision
Have good analytical and communication skills
Know how to interpret and follow established guidelines
Be fluent in English and your local language, if you’re applying from a non-English speaking country
Be living in the country for which you are applying for the last 5 consecutive years. This is because it is crucial that raters have familiarity with historical, news, and cultural affairs of their locale
Have a PC running Windows 10 with a high-speed internet connection
Degree level qualification is desired

Application Process

To apply for the web content assessor role at Lionbridge, head to its Job Search page to check if this position is available in your country. Simply type in "web content assessor" in the search field, or click here for a shortcut.

Select the listing that refers to the country where you've been living for the last 5 consecutive years. If you cannot see a Web Content Assessor listing for your country, it means that there are currently no vacancies available. In this case, you should wait a couple of days and try again.

After clicking the item corresponding to your country of residence, you will be taken to a page with a brief explanation about the position. Read the job description and then click the button Apply Now! at the bottom of the page. In doing so, you will be prompted to create a new Lionbridge account if you still don't have one. Make sure to fill out the required information on the subsequent pages.

Before submitting the application, you will be asked to complete a pre-screening questionnaire. In general, it is recommended not to mention that you work on a similar opportunity with Lionbridge or another company. The reason is that there might be some conflict of interest that they don’t make clear to applicants, which could potentially cost you your job.

Web Content Assessor Guidelines

Unlike Google, Bing does not publicly disclose its search evaluation guidelines. To be sent that document, you must be invited for the qualification exam after successfully passing the resume screening phase.

The web content assessor guidelines are officially known as UHRS One-DCG Pilot Judging Guidelines and are about 70 pages long. This is far less than Google's search evaluator guidelines, which is over 150 pages long.

You can get an idea of what those guidelines look like by registering for the OneDCG Intro webinar, which is publicly available and can be accessed through this link. You can use any email address to register to watch this.

Web Content Assessor Exam

Once your application is reviewed by Lionbridge's exam support team, you will receive an email welcoming you to the OneDCG Qualification Process. Through the link provided in this email, you will be granted access to the Guidelines, Practice Data and Learning Modules.

Lionbridge requires candidates to complete a minimum of 50 tasks in the Practice data before they can move them to the exam. In total, the practice training batch has over 250 example exercises to help you get a good understanding of the tasks.

Applicants are given one week to complete both the Practice data and the Exam. The notification of the exam is sent only after you complete the Practice data.

The exam consists of 30 tasks. For each task, you will be asked to assign ratings for Intent, Quality Credibility (QC), Location, Freshness, Aboutness, and Landing Page Satisfaction (LP-Sat). These ratings are assigned based on a given query that is provided in the task.

You may be sent an additional 15 tasks to complete if your performance on the exam is not up to par.

On its OneDCG Tips and Tricks email, Lionbridge says that successful candidates spend approximately 2.5-3.5 hours in the exam. The company advises doing a few ratings and then take a break as regular breaks generally enhance your overall performance.