Appen Arrow Project: Here is How It Works (+ Exam Parts)

Project Termination Update

Current Status of Project Arrow

As of March 19, 2024, the Appen Project Arrow has been terminated. Google ended its contract with Appen, leading to the discontinuation of this project along with others. Many raters received termination letters confirming the closure. Former Arrow participants are encouraged to explore alternative opportunities with companies like Telus, OneForma, DataForce, and WeLocalize. Notably, the Personalized Ads Assessor job at Telus International is equivalent to Project Arrow, making it a suitable alternative for those looking to continue similar work.

Project Arrow is a search engine evaluator job at Appen that is all about evaluating the quality and relevance of ads displayed by Google. It involves many different types of tasks and extensive guidelines. As a well-established, long-term online project that has been around for many years, Arrow is very sought after by both seasoned remote workers and beginners looking for online jobs that require no experience.

In this article, you will learn the details of how the Arrow project works, what the exam looks like and what to expect in general. For more information on how to work from home with Appen, check out this article.

How Project Arrow Works

The goal of project Arrow is to ensure that Google is displaying high-quality ads that are relevant to its users. Put another way, project Arrow is for paid results (ads) what project Yukon is for organic results.

To understand what this project is all about, let’s take the following examples:

Example 1: Assume you are a user searching for <airline tickets> on Google. Would you consider it useful if, after entering this query, an advertisement for a website selling bus tickets was shown?

Well, not exactly since you're looking for airline tickets. At the same time, this is not so bad because depending on the distance you need to travel, a bus trip might be a viable alternative. A much worse scenario would be if a concert ticket website was shown, as this would be completely unrelated to the intent of the user for his/her query.

Example 2: Assume you are a user browsing through a web page about vegan diet. How relevant would it be if you were shown a display ad for a meat delivery subscription company?

Well, considering their current page, it would be very unlikely that such a business would be interested in advertising to this user. Likewise, it is very unlikely that this user would be interested in this type of ad.

Types of Tasks

Project Arrow is composed of 7 different types of tasks:

• Search Ads Evaluation
• User Interest Evaluation
• Ad Creative and Landing Page Features
• Query and Mobile App Match Evaluation
• User to Keyword Evaluation
• Close Match Evaluation
• Page Evaluation – Religion

Search Ads Evaluation

Search Ads Evaluation is what represents the bulk of the tasks available in Project Arrow. To perform this task, raters need to first understand the user intent behind the query. Then, they will be requested to rate the relevance of the Ad (referred to as Ad Creative) and Landing Page for the user intent.

User Interest Evaluation

The goal of this task is to evaluate whether the ad would be relevant to the user considering the current web page that the user is visiting.

Ad Creative and Landing Page Features

In this task, you will be answering questions related to different aspects of the ad creatives and landing pages. For example, are the concepts covered by the ad and landing page related to the concepts covered by the query?

Query and Mobile App Match Evaluation

This task is about rating the relevance of YouTube ads, specifically the ads that ask the user to install mobile apps.

User to Keyword Evaluation

In this task, raters will assess the likelihood of the user being interested in things related to a specific keyword. This is evaluated considering the user’s current interests.

Close Match Evaluation

The goal of this task is to evaluate whether the user query and advertiser keywords have the same or similar intents.

Page Evaluation – Religion

In this task, raters will determine if a web page is related to some religion or not.

How Much Does Project Arrow Pay?

As with all projects in Appen, the salary one can expect to earn will vary depending on the country where the rater is located and task availability. In general, you can expect to earn anything between $5 and $20 / hour, and the average rate in most countries is between $9 and $14 per hour.

Considering an average task availability of 15 hours/week, which also varies considerably from time to time, you can expect to make between $540 and $840 per month on average with this project.

Payment Method

All Appen raters working on long-term work from home projects like this one are paid on a monthly basis through Payoneer. By signing up for a Payoneer account with our link, you will receive a $50 bonus as per the terms described on their website. Sign up today to take advantage of this campaign while it’s available.

Appen Arrow Qualification Exam

Facing Project Arrow exam can be daunting as it requires many hours of study and dedication. The exam is divided into 7 parts, each of which represents one of the types of tasks that are available on this project.

In total, candidates need to go through many pages of guidelines and there is a total of over 200 questions to be completed.

Exam Parts

• Part 1 is the Search Ads Evaluation test, which is composed of 50 questions.
  • First, raters need to evaluate how promising the ad creative is for the user’s query. Next, they will rate the landing page according to how likely it is to satisfy the user intent.
• Part 2 is the User Interest Evaluation test, which is composed of 30 questions.
  • In this test, you will have to assess how likely it is that the business would want to advertise to the user given the user’s current interest and state of mind.
• Part 3 is the Ad Creative and Landing Page Features test, which is composed of 14 questions.
  • In this part, raters will answer several questions regarding the query and ad. For example: Is the query intent clear? Are the concepts of the ad and query related? Is the ad missing some critical requirements to satisfy the user intent? Is the ad difficult to make sense of? Etc.
• Part 4 is the Query and Mobile App Match Evaluation test, which is made up of 20 questions.
  • This test asks raters if installing or using an app would satisfy the user intent. The mobile app is displayed as an add after the user issues a query on YouTube.
• Part 5 is the User to Keyword Evaluation test, and it consists of approximately 45 questions.
  • On each question, raters will be given a certain keyword. The task is to evaluate how likely the user is to be interested in things related to that keyword considering the current page that the user is visiting.
• Part 6 is the Close Match Evaluation test. It is composed of 30 questions.
  • The goal is to assess if the intent of the advertiser keyword is related to the intent of the user’s search query.
• Part 7 is the Page Evaluation – Religion test, which has 20 questions in total.
  • The task is to simply mark whether a certain page pertains to the category Religion, according to the guidelines provided.

After completing the exam, you can expect to hear back from Appen in a few days.

How to Apply for Project Arrow

If you don’t have an Appen account yet, the first step is to register as an Internet Analyst on Appen Connect (click here to go to the registration page). Once your account has been created, look for project Arrow on the ‘All Projects’ tab.

Please note that Appen has hundreds of different online projects, and only a few of them may become available on your projects page. The list of available projects depends on a variety of factors, such as rater location, experience and performance.

Since you are looking for information on Appen projects, you may also like to register on these other websites that offer work from home opportunities.

Appen Arrow vs. Yukon: Which is Better?

Both Arrow and Yukon are long-term, well-established search engine evaluator jobs for Google. While Arrow is for advertisements, Yukon is about organic search results.

There is no definitive answer as to which project is better as this will mostly depend on your personal preferences. That said, most people prefer project Yukon over Arrow, as the work is more dynamic and the availability of tasks slightly higher.

Comparing Salary & Complexity

In terms of salary, you can expect the exact same pay rate in both projects, which will vary depending on the rater’s country.

In terms of complexity, Yukon is more challenging than Arrow as it can involve dozens of different types of tasks. Many raters like this because it makes the work less tedious. However, this also requires raters to frequently learn new instructions and guidelines for each different task, which may not appeal to people looking for an easier and more predictable job.

Conclusion

If you're drawn to a position that seamlessly combines analytical prowess with a deep dive into digital advertising, Project Arrow could be the ideal addition to your professional quiver.

Eager to shape and refine the online experience for countless users around the world? With an arrow-like precision, this might be the entry-level online opportunity you've been aiming for. And yes, that's our attempt at a pun! Why not embrace the challenge and discover where this path can lead you?