Work From Home
Ad Content Filtering Images and Text – Lion Bridge Ad Accessor exam tips
Mar
v2.9: Feb 2nd 2021
| ⚠ | NOTE: Guidelines have been updated. Please pay attention to the following changes: 1. The Reference to Death category has been updated2. The Medical Treatments category has been updated3. The rating of the Santander example in ‘Note on Prominence in Video Ads’ has been changed from PG to T |
Task summary
In this task, you’ll first review an advertisement (ad), and then answer the following question: is such an ad appropriate for audiences of all ages to see? (See What Are Content Labels? below.)
If you conclude that the ad is not appropriate for audiences of all ages, you’ll indicate the minimum age level it would be appropriate for, and give us more information on why you’ve concluded this.
How will the data be used?
We want to help prevent audiences from seeing ads which may not be appropriate for their age. The data you supply will be used to train and evaluate classifiers to this end.
lionbridge internet ads assessor pay
Workflow
- Review the ad. There are two things to consider with an ad:
- The offer being advertised, e.g. the product, service, app or website described in the creative.
- The advertisement itself or ‘ad creative’
- Review the visible content in the ad, and if necessary, research important terms from the ad that you’re unfamiliar with. The ad creative may contain a video — if so, play it, with Audio enabled. You should not spend any longer than 2 minutes reviewing a video ad. Tip: If you want to repeat the video playback, right click on the video and click ‘Reload’. See section below on reviewing Youtube Video Ads.
- Answer the question “Is this ad appropriate for audiences of all ages?”
- If you answer “Yes”, you’ll move on to the next task.
- If you answer “No”, you’ll then give us an idea as to why the ad is not appropriate for all age levels by selecting check boxes that correspond with different rationales.
- You’ll indicate what age level the ad would be appropriate for.
- If you select ‘Teen’ or ‘Mature Audience’, you’ll be asked “How bad would it be for this ad to be viewed by children of all ages?”
Youtube Video Ads:
Youtube video ads will generally look like this one below (with the distinctive Youtube logo turning red upon mouse-over). In these cases, the ad consists of the full Youtube video (which can be quite long). You should follow these steps to review these ads:
- Watch the first short section of the video which loads on the task page.
- Click the ad to view it on Youtube.
- Confirm that the ad is the same one viewed on the task page . If different, only treat the video in the task page as the ad creative.
- Use the slider to scan through the frames of the video, spending no longer than 2 minutes reviewing the video.
Landing Pages:
Apart from Youtube Video Ads (mentioned above), there is no requirement to visit the page to which the ad leads (‘landing page’), but you may do so, in order to learn more about the type of offer described in the creative. Note that your decisions should be based primarily on the ad creative, and the landing page should only be used to help you research terms in the ad that you are not familiar with.
What Are Content Labels?
By “content label”, we refer to the minimum age level that a particular ad is suitable for:
 | (General Audience): Ad suitable for audiences of all ages |
 | (Parental Guidance): Ad may not be suitable for young children |
 | (Teen): Ad suitable only for teen and older audiences |
 | (Mature Audience): Ad suitable only for adults |
| ⚠ | Note: age levels do not refer to whether an offer is designed for users of a specific age. Rather, if refers to whether the ad is safe / inoffensive / innocuous / harmless for viewers of a certain age level. |
Age Level Appropriateness Guidance
The question of age level appropriateness is inherently somewhat subjective, and each offer is unique, so we’re interested to get your judgment on the age level appropriateness of each individual ad. We’ll provide a guide to various types of ads (or elements of the ads) and some instruction on how we’d expect these to be rated. For instance, we’ll ask you to rate all Dating Services ads as MA .
Follow this general guidance, but keep in mind that there will be cases where it’s appropriate for you to use your discretion: if you feel that a certain ad merits a higher (or lower) age level rating than the guidance suggests, generally because this particular ad is not typical for that category, you can apply a higher (or lower) age level rating.
For instance, we recommend that you rate ads related with References to Death such as ads for funeral parlors as T , but if you encounter an ad with more disturbing content (“Get inside the mind of a serial killer!”) you can apply the higher MA rating.
When an offer involves multiple elements that are inappropriate for some age levels, and these have different suggested ratings (e.g., ad includes both violent T and sexualized MA imagery), always select the higher (e.g. MA ) of the two recommended age level ratings.
Note on Relevance vs. Appropriateness
This task is particularly focused on identifying ads which may be inappropriate for audiences of certain age levels. For instance, offers for alcohol or weapons would not be appropriate for younger audiences.
When we say an ad is ‘appropriate for a certain age level’ we mean: the ad is safe / inoffensive / innocuous / harmless for viewers of a certain age level. You are NOT being asked to judge whether audiences of those age levels will find the ad interesting or relevant.
For example, ads for banking services, real estate listings, or tax preparations may not be of any interest to children, but it’s also inoffensive if children are exposed to such ads incidentally while playing a game.
Note on Offer vs. Ad Creative
Below, we will make a distinction between inappropriate offers and inappropriate ad creatives.
Inappropriate offers refer to products or services (like alcohol or tobacco) that are inappropriate for certain age levels.
If an offer is inappropriate for certain age levels, then it’s inappropriate for general audiences regardless of how they are being advertised.
Inappropriate ad creative, on the other hand, refers to an ad where the image/video/text shown is inappropriate for certain age levels. Such ad creatives may well promote a product that is suitable for general audiences, but the ad itself contains elements that make them unsuitable for certain age levels. For example, an ad for a chocolate bar that included sexualized language or imagery is unsuitable for certain age levels, even though chocolate itself is fine for general audiences..
Note on ‘Unrateable’: Don’t use this flag if you see areas of concern!
DO NOT mark an item as unrateable if you see some evidence that a rating higher than E is appropriate. E.g. if you see an ad creative with the text ‘Find a hot date’ along with some other foreign language text, please mark this as MA for Dating Services.
This document is divided in two Parts:
- Part 1 covers inappropriate offers
- Part 2 covers inappropriate ad creatives
Age-restricted Entertainment
NEW: When reviewing ads for entertainment media (tv shows, movies, console games), please follow these steps:
- Check the age rating of the media – do a quick Google search to identify the rating of the movie/show/console game. If the rating is MA, then rate the ad MA, even if there is no objectionable content.
- If the rating is not MA, then rate the ad as any other ad. For example, this may mean that in some cases the age rating of the media is T, but the ad itself should be rated PG or G (or media is PG and ad should be rated G), if there is nothing objectionable in the ad.
Examples:
Movie rating | Ad rating following new guidelines | Reason for ad rating | Movie title (link for video ads) | Creative (thumbnail for video ads) |
|---|---|---|---|---|
| T | Unrealistic / fleeting violence | Jumanji: The Next Level |  | |
| T | Nothing objectionable | Avengers: Endgame |  | |
| T | Scary imagery | Avengers: Endgame |  | |
| T | Nothing objectionable | Harry Potter and the Order of the Phoenix |  | |
| T | Scary imagery, violence | Harry Potter and the Order of the Phoenix |  |
When it comes to ads for games, app games are not included. This only applies to games played on an Xbox, Nintendo Switch or other gaming console or gaming computer.
If you find conflicting age ratings from different regions, please try to figure out the region at which the ad is mainly targeted, and use that rating. e.g. imagine a Polish language ad for a movie also available in English – Poland rating: 12+ ; USA Rating: R – in this case, take into account the local rating. If it is less than MA (in this case it is), then label according to the rest of the guidelines.
Age ratings don’t always align with the scale used in this project. Please research the meaning of age ratings that you are not familiar with, and map them upwards to the scale used here.
Generally, something with a minimum age rating of between 12-14 years should not be considered MA, and 15 years upwards should get an MA rating.
| ⚠ | NOTE: that this only applies to entertainment content. A mortgage ad with a warning that customers must be 18+ should still receive a G rating. |
|  |
| Even though the movie rating is T, there is nothing objectionable in the creative, so the ad rating should be G. |  |
| The ad rating stays T because of scary imagery. |
18+ Interests
Beauty / Health
| ⚠ | Please note that the following categories are not included in Beauty/Health:Offers related to medical issues affecting animals, and/or promoting vet clinics (example)Products not primarily related to body image and appearance like toothpaste, soap, toilet paper (example)Ads promoting donations to organizations that provide medical care, like Red Cross. |
Controversial
| Astrology / Occult / Paranormal Offers for psychic readings, horoscopes, dream interpretation, tarot, spells, or other offers involving occult or paranormal elements. |    |
| Politics / Religion / Sensitive Social Issues Offers relating to political candidates, causes or electionsOffers relating to religions, religious beliefs, including religious schools, religious books, etc.Offers focused on sensitive or controversial societal issues, such as abortion policy, causes of global warming, immigration policy, etc. |    |
Privacy / Safety / Gimmicks
| Mobile Subscriptions / Digital Goods Ads that seem to offer mobile subscriptions (to ringtones, viral videos, pictures, games) or which are directly selling digital goods (game upgrades, sticker packs, etc.)⚠Note that Mobile Subscriptions does NOT mean all types of subscriptions. Only offers to subscribe to the specific products listed above count as Mobile Subscriptions. Likewise, Digital Goods also narrowly covers products like the ones listed above: TV shows or films are not considered Digital Goods for the purpose of this task. |   |
| Social Networks Offers for social networks that allow users to connect with friends, colleagues, other professionals.⚠Note: this does NOT include services that are primarily used for one-on-one communication, like messaging or video-calling apps. For example, Google Duo is used primarily for private conversations with friends and family. You should NOT flag apps like this as Social Networks. |  |
| Personality Quizzes / Sweepstakes Personality quizzes, “what character am I?”Ads featuring giveaways or prizes that users have a chance to win by being randomly selected (without any entry fee).⚠Notice that NOT all offers involving cash are sweepstakes. Sweepstakes are based on chance (entering one does not guarantee a win). Other offers involving cash, e.g. cashback deals, are not based on chance (e.g., you are guaranteed to get cash back if you spend over a certain amount). |    |
| Spying / Arrest Records Offers for services that imply they will help spy on a partner (often to catch infidelity), or find non-shared personal information about a third party; services that perform public records searches for arrest records (e.g., sex offender look-up).⚠Note that “Spying” only applies to services that offer to spy on a third party. It does NOT apply to apps that record phone calls, or to VPN services that allow to visit restricted content anonymously. |     |
| Live Streaming Offers for sites or applications that primarily exist to allow users to broadcast a live stream of themselves and their surroundings to a public audience of viewers; the broadcasting user is generally involved in some activity (talking, singing, dancing, etc.). The live stream typically includes both audio and video and often allows watchers to interact with the broadcasting user via chat or voice.⚠Note that by ‘live streaming apps’ we only refer to apps where users can live-stream videos of themselves or their own surroundings.Also note that apps or sites that live-stream news, produced events (e.g. sports matches), video gaming sessions or matches are NOT considered live streaming apps or sites for this task. |    |
| Chat (with strangers) Offers for sites or applications that primarily exist to allow users to connect and communicate with other users, who are typically strangers. (Sometimes, but not always, the purpose is declared to be dating.) This might take the form of a one-to-one or many-to-many conversation (e.g. chat rooms). Users can visit existing chat rooms or they may be able to create their own public chat rooms. Conversations may take place via video, text chat or voice.⚠Note: Chat apps that are primarily used to message people that the user already knows (e.g. WhatsApp, Messenger, Signal etc.) are NOT included. |   |
| Virtual Worlds Offers for sites or applications where multiple users can create personal avatars to populate a virtual world, i.e. a simulated environment which they can explore simultaneously and independently, while participating in its activities and communicating with one another. |  |
Restricted Products
| Fireworks Offers for fireworks, sparklers, firecrackers. |   |
| Spray Paints & Glass Etching Tools Offers selling or providing information on spray paint / aerosol paint. Includes spray paint sold for domestic or business decor purposes. Does not include painting services or paint that is non-spray.Offers for products such as glass etching cream, that alter the appearance of glass by making it obscure. Covers any caustic cream, gel, liquid, or solution capable, by means of a chemical action, of defacing, damaging, or destroying hard surfaces in a manner similar to acid. Does not include privacy films that are attached to windows in sheets. Does not include offers for etching/glasswork services.Offers for graffiti products or accessories. |   |
Sex / Romance
| Dating Services / Relationship Advice Offers for dating services, matchmakers; Offers that promise to help users “meet new people”;Offers featuring romantic relationship advice or tips. |     |
| Games: Adult-themed Games featuring sexual elements or other adult-content. |   |
| References to Sex or Porn Offers related to sexual activity or pornography. |    |
| Sexual Aids & Tips / Adult Products Products or services related to sexual activity; condoms; sexual aids promising better sexual performance. |    |
| Games: Kissing-elements Promotions for games which feature characters kissing or otherwise engaged in romantic pursuit. |   |
| Romance Fiction Offers promoting fictional material (e.g, books or comics) that fall in the Romance genre. |   |
Violence / Weapons / Death
| ⚠ | Games featuring cartoonish weapons and non-realistic violence should not be included in the following categories. |
| Fight Sports / Martial Arts Offers related to boxing, wrestling, martial arts; also self-defence training. |   |
| Games: First Person Shooters / Battle Games / Virtual Weapons Games where user has a first-person perspective and shoots at other charactersGames that involve controlling armies or fighters in battles or in combat.Offers featuring weaponry for use in video games; includes skins, cases or ‘loot’ for games. |    |
| Weapons & Accessories / Concealed Carry Offers promoting real guns or weaponry; also, firearm accessories such as holsters, concealed carry permitting, firearms safety and advocacy;Offers for hunting knives, pocket knives, and other non-culinary knives.Offers for hunting trips, even if no weapons are shown in the creative.⚠The category “Weapons & Accessories / Concealed Carry” should be used only for ads promoting REAL weaponry and accessories. Online games featuring weapons should not be included. |     |
| Paintball / Airsoft Offers related to paintball, airsoft, laser tag and other play weaponry. |    |
| Reference to Death, Disaster or Abuse Offers which touch on death, murder, funerals, natural disasters, domestic violence, child abuse, bullying (e.g, ads for life insurance, pre-payment of funerals, true crime books, support for victims of violence/abuse etc.) |    |
Part 2: Ad Creatives that are inappropriate for certain age levels
The offer itself may well be for products or services that are appropriate for all age levels (e.g. a candle or a car insurance company), but its promotion includes some elements which are inappropriate for younger audiences, such as scary imagery, profanity, adult behaviors (smoking, alcohol consumption, gambling), prominent weapons etc.
Types of inappropriate content, within the creative, that are not listed above (smoking, alcohol consumption, gambling, weapons) should be flagged in Part 2 under the checkbox ‘Some other element (not listed above)’.
Note: Not all of the concerns listed in Part 1 would be concerning under Part 2 (e.g. a car advertisement that shows a person with a tattoo should NOT be flagged for the presence of tattoos, even though Tattoo offers are listed in Part 1). The reasons listed above are the main ones that we think you’ll need to flag for.
Depictions of Violence
Scary Imagery
| Scary Imagery Offers that involve imagery which may be scary for some audiences (e.g., zombies, skeletons, masks, scary clowns, blood, wounds, etc.). |     |
Crude Humor / Gross Images
| Crude Humor / Gross Images Offers employing crude or juvenile humor (references to farting, vomit, genitalia, etc.)Offers that employ gross, odd, or unsettling imagery in the promotion of a product |   |
Profane Language / Sexual Innuendo
| Profane Language / Sexual Innuendo Offers employing crude or juvenile humor (references to farting, vomit, genitalia, etc.)Offers that employ gross, odd, or unsettling imagery in the promotion of a product |     |
Bare Skin
Offers featuring men or women with exposed skin or sheer clothing. Bare Skin imagery may or may not have sexual connotation.
| ⚠ | Note: Updated guidelines for Bare Skin |
| When there is significant bare skin exposure in a sexualized context. |     |
| Non-sexualized bare skin exposure near the breasts, buttocks, or groin (typical underwear and swimwear modeling falls here) |     |
| ⚠Note: If the exposed skin is neither sexualized nor concentrated near breasts, buttocks, or groin (for instance, bare arms, bared shoulders, bare midriff or exposed skin on legs), you should NOT label the advertisement as Bare Skin. |     |
Sexual(ized) Imagery
| Sexual(ized) Imagery Offers featuring men, women or couples in sensual/sexualized/sexy poses. |     |
Note on Prominence in Video Ads
Prominence in videos is defined in terms of the severity and the duration of the sensitive content.
Severity refers to how severe, or egregious, the scene is. This includes both visual and auditory severity.
Duration refers to how long the sensitive scene is, in relation to the overall length of the video. For example, a short sensitive scene in a long video would result in a lower rating than a short sensitive scene in a short video.
See examples in the matrix below. You can click on each link and the video will open in a new tab. For videos that open on YouTube, please skip any ads that might preceed the video.
| HIGH | |||||||||||||
| • Sleepy beast | |||||||||||||
| T | MA | ||||||||||||
| • Mafia City | • AFK Arena | ||||||||||||
| MED | • Candle commercial | • King of Dance | • Partymasters | ||||||||||
| • HBO | |||||||||||||
| • Gordon Ramsay | • Santander | ||||||||||||
| • Fun Race 3d | • Virtual History Berlin Wall | ||||||||||||
| ↑Severity | LOW | • Sonic movie trailer | PG | • Dice media trailer | T | ||||||||
| • Hulu | • McDonalds | ||||||||||||
| • Disney Fantasy | |||||||||||||
| G | • Pears Bodywash | ||||||||||||
| LOW | MED | HIGH | |||||||||||
| Duration → |
See rationales for some of the above examples:
| Video Ad | Rating | Sensitive Category | Ratonale | |
|---|---|---|---|---|
| Severity | Duration | |||
| Sleepy beast |  | Sexualized imagery | High – both visual and auditory | High – throughout the whole video |
| Partymasters | | Sexualized imagery | Medium | High – throughout the whole video |
| AFK Arena | | Sexualized imagery,Reference to violence | Medium – High – Sexualized, implied violence (“kill her”), moaning | High – for most of the duration of the video |
| King of Dance |  | Bare skin, Some other element (Weapons) | Medium – it’s in an artistic setting; there’s also guns towards the end | Medium – there are fleeting scenes and many scenes are ok |
| Candle commercial | | Crude Humor, Profane Language | Medium | Medium |
| Santander | | Reference to Death, Disaster or Abuse | Medium – the ad deals with domestic violence. Although no violence is observed in the video, this falls under the category ‘Death Disaster and Abuse’. | Medium |
| Hulu | | Fleeting violence | Low | Low |
| McDonalds | | Violence, non-gory | Low | Low |
| Disney fantasy | | Bare skin | Low | Low – long video and the potentially sensitive scenes are not the focus of the video and are short compared to the length of the whole video |
| Pears bodywash | | Bare skin | Low – even though there’s bare skin, there is no focus on breasts, buttocks or groin | Medium – High |
Note on Evidence vs. Assumptions
While working on this task, you might come across offers that you know little or nothing about. A typical example would be an ad that only contains a logo of an app and no further information, e.g.
 |
In this case you might know that Instagram is a social network, which merits a T rating.
However, it’s also possible that you have never come across the name of this product or service before. In this case, before you answer any question, you should first research the name of the offer. This will allow to quickly find out that Instagram is a social network and assign the T rating.
What you should NOTdo, however, is choose a label based on what you think might possibly be shared on Instagram. It is entirely possible that a beer company has an Instagram account that they use to promote their products. But this doesn’t mean that you should give Instagram a MA rating like you would alcohol offers.
You should also avoid guessing what the likely content of an offer is, beyond what is clear from your initial investigation. For example, suppose you come across the following ad:
 |
You might be familiar with the main character of this video game, or you might look her name up and come to the conclusion that it is about a reality TV personality. What you should avoid doing is speculating what the content of the video game might be. Here are some examples of what you should and should not base your rating on:
| OK (observations based on the ad) | Not OK (speculations beyond the ad) |
| Based on the “Play free” button, it looks like this is a video game. | The character is a reality TV personality so this game is likely to have content that is sexual or involving cosmetic surgery. |
| Based on the text of the ad, it looks like the game is based on a reality TV celebrity. | Based on what I know about the main character, I think the game might contain some alcohol or adult themes. |
Examples
‘Badness’ Guidance
If you select a rating of ‘Teen’ or ‘Mature Audience’, you will be asked the question: “How bad would it be for this ad to be viewed by children ofall ages?” ‘Bad’ in this context means inappropriate for children, or potentially harmful/scary/upsetting/disgusting or some other reason.
By selecting a T or MA rating, by definition, the creative is, to some degree, inappropriate for children of all ages to view. However, some ads are more inappropriate than others, so we want to find out ‘how bad’ it would be for children to view each ad rated as T or MA.
Whether an ad is bad for children to view is a very subjective matter. However, here we will provide some examples of ads and ‘badness’ ratings. It’s important that you refer to these examples, and use them as your benchmark when coming across similar ads, even if you personally disagree with the recommended answers. You will come across ads that aren’t covered by these examples, or fall into a grey area. For those cases, try to put yourself in the shoes of a parent of a child, or imagine how you would feel about a young relative coming across this ad while using a family-safe app or webpage.
Age Groups
We are asking “How bad would it be for this ad to be viewed by children of all ages?” In other words, you should try to imagine children from various age groups viewing this ad. It could be the case that you think an ad is ‘Extremely bad’ for teenager to view, but would be merely ‘Bad’ if a younger child stumbled across it. In these cases, your rating should be always be the higher of the two – ‘Extremely bad’.
Category Broadness
Note that the ‘Extremely bad’ end of the scale could include ads that grown adults might find offensive/scary/disgusting or otherwise inappropriate. It’s important to remember that this upper end of the scale isn’t limited to ads that are offensive-to-adults. For example, you shouldn’t hold back your ‘Extremely bad’ ratings, waiting for hardcore porn or extremely graphic violence to show up, as this rating is also applicable to less extreme content as seen below.
Examples: Slightly bad
Examples: Bad
 |  |
 |  |
 |  |
Examples: Extremely bad
