Search Ads Evaluation – Unpromising and Dissatisfying Part -2 – Lionbridge Rating

Changing the User Intent

The user wants to see results matching their query, or (strongly) related results. These results should be answering to the user intent. 

  • If the intent of the user is product seeking, the results should sell, show or describe the products or acceptable substitutes. 
  • If the user is looking for information, the results should best show the information the user was looking for.

In some cases, for information seeking queries, products can be helpful, e.g. if the user is looking for information on what cats eat, and they are presented with cat food. However, in other cases product results are not desired, e.g. when a user is looking for information on HOW to use something.

Example 1

The user query is: [how to install hog wire fenses]
Click here to view the Search Results

The Ad Creative is:

Step 1
Rate the Ad Content

The query is information seeking, the user is looking for a how-to for a specific product, not for the product itself. 

The AC is promising to lead the user to a well-known e-commerce platform, where they could buy the product, but nothing indicates the user might find an answer to their actual question. The AC is changing the user intent. There might be a very slight chance that either in the product descriptions a how-to is included or that the user is also interested in purchasing.

The instructions say: 

“If the page is marginally related to the query and you think that there’s a small chance the user would be interested, use Dissatisfaction Possible.” 

Therefore, we assign Dissatisfaction Possible. Additionally, most people know Amazon, so if you know the advertiser, please also assign the “Used Prior Knowledge of Advertiser” flag.

Step 2
Evaluate the Landing Page

After submitting your AC rating, you receive the following Landing Page for evaluation: Click to view the LP

On the Landing Page there are several products matching “hog wire fence”, but the user, who is interested in information, can’t find any how-to. The information seeking intent is not answered by the result.

Here we apply the following instruction part: 

“If the page offers something that you think the user might be interested in, but not what the user was looking for and not especially close to it, use Dissatisfaction Possible. For example, if the user is looking for baseball gloves, and the landing page offers athletic socks, there’s probably some chance that the user might be interested. However, it’s not what the user was looking for, and not all that close to it, so it deserves Dissatisfaction Possible.” 

The concept of this is also applicable for the above query versus Landing Page, so we assign Dissatisfaction Possible.

Example 2

The user query is: [how do a blogger make money]
Click here to view the Search Results

The Ad Creative is:

Step 1
Rate the Ad Content

The user has a specific question / is looking for a how to. 

The Ad Creative promises to lead the user to a website where they can start their own blog, but it doesn‘t seem to lead to a result answering the user‘s question about how to make money with a blog / how somebody makes money with a blog. The Ad Creative changes the intent from information seeking to product seeking. There is a slight chance, that once the user finds the answer / the how-to, they might be interested in starting their own blog though.  

The best rating is Somewhat Unpromising.

Step 2
Evaluate the Landing Page

In this step you will be directed to the Landing Page: Click to view the LP

On the Landing Page the user can start to build their own blog, either on an online platform or using software provided by the well-known company. But the user is not looking for a product, they are looking for information. Using the product might then be of interest in the next step. Since the desired information can’t be found on the Landing Page and the user intent being changed from information seeking, so we assign a negative rating of Dissatisfaction Possible

Not satisfying for the user intent

There are also Ad Creatives and Landing Pages, that only refer to words of the query, but not the complete intent. The user intent is not answered well enough to warrant a positive rating. Ask yourself if you would be satisfied with the result, or rather irritated if you were the user.

Here are some examples to show what is meant by this.

Example 1

The user query is: [92614 civil attorneys]
Click here to view the Search Results

The Ad Creative is:

Step 1
Rate the Ad Creative

The user is looking for civil attorneys in a specific area. This usually means attorneys specialized in human rights, equality, social issues, discriminations and similar issues. It does not include criminal offenses such as DUIs. 

The Ad Creative however is for a lawyer specialized in criminal offenses. Furthermore, the user is looking for a lawyer in California. The address shown in the Ad Creative refers to Maryland, which is too far away and might have different laws and regulations. It seems the Ad Creative was triggered by the word “attorney” (often used synonymous to lawyer, though there are differences), but it doesn’t respond to the context of the query and misses to match the user intent in a satisfying way. 

The advised rating is a low Somewhat Unpromising.

Step 2
Evaluate the Landing Page

In the next step you rate the Landing Page: Click to view the LP

Just like the Ad Creative promised the lawyer shown is working on criminal cases. Users who look for a civil attorney wouldn’t be satisfied with this result. There might be very few who might have a misunderstanding of civil vs criminal cases and might deem this slightly connected, so we allow Dissatisfaction Possible. However, a high Dissatisfaction Likely rating is also allowed because of the rather strong location mismatch between California and Maryland. 

Example 2

The user query is: [jazz festival 2019 atlanta]
Click here to view the Search Results

The Ad Creative is:

Step 1
Rate the Ad Content

The user is looking for a festival in an explicit location. 

The Ad Creative is for a similar festival, but ignores the explicit location and indicates that the advertised festival is over 2000 miles away from the explicit location. This would be a 35h drive or 4-6hr Flight. This is not a reasonable distance. Most likely the Ad Creative is triggered by “jazz festival”, but ignores the very important rest of the query. While there is a connection to the query, the Ad Creative is not promising for the user with that very specific query and location. 

The advised rating is Somewhat Unpromising

Step 2
Evaluate the Landing Page

The next step is checking the Landing Page: Click to view the LP

The landing page is for a jazz festival, but just like the Ad Creative indicated, the festival is in Monterey, which we don‘t see as being in a reasonable distance to the user‘s explicit location in the query. Most to all users would be dissatisfied with the result. 

Here we would assign Dissatisfaction Possible

Example 3

The user query is: [sax lessons online]
Click here to view the Search Results

The Ad Creative is:

Step 1
Rate the Ad Content

The user is very clearly looking for saxophone lessons online, e.g. in form of video tutorials or such. The Ad Creative does not in any way respond to this query. 

The instructions say: 

Very Unpromising ads have no reasonable chance of satisfying the user. Try to put yourself in the user’s mindset – is it possible at all that the creative offers something useful to the user?  If there is no reason at all to think that the user will find the creative useful, rate it Very Unpromising
(Note: you might think “It’s always possible that someone might find anything useful, even though it has nothing to do with the query.” Don’t go that far!)”

You might try to build a connection here by saying: “well, if they find lessons, they might also finish the course or such, which then would be like graduating, and maybe they then want to invite somebody?”. This would be falling under “Don’t go that far!”. There is absolutely no plausible connection between the Ad Creative and the query. 

The correct rating here is Very Unpromising

Step 2
Evaluate the Landing Page

The next step is rating the Landing Page: Click to view the LP

The same reasoning as above also applies to the Landing Page. Here the instructions are very clear: “If the page has nothing to do with the query, use Dissatisfaction Likely.” There is no reason why we should assign a higher rating, so the correct rating is Dissatisfaction Likely.

Conclusion

We hope our course was helpful and you gathered new insights into negative ratings. 

Keep in mind that most of the time a negative rating doesn’t mean that the Ad Creative is bad, or the advertiser is bad, but that the Ad Creative and/or Landing Page do not match, or come close to what the user wants to see. 

For other users with other intents, the Ad Creatives and/or Landing Pages might be quite matching or good.

Leave a Reply