How to test whether your content is causing deliverability issues?

[vytismar]

Overview

This guide covers the methodology, logic and important points behind testing the impact of content blocks to determine whether a piece of content could cause major deliverability issue for your campaigns. In some cases this will be a URL or coding problem rather than phrasing, because phrasing does not make that much of an impact as it used to. However, it could also be a mix of a few factors, especially when it comes to folder placement.

Disclaimer: This guide has been confirmed to be working with messages bouncing due to content. However, with filtering issues it may only help in some cases, because unique mailbox engagement with particular sender or content is also a major factor in filtering decisions.

If you would like SAP Emarsys staff to assist you with such testing, you will need to open a ticket and provide account access.

Methodology and Testing Steps

The methodology is fairly simple and focuses mainly on taking one step at a time, removing content block by block and testing after each block is removed. So, the short version would be simply to remove one block at a time and send test campaigns.

However, the following steps describe a detailed explanation:

1. Create a short list of your own mailboxes with different providers.

You need different mailboxes with the largest providers, and also the mailboxes with providers that are most relevant to your target audience.

By default, the list should include the following at least:

• Yahoo


• Microsoft


• Gmail

2. Send a regular test campaign to your small list and note whether it was delivered, bounced or filtered to spam at particular mailboxes.

This part is important as the starting point of your testing.

3. Send a regular test campaign by changing the subject line to something very generic and neutral.

For example “This email is a neutral attempt” is a perfectly good and neutral testing subject line. Even though chances that changing the subject line will result in a change in filtering are minimal, it is still always wise to confirm that.

4. Send a campaign with a default template, without any of your original content.

This step will help identify whether the problem is in the content or not. The problem could lie in the infrastructure or the template itself, however, by sending a campaign with a default template will confirm that the content is OK from a deliverability perspective.

5. Identify the part of your email content that could cause the most issues.

In most cases the content element that is causing issues will be the part, which differs from the usual content you send. The most efficient way to identify the problematic part is to prioritize:

1. Remove the block with the most URLs (especially if it has a variety of different URLs).


2. Remove the blocks with most written content.


3. Remove images (unless having an image in your newsletter is unusual, you could start with removing the image block).

Example 1.

In e-commerce email marketing messages, the footer is most likely to cause problems, since it usually contains links to social media, important unsubscribe URL, and exceptional phrasing like physical addresses. In most scenarios of e-commerce email marketing message, the content will be the items for sale. So, to fasten the identification of problematic content, you will need to remove the least common content blocks.

Example 2.

In informational marketing messages, the block that could cause most issues could be either with large images or blocks with a lot of content and different URLs.

6. Keep removing one unusual content block at a time. When you remove a content block that you consider to be different from the remaining blocks - send a test campaign.

Removing content blocks one by one and sending test campaigns, eventually will result in no filtering or bounce by the email provider. So you can be sure that the problem was solved due to one of the changes you applied.

7. When you have identified the problematic block, start looking for particular parts in it causing the problem.

The same method applies here - remove parts of the last removed block instead of the whole block and send a test campaign.

8. Ensure that the block causing the issue has been identified by re-adding all other contents except that last content block.

If your test campaign with all the content - except the the block causing the issue - goes through without any problem, you have found the problem. However, if adding the rest of the content results in the same filtering or bounce result as for the first time, it may mean that the problem is either repetitive in your content or a sum of more than one problems is present. This can also be confirmed by applying the same method of testing, just it adds more complexity and more steps.

9. To have an end-to-end proof that your test is correct, use the only block probably responsible for the issue.

Use only that single block, or the part of it you have found to be troublesome. By doing so, you can finally confirm which particular content block is causing the problem, and you can either remove it or fix it.

Professional tip:

Change the subject line of your campaigns by defining which content blocks you remove in that particular test campaign. This will help you to track the progress and whether the changes you make impact the outcome.

The following image shows an example of testing the previously mentioned Gmail bounce. The rest of the tests bounced, except these. With the help of the subject line it is easy to pin point what content is being removed and that helps to come to a conclusion.

When changing subject lines, keep in mind to use only words and no special symbols. Also, keep it as short as possible. This ensures that the change in subject line will not affect your overall testing.

If you followed all these steps and were not able to indicate any issues with your content, please open a ticket to SAP Emarsys deliverability team for assistance. It could be that the mailbox provider in question is falsely flagging your emails as spam or the issue is more broad and related to sending practices, marketing strategy, IP reputation, etc.

Example of this methodology in action

A sender was not receiving their test campaign because it was bouncing with the following error on their Gmail address:

By using the method described in this guide (as seen in the screenshot above) the following issue has been identified:

• The empty UTM tag in the final redirect from "click tracker URL"

In this particular case, there were two invalid URLs in the footer to "Privacy policy" and "Terms and conditions." These look like the following:

• https://www.example.com/tos?utm_content=


• https://www.example.com/privacypolicy?utm_content=

By removing these two URLs, the message was placed  into the regular inbox. Re-adding the links in the footer results in the same bounce on Gmail and Office365.

URLs like the following were OK:

https://www.example.com/impressum?utm_medium=email

All links were redirected from a click-tracking URL.

The test was performed in an environment with a dedicated IP, fully DMARC aligned domain (same sender domain and same Returnpath domain) and link tracking domain that was the same as the organizational domain as well.

What’s next?

Please leave feedback below and/or hit the “like” button to show this type of content is useful.

You can ask questions and provide suggestions for helpful email deliverability topics in the Q&A area Q&A – SAP Emarsys Email Deliverability.

And if you’d like to find out more SAP Emarsys, you can visit the community page: SAP Emarsys CX.