Troubleshoot Automated Personalization

There are several activity setup changes that can decrease the expected time to build models, including the number of experiences in your Automated Personalization activity, the traffic to your site, and your selected success metric.

Solution: Review your activity setup and see if there are any changes you are willing to make to improve the speed at which models build.

• If your success metric is set to RPV, can you change to conversion? Conversion activities tend to require less traffic to build models. You do not lose activity data if you change the success metric from RPV to conversion.
• Is your success metric far down the sales funnel from your activity experiences? A lower activity conversion rate increases the traffic requirements needed for models to build, as a minimum number of conversions is required.
• Are there some offers or experiences you can exclude from your activity? Decreasing the number of experiences in an activity speeds the time to build models.
• Is there a higher-traffic page where this activity would be more successful? The more traffic and conversions in your activity locations, the quicker models build.

There are several factors required for an Automated Personalization activity to generate lift:

• The offers must be different enough to influence visitors.
• The offers must be located somewhere that makes a difference to the optimization goal.
• There must be enough traffic and statistical “power” in the test to detect the lift.
• The personalization algorithm must work well.

Solution: The best course of action is to first make sure the content and locations that make up the activity experiences truly make a difference to the overall response rates using a simple, non-personalized A/B test. Be sure to compute the sample sizes ahead of time. Computing sample sizes ahead of time helps to ensure that there is enough power to see a reasonable lift. You can then run the A/B test for a fixed duration without stopping it or making any changes. If an A/B test result shows statistically significant lift on one or more of the experiences, it is likely that a personalized activity is successful. Personalization can work even if there are no differences in the overall response rates of the experiences. Typically, the issue stems from the offers or locations not having a large enough impact on the optimization goal to be detected with statistical significance.

In Automated Personalization, the URL and template testing rules are added to the Target request entry constraint (for example, target-global-mbox), where they are evaluated only once. Once a user qualifies for an activity, the Target-request-level targeting rules are not reevaluated. However, the targeting audience is added to location targeting rules.

Solution: Add the necessary template rules as the input-audience of the activity. Audience evaluation happens upon each request/call.

This is expected.

In an Automated Personalization activity, once a conversion metric (whether optimization goal or post goal) is converted, the visitor is released from the experience, and the activity is restarted.

For example, there is an activity with a conversion metric (C1) and an additional metric (A1). A1 depends on C1. When a visitor enters the activity for the first time, and the criteria for converting A1 and C1 are not converted, metric A1 is not converted due to the success metric dependency. If the visitor converts C1 and then converts A1, A1 is still not converted because when C1 is converted, the visitor is released.

• If you are not able to see the preview in the new tab (due to browser cache), try refreshing two or three times. You can also copy the link and open it in a new browser or new session.
• Regenerate the Experience URL links if you changed any content and share the new links with your teammates.