Experimentation

Learn how to use Trophy to experiment with the gamification experience and increase retention and user engagement.

Trophy provides a core set of primitives to help you ship gamification experiences faster. But gamification is not something you can set and forget.

A key part of running a gamified platform is experimentation — tweaking and optimizing the user experience to ensure that users keep coming back.

Usually, you’d have to build these tools yourself, but with Trophy you get a simple experimentation stack out-of-the-box.

We have plans to extend Trophy’s experimentation capabilities in the future to support more granular A/B testing of your gamification features.

Control Ratio

Trophy has a built-in capability to automatically A/B test gamification features using a control ratio.

Found on the integration page of the Trophy dashboard, the control ratio adjusts the percentage of users that are assigned to the ‘control’ group versus the ‘experimental’ group.

Trophy returns the control attribute as true or false in most APIs that relate to user data allowing you to conditionally show gamification features to users who are in the experimental group.

If you plan to measure the overall impact of gamification features, make sure to only show those features to users with control set to false.

The control attribute

{
  "id": "user-id",
  "email": "[e-mail]",
  "tz": "Europe/London",
  "subscribedToEmails": true,
  "control": true,
  "created": "2021-01-01T00:00:00Z",
  "updated": "2021-01-01T00:00:00Z",
  "attributes": {
    "department": "engineering",
    "role": "developer"
  }
}

Trophy also doesn’t send any Emails or Push Notifications to users who are in the control group.

Analytics dashboards then compare retention and user engagement between users in the control and experimental groups allowing you to measure the impact of the features you build with Trophy and adjust mechanics where necessary.

Both retention and user engagement are important metrics to use when measuring the impact of changes on the user experience. The following sections outline what each means and how it relates to features you build with Trophy.

What Is Retention?

User retention is the percentage of users who are still using your product after a certain period.

Common time frames include 7-day, 14-day and 30-day retention with shorter time frames providing information on initial user experience and longer time frames giving insight into long-term product-market fit.

Retention Analytics

Trophy includes a retention chart for each metric on its metric dashboard and an aggregate user retention chart on the main dashboard.

The integration page allows you to control the time frame over which you want to measure retention through the ‘New User Activation Window’ setting.

What Is Engagement?

User engagement in Trophy refers to the average level of activity that your users show when using your product.

As Trophy tracks user interactions through Metrics, it can give you insights into how active your users are against each, and in aggregate.

User engagement in Trophy is the average total metric event value per daily active user. The formula is:

sum of all metric event values in a day / number of users active that day

Engagement Analytics

Trophy dashboards for each metric show engagement charts for that metric, and the main Trophy dashboard presents user engagement in aggregate.

Trophy also displays charts that show user engagement within users’ first days after signing up for your product.

We refer to this as ‘early engagement’ and it is user engagement as measured by Trophy only for users within the time frame set in the ‘New User Activation Window’ setting on the integration page.

This is a useful chart for understanding the engagement of users within their very first days with your product and to understand where blockers to initial user activation may be occurring.

Get Support

Want to get in touch with the Trophy team? Reach out to us via email. We’re here to help!