Once your bid experiment has run to completion it is time to analyze and act on the results. This page explores the Bid Experiments results dashboard and how to create a rule for successful tests. For a full outline on how to set up and launch a bid experiment, please reference our bid experiment setup guide.

Analyzing Your Bid Experiment Results

After a Bid Experiment runs to completion or is manually stopped by a user, your Bid Experiment tab will produce the results of the campaign, which can be found at Bid Experiment > Dashboard. The results displayed in this dashboard should be used to help you decide which of the following actions to take:

  1. Declare a successful experiment and implement the experiment settings as a rule
  2. Declare an unsuccessful experiment and retain your current bid settings
  3. Declare inconclusive results and re-run the experiment with a larger sample size

Using The Bid Experiment Dashboard

Your Bid Experiment Dashboard displays several charts that break down the performances of Version A (your current settings) and Version B (your bid adjustment settings). While we encourage you to closely examine each chart, it is important to focus on the Experiment Results and Explanation widgets for each experiment you run.

The Explanation widget breaks down the performance of Version A (your current settings) and Version B (your adjusted bid settings) based on their average profits per 1000 pings. Furthermore, this widget explains the confidence level of the experiment to help you decide whether or not you should trust the results of the experiment.

Bid Experiment Results

Based on the results displayed in your experiment dashboard, you should be able to determine exactly how to move forward. If you still aren’t sure how to act on the results of your experiment, please read our two-part bid experiment case study and results analysis.

Bid Experiment Rules

Bid experiment rules allow you to immediately implement the settings of your bid experiment to all future pings that meet the bid experiment’s criteria. In other words, once a bid experiment runs to completion and you determine that you would like to implement the changes, you can create a rule that applies your bid experiment criteria to all future pings instead of implementing these changes by making manual bid and filter set adjustments in the boberdoo system.

When Do We Suggest Creating Rules From Your Bid Experiments?

  1. The results of the experiment proved Version B to be a better option either considering the Buying Rate or the Average Profit Per Ping as the main performance indicator and the confidence level of the experiment is above 95%. (You may still want to implement a rule if your confidence level falls between 80%-95% if the additional experiment result charts and analysis indicate a reduced risk of uncertainty.)
  2. You want to save time setting up new filters or modifying existing ones that match the consumer segment defined in the experiment.
  3. Your profit margin is set on the SRC side (instead of the filter set side) and cannot define separate profit margins for product categories or consumer segments.

How To Set Up Bid Experiment Rules

Implementing a Bid Experiment Rule is simple.

Option 1: Open the Dashboard for the bid experiment you’d like to implement a rule for. From the Explanation widget, click Create a Rule.

Bid Experiment Results

Click OK to confirm the rule creation.

Option 2: Navigate to the Experiments tab and click the tool titled Make Rule From Experiment.

Bid Experiment Eesults

Click OK to confirm the rule creation.

Once your rule is created, you will be redirected to the Rule Management subtab. At this point your Rule is Inactive. To apply the selected experiment’s changes to future pings, you need to activate the rule by clicking the Activate Rule tool.

Bid Experiment Results

Once activated, the settings for your selected bid experiment will now apply to all qualifying pings. This rule can be deactivated and deleted at any time.

Managing Your Bid Experiment Rules

As you run more bid experiments and implement additional rules it can become harder to manager your rules and know how each ping will be handled. Because of that, please keep the following logic in mind:

A ping matches a rule if all rule attributes have a known and equivalent match within the ping’s attributes.

The rule is composed as follows: Attribute 1=A, Attribute 2=B, Attribute 3=C, Adjustment=+10%

Bid Experiment Results

If there are multiple rules matching a ping, only the first matching rule is applied.

The priority of a rule is decided based on its placement in the ordered list of rules. If an incoming ping matches the first rule, it will be applied, and no other rules will be checked. If it doesn’t match the first rule, the algorithm will check the second, third, and so on until it finds a matching rule. If no matching rule is found, the ping will be processed according to the default filter or source bid settings.

Bid Experiment Results

Rules are not be considered for pings that are picked for an experiment.

Pings assigned to Version A should be processed using the original profit margin from filter set or source setting without the matching rule’s adjustment.

Pings assigned to Version B should be processed with the adjustment set in the experiment made to the original profit margin from filter set or source setting without the matching rule’s adjustment.

Original profit margin: $5
Experiment adjustment: +50%
Matching Rule’s Adjustment: +$1

Ping Matching Both Rule And Experiment Picked For Experiment Version Original Margin Adjusted Margin
A Yes A $5 $5
B Yes B $5 $7.5
C No - $5 $6

 

Best Price Offer Limits (both Minimum and Maximum) set during the Bid Experiment setup (step 4: Adjustment) will be taken into account while the rule created from the respective experiment is live.

Bid Experiment Results