Bid Experiment Results And Rule Implementation
Once your bid experiment has run to completion it is time to analyze and act on the results. This page explores the Bid Experiments results dashboard and how to create a rule for successful tests. For a full outline on how to set up and launch a bid experiment, please reference our bid experiment setup guide.
Analyzing Your Bid Experiment Results
After a Bid Experiment runs to completion or is manually stopped by a user, your Bid Experiment tab will produce the results of the campaign, which can be found at Bid Experiment > Dashboard. The results displayed in this dashboard should be used to help you decide which of the following actions to take:
Possible Actions
• Declare a successful experiment and implement the experiment settings as a rule
• Declare an unsuccessful experiment and retain your current bid settings
• Declare inconclusive results and re-run the experiment with a larger sample size
Using The Bid Experiment Dashboard
Your Bid Experiment Dashboard displays several charts that break down the performances of Version A (your current settings) and Version B (your bid adjustment settings). While we encourage you to closely examine each chart, it is important to focus on the Experiment Results and Explanation widgets for each experiment you run.
Understanding The Explanation Widget
The Explanation widget breaks down the performance of Version A (your current settings) and Version B (your adjusted bid settings) based on their average profits per 1000 pings. Furthermore, this widget explains the confidence level of the experiment to help you decide whether or not you should trust the results.
Acting On Results
Based on the results displayed in your experiment dashboard, you should be able to determine exactly how to move forward. If you still aren’t sure how to act on the results of your experiment, please read our two-part bid experiment case study and results analysis.
Bid Experiment Rules
Bid experiment rules allow you to immediately implement the settings of your bid experiment to all future pings that meet the bid experiment’s criteria. Once a bid experiment runs to completion and you determine that you would like to implement the changes, you can create a rule that applies your bid experiment criteria to all future pings.
Bid Experiment Rules
When Do We Suggest Creating Rules?
We suggest creating rules when the experiment proves Version B is better (considering Buying Rate or Average Profit Per Ping), the confidence level is above 95%, you want to save time on filter/margin configuration, or your profit margin is set on the SRC side and cannot define separate margins for consumer segments.
How To Set Up Bid Experiment Rules - Option 1
Open the Dashboard for the bid experiment you’d like to implement a rule for. From the Explanation widget, click Create a Rule. Click OK to confirm the rule creation.
How To Set Up Bid Experiment Rules - Option 2
Navigate to the Experiments tab and click the tool titled Make Rule From Experiment. Click OK to confirm the rule creation. Once your rule is created, you will be redirected to the Rule Management subtab.
Activating Your Rule
At this point your Rule is Inactive. To apply the selected experiment’s changes to future pings, you need to activate the rule by clicking the Activate Rule tool. Once activated, the settings for your selected bid experiment will now apply to all qualifying pings. This rule can be deactivated and deleted at any time.
Managing Your Bid Experiment Rules
As you run more experiments and implement additional rules, keep the following logic in mind: Rules are composed of attributes and an adjustment. The priority of a rule is decided based on its placement in the ordered list of rules.
Rule Priority Logic
If an incoming ping matches the first rule, it will be applied and no other rules will be checked. If it doesn’t match the first rule, the algorithm checks the second, third, and so on until it finds a matching rule. If no matching rule is found, the ping will be processed according to the default filter or source bid settings.
Frequently Asked Questions
Your Dashboard displays charts comparing Version A (current settings) and Version B (adjusted settings). Focus on the Experiment Results and Explanation widgets to assess performance.
Create a rule when Version B proves better, the confidence level exceeds 95%, or you want to save time on manual filter and margin adjustments. You may also consider it at 80-95% confidence if additional charts indicate reduced uncertainty.
Create a rule from the Dashboard Explanation widget or the Experiments tab. Activate it from the Rule Management subtab. Rules are prioritized by their position in the ordered list and can be deactivated or deleted at any time.
Ready To Get Started?
Fill out the form below to schedule a demo with one of our lead distribution experts.