The Liberating “What If” Analytics Cycle

Figure 1:  The “What If” Empowerment Cycle

I like to call this the “empowerment” cycle as it empowers organizations to freely debate different ideas without having to worry about which ideas are right ahead of time.  Consequently, organizations can embrace an environment of experimentation to encourage the free flow of new ideas.  Organizations can let the tests tell them which ideas are “right” and not let the most persuasive debater or most senior person in the room make that determination.  It empowers the organizational to feel free to challenge conventional thinking, and empowers creative thinking that can surface potentially worthy ideas.  No longer do you have to spend time debating whose idea is right.  Instead, put the ideas to the test and let the data tell you!

Let’s walk through an example of how one would leverage the “What If” Empowerment Cycle:

• Step 1:  Develop hypothesis or theory that you want to test.  For example, I believe that my target audience will respond more favorably to Offer A, while my colleague believes that Offer B is more attractive to our target audience.
• Step 2:  Create an experiment (e.g., test environment with corresponding test cases) that can prove or disprove the hypothesis.  We’d also want to be clear as to the metrics that we would use to measure the test results (e.g., click through rate, store traffic, sales).  In this example, we’d create tests for three test cases: Offer A, Offer B, and a Control Group.  We’d employ some data science methodologies to select our test and control audiences, and ensure that other potential variables are held constant during the test (e.g., same time of day, same audience characteristics, same channel, timeframe, etc.).
• Step 3:  Instrument all test cases in order to measure the results of the test.   In this example, we’d want to ensure that each of the three test cases was appropriately “tagged” and that we were capturing all the relevant data to determine who responded to which offers, who didn’t respond, and what were the results of their responses.
• Step 4:  Execute the tests.  For our example, we’d now start the tests, capture, assemble, and integrate the relevant data and then conclude the test.
• Step 5:  Learn/quantify test results and move onto next test.  Finally, we’d look at the results of the tests, examine who clicked on what ads, determine the final results, and declare a winner.  And more importantly, we’d then move onto the next test.

The beauty of an organization that understands and embraces the experimentation and instrumentation cycle is to make both decisions, and then see which one works out the best.  This “What If” Analytics Cycle leverages experimentation and instrumentation to empower the organization to freely explore and test new ideas, and empowers organizations to get moving by not get bogged down in “analysis paralysis.”  In fact, I’d argue that big data is the anti-“analysis paralysis” by giving organizations the data, tools and methodologies to test ideas, learn from those tests, and move on.