Blog Striving for Analytic Maturity

Posted September 15, 2017 by Lyndsay Noble

Are You Testing and Measuring Your Digital Universe?

Businesses are typically built on relationships. It stands to reason then, that activities undertaken with the goal of building familiarity between your company and your client are of utmost importance. Sales and account teams call and try to get face-to-face meetings. Marketing teams send emails, post content, and measure their performance on clicks and downloads. However, completed phone calls, pleasant meetings, and internet clicks aren’t the true goal. Teams undertake these activities with the final goal of generating revenue for their business, and history has shown that these activities can contribute to top-line revenue.

There are four basic questions you should ask, and the ability to answer each of them is related to the analytic or digital maturity of your organization.

Question 1: How effective are your sales and marketing activities, in general, at driving revenue? The Holy Grail for answering this question is a multi-touch, online, and offline attribution analysis. If that sounds complex, that’s because it is. However, there are incremental and less-complex steps that can be taken to understand the contribution of different touchpoints to the eventual sale. I recommend starting your measurement with control group testing (also called A|B testing):

  1. Stratify your population.
    1. Identify the characteristics of your customers that are used to differentiate them within your organization. Some examples might be assets, opportunity, segment, firm, and territory.
    2. Identify the touchpoints that your customers receive that could “muddy” your evaluation. Some examples are customers who met with someone from your organization within the last 30 days and/or had a phone call within the last seven days.
    3. Assign each customer to a strata (or group) based on their characteristics and touchpoints. There may be only a few or literally dozens of groups here.
  2. Select your control group.
    1. Decide upon the size of your control group. I suggest 5% to 10% of your population.
    2. Select that percentage of customers from each stratum; these are the members of your control group.
  3. Validate your control group.
    1. Compare your chosen performance metrics between your test and control groups from recent initiatives. Look at average recent purchases and redemptions, number of calls and meetings, email clicks, and website visits.
    2. If the two groups are similar on your chosen performance metrics, you are ready. If they are not, consider adding those performance metrics into your stratification criteria and go back to step 1.
  4. Execute your touchpoint initiative.
    1. The control group will not be included in the specific initiative.
    2. The control group will be treated exactly the same as the test group in every other way (customer service, other initiatives).
  5. Evaluate the initiative.
    1. Ensure that you have allowed time for your typical sales cycle between the end of the initiative and the evaluation period.
    2. Compare inflows/revenue between your chosen test and control groups.

For more information see:

Question 2: Which activities are most beneficial at each point of the sales and marketing funnel? You can answer this question using the same methodology as the preceding general revenue driver question (attribution or A|B testing). The only changes are the performance metrics being evaluated and the population size used for evaluation.

Desired Funnel Stage Example Population Example Performance Metric
Awareness All potential customers who have never interacted with your brand Click through rate
Consideration All customers who have interacted with digital marketing but have never talked with a sales rep and have not purchased in the past 12 months Phone call completion rate
Intent All customers who have interacted with digital marketing and have personal interaction, but have never had a meeting and have not purchased in the past 12 months Meeting completion rate

Question 3: Which customers are most responsive to each type of activity? There is no one way to derive this information. Statistical-segmentation algorithms and machine-learning methodologies will be most successful due to the complexities in the data. When you have reached this level of need within your organization, you will likely have either built a strong analytics team and/or have a valuable relationship with a third-party analytics vendor(s).

Question 4: When was the last time you measured any of this? None of these evaluations are a one-time thing. Each one needs to be repeated regularly – your customers change and you will continuously need to adjust your message based on the environment, customer needs, and your objectives.

If you are using A|B testing for evaluation, you should be using a control group for every initiative. You do not need a new control group for every campaign, but you need to refresh your control group at least quarterly. Additionally, your initiatives should be coordinated such that they do not impact your ability to measure. If you are using attribution analysis, you can run your analysis annually or biannually, using the data from the previous period to aid you in making decisions for the next six to 12 months.

That is a whole lot of information in “four basic questions.” But keep in mind, that gaining “maturity” in anything is a process – a process of changes to reach certain goals. And while data and analytics should be at the center, learning the best way to handle those initiatives may take outside resources, additional training, and/or more advanced technology. Start small, show results, and gain support for bigger future initiatives. Eventually you will be at the top of the maturity curve.

“It is also very difficult to understand the effectiveness of our actions without measurements.” - Steve Killelea (2007) Peace and Sustainability: Cornerstones to survival in the 21st century

Lyndsay Noble headshot
Lyndsay Noble
Lead Analytics Consultant
FSG - Consulting
DST Systems, Inc.

categories: analytics

The views expressed in this publication are solely those of the author and do not necessarily reflect the position or policy of DST Systems, Inc. or its affiliates, subsidiaries, joint ventures, officers, directors, or management.

Of Interest...
  • Usher in a New Era of Actionable Insights
    Learn More
  • Blog: Can you Define and Track Your Unique Customers?
    Learn More
  • Digital and Analytics Quiz
    Learn More

Digital Edge Blog Subscription
Complete the form to receive Digital Edge newsletter by email


For More Information