Data is a liability until it’s proven an asset: Tableau’s new rules for B2B research

Download the PDF version here. 

Leon: Hi everyone, I’m Leon, COO of B2B research company NewtonX. We leverage AI and automation to recruit professional audiences for market research for leading brands like Tableau. We’ve been working with Tableau since 2021, and it’s been a really amazing journey. 

Jason: I’m Jason. I led the Tableau Market Insights and Research practice for two years, and recently transitioned to Salesforce’s Strategic Research team as the Principal Strategist. I’m excited to be speaking on this topic, as I’ve worked in B2B research for over 15 years and have seen a lot of fallible expectations we need to overturn. Specifically with cost — researchers all know the costs associated with research, yet they are typically called “investments.” This narrative and thinking needs to change for research leaders to ensure that their spend generates positive business outcomes. 

Investing in 100% verified samples and Custom Recruiting ensures our research yields positive business impact.

Jason Talwar

Salesforce Market Strategy Principal Strategist

As you would expect, we have a strong data culture at Tableau. That means we build collective behaviors and beliefs to value, practice, and encourage the use of data to improve decision-making. 

That also means I’m often putting myself out on the line. Market researchers think research is the answer — that you can just ask questions and call it a day. But that’s not the case. There’s lots of due diligence you need to do to ensure you’re armed with the right insights. Data is only an asset when you can tie it to positive outcomes.

Throughout all this, I’ve learned key three components to ensuring your data is an asset and not a liability:

1. Invest in 100% verified samples
2. Test and validate your research
3. Partner with vendors who can stand by their data

Leon: Starting with the first point: when it comes to investing in samples, why is B2B data more expensive than researchers think—and why is it worth every penny?

Jason: At Tableau, we recognize that data is a liability and only becomes an asset when it leads to positive business outcomes. It costs a firm time, money and resources to collect, store, visualize, analyze, transform, and share data across an organization. 

If we spend $100k on a study using a poor sample, the total cost for the sample and professional services will produce no ROI and that $100k is lost. In fact, we would argue that it would cost more, as if we are utilizing poor data this could cost us more to retract or build a false strategy. Conversely, if we were to spend $150k on the same study with higher quality data, our strategies become more refined and an ROI can be achieved.

That said, it also comes down to budgets. We have to ask ourselves: what’s most important? It’s the data, the sample. And in the context of budgets, that means making tradeoffs for your end goal. 

For example, we have $500k to run a brand tracker. Last year, it went to one vendor who covered both sample and reporting services. This year, in order to ensure good data, we didn’t have the bandwidth for professional services. Our research team had to get creative with our own operations and analysis in order to invest in the best data possible. 

If we spend $100k on a study using a poor sample, the total cost for the sample and professional services will produce no ROI and that $100k is lost. Conversely, if we were to spend $150k on the same study with higher quality data, our strategies become more refined and an ROI can be achieved.

Jason Talwar

Salesforce Market Strategy Principal Strategist

Leon: So, what do you do when your research isn’t hitting the mark? Where are researchers stopping short when instead they should be going the extra mile?

Jason: It’s in testing and validating your research. You may find that your research is not hitting the mark, or not going far enough. That’s because most people stop at the research and say, “I have the answer.” I question that. I’d respond with “show me” or “validate it”—and if you’re not using a good panel partner, you surely will not be able to validate.  

Why is validation imperative? It’s because I need to give our stakeholders the best possible data to make the best possible decisions. We live in a highly competitive environment, and any intelligence that we gather will feed into our strategy. 

So if I’m not confident in the data, then I don’t want to be sharing it with my stakeholders. That’s happened a few times — where I’ve seen a research study and said, no, we just can’t share this. And we sometimes have to throw it out. 

There are a lot of organizations that don’t trust the data, and thus their insights function. This makes it extremely hard for researchers to get a seat at the table.

That’s why we work exclusively with vendors who can validate respondents in the B2B space. Fraud rates are much higher than you think, and if your supplier cannot verify a respondent through email, LinkedIn or other means then you are increasing your risk that research will remain a liability. There are very few companies that can do this, and I find NewtonX and their Custom Recruiting approach to be very reliable.

Leon: This is where it’s rewarding for us as a research partner. At NewtonX, we’ve built the Knowledge Graph, which is our proprietary search engine scans 1.1 billion professionals across the world to pinpoint the exact professionals our clients need to speak to. We call this approach Custom Recruiting.

Custom recruiting enables us to deliver 100% verified research. Because our expert outreach is so targeted, we’re getting in higher quality people. Our high incidence rates prove we are only reaching out to exactly the right audience, because we correctly translate your requirements into specific search parameters to ensure that we get you these right people. 

Over the course of audience recruiting, the base of any discussion is always: What are the goals? How many people did we reach out to and how many people responded? How many were screened and how many are passing the screeners? And then we’re looking at the metrics and asking, where’s the gap? Do we need to increase the volume or search parameters? We’re able to ask these questions because we screen and verify all respondents to confirm they are exactly who they say they are. This ensures we’re delivering our clients data that they can confidently use to make strategic business decisions. 

It’s also exciting for us to see the impact of the data. Many of our clients have reached a breaking point. They’ve thrown out their data countless times, wondering what they’re going to say to their stakeholders about the sunk time and resources with little actionable data to show for it. They’re frustrated and demanding: there must be a better way. B2B research must change, as their careers depend on it.

The good news is, the better way is out there. With Custom Recruiting, you can source decision-quality data that restores trust in your insights organization. I’ll pass it to Jason to reveal an experiment he ran to prove this.

Jason: Thanks, Leon. NewtonX has been a key partner to us here. Sample quality is highly problematic throughout the industry and even top suppliers have major issues with fraud and controls with their sample. We were so frustrated that we conducted a blind A/B test, which you can see the results of below.

We’re looking at 20-30% deltas in data quality between verified and unverified samples. The scale of these effects is highly concerning. With traditional top tier vendors, you’re paying for a liability, guaranteeing negative business outcomes.

Using brand research as an example: With our target audience, we’ve historically seen relatively high levels of consideration for Microsoft & Tableau, and significantly lower consideration for competitors (Company 3 and Company 4) based on market share and sales data. The top tier vendor study contradicted this by deviations of up to 33% compared to the verified sample. 

Additionally, with so called top tier vendors, we’ll see a halo effect on the leading brand Microsoft due to their general market equity. But if you look at the verified sample, the halo effect dissipates quickly because the audience knows the brands in the space. Awareness, usage, consideration, and brand love accurately match up with market share and sales data. These are the reliable insights our stakeholders need to make critical business decisions.

In closing, here are two takeaways to keep in mind to help with ensuring your research becomes an asset:

For researchers: Invest in decision-quality data.

Using poor sample means that the data will always be a liability. Working with NewtonX, I feel confident on the data that we didn’t have to really make excuses as to, “hey, the qual didn’t match the quant, or why is this showing up when it really shouldn’t? Do we really trust our data?” This is where validation is imperative.

For vendors: Commit to excellence in your data.

This means developing the technology and operating models to yield good data, not retrofitting traditional panel approaches. That’s the difference between totally useless data and absolutely actionable data.