Misha Salkinder, Head of Customer Data Strategy at CaliberMind, joins our host, Camela Thompson, in this episode of the Revenue Marketing Report. Misha shares how to use attribution models to answer different questions and why changing your model may not always be the right choice.
You made it to day three. I know it has been a marathon. Misha, thank you for hanging in there. Let’s talk about why you shouldn’t over-engineer your attribution model. This is dangerous territory and I am excited to wander into it.
“Likewise. Let’s do it.”
What are some of the signs? Let me step back for a minute. Before we change an attribution model, what are the questions we should ask when people are saying it’s wrong and we need to do X, Y, or Z differently?
“There are times when tuning an attribution model to answer a specific question is a good idea. I’ll say I am a big fan of it. I think as you grow with your attribution and analytics journey, the good leaders in this space don’t just take a model as it is. They think about what are the inputs and is it answering the right question with the right methodology? I’ll just preface with that. With that said, I’ve seen many organizations start their attribution journey or even be well on it, but trying to have attribution match their expectations.
“It doesn’t begin with a question, but rather it almost starts with the answer, and then they say, let’s have this model give us the answer. Let’s have the data give us the answer. That to me, is slightly dangerous territory because what happens in engineering attribution models is you can tune the weights to tell the story you want to tell. You can remove or include events to tell the story you want to tell. However, I think that you might be stepping away from what is actually happening or from the natural utility of the model when that happens.”
I’m putting back on my sales engineer and onboarding hat, but one of the questions I like to ask is what are you trying to solve? What is broken? Who is pushing back and why? Because I think two things I saw that were the biggest red flags were, one, we want to wait for something heavier since we spend more money on it. Two, I want to “get credit,”, so I need my activity to show up. The problem with that is you need to keep in mind as Misha said, what are you using a particular model to answer? If we are trying to figure out all the most meaningful touchpoints, which are connected to winning deals.
I was never a fan of including website data and this confused many people, but I never viewed that as a meaningful touch. I wanted to focus on hand raises or things where people were actively saying, I want to do that. Now, that doesn’t mean I didn’t have a model that focused on website data as it related to opportunities, that was to help me answer a different question. I think there are red flags, there are green flags and there are times when we say, as Misha said, there’s absolutely utility in doing that. For example, Misha mentioned yesterday that your attribution model can blow up with high-volume touchpoints that may not necessarily be as meaningful as some of the transactional touchpoint website visits, for instance.
“Yeah. I would say even email clicks will probably fall in the same category for me.”
Yes, absolutely! Now is there utility in looking at that data? Probably not when you’re talking about what is most effective at generating pipeline and bookings.
“Yeah, it’s interesting. I know you are a big fan of even weighted models and I am as well. There is something to be said, particularly in the validation phase of attribution models to do the math easily. You have a $10,000 opportunity. You have two touches and the logic behind it all starts making sense that each one of those has $5,000. I get especially weary when a model gets over-engineered using weights. We can have a conversation about what data should and shouldn’t be included.
“To Camela’s point, there are certain touches that are high volume and one solution doesn’t include them ever. Another solution might be, well, can we identify the high-intent signals and only parse those out? Let’s say you’ve got a specific webpage that’s ungated, but we want those. That would be, I think, an interesting move, but to say sales isn’t going to buy into this so we need to give them higher weight to their touchpoints. At that point, I get a little bit concerned because it is a matter of whether there are better levers to move, which are better at tracking, perhaps better campaign membership recording, all these things that can help you include more of the high-value touchpoints in your model, which will live longer. It will be useful for much longer than if you say, we’re going to 10x the value of these touches.”
There is a phrase that Misha used, sales will never believe it or sales doesn’t trust it. That is what I have found at the root of many engineering journeys where we’re trying to get other departments buy-in. We pulled up attribution, they pulled up a specific deal and then they blew up. I think that’s why I like to ask who has a problem with this model and why. Because if it’s the sales team, you need to have a completely different evaluation happen.
Are you including their activity? And they expect to get “credit?” And is that the main problem or should we not be talking in terms of absolute dollars in front of them and just say, this has X times more likelihood than this thing to help push a deal over the finish? I think that’s such an interesting call out and have you seen that as well? We are bringing sales into it.
“Yes. Two thoughts come to mind. One is that, and we spoke yesterday about funnels, and this is exactly why the lines between the two miles blur for me. It is possible that splitting dollar amounts between a number of touches might give you certain answers that could be useful tactically. But, it’s possible that on one screen, you need to look at attribution models, and on the other screen, you may want to look at all funnel analytics to see what volumes are we talking about here? Oh, wait, sales is getting a ton of attribution perhaps and you’ve put strong limits on what kind of marketing touches you are allowing to be included. But you’re not doing so on sales touches. That could very much happen.
“The other thought I had as a recommendation for the marketers out there. Whenever you are presenting an attribution model or any data that’s the output of an attribution model in front of sales or maybe in general in your organization, typically, the first screens that come up are overview screens. Here’s my attribution for the quarter as a whole. I would recommend to take a step back and actually show the working of attribution, showing an event timeline and saying here’s an opportunity, and here are the five touches, and here how it was distributed. I think explaining even at the most basic level how it works can help so much with the buy-in of these, hopefully, non-over-engineered attribution models when you look at them in aggregate. Oh yeah, this touch, we’re getting attribution to this touch since it is in this window or we probably didn’t consider a long enough sales cycle. There are levers that you can influence and move, but they’re hard to see when you are looking at data in aggregate without digging into an example or two to see how the model functions.”
I think the thing that pops to mind here is so many marketers bring in attribution and treat it as a marketing project and then try to display the data and use it cross-functionally. What I mean by that is here’s our share vs. sales and they don’t involve the sales team. I’m not saying don’t try to incorporate sales data and talk about it that way. You can’t do it in a silo. It’s never going to work, ever.
Another thing I would love to touch on is the even-weighted piece that comes from years of trying to explain multi-touch attribution and failing. It is a complex topic. I tend to favor models that are more straightforward to explain and then remove touchpoints. I think in our models, we included solution pages and a couple of other pages on the website, but everything else was excluded. What was critical was the ability to say, this is an estimate. Sales agrees, they are okay with this model and we use it to make these decisions. Then you have credibility, but I feel like trying to go it alone is just misstep number one.
“Yeah. I completely agree. I think if we’re to roll out attribution modeling for an organization, one of the first stakeholders that should be included are the end users, and likely that includes someone from sales. I want to build this right for you to be useful. I want to begin from the get-go. What are the touches? For example, one thing marketing is known for is their granularity and tracking, and then we lump in, at least from my experience, there’s a sales campaign, if you can get your data on it that gets generated. So you’ve got a huge bucket of sales vs. this granular data of marketing. If you can actually help sales, can we go back to it for you? Can we help you get well-versed about the tracking of smaller campaigns for your own communication, for your own back and forth?
“If we can get to that, then all of a sudden you’ve got a true stakeholder that is interested in looking at the data because they may make a decision or they can all of a sudden say, well, interesting. These sorts of communication seem to work better than those in these stages of the funnel. That’s great! As opposed to a question of us vs. them. We’ve got a bucket of this and we are going to try to optimize this and perhaps lower the value of it so that they feel better. You can actually use analytical tools and data to help optimize sales motions as well.”
Yeah, we all have to be rowing the boat in order to cross the finish line. It’s not a matter of credit or how much credit, it’s can we figure out what’s working and what isn’t. I like that. I’m thinking through onboarding scenarios and it was interesting to me. I think it’s just human nature. We would talk to maybe demand gen and operations and they would be adamant that we’re using this for campaign optimization. We are going to focus on marketing activity. We are not going to use it externally. The minute the VP of Marketing or CMO came online and I asked, what are you planning on using this for? They would immediately say, I want to take it to the boardroom and I thought, well, back to the drawing board.
“Yeah.”
Otherwise, if you’re looking at attribution, talk to everybody and see how they’re going to use it. Get in front of those landmines before they pitch it to sales and they won’t use it anymore since sales didn’t like it.
“I couldn’t agree more. Some of the areas where these things can blow up, often have to do with expectations, mismanaged expectations, and working in silos. Like many things in life, I think it all starts with expectation-setting and communication. It’s amazing the data that exists out there today. I think it is the willingness to go and find out what would be useful because if you’ve got the right solution in place, you probably have the flexibility to build useful models for both sides of the organization.”
I like it! Misha thanks so much for stopping by the podcast.
For more content on B2B marketing trends, listen to the full Revenue Marketing Report episode at the top of the article or anywhere you podcast.
