Nic Zangre, VP of Customer Success & RevOps at CaliberMind, joins our host, Camela Thompson for this conversation on DIY Attribution. Nic shares his insights on why it’s critical for people to slow down and ask why, how those reasons change how we think about implementing attribution, and the instances where machine learning might not be the right answer and where it does fit really well.
Why is it critical for people to slow down and ask why?
Nic has accrued around 20 years of expertise in marketing operations. He started out at a company called Fast Search which was acquired by Microsoft. He got to see first-hand the fine nuances of a startup vs. enterprise environment for 6 years, implementing Marketo and Eloqua Salesforce environments. Nic has also run marketing operations and demand gen teams. More recently, he helped found CaliberMind, enabling him to solve some of the world’s toughest attribution problems.
Attribution is inherently tough and we are going to talk about why this is so today as a part of our DIY Attribution Series. “I think attribution is one of the most important aspects of any project. All of us have a picture in our heads when we hear multi touch attribution. What we think it is. And when we’re given a project, it’s really tempting to just start going out and checking out the technology to ensure it can do what we think it does or start kind of building that image in our minds”. At the beginning of this chat, Nic explained why it is essential for people to slow down and ask why before trying to implement attribution solutions.
“The few projects that I’ve been involved in that did actually fail were the ones when people don’t know why they’re buying what they’re buying. There isn’t a true understanding of the motivation. They just decide, oh! I think I need an attribution solution! A lot of my peers have it and other companies too. So I will immediately go look and research vendors without really knowing what problems am I hoping to solve or what efficiencies am I hoping to add. What am I hoping to measure to help my CMO? Those are questions you’ve to have baked before you go in earnest and evaluate build vs. buy.”
It’s interesting. I like how Nic called out it’s just the next thing we think we should do. However, maybe we shouldn’t do it yet and first ask some more questions.
“Yeah. I’ve heard, we want to prove ROI and we will implement the thing. They then realize that they’ve not put effort into documenting cost data for any of their campaigns or initiatives. Therefore, how do you know what your return on your investment is if you haven’t documented what your investment is? That’s just a small example. But you’re going to have to think of what are all the inputs into what I’m trying to get out and think through the inputs first before going to get a successful output.
Yeah. We have both heard, I spend a lot of money on digital advertising, but I don’t use Salesforce Campaigns or UTM. Oh! It’s going to make it a little bit harder!
“I wasn’t saying that working with a company like CaliberMind, you’re not going to get hand-holding and best practices and professional services. If you do end up going with a vendor, definitely check out the team that’s going to be implementing these things. Because chances are you forgot or you didn’t know some prerequisites before embarking on an attribution journey. So picking a team is important.”
Nic went on to point out some of the really great reasons why companies look for attribution.
“Oftentimes, there’s a catalyst that begins the search. It could be looking at some of the tougher challenges. We have to make cutbacks in our marketing budget. What are we going to cut and what’s actually moving the needle if we would have to choose between several channels? Maybe our marketing budget went from 100% to 50%. Do you think we are going to be able to cut the channels that are underperforming? So there’s an initiative like that.
“Or on the contrary, perhaps a company is doing really well and they just got a new round of funding. And we’re going to add channels and diversify our marketing portfolio. Therefore, you want to really have the infrastructure to report on it so you can rapidly experiment, adapt and reallocate things. So you aren’t so married to your long programs. You can stay more agile. I’ve worked with companies that have a large annual trade show, sometimes in the eight figures. Then, if that’s the case, you’re going to want to justify that revenue. What sessions worked best with the audience you invited to last year’s event and how they performed? Therefore, when planning a big event like that, sometimes, that’s a catalyst for needing multitouch attribution.
“Or if you just get tired of going to board meetings and meetings with your CFO and not being able to defend marketing’s contribution to revenue. So it’s that breaking point when you think, hey, I need something. I’ve to figure out how to tie my efforts to revenue.”
There are so many reasons and none of them are wrong. The point of this is just to really dig into that and figure out what you need to focus the most on. I hear many coming to us because they’re tired of single-touch attribution and they know that it doesn’t demonstrate reality since they’ve got a long sales cycle with lots of people involved. Or the ROI piece that Nic spoke about, that first use case was just like clean campaign optimization. What are we doing that works best and how do we double down on it?
How can each of those reasons change how we think about implementing attribution and does it change the touchpoints we’re including or how we’re including them?
Nic discussed how each of those reasons can change how we think about implementing attribution and whether it changes the touchpoints we are including or how we’re including them.
“I think the fundamentals are consistent regardless of what you’re trying to measure. However, an offline channel has a different process for capturing than a digital channel vs. a first-party vs. a third-party channel. How you capture data on your website is different from capturing it on a third-party site or content syndication play. However, all components are pretty similar. You have people in your database; leads and contacts and you want to start building a timeline for each of those people despite what the channel or platform is.
“In that timeline, an easy way to do it or a common one is using campaign members in Salesforce. That’s kind of I’d say table stakes where if I have a trade show or sponsor one and I get my attendee list. And I know if the people actually stopped by my booth, then I want a way to mark the date that all those people came to the trade show and what campaign is associated with that event. That’s what campaign members do. Then, the more sophisticated your channels get, and you might have chat functionality on your website. And they’re coming from a paid ad that has UTM in the URL parameters. So then there are more moving parts. They are setting a cookie when they land on the site of a first-party cookie and they’re registering the cookie when they chat using their email addresses.
“Then, finally, it will still go to a campaign member at the end. But there are more steps to get it to the campaign member vs. the simplified list import of booth attendees. Nevertheless, I think the blocking and tackling are similar at the end of the day.”
“Some folks when they think of attribution, think of capturing every single marketing touchpoint, particularly the digital ones. And then some people are thinking about a departmental representation of the effort that goes into closing a deal. Sometimes in those cases, I’ve seen people apply a higher bar and kind of bring in an intent level almost where they only want to include certain interactions that sales does. Let’s say we’re going to exclude emails after the opportunities are created because it’s mostly scheduling and contract negotiations. So I have seen that impact the data model a little bit.” Nic talked about other instances where he has seen interesting little caveats to how we should look at data.
“The buyer group thing is super interesting. Essentially, you’ve got what Forrester came out with, we’re no longer selling to accounts. You look at an account that might have 10,000 employees that we want to sell to. We are actually selling within a team of decision-makers or committees within that group. So if I believe that premise, then I must also want to measure my engagement in that buying group and touchpoints within it. That introduces a whole other level of hygiene and setup so you don’t have everybody with the email domain getting attribution. That’s a very generous model; you could get false positives if you sell to multiple teams within an organization.
“Therefore, what some of our customers have done or what I’ve seen is either by flagging, having a custom object, or a multi-select. Some way where on the contact you can associate which products of interest would be relevant to them. You can automate some of that based on what topics they viewed on your website. We had lead scoring more than 10 years ago and now we’re doing the more advanced topical scoring. Therefore, think of even within an existing account you want to cross-sell into a division and they’re buying their own product. Customer marketing is all the rage now and you want to know who should I sell product B to?
“A good way of doing that is building lead scoring specific around product B. So now I’m able to start identifying these other committees of people that are interested in this product, those other groups. When you look at not for profits is certainly a fun one. You think schools and you have school districts that share the same email domain across different schools. And then, those roll up to a district that could have an administration.
“Partner attribution is yet another really tricky one since if you’re a company that sells through a value-added reseller often you have a large percentage of revenue coming for marketing that you aren’t even doing. So those partners might give you lists of leads every month and now it’s not just what campaign was done. It’s how much money do we give this partner? What CRM accounts match the deals they’re registering? Therefore, there’s a hygiene process to attribution.”
So requirements gathering. We are not only figuring out what questions, but really the questions they’re trying to answer would drive a lot of what can drive it. Nic agrees.
“Yeah. think of an industry like HR Benefits. What I’m ultimately trying to measure is how many employees at my customers are using my product and then that’s going to drive how much I’m paid. So, the question could be, I want to drive employee growth at companies where they have my software. So that’s a very specific business question. But to do that now, what are all the parts? First of all, I need to know what content they’re exposed to through their training and portals in the app. Yeah, sometimes, the business question informs the inputs.”
Nic also thinks it could inform which attribution model he would recommend to a customer.
“For sure! I don’t think one attribution model is perfect for every business question. So if you really need to generate more top-of-the-funnel pipeline and perhaps you’re looking at a first touch model to start out, I need more people engaged in what channels and campaigns are really good at getting that first touch, that awareness driving the initial MQL or the lead capture.
“Or maybe I have many top-of-the-funnel leads, but I am having an issue getting them to an opportunity. Maybe I want to look at a middle touch model and I’m trying to assess what’s going to convert those known people to what’s the thing that gets them to the opportunity and that conversion. You know that even-waited is a nice Switzerland model.
“So sometimes, you just want to say, let’s treat all channels equally and we want to know the order of events, what’s the combination and mix that successful customers are taking? Even-waited could be good for that or if you have so much data and too many campaigns to sift through, perhaps you want to do something more complex like machine learning or a chain-based model. I think the state of your data and the sophistication of your team and the business question, all could lead to using different attribution models for sure.”
What are the instances where machine learning might not be the right answer and where does it fit really well?
Nic went on to talk about the situations where machine learning may not be the right answer and where it does fit really well.
“It does really well on lots of data. Let me caveat that by saying, expertly-labeled data, lots of expertly-labeled data. Therefore, when you were doing the original best practice around setting up campaigns if you have 300 campaign types and hundreds of lead sources. And you aren’t going to be able to group the data in meaningful buckets since they’ll be two in this bucket and three in this bucket instead of 200 in this bucket? So the machine sometimes has trouble drawing generalizations. If you’ve sparse data or poorly-labeled data, then you’re probably not ready for machine learning.
“And/or if you don’t trust your data. There’s definitely a data trust component. How are you going to trust a black box? It’s going to spit out incorrect predictions because they’re based on poor data. Garbage in, garbage out. Those are some of the situations where I don’t recommend machine learning. If you don’t have the fundamentals right, you’re going to overcome it and no one is going to get it. Even as we roll out machine learning, we sometimes start with some of the simpler models to gain that trust, and build that progression in crawl, walk, and run.
“You might be a good candidate for machine learning attribution if you’ve had maybe attribution built in Salesforce or a more basic version of it and you bought into it. Your company has adopted attribution and the concept of revenue and performance marketing. Once it’s all bought in and adopted and folks are data-driven, then your capabilities are in your current provider or current infrastructure.
“It’s not built for big data and machine learning. So Salesforce, even though they have products like Einstein, it all happens outside of the platform. Data that is in Salesforce, it’s a relational database. It’s not designed for big data. So when you start to think of what’s next, what’s outside of the CRM, perhaps your company has purchased Snowflake or another data warehouse, they’re starting to think holistically about data. It may be that you’re already using some form of attribution, then it’s probably a good signal that you could be ready for machine learning.”
I feel in my experience, it is just as much about explaining this in a way that makes sense, and getting buy-in isn’t about how advanced we can get so if we aren’t comfortable with something and we don’t understand it, we might be more apt to throw it out. It’s really important to gauge your organization’s maturity level and how easy this is going to be to explain.
“So some of the buy-ins might not only be on the marketing team. This could be buy-in from other stakeholders and other organizations. You don’t want buy-in from your IT team if you’re going to get this data in some warehouse and connect up Tableau to it or your BI tool that’s buy-in over there. We got the CRO and the revenue team getting buy-in, marketing getting buy-in, and finance getting buy-in. So there are a lot of boxes to check sometimes. Those other teams have many other priorities. They’re super busy.
“Another successful approach I’ve seen is doing a small pilot within your department and using that to get buy-in. Look! We built this basic version. This is conceptually what we want to do, does it hold water?”
I’m thinking back to the original use cases where people want attribution and one of them is proving ROI or marketing’s contribution to pipeline and bookings. I love what Nic said about a lot of times, it takes more than excitement to get buy-in. So it’s okay if you’re working in your department to pilot something. However, if you’re trying to put something in front of the board or if you’re trying to defend budget with finance and you independently construct something without their input. Nic explained why we don’t involve those stakeholders more in these projects.
“The pretty successful ones, they’re involved. There are a lot of committees, and a lot of stakeholders at these large, cross-functional projects at big enterprises. But I think part of it is because attribution could also have not a negative connotation, but more like a negative stereotype is a good way of phrasing it and that’s first of all, it’s marketing trying to take credit for everything. That’s assigning finger-pointing fingers and it could be very contentious saying that, oh no! We were the reason this big customer closed, not you. You didn’t do anything!
“So I think to have those conversations and perhaps to help make some inroads into those other departments. It’s important not to oversell attribution. Sell it as directional. It is not a perfect model and there are definitely other things. The more departments you include as campaigns in your model, the more holistic they will be. But even if you just start with marketing channels, you have a $1 million opportunity and the marketing team has done a very poor job of helping with this deal or at least marking in the system that they helped on this deal. There were no campaign responses, no events, and no paper trail that this ever existed except with the email that was clicked on. So you’ve got this attribution model saying, this email generated $1 million. Then, you have to be able to explain, okay, we didn’t have these processes and we didn’t have all the touchpoints in place. But directionally, this could be a great email, though we aren’t saying that sales didn’t do anything. Therefore, put a referral campaign in there as well if those are actually the things that are happening.
“Anyway, it is important to note that it’s directional. It’s by design imperfect. Every model tells a difference in solving a different question. So I think that having a little humility when going to those teams and not saying, I’m going to show you where all your deals come from and marketing’s trying to take all this credit. That’s not what it is about. It’s that we’re working as a team to try to generate more pipeline and customers. Trying to understand how we can work synergistically across the teams and do more of what’s working. I think it’s a good way of phrasing it.”
Those are all excellent points. I feel like those people who have attribution need to take a step back and ask themselves is it because it has failed me in the past or is it because we tried to roll it out in a silo and it got shot down and then it left bad feelings since I think a lot of people fall into that. Nic concurs.
“Yeah. I think of marketing ops mistakes I’ve made in the past, you go around, you do your whole roadshow internally to buy this vendor and you stick your whole reputation on it and then it falls on its face. And that’s how you learn it isn’t the vendor who might have been wrong. It just could have been the implementation or the politics. Maybe there were other reasons, bad data etc. So I think more objectively and say, I want to see that revenue table. Attribution is best, I have a marketer to prove the ROI of what I am doing. So you might not love it, but what are the alternatives?”
