Eric Westerkamp, CEO at CaliberMind, joins our host, Camela Thompson, Go-To-Market Thought Leader and B2B Insights Expert, in this episode of the Revenue Marketing Report. Eric shares ways every company can incorporate AI into their day-to-day workflows and why simpler methods sometimes are the better alternative.
Thank you for being an advocate for the show, first of all. Second of all, the reason I brought you here, there were several reasons, but the one I’m talking about is how you encourage your team to think about generative AI when it first came out and how they could incorporate it in their work streams.
“Yeah, It’s interesting that when generative AI first started to show up, we took a hard look at it, and because of the nature of some of the products we offer at CaliberMind, we have a pretty close relationship with Google and some of the things they are doing, we get access to some of the early releases of the technology. We have been taking a hard look at it and we found some very interesting things with generative AI.
“We were looking at it from two angles. One is how well it works at summarizing data, taking information, and allowing people to get it digested. The other was core to do what we did, how well is it analyzing data, taking information, and running analytics on it? We found a pretty significant gap between those use cases. When it comes to analytics and being able to take the data and to some extent, run math, we found that the models didn’t hold up.
“You had to get very specific in giving it information. I want you to run, here’s a table. Here is some data, run a standard deviation on it, and sometimes it would get it and sometimes, it wouldn’t often, it would say I don’t understand your question and that’s even when using some of the analytics models. We backed away from that.
“However, what we found out was that summarizing data and summarizing information, it was actually very good at that. We did a couple of things internally. We ran a hackathon and in it, we reached out to the whole company and said, hey! Come up with great use cases for using generative AI inside of CaliberMind and bring those back either for our customers inside the product or even, internally, how could we optimize things? We found a couple of neat use cases internally.
“On the software development side, this is probably not news to anyone, but we found that we are often looking at the configuration that is sitting in SQL files or how our business rules are developed inside of our pipelines, and often different teams need to look at the data. We found out that it was very easy for these generative AI models to take a look and say give me a summary of what’s going on here.
“For instance, we have a big customer, we’ve perhaps written some custom code to help modify their data. It gets pushed to another person in our data analytics team. They need to understand what is going on here. They would take that and drop it and say summarize what’s happening in here and give me the key points. It does a great job there.
“Another area we have found helpful was with taking information in large buyer journeys. If you think about the different customers that are moving through your buying process, they’ve got a lot of different touchpoints across different marketing campaigns, sales teams, and all of that. Oftentimes, products like ours and other products out there will put all the data on this timeline, but the challenge is a person needs to then look at the timeline, look at different individuals, see what’s going on, and try to extract out the meaning of all those different touchpoints.
“We took some of those, dropped it right into generative AI, gave it some context: This is the company you’re looking at here is some information about the company, and here are some of the things we’ve done and gave it the raw event timelines and said, tell us what’s going on. Summarize this for a salesperson and a marketer who is trying to target that organization and it did a relatively good job at that. It summarized data and told us which individuals it saw across the timeline, where they were most engaged, what they were doing, and what they were responding to.
“Then, when you gave it the persona of a marketer vs. a salesperson, for instance, it was able to parse out and say here are maybe some tactics that had been working with this customer that you want to go. Here are some people in the organization that perhaps you want to target in these different campaigns. It did a pretty good job there.”
I think that is interesting. I always saw it as having a lot of potential in two areas. One is the one Eric described. It’s handy in pointing out things that I miss. As an analyst, my value usually was pointing out these things. I am not worried about it taking over quite yet since it doesn’t quite tell you what to do next. Although Eric had a great point about what these people were doing with your campaigns and pointing those out.
The other area I was interested in is unstructured data. So freeform text, trying to figure out some kind of pattern in that. Have you explored that at all?
“Yeah, a little bit. One of the things that we have been looking at is, again, in our marketing data warehouse for our customers, we often have a lot of information about what’s going on. We may have the campaigns and campaign information and data, but often we’ve emails. We’ve all texts that are going on in email. We’ve started to experiment with looking back through here and saying, okay you have your whole BDR team and they’re doing outbound. first, which ones are having success in responses, and then what is it about those outbound emails, their outbound texts, what’s a commonality between these different or how do I take these twelve successful outbound campaigns and summarize them into what’s going on and perhaps create some new campaigns and all of that”
“I think it is the early days for some of that, but we’re starting to see that you can get some decent results out of that and it can go through these large sets of unstructured data and say, please help me understand the commonality between these. I’ve got hundreds and hundreds of emails that went out, maybe thousands of emails, but perhaps only fifteen and twenty different email campaigns. This sales rep and that rep had high success rates in getting responses, help us understand the differences between this batch and that batch. If I needed to write it using these templates, things like that.”
For the marketers wondering what does this have to do with marketing? Let’s talk a little bit about that. The reason why I nudged you about unstructured data is I’m thinking of Gong and all of these call recording tools and all of this language, this rich language that we could tap into potentially and determine dispositions, competitors, and keywords. There’s so much potential there.
Then, in terms of what Eric was describing initially, summarizing report results, tell me a little bit about what you see with customers.
“Yeah, first off, I will say that most of our customers are hyper-interested in what is going on and how they can take advantage of it. The use cases are just emerging. The challenge you get with generative AI is that it tends to be very spiky. Hey, it gives us some great results now, and now it is giving us no results and now it’s giving us some great results. And now I get something that I’m not even sure what it is. It is tough to operationalize at this point because of that.
“Technology is moving so fast. There is ChatGPT 4.0.out and now they’re talking about version 5.0 and it is moving fast. Many people are waiting until the next version since that is supposed to be so much better. Now, how are marketers taking advantage of it? I think that they are using it to take information, content, and campaigns that they are seeing are working and creating new versions of it, that kind of embody some of the characteristics of these.
“So they feed in, I had this campaign, I set all these emails out through products that send out all your emails and things like that for the sales team. They will say, here are the twenty emails I sent out. This is what is working, perhaps here’s some campaign or a piece of content that we wrote that is resonating, okay, I want to write a new piece. I want to give you the five to six bullets I want to write about. Here are some samples of content that we liked, help me rewrite this using this tone and the things that were successful over here. We’re starting to see people experiment with a lot of that.
“The key from my perspective is the measurement. You have got to figure out which of these things are working. How do you know if I rewrote this thing using ChatGPT 4.0 or Google Bard, whether it’s working or not? I think the feedback loop that people are starting to aim at is if I can take generative AI, I can measure its success. I can then feed the stuff that’s working back into it to create new content. That’s the feedback loop that you want to be able to build, but it requires combinations of AI plus measurement tools whether it’s attribution or engagement scoring, you need to measure success. Otherwise, you don’t know which pieces to double down on. You need to tell these systems this worked or this didn’t to get more content out of it.”
And if you know a topic and ask Chat GPT about it, you might get the right answer, but you may get stuff that’s all over the place. So having that feedback mechanism is important and telling it what doesn’t work and what does work in the same chat is that people underestimate how much information it needs to work well.
“Yeah.”
That is the same with machine learning. It takes quite a bit of data to be able to train it. I think there are a lot of marketing teams out there that probably have enough data collecting so much, but what are some recent developments your team has seen in the machine learning space?
“It is super-interesting that there’s so much hyperfocus on generative AI, what it is doing that you don’t hear people talking about standard machine learning algorithms. I’ll be honest. My opinion is that the majority of the value you can get at this point in time is based on a sort of standard regression analysis, standard deviations across your engagement, and things like that. There’s so much you can do with that.
“Marketing teams, to be blunt, are so far behind many other industries and taking their data and even applying these standard algorithms to it, there’s so much value they can get out of doing that even before they jump into how to leverage generative AI. How do I use these large models and things like that? First, is to measure what is working and what is not. The advances we’re seeing is that it is becoming easier and easier. Everyone has an API so it’s becoming easier to acquire data.
“Any product I use now, I’m using Gong. I can get the data out of Gong. I am using Apollo for emails or outreach or something like that. I am using one of the marketing automation platforms. There is so much data I can get, but the challenge is that you’ve all this data, it’s very distributed, they’re all in different formats and they’re coming in different ways. You have to stitch it all together and it comes back to that.
“Now, if I can get that data into a timeline or some consolidated type of information, I can then apply machine learning to it since the data has been normalized. The machine learning algorithms need data that is shaped consistently. You need to take the data from all these platforms and shape it in a way that I can then feed in machine learning and then try to create interesting results. Some cool things that we are seeing out there.
“Again, we work closely with Google. Google has released a whole new framework called BigQuery ML. It is effectively SQL where you can write and embed machine learning training and everything is right inside the SQL. We have some of the most advanced customers who are using this on top of our platform right now, where they can then take a common set of data and begin experimenting. We have customers who are saying, I want to look at all these leads and determine which of them are most likely to convert to MQLs and SQLs using a large feature set to do that. Think about it.
“If I got those leads and they are coming in from twenty different sources with different data sets, I need to normalize that data to feed it into that algorithm. But it’s becoming much easier with these frameworks to take the data when you’ve it and feed it in and train it and come up with some interesting results. The next question is what do I do about it? How do I act on that data? Hey, now I’ve got these leads that I think are good, what do I do with that data? Should I send it to marketing? How do I operationalize it?”
I think a lot of marketing leaders struggle with prioritizing data projects since you’re under so much pressure to build pipeline now. If you can put the money in that direction, it’s hard to rationalize making a long-term investment that you won’t see pay until perhaps after you’re ten years over. I don’t know.
However, the benefit of that is huge, like Eric said, you can see what’s working and what isn’t as opposed to throwing things against the wall and moving to the next thing you’re just going to throw against the wall.
“I think I read an interesting article, I think it was from Bain & Company. They said something like that. The marketing teams that leverage and have good control of their data outperform the other teams by 40%, 50%, or something like that. They’re getting significantly better results, but getting there is challenging. Data is messy. You have got to put processes and systems in place to get there, but once you do, the outcomes are much stronger for those teams.”
For more content on B2B marketing trends, listen to the full Revenue Marketing Report episode at the top of the article or anywhere you podcast.
