Episode 276 / Boaz Sobrado / Capital.com / Director of Marketing Analytics
How to Implement Predictive Modelling of Conversion Events
Subscribe to Our Newsletter
Boaz Sobrado is the Director of Marketing Analytics at Capital.com. He believes in growing as close as possible with data to spot trends and anomalies, as well as in studying the “classics” of business and marketing.
Reading old marketing books is a good strategy for uncovering timeless principles. Once you remove the blatantly outdated content, Boaz recommends a book like “"Reminiscences of a Stock Market Operator" as particularly insightful, as it touches on topics like word-of-mouth and consumer purchase instincts that remain relevant today.
In day to day marketing life, Boaz recommends that marketers develop a close relationship with data. That means delving into it on a daily basis, so you can spot trends and know how the numbers work. This will then lead to noticing anomalies and being able to see what others cannot - thus gaining a competitive advantage.
As for his shiny new object, Boaz has picked the predictive modelling of conversion events. This approach allows him to approach the lifetime value of clients differently, making algorithms work better for his business and evolving beyond a simple lead scoring method, Implementing this can be tricky, especially for teams that don’t have a data scientist, but Boaz offers some starting suggestions that anyone can start with and test, depending on your industry. Tune in to learn more.
Transcript
The following gives you a good idea of what was said, but it’s not 100% accurate.
Boaz Sobrado 0:00
If you develop relationship with the data, if you look at it every day, if you interrogate it every day, if you look at how different variables move together, you will start developing instincts around how things move together and what happens when all things happen.
Speaker 0:21
Are you a fitness loving foodie? Then defy snacks is for you. Defy offers real high quality chocolate you love, combined with functional ingredients, giving you as much as 26 grams of protein per bag. So you can be both a little bit bad and a little bit good at the same time as a female owned brand, defy dedicates a portion of its profits to other women owned businesses currently sold in the US. You can find us on defy with an eye snacks.com and follow us on social at defy snacks.
Tom Ollerton 0:54
Hello and welcome to the shiny new object podcast. My name is Tom Ollerton. I'm the founder of automated creative and this is a podcast about the future of data driven marketing, and I have the privileged position of being able to interview some amazing people from across the industry about their vision for the future of this industry. And this week is absolutely no different. I'm on a call with Boaz Sobrado, who is director of marketing analytics at Capital.com. So Boaz, for anyone who doesn't know who you are and what you do, can you give them a quick background?
Boaz Sobrado 1:30
Hi Tom. It's absolutely great to be on the podcast. I have listened to a good few episodes of your podcast, and I've really enjoyed it. I've learned a lot. So thank you for inviting me on. Yes. So in terms of my role and myself as an individual and as a professional, I work for capital.com and I'm responsible for marketing analytics. The way I usually explain what Marketing Analytics does is I use that old phrase that was attributed to a retailer over 100 years ago, which is, I know that half of my marketing budget is wasted. I just don't know which half. The way I explain what Marketing Analytics does is trying to figure out which half of the marketing budget effectively is wasted, and this is a much tougher task than you might think, because it's very much not obvious what exactly drives performance and what doesn't. So that is the sort of thing that me and my team try to help with here at Capital.com.
Tom Ollerton 2:39
So are you a marketing book reader? Or are you a... Do you watch videos or conferences? Or how do you get your knowledge about how to get better at your data driven marketing job?
Boaz Sobrado 2:53
There's... there's a wide range of sources. So I do very much enjoy listening to podcasts. I also spend probably more time than I should on Twitter, engaging in the conversations that are there around marketing, around technology, etc. But I am also a very big fan of books. And I'm a fan of books that are actually perhaps slightly different to the ones that most people would read simply because, I think the difficulty with marketing with books is that, particularly digital marketing, things move so quickly, so fast, that by the time that someone has the time to write a book and publish it, and then someone actually gets to the point of buying it, a lot of what has been written is kind of no longer actual or out of date, etc. So one thing that I've particularly liked about books is looking at books that have been published a long time ago. So you can very easily go to your local library, look up the marketing section and try to look at books that are 5060, 7080, even 100 years or more old. And the very interesting thing about them is that, obviously there's a lot of stuff there that's out of date, but it's very obvious what's out of date. So it's actually quite easy to get signal and the signal to noise ratio is a lot better. And there's a lot of things that if they were valid 100 years ago, they're very much valid still today, then they're particularly true.
Tom Ollerton 4:28
So how did you come up with that idea? And that's such a kind of odd, odd approach. Did you just stumble across that as a source of insight, or did someone suggest it to you?
Boaz Sobrado 4:39
So there's so this is something that I've heard recommended by other people in other fields as well. I mean, it's no coincidence that people to this date like reading the classics, so to speak, both in literature, but in other fields. But the way I've heard. Actually came across it is I was wandering around in the library, and I came across a very old direct response handbook from over 100 years ago. And as I started reading through it, I realized, Oh, actually, a lot of the stuff that is mentioned here, if it's not very clearly, obviously, out of date, then it's actually very useful. So that's how I stumbled into this little niche.
Tom Ollerton 5:29
That's interesting. I'm writing a book myself at the minute about the relationship between data, creativity and marketing, and it's really frustrating, because everyone wants to talk about generative AI, which I do as well, but by the time it gets published next year, then that that will be a whole different beast. You know, may have faded or morphed to, who knows? So, yes, you have to think like, okay, how can I put stuff in this book? This hopefully will be useful in 100 years time? Oh gosh, the thought of anyone doing that would be remarkable. But anyway, so moving on. So what is there a specific book that you would recommend?
Boaz Sobrado 6:03
So one of the books I recommend to all of the new joiners here at Capital is a book that's not actually related directly to marketing. It's called reminisces of a stock market operator, and it was written almost exactly 100 years ago. But although it's not directly about marketing, I think it touches a lot of marketing topics very well. It touches the importance of word of mouth, of how different narratives get constructed and how different narratives get disseminated. But also, and it's very, very powerful on this, it touches a lot on the instincts that drive people to make a purchase. And although the absolute details of the companies and markets and products that this book talks about are entirely out of date, you know, it's talking about companies that existed 100 years ago and no longer exist. Nevertheless, a lot of the sort of mechanisms and the procedures and the way this information spreads is very, very much topical, and I recommend it to everyone that joins the company.
Tom Ollerton 7:19
So what advice would you give to today's data driven marketer to be better at the discipline?
Boaz Sobrado 7:26
I think one of the big advantages we have today is that we're about whereas 100 years ago, I mean, this is something that's mentioned in the book. 100 years ago, you had to pay someone to make you a chart about whatever you wanted to see. And it took a couple of days, or maybe even weeks to get your chart. Today, you can sit down at your laptop and look at your chart as many times as you want. And I think that starting that your day like that, starting your day looking at dashboards, interrogating them, digging through them, seeing how different things move together, is basically the fastest and easiest way you can have to becoming a good data driven marketer, because if you develop a relationship with the data, if you look at it every day, if you interrogate it every day, if you look at how different variables move together, you will start developing instincts around how things move together and what happens when odd things happen. You can look at a dashboard one day you can see, oh, that that's kind of odd. Usually when that line goes up, this other one also goes up. But today it hasn't gone up, you know. And then you have to interrogate it a little bit, dig through it a little bit, and then it turns out that, for some reason, there was an issue with the Android version of your app, or there is some language that has been mistranslated to, and there's been performance issues in some countries, or something like that. The point being that if you develop a good, instinctive familiarity with the data, and you can only do this by looking at it every day, you will start spotting things that other people wouldn't spot, and that will be an advantage.
Tom Ollerton 9:04
So how do you do that specifically? Could you give me an example, maybe, in a previous role where you you've daily, had that kind of daily view of the data, and you spot the anomaly you've digged in. You dug into the anomaly, and you found an insight that's been beneficial to the growth of the business. Could you? Could you give me an anecdote or a story to articulate that?
Boaz Sobrado 9:28
Yeah, I'm happy to share in general terms, the the way I like to approach it is looking at the top line business metrics that the top business leaders would be interested in looking at the revenue numbers, the spend numbers, all of the important things that happen between all of the onboarding flows split across different channels, regions, etc, etc. So just getting an instinctive feel for the heartbeat of the business. You know, our Monday is usually stronger than Tuesdays or Fridays. The strongest day. Is the last Friday of the month unusually strong? Or is it the first Monday of the month that's unusually strong? Just sort of getting an intuition around, you know, the data, heartbeat of the business. And once you've done that, once you've acquired that instinctive feeling for it, you can immediately spot trends. So in particular, there's an anecdote where I saw that our conversion rates in the previous business were dropping significantly, and I managed to, after digging around a little bit, figure out that there was a relatively niche issue in one of the implementations of the onboarding flow that impacted the subset of users that would have otherwise taken a significant amount for someone to spot because it was such a niche issue. But nevertheless, because I had this intuitive feel for what was going on, I managed to figure out that something was wrong and relatively quickly identify it, even though, as a marketer, I was strictly responsible for the onboarding process, but nevertheless, it impacted the performance significantly so I could drag it out and try to figure it out with with the responsible Product Manager.
Tom Ollerton 11:39
This episode of the shiny new object podcast is brought to you in partnership with madfest, whether it's live in London or streamed online to the global marketing community, you can always expect a distinctive and daring blend of fast paced content, startup innovation pitches and unconventional entertainment from madfest events. You'll find me causing trouble on stage, recording live versions of this podcast and sharing a beer with the nicest and most influential people in marketing. Check it out at www dot madfest london.com.
So we're going to move on now to your shiny new object, which is predictive modeling of conversion events. Now I think I know what exactly what that is, and I'm really buzzed to talk to you about it. But can you explain to the audience what that is for someone who may have never heard of it?
Boaz Sobrado 12:32
Yeah, I think I first started thinking about this concept when I worked at my previous company, which was a consumer credit business, which effectively you can think of as kind of like a credit card. And when you're selling credit cards, when you're quote, unquote, selling money, when you're giving money to people, the outcome is actually quite different than when you're selling furniture, right? Because if you're in the credit card business, you have a customer that comes in and takes 100 pounds. Well, they can be if they're a loyal customer, and they stick around for years, and they pay back their debts on time, etc. They could be a very valuable customer, but they could also be a customer that comes in, takes 100 pounds, never pays anything back, right? So effectively, you've got this, this range of potentially negative values for a conversion event, which is not the case in E commerce, right? In E commerce, if you sell a chair, if you sell a table, if you sell a bag, whatever piece of furniture, then generally, you know how much you sold it for, and you know exactly what your margin on that is, and you can try to estimate it, and that's the conversion value that for you feedback to the advertising platform and that you optimize against. Now I know that that's not entirely true, because occasionally you get returns and etc, etc, so there's some complications there. But on the whole, it's a little bit more straightforward. However, when you work in a FinTech company and you're looking at the value of the user that you've just acquired, the range is a lot bigger, and in the case of credit, can potentially be negative as well. So I then started thinking, Okay, well, how do I do a better job actually giving the algorithms that end up optimizing all the ad delivery, etc, the data that they need in order to be able to optimize better. And that's where my thinking and obsession with these predictively modeled conversion events started.
Tom Ollerton 14:39
So I would say that was a fairly long answer. So could you be more specific? You could just help someone give me, give me a basic description of what it is.
Boaz Sobrado 14:47
So the basic description would be that instead of feeding back the raw data of what actually happened on your web platform or your app or your website, you take some function of that data, and you modify the raw data based on what you think is going to be the real value and not the actual observed value. So to give you an example, sticking to the furniture business example, when you sell a table, say, online, and you know that your margin on that table is, say, 20 pounds. Instead of giving the platform the 20 pounds value that they can then optimize against, you give it some function of the 20 pounds based on maybe, if it's a new customer, you expect them to shop again. You multiply the 20 pounds by two, and you feed it back 40 pounds. Where, if it's whereas, if it's a repeat customer that buys from you every week. You apply some sort of discount to it, and you feed it 10 pounds, for instance. So effectively, it's taking the observed values that you get from your advertising activities and modifying the real value to generate some sort of predictive or synthetic value that hopefully will drive better outcomes for you within your marketing process.
Tom Ollerton 16:04
So basically, you're taking conversion events that happen on site, furniture, financial services or otherwise, and then you're predicting the lifetime value of that customer, or a short term value of that customer, and reporting that back to the business.
Boaz Sobrado 16:20
It's effectively kind of like that, except you can also... it's basically that. So if you simplify it to individual users, let's say your your goal is user acquisition. You want to acquire a user instead of looking at the first transaction that a user makes. You can also try to make an estimate of the user's lifetime value upon the first transaction, and feed it that feedback to the algorithm itself. And that is a perfectly legitimate approach, and a lot of companies do this. In fact, it's very valuable thing to do in cases of, say, sparse data, where you have relatively few values to actually go off. So you actually need to give the algorithm a bit of a confidence boost and give it a big number, whereas you only have small numbers, could also be valuable in the case in the example that I mentioned earlier, with negative lifetime values. So for instance, if a user is probably actually not worth very much based on the behavior you've seen from them, then, instead of giving a potentially high value that their first transaction would warrant, you could actually feed the algorithm at low value. But in general, it's not just using the LTV predictions. It's about modifying the real values to make them a little bit more palatable to the algorithm that's effectively making the bidding so to give you an example, I don't know if you're familiar with UAC campaigns or universal app campaigns, but effectively, these are campaigns that are trying to optimize on Google towards an in app event. Now I've heard from people that used to work at Google that effectively any value over $500 that is fed to this algorithm is actually just truncated. So it will just take the the you know, if you send it 501 as a conversion event value, it will just truncate it to 500 right? So there is a limit to the values that the algorithm will actually accept. And there's various reasons for that, but the underlying reason for it is that the algorithms that are used to optimize advertising performance in this sort of black box world that we live in are algorithms that make certain assumptions about the distribution of the underlying data. Now what that effectively means is that if Google's doing it on their end, then maybe there's something that you can also do on your end to optimize the algorithm to performance. So truncating values, for instance, or normalizing them, or all sorts of things like this is a good example that is actually relatively commonly used across performance marketing to try to nudge the algorithm into finding the results that you want it to find.
And so how common is predictive modeling of conversion events? Now, how much of a shiny new object is in your experience?
I think it's it's pretty shiny object, in the sense of a lot of people want to do it. A lot of people try to do it. I think it is potentially something that can be quite tricky to do, simply because, depending on theinfrastructure and the size of the team that you might have, you might not actually have a dedicated data scientist sat there being able to actually put together machine learning models to try to predict the value of every single account or user that you acquire. That being said, what I think is quite common is a much simpler version of this, which is some sort of simple lead scoring algorithm. I think most companies that do user acquisition have some sort of lead scoring whereby they draw these lines of, if the user meets this requirement, then it's worth x, and if the user meets that requirement, then it's worthwhile from that perspective. It's, I think, relatively common.
Tom Ollerton 20:52
So how would you advise someone? Let's say advice to people so that you so a team that is big enough to have a data scientist and a team that isn't how would you go about kicking off predictive modeling of conversion events within those two scenarios?
Boaz Sobrado 21:10
I think the underlying philosophy, the most important thing is to be familiar with the underlying philosophy, which is the goal is to make the advertising platforms algorithm work for you as a business, not necessarily for the advertising platform itself, right? So you have to keep in mind that there, you know, most of the time there's alignment of interests, but there are some sort of scenarios in which it's not necessarily the case. I think perhaps there's this familiar anecdote. I don't know if you're familiar with it, Tom, but I'll repeat it for the sake of the audience of the pizza flyer operation around you know, there's, there's this person with a with a restaurant, and they hire these three teenagers to hand out flyers in front of the pizza shop and to hopefully get people into the restaurant. And the three teenagers go out, and every single flyer has some sort of unique identifier. So you know, you can tell which customer came from which teenager. And then what you see is first teenager at the end of the day gets three customers in, second one gets four, and the last one gets 34 and the restaurant owner is quite surprised, and he asked them, Okay, like, how? How are you getting so many people to come in? And effectively, the teenager says, Well, I hand in the flyer before they step into the restaurant. So that's sort of a long winded anecdote towards incrementality. But the point I wanted to make is, you know, you have to be aware of the dynamics of incrementality when it comes to the actual conversion events that you're feeding back to the platform, such that whether you've got data scientists or not, you need to be aware of what is actually driving value for you and your business. Now if you do not have a data scientist, I think keeping taking this to heart and being very, very conscious of the fact that not all of the conversions that the ad platforms take credit for are actually real, and they can legitimately take credit for them, I think that's the single most important thing you can do. And if you keep that in mind and you interrogate the data based on that notion, you'll do very well. With regards to when you actually do have a data scientist and you can actually work towards building these predictive lifetime value models. I think there's two things you can focus on. First of all, generally when it comes to user acquisitions, what people use as a signal to estimate the lifetime value of the user is the first or the first couple of transactions that a user makes will very much inform the value that of the user going forward. Now the that's kind of the baseline that the data scientist needs to work to, you know, on top of, and need to try to improve that model iteratively, right? Go through it, okay, I know that if you make two transactions in the first day, then you're most likely to be worth X. Okay. Well, how can I improve this model? There's the amount of times you log in before transaction inform it, et cetera, et cetera. And then that's the actual value, if you're technically capable enough and have the right infrastructure that you can feed back to the advertising platform. That's one thing that I would focus in that respect. The second is actually just testing, trying to test as much as possible. What is it that the. Algorithm will work with well with if you feed them these values versus what are the things that will simply not be functional. And the last point I wanted to make is, generally, the more data points you can feed an algorithm, the better. So the more you can multiply your conversion events, and not necessarily just limit them to the sparse, high value ones, but rather to the more frequent, occurrent ones, even if they're not necessarily truly events that drive value that will definitely help improve the quality of the ad campaigns. Now, I appreciate this has been a long answer to a simple question, but hopefully I haven't rambled too much.
Tom Ollerton 25:52
Well, look, unfortunately, we are at the end of the podcast now, and you've opened my eyes to some things, and I have to say, I'm not sure I totally get all of it, and that's how you learn, right? So thank you for showing me a different way of thinking about what is a niche of data driven marketing. So I really appreciate that. So if someone wants to get in touch with you about anything you've talked about, be it books or predictive modeling, how, how and where is a good way to get in touch with you.
Boaz Sobrado 26:25
You can find me on Twitter, on SobradoB. You can find me on LinkedIn, looking for Boaz Sobrado. There's no one else in the world with that name. And you can also find my website, boazsobrado.com.
Tom Ollerton 26:36
Brilliant. Boaz, thank you so much for your time.
Boaz Sobrado 26:39
Thank you.
Subscribe to the ‘Shiny New Object’ Podcast on Apple Podcasts, Spotify, YouTube and Soundcloud.