Episode 197 / Jennsen Fung / EF Education First / Executive VP, Digital Marketing
Next Generation User Acquisition: Combining Data Science and Behavioural Science for Competitive Advantage
Jennsen Fung is the EVP of Digital Marketing at EF Education First, the largest private education company in the world. He has a background in advertising agencies, where he developed skills in user acquisition and channel planning. In his role at EF, Jennsen works on the central digital marketing team, called Next Generation Acquisitions (NGA). NGA builds in-house machine learning models and tools and growth tactics to help EF improve its digital marketing and user acquisition efforts.
Jennsen’s Shiny New Object is called Next Generation User Acquisition, which means adding data science and behavioural science elements to user acquisition efforts in order to give organisations a competitive advantage. Jennsen believes that relying solely on channel planning can be risky, as it can make an organisation vulnerable to changes in the channel landscape and to competitors who can easily copy strategies. By integrating science and technology into user growth and acquisition tactics, organisations can make more informed and faster decisions, have more accurate attribution and prediction capabilities, and have access to features that don't natively exist on platforms like Facebook and Google Ads.
As an example of using science as a layer on top of your marketing tactics, Jennsen mentions machine learning, a subset of artificial intelligence (AI). This is a useful tool for classification tasks such as identifying spam emails or determining which users are likely to make a purchase. Jennsen also discusses the importance of data quality and governance, as well as the role of data visualisation in helping brands make sense of large amounts of data. He believes that combining data science and behavioural science with user acquisition efforts can help organizations take calculated risks, make faster decisions, and achieve bolder results.
Find out more about how Jennsen sees data and the human element interact in digital marketing and listen to his top marketing tips, on the latest episode of the podcast.
Transcript
The following gives you a good idea of what was said, but it’s not 100% accurate.
Tom Ollerton 00:11
Hello, and welcome to the shiny new object podcast. My name is Tom Ollerton. I'm the founder of automated creative. And this is a weekly podcast about the future of the marketing and advertising industries. I get to interview someone from the industry who's inspiring, smart, clever, successful about what they think is going to happen next. And this week is no different. I'm on a call with Jennsen Fung, who was EVP digital marketing at EF education first. So Jennsen, for anyone who doesn't know who you are, what you do, can you give us a bit of background?
Jennsen Fung 00:45
Sure, Tom. First of all, thank you for having me. I've been a listener to your pod as well. So my name is Jennsen. I was born and raised in Hong Kong. A few highlights of my career. So I started in advertising agencies. I've done a bunch of things in media planning, accounts. But this is where I developed my core skills in user acquisition and channel planning. And then I reckoned I wanted to see more impact in a business. The next page of my career, I moved to a on demand delivery app in Hong Kong. I was heading the global user and driver acquisition for the app. It was a very interesting part of my career, because we have a global exposure across Asia, Latin America. So very broad geographical coverage that you have a lot of resources but at the same time, you make decisions within hours across time zones, and you'll see results very quickly too. So there I pulled different growth levers like paid media, performance media discounting, couponing activations and where I enjoyed the fun to see the actual impact and learn about the power of technology and data and how they can fuel user acquisition.
Jennsen Fung 02:00
For my next role I moved to London, and I joined EF education first. So in case for people who haven't heard of the company, EF is the largest private education company in the world, people know us, typically from our language, education and outdoor business. We do everything from serious learning, like MBAs, corporate education, language schools, to casual learning, life study tools, gap years, online learning for language and casual apps. In my role as an executive VP at EF, I work in the central digital marketing team, of which we have data scientists and behavioral scientists. The team is called NGA, next generation acquisitions, super fancy, but I didn't make up the name by the way, someone in the past did. We build in-house machine learning models and tools and growth tactics, in-house DDP attribution model, and different tooling and projects. And ultimately, what we do is helping EF to get better in digital marketing and user acquisition.
Tom Ollerton 02:59
Wow, right. You sound like an analytical guy. I'm going to say that. So I'm really interested by your first choice of question, which is what has been the biggest work fuckup? Where did it all go wrong? What was that moment where you were red faced facepalming crying in the pillow with a disaster, but you're kind of glad that it happened, when you look back on it retrospectively.
Jennsen Fung 03:21
Yeah, I like how you framed the question. Let's face it, we all have a work fuck-up. But the important thing is how you embrace that and move on from that. So this is an interesting story about seeing the big picture and the impact. Let's try to help our listeners to visualize this. So the story was about a younger me, I think it was maybe senior manager, associate director at Mindshare. I was meeting a new senior client for the first time. The guy is super senior. So that was the eager young man from the agency trying to impress the big boss man, we can all see that happen in the workplace. Right. So we started by presenting some of the tactical campaigns, performance media campaigns and search campaigns, we have brand, policy, the return of investment, standard media events. And then I got a little bit ambitious, I wanted to talk business with the big boss. So we bridge into an offer type discussion. So I tried to tell them what type of offers were more profitable, than the others. And then the big boss stopped me and he told me Wait, hold on. It doesn't match with anything in my understanding. It doesn't match with my book.
Jennsen Fung 04:40
So that was the most awkward moment I had in my professional life. And then the whole meeting room went dead silence and I basically have nothing to say. Then I found out I was just looking at the paid media pot. And the boss told me Yeah, your observation is valid, but that's 7% of my whole business, so I'm not trying to say it's not important, but this is not significant. So what do you have for me? In the back picture, so I was using the wrong data plan to derive an inconclusive picture. So that was super embarrassing. And I basically walk out from meeting room with nothing to say at all, nobody bothering. But that actually got me to rethink about what is my thing? What do I want in my career, and shout out to the two gentlemen in the meeting room that day, they could be listening. And that conversation actually helped me to re plan my career that makes me want to depart from the agency and move to a more impactful and more result driven environment.
Tom Ollerton 05:52
Oh, that is, that's quite the story. But I think I would give perspective on the person that... the client, you know, they probably felt awful, you know, telling you were wrong in front of all your colleagues, or at least I hope, so I think there's probably learnings there for everyone. So that was a great story to help me understand how your career trajectory changed, but I want to get tactical now, what is the best marketing tip that you've ever had?
Jennsen Fung 06:16
Yeah, I think the best marketing tip I've ever had is, when there are multiple solutions, "a simple solution is most of the time a better solution. The reason I'm saying that is, in the tech world, there is too many new solutions happening every single day. But there is always a fine line between a complex solution as an an over engineered solution."
Jennsen Fung 06:39
I can give an example. I have heard of something called audio same targeting. So the technology is there is a bot, the bot will listen to all the TV channels for all the TV ads, so when they identify your competitors' TV ad, they will add your own ad on a mobile display. So the idea is to intercept the audience attention on their mobile phone, when they are being impacted by the competitors' TVC. I'm not trying to describe the technologically advanced solution as I also build these solutions for a living. So obviously, it has to be good, right? But we need to evaluate them in context, in the sense that what is the uplift, that solution is going to bring to the table. So usually, I would draw the line by looking at the speed to market cost, incremental performance, and does that bring a delightful user experience. So in this example, I will think about if this technology is going to create a noticeable change in user experience that will bring an uplifted user response when we have that interception happening, or that's the incremental media costs and production costs, because obviously technology costs money. And you have to spend more time and more resources to produce something that is more relevant. Does the incremental investment yield proportional performance lifts? Or is that something the user will be delighted with, or there's something just a marketing team will be delighted with and check off something on their team OKRs?
Jennsen Fung 08:11
So let's assume the interception effect is real. Which in itself, I think it's a big assumption, I will think of other alternatives that would require a fraction of the effort, such as getting a performance specialist to sit at home and watch TV all night, when they see the competitors' ad, they will just go into Google ads and turn on the campaign and update the mobile performance. Would that bring a 70% effect versus a complex solution? If the answer is yes, typically, I will actually choose a simpler solution because I can actually do that tomorrow without a need of technological investment. So the reason for choosing or comparing a simpler solution is not necessarily because it will be cheaper. But it will definitely avoid operational complication and keep your team remain focused on the big picture. So in my experience, this thinking would rule a lot of distractions for you. So you can really double down on what the next special thing is, or what is a new technology that can really pass the test and look at the big difference. But of course, this could be an extreme example for illustration. I'm a true believer in automation, technology, science, and obviously, a simple solution.
Tom Ollerton 09:27
So in a sense, you're always looking for the MVP, and I can't remember the name of the technique. I think it's called like the the Merlin technique or the Wizard of Oz technique, where you, you replicate a user experience with a human. So say, for example, you and I are building a an Alexa app for argument's sake, instead of like coding it and making it all work. You just put someone behind the screen and you go, Alexa, open the Tom and Jennsen app, and then we go Hi. It's the Jetson app and they go, if you just work out the user experience about building anything, you just... oh I can't remember what it's called. Anyway, it's that technique. It's like, strip out everything, strip out the technology. And that's so interesting that like, yes, there's lots of cool tech stuff you can do. But like, operationally, it's gonna take you ages to get there. But then obviously, there's, if it does work, and you do get 70% uplift, and it's a beautiful experience, then you can use a tech to scale it, which is the point. But you don't need scale, you need proof, right?
Jennsen Fung 10:30
Yeah, that's a private energy, I actually suspect a lot of the things under the hood could be some factories somewhere in the world, a bunch of people doing super mundane things. But once you have initial successful case, and you will be more competent to invest more to actually build out that technology, but no shame to use a scrapey and simple solution. I'm a big fan of scrapey simple things before we actually double down on the high tech sales.
Tom Ollerton 10:58
So we're going to talk about your shiny new object now, which you've given us a bit of a teaser for, but your your shiny new object is next generation user acquisition. So that kind of makes sense to me. I think I know what that is. But can you give us a crystal clear picture of what that is for the audience?
Jennsen Fung 11:14
Of course, of course. So this is my personal passion and the project I'm building. So we're thinking about spicing up user acquisitions. So with data science and behavioral science elements. So this is an extension of my experience around channel planning and user growth in my previous agency and delivery app days. Typically, a marketer will be thinking about channel planning. But if we come from a channel planning perspective, plenty of budget in different channels and use ad or use whatever organic activation tactics to activate, it uses the number of channels that are scalable, actually very limited. And everyone in the market, they're using the same set of channels. So it's very easy for everyone to duplicate successes, or catch up on the knowledge gap. So let's say your competitor is really good at Facebook, the easiest thing you can do to get that Facebook strategy or close that gap is you just post someone in that organization or in that agency. And we see that happens all the time. So there is not much competitive advantage you can generate. And also, thinking of the channel perspective solely could be dangerous, because what if the channel messed up, we have all seen the iOS 14 release two years ago, and how Facebook performance tanks and offices surprise tanks afterwards. So if we place all the bets and look at acquisition in the channel usage perspective, we could be very vulnerable and passive, to this channel changes in future and to competitors. What I believe would give an organization a massive advantage is to start integrating science and technology on top of the user growth and acquisition tactics, then you can start making more informed and faster decision. You can have more attribution and more accurate attribution and prediction. Or you can have some Facebook and Google Ads feature that they don't natively exists. While you have that, you will have that competitive advantage, you will be more confident in taking calculated risks, you have a bigger margin of error. And ultimately, your organization will go faster and go bolder. These are the pictures of shiny new object I'm interested in and they are quite difficult for our competitors to copy.
Tom Ollerton 13:47
Right? Okay, so if everyone's using the big platforms, and you... I love the idea of poaching someone from the agency to work out what their competitive strategy is. You can probably work it out yourself anyway. And you're talking about building a layer on top of that, science and tech on top of that, so I get it. But can you be specific and give me an example of that? Like, what does that actually look like?
Jennsen Fung 14:11
Of course, of course. So I think to start around the term AI, but AI is a very trendy thing to say. Within AI, you have a lot of more experimental and fancy expert like artificial general intelligence. There is a lot of academic work that goes there. In the most layman terms, if you see those AI application in a Hollywood movie that involves something, someone killing somebody else, and these are the fancy shit that you probably don't want to invest in today. So those are being pulled out from our conversation. So I think the most robust usage of AI would be machine learning, which is a category of AI. Within machine learning, and there are multiple subsets. You may or may have heard of like reinforced learning, unsupervised learning, supervised learning, stuff like that, but the more reliable and understandable use case is around classification. So we can tell, let's say we can tell which email is spam, which is not. Regression: that tells you the relationship about two things. You put in x amount or budget, does that yield a certain percentage of RI in, in a correlational, or causal sense? Recommendation and prediction. These are easy to understand. So to answer your question, I think the more tangible use case will be: Let's say we build a recommendation model in a PDF or in a website. We build LTV base, PPC bidding, modifier, we build marketing, let's...
Tom Ollerton 15:43
Oh, jump back LTV, PPC modifier. You're gonna have to unpack that a little bit for me.
Jennsen Fung 15:50
Sorry. So the more reliable machine learning use cases are classification, which should tell you, for example, which email is spam, which is not; regression, that tells you the relationship between two things, recommendation and prediction, which are easy to understand, right. So these are the more interesting and impactful marketing use cases that will happen in a workplace, such as building a recommendation engine to recommend the right product to our users, on the website, marketing mix modeling that helps you measure the ROI of offline advertising spending, personalization. So everyone will get personalized ad, personalized landing, or even product recommendation throughout the EDM journey. Uplift modeling that can tell you how much you should adjust on your product pricing, user clustering, how to cluster your users into pack enough and meaningful groups that you can generate insight, or you can act upon them. So these are the examples of the machine learning and data science, behavioral science use cases in the ideal world of next generation user acquisition. And then you can connect all these to the typical things you are seeing like media media planning, or performance media, and EDM, and so on and so forth. And this is a starting point.
Tom Ollerton 17:47
So you listed off a whole bunch of different versions of next gen user acquisition using machine learning, on creating recommendations, predictions, and so on. But can you choose one of those and hope the audience understands what that actually looks like? So what would a successful recommendation application of this technology look like? How would that work for someone who's never done that?
Jennsen Fung 18:16
Yeah, I think the recommendation is a brilliant example. Because everyone can resonate to your personal experience on Spotify and YouTube. So Spotify and YouTube a great example and recommendation. So they will provide a recommendation to you by user similarity. So let's say I'm a new user to the platform, platform thinks I look like Tom. So Tom has consumed a bunch of things probably will see those content, and then, and then they will treat the recommendation according to what I like and what I consume further, or a content similarity. So Tom likes football. The platform recommends football to me, I have consumed free football, and they will start to go deeper, let's say professional football or football in Asia, football in Europe and the world cup, or female football, stuff like that. So these are the recommendation we can build.
Jennsen Fung 19:12
If your business has a large number of products, building a recommendation engine to fit the right product to the user, in theory will uplift your conversion rates, because you are maximizing what your users are being exposed to in a set or a limited amount of browsing time. But the question to get there as a business is what kind of user data you have and what kind of product data you have. When you have enough data points around your user and the data and they start to form a pattern and you can train a recommendation model. So the recommendation model will make the right decision to you. To get there you will have you need to have the right data infrastructure. So you track different things, you document different things on a book. And then you need to have that recommendation model.
Jennsen Fung 20:02
And once the recommendation model can produce some recommendations, you can integrate them in the typical growth tactic you like, let's say website optimization. Version one is hard coded product that you make the most money for those products. So you show that those time products, and then you can have a version that on the website is recommended according to the user behavior, or on the EDM, you can do the same, or more advanced, where you can introduce some barriers to the recommendations, and test how the user response and then you can adjust your product strategy. Or there is one feature I really like on Spotify. Let's say they recommend a bunch of songs to you, you start to dislike some of the songs and they will actually remove those from the future recommendation. And they will revert back to something they ascertain that you actually like. So there is a bunch of robust usage for really advanced company like Spotify and YouTube, but even for midsize medium size ecommerce businesses, there is a lot of scalable tech usage and investment in the recommendation engine. So I like that as an example, because it's more easy to understand, it's more down to earth. And you can actually imagine the business uplift if you're getting this done correctly.
Tom Ollerton 21:30
So jumping forward a bit, I would like to understand where you think it's gonna go in terms of introducing randomness into recommendations. Because on Google or Gmail, we use Gmail as a business. And you start typing a sentence, and it suggests, you know, I'd say, looking forward to working with you, but actually, what's actually been written is looking forward to hearing from you. So the way that I understand that Google recommends that to me is because it's looked at millions and millions of millions and millions and millions of emails and gone, when someone starts to write looking at the end of an email, like 12 lines down, they normally write, looking forward to seeing you or whatever it is. Now, I always ignore the recommendation. Because that's what everyone else does. And if everyone else is doing that, therefore, the thing that I'm going to do is going to be less unique. So if you recommend me something in terms of from a creative perspective, it's because lots of other people are doing it. So what I want to know is: Have you seen any technology, do you think there's a future that is genuinely creative, as opposed to representative? You think you can take things like style transfer, this is what, this is how van Gogh painted. So we can turn this picture in a van Gogh style, it's a replication of a thing that's happened in the past, whereas true creativity, or great creativity is something that bucked the trend, right. So how do you think that brands can bring in the randomness? That is the core of a good idea, as opposed to just recommending this thing because it happened lots of times previously?
Jennsen Fung 23:15
I think that it's a wonderful question. The Google, let's try to break it down step by step. I think the Google recommendation is different from a product recommendation. And it doesn't come from an end goal that they want to inhabit creativity. Actually, I think they are trying to help you to get things done. So the reason to have that recommendation is to help you to form the sentence a little bit quicker, right? So this is something super pragmatic, but not about creativity, or helping the user to make a decision about something. And the second part of the question is, how do we introduce variants to the recommendation engine, I think that is also a very important point we need to look at, there is some recommendation or content matching engine in some of the social media, they will focus on creating an echo chamber effect. So everywhere you go, you see the same thing, which there is no variance in there. But that could be actually be damaging if we are working on a product recommendation engine. So for a product innovation engine, let's say you are going to recommend five products. Usually what we'll do is we'll isolate a certain percentage of the recommendation. And then we'll introduce something as outlier. And then we will train the model and see how the user would respond to that outlier. So we can avoid having that echo chamber effect. So if we start to see a lot of the users actually interact with the outlier a little bit better than the recommendation, the recommendation engine is going to know Oh, actually, I'm missing something.
Jennsen Fung 24:56
So as you start incorporating some of the features, this is how we introduce variants to avoid the echo chamber effect, which I think it's where your question is about, the last bit is about creativity. So I think there is a form of tension between the use of science and how the human brains come in place. And that help us to become creative and produce new things, or are we going to live in the machine life? My answer is, like I said, in the very beginning about sourcing a simpler solution. I wouldn't wish a recommendation or machine to do everything for us. There are layers in marketing that will require creativity, common sense. And in a sense, the romance of creativity, I would never discredit those elements. I think that's just don't be obsessive in science and technology. But always take some time to appreciate real life interaction and user insight from a real human being or creativity from a copywriter. And these are over important and make the tech life and digital marketing life a little bit more fun and sexy. So I think that is also important to think about.
Tom Ollerton 26:21
And that is a beautiful way to finish the podcast. I could talk about this all day. And I do talk about this all day anyway. But thank you for recording this for me. It's been fantastic. So Jennsen, if someone wants to get in touch with you about this, or anything else, where do you want them to get in touch with you, and what makes a great outreach message to you?
Jennsen Fung 26:40
I think LinkedIn is the best channel. I can be searched on LinkedIn. The best outreach message is just to be very upfront about how can I help you? And why do you want to be in touch? Let's say, send me a message. Say, Tom said to me, I listened to your podcast on shiny new object. I think you're wrong. I'm doing something similar but better. Totally welcome. I would love to hear more about how our friends in the market are doing different things and how they feel about the use of technology and science in marketing.
Tom Ollerton 27:22
Fantastic. Jennsen, thanks so much for your time.
Jennsen Fung 27:24
Thank you for invitation, Tom, it's been fun.
Subscribe to the ‘Shiny New Object’ Podcast on Apple Podcasts, Spotify, YouTube and Soundcloud.