john-cordier_1_05-10-2024_113920: And when we actually record it, it'll be like our pre done one. Um, and then we'll just flow right into the conversation.

matt-madden_1_05-10-2024_113918: Sounds great.

john-cordier_1_05-10-2024_113920: All right. Welcome to the next episode of the flux, where we talk data decisions and stories of people asking the what if questions to create an intentional impact on the future. In each of our episodes, we hope we can turn lessons and hindsight from our guests into foresight.

So you can uncover how you can create an intentional impact. For others in your actions going forward today, we're on with Matt Madden, a BYU marketing lab director, where he gets to help make the really nerdy stat stuff. Accessible to everybody else. So Matt, why don't you start chatting about a little bit of, uh, you know, what you're doing at the BYU marketing lab and know, what you're working on today.

What are some

matt-madden_1_05-10-2024_113918: the lab is a really interesting experiment that the [00:01:00] BYU Business School started a number of years ago. Essentially, we're getting a bunch of students that are in their last year of the MBA program, or are seniors in the marketing program, and putting them on real consulting jobs. So we're bringing in outside clients and having these students come in.

Practice all of the skills they learned in analytics and consulting and research, um, to do some really awesome work for clients. So, uh, this past year we got to do, um, some segmentation and targeting work for TruFru, which is a, an awesome local brand. Um, that's got a ton of good momentum, um, and a number of other things, but I just, I love watching these students, um, go from statistics and analytics is.

It's kind of the scariest part of getting into a business program for most people, uh, to just loving it, um, once they get it and they see the value of it.

john-cordier_1_05-10-2024_113920: of the common things that make him flip the switch of like, Oh, the stats stuff is actually really important. Like [00:02:00] in some ways they have a, an aha moment. Like, is there a common, story or case study that you use to get them into it?

matt-madden_1_05-10-2024_113918: I mean, for, for a lot of my students, I think it's when we get into this, uh, market research technique called conjoint or choice modeling. Um, like everybody out there has seen a thousand surveys. Most of the students, by the time they get into our program, like they've written their own surveys. We've got a partnership with Qualtrics at BYU, so they've used Qualtrics, or worst case, like a free SurveyMonkey trial or Google Forms or something, but the thing is they've like really done bad survey work.

And they don't know it. Uh, like if you ever get a survey from your kid's school or something,

john-cordier_1_05-10-2024_113920: hmm. Mm

matt-madden_1_05-10-2024_113918: are the worst surveys. They're just, they're poorly designed and you think, you're going to get all this data back and you guys aren't going to have any idea what to do with any of the questions really. So, um, And you teach this technique called conjoint, which probably a lot of the listeners haven't heard of, but [00:03:00] I imagine instead of just going around and asking people, well, how much do you like that on a 10 point scale?

And how much do you like this other thing on a 10 point scale? And what about this other thing on a 10 point scale, which is what we're used to seeing in surveys all the time. Um, instead we like, we design a product, um, you know, breaking it down and it's kind of components. And then we put multiple products in front of people and we say, Okay, if you went shopping in this category, and here's three products, which of the three products would you buy?

You say, oh, well, that's easy. Like I do that all the time. I always go shopping And so the idea of Conjoint is to essentially build these concepts on the fly, put them in front of people, have them You know, give us eight to twelve responses on which is their favorite product from these different product sets that we design on the fly And from that you understand exactly what they care about Right, so I might struggle to actually ask you, Okay, when you're shopping for a car, what do you really want?

Right, and you say, oh, well, you know, First thing, I need to get a safe car. Well, it turns out, like, [00:04:00] you would say that out loud, But the reality is, if I showed you four cars side by side, And some of them had four star ratings, and some had five star safety ratings, You would pick ones with the four star safety ratings all day, every day, if they're cheap enough or it's a brand that you care about or it gets better gas mileage, right?

So you might not even know that out loud, but once I show you some concepts and make you actually go shopping, understand what's going on inside your head. So we get that unit to the students and that's sort of, I think for a lot of them, their first aha moment of like, holy cow, this data can do some really cool things to help understand consumers and people.

john-cordier_1_05-10-2024_113920: So is one major part of and things that you try to, you know, like it's a different lens that you approach marketing analytics with is more of like a decision science behavioral economics type lens.

matt-madden_1_05-10-2024_113918: Yeah, that's a lot of what I do. I mean, I try not to call it even, it's still called the market research class, but I feel like market research is like this old school term now that nobody wants to talk about. [00:05:00] So we more label it consumer insights now. Um, it's what drives people's behaviors and how do you understand people's emotions so that you can.

Really be a benefit to customers out there. Um, I think marketing has a mixed reputation, right? I'm a historically I'm a statistics guy, but I ended up in marketing. And so on my first day of class, I always welcome students to a marketing program and I say, you now have a complicated reputation in the world.

You're in a business school and you're a marketer. Um, and we just try and really push it as good marketing benefits. Lots of people. Um, You help, you help them out. You're understanding what they need. You're helping your clients save money and come up with better innovations and better ways to message to people.

Um, so for us, it's like really understanding people, not kind of the classy, classic, maybe sleazy marketing that you get on Mad Men where it's almost like [00:06:00] manipulative. Yeah,

john-cordier_1_05-10-2024_113920: stats into marketing? Like tell us about that journey.

matt-madden_1_05-10-2024_113918: so I mean, the stats journey is interesting. So I started in math education when I was a freshman at BYU and every high school teacher I had that I caught up with over Christmas break after my first semester was like, don't do it. Which is so sad. We need more good teachers out there, but they all just said like, you know what, you're smart enough, you could do something else.

Teaching is fun. It does not pay a lot. Like you're going to have a lot of struggles, find something different. So I went from math education, tried math. The math teachers were just weird. Um, I didn't love it and ended up sitting in on a guest lecture where some guy came in from BASF. They're like the, the company that like improves products all the time.

Um, and he was a [00:07:00] stats guy and he talked about the stuff he did in his work. And it was like my first moment of understanding math can be really practical. Like math was always fun, it was just like solving puzzles all the time. And all of a sudden math became practical when he talked about stats. So I started that program, um, loved it.

Unlike probably 90 percent of people, um, who took stats and hated it, I was the 10 percent who took stats and loved it. And it was basically a choice between doing like bio stats and healthcare stats or business stats. Um, and I really loved the healthcare stats. My master's thesis, um, was kind of a cool use of, of modeling and predictions.

Um, if you've got time for a story, um,

john-cordier_1_05-10-2024_113920: Yeah. Has

matt-madden_1_05-10-2024_113918: um, she was, uh, Laird Wolfson, she was a professor at BYU and then she was on sabbatical doing work for the WHO. And so she knew kind of that I had been doing some internship work, um, with some advanced modeling. She said, Hey, you want to do your master's thesis on this?

[00:08:00] And it's something called verbal autopsies. So if you go into like Sub Saharan Africa, um, right, the healthcare is not great. And you're trying to watch for these kinds of epidemics that start to spread. Um, the thing is it's really hard to track. There's not a lot of data. And the only real way to tell if a bunch of people are starting to die from a certain disease.

Um, is to like do an autopsy and figure out, okay, what did they die from, but having like these really smart, you know, medically trained physicians doing autopsies is not a good use of their time to actually save people's lives. Um, so what they did is they got this database of, uh, hundreds of autopsies that they had done.

Um, so they know the cause of death and then they created this survey that they went and gave to like the families or the last caregivers of the people, right before they passed away. Okay. That asked them to kind of rate symptoms and timing and other things like that to understand what was going on before, you know, your, um, patient or your loved one passed away.

Um, so my like first real foray into [00:09:00] like predictive modeling was using those surveys to classify cause of death on hundreds and hundreds of autopsies to see like, can we replace all the time that physicians in Africa take, um, you know, doing autopsies essentially. Yeah. Have them stop doing that and just keep these surveys out there instead.

And that's going to give us like an early indicator if some disease is starting to hit outbreak level.

john-cordier_1_05-10-2024_113920: that evolved into something that's stuck today? Like, are they still, are these surveys still like in motion? And that's

matt-madden_1_05-10-2024_113918: Yeah, they still do that. I mean, they've got different data now where they contracted and not just with like these primary surveys, but they've got more data certainly in that area of the world than they had. Um, you know, this is over 20 years ago and kind of old now when we were doing Um, but yeah, we did that with the WHO.

Um, it was, uh, the adult morbidity and mortality project with the Tanzania ministry of health and like the university of Newcastle in England. Um, so it was pretty cool to like do [00:10:00] that and have an impact. And then I got sucked into marketing cause I took an internship at a startup, um, over the summer. And so my budding health analytics career was waylaid by the awesomeness of consumer insights instead.

john-cordier_1_05-10-2024_113920: uh, I mean, within like the healthcare world, do you see healthcare today being a little bit behind the business stats, business marketing side? Or is healthcare in some ways catching up Like where, how do you position the two against one another?

matt-madden_1_05-10-2024_113918: No, it's a good, it's a good question. So there are parts of healthcare that are right on the cutting edge of everything, especially for like early detection, diagnostics, um, what, you know, these pharma companies that kind of like marketers have mixed reputations, right? Some of the stuff they're doing to try and like create new, um, new treatments, new drugs, like it's incredible work the way that they're trying to like get down.

simulate effects and then actually [00:11:00] create a drug and start testing it out and getting it to trials like they're on the cutting edge. There's other stuff though where you can tell there's still a little bit handcuffed by policy if you want to get stuff passed like an FDA it's still kind of old school you have to kind of line up your tests that you want to do you can't you know do anything like p hacking And so it just feels kind of old and rigid and you can't use a lot of the latest techniques to analyze the data because they want, you know, more like tried proven, you know, almost old school techniques that are just, um, like they're good, they work really well, but they're also like really obvious what the answer is, like you can't get too clever or too creative because, and I understand that you can get a little manipulative with your data if you did, but I think there's pockets of healthcare that are like right on the cutting edge, other areas.

Yeah, maybe not so much. Maybe especially those that are politically involved.

john-cordier_1_05-10-2024_113920: That makes a lot of sense. [00:12:00] So, you know, you've kind of been the field kind of leading edge stuff, like surrounded by that in the academic community. Certainly is most exciting about where modeling within marketing analytics, where do you, where's it headed and what's most exciting to you?

matt-madden_1_05-10-2024_113918: Um, that's a good question. I think, uh, for me the exciting part of modeling and analytics is just going to be the availability of more data in easier ways. Um, So I'm classically a market research person. If you talk to some people, they hate market research. Um, not because they hate like the methodology.

They just say, I've seen market research. I got answers. They were not accurate at all. Um, I, and I didn't find it very useful. Um, so there's a lot of bad research out there. There's also a lot of really good research out there. Um, I think the good research is getting more prevalent, but even the good research, um, You know, it's, [00:13:00] it's hard to do because more and more everybody's being asked to take surveys constantly, right?

Like every time you leave, you know, a Chipotle or a Cafe Rio at the bottom of your seat, they're like, Hey, please take our customer experience survey. Um, and you're just being like badgered all the time by everybody to get your data and your reactions. Um, which is like, it's cool at a macro level that we're trying to understand people and get their feedback.

But as a consumer, it's a total drag. And you've got a self selection bias, um, you know, where people are, you know, it's like the Yelp effect where you basically either leave a glowing positive review because it's somebody you know and love, um, or you leave a scathing review because something really terrible happened and all the people who had a perfectly pleasant experience, just like they don't bother logging on and doing that survey to get, you know, free chips and salsa on their next visit.

They just, they don't care. Um, But you're starting to get all of these really tool good [00:14:00] tools coming online that are getting people's data without having to badger them about it as much, right? Like some of that's just like the data scraping. It's data that, you know, we're kind of voluntarily putting out there, especially in the US where we don't have like privacy lockdown on our online data.

I think to me that's like the cool area of the future of modeling is like doing like awesome primary research, um, you know, paying for that data, going out to collect, you know, a good survey sample, um, using good techniques, um, in segmentation or conjoint, but then easily linking it to corporate data that people have readily available that used to take, you know, days or weeks to put together that now they can just like pull all that data in in a day or less.

Um, or they can scrape data and feedback from conversations and product reviews and, you know, people who've been on with the service centers because they're transcribing all of that. You know, so like the availability of making all of that data that used to take a ton of time to process doesn't [00:15:00] take long anymore.

Um, like to me, that's going to make like the field of analytics just kind of have a heyday in the next decade.

john-cordier_1_05-10-2024_113920: Anything about that concerning with like all different types of data being available, like all the newer technologies, you know, taking advantage of some of that, you know, anything, give you pause.

matt-madden_1_05-10-2024_113918: So, I don't know. Some people are really big into privacy. I, for the most part, like when I get asked to, you know, opt in, do you accept cookies and all that? Like, I just say yes. Maybe it's because I'm a marketing guy. And so like, I know how that data is used. It's like, you could say no. And then, you know, you're going to be a 25 year old watching Hulu and the Hulu ads are going to come up and it's going to show you some like blood pressure medication drug commercial geared at like a 60 year old.

Because you've opted out of letting them track you. And I don't know, for me, I just, so I'm not too worried about the privacy game. The one thing I am worried [00:16:00] about is, um, how much we're just going to rely on, on the past to like make all of our next decisions. This is like the downside of just data analytics in general probably, um, is the more we rely only on past data to make our, make our next decisions.

We will use a little less judgment. We'll have a little less creativity. Um, we had this big conversation a few weeks ago, um, when we were wrapping up, um, the MBA analytics class, um, that I'm taking over next year. And we were talking about like generative AI and what's, what's going on. Um, and there's some cool stuff going on.

My colleague, Jeff Dodson, um, he's at BYU. He's going to Ohio state next year, the Ohio state. Um, He's done some really cool work, um, trying to value, like, um, what is it worth to have a specific artist or artistic style applied to like an image. [00:17:00] Right? So if you were going to be buying a teacher, I could just have an image of somebody smiling, or I can do that in the style of Bob Ross or, you know, a Thomas Kinkade or, you know, lots of different.

Um, you know, like these amazing artists that are out there. And so everybody's like generating these images and they're leaning on these existing artists portfolios because that's what they've trained these models on. And what he's trying to do with his research is figure out in a way, like how are we going to help compensate these artists that have like their works have gone into training the models.

Um, we can prove how much more people like it when you style it after their, um, you know, their artistic style versus another person's. Um, um, But the conversation from that and just like valuing the art and what it, you know, what value it brings to the training models that these generative AIs all have, um, is like who's going to make the new art if we're just overloaded with massive amounts of art that are all just based on what's already been done up until [00:18:00] 2024.

And if nothing new can get popular because we're just so obsessed with stuff from the past. That's maybe a little bit of a scary spot to be in. Right. And that's the gen AI thing, but I think you've got similar, similar issues with, um, you know, it's good to make data driven decisions, but at some point, like you can't let it stifle your creativity and, you know, really good entrepreneurs and business managers need to be willing to like take risks in new areas that might not have a, you know, a proven track record.

Yeah. But it might be the next big thing. Data won't tell you every single answer. It can maybe give you some guardrails to tell you like, yeah, these things have been done and tried. We know those are disasters. But I do worry that if people overuse data, they're just going to do the same thing over and over again.

john-cordier_1_05-10-2024_113920: So within that, you're bringing up a topic somewhat related to forecasting. [00:19:00] And as we're speaking, like a lot of the news is coming out about some of the challenges that the health insurance industry in the United States has been facing because a lot of their forecasting methodologies look at what happened last year. Let's understand those trends and let's carry it forward rather than, no, there's some behaviors that are changing. And through that behavior change, you're actually seeing Dramatic increases in utilization of services and the like, uh, you know, what are some use cases where it's good to use, know, backwards looking data, take it forward, maybe there's like an aspect of time that that matters for, you know, you could look at it from a timespan perspective, or are there scenarios where you're like, you definitely should rely less on some of the backwards looking data. When things are changing going forward, like, are there industries that you would say are [00:20:00] amenable to looking to make projections about the next year compared to others?

matt-madden_1_05-10-2024_113918: Um, well, I mean, the one that comes to mind right now, it's, um, probably cause I'm, I'm working on a project related to electronic vehicle adoption in the U S. Um, that's a really tricky one because, you know, if you look at any of the curves for like expected EV adoption. It's going up, but nobody really knows how much it's going up.

Right. So like, have we, we've kind of got our like first phase of popular EVs out there, right? Like a Tesla model three, it's at a really accessible price point. Um, you know, but you've even seen like Tesla struggling to, to move product right now because there's a limited number of people in the population that were willing to spend that much on electric vehicle in the U S.

Um, and probably in lots of other countries too. And so it's like, you've got this projection going up just because we know like, yeah, Evie's probably some [00:21:00] version of that is the future. Like, you don't know how quickly you've tapped out the group of people that were your initial buyers and considerers.

john-cordier_1_05-10-2024_113920: Mm

matt-madden_1_05-10-2024_113918: if you've tapped them out, like you need to dial back your forecast. Um, Or you need to rethink like what, what technological changes have to happen so that you grow that population that are going to start considering more vehicles. Like there's some really cool stuff, um, coming down the pipeline. If you look at it, like, um, Dodge has this Ram charger, um, that basically it's, um, it's not like a classic hybrid where you have an electric engine and a gas engine when you run out of battery, it's just an electric engine.

Um, But that electric engine can be recharged by basically a really advanced gas based generator so that you can get, you know, six or 700 miles on a tank of gas where, you know, most of the time you're never using the gas. Um, but a lot of people don't want EVs because they're afraid of getting [00:22:00] stranded.

It, you know, it sounds like a massive pain if you're going to take a road trip with an EV. You've got to like literally plan out all your stops. Um, You know, this kind of tech might be the thing that like solves that, like massive concern for a lot of American drivers at least. Um, so suddenly like, okay, well you've started to tap out your sales and this new technology is going to like bump up how many people actually consider it.

john-cordier_1_05-10-2024_113920: hmm.

matt-madden_1_05-10-2024_113918: that's one of those where you can't just look at, you know, last year's numbers and the year, you know, before that and draw a line and project it. Like you've got to pay attention to the population. Like, have you sold to all of your like readily available customers or not? And what are you doing to bring more people in to the fold to be considerers so that you can sell to them now?

Next?

john-cordier_1_05-10-2024_113920: Yeah. I mean, one of the other points you made earlier in the conversation, the stifling of creativity, if we're just looking at the past. in some ways, like BYU is one of the most innovative ton of entrepreneurs coming out of BYU. [00:23:00] In fact, like when you look at somebody who's trying to start something, if you're looking at past data before it even started, like it's, there's nothing to go off of.

matt-madden_1_05-10-2024_113918: Yeah.

john-cordier_1_05-10-2024_113920: like how, you know, How many times have you seen like an entrepreneur or somebody in like the investing community being like, well, you don't have the numbers to prove it out, but then you, you know, you have to understand, like, some set of behaviors are going to change to make the adoption of this thing actually take off. Like, how do people do that today other than just finger to the wind and guessing?

matt-madden_1_05-10-2024_113918: Um, I mean, like most people, so like one of the people on our marketing advisory board at BYU is Chad Gehring. He basically helps run a, a toy manufacturing company. He's in the weeds with the data. He's always kind of on like the cutting edge of, um, you know, like the technology to like manufacture toys, especially more sustainable toys in the future.

But one thing Chad is awesome at is just he talks to people all the time. And so he can tell like, is there a [00:24:00] trend that people are just hopping on and loving? Is there something people are just constantly frustrated about? Um, do people say, you know, and I go down the toy aisle and I just, I see the same things again and again and again, and there's nothing that looks different.

He's Um, so I think for a lot of entrepreneurs, they might not have like quantitative data other than, you know, maybe a little bit of just like needs and complaints that you can find quantitatively online through some scraping, but they just talk to other people and they ask really good questions and they're just constantly inquisitive.

So they're getting, you know, qualitative data when you talk to enough people starts to appear quantitative at least.

john-cordier_1_05-10-2024_113920: Sure,

matt-madden_1_05-10-2024_113918: And that's what I think the best entrepreneurs do, right? Is they're just, they're hearing what people talk about and you know, they're following, you know, the frustrations and kind of the exciting things that people talk about.

john-cordier_1_05-10-2024_113920: What's been one of the challenges that you've seen getting qualitative data at scale? Like, is that, is that something that someone's trying to solve for? Is that [00:25:00] what the, you know, origins of survey companies were trying to do? Like, how's that changed over the years?

matt-madden_1_05-10-2024_113918: Yeah. I mean, survey companies were built really to get quant data. So it's, you know, it's all the, like the qual shops that are trying to get out there and do interviews and focus groups and all that. Um, they're the ones trying to solve quality scale and they've done a, like Really good job of it. Um, there's a bunch of great companies out there now.

And I think this is where kind of these AI tools that are so good at transcribing conversations are just making it, um, to making it so easy. Now, um, we were, uh, like we did a project with my lab, um, working with our continuing education team at BYU and they had us do a bunch of employee interviews and it's like, we recorded every interview.

So we have the discussion guide. We did the typical 12 thing. Um, and then we used Otter. ai as a software tool that basically just transcribed it. Um, and you can throw the transcriptions into like a chat GPT or Otter. ai would [00:26:00] come up, um, you know, basically transcribe it for you. You could query it, ask some questions, figure out what the topics were, summarize the top five frustrations that employees talked about, um, or, you know, list the top three things employees wish that, you know, their department did more of to make their jobs even better.

Um, So I feel like the AI side of things is making qual more accessible. The, the main issue is like, there's a difference between primary data and secondary data. And so, all these tools are awesome on secondary data because it's just, it's out there. You can, you can either buy it or access it through APIs or other sources.

But like, secondary data is just people talking about what they want to talk about. Whereas primary data is You have a specific query. You're asking a question with a goal in mind to learn about a certain topic. Um, and there's still not an amazing way to get primary data at scale.

john-cordier_1_05-10-2024_113920: Mhm.

matt-madden_1_05-10-2024_113918: some of that's just surveys, you know, [00:27:00] we've had like syndicated surveys and omnibus surveys forever, but it still just feels like it's kind of the same game.

Um, there's some, you know, cool people. Steven Snell is a local guy trying to work on like the data quality problem. Um, Um, and then you've got bots that are out there. You've got people that are essentially programming software to take surveys, to make a little bit of money here and there all day long, every day.

And there's, you know, there's basically adding a tax on top of the cost for your primary data collection. So for me, it's trying to figure out how do we ask people good questions at scale and motivate them to like give good, honest answers. Without overwhelming them without getting a bias sample, right?

Like the Yelp review problems. Um, and everybody's kind of doing their own thing. And until I don't know, it's almost like until the industry comes together and decides like, okay, we've got to like limit what we're doing to people. Like we're, we're overexposing consumers [00:28:00] to surveys all day, every day.

We're probably going to continue to have a problem on just like getting really good primary survey. And primary qualitative data. Um, I think most people understand the problem of bots. I don't think that very many people in the industry have really. Tackle the problem of consumers are sick of being asked for their feedback nonstop.

Um, and they're certainly tired of being asked for their feedback in ways that clearly don't let them give feedback that matters, right? Like if all you're asking me, it's like, Hey, we have a quick two question survey. How are we doing? And it's two 10 point scales. Um, [00:29:00] Like, you're not going to do anything about what I was frustrated about.

Like, you're just tracking your store to see which of your stores are the worst performers so that you can, like, reward the best ones and punish the worst ones. Like, that's not my job. I don't really care to give you that kind of feedback. But, you know, when you give meaningful feedback, most of the time you know it goes into some kind of black box.

Um, you know, it disappears, it doesn't go anywhere, no one does anything about it. Um, and even if they do something about it, Um, most of the time you don't know. The only time, literally like, okay, so four weeks ago was the first time I think I've ever gotten a call back from any company in my entire life about a problem that I reported.

And it was from Ally Bank. So I use Ally Bank, it's like the online banking. Um, I still don't use them for my primary checking, although I probably should. Um, I still use Wells Fargo because I've had them since I was a kid. And they never ripped me off with the whole account creation scandal, so I'm like alright, I guess they've been good to me, I won't quit them yet.

Um, but he's an ally for like all my [00:30:00] savings accounts, and it's like, it's really cool, you can like bucket stuff out for your kids. Anyway, I like reported this problem to them because I pulled money out, because we're working on like my, we moved last summer, we're working on yard work, so I pulled money out, and you can like specify which savings buckets the money comes out of.

And I did that, And it just like completely ignored my buckets and it just like pulled all of our money out of like our vacation savings account instead of our like house savings account. And I had to like go through and it like emptied two or three accounts because it just kind of does it in this random order.

So it like took two or three accounts to zero and then the rest from vacation. I was like, this is a hassle. Why are you doing this to me? And I reported it and like they actually called me back and they're like, hey, I'm sorry. You complain about this thing. We send it to our engineers. Like, can they ask you if you follow up questions on it now?

And I was like, Oh my gosh, like

john-cordier_1_05-10-2024_113920: Yeah.

matt-madden_1_05-10-2024_113918: listened. Yeah. Like that thing alone was like, I should probably go buy a stock and ally because they might be the only company actually doing this,[00:31:00] 

john-cordier_1_05-10-2024_113920: Cool.

matt-madden_1_05-10-2024_113918: don't care otherwise, if you don't show that you care about them, like they're not going to care enough to give you honest feedback.

john-cordier_1_05-10-2024_113920: Yeah. I mean, and that comes in from, you know, data that consumers are putting out, uh, data that is coming in from internet searches, web, whatever it might be. ultimately the, you know, this, this breadth of data that's coming out, there to help us make better decisions as businesses. So,

matt-madden_1_05-10-2024_113918: Well, yeah, but better decisions of businesses, but like better for the consumer, right? And they don't see that enough. So that to me is the big, like, unspoken secret in the industry is like, even the ones who want to serve consumers, the consumers haven't really bought into the fact that you aren't trying to help them.

Not often enough.

john-cordier_1_05-10-2024_113920: Mm hmm.

matt-madden_1_05-10-2024_113918: We've got to figure that out.

john-cordier_1_05-10-2024_113920: So have you seen a scenario where somebody hasn't used data, went with a decision, then it's had, you know, a number of unintended consequences, like, you know, the, the downside of not using [00:32:00] modeling or not using data?

matt-madden_1_05-10-2024_113918: I mean, well. All the people I know are like either data nerds or they're my clients and they're using data. That's why I know them. Um, I think we're probably at a point where it's, um, it's the smaller businesses that aren't using data enough, right? So it's like they're small or medium enterprises that are getting bigger.

Those in the B to B space tend to use data really well. But those that are just in like the consumer space, they just kind of keep doing what they know has been working. Um, I think it makes them probably slow to adapt to changes unless they're a really good listener. Um, but it's hard to get them data at scale like inexpensively enough that it's worth their time, right?

Like if you're, you know, if you're Microsoft or, you know, even like my lab client, true fruit, right? Investing a little bit in like understanding your consumer is going to pay off because you've got enough consumers that like, Oh yeah, if we, you know, increase [00:33:00] something by, you know, 2 percent or, you know, if you're Microsoft by 0.

01%, like a 0. 01 percent increase. Yeah, that covered the budget for the research project easily.

john-cordier_1_05-10-2024_113920: Yep.

matt-madden_1_05-10-2024_113918: when you're a small company, I've literally, we have a lot of small companies reach out to us in the lab and like, we don't do a lot of primary research with them unless they have their own customer list.

Um, I tend to like give them to my digital marketing team to, you know, Run some campaigns and like just do like the campaign analytics that you can get out of a Google or a meta

john-cordier_1_05-10-2024_113920: Mm hmm.

matt-madden_1_05-10-2024_113918: Because like the ROI for a you know, even a 15 to 20 thousand dollar research project, which is really cheap Like that's too much for them to ever get like a positive ROI out of it.

They just can't So democratizing like really good data for smaller businesses and smaller companies. I Don't know. I'd love to see that happen I feel like one cool thing about the lab at BYU is that we are, since it's student run, it makes us less expensive.

john-cordier_1_05-10-2024_113920: Mm

matt-madden_1_05-10-2024_113918: they're [00:34:00] still really smart and they're good at what they do, and I still get to be involved in projects and supervise it.

It's like one thing we're doing to like, get more data accessible to smaller companies. Um,

john-cordier_1_05-10-2024_113920: Mm-Hmm.

matt-madden_1_05-10-2024_113918: I think there's going to be more solutions out there. There's some like good Zappy and some other kind of DIY tools that are doing it. You know, Qualtrics is trying to build stuff in. But Qualtrics is going to be focused on the really high end people.

You know, it might have to be a survey monkey or somebody like that that ends up getting the, the small businesses more involved, otherwise they're just making decisions on gut

john-cordier_1_05-10-2024_113920: Sure,

matt-madden_1_05-10-2024_113918: and oftentimes they're good enough at it that it works.

john-cordier_1_05-10-2024_113920: Yeah. So we like to ask this question to all of our guests. Um, we consider it the, the what, the flux type moment. So, uh, have you been a part of a what, the flux type moment or, you know, have one of these clients come to you after they've had one of those? Kinda, if we could go back,

matt-madden_1_05-10-2024_113918: Oh, good heavens.

john-cordier_1_05-10-2024_113920: to help us. You know, [00:35:00] this is kind of like the having hindsight and like, Oh, I wish we had that foresight two years ago or five years ago.

matt-madden_1_05-10-2024_113918: Yeah. So the modelers we used to get probably one in, I don't know, 10 to 15 projects was just like a disaster recovery project where somebody went out, did a bunch of research, collected a bunch of data and found out it just wasn't useful or the analytics were any good. Um, like probably one of the best stories we got, um, I don't remember the end client now.

There were, there were pharmaceutical companies, so they were doing like over the counter drugs. So you know, it would have been like a Tylenol or Advil type thing. Um, and they got all this data from, you know, I don't know if it was Kroger, Smith's, but basically all this like loyalty shopper data that was accessible, right?

Like this is why you have to type in your phone number to get points at Smith's. It's not that they want to give you a coupon. Is that they can tie all the purchases back to you. Um, so they got access to this data and they were so excited about it. I said, okay, well we need to [00:36:00] figure out like when we need to ramp up production and when it's worth going and like putting a merchandising display in the stores and stuff so that we can sell more of our product right, you know, around the right time when, you know, flu illness season is going to hit, which is different in like every region and city around the world.

Um, they were pumped about it and they had some people. just run models. And basically I think what they did is they just like selected the sales of, you know, Tylenol Advil and just correlated it with the sales of every other product in the entire store. So it was like, Oh, it's correlated with frozen pizza or you know, hot dog buns or how many people bought Milky Way candy bars or like stuff that you know has no bearing at all on, Helping us predict like when is flu season going to hit based on what other people are starting to buy?

Um, you know, this is the downside of having access to too much data is, you know, people do this data mining [00:37:00] and they just find spurious correlations all the time. If you want to be entertained, go to spurious correlations. com. It has some of the funniest spurious correlations you'll ever see. Um, you know, it's like the, you know, shark attacks being predicted by the number of Kevin Bacon movies released in a year.

really tightly correlated for the past 20 years, you know,

john-cordier_1_05-10-2024_113920: Nice.

matt-madden_1_05-10-2024_113918: um, clearly unrelated unless Kevin Bacon's doing something we don't know about. Um, you know, so it's disaster recovery like that, where they think they have a lot of data and therefore they have a lot of potential insight. Um, more data doesn't always make for better insights if you don't know how to use it well.

Right. And especially if the data is just not set up properly, if you're trying to do forecasting, you better know how. To like do causal analysis and look at time lags and set some hypotheses up that actually might make sense so that when you get an answer out, you know, the client doesn't laugh you out of the room.

Um, yeah, so that client basically spent a bunch with some other analytics agency, laughed them out of the [00:38:00] room and then started asking for referrals and then came to us to, to build the backup model.

john-cordier_1_05-10-2024_113920: Noted.

matt-madden_1_05-10-2024_113918: it was not the first nor the last time it happened.

john-cordier_1_05-10-2024_113920: so, uh, with the modelers and then even with what you're seeing, it's like some of the students can be into today. Uh, like where do you see like modeling and modeling forecasting simulation being used or not used in machine learning? And like where, you know, where's there a good overlap? Where's there tension? And do you think at some point in time, like modeling and simulation would become as common speak as machine learning or AI is today?

matt-madden_1_05-10-2024_113918: Probably, they're, they're probably going to end up turning into the same thing, eventually. Hopefully. Hopefully. Because, like, if you ask most people, they, if you say, oh yeah, what do you think about modeling versus machine learning, they'd be like, it's the same? Like they don't even know, right? If they know the word regression, they're in the probably top 10 [00:39:00] percent of most educated people on earth.

From a practical standpoint though, I don't see, I don't see how the two don't end up merging into one, um, just because we're going to get bigger and bigger. Basically like software architectures that are going to bring in like the simulation side that has like really good data that represents the way people actually behave.

Um, and it's going to be, you know, used to make really accurate predictions and the predictions are going to get better because it's going to use all of this machine learning and generative AI, you know, these large language models, um, input and influence as part of the forecasting process. Because. While it used to be ridiculously hard, you know, the people that are building out these architectures are making it really easy and accessible to more people that just are like, you know, good, smart, general business people that don't have to have an advanced degree in stats or analytics

john-cordier_1_05-10-2024_113920: Mhm.

matt-madden_1_05-10-2024_113918: use the tools anymore, [00:40:00] um, which, you know, for better or worse, like I think for the most part is probably going to be a good thing for the world and a good thing for business, um, You know, that's always what scares people, like in the transition time, it's like, man, I look at what I can teach my students to do now, and it's like, I'm teaching them stuff that used to be so advanced when I was in school, and they're just like, they can churn through it, the software is awesome, they don't have to really understand what an experimental design is the way that I used to.

Cause like the software just kind of takes care of that for them for the most part. Um, you know, so all these assumptions and levels of learning, like they're just getting baked into the like system and the software, um, I don't know, someone out there is going to have to keep learning the basics so that they can like keep building the next generation of software, but for everyone else, I don't know, it's kind of awesome that you don't have to have an advanced stats degree to do some like really cool analytics work anymore because the software tools are so helpful now.[00:41:00] 

john-cordier_1_05-10-2024_113920: Do you think that's been more of an advancement in like user interface design or visualizations? Like, what's the main driver of like when there's, know, and again, like we're coming at this from, you know, agent based simulation stuff, like challenging to build models and the like. Um, is it the ease of building a model or is it really the time to insight from what's coming off of those models that has spurred the adoption more quickly?

matt-madden_1_05-10-2024_113918: Yeah, I think it's time to insight. I think it's because people like you are doing the hard work of making a really robust model on the back end that nobody has to like see. the complexity of how that model is running, because that's the really hard part. And that's the part that nobody else knows how to do or wants to do, or at least, you know, 90, 95 percent of people don't want to do, nor do they even really want to understand it.

So it's like people like you are putting all the hard effort in on the back end to make the model [00:42:00] work. Um, and now it's getting to the point where you can Put that into a piece of software and link it with other data points. And like these queries can come in from online so easy now compared to what it used to be.

You know, we're still sort of teaching some of our students how to write SQL queries. I mean, I'm guessing in five to 10 years, no one's going to know how to write a SQL query coming out of a college program anymore. It's, it's all point and click.

john-cordier_1_05-10-2024_113920: Yep. Uh, I think there is a job description that came out from NASA years ago. Like, Hey, like who knows how to program in cobalt because like so much of our stuff is built in cobalt and, uh, like we can't really find people come out of school with that skillset anymore. And like, running satellites up in space from, you know,

matt-madden_1_05-10-2024_113918: I think like the entire like New York Metro system is running on it. So they were trying to update it and they're like only able to hire these, you know, 55 year olds that used to just have this like old coding language and they sort of [00:43:00] died out and they had to shift to more modern languages and now they're making bank on their old cobalt skulls.

It's,

john-cordier_1_05-10-2024_113920: yeah. Like, do you see. I guess it'd be like language, like people are doing everything in Python now, a little bit of R, like, or what else are people learning today?

matt-madden_1_05-10-2024_113918: um, yeah, everything I see right now, it's, it's R and Python. I mean, if you're still kind of in the stats and analytics game, R has a kind of a good place in, in the world. But I mean, if you look at the rates of Python adoption, it's, it's off the charts, like even the stuff I'm doing, um, like my coding language, my first coding languages really are, um, but like with.

You know, with the tools like a chat GPT pro license, I can just feed it my R code and say, turn this into Python code. And now I know how to code in Python. Like there's certain problems you obviously can't solve unless you really know how to code. [00:44:00] Um, but I think that's, everybody's going to end up, you know, in a couple of different platforms.

It's going to be Python. It's going to be, you know, some of the app based languages, if you're going to create apps for Google or Apple, um, you know, they're still going to have some of their own languages. Microsoft's tried to open up. You know, even though, you know, they've tried to open up kind of coding and language so that it's a much more open source game, um, at least that people start in.

So, eh, I don't know, we'll see. The, the issue is it's just going to be higher level programming languages that end up being what people write. And it's just like natural language queries. And that's why That's why these generative AI tools got so cool so fast because it's like, Oh, I don't have to know how to write in a programming language.

I can just write plain English and it gives me stuff back in plain English. Uh, yeah, I mean the last project I did, I literally, when I was, I was coding some stuff up in R and doing a couple of things I hadn't done before, and even some things I had done before, instead of [00:45:00] like digging through my own documentation and past projects, I literally just had a GPT window open and I was just like, Hey, remind me how to.

Do X, Y, Z, and it's like, Oh, here's an example of how to do it. I'm like, Oh yeah, recognize that copy, paste, and then adjust it to my own data. That's how the game is starting to change for most developers. At least, um, I don't know. I think the next gen it's just, you're going to have a very small group of people that know how to actually code in programming languages and everybody else is going to be no code.

And in some ways that's going to be. Amazing. And in other ways, it's like, all right, I guess people who are like super committed to still knowing how to code. We'll probably have a big advantage, um, you know, in the years to come because he's going to maintain. Yeah. Like who's going to maintain all this stuff?

Somebody has to do it, right? They're going to be the next generation of the cobalt programmers.

john-cordier_1_05-10-2024_113920: Cool. Uh, time for like one or [00:46:00] two more questions. Um, You know, any shout outs that you want to give to folks at BYU or others that you've worked with on like, these have been some of your heroes in modeling and stats in the past.

matt-madden_1_05-10-2024_113918: Um, yes, I mean, I've got so many awesome BYU colleagues. Um, I love working with them. Um, you know, like I said, Jeff Dotson is one of the smartest people I know. Um, and, um, Mark Dotson, his, his brother, who I used to work with at the modelers before he got a PhD and went to BYU, um, they do some awesome work in analytics.

Um, yeah. So if people are looking for somebody to like solve a tough problem and you're trying to get a freelancer, I would say, yeah, get, give those guys a call. Um, you know, obviously you can call me in my marketing lab, but I love those two guys. Um, and then, you know, Jeff Brazil was kind of my mentor.

He's one of the founders of the modelers. Um, he got a PhD under Jordan Luviere. Who's like the godfather of max diff and conjoint. Um, [00:47:00] and he runs a company called blue owl AI now. Um, and they're doing some really cool work, um, that's basically just trying to take this advanced analytics thing that we were doing at the modelers, you know, that we started 20 years ago and modernize it with more of the AI tools, you know, basically make time to insight faster and faster, um, but not lose the advantages of the advanced analytics because a lot of the quick, like faster time to insight, it's being done without like real analysis getting done.

And it's just like, Oh yeah, we can. You know, go query that or point and click our way to get some data out, but it's just pulling a bunch of averages. It's still not doing anything to like find underlying drivers of how people are making decisions. So yeah, Jeff at Blue Owl is doing some awesome stuff in that space.

john-cordier_1_05-10-2024_113920: Cool. final question. When you, uh, you know, students are graduating, uh, it's kind of that time of year right now. What is some of your best advice or, uh, a lesson that you hope they take, [00:48:00] take with them once they finish the program?

matt-madden_1_05-10-2024_113918: Oh, that's good. Yeah, mine graduated. I got to read names at graduation this year, which is kind of awesome as they come up and you're trying to keep it like decorum at the convocation as you read their names, but it's like, I know all these students and so I'm just like, got this stupid grin on my face the whole time.

And some of them trying to like high five or fist bump as they come through. And I'm just like. Dean Madrian's not going to like that. This is a decorous occasion. Um, as they leave, there's basically two things I go through. Um, one, I try and remind them all about kind of these principles of resiliency. So I teach a resiliency class.

Um, it's kind of a unit that I cover in my marketing consulting class. Uh, it's a class that I teach that kind of preps them to work in marketing lab for me. So they take it their first year in the program. Um, but like burnout is a real thing. Um, and you don't avoid burnout by avoiding hard work, right?

Like you avoid burnout by building up, um, your capacity to recover from [00:49:00] setbacks and difficult times. And, and you build it up by like developing good habits. Like how do you reset during the day? Right now we're all, right? We've got these stupid things in our pocket. And so it's like you have a spare moment and it's just so easy to open it up and catch up on emails and texts.

And then you dive into like a social media feed. It's not a very healthy way to reset your brain. Um, because you need to have more, you know, I, what we call positive but low energy experiences. Whereas like using your smartphone, It's not always a positive experience. There's a ton of outrage on there. Um, it is low energy, but if it's low energy, negative experiences, that's not how you kind of bounce back and recover to avoid burnout.

john-cordier_1_05-10-2024_113920: Mm

matt-madden_1_05-10-2024_113918: Um, you know, so it's like, go get out, go for a walk, talk to a friend, give somebody a call that you haven't talked to in a while, like go engage, do something else that brings you a little bit of joy, um, besides getting sucked into the pathologies [00:50:00] of smartphones. Um, so that's like the number one thing I give them and then I just tell them they need to learn really carefully, um, when to say yes versus no, because at least the students coming out of my program, they say yes to everything like there there's, and they're, and they crush it anyway.

That's what's ridiculous, right? Like these kids coming out of the BYU business school, like they're so smart, they're so driven, they work so hard, they say yes to everything and then they do well at everything. Um. Until finally they don't and it's like their big wake up call. They're like, oh my gosh, I really can't do it all because people will just keep giving me more and more to do.

Um, so I had a student ask me this. She was like, hey, what's the one piece of advice that you'd give all of us? And I just, I went on the board and I just wrote, no thanks. Like learn how to say this to people. Hey, are you interested in this? You know, that sounds really cool. I'm excited that you're doing that.

Thank you for the invite. Um, I can't do that right now cause I've got. These other things that I gotta take [00:51:00] care of that are priorities at the moment. So for young Energetic students like learning when to say no is so important and don't say no to everything say it But yes, say yes to the right things and say no to more.

At least my students you all say yes to too much Stop it Prioritize it

john-cordier_1_05-10-2024_113920: Now that's a great way to end.

​