Chris Parker: Hello, this is Chris Parker with the Business Simplicity podcast and I had a conversation with Hugo Alves who’s the chief product officer of Synthetic Users and this is used by enterprises to do market research with almost an infinite number of synthetic users using Gen AI. I have experimented it with myself and so we go through the journey the origin story of Synthetic Users as a company which is fascinating and Hugo’s coming from psychology research and how it evolved into this quite amazing product and then we also go into my own experience and what I got right what I got wrong and what I learned and how it really shifted my own perspective about my own proposition which I really did not expect going through this it was very confronting in a very positive way. So Hugo, why would it be valuable for people to listen to this conversation?

Hugo Alves: So I think one of the biggest value that they can take is that they can understand what is the purpose of Synthetic Users. Get a little bit of an understanding of how it is built and why we perform the way we do and why we are different from other more general purpose tools like Chat GPT. It’s so easy to think, oh, but this could be done with Chat GPT. And we go in depth as to why those differences really matter when we’re trying to get business effective business results.

Chris Parker: Welcome to the Business Simplicity podcast where Chris Parker explores how leaders cut through complexity to accelerate strategy, execution, and growth with calm, clarity, and confidence. Welcome back to the Business Simplicity podcast. This is Chris Parker and I’m having a conversation with Hugo Alves who is the chief product officer of Synthetic Users. He’s based in Portugal, Lisbon. I think I first came across him when he was presenting this amazing solution. I think it’s amazing on the Gen AI circle meetups and then Gen AI circle is a group bit curated group of about 400 people around the world that are really pushing the limit on AI Gen AI and Synthetic Users is definitely in that space. So, what we’re going to do today is hear a little bit of the origin story from Hugo. Discover a bit who he is and what the company’s all about. And then I’ve also actually applied Synthetic Users to my own company and another company. And I’ve got some questions, feedback, and I just want to unpack that in a really interesting way. So there’s going to be kind of two parts to this. So one, let’s discover what the company and solutions all about and then go into my practical experience. So outstanding. So Hugo, thank you so much for joining. Can you maybe kick us off like the word Synthetic Users? I guess it’s in the name, but can you unpack a little bit about just to start it off strong, what is Synthetic Users and why does it exist?

Hugo Alves: Okay. I think Synthetic Users is one of those products that it does what it says on the tin. The name is quite explicit but for those who might not be as familiar with the space we create synthetic people that can be used for the purposes of research. It can be user research. It can be market research. So what we create is person descriptions that could belong to a particular group to whatever requirement my customers provide. They can be busy moms from Texas. It can be B2B marketing managers at large corporations. It can be even rural farmers from Botswana. And we create versions of people that could belong to those groups that can be used for to be they can be interviewed for any purpose that you want. But typically when you’re building products on when you’re creating new services, you want to have an understanding of the people you’re building for. And sometimes it’s not that easy to find them. And we can accelerate that process. We’re not a replacement for speaking with humans, but we’re a complement for those situations in which you’re willing to do a little bit of a trade-off between speed and accuracy and or you just want to have a first way of the lands to be better informed when you go talk to organic people.

Chris Parker: And who is in your mind the perfect user for this like and what is the job that they’re doing? So if someone’s coming to Synthetic Users, they have a problem in mind that they’re trying to resolve who is that ICP if you will and what is the job they’re doing with Synthetic Users?

Hugo Alves: So typically the way we look at it, it’s insights managers. So people work at large corporations that in which there are different groups working in different service areas, working with different new products in innovations and there’s a constant need for understanding a particular group of people, understanding how a particular new concept might resonate with those people to get feedback, early feedback. So it will be sure that you’re not going on a path that you will end up having to walk back from because humans end up saying that doesn’t work. So whoever is responsible for large research projects and frequent research projects in which innovation and product creation and service design are a focus.

Chris Parker: Yeah. And I guess Because you said large corporates, but I also when I was using it could attribute it to like lean startup, meaning validate fast and kill fast. I’m sure there’s incredible value for smaller organizations that are like, hey, I’ve got this cool idea. It’s really cool but and let me talk to a number of users. And instead of having to go down to the local mall and beg people to look at your widget.

Hugo Alves: Indeed. Indeed. And to be fair, that’s when we started when me and Pwame first started the product the way we were thinking about our GTM or go to market strategy or whatever people like to call it. We wanted to be Canva for research. Cool. And kind of the analogy was always that in the same way that my mom doesn’t open Photoshop to make a birthday card for my son, she can she does use Canva to do that because what it did was it kind of democratized something that was the domain of only experts and all of that. And we thought we can do the same for research. We can make research accessible for people who don’t know how to do it. They don’t know how to do a non-leading question. they don’t know how to recruit. They don’t know all the care that you need to put in research for the results to be valuable. And we can help those people. We can help UX freelancers that change customers every month and need to get up to speed with a new vertical or a new domain every month. But we can we come across a couple of challenges. Some of one of is if we’re trying to sell these to people who don’t recognize the value of research, there’s a huge education piece that you need to do and that is always a challenge when building new services and new products. And if you’re talking about UX freelancers and all of that, the it’s not recurring enough the challenge to justify a subscription-based model, which was what we were going for for different reasons, predictability, all of that. And then another thing happened was we were struggling to go for that what’s typically called the lower end of the market. It we weren’t getting as much penetration as we would like to. And then I suddenly start getting emails that I believed were phishing emails because I was like there’s no way this company I can’t name names because this is kind of a too hot of a topic on mind for people to be able to say yes we use Synthetic Users. They use it. happy, but getting into a flame board on LinkedIn is not really what these people want. But I was getting emails from companies that I thought it’s impossible that I’m getting an email from one of the biggest financial firms in the world. It’s impossible. This is a phishing email. And I’m talking to Pwame. I say I’m pretty sure this is a phishing email. They’re asking for a demo, but I’m not sure. Then I started digging and I was like, no, no. I we really are getting an email from this company. And we we ended up closing them as a customer after one year and six months because that’s how enterprise compliance and all of that works but we started filling some pull from larger firms and larger firms if you think about it are better match for us because that point that I mentioned have different groups working with different in different products in different services. So the need for research is constant and it it makes a lot more sense with the business model that we were going for. So we decided to focus for now on more of the enterprise side of businesses. Some we have some Fortune 500 companies. We have some Fortune 100 companies. And we decide to focus on that. We still have this aspiration to be to be able to help mom and pop shops. But I have this thing on my mind. I’ve been I’ve been trying this. I’ve been tagging him quite frequently on Twitter. I have this on my mind that one ideal customer for us would be Shopify because Shopify helps so many people build businesses and I think they could perform even better if they provided the service like ours to their merchants. So the person who’s selling now some jewelry online that is homemade and all can understand what other types of jewelry could they sell. Maybe someone who sells dog food could figure out what other animals could also I don’t know. I think Synthetic Users could be an amazing partner for Shopify. Toby, if you’re listening, here I am again trying to convince you. I think we could be an amazing partner for Shopify because it would allow us to do something that we wanted to do in the beginning to help small businesses understand their customers better and build better products and services, but do it immediately at a scale that will have an amazing impact.

Chris Parker: Yeah. Outstanding. And can you just give me a few words on where it came from? So like how did this emerge because I see on your LinkedIn that you’ve been the chief product officer there since 2023. So it’s not very old. But it must have emerged from some sort of trigger or need in your life or some observation. And then here we are. So where did this come from?

Hugo Alves: So this goes back a lot. So my academic background is quite different. It’s not a techie background. My academic background is clinical psychology. I did a masters in clinical psychology here in Lisbon. And when I finished my masters, I started working at my university as the lab manager for six years. So for 6 years I was the person responsible for managing the day-to-day operations of the psychology lab at ISCTE. This actually my job was helping run experimental studies helping with data analysis figuring out new experimental paradigms and a really core part of my job was managing the participant pool. The participant pool is essentially the who is available to be part of studies. And if you’re familiar with academic research in cognitive science, psychology and a lot of topics, you know that first and second year students are kind of the bread and butter of a participant pool. They go they spend half an hour in front of computer making some task and then they get some course credits. And every semester I ran into a big challenge which was I would have some researcher come knock on my door and say Hugo I need 200 participants for this study I need 50 for this one and I need 100 more to pre-test some materials that I want to use in 6 months and every semester I would end up with a balance sheet in terms of participants that didn’t fit. I’ve always I was always lacking participants and this so this challenge of finding people for us to be able to understand and to be able to make sense of what is the human condition and how does do humans work what do they feel what do they think what do they reason about is something that I’ve kind of chased me throughout my life because when I when I was working there I decided this was early days of mobile mobile was finally taking off. So people were starting to have smartphones and tablets. It started to become something quite normal. And I thought people have kind of moments in their lives which when they’re bored, they don’t have anything to do. Why don’t I build a business a service in which researchers can publish their studies to an app and people can be waiting for in the doctor’s office and they can be helping science and making money at the same time. So I left my job at my university to build this but I didn’t have a developer. I wasn’t a developer. So I joined my co my co-founders agency had a design and development and innovation agency at the time. I joined his agency with the idea of stealing a developer. I’m going to join as a social media manager and while I’m there I’m going to convince one of the developers that this is the best idea in the world and we’re going to be building we’re going to build this together. Truth is that I didn’t wasn’t able to convince a developer to join me. Developers are pitched ideas all the time and they’re quite picky in correctly so in what they decide to pursue. But what ended up happening was that I became a product manager and I’ve been a product manager for the last 11 or 12 years in a lot of different domains B2C B2B really early stage companies more on the growth side some AI stuff there too and one of the challenges that as a product manager you keep running into is you know how important it is to do research you know how intro to building great products and great services is to having understanding whoever you’re building for. And you also know that it’s not as easy as it should be that sometimes you’re building products for people are really hard to reach either because they have some rare disease or because they’re the CEO of companies and CEOs don’t just stop and spend half an hour talking with you about the challenges they face with CRM. It’s not it. So there’s challenges in terms of recruiting who who hasn’t been in a session that simply the participant doesn’t show up. It’s expensive. It takes a lot of time. the analysis the all takes it’s it has too much friction and when me and Pwame decided to start building with large language models this was mid 22 before Chat GPT and we we met 11 years ago when I when I was trying to poach one of his developers but we stayed friends and we worked together in a couple of different things before steps And in 2022 I’m sending him messages. Hey, have you seen this with GPT-3 at the time showing him what was now possible and one day he turns to me and says hey come by the office on Monday and on Monday I was in the office and I was like let’s figure out this is a fundamental revolution in terms of capabilities. let’s figure out how can we apply this new stack to help people build better services and better products. But at the time we weren’t sure of how to do that. So we did a lot of different Miro prototypes. So we we decided like okay so Miro is one of those tools that almost everyone uses in product be it the UX researchers be it the designers be it the PMs be it the engineers so let’s build some tools on top of Miro to help and we built a concept map explorer you would put a concept in the center click a button it would create a mind map now this is kind of common but I think we were the first to do this we built a planet-centric canvas assistant. So planet-centric canvas is essentially a way for you to be more aware of second order impacts of the products that you build. Instead of just mapping the user, you map the communities in which the user is involved or the communities that are involved in the sourcing of the materials of whatever you want to build. So it helps teams be more aware of the global impact that their products might have. We build a business model canvas assistant. So you would just give it the your idea of your business and it would fill all the sections of the business model canvas and then we built a persona canvas and we ended up pitching this all of these tools to one big VC firm. I remember being in the call with them and we’re showing all the tools and then being asked so which one are you doing? And I was which one? No, I’m doing all of them. This is kind of a product suite. This is going to help people with different needs. And and they said, “Oh, okay.” And then we we I left the call and I was like, “They’re right. They’re right. They’re absolutely right. I can’t be building all of this.”

Chris Parker: In the call that I seem to remember on the Gen AI circle, did they like confront you with that and they say, “Hugo, you got to decide.” Or did you reveal Did you figure that out afterwards on your own?

Hugo Alves: They didn’t tell me I needed to pick. They implied that they were expecting me to pick one of them. There’s some wisdom there, though.

Chris Parker: Yeah. Yeah. Yeah. Yeah. Of course. Of course. I’m glad they did.

Hugo Alves: So me and Pwame after the call I turn to Pwame like we need to go do some user research. we need to go talk to some PMs and some UX designers and we need to do some user research and like oh my god it’s going to take so long and suddenly I take the persona canvas that I had built and I I create product design and I ask him hey what are the main challenges you face and suddenly he replies with oh it’s hard to balance aesthetics with functionality it’s hard to recruit people to understand what they need and I’m like Ah, we have something here. We have a people simulator. This is this is going to be our thing. First of all, we don’t want to go for the oh, we’re going to summarize user interviews that they’re going to be hundreds of those companies. There going to be people better positioned than us. Let’s go for this outrageous idea of making of creating people. This is going to be I’m pretty sure that everyone’s going to love us. we were quite aware of how polarizing our product could be. So, we decided to focus exclusively on that. And funny enough, it was called not Synthetic Users, it was called Synths with a Z at the end because I’m really bad at naming. And two days before we launch the our product designer says, “Oh, I was talking with my boyfriend yesterday and he was like, I I don’t know how to pronounce your name and it’s kind of weird to pronounce.” And I just said, “I don’t care. I’m we’re going to be launching in two days anyway. Let’s just call it Synthetic Users for now and then we decide.” What I wasn’t aware at the time was that this name would end up being kind of the de facto category name for this kind of product. And we’ve been quite lucky with having this name because people just search for the Herobot Synthetic Users. They search for it and end up on our website.

Chris Parker: Yeah, it’s nice. The Kleenex of synthetic so to speak. So before I fascinate I love these origin stories because it evolved and someone I researched and I’ve written a book about is Rumi the Persian mystic and philosopher and poet and one of his statements was you will discover the way as you’re walking on the way rough paraphrase and it sounds like you’ve just been with curiosity and intention just walking and then and then also being not too married to your darling. So sounds like you had to kill a lot of darlings along the way. Before I share a little bit about my experience with Synthetic Users and figure out from you what I did right and not right. What is what is the tech stack because this was this is LLM enabled. So you couldn’t have done this really five years ago. So without revealing any secret sauce what is powering these synthetic users?

Hugo Alves: So large language models now we’re supporting vision so it’s not just they’re multimodal models right now too but essentially large language models what we understood was that’s and that’s the reason why we have this prompt engineering description is that these models are extremely capable but you need to be able to you need to understand them really well, to know what are their shortcomings, what are their biases, what are their tendencies, you need to understand even the training process to understand why these models behave in the ways they behave. And each of them has a slightly different flavor. So what we do is I always tell this to investors is that I’m building Legos. someone else is building the Lego pieces and I’m the one who’s assembling them in a way to create something novel. So, we don’t train our own models. We it’s something that is extremely expensive and one lesson that has come throughout these three years of building with AI is that even when you just fine-tune a model, which is something easy, cheaper, all of that, you spend some effort. It’s not expensive, but you need to do put a lot of effort into fine-tuning, creating a good data set to fine-tune a model and then two months after a new state-of-the-art model comes that just beats your fine-tuned model by a lot. So the it’s in my perspective, it’s not worth to invest a lot right now in fine-tuning because the big labs are doing what they do, competing, and the prices are going down. So I’m just privileged to be able to use the models that exist out there. So essentially we use large language models. We have a complex infrastructure because exactly because of the limitations of the models. Sometimes you can’t ask the model to do everything at once. You need to have multi-steps or you have some kind of agent on the background that is managing stuff for you. And we use maybe I would say right now we are using maybe seven or eight different models at different stages of the process of in our in our platform because in some cases speed is extremely important at that stage in that step I need a really fast model. So I’m willing to trade off a little bit of the quality for the speed. In some other cases, it’s like I don’t care. It could take 10 minutes, but I want the best model there is right here. I wanted to reason. I wanted to do all of that. So, it’s a mix of models with then a lot of infrastructure on top of it to coordinate and to kind of crystallize workflows and data streams that we feel can build the best models.

Chris Parker: No. Well, I think let me let me take over and let me explain what I did with the platform and I’m really curious and I think I have some learnings already through that experience. And it and it maybe goes back to your investor thing around focus because I think some of my research might have been not focused enough. But I’ve done five research projects. I was able to get up and running by watching like a nine-minute how-to video. There’s a video someone yeah I guess he said it would be 30 seconds but he immediately said this won’t be 30 seconds but it was like 10 minutes and once you understand that the basic gist of it okay now it’s that so basically it’s that simple okay and I after watching a 10-minute video I can understand how roughly things are happening in the portal and what I decided to do for the first projects is for my own company Ebullient business design I have three propositions there’s strategy Strategy acceleration, execution acceleration, growth acceleration as in personal growth and coaching. And I have three rough personas for that. Strategy is more leadership, execution is more program, get stuff done. And then coaching can be people that are more vibing with me that call me to say, hey, help me out on this challenge or whatever. And I and I ran three projects tying those things together. So, persona 1 to offer one, linearly like that. Fortunately, in the last couple of months, I’ve documented my company pretty extensively using my simple business design framework. And so, I had a real nice wealth of knowledge. And what I use that business design document for is in GPT, I have a I call it my mission bot. And so that helps me like if I ever if I ever need to write something from the context of my business or ask questions I have that in the GPT project as well as some other artifacts and so it’s always context aware exactly what my business is all about. So I had basically everything at my fingertips and I was able to just go fast. The fourth one is I’m involved in a leadership development network and we’re getting together in Switzerland in two weeks and it’s about positioning and sales. So I was like well let me see if I can discover anything from that. Their proposition and persona were less clear. So I had to gather some other artifacts and did some pre-work in GPT in order to prepare it so I could figure out okay what is the proposition what is the customer and then what’s the question as well. I think refining the question is really really important. And then and honestly after doing those four experiments what I also did because you can there’s you can actually it’s amazing to like actually see the answers of these humans. it’s, they’re bought humans, but they’re talking like, oh yeah, last Thursday out over lunch with my wife, I reflected on this and, it was like really human interaction in a really really fascinating way. And then you could then it generates a overall report. There’s some sort of mind map that I was never able get to work in either of them. So, but but anyways, I was I was just chatting with with the rest of the team because I broke the mind the knowledge graph the mind map and I need to fix it. I was like I need this is the my task for the day.

Hugo Alves: Well, I noticed that wasn’t that wasn’t but anyways the overall summary report is really revealing and then you could also download the entire research set in multiple formats. And as an experiment, I dropped a research set into Notebook LM and had it generate a podcast for me because I was like, “Okay, well, what is the essence of this?” And I did that for the leadership development network. And it was it was incredible. It was even though the proposition and persona weren’t totally bang on. I think it was really affirming of a lot of the assumptions we had. And then that got me suspicious. I was like wait a second wait a second because I don’t like affirming. Exactly. Wait like wait a second is this just GPT wanting to please me and just like oh Chris you’re the greatest thing ever like no you little lying bastard machine. And so what I did is I and I advise this to my clients sometimes when I do custom experience work. I use GPT to spin up the most aggressively horrible anti-persona I possibly could. Just everything that I don’t want. Everything everything that is just not who I want to work for. And then that was the fifth experiment I ran and I said, “Okay, let me let me do the same essential questions of my first three, but asking this anti-persona.” How would he love or she love to buy from me? And and and fortunately at the end of like no, they are just not interesting. Some quotes, I’d be cautious about buying from this company right now. While the solution addresses some of the key challenges like alignment and execution delays, there are too many unknowns, unclear ROI, vague implementation, no guarantee and I was like okay okay all right that it was confronting to read it honestly the next one honestly I’d be cautious about buying from this company right now good in theory but I want use cases I want proof I want demonstrated tangible ROI I asked it to be the most hard ass a type personality. ENTJ. fact driven, not empathetic, everything that I am not. And fortunately it there was not a fit. There was not a fit. But what what I but what I discovered through that process was these are valid criticisms of my positioning and my proposition. So I was like okay well a I do want to repel these people but these are this is real valid feedback about gaps in my messaging and gaps in my positioning and so it was really educational. Now, my learning from this and then I’ll hand it over to you to maybe fill in some of the blanks is knowing upfront and this is where because I’m not a research professional, maybe I got it too wrong. I probably went too broad too broad. Knowing upfront what is the hypothesis that you want the actionable answer to I think would for me would have made this more valuable because this has been an amazing valuable experience but maybe having a tighter line on this is the experiment I’m running and I’m sure proper research analysts have frameworks for that. So was but this is the journey that I went through and I would just throw it back to you and I’m just curious what did I do right and what did I just completely miss in this whole process.

Chris Parker: I don’t think you completely missed anything. I think in general you use the tool as we have kind of the expectation one there’s a really important aspect here which is it’s just a tool. it’s just a tool. Any tool that we interact with has a modeling curve, has a let me figure out how to best use this. Some of them the modeling curve is really shallow. It’s like, hey, in 5 minutes I’m an expert. In some of them, the learning curve is really hard. If you talk about people who use Salesforce, they’re going to tell it’s a juggernaut. it’s a juggernaut and it’s not easy to use because the use cases for in which it is used they’re complex ones and there’s a lot of challenge. So there’s huge variability in tools and tools involve a learning process of how to best interact with them. And I know you didn’t have a lot of time because I sent you the link to sign up quite recently. But that’s it. you’re we it’s the same with anything a designer when Figma first came out there was already sketch as kind of the anchor to that kind of interaction. But but Figma was a new one. It has different names. It have different flows and ways of building it and it was Sketch was even more different from Photoshop. So when Sketch first came out, it was quite different from Photoshop, which was a lot of what was used for this. So people need to get used to the tool. It’s just that. So I’m pretty sure that if you keep using Synthetic Users, you will get better at Oh, I know I can’t be too generic. So I don’t know how you gave the input. Just give a little bit context for people who are listening to us. The most basic kind of study. The most the simplest kind of study we have in our in our platform is one that we call research goal. Essentially just takes two inputs. Takes an audience. You describe the audience or you give us what would be your recruitment criteria and the research goal. So what do you want to learn about? So who and what you want to word about and I’m not sure how you went about it but we have that conversation of interface when you first enter this and the reason why we built the conversation of interface was particularly early on when we were still free and people anyone could drop by and run some research. What happened a lot was that on the audience people would people would sign up to experiment would just say people and if you’ve run research and if you’ve run anything you know that people is not a group to be studied. It is in psychology we are trying to understand people in general but we know that for purposes of user or market research there’s no such thing as people because people involve an old person in rural China that lives just from what they cultivate and it involves the highest paid Jeff Bezos so people is too much of a diverse group and that conversational interface that we have there is there exactly to make sure it asks probing questions. You say people and says, “Oh, okay. But what type of people? What?”

Hugo Alves: What I discovered was indeed it was it was probing and it was conversational. It wasn’t just two inputs. It says, “Okay, who who are who’s the persona?” And I gave it my definition of a persona discuss description. And then it asked clarifying questions and then it says, “Okay, what’s your goal?” I said, “Well, what do they think of this proposition?” And then it asked refining questions there. And then at some point, I said, “Just go for it.” And then

Chris Parker: Yeah. Yeah. Yeah. And then yeah it kept it discovery. Yeah. It does exactly because of that because we know how important everyone knows this in conventional research and synthetic research the same the bad the inputs make a lot of difference in terms of the outputs. So if your research question is a crisp one something that is already kind of purpose with the context that you need all of that your results are going to be best. the same with synthetic users. It’s just the same. The big advantage and this is I think people sometimes forget how big this is. The big advantage of a product like mine is you can learn my product by spending an afternoon because you run a study and you’re like a little bit too generic. Okay, let me run another one without going through the recruitment process without anything. I can generate anyone I want in five minutes. and let’s iterate. Let’s run. Oh, this one is a little bit better. or you feel it’s a little bit too positive and you look at it like oh my research goal is was kind of a leading was written in a leading way. So let me just rephrase it to say that I’m working with some company and I want this to be destroyed and then the interviewer that we generate is going to be a lot more aggressive in the questions that he ask. So the ability to quickly iterate on both the audience we might feel we might generate the first one and say oh I forgot to say that I only want people from Germany right now because we’re focused on the German market and it gave me people from all across Europe. You can do a new study in five minutes. So this the ability to iterate and to make studies not a oh let’s do a study and let’s spend two weeks recruiting and doing all of that to hey I just ran one and let me run another one.

Hugo Alves: Yeah, I think I did three sessions because indeed after I did the first three on myself and then gave it a night and then after reading it and that was probably two hours of work maybe and then and then all this other stuff starts sparking and you’re like okay well what okay let me do it from another perspective and then let me go out and get that enablers network thing and brought that in and then and then reading that and then that’s when I got this this is very affirmative I want to come at it a different way but it took hours not days.

Chris Parker: Yeah. Yeah. Right. The affirmative. So the affirmative is a common criticism because people are used to interact with products like Chat GPT which recently had one big challenge with the sycophancy issue is an issue since day one with large language models. By the way they are trained. They are pleasing engines. They want to please whoever they’re interacting with. And that’s why a product, a specialized product like mine comes in because if you go to Chat GPT and you say, can you can you generate some personas that could be mashed potato lovers? And he gives you five personas. You say, oh, okay, now pretend you’re one of them and give me feedback on a really terrible idea. And they’re going to say, oh, I love that idea. Because Chat GPT by the way the system prompt is designed by all of the training process that goes there tends to be that pleasing companion to a fault to a huge fault and people don’t people are now kind of pushing back because people want to be challenged. Of course we get comfort from being confirmed but we know that for us to be successful and to be good people in the world we need to be challenged by our family by our friends by our colleagues. So we know the importance of it. So if you use Chat GPT, you’re going to get that kind of result. That’s one of the things that our product I think does best, but it still sometimes goes as you saw into more the affirmative side is to be more challenging because if you run if you create a mashed potato lover on Synthetic Users and then you promote some kind of stupid idea to him, he’s going to say, “No, I don’t like that idea. that idea is worthless or I don’t see why you’re even spending time doing that. So we are not the sycophancy engine that Chat GPT is although it’s something that we need constantly through our evaluation process to be evaluating to know exactly which models have more of this tendency which models need more of a prompt towards a different direction.

Hugo Alves: No. And when I was setting up the research and I don’t know the right answers, you have to pick the number of people, synthetic people. And I think I picked eight or five, I don’t know, but it So, a couple questions on that. What’s What’s the sweet spot of number of respondents? And is there like this library of respondents out there or does it really spin up?

Chris Parker: No, they spin it spins up. it would be impossible for me to predict the respondents that my customers would ask me to generate. So there’s no pre-built. It’s all on the fly. I think that’s one of the powers. The how many interviews should I write on each study. It’s my failure as a product person to have been asked that question so many times. It’s my failure. I don’t know how to sort this out. It’s I always go back to the it’s a tool. It’s a matter of you experimenting. But we don’t allow for example we just increased the maximum amount of interviews per study to 20. Before we it was only 10. Why did we only have 10? Because in the same you have the same concept with organic research which is saturation. Saturation means that you’re running a new interview and no new topics are surfacing. you’ve explored in the previous interviews, you’ve explored what was there to be explored and you’re not getting anything new. So at that time it’s not worth running any more research at least with the same script with the same research guide with the same goal. You might narrow, you might drift, you might go to some other adjacent topic, but with the same one it’s not worse. And what we discovered or VR is one thing that you’re going to notice in our interviews in comparison to organic ones is that the answers that synthetic users give are more are richer are more in depth than humans. And in the early days there were two things that in the early days we had to kind of decide was do we want to make the the so there’s two important aspects in any answer. One is the content which is kind of what topics are mentioned, what challenges, what pains, what goals. So is the the semantic content of the answer the same that you would get from a human interview. And the other one is the form. And when you run user interviews, people some you go through the transcripts and they’re hard to read because when people are speaking, they go on detours. They stop mid-sentence. It’s not fluent. And early on we had to make a decision. We want to make this as human as possible. So both content and form and it’s harder for my customers because they’re going to have what it’s going to be less it’s not going to be that easy to read an interview and we need to run longer interviews to extract everything or do we want just to focus on the content and not the form and we decide just for that.

Hugo Alves: My feeling is eight was more than enough for me. Maybe maybe too many. Maybe five would have been enough. But then but it’s amazing how human it is. And so one of the personas on my anti- study was Marcus Chen. And then there’s this sort of openness, conscientiousness, extraversion, agreeableness, neuroticism. which was quite those are quite tight.

Chris Parker: Those are that’s called the big five or the ocean model. It’s the most well validated psychology model. Myers-Briggs tends to be the one that people talk about more, but this is the most scientific one. And I’ll think also I hear about this big five more in marketing and research indeed. And his name’s Marcus Chen. He’s 44 from New York. He’s a CEO. always decision-making style data driven leadership approach command and control risk tolerance moderate work life balance work focused communication preferences direct and concise so not somebody I would hang out with a lot but the first question is what are the most pressing challenges you’re currently facing so that and I didn’t ask that question that the platform asked that because I think it was kind of an orientation question and indeed you can get to know this person and actually also sympathize with this person like like well maybe Marcus wouldn’t be so bad because it’s like one of the biggest challenges I’m dealing with right now is is driven by the inconsistency I see across departments when it comes to executing on key initiatives okay targets are often clear we put the metrics on paper align with them with broader organizational goals laid out the plan but somewhere in the middle there’s a disconnect and honestly that’s my sweet spot that’s why I call it strategy acceleration I’m not going to make strategy for Marcus but I could help Marcus land the strategy with the humans and it’s so I’m like wow I could Marcus okay I can get you and then couple paragraphs down the common thread here is I’m running up against resistance misalignment operational friction that’s draining momentum and for someone like me who thrives on speed performance and results becomes not an organizational issue but a personal frustration too it’s like well this is all catnip for me like wait these tensions are super relevant and that’s all stuff so I expected to hate these people. But as going through this, I was like, “Yeah, there’s value over there.” It was really really fascinating and deeply human and it inspired an empathy from within me through reading through these things. So

Hugo Alves: I think a product like ours can be we joke around where they’re empathy engines. So there’s the products that allow you to build empathy. Empathy you know the I walk on someone’s shoes. You don’t walk on someone’s shoes but you understand them and you understand their motivations and their needs and what better after reading an interview than not at all. I’ve seen people saying oh no the is so bad that it’s better not to do any research at all and I there’s a joke now on LinkedIn. I normally go there and say, “Have you tried it?” Because that can only come from someone who didn’t experience the product because the product is something that allows you to understand how someone else looks at the world in a quite empathic way. Although we tend to think of, oh no, but it’s this are algorithms and it’s just just this it’s mass. It’s not empathic. What we discovered with AI is that math can be quite empathic which is kind of weird even for me who works on this.

Chris Parker: No, it’s incredible and I guess for me the value is now I also have this artifact meaning I have these full research studies that in the future. Did you ask follow-up questions? So we have this ability on any interview to ask follow-up questions.

Hugo Alves: Yeah. Yeah. Well, that’s the thing is in, and in preparation for this, I went back this morning and I asked and you can tag this little ask to all users or not. I asked a lot of follow-up questions and I tried to keep them consistent. because it seems like the system generates the first 10 questions or so or number amount of questions that gives you an orientation. I’m like, well, I want more sharp. And the last one was, would you buy from this company? Why or why not be as brief as possible? And this is where, I’d like to see proven track record first. And maybe that’s because I didn’t feed it a track record, because I have case studies and credibility, but but this is what this persona is looking for is proof, so that like, okay.

Chris Parker: Yeah. And I for me that the shift has been I believe there’s some real potential value that I can bring to the Marcus Chen of the world as opposed to only working people that I’m more sympathetic with and I think my own messaging and positioning can be shifted on that. And for me this is it’s only through this empathy this reading through this is like wow really interesting. Now, what do people do like? Okay. So, I guess it comes back to the goal of the research and I guess if you’re a professional researcher, you have that goal and you know you’re going to get some outcome and you’re going to make a yes no decision or something like that. Is that the typical where the value really comes from this? So, it’s like if someone’s going to be use Synthetic Users to do some research, the value comes from a decision they take after they’ve done the research, right? or is there is it more?

Hugo Alves: I think right now most people use us not as a replacement of conventional organic research but as a complement in several different stages. So one is just knowing the language. So let me give you a more concrete example. Imagine we’re talking you’re talking about building a new product for or let’s say collectors of stamps. I’m going to go with a kind of weird one. And you feel that hey I collect cards. I’ve built a business around collecting cards but maybe I could extend my business to people who collect stamps. And let me see how this could be played out. So you instead of going immediately and try to find go to forums and try to find stamp collectors, you generate a couple of stamp collectors on Synthetic Users and you generate some interviews to understand what their challenges are, what they would need help with. And from the moment you read the interviews, you understand that they have their specific language to talk about stamps, to talk about businesses, to talk about why they do it. And then when you go talk to a organic to a person out there in the world who’s really a stamp collector, you’re the ability of you to be able to build rapport with that person just went through the roof because you already know the differences between this kind of stamps and these kind of stamps because you already know that one of the big pains is having a hard time keeping them in a pristine condition. So you’ve already built that muscle that now you can do an organic interview and take 10 times the value that you would get you would take if you were directly there. So this is one of the things which is kind of as a preparation for for going talking to a human. One thing that I see a lot happen in my product is that we are used to narrow down options. So you’re a big company. You’re doing an innovation workshop because you want to build a product for a particular demand space and you have at the end of the workshop or mid workshop you have 12 concepts that could play out. You either go and run interviews with with people for each of those concepts to get feedback and decide on which ones to narrow down or you use Synthetic Users to get those 12 to four or five and then you go and use those four or five and go talk to humans to validate that. And the cool thing about this one is that you can then look at what synthetic user said and what human said. And what typically happens with this is that people come back to me and say, “Oh, Hugo, it just boosted my confidence in your product so much because we ended up going talking to humans and humans are talking about one concept and I’m having a deja vu because that’s what your product said about that concept.” And I’m like, “Yes.” So that’s this is one of the things that I say to people and you saw we were talking before we started recording about the post that I did on LinkedIn essentially telling people, hey, if you have an audience and a research goal and you want me to run a study for you, I’ll do it. And one thing that I do to people while I’m demoing is they ask me, “Oh, okay. But how do I know this works?” And I tell them, “Hey, we have a case study. There’s academic research. I could bombard you with academic research in which people who have zero interest in my product. They don’t have anything to do with me. They are testing this idea of large language models being able to reproduce completely different aspects of human nature. So I could kind of bombard you with that. But what I always tell people to do is you’ve probably run some research recently, something that is not kind of obvious. So what I what you should do is you should cut Synthetic Users share of course zero of the outcomes of the research but share the inputs share what was the how did you recruit people and what did you want to learn and run interviews and then you by yourself without me knowing what research you had run or what did my product say you can judge by yourself because there’s nothing like seeing the product work.

Chris Parker: Yeah. Yeah, this is something that we discovered is that if we can get people to this stage of running a comparison study of their research with with what we provide, we’re going to have a quiet we’re going to have a customer. So, is in the spirit of sadly wrapping up because I’m thoroughly enjoying this, but is this the call to action, if you will? So if you are a research analyst and you’re doing some research and you’re triggered by this potential then to reach out to you on LinkedIn and run this kind of sort of blind studies. So take take something you’ve already done maybe in a traditional way and bring in the inputs not the outputs and then compare and contrast in the?

Hugo Alves: Exactly. Yeah. Exactly. Exactly. Exactly. That’s it. That’s for me that’s the ideal situation is come to me you have you’re going to spend zero I’m going to do it for free come to me send me a message and say Hugo can you run this I just need recruitment criteria how did you find the people you interviewed and what were either your questions your goal whatever it is and I’ll just use the inputs of your research and I’ll give you the output of my platform And then it’s up to you to check and compare.

Chris Parker: Incredible. Well, I found it amazing and fascinating and also also in a SaaS, Gen AI product space to see how you clearly have linked together different engines at different steps of the way, to craft people’s, to prompt them through setting up the research and then, of course, executing the research and then and generating the output. Yeah, outstanding. And again for me, it confronted me about some of my biases, some of my assumptions. So, it definitely did the job for me. So, and we’re just humans. We’re just humans and we’re now we’re benefiting from these giant mirrors, the these LLMs that are able to, reflect back. And I believe if you stay curious and cautious sometimes because GPT wants to please you. So be careful. Don’t make life decisions based on GPT advice please. But this does it at a scale. So, if you’re validating, if you’re, I think if you’re making a major decision on a company product, launch, something to validate, there’s really no reason not to do this. Just to say, hey, am I on the right track? What am I missing? How, if this is what I feel this persona group would really appreciate, am I right? because because we oftentimes are so in love with our product that we’re just blind to the irrelevance or whatever. So, some sort of fly in the ointment that could screw it all up.

Hugo Alves: Yeah. I think when we first launched, we didn’t so what you experience now, it’s what we call a dynamic interview. So we generate a synthetic user on the one side and then we generate some kind of an interviewer that will decide on what questions to ask interactively. In the beginning that was not the case in the beginning that it was we generated the synthetic user but the questions were fixed questions for all the two types of interviews we did and one of the interview types was concept testing and at that time four of the eight questions were negative questions were what concerns do you have about this idea or this concept? What problems do you see in the implementation or the consumption of the product? what ways do you see this product being improved? So my questions were disconfirming questions. I didn’t want that my product to be a yes, I think dog food that your baby can also eat is an amazing idea. No, no, I wanted my product to provide.

Chris Parker: That’s you have something. No, the dog will works the other way though. The dog will eat the baby food.

Hugo Alves: Exactly. Exactly. So I wanted the product to provide to not be a yes man that is not helpful. That we know that we know that if and that’s the concern with people using Chat GPT for this is that Chat GPT is too much of a yes man to be useful in most cases.

Chris Parker: Outstanding. Hugo Alves is coming to us from from Portugal from Lisbon and you can find him at syntheticusers.com. I’ll put the link in the show notes. I’ll also put the LinkedIn there. So, if you are running research and if you’re triggered and inspired by this, I really urge you to reach out to Hugo on LinkedIn and run that test. If you happen to do it, then also let me know or this will be posted on LinkedIn. I’d love it if you go back there and and then let us know what happened with that experiment. I would be just excited to hear that. So, Hugo, thank you so much.

Hugo Alves: Thank you, Chris. It’s always a pleasure talking to you. someone who really values how to build great businesses, how to build effective businesses, and someone who’s curious about this new world of having artificial intelligence helping us helping us help other people. Curious every day. Thank you, Hugo. Cool.

Chris Parker: Thank you for listening to the Business Simplicity podcast. If this conversation resonated, please share it with a fellow leader. Navigating complexity. Visit Ebullient.com to discover how we can partner to simplify your strategy, align your teams, and accelerate meaningful growth.