Operators Ep 25 Transcript

Delian:

Hi, everyone. My name is Delian, and I'm a principal at Founders Fund, a venture capital firm based in San Francisco. This is Operators, where I interview non-VC, non-CEO, non-founder operators that make the startup world go round. Today, I'm interviewing Erik Bernhardsson, former CTO at better.com. Prior to joining Better, Erik spent seven years at Spotify as an engineering manager focused on their recommendation and discovery system. I hope you enjoy the show. Sweet. Erik, thanks so much for taking the time to come on to the podcast. Excited to chat today.

Erik Bernhardsson:

Sure, it's fun to be here.

Speaker 2:

Cool. Well, [inaudible 00:00:49] conversations, before we dive into your current work, I always like starting from the beginning and talking about where people started off, and even education, things like that. You studied back, both in high school, Ignazio, and in university, physics and mathematics. What drew you to that early on? Was it something you had a natural talent for, and was it something you pursued, specifically, with the intent of eventually getting involved in technology and computer engineering?

Erik Bernhardsson:

Yeah, I mean, sort of. I grew up coding, since I was eight, or something like that. So it's been almost 30 years now, and, I don't know, I was always doing it just for fun. I guess, it's funny, in a way, I always think about how lucky I am. My hobby turns out to be something you can make money from. All my friends were collecting ice hockey cards, or whatever, and they'd probably didn't make any money from that. But, I grew up coding a lot, and then, in a way, I actually got into math, because I was like, I wanted to make something spin when I was coding, then I had to learn sine and cosine, and kept going deeper and deeper into math, and then, eventually, I had to make the choice of going to high school, and I picked a special math program.

            And I think it was a coincidental, or a random choice, but it was a life changing way, in a thing. In a way, ended up at this high school with a bunch of smart kids from all over Sweden, who were just super smart, and interested in... A lot of them are coding, and a lot of them are super into math, and we... This culture of competition in a way; everyone's trying to be better than each other in solving math problems and stuff. So it was a lot of fun.

Speaker 2:

Definitely. I was at "mathlete" when I was in high school, so definitely, I like that world. But yes, it looks like you did obviously start to... Obviously, you could when you're a kid, but actually got your first couple of professional gigs, various internships, and then eventually landed at Spotify in Sweden when they were... Quite early on. Can you talk us through how those first couple internships landed, what you learned there, and then, eventually, how you made it over to Spotify in the early years?

Erik Bernhardsson:

Yeah, totally. School in Sweden is lax. University in Sweden, it's like, you have exams every once in a while, and you have to go and take them but other than that, there's no really strict requirement on actually being present at the university. So, I started exploiting that to some extent, and throughout my time at university, I had a lot of different internships. And I'm so happy I did that. I tried a lot of different things; I worked at Google, a stint, I did some high frequency trading in Sweden, I took some time off and worked with this telco, and did a lot of different stuff. And then, my school ended up taking a little bit longer than the plan, but it was fine. I think when I graduated school, I had two or three years of full-time experience. There's few times in your life when you have that opportunity to sample things and figure out what you want to do, and one thing I did a lot in school was, also, I did a lot of programming competitions.

            There was this group in my school, most of them were a little bit older than me, and we went to all these programming competition. I participated a lot too. Me and my team, we won the Swedish championship, I think, five or six times, or something like that. And then when I graduated, I came back from my internship at a hedge fund in New York, and I was back in Sweden, and some of these older students that I used to go to programming competition, they were working on this thing called Spotify. I was interested, like, "What's this thing? It seems really cool." I didn't even care what it is, I want to work with these people. Clearly, they're very smart. I think, maybe that was my experience from my high school, too, it's like, just being around smart people is generally what I like to do. I feel like life's too short not to work with smart people. And so, I finagled my way into Spotify. I didn't have any experience building music, machine learning or anything like that, but I convinced them, "Why don't I come work with you and build a prototype music recommendation system?"

            I mean, back then, no one was doing machine learning, so I was as good as anyone else, in a way. No one had experienced. This was 2008. So they hired me... Or not hired, they let me basically write a master's thesis, and then they hired me full-time early in 2009 to build the music recommendation system, which is a lot of fun.

Speaker 2:

And you went straight from hire to building up a team, initial, prototype, et cetera, et cetera, where pretty quick period of time. I guess, can you give us a little bit of understanding, how large was Spotify at the time when you joined in terms of team, and then, what was that initial project, how did you prove value to the company, build out that team, and showed that this was some things worth investing into?

Erik Bernhardsson:

Yeah, in a way, I failed at that. I think, Spotify was maybe 50 people when I joined. And so, I joined early 2009 to do this thing, and pretty quickly, we of... I wasn't managing initially, and then we pretty quickly realized... Actually, Spotify, it's like a lot more foundation from the salt. And I think about it a lot, a lot of companies, machine learning is maybe not the first thing you should do. You should... When Spotify, early days, the number one goal is to make sure you click play, music plays. And there was so many just core things like that, that had to be solved first, and one of them at that time was just reporting and analytics. There was nothing existing. I actually, reluctantly switched focus to that, and initially, I was a little bit disgruntled about it, because I joined thinking I was going to do all these cool machine learning and stuff, now, I'm just doing this boring reporting. I think, I would say in a way, also a lucky thing for me, in hindsight, because I think... I don't know.

            I grew up thinking I was this algorithm dude, and I would only to solve hard math problem, but then I ended up working with all these MBA type people, and I started developing this appreciation for how a company is built, because they would come to me with all these questions, like, what is our user retention look like? And I was like, first of all, I pull a query, and show them, and then that didn't answer their question, and then I have to pull... And then, eventually, I would try to probe deeper. I was like, "What are you trying to show here? What's our investor asking for? What are you trying to prove?" And over a year or so, I ended up just getting deeper and deeper, really understanding how investors think about Spotify? How do we make this work? And despite that engineering work maybe being less interested, I thought commercial side was super fascinating, and it was this side I never thought about myself, because I never grew up thinking of myself as particularly commercial interested.

            And I built up a small team at Spotify at that time, and did this for a couple years. And then, in 2011, decided, for whatever reason, maybe a mistake in hindsight, that I wanted to do something new, and so I moved to New York and started at a hedge fund doing high frequency trading again.

Speaker 2:

And you ended up coming back to Spotify, and I'm curious, just, in that first stint Spotify that you were there, was it already at the point where Spotify was monetizing? Were you actually starting to dig into just the standard like whatever, CAC, retention, payback periods, things like this, or is this still in... I forget, I think Spotify was free only for quite some time if I remember the early days. Maybe I'm wrong about that, but I'm curious, was that also part of what [inaudible 00:08:34], was actually just seeing the money flowing and being able to analyze that, not just the users, but the actual monetary levers?

Erik Bernhardsson:

Yeah, totally. And I don't even remember if people used terms like CAC and LTV and things like that. I don't remember using them, but that's what we're doing. A lot of what I've built up, instead of... There's no literature on this, but I ended up building all the cohort models, and looking at premium promotion over time, and one of the sort of breakthroughs that we made was just plotting that over time and showing that, you have this free product that's big, we were just losing money from it, barely breaking even on the ads. However, if you look at the conversion rate over time of those customers, it might take a year, it might take two years, but it keeps just going up. It just keeps going up. And so, what that means is, a company like Spotify has, in a way, a high CAC in the form of... You can almost think of the free product as CAC, but a quite high LTV, just a long payback period. And so, looking at cohort models, I feel like cohort models has always been my secret weapon in all analytics I've done and throughout my life. And that's something I started back at Spotify 2009, but you're right, that was just around the time we started monetizing it. I think the premium product rolled out end of 2008 or something like that [crosstalk 00:09:58].

Speaker 2:

It's funny, I had a somewhat similar realization. I used to be a software engineer, and then in the summer of 2012, I was an intern at Square, and I saw somebody present in front of the whole company a cohort analysis, and they were churning my engineering friends. I mean, that's statistical black magic. I didn't even realize it, that's how people thought about businesses, and that was when I had maybe somewhat similar realization of... I was like, "Oh, you can combine math and statistics, plus business, and then you can use that to operate businesses better than anybody else in the world." I was like, "That's what I want to go and learn about."

Erik Bernhardsson:

Yeah, and it's so fascinating, and I feel like it's still this black magic thing. Some stuff that I was working on at Better, and we can get back to that later is, not just plotting the cohorts, but also, you can actually model it. You can look at cohorts and... That turns out a lot of cohorts have quite predictable behavior. They actually, at Better, turns out, they all fit this generalized gamma distribution, which only has three parameters, so you can fit those parameters, and then you can extrapolate it out into infinity, and put an LTV on that cohort very early. And if you do that well, now you can suddenly compute ROI of ad campaigns super early. You don't have to wait for three months to see how the final commercials looks like. You can do it quite early. And so, I think there's this super interesting... I don't know. There's a whole rabbit hole I could talk about hours, but I love what you're saying, this idea of business plus statistics. I think if you know both, you can be very dangerous.

Speaker 2:

And so, you end up making the mistake, I guess, of leaving Spotify briefly to go join a hedge fund. I guess it was, partially, because you're interested in applying more of your mathematics skills, and a quantitative hedge fund [inaudible 00:11:35] the right way, and so, is that why you decided, and then, what got you immediately, quite quickly, back to Spotify?

Erik Bernhardsson:

Yeah, that was it. I mean, as I mentioned, I didn't really work that much on the music recommendation system the first couple of years at Spotify. And I think I did miss that, to some extent. I also had an internship at [die-shai 00:11:56] in New York in 2007, throughout in my school, and I loved New York. I think Sweden's a great country in many ways, but I think I just... I'm a person who likes big cities, and all this stuff happening. So, I missed New York in a certain way. So, I decided, this is a way to work on hard math problems and live in New York. And so moved to New York in 2011, and did the reverse commute up to Connecticut for five months, realized I hated it. I mean, it was a good, still, personal decision to do that, because I ended up in New York, and I just passed my 10 years here. So, I guess I'm a New Yorker now, but I am very happy about the fact that I came here. I realized in the... I love New York, but didn't like my job. And so, when the opportunity came up to come back to Spotify, I think I jumped on it immediately.

            I guess another thing I realized in hindsight is also, your first job out of school, you don't necessarily have anything to compare against. I don't think I fully appreciated when I worked at Spotify in the early days, what a unique place that was, because for me, that was just... And I was very lucky in hindsight, it was just some random company I joined straight out of school. And the fact that this was about to become biggest startup out of Sweden in a whole generation, I guess I didn't really appreciate it at that time, but I realized that when I left Spotify, how much I missed it. And so, I went back and was very happy to come back, but in New York.

Speaker 2:

It's sometimes impossible to translate. I'm going through that same thing. My little brother was the 25th employee at Ramp, and he's going through particularly special experience there, but it's impossible for me to describe to him how anomalous his experience is in the world of startups, because he doesn't know anything else. He's like, "Oh, every startup grows this quickly, and has such a high quality team, and fundraises every six months and has massive [crosstalk 00:13:54] and is amazing."

Erik Bernhardsson:

Totally. I mean, it's probably a good thing for your career sometimes to take a shitty job too, at a stagnant company, almost. Go work for some insurance company for six months just to see what it's like.

Speaker 2:

You learn to just avoid that as much as possible for the rest of your career, because it's like, even six months of that could be more draining and tiring than four years of high growth startup, because it turns out banging your head against the wall when things aren't working isn't particularly interesting.

Erik Bernhardsson:

Exactly. Yeah. But yeah, I think Spotify was... It was an amazing place. And when I came back at Spotify, there was a lot of focus on machine learning and music recommendation. There were a lot of desire to go deeper. So, when I came back, I actually was able to pick up machine learning and really start to go deep on building this music recommendation system, which I always kept hacking on it on the side, even before when I was at Spotify, even though I wasn't really supposed to, but I spent a lot of my spare time on it. So, I mean, eventually, we built this music recommendation system. Pretty much, 80% of my code is still powering the system from what I heard at people still at Spotify. So, that was a fun thing, having built something, and then it took two, three years to do it, because a lot of technology back then didn't really exist. But, I like the idea of building something that makes millions of people happy. So, that's my... In a way, something I'm always going to look back at.

Speaker 2:

I was going to say, I assume that, eventually that work led to this. Now that you left a little bit before it finally launched, but I assume all this work led into eventually what people love today, to discover weekly playlist. I assume the ML and recommendation work that you worked on fed into that.

Erik Bernhardsson:

Yeah, absolutely. It's all a bunch of algorithms. I mean, there's many other people. I don't want to discredit things that... This is six years since I left, so there's been many, many improvements, I'm sure, since I left, but a lot of the foundation, I think, from what I hear, is my algorithms. I mean, Spotify was a crazy place. I think, going back to what I said, how unique it was, it was very chaotic in a very, very good way. When I joined, no one told me who was my manager. I didn't know that for a year. That was maybe bad chaos, but I think the upside was, there's just a bunch of smart people who just wanted to get shit done, and build a startup. And I think in hindsight, it ended up working out reasonably well. So, I think I'm always going to have a special place in my heart for those first few years at Spotify, what a unique environment it was.

Speaker 2:

If you had some lessons to take away from both your stints at Spotify that would describe either what you learned about core engineering and algorithms, or management of engineering teams shipping product, or even just company building, if you had to distill that into a couple of lessons learned that were imparted upon you in your future career moves, what do you feel like those would be?

Erik Bernhardsson:

I think that the quality of the early tech team is just 90% of a company's success, in my opinion. And I think Spotify nailed it. The early tech team was just phenomenal, and if you get that right, you create this snowball effect, because other people just want to join, because there's other smart people there. And I think you also have this positive culture where smart people... They like to bicker a little bit. They're always challenging each other, like, "I bet I can do this better. I bet I can do this faster. We don't need as many machines as you think, because I have this trick up my sleeve.' I think that just created this culture where people worked really hard, and kept coming up with creative solutions. So, to me, that's probably the best takeaway from Spotify, is just how much the early tech team matters. I mean, not just the tech team, but those people that are working.

Speaker 2:

And so, from there, you ended up joining Better as one of the first early employees, and I assume they... Maybe they didn't do it immediately, but eventually gave you this CTO offer. So, how did that come about? Why did you decide to join them? It's a, obviously, wildly different company than what Spotify does. [crosstalk 00:18:22].

Erik Bernhardsson:

I think everyone thought I was crazy for doing this. I mean, in 2008, when I took the job, or 2009, when I took the job at Spotify, I actually had an offer from Google, and Spotify back then was this obscure music startup. And, all my friends and family were like, "You're really dumb. You're going to this startup that has no revenue, and they're in this space where you can ever make money." And, "Why are you doing this? Don't you want to go work at Google?" Because Google is such a cool company. But I don't know, I just want to work with smart people. And so, I think I... My takeaway from that was, I think, in a way, sometimes not going for the most prestigious thing can be... I think some people... It's easy to fall into a trap of just picking the most prestigious thing, and I'm very happy I didn't do that with Spotify. And I think, again, when the opportunity with Better came up, it was in the mortgage industry. I was like, "Why the fuck do you want to go into mortgage industry?" They're like, "Isn't that the industry that took down the economy in 2008? Why do you want to go work there?" And, I was like, "You know what? I think it's such a..."

            I thought it was an amazing opportunity, because I spent a lot of time with Vishal, the CEO, and was so struck with, first of all, his intelligence, but also views on the world and the market and all the things. And I think at that time, I also thought a lot about, in a way, Internet is having changed things like entertainment and communication and media, and the case with Spotify with publishing and all these things, but what about the rest of the world? What about real estate, or banking, or healthcare, and education? And I still think that's a thesis that makes sense. And so, I think there was a lot of things that clicked with Better. It checked the box of something I believed in that made sense, but also, I also really got along with Vishal, the CEO. We both have this... I think, coming out of Spotify, seeing how a group of smart people can change the world in a way, I think you get this confidence and almost megalomania that like, "This industry is broken. Let's fix it."

            And that really resonated, both Vishal and I got along really well in how he looked at it. So, I don't know. I think it worked out reasonably well. It's not a big company, and I'm very happy I ended up joining Better, even though everyone thought I was stupid for doing it.

Speaker 2:

And maybe before we dive into Better a little bit more, I realized I had one thing that I wanted to ask about the Spotify days, which is, I think, in some ways, both of you were extremely early to the world of doing recommendation algorithms, but then also, I think music in particular, essentially, difficult recommendation problem, because there's not as... I don't know, I mean, maybe you're just looking at what their friends or people who listen to similar songs, that you're really looking at, so you're not necessarily having to analyze the actual music that much, but I'd love to understand, what were some of the most impactful, either projects, or features that you would pull from the music, or ideas you had around recommendations that ended up changing the actual fundamental, let's say, algorithms, and how do you actually recommend music, or how to present it to users, or what to index off of to use for recommendations, over there, three or four years of working on it?

Erik Bernhardsson:

Yeah, totally. I mean, just to put a little bit of context, back then when I started working on it, machine learning... I mean, there was people doing machine learning, but I don't think it had this same hype around it than compared to now, nowhere near. And so, it was much harder to find actual implementation quality stuff. I had to read a lot of papers and trying to implement stuff myself. I think that main thing that existed that was extremely helpful back then was the Netflix prize. Netflix had won million dollar contest at some point to predict movie ratings. And so, out of that research, there was a particular one thing that I thought was really helpful, which was, collaborative filtering. The idea that, actually, you don't need to understand anything about the movies, or you don't need to understand anything about what anything is doing, you just need to look at patterns in how people rate similar movies. Instead of, look at, "Oh, these two movies seem correlated statistically." So, if someone likes movie A, then they're probably more likely to like movie B, and vice versa.

            And so, same idea with music. We never really looked at the content of audio, we started doing that much later using deep learning on the audio, maybe 2014, but in the early days, and still, I think, 95% of what powers the music recommendation system at Spotify is really just finding statistical patterns in consumption behavior. We don't have ratings on Spotify. We just look at what people consume. And it's, intuitively, if track P and track Q co-occur in a lot of playlists, they're probably similar. So that's the idea. And I actually think music recommendation is also a simpler problem than other types of recommendation, because music is a simpler object. When you like a movie, there's a lot of reasons you can like the movie; maybe you like the photography, or that's it's... Shows dark side about humanity, or you like that it's like... Takes place in Baltimore, whatever, I don't know. And, with music, I think it's a little bit simpler. It's the genre, and the style, and... The objects are more low dimensional, in a certain sense. We used a lot of dimensionality reduction to build, basically, the vector representation of music. And I think it intuitively feels like it should work better for music than maybe other more complex objects. So, I think those are some of the... I mean, there's a lot of devil in the detail, like how do you actually implement this?

            And then, back then, it was hard. We have to use Hadoop and other things, because it was hard to scale it, but I think a lot of it came from collaborative filtering and latent factor models, and not really caring about the music itself.

Speaker 2:

You're probably one of the few people in the world that I can ask this question of, do you think that there have been any downsides in the fact that everything from Netflix, to YouTube, to Spotify, have, again, maybe more recently started to do vector representations of the content, but, for a long time, we're primarily reliant on collaborative filtering, which, by default, makes it so that if you are a less listened to artist or creator, in some ways, is inherently more difficult to actually build up a set of data around who would be likely to like your content, given that you can't necessarily do collaborative filtering if not for having somebody? Basically, the most listened to get listened to more versus the unlistened to, don't get nearly as much, versus...

            I feel like TikTok is one of the first times where you see somebody... I'm not sure what the underlying algorithms are, but you can sense it in just like their "for you" page works, in that, it does not feel like it is truly just collaborative filtering, it feels like something that is more fundamentally different, or at the very least, introduces more spontaneity or randomness into it, in that, they artificially force low viewed content forward in order to start to build up that data set around collaborative filtering. Do you think that's changed consumer behaviors or how music gets discovered, in a way that is-

Erik Bernhardsson:

Maybe. I mean, I would argue that it's like, if anything changes, to the better. Look, what you're talking about is the filter bubble to some extent, or... I remember there was this paper many years ago, I used to know a lot more about these things, I'm sure there's a lot of research in the last 10 years that I missed, but where they built this music website, and they randomly split the users into different cohorts. And then, what they ended up seeing was, some tracks would end up on the top list on one of the courts, but not the other cohorts, and then, you get the Matthew effect, this self reinforcing feedback loop, where that thing got more and more prominence. And so, that's even without recommendations, and when I look back at the world in their 70s, or 80s, I don't know, I don't think music really spoke for itself. I think that music that ended up being more and more popular in the '90s and 2000s, too, was that driven by the quality of the music in itself? I don't know. I think, it could be small deviations in their initial listening, but then amplified through people analyzing charts, or maybe music record bosses wanting to invest in one type of thing or not the other thing, because they don't like a certain artist or whatever.

            I think, in a way, music recommendations, or any recommendations, it actually lets that music speak for itself. And by the way, when you're implementing the systems, you try to avoid these feedback loops, because they also generate bad behavior in the system. So, it's quite important when you train on data, in a way, you want to train on data that's not output from the music recommendation system in itself. And there's a lot of work that goes into, if someone listens to a track from the music recommendation system, and then they put it in a playlist, you also need to filter that up. I actually think recommendation system probably do a better job at flattening the long tail, and leading to more obscure content gaining prominence, because it's good. People discover it's good. I don't know, maybe I'm rationalizing it, who knows?

Speaker 2:

I guess I'm curious, as you guys started to implement deep learning and vector representations of the music, were there any major shifts in behaviors, or statistically significant events that changed which type of music blew up or got discovered, because of the difference in either analyzing the underlying object as opposed to just analyzing the user's behaviors?

Erik Bernhardsson:

I mean, analyzing the audio, that was... I think it works as a supplement to the statistical analysis, the collaborative filtering stuff, at best. And I think the other thing that is helpful for is cold start problem. Let's say a new album comes into Spotify, we don't have any listening data for it, so we need to rely on something else, and for that, it's good to have that as a fallback mechanism to analyze the audio data in itself, but for everything else, it's much, much better to actually just look at listening behavior. And you don't need a lot of data. Already with a few 100 plays, you can see what types of users are listening to this audio, and already draw a lot of conclusions from that. So, generally collaborative filtering wins out with very little data, already, and Spotify, now, has a tremendous amount of data. So, generally, I think the audio, it's a supplement, but the core power of the recommendations, I think, for almost any person who implements this in practice is always going to be collaborative filtering type algorithms.

Speaker 2:

And in relation to all this, I'd love to now talk a little bit about Better. Can you talk about some of the more interesting, let's say, machine learning or data science problems you started to tackle there, and then how that related to whether it's underwriting, repayment periods, understanding the LTV of these cohorts? What were some of the more, let's say, interesting or counterintuitive signals you guys dug into that ended up being really significant for the business?

Erik Bernhardsson:

I mean, I think there's a little bit of a... I think it's stereotypical, FinTech company where you develop this black box, default risk prediction model, and then you try to skim the cream off the top in terms of customers or whatever. With Better, it's actually quite different. Better, we sell our loans on the secondary market, and instead of Fannie implicitly, basically drives all the underwriting rules in America directly or indirectly. So, Better, I mean, it's actually not as machine learning driven as you would think. I think also, in a way, as I mentioned, I started out being this algo guy, I love math, whatever, that's all I want to do, I think, throughout my career, I realized, no, I actually wanted to build companies, and machine learning... I get excited when I get to use machine learning, but I also think, at the end of the day, it's just a tool out of many things in the toolbox, and I think... First of all, I think, one of the things I learned about doing a lot of machine learning is, there's a lot of stuff that machine learning is not good for. I joke about that, and it's like, that's a benefit for me, is, I'm not as excited about using machine learning as maybe other people, because I know what it's good for, and I know what it's not good for.

            And I also think, in a way, a lot of companies, it's just so much basic stuff you need to do first. And so, Better, I mean, we had a lot of hard technical problems, and a lot of difficult things to build, but I wouldn't say machine learning is a core to what we do, and I think, I actually find that, for most companies, that's probably true, actually. You get to a certain stage, then machine learning becomes great to squeeze out an additional 10%, but you still need to build the first 100% of it.

Speaker 2:

I guess speaking of, let's say, just building the company, rather than just trying to do machine learning, I'd love to understand, before this, I think, at Spotify, your largest team size got to maybe 20, 25, versus Better, the company's over, it sounds like 3000 employees now, then the team's over 130 people. I imagine that level of rapid scaling, I think Spotify took a little bit longer to grow into the scale that it's gotten to today, versus, I think, the slope at Better has been a lot steeper. I'm curious, how is that scaling process different, what lessons did you take from the actual, let's say, company building from Spotify that translated into Better and made it, let's say, easier to ride that steep slope?

Erik Bernhardsson:

I mean, it's even bigger at Better. I think there's 300 engineers now, and the whole company is 6000. I mean, there's been some crazy hiring, and I think, as I mentioned, my takeaway from Spotify was how much the quality of the early team matters. And then, once you go a little bit deeper on it, I've developed this obsession with hiring people, which is something actually, in a way, I never thought of myself. I'm not a people person, in a way. I like to solve technical problems, that sort of... If anything I do in my spare time, it's more of that, but I also like to build companies, and I think when you... As a CTO, what is your number one job? It's, you're not going to do most of the work. It's going to be the people you hire. And so, sometimes I meet CTOs, and they're like, "Oh, it's so hard to hire people. How do you do it?" And I'm like, "How much time do you spend hiring?" And they're like, "10% of my time." It's like, "No." Of course, it's going to be hard. I mean, I don't know, I spent, I don't know, 70% of my time in the first year hiring.

            I was doing 40 interviews a week at peak. I was doing so much interviewing I got an ear infection, because I was talking on the phone all the time. The week before my wedding, I was on painkillers throughout my whole weddings. So, I had a horrible... Anyway... But, my point is, I think going back to first principles of, what is your job as the CTO? I think especially in the first year or two, it's just hiring. It's just... And even to hire crazy quantities in the early days, it's just hard. You just need to meet with so many people, because the conversion rate is going to suck, and especially maybe because we were doing something in the mortgage space. People are just not that excited about it. So, at a first order approximation, I think hiring is just a numbers game, you just got to go out and talk to so many people. And, initially, conversion rate is going to be bad, and so you're going to have to figure out how to improve that conversion rate, you're going to have to be better at sourcing, you have to be better at selling and all these things, and just do it all the time. So that was my...

            I think recruiting is a little bit like learning Spanish. It's not hard, everyone can learn Spanish, but don't underestimate how much time it takes. It really takes a lot of time to learn a language and be fluent in it. But everyone can do it, clearly. Kids speak Spanish. So, I feel like sometimes people, hard means two different things in English, it means it's hard in terms of quantum mechanics, only certain people can do it, or it's hard in the sense of learning Spanish. And recruiting is definitely the latter. You just need to do it a lot. So that was something I think I actually ended up enjoying quite a lot, because you start to... My pattern matching part of my brain started picking up, what are people that are good? What do they say? What do they look like on their resume? How do I find more of those? And you make it a game, and then it's fun.

Speaker 2:

In terms of, not only scaling the company, but then pattern matching to people that can help you scale, or teach people Spanish, let's say, as you call it, what do you feel Better did well in terms of, not only, obviously, hiring the engineers, but then hiring engineers that could hire more engineers, and not only basically scale the company and scale the system, but find the people that can help the system scale itself?

Erik Bernhardsson:

I think my... This might sound naive, and somewhat aspirational, but my goal for the company I want to create is always, I want a company where people just come in every morning and just ask themselves, "What can I do that increases the value of the business?" And then they just self organize and do that. And then at that point, by the way, I'm no longer needed, because people just do things without a manager. So, clearly you're never going to get there fully, but I think it's worth asking yourself, how can you get to it as close to that as possible? How can you find people where, as a manager, you don't have to spend too much time supervising them, because you trust them, because they're good, and you know that they're going to go after the right problems? So, a lot of it, I think, it has to boil down to, how do you set the right context? I mean, Netflix always talks about context versus control, telling people what's the most important thing for the business that they can figure out how that effects their day to day, but also hiring people who are quite autonomous, or quite entrepreneurial, I think is very important for a startup in the early days.

            People sort of... They're curious. They're like, "I think this company needs this thing. I'm going to figure out, today, as I implement this feature, I'm going to make this decision, because it builds future optionality and to expanding to this other thing, versus doing this other thing, which doesn't." And I also think, probably early on as a startup, you want to hire people who are generalists, because that also helps. I think there's an axis of goal oriented people versus tools oriented people. I used to be a tools oriented person. I grew up thinking I just want to work on hard math problems, or whatever. And all those, I actually don't give a shit. I just want to solve problems. I just want to... "Here's the business problems, let's get together and figure out what to do." And if it entails me having to hire people, or spend a whole day going through customer complaints in a spreadsheet and classify them, that's fine. So you want those people who are more goal oriented, people who are excited about a company's mission.

            If someone is like, "All I want to do is work on functional programming" That's fine for some types of jobs, but I think, if I hire that person, I know that their career objectives might be at odds with what I want them to do today, and that might be like a bad conflict of interest later, and so, I'm quite hesitant to hire those people for a startup. Those more tools oriented people. It goes for any tool, deep learning, or whatever it is. And maybe you need those people later as you get bigger, you can afford a specialist a bit more, but early on, you want autonomous people, entrepreneurial people, generalists, people who can take a business problem and work on autonomously for three months without too much supervision.

Speaker 2:

And then, beyond just the actual... I think it's a really great pattern matching for the people that you're looking for, but obviously, there's a lot to scaling a company that isn't just the people, but it's also the systems and structures around it. And it sounds like you guys have done a really solid job of that at Better, everything from the career ladder, to the salary reviews, to onboarding people effectively. Can you talk to a little bit about just... What were some of the most... Was a lot of that just following a pre-existing playbook that you found from other companies that scaled like Spotify? Was that reinventing the wheel for what worked for Better? What were some of the most effective strategies and systems that you put in place that allowed the company to continue to scale as it got larger?

Erik Bernhardsson:

Yeah. I mean, spending seven years at Spotify, I learned a lot about what worked and what didn't work. And I think Spotify was a good starting point in that. I mean, Spotify is quite successful, and so, I think, a lot of the things that Spotify did was very good. I think it's hard to dispute that, but there were also things that Spotify did that I didn't like, and I think were bad, like the complicated matrix model. People talk about, Spotify... That's just one example. Spotify's matrix model is this famous thing that Spotify, by the way, doesn't even use, but they used it for a while when I was there, and I think it led to poor accountability and poor supervision. So, that was an example of something that I took away that I really don't want to do. And another example was, Spotify had a very decentralized recruiting model. And I think the problem with that was, you have a lot of junior hiring managers with really aggressive headcount goals, and when you create those incentive structures, of course, they're going to lower their bar for who they hire.

            And I think, it's no one's fault, but that leads to poor oversight in who you bring in. I think that's another example where I'm a big fan of centralizing, not maybe the all of it, like interviewing and hiring, but centralizing a lot of the criteria and what the bar looks like for hiring. But with Spotify, it's so many things. They hired great people, they gave people a lot of autonomy, and trusted people to do the right things, give people context, instead of control. There's so many positive things that Spotify did that I definitely emulated at Better.

Speaker 2:

And then, it sounds like also, you've always been interested in the, let's say, "business" side of it, and then, at Better, you've been actually pretty involved in the fundraising process. Can you talk through why those are both intriguing to you, how you got involved in... I love the quote from [inaudible 00:41:56] which is, "Keeping the board happy." I'm a big fan of, if you keep your internal investors happy, you do a really good job with clean and effective board meetings that can actually just make your life as a company a lot easier, because future fundraisers, the strategy, et cetera, keeps everybody in line, makes life a lot easier, I think, as a founder, when you have a very, let's say, productive board, and happy board, and effective board.

Erik Bernhardsson:

Yeah, I mean, I think, you end up at the board, or at the investors if you just keep asking why. With any company question, "This week we're going to focus a lot on growth, not on customer experience," or, "Next week, it's going to be the other way around." If you start asking why, you often end up at that level. I mean, at its core, it's like, there is demand out there in the world for return on capital with a certain risk profile and a certain time period. That's the ultimate answer to a lot of these things. And I think, being a curious person, I kept asking why, and I think... I love being a part of Better and seeing how that sausage is made, and developing and better understanding for, what do different investors look like? What do growth investors look like, what do they look for? What do early stage investors, what do late stage investor, what do West Coast and East Coast investors look for? They will look for certainly different things. And I think it's really helpful to know that, because, then you also know, in a way, what to manage for.

            I mean, obviously, you always have to have your own point of view of how to build a successful company, but I think it's important to find investors who also share that point of view, because otherwise, you might have a conflict of opinion later when some investor wants you to go super aggressive, and increase burn, and other investor wants to be more... Whatever it is. So, I think... I don't know. I find that, for any engineer, if you just keep being curious and keep asking for the question behind the question, on some level, you end up trying to under second guess what investors want.

Speaker 2:

I assume your answer to this question is going to be somewhat related to the pattern matching that you're doing for trying to find the ideal engineers at Better, but I'd be curious if you were to look back at, or if you're advising, let's say, a fresh German mathlete/IOI competitor that wants to get into the world of startups and understand business, et cetera, what would be your ground level advice, or just, how to think about their career, what opportunities to prioritize, where to go, but maybe, caveated for the world of 2020, as opposed to the world of 2007, I would say?

Erik Bernhardsson:

I mean, I think there's just so much, and I think, in the hindsight, I think I was quite lucky to have joined Spotify, I think it's so... I also want to discount my own advice, it's hard to tell everyone go join [inaudible 00:44:53] school, that random company that turns out to be the biggest unicorn in your country in that generation. I was very lucky. But, a couple of things I recommend everyone to do is, I did a lot of internship, I mentioned that, and I think it was so helpful to learn what different types of companies look like. I think everyone benefits from a little bit having generalist skills. Let's say you're super interested in machine learning. It's probably good to also know how to spin up an AWS instance, and build a web server, and set up a database, and build a website. Those skills will unblock you from working with... It will make you finish your tasks a lot easier with more autonomy. So, I think that's super important. I think also figuring out, are you a goals oriented people, versus are you a tools oriented person? Do you want to go super deep? Do you want to be the world's best expert at whatever, super obscure thing? Then maybe you should go work at Google.

            And that's fine. A lot of people are super happy there, going super deep and doing things. Or you want to go to the other side of the spectrum, you're super curious about how the world fits together. You should probably go work at a startup. I know, there's a lot of those things. I think another thing I touched on is, don't optimize for what your friends and family tell you to do. I'm so happy I went to Spotify and not Google, and I'm so happy... I mean, Google is a great company. I feel like I'm throwing them under the bus, but I really not. And I'm so happy I went to Better. I don't know. I think in general, just doing what you think is right, not caring too much about what other people think, I generally found that to be the best choices in my career.

Speaker 2:

Awesome. I think it's a great final piece of advice and note to wrap on. Erik, really appreciate you taking the time to hop on to the podcast today, it was a really enjoyable conversation.

Erik Bernhardsson:

Yeah, of course. It was super fun to discord or ramble a little bit. I hope people enjoy all these various collection of thoughts.

Delian:

Thanks for listening, everyone. If you'd like to support the podcast, please sign up for a paid Substack subscription, which we use to pay for transcripts, mics, and other improvements. If you have any comments or feedback on what kinds of questions I should ask, who should come on the show, or anything else, please do let me know. Have a great rest of your day.