The State of AI Startups in 2024 [LS Live @ NeurIPS]
00:00:00.000 | (upbeat music)
00:00:02.580 | - Okay, I think we're gonna kick this off.
00:00:08.900 | Thanks to everyone who made it early morning.
00:00:11.880 | It's like really weird experiments that we wanted to try
00:00:14.520 | because one, we saw this space,
00:00:16.920 | but two also, I've been to a number of these things now
00:00:19.640 | and I always felt like there was not enough
00:00:21.920 | like industry content for people
00:00:23.880 | and we wanted an opportunity while everyone is in town
00:00:27.000 | in like one central spot to get everyone together
00:00:29.920 | to talk about the best stuff of the year, review the year.
00:00:33.000 | It's very nice that New York is always the end of the year.
00:00:36.000 | And so I'm very honored that Sarah and Pranav
00:00:39.200 | have agreed to help us kick this off.
00:00:42.320 | Sarah, I've known for, I was actually counting, 17 years.
00:00:45.380 | - Sounds very much.
00:00:48.780 | (laughing)
00:00:51.520 | - But she's been enormously successful as an AI investor.
00:00:55.500 | Even when you're doing your Greylock days,
00:00:57.600 | I was tracking your investing
00:00:59.320 | and it's come a long way since then.
00:01:01.540 | And Pranav, I've known shorter,
00:01:05.260 | but he's also starting to write really incredible posts
00:01:07.960 | and opinions about what he's seeing as an investor.
00:01:10.500 | So I wanted to kick this off with an industry session.
00:01:13.360 | We have a great day of sort of like best of year recaps
00:01:17.520 | lined up.
00:01:18.360 | I think Vic is here as well and the RoboFlow guys.
00:01:22.840 | So I would just let you kick it off.
00:01:25.480 | Thank you.
00:01:27.320 | (clicking)
00:01:30.200 | - Hi everyone.
00:01:31.120 | My name is Sarah Guo and thanks to Sean and friends here
00:01:34.480 | for having me and Pranav.
00:01:36.520 | So I'd start by just giving 30 seconds of intro.
00:01:40.440 | I promise this isn't an ad.
00:01:42.120 | We started a venture fund called Conviction
00:01:44.440 | about two years ago.
00:01:45.520 | Here is a set of the investments we've made.
00:01:48.920 | They range from companies at the infrastructure level
00:01:53.320 | in terms of feeding the revolution
00:01:55.180 | to a foundation model companies, alternative architectures,
00:01:58.560 | domain specific training efforts,
00:02:00.360 | and of course applications.
00:02:02.840 | And the premise of the fund,
00:02:04.760 | Sean mentioned I worked at Greylock
00:02:06.360 | for about a decade before that
00:02:07.680 | and came from the product engineering side
00:02:10.360 | was that we thought that there was a really interesting
00:02:13.480 | technical revolution happening,
00:02:15.920 | that it would probably be the biggest change
00:02:17.800 | in how people use technology in our lifetimes.
00:02:20.240 | And that represented huge economic opportunity
00:02:22.720 | and maybe that there'd be an advantage
00:02:24.880 | versus the incumbent venture firms
00:02:27.400 | in that when the floor is lava,
00:02:29.680 | the dynamics of the markets change,
00:02:31.440 | the types of products and founders that you back change,
00:02:35.280 | it's a lot for existing firms to ingest
00:02:38.120 | and a lot of their mental models
00:02:39.320 | may not apply in the same way.
00:02:41.840 | And so there was an opportunity
00:02:43.040 | for first principles thinking,
00:02:44.320 | and if we were right, we'd do really well
00:02:45.960 | and get to work with amazing people.
00:02:47.400 | And so we are two years into that journey
00:02:49.360 | and we can share some of the opinions
00:02:51.000 | and predictions we have with all of you.
00:02:53.000 | Sorry, I'm just making sure that isn't actually
00:02:58.640 | blocking the whole presentation.
00:03:00.600 | And Pran's gonna start us off.
00:03:02.160 | - So quick agenda for today,
00:03:04.520 | we'll cover some of the model landscapes and themes
00:03:06.680 | that we've seen in 2024,
00:03:08.680 | what we think is happening in AI startups
00:03:10.160 | and then some of our latent priors
00:03:12.200 | on what we think is working in investing.
00:03:14.560 | So I thought it'd be useful to start from like,
00:03:17.960 | what was happening at NeurIPS last year in December, 2023.
00:03:21.760 | So in October, 2023,
00:03:23.640 | OpenAI had just launched the ability
00:03:24.920 | to upload images to ChatGPT,
00:03:26.440 | which means up until that moment,
00:03:27.920 | it's hard to believe, but like roughly a year ago,
00:03:29.440 | you could only input text and get text out of ChatGPT.
00:03:33.160 | The Mistral folks had just launched the Mixtral model
00:03:35.960 | right before the beginning of NeurIPS.
00:03:38.000 | Google had just announced Gemini.
00:03:39.600 | I very genuinely forgot about the existence of Bard
00:03:42.320 | before making these slides.
00:03:44.040 | And Europe had just announced
00:03:45.600 | that they were doing their first round of AI regulation,
00:03:47.960 | but not to be their last.
00:03:49.520 | And when we were thinking about like,
00:03:51.520 | what's changed in 2024,
00:03:53.160 | there's at least five themes that we could come up with
00:03:55.600 | that feel like they were descriptive
00:03:57.080 | of what 2024 has meant for AI and for startups.
00:04:01.040 | And so we'd start with,
00:04:02.680 | first, it's a much closer race on the foundation model side
00:04:04.840 | than it was in 2023.
00:04:06.080 | So this is Elm Arena,
00:04:08.200 | they're asking users to rate the evaluations
00:04:11.280 | of generations from specific prompts.
00:04:13.960 | So you get two responses from two language models,
00:04:16.240 | answer which one of them is better.
00:04:17.560 | The way to interpret this
00:04:18.400 | is like roughly 100 Elo difference
00:04:20.040 | means that you're preferred two thirds of the time.
00:04:22.400 | And a year ago, every OpenAI model
00:04:24.760 | was like more than 100 points better than anything else.
00:04:27.360 | And the view from the ground was roughly like,
00:04:29.440 | OpenAI is the IBM, there is no point in competing.
00:04:32.040 | Everyone should just give up, go work at OpenAI
00:04:34.040 | or attempt to use OpenAI models.
00:04:36.320 | And I think the story today is not that.
00:04:39.680 | I think it would have been unbelievable a year ago
00:04:41.960 | if you told people that, A, the best model today on this,
00:04:44.600 | at least on this eval is not OpenAI.
00:04:47.120 | And B, that it was Google
00:04:48.520 | would have been pretty unimaginable
00:04:49.920 | to the majority of researchers.
00:04:51.560 | But actually there are a variety
00:04:53.920 | of proprietary language model options
00:04:56.360 | and some set of open source options
00:04:57.720 | that are increasingly competitive.
00:04:59.400 | And this seems true, not just on the eval side,
00:05:01.400 | but also in actual spend.
00:05:03.080 | So this is RAMP data.
00:05:04.120 | There's a bunch of colors,
00:05:04.960 | but it's actually just OpenAI and Anthropic spend.
00:05:07.440 | And the OpenAI spend at the beginning,
00:05:09.560 | at the end of last year in November of '23
00:05:11.240 | was close to 90% of total volume.
00:05:13.720 | And today, less than a year later,
00:05:16.160 | it's closer to 60% of total volume,
00:05:18.520 | which I think is indicative
00:05:19.480 | both that language models are pretty easy APIs to switch out
00:05:22.480 | and people are trialing a variety of different options
00:05:24.840 | to figure out what works best for them.
00:05:27.080 | Related, second trend that we've noticed
00:05:29.520 | is that open source is increasingly competitive.
00:05:31.480 | So this is from the scale leaderboards,
00:05:34.440 | which is a set of independent evals
00:05:36.520 | that are not contaminated.
00:05:38.320 | And on a number of topics that actually
00:05:40.400 | the foundation models clearly care a great deal about.
00:05:43.320 | Open source models are pretty good
00:05:44.520 | on math instruction following and adversarial robustness.
00:05:47.760 | The Lama model is amongst the top three of evaluated models.
00:05:51.680 | I included the agentic tool use here,
00:05:53.080 | just to point out that this isn't true across the board.
00:05:55.360 | There are clearly some areas
00:05:56.680 | where foundation model companies have had more data
00:05:59.280 | or more expertise in training against these use cases,
00:06:01.600 | but models are surprisingly an increasing,
00:06:03.560 | open source models are surprisingly increasingly effective.
00:06:06.440 | This feels true across evals.
00:06:07.840 | This is the MMLU eval.
00:06:09.880 | I wanna call out two things here.
00:06:11.080 | One is that it's pretty remarkable
00:06:13.760 | that the ninth best model and two points
00:06:15.600 | behind the best state-in-the-art models
00:06:18.680 | is actually a 70 billion parameter model.
00:06:20.480 | I think this would have been surprising
00:06:22.600 | to a bunch of people who were,
00:06:23.640 | the belief was largely that most intelligence
00:06:25.960 | is just an emergent property,
00:06:27.160 | and there's a limit to how much intelligence
00:06:28.640 | you can push into smaller form factors.
00:06:31.080 | In fact, a year ago, the best small model
00:06:33.120 | or under 10 billion parameter model
00:06:34.880 | would have been Mistral-7b, which on this eval,
00:06:37.280 | if memory service is somewhere around a 60,
00:06:39.440 | and today that's the LLAMA-8b model,
00:06:41.640 | which is more than 10 points better.
00:06:43.200 | The gap between what is state-of-the-art
00:06:45.160 | and what you can fit into a fairly small form factor
00:06:49.120 | is actually shrinking.
00:06:50.760 | And again, related, we think the price of intelligence
00:06:54.720 | has come down substantially.
00:06:55.800 | This is a graph of flagship OpenAI model costs,
00:06:59.000 | where the cost of the API has come down roughly 80, 85%,
00:07:02.920 | and call it the last year, year and a half,
00:07:05.840 | which is pretty remarkable.
00:07:06.800 | This isn't just OpenAI 2.
00:07:08.200 | This is also the full set of models.
00:07:09.920 | This is from artificial analysis,
00:07:11.280 | which tracks cost per token across a variety
00:07:13.520 | of different APIs and public inference options.
00:07:15.720 | And we were doing some math on this.
00:07:17.720 | If you wanted to recreate the kind of data
00:07:20.960 | that a text editor had, or something like Notion or Coda,
00:07:24.000 | that's somewhere in the volume of a couple thousand dollars
00:07:26.680 | to create that volume of tokens,
00:07:28.360 | that's pretty remarkable and impressive.
00:07:30.280 | It's clearly not the same distribution of data,
00:07:32.560 | but just as a sense of scope,
00:07:35.200 | there's an enormous volume of data that you can create.
00:07:38.160 | And then fourth, we think new modalities
00:07:40.640 | are beginning to work.
00:07:41.920 | Start quickly with biology.
00:07:43.480 | We're lucky to work with the folks at Chai Discovery,
00:07:46.160 | who just released Chai 1, which is open source model
00:07:48.440 | that outperforms AlphaFold 3.
00:07:49.840 | It's impressive that this is like roughly a year of work
00:07:52.680 | with a pretty specific data set
00:07:54.440 | and then pretty specific technical beliefs.
00:07:56.120 | But models in domains like biology are beginning to work.
00:07:59.400 | We think that's true on the voice side as well.
00:08:01.760 | Point out that there were voice models
00:08:03.280 | before things like 11 Labs have existed for a while,
00:08:05.480 | but we think low latency voice is more than just a feature,
00:08:08.320 | it's actually a net new experience and interaction.
00:08:11.240 | Using voice mode feels very different
00:08:12.960 | than the historical transcription first models.
00:08:14.920 | Same thing with many of the Cartesian models.
00:08:17.880 | And then a new nascent use case is execution.
00:08:20.880 | So cloud-launched computer use,
00:08:22.360 | OpenAI launched code execution inside of Canvas yesterday.
00:08:25.120 | And then I think Devin just announced
00:08:26.760 | that you can all try it for $500 a month,
00:08:29.520 | which is pretty remarkable.
00:08:30.600 | It's a set of capabilities
00:08:31.920 | that have historically never been available
00:08:33.480 | to vast majority of population.
00:08:35.040 | And I think we're still in early innings.
00:08:36.600 | Cognition, the company was founded under a year ago.
00:08:39.000 | First product was roughly nine months ago,
00:08:40.960 | which was pretty impressive.
00:08:42.520 | - If you recall, like a year ago,
00:08:45.040 | the point of view on Sui Bench was like,
00:08:47.240 | it was impossible to surpass 15% or so.
00:08:50.800 | And I think the whole industry now considers that,
00:08:53.880 | if not trivial, accessible.
00:08:56.520 | - Yeah.
00:08:57.360 | Last new modality we wanted to call out,
00:09:00.440 | although there are many more, is video.
00:09:02.760 | I got early access to Sora
00:09:05.160 | and managed to sign up before they cut off accesses.
00:09:07.200 | So here's my favorite joke in the form of a video.
00:09:10.960 | Hopefully someone here can guess it.
00:09:12.720 | Yeah, you're telling me I shrimp fried this rice.
00:09:17.960 | It's a pretty bad joke, but I really like it.
00:09:20.960 | And I think this one, the next video here
00:09:24.840 | is one of our portfolio companies, Heygen,
00:09:27.240 | that translated and does the dubbing for,
00:09:31.480 | or lip sync and dubbing for live speeches.
00:09:34.040 | So this is Javier Millet, who speaks in Spanish,
00:09:36.800 | but here you will hear him in English if this plays.
00:09:40.800 | And you can see that you can capture
00:09:42.080 | the original tonality of his speech and performance.
00:09:45.360 | I think audio here doesn't work,
00:09:46.600 | but we'll push something publicly.
00:09:48.840 | - Let's give it a shot.
00:09:51.680 | - Yeah.
00:09:52.520 | Excellent.
00:09:56.400 | Yeah, and you can hear that this captures
00:09:58.080 | his original tone and the emotion in his speech,
00:10:02.000 | which is definitely new and pretty impressive
00:10:04.160 | from new models.
00:10:05.440 | So the last...
00:10:08.080 | Yeah, that makes sense.
00:10:12.240 | The last point that we wanted to call out
00:10:13.520 | is the much purported end of scaling.
00:10:15.520 | I think there's a great debate happening here later today
00:10:18.160 | on the question of this, but we think at minimum,
00:10:21.080 | it's hard to deny that there are at least some limits
00:10:23.800 | to the clear benefits to increasing scale,
00:10:27.880 | but there also seems like there are new scaling paradigms.
00:10:29.680 | So the question of test-time compute scaling
00:10:31.880 | is a pretty interesting one.
00:10:32.720 | It seems like OpenAI has cracked a version of this
00:10:34.880 | that works, and we think, A, foundation model labs
00:10:37.040 | will come up with better ways of doing this,
00:10:38.680 | and B, so far it largely works for very verifiable domains,
00:10:43.320 | things that look like math and physics
00:10:44.880 | and maybe secondarily software engineering,
00:10:46.400 | where we can get an objective value function.
00:10:48.800 | And I think an open question for the next year
00:10:50.320 | is going to be how do we generate those value functions
00:10:52.080 | for spaces that are not as well-constrained or well-defined?
00:10:55.080 | And so the question that this leaves us in is like,
00:10:58.240 | well, what does that mean for startups?
00:11:00.120 | And I think a prevailing view has been
00:11:02.440 | that we live in an AI bubble.
00:11:04.080 | There's an enormous amount of funding
00:11:05.900 | that goes towards AI companies and startups
00:11:07.760 | that is largely unjustified based on outcomes
00:11:09.960 | and what's actually working on the ground,
00:11:12.320 | and startups are largely raising money on hype.
00:11:14.640 | And so we pulled some pitch book data,
00:11:16.640 | and the 2024 number is probably incomplete
00:11:18.920 | since not all rounds are being reported,
00:11:20.360 | and largely suggests like actually there is
00:11:22.440 | a substantial recovery in funding,
00:11:24.440 | and maybe 2025 looks something like 2021.
00:11:27.240 | But if you break out the numbers here a bit more,
00:11:30.640 | the red is actually just a small number
00:11:32.600 | of foundation model labs,
00:11:33.480 | like what you would think of
00:11:34.320 | as the largest labs raising money,
00:11:36.380 | which is upwards of 30 to $40 billion this year.
00:11:39.000 | And so the reality of the funding environment
00:11:41.040 | actually seems like much more sane and rational.
00:11:43.480 | It doesn't look like we're headed to a version of 2021.
00:11:45.800 | In fact, the foundation model labs account
00:11:47.900 | for an outsized amount of money being raised,
00:11:50.500 | but the set of money going to companies
00:11:53.580 | that are working seems much more rational.
00:11:55.600 | And we wanted to give you,
00:11:56.680 | we can't share numbers for every company,
00:11:58.760 | but this is one of our portfolio companies
00:12:01.000 | growing really, really quickly.
00:12:03.260 | We think zero to 20 and just PLG style spending
00:12:05.940 | is pretty impressive.
00:12:06.780 | If any of you are doing better than that,
00:12:08.360 | you should come find us.
00:12:09.200 | We'd love to chat.
00:12:10.040 | And so what we wanted to try and center discussion on,
00:12:16.060 | this is certainly not all of the companies
00:12:17.460 | that are making 10 million more or revenue and growing,
00:12:19.900 | but we took a selection of them
00:12:21.140 | and wanted to give you a couple ideas
00:12:23.060 | of patterns that we've noticed
00:12:24.700 | that seem to be working across the board.
00:12:26.740 | The first one that we've noticed
00:12:28.920 | is like first wave service automation.
00:12:30.340 | So we think there's a large amount of work
00:12:32.780 | that doesn't get done at companies today,
00:12:35.100 | either because it is too expensive
00:12:36.700 | to hire someone to do it.
00:12:37.560 | It's too expensive to provide them context
00:12:40.020 | and enable them to be successful
00:12:42.100 | at whatever the specific role is,
00:12:44.380 | or it's too hard to manage those set of people.
00:12:46.740 | So prescribing it's too expensive
00:12:48.540 | to hire the specific set of people.
00:12:49.980 | For Sierra and Decagon,
00:12:50.940 | for customer support style companies,
00:12:52.260 | it's really useful to do like next level automation.
00:12:54.800 | And then there's obviously growth in that.
00:12:56.220 | And for Harvey and Even Up,
00:12:57.060 | the story is you can do first wave professional services
00:13:01.300 | and then grow beyond that.
00:13:02.860 | Second trend that we've noticed
00:13:05.100 | is a better search new friends.
00:13:07.320 | So we think that there is a,
00:13:09.140 | it's pretty impressive
00:13:09.980 | like how effective text modalities have been.
00:13:12.060 | So Character and Replica
00:13:13.220 | have been remarkably successful companies.
00:13:15.060 | And there's a whole host of not safer work chatbots as well
00:13:18.060 | that are pretty effective at just text generation.
00:13:20.780 | They're pretty compelling mechanisms.
00:13:22.100 | And on the productivity side,
00:13:22.940 | Perplexity and Glean have demonstrated this as well.
00:13:24.740 | I worked at a search company for a while.
00:13:26.180 | I think the changing paradigms
00:13:27.800 | of the how people capture
00:13:28.940 | and learn information is pretty interesting.
00:13:31.220 | We think it's likely text isn't the last medium.
00:13:33.800 | They're infographics for sets of information
00:13:35.760 | that seem more useful
00:13:36.600 | or sets of engagement that are more engaging.
00:13:38.900 | But this feels like a pretty interesting place to start.
00:13:43.220 | - Oh, yeah.
00:13:45.580 | Okay, Mike.
00:13:46.540 | So one thing that I've worked on investing in
00:13:49.940 | in a long time is democratization of different skills,
00:13:52.580 | be they creative or technical.
00:13:54.420 | This has been an amazing few years for that
00:13:56.860 | across different modalities, audio, video,
00:14:01.400 | general image, media, text,
00:14:03.660 | and now code and really fully functioning applications.
00:14:07.320 | One thing that's really interesting
00:14:09.620 | about the growth driver for all of these companies
00:14:12.540 | is the end users, in large part,
00:14:15.100 | are not people that we thought of as,
00:14:18.020 | we, the venture industry, you know, the royal we,
00:14:20.300 | thought of as important markets before.
00:14:22.540 | And so a premise we have as a fund
00:14:24.980 | is that there's actually much more instinct for creativity,
00:14:28.860 | visual creativity, audio creativity, technical creativity,
00:14:31.900 | than like there's latent demand for it.
00:14:35.180 | And AI applications can really serve that.
00:14:37.680 | I think in particular,
00:14:38.740 | Midjourney was a company that is in the vanguard here
00:14:41.220 | and nobody understood for a long time
00:14:43.100 | because the perhaps outside view
00:14:45.540 | is like how many people want to generate images
00:14:47.900 | that are not easily, you know,
00:14:50.340 | they're raster, they're not easily editable,
00:14:52.120 | they can't be using these professional contexts
00:14:54.020 | in a complete way.
00:14:55.340 | And the answer is like an awful lot, right,
00:14:57.340 | for a whole range of use cases.
00:14:58.580 | And I think we'll continue to find that,
00:15:00.080 | especially as the capabilities improve.
00:15:02.180 | And we think the range of quality and controllability
00:15:07.180 | that you can get in these different domains
00:15:10.840 | is still, it's very deep and we're still very early.
00:15:13.440 | And then I think as,
00:15:15.940 | if we're in the first or second inning of this AI wave,
00:15:19.900 | one obvious place to go invest
00:15:22.820 | and to go build companies is the enabling layers, right?
00:15:26.180 | Shorthand for this is obviously compute and data.
00:15:29.260 | I think that the needs for data
00:15:31.980 | are largely changed now as well.
00:15:34.380 | You need more expert data.
00:15:36.260 | You need different forms of data.
00:15:37.900 | We'll talk about that later in terms of who has,
00:15:40.140 | like let's say reasoning traces in different domains
00:15:43.080 | that are interesting to companies doing their own training.
00:15:46.360 | But this is an area that has seen explosive growth
00:15:49.560 | and we continue to invest here.
00:15:51.120 | Okay, so maybe time for some opinions.
00:15:55.480 | There was a prevailing narrative
00:16:00.080 | that some part from companies, some part from investors,
00:16:05.000 | it's a fun debate as to where is the value in the ecosystem
00:16:08.560 | and can there be opportunities for startups?
00:16:11.660 | If you guys remember the phrase GPT wrapper,
00:16:13.900 | it was like the dominant phrase
00:16:15.500 | in the tech ecosystem for a while of,
00:16:18.420 | and what it represented was this idea
00:16:20.520 | that there was no value at the application layer.
00:16:23.020 | You had to do pre-training
00:16:24.500 | and then like nobody's gonna catch open AI in pre-training.
00:16:27.040 | And this isn't like a knock on open AI at all.
00:16:32.040 | These labs have done amazing work enabling the ecosystem
00:16:35.140 | and we continue to partner with them and others.
00:16:37.940 | But it's simply untrue as a narrative, right?
00:16:42.940 | The odds are clearly in favor
00:16:44.740 | of a very rich ecosystem of innovation.
00:16:47.860 | You have a bunch of choices of models
00:16:49.780 | that are good at different things.
00:16:51.700 | You have price competition, you have open source.
00:16:55.100 | I think an underappreciated impact of test time scaling
00:16:58.660 | is you're going to better match user value
00:17:01.340 | with your spend on compute.
00:17:03.080 | And so if you are a new company
00:17:05.100 | that can figure out how to make these models
00:17:06.780 | useful to somebody, the customer can pay for the compute
00:17:09.580 | instead of you taking as a startup,
00:17:11.740 | the CapEx for pre-training or RL upfront.
00:17:16.740 | And as Pranav mentioned, small models,
00:17:21.340 | especially if you know the domain
00:17:22.460 | can be unreasonably effective.
00:17:24.340 | And the product layer has,
00:17:26.380 | if we look at the sort of cluster of companies
00:17:29.060 | that we described,
00:17:30.060 | shown that it is creating and capturing value
00:17:32.300 | and that it's actually a pretty hard thing
00:17:33.540 | to build great products that leverage AI.
00:17:36.820 | So broadly, like we have a point of view
00:17:39.940 | that I think is actually shared by many of the labs
00:17:41.940 | that the world is full of problems
00:17:44.180 | and the last mile to go take even AGI
00:17:47.740 | into all of those use cases is quite long.
00:17:50.240 | Okay, another prevailing belief is that,
00:17:54.860 | or another great debate that Sean could host is like,
00:17:57.820 | does the value go to startups or incumbents?
00:17:59.940 | We must admit some bias here,
00:18:01.740 | even though we have friends and portfolio,
00:18:04.100 | former portfolio companies
00:18:05.060 | that would be considered incumbents now.
00:18:06.980 | But, oh, sorry, swap views.
00:18:11.980 | Sorry, there are markets in venture
00:18:16.940 | that have been considered traditionally like too hard, right?
00:18:19.700 | Like just bad markets for the venture capital spec,
00:18:23.540 | which is capital efficient, rapid growth.
00:18:25.980 | That's a venture backable company
00:18:27.820 | where the end output is a tens of billions of dollars
00:18:32.060 | of enterprise value company.
00:18:34.660 | And these included areas like legal healthcare,
00:18:37.300 | defense, pharma, education,
00:18:39.620 | any traditional venture firm would say like bad market,
00:18:43.820 | nobody makes money there, it's really hard to sell,
00:18:45.940 | there's no budget, et cetera.
00:18:47.420 | And one of the things that's interesting
00:18:48.860 | is if you look at the cluster of companies
00:18:50.460 | that has actually been effective over the past year,
00:18:52.980 | some of them are in these markets
00:18:54.700 | that were traditionally non-obvious, right?
00:18:56.580 | And so perhaps one of our more optimistic views
00:19:00.580 | is that AI is really useful.
00:19:02.420 | And if you make a capability that is novel,
00:19:05.580 | that is several magnitudes, orders of magnitude cheaper,
00:19:09.940 | then actually you can change the buying pattern
00:19:11.700 | and the structure of these markets.
00:19:13.380 | And maybe the legal industry didn't buy anything
00:19:15.980 | 'cause it wasn't anything worth buying
00:19:17.180 | for a really long time, that's one example.
00:19:19.660 | We also think that like,
00:19:21.020 | what was the last great consumer company?
00:19:23.820 | Maybe it was Discord or Roblox
00:19:25.580 | in terms of things that started
00:19:26.660 | that have just like really enormous user basis
00:19:29.860 | and engagement until we had these consumer chatbots
00:19:34.260 | of different kinds and like the next,
00:19:36.300 | perhaps the next generation of search.
00:19:38.220 | As Pranav mentioned, we think that the opportunity
00:19:42.860 | for social and media generation and games
00:19:45.340 | is large and new in a totally different way.
00:19:49.860 | And finally, in terms of the markets that we look at,
00:19:53.580 | I think there's broad recognition now
00:19:56.180 | that you can sell against outcomes and services
00:19:59.260 | rather than software spend with AI
00:20:01.380 | because you're doing work versus just giving people
00:20:03.580 | the ability to do a workflow.
00:20:05.660 | But if you take that one step further,
00:20:07.740 | we think there's elastic demand for many services, right?
00:20:11.340 | Our classic example is there's on order of 20 to 25 million
00:20:16.340 | professional software developers in the world.
00:20:19.180 | I imagine much of this audience is technical.
00:20:22.740 | Demand for software is not being met, right?
00:20:27.100 | If we take the cost of software
00:20:29.180 | and high quality software down to orders of magnitude,
00:20:32.220 | we're just gonna end up with more software in the world.
00:20:34.700 | We're not gonna end up with fewer people doing development.
00:20:37.420 | At least that's what we would argue.
00:20:39.700 | And then finally, on the incumbent versus startup question,
00:20:44.700 | the prevailing narrative is incumbents
00:20:47.260 | have the distribution, the product surfaces and the data.
00:20:50.220 | Don't bother competing with them.
00:20:51.460 | They're gonna create and capture the value
00:20:52.980 | and share some of it back with their customers.
00:20:54.940 | I think this is only partially true.
00:20:57.540 | Incumbents have the distribution.
00:20:59.260 | They have always had the distribution.
00:21:00.860 | Like the point of the startup is you have to go fight
00:21:02.820 | with a better product or a more clever product
00:21:05.860 | and maybe a different business model
00:21:07.660 | to go get new distribution.
00:21:09.580 | But the specifics around the product surface and the data
00:21:12.980 | I think are actually worth understanding.
00:21:15.100 | There's a really strong innovators dilemma.
00:21:17.140 | If you look at the SaaS companies that are dominant,
00:21:19.700 | they sell by seat.
00:21:21.100 | And if I'm doing the work for you,
00:21:22.660 | I don't necessarily wanna sell you seats.
00:21:24.220 | I might actually decrease the number of seats.
00:21:26.900 | The tens, the decades of years,
00:21:30.500 | the millions of man and woman hours of code
00:21:33.460 | that have been written to enable a particular workflow
00:21:38.460 | in CRM, for example,
00:21:41.140 | may not matter if I don't want people to do that workflow
00:21:43.860 | of filling out the database every Friday anymore.
00:21:46.460 | And so I do think that this sunk cost
00:21:49.580 | or the incumbent advantage gets highly challenged
00:21:52.780 | by new UX and code generation as well.
00:21:55.620 | And then one disappointing learning
00:21:57.220 | that we found in our own portfolio
00:21:59.460 | is no one has the data we want in many cases, right?
00:22:03.060 | So imagine you are trying to automate
00:22:06.700 | a specific type of knowledge work.
00:22:09.620 | And what you want is the reasoning trace,
00:22:13.580 | all of the inputs and the output decision.
00:22:16.780 | Like that sounds like a very useful set of data.
00:22:19.940 | And the incumbent companies in any given domain,
00:22:22.380 | they never save that data, right?
00:22:23.940 | Like they have a database with the outputs some of the time.
00:22:26.740 | And so I would say one of the things
00:22:29.940 | that is worth thinking through as a startup
00:22:32.300 | is when an incumbent says they have the data,
00:22:35.420 | like what is the data you actually need
00:22:36.580 | to make your product higher quality?
00:22:39.100 | Okay, so in summary,
00:22:41.260 | our shorthand for the set of changes that are happening
00:22:45.060 | is software 3.0.
00:22:46.300 | We think it is a full stack rethinking
00:22:48.740 | and it enables a new generation of companies
00:22:52.420 | to have a huge advantage.
00:22:54.020 | The speed of change favors startups.
00:22:56.820 | If the floor is lava,
00:22:57.820 | it's really hard to turn a really big ship.
00:23:00.380 | I think that some of the CEOs of large companies
00:23:02.740 | now are incredibly capable,
00:23:04.100 | but they're still trying to make 100,000 people
00:23:06.380 | move very quickly in a new paradigm.
00:23:08.820 | The market opportunities are different, right?
00:23:10.940 | These markets that we think are interesting and very large,
00:23:13.740 | like represent a trillion dollars of value
00:23:15.660 | are not just the replacement software markets
00:23:18.540 | of the last two decades.
00:23:20.940 | It's not clear what the business model
00:23:22.660 | for many of these companies should be.
00:23:24.500 | Sierra just started talking about charging for outcomes.
00:23:27.420 | Outcomes-based pricing has been this holy grail idea
00:23:30.180 | in software, and it's been very hard,
00:23:31.900 | but now we do more work.
00:23:33.340 | There are other business model challenges.
00:23:37.580 | And so, our companies, they spend a lot more on compute
00:23:41.580 | than they have in the past.
00:23:43.220 | They spend a lot with the foundation model providers.
00:23:45.380 | They think about gross margin.
00:23:47.580 | They think about where to get the data.
00:23:49.340 | It's a time where you need to be really creative
00:23:51.180 | about product versus just replace the workflows of the past.
00:23:56.180 | And it might require ripping out those workflows entirely.
00:23:59.020 | It's a different development cycle.
00:24:01.460 | I bet most of the people in this room have written evals
00:24:04.660 | and compared the academic benchmark to a real-world eval
00:24:08.220 | and said, "That's not it."
00:24:10.540 | And how do I make a user understand
00:24:14.900 | the non-deterministic nature of these outputs
00:24:18.700 | or gracefully fail?
00:24:19.700 | I think that's a different way to think about product
00:24:22.420 | than in the past.
00:24:24.140 | And we need to think about infrastructure again.
00:24:27.260 | There was this middle period where the cloud providers,
00:24:30.660 | the hyperscalers, took this problem away
00:24:32.180 | from software developers,
00:24:33.300 | and it was all just gonna be front-end people at some point.
00:24:36.220 | And it's like, we are not there anymore.
00:24:37.620 | We're back in the hardware era where people are acquiring
00:24:41.060 | and managing and optimizing compute.
00:24:42.580 | And I think that will really matter
00:24:43.660 | in terms of capability in companies.
00:24:46.380 | So I guess we'll end with a call to action here
00:24:50.740 | and encourage all of you to seize the opportunity.
00:24:55.340 | It is the greatest technical and economic opportunity
00:24:57.380 | that we've ever seen.
00:24:58.340 | We made a decade-plus career-type bet on it.
00:25:02.340 | And we do a lot of work with the foundation model companies.
00:25:07.340 | We think they are doing amazing work,
00:25:10.140 | and they're great partners and even co-investors
00:25:12.660 | in some of our efforts.
00:25:13.660 | But I think all of the focus on their interesting missions
00:25:17.740 | around AGI and safety do not mean
00:25:21.340 | that there are not opportunities
00:25:22.660 | in other parts of the economy.
00:25:24.260 | The world is very large, and we think much of the value
00:25:27.060 | will be distributed in the world through an unbundling
00:25:29.260 | and eventually a re-bundling,
00:25:31.260 | as often happens in technology cycles.
00:25:34.540 | So we think this is a market
00:25:35.900 | that is structurally supportive of startups.
00:25:37.940 | We're really excited to try to work
00:25:39.900 | with the more ambitious ones.
00:25:41.580 | And the theme of 2024 to us has been like,
00:25:45.420 | well, thank goodness, this is an ecosystem
00:25:47.900 | that is much friendlier to startups than 2023.
00:25:51.660 | It is what we hoped.
00:25:53.420 | And so, you know, please ask those questions
00:25:56.700 | and take advantage of the opportunity.
00:25:58.580 | - Thanks for joining our first talk
00:26:07.660 | from Latent Space Live at NeurIPS 2024 in Vancouver.
00:26:11.620 | As always, this is your AI co-host, Charlie.
00:26:15.140 | I want to give a huge thank you to Sarah Guo
00:26:17.420 | and Pranav Reddy for sharing their invaluable insights
00:26:20.300 | on the state of AI in 2024.
00:26:23.020 | Be sure to check the link in the description
00:26:24.900 | for their presentation slides, social media links,
00:26:27.940 | and additional resources.
00:26:29.860 | Watch out and take care.
00:26:31.500 | (upbeat music)