Windsurf: The Enterprise AI IDE

00:00:00.000 | (upbeat music)

00:00:02.580 | - Hey, everyone.

00:00:05.860 | Welcome to the Latent Space Podcast.

00:00:07.480 | - This is Alessio, partner and CTO at Decibel Partners.

00:00:10.200 | And I'm joined by my co-host Swiggs,

00:00:11.800 | funder of Small.ai.

00:00:13.000 | - Hey, and today we are delighted to be,

00:00:16.000 | I think the first podcast in the new Codium office.

00:00:18.560 | So thanks for having us and welcome Farun and Anshul.

00:00:22.080 | - Thanks for having us.

00:00:23.040 | Yeah, thanks for having us.

00:00:24.240 | - This is the Silicon Valley office?

00:00:25.640 | - Yeah.

00:00:26.480 | - So like, what's the story behind this?

00:00:27.960 | - The story is that the office was previously,

00:00:31.440 | so we used to be on Castro Street.

00:00:33.120 | So this is in Mountain View.

00:00:34.200 | And I think a lot of the people at the company previously,

00:00:37.640 | you know, were in SF or still in SF.

00:00:39.960 | And actually one thing if you notice about the company

00:00:42.160 | is it's actually like a two minute walk from the Caltrain.

00:00:44.520 | And I think we were, we didn't want to move the office

00:00:47.360 | like very far away from the Caltrain.

00:00:48.840 | That would probably, you know,

00:00:50.240 | piss off a lot of the people that lived in San Francisco,

00:00:53.080 | this guy included.

00:00:54.820 | So we were like scouting a lot of spaces in the nearby area.

00:00:57.880 | And this area popped up.

00:00:59.600 | It previously was being leased by,

00:01:02.120 | I think, Facebook/WhatsApp.

00:01:04.000 | And then immediately after that, Ghost Autonomy.

00:01:06.480 | And then now here we are.

00:01:08.220 | And we also, you know,

00:01:09.560 | I guess one of the things that the landlord told us was

00:01:11.400 | this was the place that they shot all the scenes

00:01:13.560 | for Silicon Valley, at least like externally

00:01:15.320 | and stuff like that.

00:01:16.160 | So that just became a meme.

00:01:17.400 | Trust me, that wasn't like the main reason why we did it.

00:01:20.200 | But we've leaned into it.

00:01:21.580 | - It doesn't hurt.

00:01:22.420 | - Yeah.

00:01:23.240 | - Yeah.

00:01:24.080 | And obviously that played a little bit

00:01:24.920 | into your launch with Windsurf as well.

00:01:27.040 | So, let's get caught up.

00:01:28.680 | You were guest number four?

00:01:30.560 | - I think I was two.

00:01:31.640 | - Maybe it was two.

00:01:32.680 | - Might have been two.

00:01:34.660 | - So a lot has happened since then.

00:01:36.280 | You've raised a huge round

00:01:38.280 | and also just launched your idea.

00:01:39.720 | Like what's been the progress over the last year or so

00:01:42.120 | since the in-space people last saw you?

00:01:44.560 | - Yeah.

00:01:45.380 | So I think the biggest things that have happened

00:01:47.040 | are Codiums extensions have continued to gain

00:01:50.200 | a lot of sort of popularity.

00:01:52.280 | You know, we have over 800,000 sort of developers

00:01:55.040 | that use that product.

00:01:56.240 | Lots of large enterprises also use the product.

00:01:58.400 | We were recently awarded

00:01:59.640 | JP Morgan Chase's Hall of Innovation Award,

00:02:01.720 | which is usually not something a company gets,

00:02:03.880 | you know, within a year of deploying an enterprise product.

00:02:06.440 | And then large companies like Dell and stuff use the product.

00:02:08.800 | So I think we've seen a lot of traction

00:02:10.080 | on the enterprise space.

00:02:11.180 | But I think one of the most exciting things

00:02:13.000 | we've launched recently is this actually IDE

00:02:16.240 | called Windsurf.

00:02:17.360 | And I think for us,

00:02:18.560 | one of the things that we've always thought about is

00:02:20.480 | how do we build the most powerful AI system

00:02:22.560 | for developers everywhere?

00:02:23.920 | The reason why we started out with the extension system

00:02:25.800 | was we felt that there were lots of developers

00:02:28.740 | that were not going to be on one platform.

00:02:30.520 | And that still is true, by the way.

00:02:32.040 | Outside of Silicon Valley,

00:02:33.240 | a lot of people don't use GitHub, right?

00:02:34.960 | This is like a very surprising finding,

00:02:36.880 | but most people use GitLab, Bitbucket,

00:02:39.960 | Garrett, Perforce, CVS, Harvest, Mercurial.

00:02:42.680 | I could keep going down the list,

00:02:43.920 | but there's probably 10 of them.

00:02:45.200 | GitHub might have less than 10% penetration

00:02:47.680 | of the Fortune 500, full penetration.

00:02:49.720 | It's very small.

00:02:50.680 | And then also on top of that,

00:02:52.320 | GitHub has very high switching costs

00:02:54.200 | for source code management tools, right?

00:02:56.080 | Because you actually need to switch over

00:02:57.360 | all the dependent systems on this workflow software.

00:02:59.600 | It's much harder than even switching off of a database.

00:03:02.040 | So because of that,

00:03:03.260 | we actually found ways in which

00:03:04.440 | we could be better partners to our customers,

00:03:06.080 | regardless of where they started their source code.

00:03:08.000 | And then more specifically on the IDE category,

00:03:10.080 | a lot of developers, surprise, surprise,

00:03:11.560 | don't just write TypeScript and Python, right?

00:03:13.480 | They write Java, they write Golang,

00:03:15.840 | they write a lot of different languages.

00:03:17.320 | And then a high quality language servers

00:03:18.960 | and debuggers matter.

00:03:20.200 | Very honestly, JetBrains has the best debugger for Java.

00:03:23.560 | It's not even close, right?

00:03:24.860 | These are extremely complex pieces of software.

00:03:27.420 | We have customers where over 70%

00:03:29.120 | of their developers use JetBrains.

00:03:30.440 | And because of that,

00:03:31.280 | we wanted to provide a great experience

00:03:32.940 | wherever the developer was.

00:03:34.320 | But one thing that we found was lacking was,

00:03:36.520 | you know, we were running into the limitations

00:03:38.200 | of building within the VS Code ecosystem

00:03:40.300 | on the VS Code platform.

00:03:41.600 | And I think we felt that there was an opportunity

00:03:44.620 | for us to build a premier sort of experience.

00:03:47.440 | And that was within the reach of the team, right?

00:03:49.240 | The team has done all the work,

00:03:50.400 | all the infrastructure work

00:03:51.660 | to build the best possible experience, right?

00:03:53.320 | And plug it into every ID.

00:03:54.800 | Why don't we just build our own ID

00:03:56.580 | that is by far the best experience?

00:03:58.680 | And as these agentic products

00:04:00.560 | sort of become more and more possible,

00:04:02.480 | and all the research we've done on retrieval

00:04:04.160 | and just reasoning about code bases

00:04:05.720 | became more and more to life.

00:04:07.200 | We were like, hey, if we launch this agentic product

00:04:09.900 | on top of a system

00:04:11.300 | that we didn't have a lot of control over,

00:04:13.320 | it's just gonna limit the value of the product

00:04:14.920 | and we're just not gonna be able to do the best tool.

00:04:16.840 | That's why we were super excited to launch Windsurf.

00:04:19.360 | I do think it is the most powerful IDE system

00:04:21.700 | out there right now.

00:04:23.160 | In the capability, right?

00:04:24.520 | And this is just the beginning.

00:04:25.440 | I think we suspect that there's much, much more we can do,

00:04:28.720 | more than just the auto-complete sort of side, right?

00:04:30.960 | When we originally talked,

00:04:31.900 | probably auto-complete was the only piece of functionality

00:04:33.880 | the product actually had.

00:04:35.120 | And we've come a long way since then, right?

00:04:37.560 | These systems can now reason about large code bases

00:04:40.000 | without you adding everything, right?

00:04:41.960 | Like when you use Google,

00:04:42.800 | do you say like @NewYorkTimesPost, blah, blah, blah,

00:04:45.160 | and like ask it a question?

00:04:46.400 | No.

00:04:47.240 | We want it to be a magical experience

00:04:48.440 | where you don't need to do that.

00:04:49.640 | We want it to actually go out and execute code.

00:04:51.160 | We think code execution is a really, really important piece.

00:04:54.160 | And when you write software,

00:04:55.800 | you not only just kind of come up with an idea,

00:04:57.960 | the way software kind of gets created

00:04:59.960 | is software is originally this amorphous blob.

00:05:02.560 | And as time goes on and you have an idea,

00:05:05.280 | the blob and the cloud sort of disappear

00:05:07.080 | and you see this mountain.

00:05:08.320 | And we want it to be the case

00:05:09.200 | that as soon as you see the mountain,

00:05:10.880 | the AI helps you get to the mountain.

00:05:12.400 | And as soon as you see the mountain,

00:05:13.960 | the AI just creates the mountain for you, right?

00:05:16.080 | And that's why we don't believe in this sort of modality

00:05:18.360 | where you just write a task

00:05:19.560 | and it just goes out and does it, right?

00:05:20.840 | It's good for zero to one apps.

00:05:21.840 | And I think people have been seeing

00:05:23.080 | Windsor is capable of doing that.

00:05:24.200 | And I'll let Anshul talk about that a little bit.

00:05:26.120 | But we've been seeing real value

00:05:28.280 | in real software development,

00:05:29.720 | which is more to say,

00:05:30.760 | this is not to say that current tools can't,

00:05:32.960 | but I think more in the process of actually evolving code

00:05:36.200 | from a very basic idea.

00:05:37.720 | Code is not really built as you have a PRD

00:05:39.600 | and then you get some output out.

00:05:41.760 | It's more like you have a general vision.

00:05:43.880 | And yes, and as you write the code,

00:05:45.600 | you get more and more clarity

00:05:46.640 | on approaches that don't work and do work.

00:05:48.200 | You're killing ideas and creating ideas constantly.

00:05:50.880 | And we think Windsor is the right paradigm for that.

00:05:53.040 | - Can you spell out what you couldn't do in VS Code?

00:05:57.000 | Because I think when we did the cursor episode,

00:05:59.480 | explain that everybody on Agri News is like,

00:06:01.920 | oh, why did you fork?

00:06:03.960 | Why you could have done it in an extension?

00:06:05.560 | Like, can you maybe just explain more of those limitations?

00:06:09.080 | - I mean, I think a lot of the limitations

00:06:11.280 | around like APIs are pretty well-documented.

00:06:13.520 | I don't know if we need to necessarily

00:06:15.400 | go down that rabbit hole.

00:06:16.720 | I think it was when we started thinking,

00:06:18.480 | okay, what are the pieces that we actually need to give

00:06:20.640 | the AI to get to that kind of emergent behavior

00:06:24.520 | that Bruin talked about, right?

00:06:26.320 | And yes, we were talking about

00:06:27.920 | all the knowledge retrieval systems

00:06:29.320 | that we've been building for the enterprise all this time.

00:06:31.360 | Like, that's obviously a component of that.

00:06:33.600 | You know, we were talking about all the different tools

00:06:35.000 | that we could give it access to,

00:06:36.000 | so they can go like do that kind of terminal execution,

00:06:38.120 | things like that.

00:06:38.960 | And the third main category that we realized

00:06:41.080 | would be like kind of that magical thing

00:06:42.480 | where you're not out there writing out a PRD.

00:06:44.760 | You're not scoping the problem for the AI.

00:06:46.680 | Is that if we're actually being able to understand

00:06:48.400 | the kind of the trajectory

00:06:49.240 | of what developers are doing within the editor, right?

00:06:51.400 | If we actually are be able to see like,

00:06:52.600 | oh, the developer just went and opened up

00:06:55.440 | this part of the directory and tried to view it,

00:06:56.880 | then they made these kind of edits.

00:06:58.280 | And they tried to do like some kind of commands

00:06:59.920 | in the terminal.

00:07:00.760 | And if we actually understand that trajectory,

00:07:02.800 | then our ability for the AI to just be immediately be like,

00:07:05.600 | oh, I understand your intent.

00:07:06.840 | This is what you want to do

00:07:07.720 | without you having to spell it all out for it.

00:07:09.880 | That is one like that kind of like magic

00:07:11.360 | would really happen.

00:07:12.360 | I think that was kind of like that intuition.

00:07:13.840 | So you have the restrictions of the APIs

00:07:16.280 | that are well-documented.

00:07:17.560 | We have the kind of vision of like

00:07:19.840 | what we actually need to be able to hook into

00:07:21.680 | to really expose this.

00:07:22.680 | And I think it was that combination of those two

00:07:24.440 | where we were like,

00:07:25.360 | I think it's about time to do the editor.

00:07:26.840 | The editor was not like a necessarily like a new idea.

00:07:28.960 | I think we've been talking about the editor

00:07:30.160 | for a very long time.

00:07:31.520 | I think it's like, of course,

00:07:32.720 | we just pulled it all together in the last couple of months,

00:07:35.080 | but it was always something in the back of the mind.

00:07:37.080 | And it's only when we started realizing,

00:07:38.320 | okay, the models are not capable of doing this.

00:07:40.880 | We actually can look at this data.

00:07:42.240 | Like we have a really good context awareness system.

00:07:43.680 | We're like, I think now's the time.

00:07:45.440 | And we went on and executed on it.

00:07:47.360 | - So it's basically not,

00:07:49.160 | it's not like one action you couldn't do,

00:07:50.760 | but it's like how you brought it all together.

00:07:52.840 | It's like the VS Code is kind of like sandbox, so to speak.

00:07:55.240 | - Yeah, let me maybe like even just to go one step deeper

00:07:58.240 | on each of the aspects that Anshul talked about.

00:08:00.320 | Let's go with the API aspect.

00:08:02.000 | So right now, I'll give you an example.

00:08:03.560 | Supercomplete is actually a feature

00:08:04.880 | that I think is like very exciting about the product, right?

00:08:07.360 | It can suggest refactors of the code.

00:08:09.720 | I think you can do it quickly and very powerfully.

00:08:12.040 | On VS Code, actually the problem for us

00:08:14.440 | wasn't actually being able to implement the feature.

00:08:16.000 | We had the feature for a while.

00:08:17.320 | Problem was actually even to show the feature,

00:08:19.000 | VS Code would not expose an API for us to do this.

00:08:21.320 | So what we actually ended up doing

00:08:22.600 | was dynamically generating PNGs

00:08:25.240 | to actually go out and showcase this.

00:08:27.000 | It was not really aligned.

00:08:28.120 | We actually ended up doing it ourselves

00:08:29.400 | and it took us a couple hours

00:08:30.360 | to actually go out and implement this, right?

00:08:32.360 | And that wasn't because we were bad engineers.

00:08:34.200 | No, our good engineering time was being spent

00:08:36.080 | fighting against the system rather than being a good system.

00:08:38.400 | Another example is we needed to go out

00:08:40.000 | and find ways to refactor the code.

00:08:41.760 | The VS Code API would constantly keep breaking on us.

00:08:44.160 | We'd constantly need to show a worse and worse experience.

00:08:46.680 | This actually comes down to the second point

00:08:48.800 | which Anshul brought up, which is like,

00:08:50.080 | we can come up with great work and great research.

00:08:52.040 | All the work we have here is not like,

00:08:53.840 | the research on Cascade is not like a couple month thing.

00:08:55.720 | This is like a nine months to a year thing

00:08:58.120 | that we've been investigating as a company.

00:09:00.080 | Investing in on evals, right?

00:09:01.640 | Even the evals for this are a lot of effort, right?

00:09:04.400 | A lot of actually systems work to actually go out and do it.

00:09:06.480 | But ultimately, this needs to be a product

00:09:08.000 | that developers actually use.

00:09:09.360 | And I think, let's even go for Cascade, for example,

00:09:12.320 | and looking at the trajectory.

00:09:13.440 | - And can you define Cascade

00:09:14.560 | because that's the first time you brought it up?

00:09:15.920 | - Yeah, so Cascade is the product

00:09:17.600 | that is the actual agentic part of the product, right?

00:09:20.760 | That is capable of taking information

00:09:23.480 | from both these human trajectories and these AI trajectories,

00:09:25.680 | what the human ended up doing,

00:09:27.000 | what the AI ended up doing,

00:09:27.880 | to actually propose changes and actually execute code

00:09:30.800 | to finally get you the final work output, right?

00:09:33.160 | I'll even talk about something very basic.

00:09:34.520 | Cascade gives you a bunch of code.

00:09:35.720 | We want developers to very easily

00:09:37.120 | be able to review this code, okay?

00:09:39.040 | Then we can show developers a hideous UI

00:09:41.000 | that they don't want to look at.

00:09:42.280 | And no one's gonna really use this product.

00:09:43.840 | And we think that this is like a fundamental building block

00:09:45.960 | for us to make the product materially better.

00:09:47.840 | If people are not even willing to use the building block,

00:09:49.920 | where does this go, right?

00:09:51.480 | And we just felt our ceiling was capped

00:09:53.280 | on what we could deliver as an experience.

00:09:55.440 | Interestingly, JetBrains is a much more configurable paradigm

00:09:58.520 | than VS Code is.

00:10:00.120 | But we just felt so limited

00:10:01.960 | on both the sort of directions that Anshul said,

00:10:05.000 | that we were just like,

00:10:05.840 | "Hey, if we actually remove these limitations,

00:10:08.440 | "we can move substantially faster."

00:10:10.160 | And we believe that this was a necessary step for us.

00:10:13.200 | - I'm curious more about the evals side of it,

00:10:15.560 | 'cause you brought it up.

00:10:16.560 | And we have to ask about evals

00:10:18.120 | anytime anyone brings up evals.

00:10:19.760 | How do you evaluate a thing like this

00:10:21.120 | that is so multi-step and so spanning so much context?

00:10:26.120 | - So what you can imagine we can sort of do,

00:10:28.680 | and this is like one of the beautiful things about code

00:10:30.400 | is code can be executed.

00:10:31.840 | We could go take a bunch of open source code,

00:10:33.880 | we can find a bunch of commits, right?

00:10:35.880 | And we can actually see if some of these commits

00:10:38.680 | have tests associated with them.

00:10:40.400 | We can start stripping the commits,

00:10:41.680 | and the approach of stripping the commits is good

00:10:44.200 | because it tests the fact that the code

00:10:45.800 | is in an incomplete state, right?

00:10:47.040 | When you're writing the commit,

00:10:48.120 | the goal is not the commit has already been written for you,

00:10:49.960 | you're giving it in a state

00:10:51.120 | that where the entire thing has not been written.

00:10:52.920 | And can we go out and actually retrieve the right snippets

00:10:55.200 | and actually come up with a cohesive plan

00:10:57.600 | and iterative loop that gets you to a state

00:10:59.880 | where the code actually passes?

00:11:01.080 | So you can actually break down

00:11:02.600 | and decompose this complex problem

00:11:04.600 | into like a planning, retrieval,

00:11:06.600 | and multi-step execution problem.

00:11:07.960 | And you can see on every single one of these axes

00:11:09.720 | is it getting better.

00:11:10.560 | And if you do this across enough repositories,

00:11:12.600 | you've turned this highly discontinuous

00:11:14.840 | and discrete problem of make a PR work

00:11:16.880 | versus make it not work into a continuous problem.

00:11:19.120 | And now that's a hill you can actually climb,

00:11:21.080 | and that's a way that you can actually apply research

00:11:23.080 | where it's like, "Hey, my retrieval got way better.

00:11:25.080 | "This made my eval get better," right?

00:11:27.040 | And then notice how the way the eval works is

00:11:29.720 | I'm not that interested in the eval

00:11:31.040 | where purely it's a commit message

00:11:32.720 | and you finish the entire thing.

00:11:34.080 | I'm more interested in the code is in an incomplete state,

00:11:36.920 | and the commit message isn't even given to you

00:11:38.840 | because that's another thing about developers.

00:11:40.560 | They are not willing to tell you

00:11:41.720 | exactly what's in their head.

00:11:43.080 | That's the actual important piece of this problem.

00:11:45.440 | We believe that developers will never

00:11:47.360 | completely pose the problem statement, right?

00:11:50.720 | Because the problem statement lives in their head.

00:11:52.600 | Conversations that you and I have had at the coffee area,

00:11:55.600 | conversations that I've had over Slack,

00:11:57.040 | conversations I've had over Jira, right?

00:11:59.160 | Maybe not Jira, let's say Linear, right?

00:12:00.840 | That's the cool thing now.

00:12:02.520 | - Don't talk about Jira.

00:12:03.360 | - Yeah, so conversations I've had on Linear,

00:12:06.560 | and all of these things come together

00:12:08.440 | to actually finally propose sort of a solution there,

00:12:10.920 | which is why we want to test the incomplete code.

00:12:13.120 | What happens if the state is in an incomplete state

00:12:15.280 | and am I actually able to make this pass without the commit?

00:12:17.960 | And can I actually guess your commit well?

00:12:19.800 | Now you can convert the problem

00:12:21.200 | into a mask prediction problem

00:12:23.120 | where you want to guess both the high-level intent

00:12:25.240 | and as well as the remainder of changes

00:12:26.920 | to make the actual test pass.

00:12:28.120 | And you can imagine if you build up all of these,

00:12:30.040 | now you can see, hey, my systems are getting better.

00:12:31.800 | Retrieval quality is getting better.

00:12:32.920 | And you can actually start testing this

00:12:34.120 | on larger and larger code bases, right?

00:12:35.920 | And I guess that's one thing that we, honestly,

00:12:38.400 | to be honest, we could have done a little faster.

00:12:40.480 | We had the technology to go out

00:12:41.720 | and build these zero-to-one apps very quickly.

00:12:43.200 | And I think people are using Windsurf to actually do that.

00:12:45.360 | And it's extremely impressive.

00:12:47.040 | But the real value, I think,

00:12:48.720 | is actually much deeper than that.

00:12:50.360 | It's actually that you take a large code base,

00:12:52.440 | and it's actually a really good first pass.

00:12:53.560 | And I'm not saying it's perfect,

00:12:55.080 | but it's only going to keep getting better.

00:12:57.560 | We have deep sort of infrastructure to that

00:12:59.440 | actually is validating

00:13:00.440 | that we are getting better on this dimension.

00:13:02.160 | - We mentioned the end-to-end evals that we have

00:13:04.240 | for this system, which I think are super cool.

00:13:06.120 | But I think you can even decompose

00:13:07.360 | each of those steps, right?

00:13:08.280 | The ideas of, just take a retrieval, for example, right?

00:13:11.240 | Like, how can we make eval for retrieval really good?

00:13:13.840 | And I think this is just a general thing

00:13:15.840 | that's been true about us as a company is like,

00:13:17.560 | most evals and benchmarks that exist out there

00:13:20.280 | for software development is kind of bogus.

00:13:22.800 | There's not really a better way of putting it.

00:13:24.440 | Like, okay, you have Sweebench, that's cool.

00:13:26.840 | No actual professional work looks like Sweebench.

00:13:29.760 | Like, human eval, same thing.

00:13:31.400 | These things are just a little kind of broken.

00:13:33.080 | So when you're trying to optimize against a metric

00:13:35.480 | that's a little bit broken,

00:13:36.320 | you end up making kind of suboptimal decisions.

00:13:38.120 | So something that we're always very keen on is like,

00:13:40.120 | okay, what is the actual metric that we want to test

00:13:42.480 | for this part of the system?

00:13:43.400 | And so take retrieval, for example.

00:13:45.000 | A lot of the benchmarks for these embedding-based systems

00:13:47.400 | are like needle-in-the-haystack problems.

00:13:49.280 | Like, I want to find this one particular piece

00:13:51.880 | of information out of all this potential context.

00:13:54.560 | That's not really what actually is necessary

00:13:56.640 | for doing software engineering

00:13:57.560 | because code is a super distributed knowledge store.

00:13:59.960 | You actually want to pull in snippets

00:14:01.320 | from a lot of different parts of the code

00:14:02.520 | based in order to do the work, right?

00:14:04.120 | And so we built systems that,

00:14:06.800 | instead of looking at retrieval at one,

00:14:08.680 | you're looking at retrieval at like 50.

00:14:10.920 | What are the 50 highest things

00:14:12.280 | that you can actually retrieve?

00:14:13.120 | And are you capturing all of the necessary pieces for that?

00:14:16.320 | And what are all the necessary pieces?

00:14:17.480 | Well, you can look again back at old commits

00:14:19.920 | and see what were all the different files

00:14:21.760 | that together were edited to make a commit

00:14:24.240 | because those are semantically similar things

00:14:26.320 | that might not actually show

00:14:27.880 | if you actually try to map out a code graph, right?

00:14:29.960 | And so we can actually build these kind of golden sets.

00:14:32.000 | We can do this evaluation even for sub-problems

00:14:34.200 | in the overall task.

00:14:35.400 | And so now we have an engineering team

00:14:37.040 | that can iterate on all of these things

00:14:38.560 | and still make sure that the end goal

00:14:40.400 | that we're trying to build too is really, really strong

00:14:42.400 | so that we have confidence in what we're pushing out.

00:14:44.120 | - And by the way, just to say one more thing

00:14:46.720 | about the SuiteBench thing,

00:14:47.840 | just to showcase these existing metrics,

00:14:49.960 | I think benchmarks are not a bad thing.

00:14:51.440 | You do want benchmarks.

00:14:52.280 | Actually, I would prefer if there were benchmarks

00:14:54.080 | versus, let's say, everything was just vibes, right?

00:14:56.640 | But vibes are also very important, by the way,

00:14:58.560 | because they showcase where the benchmark is not valuable

00:15:01.240 | because actually, vibes sometimes show you

00:15:02.760 | where criminal issues exist in the benchmark.

00:15:05.920 | But you look at some of the ways

00:15:07.120 | in which people have optimized SuiteBench.

00:15:08.600 | It's like, make sure to run PyTest every time X happens.

00:15:11.120 | And it's like, yeah, sure.

00:15:12.520 | You can start prompting it in every single possible way.

00:15:15.760 | And if you remove that, suddenly it doesn't get good at it.

00:15:18.040 | It's like, what really matters here?

00:15:19.920 | What really matters here is across a broad set of tasks,

00:15:23.000 | you're performing high-quality suggestions for people,

00:15:26.440 | and people love using the product.

00:15:27.800 | And I think, actually, the way these things work

00:15:30.000 | is beyond a certain point.

00:15:31.240 | Because yes, I actually think it's valuable

00:15:33.080 | beyond a certain point.

00:15:33.920 | But once it starts hitting the peak of these benchmarks,

00:15:36.200 | getting that last 10% actually probably is counterintuitive

00:15:39.640 | to the actual goal of what the benchmark was.

00:15:41.760 | Like, you probably should find a new hill to climb

00:15:43.880 | rather than sort of p-hacking or really optimizing

00:15:46.720 | for how you can get higher on the benchmark.

00:15:49.240 | - Yeah, we did an episode with Anthropic

00:15:50.920 | about their recent SuiteAgent, SuiteBench results.

00:15:54.320 | And we talked about the human eval versus SuiteBench.

00:15:56.960 | Or human eval is kind of like a Greenfield benchmark.

00:15:59.800 | You need to be good at that.

00:16:00.960 | SuiteBench is more excessing.

00:16:02.200 | But it sounds like, I mean, your eval creation

00:16:04.960 | is similar to SuiteBench as far as using get up commits

00:16:08.200 | and kind of like that history.

00:16:09.240 | But then it's more like masking at the commit level

00:16:12.160 | versus just testing the output of the thing.

00:16:15.520 | - Cool.

00:16:16.360 | We have some listener questions, actually,

00:16:17.560 | about the Windsurf launch.

00:16:18.560 | And obviously, I also want to give you the chance

00:16:20.440 | to just respond to Hacker News.

00:16:21.880 | (laughing)

00:16:23.280 | - Oh, man.

00:16:24.600 | (laughing)

00:16:25.440 | - Hey, let me tell you something very, very interesting.

00:16:28.200 | I love Hacker News as much as the next person.

00:16:31.040 | But the moment we launched our product,

00:16:33.120 | the first comment, like this was a year ago,

00:16:35.200 | the first comment was, "This product is a virus."

00:16:37.720 | And we were like--

00:16:38.560 | - This was the original Codium launch like two years ago.

00:16:40.520 | - This is the original.

00:16:41.360 | - Like, "I am analyzing the binary as we speak.

00:16:44.040 | "We'll report back."

00:16:45.200 | - And then he was like, "It's a virus."

00:16:47.720 | And I was like, "Dude, like, it's not a virus."

00:16:49.880 | (laughing)

00:16:51.480 | - We just want to give autocomplete suggestions.

00:16:53.240 | That's all we want to do.

00:16:54.440 | - Yeah.

00:16:55.280 | - Okay.

00:16:56.120 | - Wow, I didn't expect that.

00:16:57.600 | - And then there was, like, TO drama.

00:16:58.880 | There's enough drama on the launch to cover,

00:17:01.200 | but I don't know if we want to just

00:17:02.960 | make this a Cascade piece.

00:17:04.880 | But we had a bunch of people in our Discord

00:17:06.640 | try out the product, give a lot of feedback.

00:17:08.400 | One question people have is, like,

00:17:10.040 | to them, Cascade already felt pretty agentic.

00:17:12.720 | Like, is that something you want to do more of?

00:17:14.960 | You know, obviously, since you just launched on IDE,

00:17:17.440 | you're kind of like, you're focusing

00:17:18.800 | on having people write the code.

00:17:20.480 | But maybe this is kind of like the Trojan horse

00:17:22.600 | to just do more full-on, end-to-end, like, code creation.

00:17:26.800 | - Dev-in style.

00:17:27.640 | - Yeah, I think it's, like, how do you get there

00:17:29.760 | in a real, principled manner, right?

00:17:32.960 | We have, obviously, enterprise asking us all the time,

00:17:35.040 | like, "Oh, when's it going to be end-to-end work?"

00:17:36.880 | The reality is, like, okay, well,

00:17:38.200 | if we have something in the IDE that, again,

00:17:39.720 | can see your entire actions and get a lot of intent

00:17:41.920 | that you can't actually get if you're not in the IDE,

00:17:44.320 | if the agent there has to always get human involvement

00:17:49.000 | to keep on fixing itself, it's probably not ready

00:17:51.600 | to become a full end-to-end automated system,

00:17:53.520 | 'cause then we're just going to turn into a linter

00:17:54.840 | where, like, it produces a bunch of things

00:17:56.920 | and no one looks at any of it.

00:17:58.120 | Like, that's not the great end state.

00:18:00.840 | But if we start seeing, like, oh, yeah,

00:18:02.160 | there's common patterns that people do

00:18:03.520 | that never require human involvement,

00:18:05.720 | just end-to-end just totally works

00:18:07.040 | without any intent-based information,

00:18:09.440 | sure, that can become fully agentic,

00:18:11.080 | and we'll learn what those tasks are pretty quickly,

00:18:13.680 | 'cause we have a lot of data.

00:18:14.840 | - Maybe add on to that.

00:18:16.080 | I think that if the answer is, like,

00:18:18.720 | fully agentic is Dev-in, I think, like, yes.

00:18:22.440 | The answer is this product should become fully agentic,

00:18:25.600 | and limited human interaction is the goal,

00:18:28.280 | is 100% the goal, and I think, honestly,

00:18:30.400 | of all usable products right now,

00:18:32.080 | I think we're the closest right now,

00:18:33.600 | of all usable products in an IDE.

00:18:35.360 | Now, let me caveat this by saying,

00:18:37.400 | I think there are lots of hard problems

00:18:38.840 | that have yet to be solved that we need to go out

00:18:40.800 | and solve to actually make this happen.

00:18:42.200 | Like, for instance, I think one of the most annoying parts

00:18:44.120 | about the product is the fact that you need to accept

00:18:46.640 | every command that kind of gets run.

00:18:48.520 | It's actually fairly annoying.

00:18:49.880 | I would like it to go out and run it.

00:18:51.240 | Unfortunately, me going out and running arbitrary binaries

00:18:53.720 | has some problems in that if it, like,

00:18:55.520 | RMRs my hard disk, I'm not gonna be--

00:18:56.920 | - It's actually a virus.

00:18:58.000 | - I'm not gonna actually--

00:18:58.840 | - It's a virus, yeah.

00:18:59.680 | - The hacker needs to be with you.

00:19:00.520 | Yeah, it does become a virus.

00:19:03.120 | I think this is solvable with, like, with complex systems.

00:19:06.440 | I think we love working on complex systems infrastructure.

00:19:09.400 | I think we'll solve it.

00:19:10.400 | Now, the simpler way to go about solving this is,

00:19:12.600 | don't run it on the user's machine and run it somewhere else

00:19:14.640 | because then if you bork that machine,

00:19:16.200 | you're kind of totally fine.

00:19:17.440 | Now, I think, though, maybe there's a little bit

00:19:19.840 | of trade-off of, like, running it locally versus remotely,

00:19:21.760 | and I think we might change our mind on this,

00:19:23.600 | but I think the goal for this is not for this

00:19:25.600 | to be the final state.

00:19:26.440 | I think the goal for this is, A, it's actually able to do

00:19:28.960 | very complex tasks with limited human interaction,

00:19:30.960 | but it needs to know when to actually go back to the human.

00:19:34.520 | Also, on top of that, compress every cycle

00:19:36.720 | that the agent is running.

00:19:37.800 | Right now, actually, I even feel like the product

00:19:39.680 | is too slow for me sometimes right now.

00:19:41.760 | Even with it running really fast,

00:19:43.920 | it's objectively pretty fast.

00:19:45.320 | I would still want it to be faster, right?

00:19:47.560 | So there is, like, systems work and probably modeling work

00:19:50.120 | that needs to happen there to make the product even faster

00:19:52.520 | on both the retrieval side and the generation side, right?

00:19:55.160 | And then finally speaking, I think another key piece here

00:19:57.600 | that's, like, really important is I actually think

00:19:59.960 | asking people to do things explicitly

00:20:01.840 | is probably going to be more of an anti-pattern

00:20:04.280 | if we can actually go and passively suggest

00:20:06.400 | the entire change for the user.

00:20:08.040 | So almost imagine, as the user is using the product,

00:20:10.880 | that we're gonna suggest the remainder of the PR

00:20:13.040 | without the user kind of, like, even asking us for it.

00:20:16.280 | I think this is sort of the beginning for it,

00:20:17.880 | but yeah, like, these are hard problems.

00:20:19.440 | I can't give a particular deadline for this.

00:20:21.480 | I think this is, like, a big step up than what we had,

00:20:23.760 | particularly in the past.

00:20:25.160 | But I think what Anshul said is 100% true,

00:20:28.000 | but the goal is for us to get better at this.

00:20:30.360 | - I mean, the remote execution thing is interesting.

00:20:32.320 | You wrote a post about the end of localhost.

00:20:34.760 | - Yeah.

00:20:35.600 | - And that's almost like, then we were kind of like,

00:20:37.240 | well, no, maybe we do need the internet

00:20:39.040 | and, like, people want to run things.

00:20:40.400 | But now it's like, okay, no, actually, I don't really care.

00:20:42.880 | Like, I want the model to do the thing.

00:20:44.160 | And if you were, like, you can do a task

00:20:46.120 | end to end, but it needs to run remotely,

00:20:48.160 | not on your computer.

00:20:49.000 | I'm sure most people would say, yeah.

00:20:50.360 | - No, I agree with that.

00:20:51.200 | - I'm cool with it.

00:20:52.040 | - I actually agree with it running remotely.

00:20:53.160 | That's not a security issue.

00:20:54.480 | I actually, I totally agree with you

00:20:55.800 | that it's possible that everything could run remotely.

00:20:59.320 | - That's how it is at most, like, big companies,

00:21:01.600 | like, Facebook, like, nobody runs things locally.

00:21:04.240 | - No one does.

00:21:05.080 | In fact, you connect to a remote machine.

00:21:05.920 | - It's essentially to the mainframe.

00:21:07.600 | - You're right on that.

00:21:08.440 | Maybe the one thing that I do think is kind of important

00:21:11.240 | for these systems that is more than just running remotely

00:21:13.400 | is basically, like, you know,

00:21:15.000 | when you look at these agents,

00:21:15.960 | there's kind of, like, a rollout of a trajectory.

00:21:18.120 | And I kind of want to roll this trajectory back, right?

00:21:20.280 | In some ways, I want, like, a snapshot of the system

00:21:22.200 | that I can, like, constantly checkpoint

00:21:23.720 | and move back and forth.

00:21:24.920 | And then also on top of that,

00:21:25.760 | I might want to do multiple rollouts of this.

00:21:28.160 | So basically, I think there needs to be a way

00:21:29.680 | to almost, like, move forward and move backwards the system.

00:21:33.120 | And whether that's locally or remotely,

00:21:34.880 | I think that's necessary.

00:21:35.960 | But every time, if you move the system forward,

00:21:37.760 | it, like, destroys your machine.

00:21:39.640 | It's probably going to be a hard system to kind of,

00:21:41.880 | or potentially destroys your machine.

00:21:43.280 | That's just not a workable solution.

00:21:44.760 | So I think the local versus remote,

00:21:46.520 | I think you still need to solve the problem of

00:21:48.440 | this thing is not going to destroy your machine

00:21:50.480 | on every execution, if that makes sense.

00:21:52.080 | - Yeah, yeah.

00:21:53.360 | There is a category of emerging infrastructure providers

00:21:55.800 | that are working on time travel VMs.

00:21:58.880 | - And if Verne's first episode on this podcast

00:22:01.120 | was an indication, we like infrastructure problems.

00:22:03.320 | - Yeah, okay.

00:22:04.160 | All right, oh, so you're going there.

00:22:05.000 | All right, okay.

00:22:05.840 | - Well, that's funny, right?

00:22:06.680 | It's like, when we first had you,

00:22:08.400 | you were doing so much on, like,

00:22:10.160 | actual model inference, optimization, all these things.

00:22:12.680 | And today, it's almost like--

00:22:14.120 | - It's Cloud, it's 4.0.

00:22:15.880 | - It's like, you know,

00:22:16.720 | people are, like, forgetting about the model.

00:22:18.720 | You know, and now it's all about

00:22:19.680 | at a higher level of extraction.

00:22:21.360 | - Yeah.

00:22:22.200 | So maybe I can say, like, a little bit about

00:22:23.840 | how our strategy on this has, like, evolved.

00:22:25.960 | Because it objectively has, right?

00:22:27.600 | I think I would be lying if I said it hasn't.

00:22:30.400 | The things, like, autocomplete and supercomplete

00:22:32.120 | that run on every keystroke are entirely, like,

00:22:34.120 | our own models.

00:22:35.080 | And by the way, that is still because properties,

00:22:37.240 | like FIM, fill-in-the-middle capabilities,

00:22:39.440 | are still quite bad with the current model.

00:22:41.520 | - Non-existent.

00:22:42.560 | - They're all, they're very bad.

00:22:44.000 | Non-existent, they're not good, actually, at it.

00:22:46.280 | - Because FIM is an actual, like,

00:22:47.920 | how you order the tokens.

00:22:49.360 | - It's how you order the tokens, actually, in some ways.

00:22:51.720 | And this is the, this is sort of,

00:22:53.120 | if you look at what these products have sort of become,

00:22:55.160 | and this is great, is a lot of the clods

00:22:57.120 | in the open AIs have focused on, kind of,

00:22:58.520 | the chat, like, assistant API,

00:23:00.160 | where it's, like, complete pieces of work,

00:23:01.760 | message, another complete piece of work.

00:23:03.320 | So multi-turn, kind of, back-and-forth systems.

00:23:06.000 | In fact, like, actually, even these systems

00:23:07.840 | are not that good at making point changes.

00:23:09.760 | When they make point changes,

00:23:10.680 | they kind of are, like, off here and there

00:23:12.240 | by a little bit.

00:23:13.280 | Because, yeah, when you, like,

00:23:14.800 | are doing multi-point, kind of, like, conversations,

00:23:17.160 | it's, you know, exact GIFs getting applied

00:23:19.600 | is not, like, even a perfect science, still yet.

00:23:21.800 | So we care about that.

00:23:23.240 | The second piece where we've actually, sort of,

00:23:25.440 | trained our models is actually on the retrieval system.

00:23:27.640 | And this is not even for embedding,

00:23:28.880 | but, like, actually being able to use high-powered LLMs

00:23:31.600 | to be able to do much higher-quality retrieval

00:23:34.080 | across the code base, right?

00:23:35.960 | So this is actually what Anshul said.

00:23:37.960 | For a lot of the systems, we do believe embeddings work,

00:23:40.400 | but for complex questions,

00:23:41.920 | we don't believe embeddings can encapsulate

00:23:43.960 | all the granularity of a particular query.

00:23:46.560 | Like, imagine I have a question on a code base of,

00:23:49.520 | find me all quadratic-time algorithms in this code base.

00:23:52.000 | Do we genuinely believe the embedding

00:23:54.160 | can encapsulate the fact that this function

00:23:55.760 | is a quadratic-time function?

00:23:57.240 | No, I don't think it does.

00:23:58.600 | So you are going to get

00:23:59.440 | extremely poor precision recall at this task.

00:24:01.640 | So we need to apply something a little more high-powered

00:24:03.920 | to actually go out and do that.

00:24:04.760 | So we've actually built, like, large distributed systems

00:24:07.480 | to actually go out and run these at scale,

00:24:09.400 | run custom models at scale across large code bases.

00:24:12.000 | So I think it's more a question of that.

00:24:13.720 | The planning models right now, undoubtedly,

00:24:16.120 | I think the CLODS and the OpenAIs have the best products.

00:24:18.920 | I think LLAMA4, depending on where it goes,

00:24:20.800 | it could be materially better.

00:24:22.200 | It's very clear that they're willing to invest

00:24:24.160 | a similar amount of compute

00:24:25.640 | as the OpenAIs and the Anthropics.

00:24:28.120 | So we'll see.

00:24:28.960 | I would be very happy if they got really good,

00:24:30.840 | but unclear so far.

00:24:32.360 | - Don't forget Grok.

00:24:33.400 | - Hey, dude, I think Grok is also possible.

00:24:35.240 | - Yeah. - Right?

00:24:36.080 | I think, don't doubt Elon, yeah.

00:24:38.240 | - Okay, so I didn't actually know,

00:24:40.000 | it's not obvious when I use Cascade.

00:24:41.800 | I should also mention that, you know,

00:24:42.920 | I was part of the preview, thanks for letting me in,

00:24:44.880 | and I've been maining Windsurf for a long time.

00:24:47.960 | It's not actually obvious.

00:24:48.800 | You don't make it obvious

00:24:49.620 | that you are running your own models.

00:24:50.800 | - Yeah. - I feel like you should,

00:24:52.060 | so that, like, I feel like it has more differentiation.

00:24:55.520 | Like, I only have exclusive access to your models

00:24:58.040 | via your IDE than having the dropdown that says

00:25:00.960 | Cloud and 4.0, 'cause I actually thought

00:25:02.760 | that was what you did.

00:25:03.680 | - No, so actually, the way it works

00:25:05.120 | is the high-level planning that is going on in the model

00:25:07.480 | is actually getting done with products like the Cloud,

00:25:09.860 | but the extremely fast retrieval,

00:25:12.160 | as well as the ability to, like, take the high-level plan

00:25:14.600 | and actually apply it to the code base

00:25:16.040 | is proprietary systems that are running internally.

00:25:18.160 | - And then the stuff that you said

00:25:19.720 | about embeddings not being enough,

00:25:21.940 | are you familiar with the concept of late interaction?

00:25:24.240 | - No, I actually have never--

00:25:25.200 | - Yeah, so this is Colbert, or the guy, Omar Khattab,

00:25:28.680 | from, I think, Stanford, has been promoting this a lot.

00:25:31.480 | It is basically what you've done.

00:25:32.560 | - Okay. (laughs)

00:25:34.280 | - Sort of embedding on retrieval rather than pre-embedding.

00:25:37.160 | - Okay, that's a--

00:25:38.000 | - In a very loose sense.

00:25:39.120 | - I think that sounds like a very good idea

00:25:41.080 | that is very similar to what we're doing.

00:25:42.640 | - Sounds like a very good idea.

00:25:43.700 | (laughs)

00:25:44.540 | I don't think we'd say that.

00:25:45.360 | - That's like the meme of Obama

00:25:46.800 | giving himself a medal right there.

00:25:48.200 | (laughs)

00:25:49.880 | - Well, I mean, there might be something to learn

00:25:52.680 | from contrasting the ideas and seeing where--

00:25:54.320 | - Absolutely.

00:25:55.160 | - Like, the subtle opinion differences.

00:25:56.380 | It's also been applied very effectively

00:25:58.240 | to vision understanding.

00:25:59.840 | Because vision models tend to just consume the whole image,

00:26:03.320 | if you are able to sort of focus on images

00:26:05.520 | based on the query, I think that can get you

00:26:08.320 | a lot of extra performance.

00:26:09.960 | - The basic idea of using compute

00:26:12.080 | in a distributed manner to do operations

00:26:14.880 | over a whole set of raw data rather than a materialized view

00:26:18.040 | is not anything new, right?

00:26:19.440 | I think it's just like, how does that look like for LLMs?

00:26:21.620 | - When I hear you say build large distributed systems,

00:26:24.320 | you have a very strange product strategy

00:26:26.420 | of going down to the individual developer

00:26:28.440 | but also to the large enterprise.

00:26:30.080 | Is it the same in front that serves everything?

00:26:31.940 | - I think the answer to that is yes.

00:26:33.840 | The answer to that is yes.

00:26:34.680 | And the only reason why for us the answer is yes.

00:26:37.440 | And to be honest, our company is a lot more complex

00:26:39.800 | than I think if we just wanted to serve the individual.

00:26:42.440 | And I'll tell you that because we don't really pay

00:26:45.000 | other providers to do things for our indexing.

00:26:47.600 | We don't pay other providers to do our serving

00:26:50.940 | of our own customer models, right?

00:26:52.960 | And I think that's a core competency within our company

00:26:55.120 | that we have decided to build.

00:26:56.080 | But that's also enabled us to go and make sure

00:26:58.640 | that when we're serving these products

00:27:00.160 | in an environment that works for these large enterprises,

00:27:02.360 | we're not going out and being like,

00:27:03.400 | we need to build this custom system for you guys.

00:27:05.440 | This is the same system that serves our entire user base.

00:27:08.480 | So that is a very unique decision we've taken as a company.

00:27:11.680 | And we admit that there are probably faster ways

00:27:14.400 | that we could have done this.

00:27:15.400 | - I was thinking, you know, when I was working with you

00:27:16.840 | for your enterprise piece, I was thinking like

00:27:18.240 | this philosophy of go slow to go fast,

00:27:20.180 | like build deliberately for the right level of abstraction

00:27:23.440 | that can serve the market that you really are going after.

00:27:26.200 | - Yeah, I mean, I would say like I was writing,

00:27:28.120 | when writing that piece, you're like looking back

00:27:29.920 | and reading it back, it sounds so like almost obvious

00:27:32.280 | and not all of those are really conscious decisions we made,

00:27:34.960 | like I'll be the first to admit that.

00:27:36.480 | But like, it does help, right?

00:27:37.440 | When we go to like an enterprise

00:27:39.160 | that has tens of thousands of developers,

00:27:41.120 | and they're like, oh, wow, like, you know,

00:27:42.320 | we have tens of thousands of developers

00:27:44.040 | and like, does your infrastructure work

00:27:45.640 | for tens of thousands of developers?

00:27:47.120 | We can turn around and be like,

00:27:48.040 | well, we have hundreds of thousands of developers

00:27:50.240 | or an individual plan that we're serving.

00:27:51.680 | Like, I think we'll be able to support you, right?

00:27:53.280 | So like, being able to do those things,

00:27:55.600 | like we started off by just like,

00:27:56.920 | let's give it to individuals, let's see what people like

00:27:58.800 | and what they don't like and learn.

00:28:00.440 | But then those become value propositions

00:28:01.760 | when we go to the enterprise.

00:28:02.800 | - And to recap, when you first came on the pod,

00:28:05.400 | it was like auto-completion is free

00:28:08.200 | and co-pilot was 10 bucks a month.

00:28:10.160 | And you said, look, what we care about

00:28:12.080 | is building things on top of code completion.

00:28:14.680 | How did you decide to like just not focus

00:28:17.360 | on like short-term kind of like growth monetization

00:28:19.680 | of like the individual developer and like build some of this

00:28:22.120 | because the alternative would have been,

00:28:23.880 | hey, all these people are using it.

00:28:25.600 | It's like, we're going to make this other

00:28:26.800 | like five bucks a month plan monetized.

00:28:29.760 | - I think this might be a little bit of like

00:28:32.200 | commercial instinct that the company has

00:28:34.640 | and unclear if the commercial instinct is right.

00:28:36.840 | I think that right now, optimizing for making money

00:28:40.320 | off of individual developers

00:28:42.000 | is probably the wrong actually strategy.

00:28:44.920 | Largely because I think individual developers

00:28:46.600 | can switch off of products like very quickly.

00:28:48.600 | And unless we have like a very large lead,

00:28:51.400 | trying to optimize for making a lot of profit

00:28:53.120 | off of individual developers,

00:28:54.880 | it's probably something that someone else

00:28:56.560 | could just vaporize very quickly.

00:28:57.920 | And then they move to another product.

00:29:00.360 | And I'm going to say this very honestly, right?

00:29:01.880 | Like when you use a product like Codium

00:29:03.600 | on the individual side,

00:29:05.360 | there's not much that prevents you

00:29:07.560 | to switch onto another product.

00:29:08.960 | I think that will change with time

00:29:10.120 | as the products get better and better and deeper and deeper.

00:29:12.600 | I constantly say this,

00:29:13.480 | like there's a book in business called like seven powers.

00:29:16.240 | And I think one of the powers that a business like ours

00:29:18.040 | need to have is like real switching costs.

00:29:20.080 | But like you first need something in the product

00:29:22.280 | that makes people switch on and stay on

00:29:24.200 | before you think about how do you make people switch off.

00:29:26.640 | And I think for us,

00:29:28.400 | we believe that there's probably much more differentiation

00:29:31.000 | we can derive in the enterprise

00:29:32.400 | by working with these large companies

00:29:34.000 | in a way that is like,

00:29:34.960 | that is interesting and scalable for them.

00:29:36.840 | Like I'll be maybe more concrete here.

00:29:39.080 | Individual developers are much more

00:29:40.880 | sort of tuned towards small price changes.

00:29:43.600 | They care a lot more, right?

00:29:44.640 | Like if our product is 10, 20 bucks a month

00:29:47.000 | instead of 50 or 100 bucks a month,

00:29:48.920 | that matters to them a lot.

00:29:50.200 | And for a large company

00:29:51.040 | where they're already spending billions of dollars

00:29:52.840 | on software, this is much less important.

00:29:54.480 | So you can actually solve maybe deeper problems

00:29:56.520 | for them and you can actually kind of provide

00:29:59.520 | more differentiation on that angle.

00:30:01.240 | Whereas I think individual developers

00:30:03.160 | could be churny as long as we don't have the best product.

00:30:05.160 | So focus on being the best product,

00:30:06.720 | not trying to like take price

00:30:07.880 | and make a lot of money off of people.

00:30:09.600 | And I don't think we will, for the foreseeable future,

00:30:12.600 | try to be a company that tries to make a lot of money

00:30:14.600 | off individual developers.

00:30:15.880 | - I mean, that makes sense.

00:30:17.040 | So why $10 a month for Windsurf?

00:30:19.280 | - Why $10?

00:30:20.120 | - $10 a month was actually the pro plan.

00:30:21.960 | So we launched our individual pro plan

00:30:23.800 | before Windsurf existed.

00:30:25.320 | Because I think there's, let's all chat.

00:30:27.200 | We all said to be financially responsible.

00:30:28.760 | (laughing)

00:30:30.400 | - Yeah, yeah, we can't run out of money.

00:30:32.880 | - Maybe it's $150 million, you're good.

00:30:35.160 | - There's a lot of things

00:30:36.000 | because of our infrastructure background

00:30:36.920 | we can give for essentially free,

00:30:39.000 | like unlimited auto-complete,

00:30:40.840 | unlimited chat on our faster models.

00:30:44.120 | We give a lot of things actually out for free.

00:30:46.240 | But yeah, when we started doing things

00:30:47.440 | like the super-completes

00:30:48.480 | and really large amounts of indexing

00:30:50.480 | and all these things, there is real cogs here.

00:30:53.200 | We can't ignore that.

00:30:54.520 | And so we just created $10 a month pro plan

00:30:57.600 | mostly just to cover the cost.

00:30:58.960 | We're not really operating, I think,

00:31:00.320 | on much of a margin there either.

00:31:01.640 | But like, okay, just to cover us there.

00:31:04.080 | So for Windsurf, it just ended up being the same thing.

00:31:07.000 | And everyone who downloads Windsurf in the first,

00:31:09.720 | I forget, a couple of weeks,

00:31:10.840 | they get two weeks for free.

00:31:12.040 | Let's just have people try it out,

00:31:13.200 | let us know what they like, what they don't like.

00:31:14.440 | And that's how we've always operated.

00:31:16.360 | - I've talked to a lot of CTOs in the Fortune 100

00:31:19.520 | where most of the engineers they have,

00:31:21.560 | they don't really do much anyway.

00:31:23.080 | The problem is not that the developer costs 200K

00:31:26.560 | and you're saving 8K.

00:31:28.120 | It's like that developer should not be paid 200K.

00:31:30.600 | But that's kind of like the base price.

00:31:33.280 | But then you have developers getting paid 200K,

00:31:35.400 | they should be paid 500K.

00:31:36.960 | So it's almost like you're averaging out the price

00:31:40.160 | because most people are actually not that productive anyway.

00:31:42.400 | So if you make them 20% more productive,

00:31:44.120 | they're still not very productive.

00:31:45.560 | And I don't know in the future,

00:31:47.440 | is it that the junior developer salary is like 50K?

00:31:52.160 | And it's like the bottom of the end

00:31:53.640 | gets kind of like squeezed out

00:31:54.760 | and then the top end gets squeezed up?

00:31:56.840 | - Yeah, maybe Alessio, one thing that I think about a lot,

00:31:58.960 | because I do think about this, the per seat, anything,

00:32:01.520 | all of this stuff, I think about a good deal.

00:32:03.280 | Let's take a product like Office 365.

00:32:05.280 | I will say a lawyer at Codium

00:32:06.800 | uses Microsoft Word way more than I do.

00:32:08.760 | I'm still footing the same bill.

00:32:10.360 | But the amount of value that he's driving

00:32:11.680 | from Office 365 is probably tens of thousands of dollars.

00:32:15.400 | By the way, everyone, Google Docs, great product.

00:32:18.080 | Microsoft Word is a crazy product.

00:32:20.480 | It made it so that the moment you review anything

00:32:22.560 | in Microsoft Word, the only way you can review it

00:32:24.320 | is with other people in Microsoft Word.

00:32:26.080 | It's like this virus that penetrates everything.

00:32:28.160 | And it not only penetrates it within the company,

00:32:29.840 | it penetrates it across company too.

00:32:31.800 | The amount of value it's driving is way higher for him.

00:32:34.280 | So for these kinds of products,

00:32:35.800 | there's always going to be, for these kinds of products,

00:32:37.880 | this variance between who gets value from these products.

00:32:41.120 | And you're right, it's almost like a blended.

00:32:42.840 | 'Cause you're actually totally right.

00:32:44.160 | Probably this company should be paying that one developer

00:32:46.280 | maybe like four times as much.

00:32:47.720 | But in a weird way, software is this team activity enough

00:32:50.640 | that there's a bunch of blended outcomes.

00:32:52.440 | But hey, 20% of the four times, and there are four people,

00:32:55.560 | is still going to cover the cost

00:32:56.800 | across the four individuals, right?

00:32:58.080 | And that's how roughly these products

00:33:00.080 | kind of get priced out.

00:33:01.680 | - I mean, more than about pricing,

00:33:02.960 | this is about the future of the software engineer.

00:33:06.280 | - We could be very wrong also.

00:33:07.600 | - Yeah, I think nobody knows.

00:33:10.040 | - Reserve the right to be incredibly off.

00:33:11.880 | - Yeah.

00:33:12.720 | - I mean, business model does impact the product,

00:33:14.360 | product does impact the user experience.

00:33:16.520 | So it's all of a kind, I don't mind.

00:33:18.360 | We are, we do, are as concerned about the business of tech

00:33:21.360 | as the tech itself.

00:33:22.520 | - That's cool.

00:33:23.360 | - Speaking of which, there's other listener questions.

00:33:25.360 | Shout out to Daniel Infeld,

00:33:26.640 | who's pretty active in our Discord,

00:33:27.880 | just asking all these things.

00:33:29.720 | Multi-agent, very, very hot and popular,

00:33:32.240 | especially from like the Microsoft research point of view.

00:33:34.760 | Have you made any explorations there?

00:33:37.360 | - I think we have.

00:33:38.680 | I don't think we've called it a multi-agent,

00:33:40.800 | which is more so like once you,

00:33:42.640 | this notion of having many trajectories

00:33:44.680 | that you can spawn off,

00:33:46.200 | that kind of like validate

00:33:47.680 | sort of some different hypotheses

00:33:49.040 | and you can kind of pick the most interesting one.

00:33:50.840 | This is stuff that we've actually analyzed

00:33:52.760 | internally at the company.

00:33:53.600 | By the way, the reason why

00:33:54.840 | we have not put these things in,

00:33:56.520 | actually, is partially because we can't go out

00:33:58.520 | and execute some random stuff in peril in the meantime.

00:34:01.560 | In the meantime--

00:34:02.400 | - Because of the side effects.

00:34:03.240 | - Because of the side effects, right?

00:34:04.320 | So there are some things that are a little bit

00:34:06.200 | dependent on us unlocking

00:34:08.360 | more and more functionality internally.

00:34:09.920 | And then the other thing is, in the short term,

00:34:11.920 | I think there is like also a latency component.

00:34:13.800 | And I think all of these things can kind of be solved.

00:34:15.840 | I actually believe all of these things are solvable from.

00:34:17.560 | They're not unsolvable from.

00:34:18.800 | And if you want to run all of them in parallel,

00:34:20.640 | you probably don't want end machines to go out and do it.

00:34:22.520 | I think that's unnecessary,

00:34:23.800 | especially if most of them are I/O bound kind of operations

00:34:26.560 | where all you're doing is reading a little bit of data

00:34:28.680 | and writing out a little bit of data.

00:34:29.760 | It's not extremely compute intensive.

00:34:31.600 | I think that it's a good idea

00:34:33.240 | and probably something we will pursue

00:34:35.240 | and is going to be in the product.

00:34:36.720 | - I'm still processing what you just said

00:34:38.120 | about things being I/O bound.

00:34:39.600 | So you can, so for a certain class of concurrency,

00:34:42.080 | you can actually just run it all in one machine.

00:34:43.640 | - Yeah, why not?

00:34:44.480 | Because if you look at the changes that are made, right?

00:34:47.200 | And for some of these,

00:34:48.160 | it's writing out like what, a couple of thousand bytes?

00:34:50.320 | Maybe like tens of thousands of bytes on every,

00:34:52.680 | it's not a lot, very small.

00:34:54.480 | - What's next for Cascade or Windsurf?

00:34:56.760 | - Oh, there's a lot.

00:34:58.000 | I don't know.

00:34:58.840 | We did an internal poll and we were just like,

00:35:00.920 | are you more excited about this launcher

00:35:02.480 | or the launch that's happening in a month?

00:35:04.360 | Or like what we're going to come out with in a month?

00:35:06.000 | And it was like almost uniformly in a month.

00:35:08.040 | I think like, you know, there's some like obvious ones.

00:35:10.080 | I don't know how much for you want to say.

00:35:11.440 | I don't want to speak of,

00:35:13.920 | but I think you'd look at all the same axes of the system,

00:35:17.160 | right?

00:35:18.000 | Like how can we improve the knowledge retrieval?

00:35:19.200 | Like we'll always keep on figuring out

00:35:20.520 | how to improve knowledge retrieval.

00:35:21.480 | In the, in our launch video,

00:35:22.920 | we even showed some of like the early explorations

00:35:24.840 | we have about looking into other data sources.

00:35:27.280 | That might not be the coolest thing

00:35:28.280 | to the individual developer building a zero to one app,

00:35:30.600 | but you can really believe that like

00:35:32.240 | the enterprise customers really think

00:35:33.720 | that that's very cool, right?

00:35:35.120 | I think on the tool side,

00:35:36.880 | I think there's a whole lot more that we can do.

00:35:38.240 | I mean, of course, I mean,

00:35:39.440 | everyone's talked about not just suggesting

00:35:41.280 | the terminal command, but actually executing them.

00:35:42.800 | Like, I think that's going to be huge.

00:35:44.640 | Unlock, you look at that,

00:35:46.000 | the actions that people are taking, right?

00:35:47.720 | Like the human actions,

00:35:48.560 | the trajectories that we can build,

00:35:49.600 | like how can we make that even more detailed?

00:35:51.640 | And I think all of those things,

00:35:52.880 | and you make some like even a cleaner UI,

00:35:55.840 | like the idea of looking at future trajectories,

00:35:58.680 | trying a few different things,

00:35:59.640 | and like suggesting potential next actions to be taken.

00:36:02.920 | That doesn't really exist yet,

00:36:04.200 | but like it's pretty obvious,

00:36:05.720 | I think, how that would look like, right?

00:36:06.840 | You open up Cascade,

00:36:08.360 | and instead of like starting typing,

00:36:09.800 | it's just like,

00:36:10.640 | here's a bunch of things that we want to do.

00:36:12.160 | We kind of joke that's like Clippy's coming back,

00:36:14.000 | but like maybe now's the time for Clippy to really shine,

00:36:16.400 | right?

00:36:17.240 | So I think there's a lot of ways that we can take this,

00:36:19.120 | which I think is like the very exciting part.

00:36:21.000 | We're calling each of our launches waves, I believe,

00:36:23.000 | because we want to really double down on the aquatic themes.

00:36:25.800 | - Oh yeah, does someone actually windsurf for the company?

00:36:27.920 | Is that?

00:36:28.760 | - I don't think.

00:36:29.600 | (laughing)

00:36:30.440 | - We're living out our dream of being cool enough

00:36:32.720 | to windsurf through the rocks.

00:36:33.560 | - Yeah, I don't think we can.

00:36:35.320 | - Yeah, all right.

00:36:36.160 | (laughing)

00:36:37.000 | - That's something we learned,

00:36:37.960 | 'cause I don't think any of us are windsurfers.

00:36:39.720 | Like in our launch video,

00:36:40.560 | we have someone like using windsurf on a windsurf.

00:36:43.040 | That was like--

00:36:43.880 | - You saw that?

00:36:44.720 | - You saw that in the beginning of the video.

00:36:45.540 | Someone has a computer.

00:36:46.380 | And we didn't realize like now apparently

00:36:47.840 | is like the time of the year

00:36:48.720 | where there's like not enough wind to windsurf.

00:36:50.960 | So we were trying to figure out how to do this like,

00:36:52.760 | you know, launch video with windsurf on the windsurf.

00:36:54.920 | Every windsurfer we talked to were like,

00:36:56.400 | yeah, it's not possible.

00:36:57.240 | And there was like one crazy guy who was like,

00:36:59.040 | yeah, I think we can do this.

00:37:00.200 | And we made it happen.

00:37:01.800 | - Oh, okay.

00:37:03.120 | - That's funny.

00:37:04.080 | Is there anything that you want feedback on?

00:37:05.640 | Like maybe there's a fork in the road,

00:37:07.120 | you want feedback,

00:37:07.960 | you want people to respond to this podcast

00:37:10.320 | and tell you what they want?

00:37:11.840 | - Yeah, I think there's a lot of things

00:37:14.160 | that I think could be more polished about the product

00:37:16.640 | that we'd like to improve.

00:37:18.760 | Lots of different environments

00:37:19.960 | that we're going to improve performance on.

00:37:21.380 | And I think we would love to hear from folks

00:37:24.360 | across the gamut.

00:37:25.200 | Like, hey, like, if you have this environment,

00:37:26.840 | you use Windows and X version,

00:37:28.560 | it didn't work or this language.

00:37:30.000 | - Oh yeah.

00:37:30.840 | - It was like very poor.

00:37:31.660 | I think we would like to hear it.

00:37:33.280 | - Yeah, I gave Prep and Kevin a lot of shit

00:37:35.160 | for my Python issues.

00:37:36.480 | - Yeah, yeah, yeah.

00:37:37.320 | And I think there's a lot to kind of improve

00:37:39.440 | on the environment side.

00:37:40.280 | I think like, for instance,

00:37:41.240 | even just a dumb example,

00:37:42.440 | and I think sort of this was a common one.

00:37:44.880 | It's like, yeah, like the virtual environment,

00:37:46.480 | where's the terminal running?

00:37:47.500 | What is all this stuff?

00:37:48.520 | These are all basic things that like,

00:37:50.160 | to be honest, this is not rocket science,

00:37:51.720 | but we need to just fix it, right?

00:37:53.500 | We need to fix it.

00:37:54.340 | So we would love to hear like all the feedback

00:37:56.480 | from the product.

00:37:57.300 | Like, was it too slow?

00:37:58.160 | Where was it too slow?

00:37:59.280 | What kind of environments could it work way more in?

00:38:01.400 | There's a lot of things that we don't know.

00:38:02.760 | We, luckily, we're daily users of the product internally.

00:38:06.320 | So we're getting a lot of feedback inside,

00:38:08.120 | but I will say like,

00:38:08.960 | there's a little bit of Silicon Valley-ism

00:38:10.320 | in that a lot of us develop on Mac.

00:38:12.040 | A lot of people, once again,

00:38:13.440 | over 80% of developers are on Windows.

00:38:15.680 | So yeah, there's a lot to learn

00:38:17.060 | and probably a lot of improvements down the line.

00:38:18.720 | - Are you personally tempted,

00:38:19.560 | as your CEO of the company,

00:38:21.320 | to switch to Windows just to feel something?

00:38:24.560 | (laughing)

00:38:25.400 | - Feel the Windows.

00:38:26.400 | - You know what?

00:38:27.240 | - To feel the pain.

00:38:28.060 | - You know what?

00:38:28.900 | Maybe I should.

00:38:29.740 | Actually, I think I will.

00:38:31.120 | - Your customers, everyone says it's 80, 90% are on Windows.

00:38:35.820 | You live in Windows.

00:38:36.760 | You will never not see something that missed.

00:38:40.800 | - So I think in the beginning,

00:38:41.840 | part of the reason why we were hesitant to do that

00:38:44.260 | was a lot of our architectural decisions

00:38:46.720 | to work on across every IDE

00:38:48.200 | was because we built a platform-agnostic way

00:38:50.880 | of running the system on the user's local machine

00:38:53.520 | that was only buildable,

00:38:54.920 | easily buildable on dev containers

00:38:57.160 | that lived on a particular type of platform.

00:38:59.340 | So Mac was nice for that.

00:39:00.680 | But now, there's not really an excuse

00:39:02.600 | if I can also make changes to the UI and stuff like that.

00:39:05.800 | And yeah, WSL also exists.

00:39:07.400 | That's actually something that we need to add to the product.

00:39:09.400 | That's how early it is that we have not actually added that.

00:39:11.880 | - We don't have remote.

00:39:13.120 | - Anything else about Codium at large?

00:39:16.360 | You still have your core business of the enterprise Codium.

00:39:20.760 | Anything moving there

00:39:21.720 | or anything that people should know about?

00:39:23.360 | - Anshul, you want to take that?

00:39:24.200 | - I think a lot are still moving there.

00:39:26.120 | I think it would be a little bit

00:39:27.920 | very kind of egotistical of us to be like,

00:39:29.360 | oh, we have Windsurf now.

00:39:30.400 | All of our enterprise customers

00:39:31.480 | are going to switch to Windsurf and this is the only,

00:39:32.900 | like, no, we still support the other IDE.

00:39:34.600 | - You just talked about your Java guys loving JetBrains.

00:39:37.560 | They're never going to leave JetBrains.

00:39:38.400 | - They're not.

00:39:39.240 | Like, I mean, forget JetBrains.

00:39:40.560 | There's still tons and tons of enterprise people on Eclipse.

00:39:43.200 | Like, we're still the only code assistant

00:39:45.280 | that has an extension in Eclipse.

00:39:46.440 | That's still true years in, right?

00:39:48.640 | And but like, that's because that's our enterprise customers.

00:39:50.640 | And the way that we always think about it is like,

00:39:52.540 | how do we still maximize the value of AI

00:39:54.620 | for every developer?

00:39:55.460 | And I don't think that part of who we are

00:39:57.580 | has changed since the beginning, right?

00:39:59.500 | And there's a lot of like,

00:40:00.340 | meeting the developers where they are.

00:40:01.540 | So I think on the enterprise side,

00:40:03.140 | we're still pretty invested in doing that.

00:40:05.220 | We have like a team of engineers dedicated

00:40:07.340 | just to making enterprise successful

00:40:08.700 | and thinking about the enterprise problems.

00:40:10.460 | But really, if we think about it

00:40:11.440 | from the really macro perspective,

00:40:13.220 | it's like, if we can solve all the enterprise problems

00:40:15.620 | for enterprise, and we have products

00:40:18.100 | that developers themselves just truly, truly love,

00:40:21.400 | then we're solving the problem from both sides.

00:40:23.300 | And I think it's one of those things where,

00:40:25.100 | I think when we started working with the enterprise

00:40:26.900 | and we started building like dev tools, right?

00:40:28.340 | We started as an infrastructure company,

00:40:29.420 | now we're building dev tools for developers.

00:40:32.940 | You really quickly understand and realize just

00:40:35.580 | how much developers loving the tool

00:40:37.860 | make us successful in an enterprise.

00:40:39.740 | There's a lot of enterprise software that developers hate.

00:40:41.780 | - I want to draw this flywheel.

00:40:42.900 | - But like, we're giving a tool for people

00:40:45.420 | where they're doing their most important work.

00:40:47.500 | They have to love it.

00:40:48.340 | And it's not like, we're like trying to convince,

00:40:50.980 | the executives at this company

00:40:52.300 | also ask their developers a lot, do you love this?

00:40:54.340 | Like, that is like, almost always a key aspect

00:40:56.860 | of whether or not Codium is accepted or not,

00:40:58.300 | like into the organization.

00:41:00.060 | I don't think we go from zero to 10 million ARR

00:41:02.580 | in less than a year in an enterprise product

00:41:04.420 | if we don't have a product that developers love.

00:41:06.180 | So I think that's why we're just,

00:41:07.940 | the IDE is more of a developer love kind of play.

00:41:10.500 | It will eventually make it to the enterprise.

00:41:12.260 | We still solve the enterprise problems.

00:41:13.500 | And again, we could be completely wrong about this,

00:41:16.020 | but we hope we're solving the right problems.

00:41:17.580 | - It's interesting, I asked you this

00:41:18.940 | before we started rolling, but like, it's the same team.

00:41:21.740 | That's the same engine team.

00:41:22.820 | Like I, in any normal company or like, you know,

00:41:26.340 | my normal mental model of company construction,

00:41:28.420 | if you were to have like effectively two products like this,

00:41:31.340 | like you would have two different teams

00:41:32.620 | serving two different needs, but it's the same team.

00:41:34.900 | - Yeah, I think one of the things

00:41:36.220 | that's maybe unique about our company is like,

00:41:38.300 | this has not been one company the whole time, right?

00:41:41.140 | Like we were first, like this GPU virtualization company

00:41:43.940 | pivoted to this.

00:41:45.020 | And then after that, we're making some changes.

00:41:46.860 | And like, I think there's like a versatility of the company

00:41:49.980 | and like this ability to move where we think the instinct,

00:41:53.700 | we have this instinct where,

00:41:54.740 | and by the way, the instinct could be wrong,

00:41:56.020 | but if we smell something, we're going to move fast.

00:41:58.300 | And I think it's more a testament

00:41:59.700 | to I think the engineering team rather than any one of us.

00:42:02.500 | - I'm sure you had December 19th, 2022.

00:42:06.180 | You had one of our guests post,

00:42:08.460 | "What building Copilot for Rx really takes?"

00:42:11.300 | Estimate inference to figure out latency quality.

00:42:14.060 | Build first party instead of using third party as API.

00:42:17.020 | Figure out real time because ChadGBT and Dali,

00:42:20.300 | Dali, the RIP, are too slow.

00:42:23.220 | Optimize prompt because context window's limited,

00:42:26.020 | which is maybe not that true anymore.

00:42:27.860 | And then merge model outputs with the UX

00:42:30.620 | to make the product more intuitive.

00:42:32.540 | Is there anything you would add?

00:42:34.500 | - I'd give myself like a B minus on that.

00:42:36.340 | - Yeah, no, it's pretty good.

00:42:37.180 | - So some parts of that are accurate.

00:42:38.540 | Even like the context, like the one that you called out,

00:42:39.940 | like yeah, models have like larger context.

00:42:41.820 | Like so now that's absolutely true.

00:42:43.220 | Like it's grown a lot,

00:42:44.100 | but look at like an enterprise code base.

00:42:45.900 | They have like tens of millions of lines of code.

00:42:48.460 | That's hundreds of billions of tokens.

00:42:50.180 | - Never going to change.

00:42:51.740 | - Still being really good at like being able

00:42:53.460 | to piece together this like distributed knowledge

00:42:55.020 | for what's important.

00:42:55.860 | So I think like there are figures there

00:42:57.020 | that I think are still pretty accurate.

00:42:58.740 | There's probably some that are less so.

00:43:00.940 | - First party versus third party.

00:43:01.780 | - First party versus third party.

00:43:02.620 | I think we're like, we're wrong there.

00:43:03.460 | - We just got it wrong.

00:43:04.300 | - I think I would nuance that to be like,

00:43:06.100 | there are certain things

00:43:06.940 | that it's really important to first party.

00:43:08.140 | Like, you know, autocomplete,

00:43:09.300 | you have a really specific application

00:43:11.220 | that you can't just prompt engineer your way out of

00:43:13.500 | or just maybe even like fine tune afterwards.

00:43:15.580 | Like you just can't do that.

00:43:16.820 | I think there's truth there,

00:43:17.700 | but like, let's also be realistic.

00:43:18.980 | Like the stuff that's coming out

00:43:20.100 | for the third model providers,

00:43:21.340 | like Cascade and Windsurf would not have been possible

00:43:24.000 | if it wasn't for the rapid improvements

00:43:26.220 | with 4.0 and 3.5.

00:43:27.620 | Like that just wouldn't have been possible.

00:43:29.140 | So I'll give myself a B minus.

00:43:31.020 | I'll say I passed, but yes, two hours, two years later.

00:43:33.980 | - Just to be clear, we're not grading.

00:43:36.060 | It's more of a, what would you, you know.

00:43:38.940 | - Like where are they now?

00:43:39.780 | - What would you have added?

00:43:40.900 | What would you like?

00:43:41.860 | - Yeah, I mean like that first post, right?

00:43:43.620 | Like that was when we had literally,

00:43:45.300 | I think that was like a few weeks

00:43:46.780 | after we had launched Codium.

00:43:48.060 | I think that's like, you know,

00:43:48.900 | Swiggs and I were talking like maybe we can write this

00:43:51.060 | 'cause we're like one of the first products

00:43:52.180 | that people can actually use with AI.

00:43:53.260 | That's cool.

00:43:54.100 | - I specifically like the Co-Pilot for X thing

00:43:56.460 | 'cause everyone, it is so hot.

00:43:57.780 | Like everyone wanted to do Co-Pilot.

00:43:58.620 | - Everyone was just like, you know,

00:43:59.900 | chat GPT, that's all that was.

00:44:01.820 | So, but I think like, you know,

00:44:03.340 | that we didn't have an enterprise product.

00:44:04.900 | I don't even think we were like necessarily thinking

00:44:06.620 | of an enterprise product at that point, right?

00:44:08.420 | So like all of the learnings that like, you know,

00:44:10.980 | we've had from the enterprise perspective,

00:44:12.740 | which is why I loved coming back

00:44:13.980 | for like a third time now on the blog.

00:44:16.420 | Some of those, I think we kind of like figured,

00:44:18.780 | some of those we just honestly walked backwards into.

00:44:21.820 | Had to get lucky a lot of the ways.

00:44:23.220 | Like we had many, we just did a lot.

00:44:26.060 | Like there's so many of like opportunities

00:44:27.980 | and deals that we had that we like lost

00:44:29.580 | for a variety of reasons that we had to like learn from.

00:44:31.940 | There's just so much more to add

00:44:33.100 | that there's no way I would have gotten that right in 2022.

00:44:35.780 | - Can I mention one thing that I think is,

00:44:37.780 | hopefully this is not very controversial,

00:44:39.900 | but it's like true about our engineering team as a whole.

00:44:42.100 | I don't think most of us got much value from ChaiGBT.

00:44:45.180 | Largely because I think the problem was,

00:44:47.020 | and this is maybe a little bit of a different thing.

00:44:49.020 | It's like a lot of the engineers at the company

00:44:51.020 | who have been writing software for like over eight years.

00:44:53.460 | And this is not to say they know everything

00:44:54.740 | that ChaiGBT knows, they don't.

00:44:56.500 | They'd already gotten good enough

00:44:58.780 | at searching for Stack Overflow.

00:45:00.980 | Invested a lot in searching code base, right?

00:45:04.100 | They can very quickly grab through the code.

00:45:06.100 | Incredibly fast, like every tool.

00:45:07.620 | And they've spent like eight years mastering that skill.

00:45:10.260 | And ChaiGBT being this thing on the side

00:45:12.300 | that you need to provide a lot of context to,

00:45:14.100 | we were not able to actually get,

00:45:16.060 | like my co-founder just basically never used ChaiGBT at all.

00:45:18.820 | Literally never did.

00:45:19.940 | And because of that, probably at the time,

00:45:21.940 | one of our incorrect sort of assumptions was probably that,

00:45:25.100 | hey, like a lot of these passive systems need to get good

00:45:27.020 | because they're always there

00:45:27.860 | and these active systems are going to be behind.

00:45:29.660 | I think actually Cascade was the big thing.

00:45:32.140 | Is there a company where everyone is now using it?

00:45:34.020 | Literally everyone.

00:45:34.980 | Biggest skeptics.

00:45:35.820 | And we have a lot of people at the company

00:45:36.900 | that are skeptical of AI.

00:45:38.420 | I think this is actually important.

00:45:39.260 | Don't hire them.

00:45:40.140 | No, I think here's the important thing.

00:45:42.260 | Those people that were skeptical about AI

00:45:44.100 | previously worked in autonomous vehicles.

00:45:45.940 | These are not crypto people.

00:45:47.180 | These are people that care about technology

00:45:49.140 | and want to work on the future.

00:45:50.380 | Their bar for good is just very high.

00:45:52.620 | They will not form a cult of this is awesome.

00:45:55.580 | This is going to change the world.

00:45:56.500 | They were not going to be the kind of people on Twitter

00:45:58.100 | that are like, yeah, this changes everything.

00:46:00.860 | Like software as we know it is dead.

00:46:02.380 | No, there are people that are going to be incredibly honest.

00:46:04.700 | And we know if we hit the bar that is good for them,

00:46:07.420 | we found something special.

00:46:08.660 | And I think at that time,

00:46:10.220 | we probably had a lot of sentiment like that.

00:46:12.260 | That has changed a lot now.

00:46:13.660 | And I think it's actually important that you have

00:46:15.660 | believers that are incredibly future looking

00:46:18.220 | and people that kind of reign it in.

00:46:20.100 | Because otherwise you just have,

00:46:21.500 | you know, this is like autonomous vehicles.

00:46:23.060 | You have a very discreet problem.

00:46:25.340 | People are just working in a vacuum.

00:46:27.140 | And there's no signal to kind of bring you down to reality.

00:46:29.740 | But you have no good way to kill ideas.

00:46:31.900 | And there are a lot of ideas we're going to come up with

00:46:33.580 | that are just terrible ideas.

00:46:34.980 | But we need to come up with terrible ideas.

00:46:36.340 | Otherwise, like how does anything good come out?

00:46:37.980 | And I don't want to call these skeptics.

00:46:39.220 | Skeptics suggest that they don't know.

00:46:40.700 | They're realists.

00:46:41.740 | They're the type of people that when they see

00:46:43.340 | waitlist on a product online,

00:46:44.900 | they just will not believe it.

00:46:45.780 | They will not think about it at all.

00:46:47.140 | - Kudos for launching without a waitlist.

00:46:48.980 | - Yeah, yeah.

00:46:49.820 | By the way, we will never launch with a waitlist.

00:46:51.580 | We will never launch with a waitlist.

00:46:52.620 | That's the thing at the company.

00:46:53.700 | We'd much rather be a company

00:46:54.780 | that's considered the boring company

00:46:56.460 | than a company that launches once in a while

00:46:58.580 | and like, hopefully it's good.

00:46:59.420 | - My joke is generative AI has gotten really good

00:47:02.980 | at generating waitlists.

00:47:04.140 | - Yeah, absolutely.

00:47:05.380 | Also, just to clarify,

00:47:06.420 | both of us used to work in autonomous vehicles,

00:47:07.980 | so it doesn't come across as--

00:47:09.220 | - Oh, yeah, yeah.

00:47:10.060 | - Autonomous driving.

00:47:10.900 | - Autonomous vehicles.

00:47:12.100 | No, we love that technology.

00:47:13.460 | - We love it.

00:47:14.300 | Like, I love hard technology problems.

00:47:15.660 | That's what I live for.

00:47:16.820 | - Amazing.

00:47:18.220 | Just push back on the first party thing.

00:47:19.700 | I accept that the large model labs

00:47:22.620 | have just like done a lot of work for you

00:47:24.460 | that you didn't need to duplicate.

00:47:27.060 | But you now are sitting on so much proprietary data

00:47:30.860 | that it may be worth training on the trajectories

00:47:34.700 | that you're collecting.

00:47:35.700 | So maybe it's a pendulum back to first party.

00:47:38.220 | - Yeah, I mean, I think like,

00:47:39.540 | I mean, we've been pretty clear

00:47:40.580 | from like a security posture perspective.

00:47:42.220 | Like, I think there's like, both like, you know,

00:47:44.260 | customer trust and like--

00:47:46.100 | - I mean, I kind of want, like, let me opt out.

00:47:47.420 | - I think that there is signals that we do get

00:47:49.500 | from our users that we can utilize.

00:47:51.020 | Like, there's a lot of preference information

00:47:52.460 | that we get, for example.

00:47:53.300 | - Which is effectively what you're saying,

00:47:54.420 | of like, our trajectories--

00:47:55.780 | - Go ahead. - Our trajectory's good.

00:47:56.700 | We, like, I will say this,

00:47:58.220 | the super complete product that we have

00:47:59.740 | has gotten materially better

00:48:01.220 | because of us not only using synthetic data,

00:48:03.700 | but also getting the preference data from our users

00:48:06.220 | of like, hey, given these set of trajectories,

00:48:08.580 | here's actually what a good outcome is.

00:48:10.180 | And in fact, one of the really beautiful parts

00:48:12.100 | about our product that is very different

00:48:14.140 | than a chat GPT is,

00:48:15.220 | we can not only see if the acceptance happened,

00:48:17.380 | but if something more than the acceptance happened,

00:48:18.980 | and it happened even more than that, right?

00:48:20.860 | Like, let's say you accepted it,

00:48:22.220 | but then after accepting it,

00:48:23.260 | you deleted three or four items in there.

00:48:25.740 | We can see that.

00:48:26.780 | So that actually lets us get

00:48:28.060 | to even better than acceptance as a metric,

00:48:31.940 | because we're in the ultimate work output of the developer.

00:48:35.060 | - It's the preference between the acceptance

00:48:36.580 | and what actually happened.

00:48:37.660 | If you can actually get ground truth

00:48:39.020 | of what actually happened,

00:48:40.140 | this is the beauty of being an ID,

00:48:41.660 | then like, yeah, you get a lot, a lot of information there.

00:48:43.580 | So that's data that's helpful.

00:48:44.420 | - Did you have this with the extension,

00:48:45.380 | or is this pure Windsurf?

00:48:47.060 | - We had this with the extension, yes.

00:48:47.940 | - Yeah, okay, all right. - Yes.

00:48:49.140 | - The Windsurf just gives you more of the ID.

00:48:50.620 | - Yes.

00:48:51.460 | So that means you can also start getting more information.

00:48:54.100 | Like, for instance, the basic thing that Anshul said,

00:48:56.300 | we can see if like, a file explorer was opened.

00:48:58.740 | It's actually just a piece of information

00:49:00.260 | we just cannot see previously.

00:49:01.860 | - Sure.

00:49:02.700 | A lot of intent in there, a lot of intent.

00:49:04.980 | - Second one. - Oh boy.

00:49:06.020 | - How to make AI UX your mode.

00:49:07.860 | - Oh man, isn't that funny that we now created

00:49:10.540 | like, the full UX experience in an ID?

00:49:12.180 | I think that one, that one is pretty accurate.

00:49:14.300 | - That one's an A?

00:49:15.140 | - I think that one, I'll give myself,

00:49:16.260 | I think like, we were doing that within the extensions.

00:49:18.860 | I still think that's true

00:49:19.700 | within the extensions as well, right?

00:49:20.700 | Like, we got very, very creative with things.

00:49:22.940 | Like, Roon mentioned the idea of just like, you know,

00:49:24.700 | essentially rendering images to display things.

00:49:26.820 | Like, we get creative to figure out

00:49:28.580 | what the right UX is doing there.

00:49:29.820 | Like, we could create a really like, dumb UX

00:49:32.220 | that's like, a side panel, like, whatever.

00:49:33.660 | Like, but no, actually going that extra mile

00:49:36.020 | does make that experience as good as it possibly can there.

00:49:38.900 | But yeah, now like, look at some of the UX

00:49:40.540 | that we're able to build in like, in Windsurf.

00:49:42.780 | And it's just like, it's fun.

00:49:44.460 | The first time I saw,

00:49:46.380 | 'cause now we can do command in the terminal.

00:49:48.260 | Like, you can not have to search for a batch command.

00:49:50.460 | The first time I saw that, I was like,

00:49:52.340 | I just started smiling.

00:49:53.180 | And like, it's like, it's not, it's not like, Cascade.

00:49:55.180 | It's not like, a gentic system right in the lab.

00:49:56.460 | But I'm like, that is just a very, very cool UX.

00:49:58.020 | - We literally couldn't do that in VSCode.

00:50:00.020 | - Yeah, I understand that, yeah.

00:50:01.700 | I've implemented a 60-line batch command called please.

00:50:05.300 | And you can, you know, do that.

00:50:07.140 | - Oh, wow, that's cool.

00:50:08.500 | - Yeah, so please, English, and then--

00:50:10.100 | - You know, that's actually really cool

00:50:11.100 | because one of the things I think we believe in

00:50:13.420 | is actually, I like products like autocomplete

00:50:15.900 | more than command, purely because

00:50:17.620 | I don't even want to open anything up.

00:50:19.740 | So that thing where I just can type

00:50:21.540 | and not have to press some button shortcuts

00:50:23.140 | to go in a different place, I actually like that too.

00:50:25.140 | - Yeah, and I actually adopted WARP, the terminal WARP,

00:50:28.300 | initially for that, 'cause they gave that away for free.

00:50:30.740 | But now it's everywhere, so I can turn off a WARP

00:50:32.940 | and not give Sequoia my batch commands.

00:50:35.580 | (all laughing)

00:50:37.660 | - I'm with you.

00:50:38.500 | No, I use, I use WARP, no, no, look.

00:50:42.620 | I use, okay, I don't know.

00:50:44.660 | Gotta go on a rant.

00:50:45.500 | Hopefully somebody at WARP.

00:50:46.940 | This is like WARP product feedback.

00:50:48.660 | But they basically had this thing where you can do

00:50:50.740 | kind of like pound, and then write in Azure language.

00:50:54.380 | But then they have also the auto-infer

00:50:56.820 | of what you're typing is natural language,

00:50:58.860 | and those are different.

00:50:59.820 | When you do the pound, it's only like it gives you

00:51:02.060 | a predetermined command.

00:51:03.300 | When you like talk to it, it generates a flow.

00:51:06.220 | It's a bit confusing of a UX, but going back to your post,

00:51:09.980 | you had the three Ps of AI/UX.

00:51:12.460 | - What were they again?

00:51:13.300 | - Present, practical, powerful.

00:51:16.220 | - Actually, that was really good.

00:51:17.180 | I liked it.

00:51:18.020 | - And I think like in the beginning,

00:51:20.460 | being present was enough.

00:51:21.620 | Maybe even when you launch, it's like,

00:51:23.580 | oh, you have AI, that's cool, other people don't have it.

00:51:26.140 | Do you think we're still in the practical,

00:51:29.220 | where like the experience is actually,

00:51:31.100 | like the model doesn't even need to be that powerful,

00:51:33.420 | like just having better experience is enough?

00:51:35.940 | Or like do you think like really being able to do the whole,

00:51:39.460 | because your point was like you're powerful

00:51:41.620 | when you generate a lot of value for the customer.

00:51:44.020 | You're like practical when you're basically

00:51:45.820 | like wrapping it in a nicer way.

00:51:47.620 | Yeah, where are we in the market?

00:51:48.700 | - I think there's always gonna be room

00:51:50.180 | for like practical UX, like getting it.

00:51:52.260 | I mean, the command terminal,

00:51:53.580 | that's like a very practical UX, right?

00:51:55.100 | Like I do think with things like Cascade

00:51:57.540 | and these agentic systems,

00:51:58.420 | like we are starting to get onto powerful.

00:52:00.820 | 'Cause like there's so many pieces,

00:52:02.780 | like from a UX perspective that make Cascade really good.

00:52:05.580 | Like it's just really like micro things

00:52:08.060 | that are like just all over the place.

00:52:09.700 | But as you know, we're streaming in,

00:52:11.540 | we're showing like the changes,

00:52:12.820 | we're like allowing you to jump and open diffs and see it.

00:52:15.580 | We can run background terminal commands.

00:52:17.460 | You can see what like is what's running,

00:52:19.220 | background processes are running.

00:52:20.180 | Like there's all these small UX things

00:52:22.300 | that together come to a really powerful and intuitive UX.

00:52:25.700 | I think we're starting to get there.

00:52:27.540 | It's definitely just the start.

00:52:29.580 | And that's why we're so excited

00:52:30.740 | about where all this is gonna go.

00:52:32.420 | I think we're starting to see the glimpses of it.

00:52:33.820 | I'm excited.

00:52:34.740 | It's gonna be a whole new ballgame.

00:52:36.260 | - Yeah, awesome.

00:52:37.580 | First of all, it's just been really nice to work with you.

00:52:39.420 | It's like, you know, I do work with a number

00:52:41.700 | of guests, posters, and you know,

00:52:43.900 | not everyone makes it through to the end

00:52:45.580 | and nobody else has done it three times.

00:52:47.380 | So kudos.

00:52:49.460 | (all laughing)

00:52:52.700 | This one was more like the money one,

00:52:54.660 | which I, you know, it's funny

00:52:55.860 | 'cause I think developers are like

00:52:57.820 | quite uninterested in money.

00:52:59.940 | Isn't it weird?

00:53:00.980 | - Yeah, I mean, I think like,

00:53:02.340 | I don't know if this is just the nature of our company.

00:53:04.620 | Like, I think there's something you said,

00:53:06.300 | like there's all like the San Francisco AI companies

00:53:08.380 | and like, everyone's like hyping each other,

00:53:09.660 | like on the tech and everything, which is like great.

00:53:11.020 | The tech's really important.

00:53:12.100 | We're here in Mountain View, beautiful office.

00:53:14.220 | We just really care about like actually driving value

00:53:16.340 | and making money, which is kind of like a core,

00:53:18.820 | a core part of the company, but yeah.

00:53:20.660 | - I think maybe the selfish way of saying that,

00:53:23.260 | or like a little more of the selfless way is like,

00:53:25.260 | yeah, we can be kind of like this VC funded company forever,

00:53:28.980 | but ultimately speaking, you know,

00:53:30.700 | if we actually want to transform the way software happens,

00:53:33.340 | we need this part of the business that's cash regenerative

00:53:36.060 | that enables us to actually invest tremendously

00:53:38.380 | in the software.

00:53:39.300 | And that needs to be durable cash,

00:53:41.220 | be cash that like churns the next year.

00:53:43.380 | And we want to set ourselves up to be a company

00:53:46.180 | that is durable and can actually solve these problems.

00:53:49.820 | - Yeah, yeah, excellent.

00:53:50.700 | So for people, obviously we're going to link

00:53:52.540 | in the show notes,

00:53:53.380 | but for people who are listening to this for the first time,

00:53:56.820 | I had a lot of trouble naming this piece.

00:53:59.140 | So we originally called it,

00:54:01.020 | you had like how to make money something.

00:54:03.300 | - I, I, it was, it was, I apologize.

00:54:07.740 | I was super bitty.

00:54:08.580 | I was, I think it was like writing part of that story,

00:54:10.700 | like on a plane flight.

00:54:11.980 | So I apologize for that.

00:54:12.820 | - No, you had like $3 signs in the title.

00:54:14.220 | - Oh, I absolutely had $3.

00:54:15.060 | I was like, I'm, I can't do that.

00:54:17.060 | So it's either building AI for the enterprise.

00:54:18.660 | And then I also said the worst,

00:54:19.980 | the most dangerous thing an AI startup can do

00:54:21.660 | is build for other AI startups,

00:54:22.820 | which I think you all, both of you will co-sign.

00:54:24.700 | And I think basically the main thesis,

00:54:26.580 | which I really liked was like, go slow to go fast.

00:54:29.100 | Like here's the, if you actually build for like security,

00:54:33.020 | compliance, personalization, usage analytics,

00:54:35.100 | latency budgets, and scale from the start,

00:54:37.700 | then you're going to pay that cost now,

00:54:40.020 | but eventually it's going to pay off in the long run.

00:54:42.260 | And this is the actual insight.

00:54:44.740 | You cannot do this later.

00:54:46.300 | Like if you build the easy thing first as an MVP,

00:54:49.020 | it's like, yeah, I like just ship it

00:54:50.300 | with like whatever's easy to do.

00:54:52.460 | And then you tack on the enterprise ready.io

00:54:55.140 | set of like 12 things that you have,

00:54:57.100 | you actually end up with a different product

00:54:58.580 | or you end up worse off

00:55:00.380 | than if you had started from the beginning.

00:55:01.620 | So that I had never heard before.

00:55:03.660 | - Yeah. I mean, we see that like repeatedly.

00:55:05.780 | I mean, just like right now,

00:55:07.100 | you know, we're have a lot of customers

00:55:08.780 | in like the defense space, for example,

00:55:10.340 | we're going through FedRAMP accreditation right now

00:55:13.660 | and people that we're working with,

00:55:15.060 | they saw like all the fact like,

00:55:16.860 | oh yeah, we already have a containerized system.

00:55:18.900 | We can already like deploy in these manners.

00:55:21.140 | We can already do like X,

00:55:21.980 | we've already gone through like security.

00:55:23.820 | They're like, oh, you guys are going to have

00:55:25.620 | a much easier time doing this, right?

00:55:27.580 | Then most companies that are just like,

00:55:28.780 | okay, we have like a big SaaS blob

00:55:30.180 | and now we need to like do all these things.

00:55:31.820 | It might sound like a really deep thing.

00:55:33.220 | I think like it's just anyone who's like worked

00:55:34.780 | like, you know, for like an extended period of time

00:55:36.740 | at like a company, like on a certain project

00:55:39.140 | has probably seen this happen, right?

00:55:41.260 | Like the technology just keeps on improving.

00:55:43.220 | And then you realize that like,

00:55:44.660 | you have to now like re-architect your whole system

00:55:46.580 | to get something improving.

00:55:47.420 | Like just making that kind of change

00:55:49.660 | when you've invested so much effort,

00:55:51.460 | like people have like important hours,

00:55:53.380 | they're emotionally invested in whatever it might be.

00:55:54.900 | It's really hard to make that change.

00:55:57.140 | So I'm sure we're going to hit that also.

00:55:59.460 | Like, yes, I think we've done things

00:56:01.660 | a little bit earlier than most companies.

00:56:03.780 | I think we're going to hit points

00:56:05.300 | where we're going to see parts of our systems

00:56:06.620 | where like, oh, we really need to re-architect that.

00:56:08.780 | Actually, we've definitely hit that already, right?

00:56:10.660 | And I think that's just like at like the project level,

00:56:12.940 | the product level, or is that like your whole company, right?

00:56:15.740 | I think the thesis behind here is like,

00:56:17.700 | to some degree, your company needs to have this DNA

00:56:20.140 | from the beginning.

00:56:21.180 | And I think then you'll be able to go through those bumps

00:56:23.380 | a lot more smoother and be able to drive the value.

00:56:26.260 | I don't know, Varun.

00:56:27.100 | - Yeah, can I say two points?

00:56:28.660 | So first point I'd like to say is,

00:56:30.940 | this is something that me and Douglas,

00:56:32.100 | my co-founder, talk about a lot.

00:56:33.580 | It's like, you know, there's this constant thing

00:56:35.420 | of like build versus buy.

00:56:36.820 | I think the answer is like,

00:56:38.060 | a lot of the time the answer should be buy, right?

00:56:39.820 | Like we're not going to go build our own sales tool.

00:56:41.820 | We should go buy Salesforce, right?

00:56:43.340 | That's kind of dumb.

00:56:44.180 | That's undifferentiated.

00:56:45.340 | And the reason why you go with buy instead of build is,

00:56:48.060 | hey, like, look, the ROI of what exists out there is good.

00:56:51.060 | From like an opportunity cost standpoint,

00:56:52.820 | it's better to actually go out and buy it

00:56:55.100 | than build it and do a shittier job, right?

00:56:56.660 | There's a company that's actually going out

00:56:58.020 | and focused on that.

00:56:59.020 | But here's the hidden thing that I think

00:57:00.420 | is like really important.

00:57:01.460 | When you go out and buy,

00:57:02.660 | you're losing a core competency inside the company.

00:57:04.860 | And that's a core competency you can never get.

00:57:07.140 | It's always very hard.

00:57:08.060 | Like startups are so limited on time.

00:57:09.700 | Let me just say, like, let's say as a company,

00:57:11.900 | we did not invest in, I don't know, model inference.

00:57:15.060 | Yeah, we have like a custom inference runtime.

00:57:17.540 | We give that up right now.

00:57:18.620 | We will never get it back.

00:57:20.260 | It's going to be very hard to get it back, right?

00:57:22.300 | - You can't just use VLM and Telsa RT.

00:57:25.020 | - I mean, that would be your only option.

00:57:26.820 | - Or just let me put it, if we use VLM,

00:57:29.100 | we would not be talking with you right now.

00:57:31.060 | Like, yeah, we would have, yeah.

00:57:32.700 | But the point is, this is more a question of like,

00:57:35.060 | like, you know, I try to think about it

00:57:36.340 | from first minutes, it was like,

00:57:37.180 | Google's a great company, makes a lot of money.

00:57:38.700 | What happens if they actually made the search index

00:57:40.500 | of the product something that someone else bought for them?

00:57:42.380 | It's like, they could.

00:57:43.220 | Maybe someone else could have done a good job.

00:57:44.980 | Maybe that's like a bad example.

00:57:46.580 | But like, particularly because Google is a search index,

00:57:49.180 | but like, tough luck getting that core competency back.

00:57:51.700 | You've lost it, right?

00:57:53.020 | And I think for us, it's more a question of like,

00:57:54.900 | what core competencies do we need inside the business?

00:57:57.580 | And yeah, like, sometimes it's painful.

00:57:59.260 | Like, sometimes actually, like,

00:58:00.260 | some of these core competencies are annoying.

00:58:01.540 | Sometimes we'll be behind, behind what exists out there.

00:58:04.540 | Right, and we need, just need to be very honest.

00:58:05.860 | That's where the truth-seekingness of the company matters.

00:58:08.260 | Like, are we really honest about this core competency?

00:58:11.420 | Can we actually keep up?

00:58:12.660 | If the answer is we truly can't keep up,

00:58:14.180 | then why are we keeping up with the trade?

00:58:15.300 | We should just buy, right?

00:58:16.580 | Like, let's not build.

00:58:17.700 | If the answer is we can,

00:58:18.540 | and we think that this will differentiatedly

00:58:20.060 | make our company a better company in the long-term,

00:58:22.060 | then the answer is we need to.

00:58:23.580 | We need to, because like,

00:58:24.580 | the race is not won in the next year.

00:58:26.020 | The race is won over the next, like, five, 10 years, right?

00:58:28.100 | Maybe even longer, right?

00:58:29.180 | So that's like, that's maybe one thing.

00:58:30.540 | And then the second thing, actually,

00:58:32.020 | from like, the enterprise standpoint,

00:58:33.700 | I think one of the unique parts of the company is now is,

00:58:36.260 | we have both this individual and enterprise side,

00:58:38.260 | and usually companies stick to one or the other.

00:58:40.460 | And I think that needs to be part of the DNA.

00:58:42.740 | I think kind of early on in the company, as Anshul said,

00:58:45.180 | I mean, there's stories of companies like Dropbox

00:58:47.180 | and stuff that tried.

00:58:48.660 | And Dropbox is an amazing company, fantastic company,

00:58:51.220 | that one of the fastest-growing consumer companies

00:58:53.460 | of all time, consumer more on the software company

00:58:55.860 | of all time.

00:58:56.700 | But yeah, like, when you have everyone

00:58:58.340 | sort of product-oriented on the consumer side,

00:59:00.300 | the enterprise is just, it's checking off a lot of boxes

00:59:03.660 | that ultimately do not help the consumer at all,

00:59:06.740 | doesn't help your growth metrics.

00:59:08.220 | And effectively, if the original group of people

00:59:10.260 | didn't care, it's incredibly hard to get them

00:59:12.500 | to care down the line, right?

00:59:14.420 | It's incredibly hard, why do it?

00:59:16.020 | And you need to feel like, hey, this is like,

00:59:17.780 | this is an important part for the company's viability.

00:59:20.700 | So I think that there's a little bit of like,

00:59:22.820 | the build versus buy part, and then also like,

00:59:24.660 | the cultural DNA of the company

00:59:26.300 | that I think are both really important.

00:59:27.620 | And yeah, it's something we think about all the time.

00:59:29.780 | - I have the privilege of being friends

00:59:31.380 | with you guys off the air.

00:59:32.940 | I don't feel like, like, I think I know your work histories.

00:59:35.580 | Like, you say cultural DNA, but like,

00:59:37.580 | it's not like you've built like,

00:59:39.020 | giant enterprise SaaS before, right?

00:59:41.660 | - Yeah, I think, yeah.

00:59:44.100 | - So like, where are you getting this from?

00:59:46.140 | - Yeah, in fact, I think the only other sort of,

00:59:50.020 | I guess like, when I look at my previous internships,

00:59:52.660 | maybe Anshul can provide some context here.

00:59:54.340 | It's like, I worked at like, LinkedIn,

00:59:56.180 | and then Quora, and then Databricks.

00:59:57.340 | And to be honest, like, I was not that interested

00:59:59.340 | in B2B ETL software that much.

01:00:01.540 | That's not what drives me when I wake up at night.

01:00:03.500 | So, because of that, I decided to go work

01:00:05.900 | in an autonomous vehicle company immediately after.

01:00:08.460 | I think part of it comes down to,

01:00:10.620 | maybe a little bit of the unique aspect of the company,

01:00:12.860 | and the fact that we pivoted as a company,

01:00:14.700 | is like, we want to be a durable company.

01:00:18.380 | And then the question is,

01:00:19.260 | how do you work backwards from that?

01:00:20.740 | There's a lot of things about being very honest

01:00:22.340 | about what we're good at and what we're not good at.

01:00:24.220 | Like, I think, surprisingly, enterprise sales

01:00:26.180 | is like, not something that like,

01:00:27.540 | it came out of the womb knowing how to do.

01:00:29.260 | I didn't really know.

01:00:30.180 | And because of that, like, obviously like,

01:00:31.900 | a lot of sales happen between sort of folks

01:00:34.380 | like Anshul and I helping partner with companies.

01:00:37.100 | But very soon, we hired actually a VP of sales,

01:00:39.340 | and we've actually been deeply involved

01:00:41.060 | in the process of scaling out like,

01:00:42.460 | a large go-to-market team.

01:00:43.820 | And I think it's more a question of like,

01:00:45.780 | what matters to the company,

01:00:47.100 | and how do you actually go out and build it?

01:00:48.340 | And I think one of the people that I think about a lot,

01:00:50.100 | actually, is someone like Alex Wang.

01:00:51.700 | He dropped out of college.

01:00:52.620 | He was a year younger than us at MIT.

01:00:54.420 | And he has figured out how to constantly change

01:00:57.620 | the direction of the company.

01:00:58.740 | Effectively, it starts out as like,

01:01:00.300 | you know, human task interface,

01:01:01.620 | then an AV labeling company,

01:01:03.100 | then a cataloging company,

01:01:04.340 | then now a generative AI labeling company.

01:01:06.500 | And every time, the revenue of the company

01:01:07.980 | kind of goes up by a factor of 10,

01:01:09.180 | even though the business is doing something

01:01:10.580 | largely different. - I mean, now it's all

01:01:11.420 | about military contracts. - Yeah, now it's probably

01:01:12.900 | gonna be military, and then after that,

01:01:13.980 | it might be taking over the world.

01:01:15.020 | Like, he's just gonna keep increasing the stakes.

01:01:17.420 | And like, there's no playbook on how this really works.

01:01:19.940 | It's just a little bit of like, you know,

01:01:22.340 | solve a hard problem and work backwards from that, right?

01:01:26.060 | - And we'll get lucky along the way.

01:01:27.260 | Like, I don't think like, you think everything

01:01:29.220 | from first principles to the best of our abilities,

01:01:31.540 | but there's just so many variable unknowns that,

01:01:34.380 | yeah, like, we don't know everything that's happening

01:01:36.620 | in every company out there,

01:01:37.940 | and everyone knows how fast the AI space is moving.

01:01:40.180 | Like, we have to be pretty good at adapting.

01:01:42.740 | - I wanna double-click on one thing,

01:01:44.060 | just because you brought it up,

01:01:45.660 | and it's like a rare thing to touch on.

01:01:47.220 | VP of sales.

01:01:48.180 | We don't get to actually,

01:01:49.020 | we talk to pretty early-stage founders mostly.

01:01:50.940 | They don't usually have a pretty built-out sales function.

01:01:54.060 | Advice, what kind of sales works in this kind of field?

01:01:58.180 | You know, what didn't work?

01:02:00.020 | You know, anything that you can share with other founders?

01:02:02.060 | - I think one of the hard parts

01:02:03.260 | about hiring people in sales,

01:02:04.860 | and I really, like, Graeme Unschul can also attest,

01:02:07.420 | like, we have amazing VP of sales at the company.

01:02:10.180 | One of the things is like,

01:02:11.260 | if you're purely a developer salespeople,

01:02:13.180 | their job is to like, talk like, really well,

01:02:16.060 | prim and proper.

01:02:16.900 | I mean, very obvious if you hear like, me talk,

01:02:19.180 | like, I'm not a very polished person.

01:02:20.860 | - You're great, by the way.

01:02:21.700 | I don't know.

01:02:22.540 | - Compared to most pure, pure salespeople.

01:02:25.900 | So actually, just checking based on the way they speak

01:02:29.220 | is not that interesting.

01:02:30.260 | I think like, you know, what matters in a space like ours

01:02:32.500 | that is very quickly, moving very quickly,

01:02:34.900 | I think is like, intellectual curiosity is very important.

01:02:37.460 | Intellectual horsepower.

01:02:38.860 | Understanding how to build a factory.

01:02:41.340 | I'm not trying to minimize it,

01:02:42.540 | but in some ways, you need to build

01:02:45.100 | something incredibly scalable here, right?

01:02:47.580 | It's almost like every year,

01:02:49.100 | you're kind of making this factory twice, thrice,

01:02:51.620 | maybe as big, right?

01:02:52.900 | Because in some ways,

01:02:53.740 | you have people that are quota-carrying,

01:02:54.900 | you need some number of people,

01:02:56.580 | and you need to make the math work.

01:02:57.780 | And you actually, the process of building a factory

01:03:00.020 | is not something you can just take someone

01:03:01.540 | who is a great rep at another company

01:03:03.260 | and just make them build a factory.

01:03:04.740 | This is actually a very different skill.

01:03:06.540 | How do you actually make sure you have hundreds of people

01:03:08.660 | that actually deeply understand the product?

01:03:10.060 | Actually, Onshore works very closely also with sales

01:03:12.900 | to make sure that they're enabled properly.

01:03:14.540 | Make sure that they understand the technology.

01:03:16.020 | Our technology is also changing very quickly.

01:03:18.020 | Let's maybe take an example on how our company

01:03:20.060 | is very different than a company like MongoDB.

01:03:21.860 | When you sell a product like MongoDB,

01:03:23.380 | no one at the company is interested

01:03:24.980 | in how the data is being stored.

01:03:26.580 | It's not that interesting, right?

01:03:27.940 | I love databases.

01:03:29.060 | I would be interested.

01:03:29.940 | But most people are like,

01:03:30.780 | solve the application problem I have at hand.

01:03:32.740 | People are curious about how our technology works.

01:03:34.940 | People are curious about RAG, right?

01:03:36.580 | People that are buying our technology.

01:03:38.180 | And imagine we had a sales team that is scaling

01:03:40.060 | where no one understands any of this stuff.

01:03:42.020 | We're not gonna be great partners to our customers.

01:03:44.100 | So how do you create almost this growing factory

01:03:47.180 | that is able to actually distribute the software

01:03:49.220 | in a way that is true to our partners?

01:03:51.220 | And also at the same time,

01:03:52.580 | taking on all the new parts of our product, right?

01:03:54.900 | They're actually able to expound

01:03:56.740 | on new parts of our product.

01:03:57.860 | So, sorry, that was more a statement

01:04:00.220 | of building a scalable sales team.

01:04:01.820 | But in terms of who you hire is,

01:04:03.900 | you just need to have a sense.

01:04:05.580 | In some ways, this is maybe an example

01:04:07.980 | of talk to enough people,

01:04:09.260 | find out what good looks like potentially in your category

01:04:11.420 | and find someone who's good and humble

01:04:12.900 | and willing to work with you.

01:04:13.740 | - Yeah, that's just generic hiring.

01:04:14.820 | - It's just generic hiring.

01:04:15.980 | - I think here, there's sales for AI

01:04:20.180 | or sales for AI infrastructure.

01:04:22.220 | And then there's also the sales feeding into products

01:04:25.380 | in a way that we're talking about here, right?

01:04:27.420 | Where they basically tell you what they need.

01:04:29.780 | I imagine a lot of that happened.

01:04:31.340 | - I think a lot of that happened.

01:04:32.460 | I mean, it still happened.

01:04:33.300 | Varun mentioned, Varun, myself,

01:04:35.380 | a number of other people who are developers

01:04:37.460 | by trade engineers.

01:04:38.500 | We're pretty involved in the sales process

01:04:40.540 | 'cause there's a lot to learn, right?

01:04:42.980 | Before we went out and hired a sales leader,

01:04:45.740 | yeah, if all we went is like,

01:04:47.020 | neither of us had ever done a sale for Codium in our lives

01:04:49.940 | and we went and tried to find a sales leader,

01:04:51.500 | we probably would have not hired the right person.

01:04:53.660 | - Yeah, we had sold a product

01:04:54.780 | to like 30 or 40 customers at that time.

01:04:56.660 | - We had done like hundreds and hundreds of deals cycles

01:04:59.580 | ourselves personally, right?

01:05:01.460 | Without, I mean, we read a lot of books

01:05:03.020 | and we just did a lot of stuff

01:05:04.020 | and we learned like what messaging worked,

01:05:05.420 | like what did we need to do?

01:05:07.060 | And then I think we found like the right person, right?

01:05:09.420 | Second Varun, like Graham's amazing

01:05:11.020 | and who we brought on as our VP of sales.

01:05:13.060 | That just has to be part of the nature

01:05:14.780 | and it doesn't stop now.

01:05:15.940 | Like just because we have a VP of sales

01:05:17.540 | and people dedicated to sales,

01:05:19.300 | it doesn't stop that we can't be involved

01:05:22.060 | or like engineering can't be involved, right?

01:05:23.620 | Like we have lots of people,

01:05:25.300 | like we hire plenty of deployed engineers, right?

01:05:27.660 | These are people like, you know,

01:05:28.660 | I think like Palantir kind of made this really famous.

01:05:30.860 | - Four deployed engineers.

01:05:31.700 | - Like deployed engineers like work very, very closely

01:05:33.980 | with the sales team on very technical aspects

01:05:35.860 | because they can also understand

01:05:37.300 | like what are people trying to do with AI?

01:05:39.260 | - As in they work at Codium as deployed engineers?

01:05:41.260 | - Yeah. - Okay.

01:05:42.380 | - And then they partner with our account executives

01:05:44.820 | to like make our customers successful

01:05:46.740 | and like learn what is it that people

01:05:48.500 | are actually getting value with AI, right?

01:05:50.260 | And like that's information that we keep on collating.

01:05:52.460 | And it's like, we will both jump into any deal cycle

01:05:55.060 | just to learn more

01:05:56.300 | because that's how we're going to just keep on building

01:05:59.180 | like the best product.

01:06:00.020 | It comes back to the same, like just care, I don't know.

01:06:02.700 | And hopefully we build the right thing.

01:06:04.700 | - Cool guys.

01:06:05.540 | Thank you for the time.

01:06:07.060 | It's great to have you back on the pod.

01:06:08.500 | - Yeah, thanks a lot for having us.

01:06:09.500 | Hopefully in a year we can do another one.

01:06:11.220 | - Yeah.

01:06:12.060 | - We'll be a billion by then.

01:06:12.900 | - Yeah, exactly.

01:06:13.740 | At this rate, then next year.

01:06:15.980 | - We try not thinking about that.

01:06:17.300 | - Try to not be a zero billion company.

01:06:18.660 | - That's, well, I'm glad there's that, yeah.

01:06:21.340 | - All right, cool, that's it.

01:06:22.180 | - Awesome.

01:06:23.580 | (upbeat music)

01:06:26.180 | (upbeat music)

01:06:28.760 | (upbeat music)

01:06:31.340 | (upbeat music)

01:06:33.920 | (upbeat music)

Windsurf: The Enterprise AI IDE

Chapters