back to index

Copilots Everywhere: Thomas Dohmke and Eugene Yan


Whisper Transcript | Transcript Only Page

00:00:00.000 | I'm delighted and honored to welcome a very special guest, Thomas Domke.
00:00:18.360 | Thomas has been fascinated by software development since his childhood in Germany,
00:00:23.260 | and he's built a career building tools, dev love, and accelerating innovations
00:00:28.780 | that are changing software development.
00:00:30.440 | Currently, Thomas is CEO at GitHub,
00:00:32.840 | where he has overseen the launch of the first at-scale AI developer tool, GitHub Copilot.
00:00:39.240 | So please join me in welcoming to the stage, Thomas Domke.
00:00:43.920 | Thank you, Thomas.
00:00:57.560 | Oh, it's bright out here.
00:01:00.380 | Yeah.
00:01:01.000 | Well, thank you, everyone.
00:01:02.980 | Thank you, Thomas.
00:01:03.540 | Let's start with Copilot.
00:01:06.340 | Many people have shared their own takes on the Copilot origin story.
00:01:11.480 | So, but what was your personal experience seeing it in GitHub?
00:01:14.980 | I don't know.
00:01:15.400 | You had a sneak preview.
00:01:16.300 | Take us back to the start in 2020.
00:01:18.100 | So, imagine it's 2020.
00:01:20.420 | It's lockdown here in San Francisco, in Seattle, everywhere where GitHub engineers are sitting.
00:01:25.640 | So, like all of you probably were on a Zoom call.
00:01:29.100 | One of us had early access to a new model that OpenAI had just released in preview,
00:01:36.000 | a version of GPT-3 called Codex.
00:01:38.580 | And, you know, one had, Uge, I think, had the keyboard, the leader of GitHub Next at the time.
00:01:44.240 | And we were dictating prompts and asked the model to write some code.
00:01:49.460 | And I think the first aha moment that I had is that you could ask it to write JavaScript code
00:01:54.460 | and put the curly braces in the right places and whatnot.
00:01:57.320 | And you could write it, ask it to write Python code.
00:01:59.720 | And the model, in a way, you know, it doesn't work like a compiler.
00:02:03.300 | It doesn't have a syntax tree.
00:02:04.940 | It doesn't know these things.
00:02:06.540 | Or you could also argue it knows them exactly like we'd know it.
00:02:09.560 | So, that was probably the first moment we kept building, we kept exploring the model
00:02:15.100 | and then decided we built this autocompletion copilot that, you know, was the first copilot.
00:02:19.100 | And we built it all, you know, while being remote, while being on lockdown.
00:02:23.240 | So, if your investors tell you today you need to be in a room and in front of a whiteboard,
00:02:29.800 | you can innovate if you want to while being in your home offices around the world.
00:02:35.560 | I think the next moment was that we shipped a preview to our internal engineers,
00:02:42.000 | and we call it a staff ship at GitHub.
00:02:43.600 | And the NPS survey with those engineers was through the roof, I think, 72.
00:02:48.960 | 72, 73, something like that.
00:02:50.120 | And typically, our early stage products, especially, you know, with the large language model
00:02:53.880 | and, you know, all the hallucinations and the UI wasn't really figured out yet, it's much lower.
00:02:59.540 | So, that was kind of like a holy shit moment that we had.
00:03:03.300 | And as the product then shipped in mid-2021, and, you know, COVID was still going,
00:03:08.580 | we started looking at telemetry, and the team came and says it writes about 25% of the code
00:03:14.720 | in those files where it was enabled.
00:03:16.240 | And I remember saying, don't believe this.
00:03:19.360 | Like, you know, telemetry is wrong.
00:03:21.080 | Please go back and validate that.
00:03:22.820 | And it turned out, you know, that was actually right.
00:03:25.300 | And by now, it's about half, you know, the code that's written.
00:03:29.100 | Some languages like Java even have a higher acceptance rate and more lines written.
00:03:33.160 | And so, I think those kind of, this journey that we went through over the first two years
00:03:37.940 | really was like one moment after another where we saw the future of AI long before ChatGPT
00:03:44.820 | actually opened everybody else's mind.
00:03:46.480 | Amazing.
00:03:47.380 | And now it's available to everyone here as well.
00:03:49.180 | So, I think Copilot started as an autocomplete IDE, and now it's all over GitHub.
00:03:54.440 | I know I have PR boards, et cetera.
00:03:56.040 | What do you do to make Copilot and integrate it across all of GitHub?
00:04:01.740 | Like, what are some experiments?
00:04:02.740 | What worked?
00:04:03.180 | What didn't work?
00:04:03.840 | I think the first thing is to think about, you know, what do I do as a leader, as the CEO
00:04:08.520 | of a company?
00:04:09.240 | And it's really about constantly reconfiguring our approach.
00:04:12.920 | So much of, you know, the AI world is changing almost daily.
00:04:17.140 | And there's, you know, some news on the information elsewhere every morning.
00:04:21.300 | And so, there is no more A.
00:04:23.080 | I have a long-term strategy.
00:04:24.440 | I have my features all laid out, and it's worked through the backlog.
00:04:27.840 | It's really like operating as agile as possible, even as we are, you know, a 3,000-person company
00:04:33.140 | as part of, you know, one of the largest companies on the planet.
00:04:36.840 | The second is that we really try to meet, you know, the developer where they are.
00:04:41.800 | We say, you know, we're not trying to build an AI engineer.
00:04:45.120 | We are trying to build AI for engineers, a human-centric approach.
00:04:48.680 | You know, that's where the name, what the name Copilot ultimately visualizes.
00:04:53.420 | But also, you know, we're trying to make the developer's lives better because we are developers
00:04:58.760 | ourselves, and every productivity improvement we can find ultimately helps us at GitHub to
00:05:04.460 | build, you know, our AI product.
00:05:05.960 | So, that really is the approach, like, looking at what can we do next to make, you know,
00:05:10.840 | our work a little bit easier of building more features for Copilot.
00:05:14.480 | You mentioned a great point.
00:05:15.700 | You're trying to meet the developer where they are.
00:05:17.340 | So, for now, we've been bringing the AI to the IDE.
00:05:20.500 | Yeah.
00:05:21.120 | Can you...
00:05:21.920 | Are we going to try to bring the developer, the IDE, closer to the AI?
00:05:25.500 | How are you thinking about that?
00:05:27.300 | You know, the idea of bringing AI into the IDE, or really into ghost text, you know, auto-completions,
00:05:33.360 | was a way of getting around hallucinations.
00:05:36.500 | It was a way of saying, okay, the model's not always going to be perfect, but neither are
00:05:40.820 | auto-completions, right?
00:05:42.020 | Like, whether you have auto-completions in your Google Docs, or in your email, or in your
00:05:45.840 | editor, in the old IntelliSense way, as you're typing, it cannot know what you want it to type.
00:05:51.260 | And so, you're used to adjusting your typing, and then you find this moment when you press
00:05:55.640 | the tab key.
00:05:56.320 | And even without auto-completions, if you're thinking about what developers do in the editor,
00:06:00.920 | while they write code, and the best developers write a lot of code before they get stuck,
00:06:04.540 | and the newbies, and those that rarely write code like I, you know, get stuck more often.
00:06:09.820 | And then you, you know, control-tap or command-tap into your browser, and you open Google or Stack
00:06:18.120 | Overflow, GitHub, right?
00:06:19.760 | And what you do there is you find code, and you argue with other developers, and then you
00:06:23.780 | copy and paste that code into your editor, and you modify that as well.
00:06:27.920 | So, it's kind of like, in a way, Stack Overflow has as many hallucinations as the model that might
00:06:33.720 | have, and not because the answers are bad, but because the world is changing so much.
00:06:38.040 | You know, I code a little bit on iPhone projects in Swift, and there's always a new Swift version
00:06:42.700 | after WWDC, or a new Xcode version, and so things have changed of how you use the APIs.
00:06:47.300 | And so, it keeps the developer in the flow.
00:06:49.840 | That really, the crucial thing here was, we didn't, you know, in the world, you know, 10 years
00:06:55.020 | ago, we probably wouldn't even call this AI.
00:06:56.720 | We would just call it, you know, smarter auto-completion.
00:06:59.740 | And the AI piece is not the core piece.
00:07:02.900 | The core feature of Copilot is that it helps developers to stay in the flow to get the job
00:07:08.680 | done, and not be in this constant distraction between the editor and the browser.
00:07:12.560 | That's a great point.
00:07:13.900 | And I think, a few months ago, you wrote this post about workspace.
00:07:18.880 | What was the journey to creating workspace?
00:07:22.020 | And maybe for folks who are not unfamiliar with it, what is workspace?
00:07:25.940 | Yeah, so, you know, you already mentioned auto-completion.
00:07:29.080 | That's how we started in November 2022.
00:07:32.860 | ChatGPT happened.
00:07:33.860 | So, early 2023, we had a chat and GPT-4 to copilot in the IDE as a separate sidebar window.
00:07:42.640 | So, we have that available.
00:07:43.520 | And it has reg and all the information, the context available in the IDE.
00:07:47.520 | But ever since, we've been thinking, how can we make the developer flow even easier?
00:07:51.500 | And workspace does exactly that.
00:07:52.960 | It takes a GitHub issue or just a task, an idea that you write down on GitHub.com.
00:07:58.100 | And it helps you then, as part of your code-based repositories, to figure out how to implement
00:08:02.500 | that change.
00:08:03.000 | It bridges from the issue, you know, from the task description into the pull request, into
00:08:08.300 | code.
00:08:09.300 | And the magic behind this is that, A, the human is still in the center.
00:08:12.080 | So, every step of that way, you know, writing a specification, analyzing the current repo,
00:08:17.320 | the current behavior, and then using your description to figure out how do you modify this.
00:08:21.080 | And then writing the plan, which shows you how to change all the files to the implementation,
00:08:27.080 | which is the div view, if you will.
00:08:28.080 | The human can interact, can change those bullet points, can change the code.
00:08:32.080 | And what that really does is it gives you a pair programmer that helps you to explore
00:08:38.240 | the code base, right?
00:08:39.800 | Because the challenge we all have as engineers is that as soon as you get moved onto a new
00:08:44.840 | project or you want to, you know, modify an open source project or you're just, you know,
00:08:49.580 | coming back from vacation, you're trying to remember what is implemented where in your
00:08:53.680 | thousand-plus files, that is navigating the code base is the first challenge you have,
00:08:58.080 | figuring out what's the current behavior and what's the new behavior.
00:09:01.180 | So, you're having an AI native, a co-pilot native developer environment that helps you
00:09:05.460 | along that journey that you're naturally also doing in your IDE.
00:09:08.740 | And that really is the key here.
00:09:10.060 | It's not about, you know, building an autonomous agent.
00:09:13.180 | I'm sure you've heard a lot about that in the last three days.
00:09:15.880 | It's about building agents that helps us as humans to solve a task and learn along the
00:09:21.000 | way as we figure out, oh, you know, there's this test file that I also have to modify if
00:09:24.840 | I want to implement this feature.
00:09:26.340 | I love the point that you mentioned, which is not building autonomous agents and also helping
00:09:30.280 | the developers.
00:09:31.280 | So, how should none developers use Workspace?
00:09:34.560 | They can.
00:09:35.560 | And in fact, you know, once we announced this last year at GitHub Universe in November, I
00:09:40.560 | think the first email we got with feedback was from a program manager or product manager saying,
00:09:44.560 | "This is awesome because now I can not only write, you know, a user story or a work item,
00:09:49.560 | I can also see what that would mean to implement in the code base.
00:09:53.840 | And in many ways, you know, the biggest challenge we have today is can we be as specific as
00:09:58.120 | possible when we write down a task, you know, as product managers or as engineers ourselves.
00:10:02.120 | You know, often everything is obvious until it is not.
00:10:06.400 | And then, you know, you kind of need to size the task, right?
00:10:11.400 | Like, how long will it take and the mythical man month?
00:10:14.680 | I think the pragmatic engineer had that a couple of weeks ago is still true.
00:10:18.640 | Most estimates are half the time that the job actually takes.
00:10:23.180 | And so, we're really bad at estimating how much time it takes to get work done, whether
00:10:27.720 | it's encoding or whether it's building houses or roads or infrastructure.
00:10:31.620 | And so, Workspace helps you with that as it helps you to figure out what I just described.
00:10:36.020 | Is it actually specific enough to write the code for that or to even figure out what the plan
00:10:39.920 | would look like?
00:10:41.720 | Can you share a bit about your vision on how you think we will build and code in natural
00:10:47.900 | language and how it will help us collaborate better, devs and PMs, coders and non-coders,
00:10:54.060 | across languages and across the world?
00:10:56.900 | For me, you know, the very first thing when you say natural language is, and I have it on
00:11:00.940 | my t-shirt here, Copilot Speaks Your Language, is because chat, these large language models that
00:11:05.200 | we're using today in GitHub Copilot and many other AI applications are the same models that
00:11:09.760 | are also helping us in chat agents.
00:11:11.560 | They speak almost any language, any major human language.
00:11:15.460 | And so, whether you want to explore coding in English and you don't understand the concepts
00:11:20.040 | of true/false, Boolean logic yet, or whether you want to learn that in German, in Hindi,
00:11:25.560 | in, you know, Brazilian Portuguese, in Spanish and Chinese, you can do that now.
00:11:29.760 | And if I, you know, look at kids today in school, most of them are growing up with mobile phones.
00:11:35.840 | You know, when you go into a restaurant here in San Francisco or in Seattle or elsewhere in the world,
00:11:40.360 | at night, you probably see a family with little kids where the kids have their phone because the parents want to enjoy five minutes on their own.
00:11:46.260 | And then as then kids grow up, you know, they see Super Mario or Minecraft and they get into gaming and that naturally,
00:11:51.960 | that means, how can I create my own game?
00:11:53.960 | How can I create my own web page?
00:11:55.960 | Copilot enables that and it enables that in the language that the kids grow up with, which, you know, for the majority of the humans
00:12:02.060 | of the, on the, on the, on this planet is not English.
00:12:04.360 | And so that's number one.
00:12:05.760 | It democratizes access to technology.
00:12:07.760 | It also democratizes access for those that don't have parents at home that have a technical background
00:12:12.360 | or that don't have parents at home that have infinite patience, but most parents do not.
00:12:16.460 | I have two kids.
00:12:17.160 | I speak from my own experience.
00:12:18.860 | At some point you're just done, you know, with explaining the world to your kids and you just want to, you know,
00:12:22.260 | switch on the TV and, and watch a, watch a Netflix show.
00:12:25.760 | And, and, and, but that keeps going.
00:12:28.060 | If you look into the professional context, one of the biggest challenge we have is, you know,
00:12:31.260 | if you would join my company or I join your company tomorrow,
00:12:35.260 | the biggest challenge we have is what's all the institutional knowledge?
00:12:38.060 | How are things being done?
00:12:39.260 | You know, and what we don't like as humans is ask a thousand questions.
00:12:43.160 | Especially if you're a new employee in a, in a big company, you're like having this anxiety in your head
00:12:48.360 | that everybody else thinks you are, you are dumb.
00:12:50.260 | You know, why did you get hired in the first place?
00:12:52.060 | So a copilot also democratizes access to all the information and companies.
00:12:55.960 | And I think that is going to be changing how we work and not only for developers in the workforce,
00:13:00.860 | but for really every human.
00:13:02.360 | Thank you.
00:13:03.860 | Thank you, Thomas, for sharing your vision.
00:13:05.160 | I guess the next thing I want to ask is maybe a little bit more unhinged.
00:13:08.660 | Speaking of agents, in your opinion, what makes an agent or a copilot?
00:13:14.660 | What's the definition, what, what's your definition of an agent?
00:13:18.260 | I think an agent, you know, is like an AI dishwasher.
00:13:22.060 | You fill it with, you know, the dishes and you let it, let it do its thing.
00:13:26.060 | And then at the end, just to take the output and you put it back into the shelves.
00:13:29.760 | Right.
00:13:30.260 | And today we have, you know, we called it, used to call it bots, you know, or CI/CD.
00:13:37.260 | And in many ways that's an autonomous agent, right?
00:13:38.760 | You push your pull request and you run your CI/CD GitHub Actions or a similar product.
00:13:44.060 | And many compute primitives that we have today are agents as they get a job done on their own.
00:13:50.760 | My monitoring, you know, to figure out if GitHub up or down is somewhat autonomous.
00:13:54.760 | Hopefully pages somebody without us hearing from you that you cannot access your repository.
00:13:59.760 | So, you know, I think in many ways what we are building is still tools that help us to get the job done.
00:14:06.260 | And there's many jobs that developers have to get done.
00:14:08.260 | Many jobs that now AI engineers need to get done.
00:14:11.260 | You saw it on the slide earlier.
00:14:12.260 | All the things that are also still true, you know, even though you can automate things with large language models.
00:14:17.260 | And a lot of work in software engineering is bogging us down.
00:14:22.260 | A lot of boilerplate, a lot of security compliance.
00:14:25.760 | You know, that Friday evening when you want to, you know, enjoy the barbecue because the sun is out and instead you have to update all your log4j dependencies, right?
00:14:33.760 | Like security tooling, in fact, you know, is creating more work.
00:14:36.760 | It's not a dishwasher.
00:14:37.760 | It's actually a tool that tells you that the dishes are dirty and then you have to do the dishes yourself today.
00:14:43.760 | And so that's security tooling, right?
00:14:45.760 | It just adds stuff to our backlog while we actually want to work on the creative side and we want to build new features.
00:14:51.760 | You want to build innovative product that are creative things.
00:14:54.260 | And I think many software developers do not understand themselves as a production worker.
00:14:58.260 | They understand themselves as artists, as creators.
00:15:01.260 | But, you know, our companies, our governments, you know, the world is requiring us to do a lot of other work.
00:15:06.260 | And then we need AI tools, auto-fix, you know, things that scans not only for security issues but then fixes those security issues.
00:15:14.760 | We need those pieces supported by AI so we have more time for the things we do want to do and AI takes over the things we don't want to do.
00:15:22.760 | And that's where the agents will go.
00:15:24.760 | Fantastic.
00:15:25.260 | What's an agent you want to have?
00:15:27.260 | And how far are we from it?
00:15:29.260 | I mean, I want to have these agents that burns down all my security backlog.
00:15:33.260 | As in any company, the challenge is that I have way too many of these items and there isn't really a book you can buy that tells you as an engineering manager of how to balance those two things.
00:15:45.760 | You cannot do all your work into security compliance, accessibility and whatnot and you cannot put all your work into innovation because your customers will lose all your trust the moment you have a security issue that threatens their data.
00:15:59.260 | And as such, you have to balance those two things or you find AI agents that brings the work down.
00:16:04.260 | And I think as any, you know, leader of a software development company, I always want to go faster.
00:16:09.260 | I always want to get that feature done faster and I'm sure it's, you know, the same for you folks at Amazon.
00:16:15.260 | When I have an idea and I ask my folks how long will it take to implement that, the estimate I'm getting is like I'm scratching my head and I'm thinking I can do that myself faster than waiting for my team to do it.
00:16:27.260 | But of course, that's not the truth.
00:16:28.760 | The truth is that there's so many other things in the process these days that we need to find new abstraction layers that help us to get control over our development lifecycle again.
00:16:39.760 | That's a great point.
00:16:40.760 | So, last question.
00:16:41.760 | Do you have any advice for devs, both new and experienced, on how they should navigate this new world of tools, this new world of abstractions, in what some say is the biggest technology innovation since the internet?
00:16:56.760 | I think, you know, the most exciting thing about this new technology is, and you saw it hopefully over the last three days at this conference, is that we are moving into a new world of software development.
00:17:09.760 | And there have been multiple step functions, you know, over my life.
00:17:13.760 | I was born right before the PC was invented.
00:17:16.760 | I remember getting my Commodore 64 on a PC in the 90s.
00:17:20.760 | I remember the open source and the internet.
00:17:22.760 | And, you know, internet open source before the internet was buying CDs and DVDs in bookstores.
00:17:27.760 | The internet came, you know, SourceForge, and then GitHub came.
00:17:30.760 | All of a sudden, developers started collaborating.
00:17:32.760 | The mobile wave came.
00:17:33.760 | And every time we had those step functions, software development got more exciting.
00:17:37.760 | And I think, you know, we are again at that step function.
00:17:40.760 | It means we can embrace our nerditude.
00:17:43.760 | We can build new shit.
00:17:45.760 | And I think, you know, it's really like, you know, for me as the CEO of GitHub, I don't get to touch code often.
00:17:52.760 | And so when I get to touch code on a Sunday afternoon, I don't want to spend all my time of updating all my dependencies.
00:17:58.760 | Okay, that's all we had.
00:17:59.760 | Thank you, Thomas.
00:18:00.760 | Please join me.
00:18:01.760 | Thank you so much.
00:18:02.760 | Thank you.
00:18:03.760 | Thank you.
00:18:04.760 | Thank you.
00:18:05.760 | Thank you.
00:18:06.760 | Thank you.
00:18:07.760 | Thank you.
00:18:07.760 | Thank you.
00:18:08.760 | Thank you.
00:18:09.760 | Thank you.
00:18:09.760 | Thank you.
00:18:09.760 | Thank you.
00:18:10.760 | Thank you.
00:18:11.760 | Thank you.
00:18:11.760 | Thank you.
00:18:12.760 | Thank you.
00:18:13.760 | Thank you.
00:18:13.760 | Thank you.
00:18:14.760 | Thank you.
00:18:15.760 | Thank you.
00:18:15.760 | Thank you.
00:18:16.760 | I'll be back home and we'll be back home and we'll be back home.