back to index

GitHub Next Explorations: Rahul Pandita


Whisper Transcript | Transcript Only Page

00:00:00.000 | Harul Pandita Reviewer: Rhonda Jacobs
00:00:07.000 | My name is Rahul Pandita, and I am a researcher at GitHub Next.
00:00:19.000 | And today, we're going to talk about some of the GitHub Next explorations.
00:00:23.000 | Now, before we begin, who among you have heard of GitHub Next?
00:00:27.000 | Oh, cool. Quite a few of you.
00:00:30.000 | We'll make it go much easier and much faster.
00:00:33.000 | All right.
00:00:34.000 | For those of you who don't know us,
00:00:36.000 | we are about 20 bunch of researchers, seniorish-level developers,
00:00:42.000 | and mostly tool builders who work outside of the regular product
00:00:49.000 | and report directly to our CEO.
00:00:52.000 | And that's by design.
00:00:53.000 | And our goal is to explore the future of software engineering
00:00:57.000 | like you all are doing in your day-to-day jobs.
00:01:00.000 | And the reason for exploring that is that, like, once we do our explorations,
00:01:06.000 | we toss it on and we pass it on our learnings to the product and development teams
00:01:11.000 | so that they can build really compelling products like the co-pilot
00:01:14.000 | that you all have used, hopefully, at some point of time.
00:01:17.000 | as an aside, for people who are following us on Twitter, I don't look anything like my picture over here.
00:01:24.000 | I'm the one in the green background.
00:01:26.000 | But we do have Devon in our team.
00:01:28.000 | He's not an automated AI.
00:01:29.000 | He's a very real person, and he looks exactly like the person on the top right corner on that slide.
00:01:34.000 | Since we have gotten that out of the way, let's get back to the future of software engineering with regards to Gen AI.
00:01:44.000 | So here's what Andrew Ning, who single-handedly trained a whole generation of machine learning engineers,
00:01:52.000 | has to say about AI, that it's just as electricity.
00:01:57.000 | It's the new electricity.
00:01:58.000 | It's going to transform the software development and almost every other field,
00:02:02.000 | just like electricity did 100 years ago.
00:02:05.000 | So what does that mean?
00:02:07.000 | Here's a picture of what a manufacturing facility looked like before electrification.
00:02:14.000 | There used to be a giant, mostly coal-powered steam turbine or steam engine located centrally,
00:02:21.000 | which used to turn these giant shafts, which would turn these auxiliary shafts, so forth and so on,
00:02:28.000 | and individual workers would connect to these shafts using the belt and pulley system.
00:02:32.000 | Right?
00:02:33.000 | And these engines were like really, really huge.
00:02:35.000 | So it was the workers.
00:02:37.000 | The whole architecture of the factory was designed around this steam engine.
00:02:41.000 | And the whole workflow was around the steam engine.
00:02:44.000 | And it was the workers who were working around the technology,
00:02:50.000 | rather than the technology working for people.
00:02:52.000 | Right?
00:02:53.000 | And along in 1880 came these electric motors.
00:02:57.000 | And they had the potential to revolutionize the manufacturing sector.
00:03:03.000 | Why?
00:03:04.000 | Because unlike steam engines or steam motors, they retained their efficiency when they were smaller.
00:03:11.000 | so you could basically redesign the entire factory floor plan.
00:03:16.000 | So you would think that, wow, this is great, and everyone would jump on this.
00:03:20.000 | But it was not until 1920s where these became the mainstream.
00:03:26.000 | So early 1880s to late 1920s.
00:03:29.000 | What was happening for about these 40 years?
00:03:32.000 | What was happening was exploration and experimentation.
00:03:35.000 | People were trying to figure out how to use this technology, how to make it better,
00:03:41.000 | how to de-risk it to a point that the use of this technology becomes the norm rather than the exception.
00:03:50.000 | And that's what we do at GitHub Next, right?
00:03:53.000 | Our charter is to explore the future of software engineering.
00:03:56.000 | And with the emphasis on the word explore, right?
00:03:59.000 | Because if we knew what the future of software engineering in context of AI looks like,
00:04:03.000 | we would just build it.
00:04:04.000 | That's more efficient.
00:04:05.000 | But unfortunately, we do not.
00:04:07.000 | So what we have to resort to is exploration.
00:04:10.000 | We just try out different things, rapidly prototype, experiment,
00:04:13.000 | and figure out whether something works or not.
00:04:15.000 | And if it works, then we put it out in front of our customers or in users,
00:04:20.000 | and we learn from them, and then we finally transform into a product.
00:04:24.000 | Oftentimes, an idea begins inside our next as a functional prototype,
00:04:30.000 | which goes through heavy dogfooding inside the next team.
00:04:34.000 | If it survives that, then we move on to the next level of dogfooding that is inside the company.
00:04:39.000 | If it survives that, then we move on to the next level,
00:04:42.000 | which is releasing it as a tech preview to the early adopters.
00:04:46.000 | We learn from that.
00:04:47.000 | If it survives that, then it may have a chance to become a product like that,
00:04:51.000 | a product in the future.
00:04:52.000 | And we can kill or we can shelve any of these explorations at any point of time
00:04:56.000 | if we are not getting the right signal so that we can explore other areas.
00:05:01.000 | We did that with the co-pilot.
00:05:03.000 | So, yes, co-pilot started off as a next experiment.
00:05:06.000 | And since that, we have created many other experiments like co-pilot for CLI,
00:05:11.000 | co-pilot voice, GitHub blocks, spec langs, so forth and so on.
00:05:15.000 | A lot of these have transformed into a product of their own.
00:05:18.000 | So you can see some of them as GitHub product offerings.
00:05:21.000 | A lot of them have been absorbed into existing products.
00:05:26.000 | And you will see them as a part of the existing products.
00:05:29.000 | And a significant number of them have been shelved.
00:05:32.000 | We've learned what we learned from those experiments
00:05:34.000 | and figured out that this is not the right time for that kind of exploration
00:05:37.000 | or the exploration itself was flawed.
00:05:39.000 | But we learned from them and we will keep that learning
00:05:42.000 | and use that in our next explorations.
00:05:49.000 | So there was an overview of GitHub Next.
00:05:51.000 | And today I'm going to talk about two specific explorations.
00:05:54.000 | One is the next edit suggestions in the co-pilot workspace
00:05:57.000 | that are currently active from GitHub Next's perspective.
00:06:01.000 | And specifically, I'm going to talk about what their motivations was
00:06:05.000 | and how they came to be and what are the future plans for that.
00:06:09.000 | So, first off, co-pilot next edit suggestions.
00:06:13.000 | Right?
00:06:14.000 | So what if, it started off with this question,
00:06:16.000 | what if ghost text could be more intelligent?
00:06:19.000 | Right?
00:06:20.000 | So we all know what co-pilot does.
00:06:22.000 | It provides you the code completions in your current context.
00:06:26.000 | Right?
00:06:27.000 | It's like really, really good at creating new code,
00:06:30.000 | but that's not what we all do.
00:06:32.000 | Right?
00:06:33.000 | We almost always edit existing code,
00:06:37.000 | which involves editing, adding, deleting lines
00:06:40.000 | at multiple locations in a program.
00:06:42.000 | Right?
00:06:43.000 | What if ghost text was good at that as well?
00:06:46.000 | And that's what this exploration is.
00:06:48.000 | We call it next edit suggestion,
00:06:50.000 | which provides you suggestions not only at the current cursor level,
00:06:53.000 | but provides you suggestions what else needs to change in a program.
00:06:57.000 | But enough talking.
00:06:59.000 | Let's jump on to a demo.
00:07:02.000 | Right?
00:07:03.000 | Here, I am going to add this parameter in this Python program.
00:07:07.000 | And the next edit suggestion automatically picks it up
00:07:09.000 | and says that, hey, you need to update your method definition.
00:07:12.000 | Once we update the method definition,
00:07:14.000 | it says that, hey, you need to add these arguments.
00:07:18.000 | And once that has been updated, then it will go back and say, hey,
00:07:22.000 | now the code document is not in line with what the code is actually doing.
00:07:27.000 | And it goes ahead and edits that and updates that as well.
00:07:31.000 | And the same thing repeats when I add one more parameter.
00:07:39.000 | So that was Copilot next edit suggestions experiment.
00:07:45.000 | We are still not ready yet.
00:07:47.000 | We are still experimenting with a bunch of other stuff.
00:07:51.000 | Like, you know, is the ghost text completion the right modality for it?
00:07:56.000 | Or do we need to figure out a different way of presenting those suggestions?
00:08:01.000 | What if the location of the next edit is not visible in the current viewport?
00:08:06.000 | Or what if the location is in a file that is not even open in an editor?
00:08:11.000 | Most importantly, we are also working on fine-tuning the models specifically for this use case.
00:08:16.000 | The idea being that, like, if we want the next edit suggestions to be accurate
00:08:20.000 | and we want it to be very useful, then the suggestions needs to be on point.
00:08:25.000 | And once we are done with these further sub-explorations
00:08:29.000 | and we feel that it has gotten through our internal dog fooding standard,
00:08:33.000 | Next edit suggestions would be coming out either as a standalone tech preview from Next
00:08:38.000 | or as a part of an existing Next product sometime in your IDE in the next few months.
00:08:44.000 | All right.
00:08:46.000 | So there was code completions.
00:08:48.000 | But let's move from the code completions to the task completions land.
00:08:51.000 | Why do we ask?
00:08:52.000 | Why move from the task completions?
00:08:54.000 | It just turns out that while code is, like, an important artifact
00:09:00.000 | that comes out of software development,
00:09:02.000 | but it's not the only artifact.
00:09:04.000 | Software development involves this inner loop where you begin with a task.
00:09:08.000 | The idea is, like, what am I supposed to do?
00:09:10.000 | How am I?
00:09:11.000 | What is the specific thing that I'm trying to do?
00:09:14.000 | And followed by, how do I go about doing that thing?
00:09:18.000 | What are the frameworks that are at my disposal?
00:09:20.000 | What are the programming languages that are at my disposal?
00:09:23.000 | What is the existing code that's there?
00:09:27.000 | How do I write a new code that is consistent with those codes?
00:09:30.000 | So that becomes a sort of a specification.
00:09:32.000 | And once you understand where you are, then you sort of try to decide, like, where am I going with it?
00:09:37.000 | Like, how does the final product look like?
00:09:39.000 | Once you have zeroed in on that, then you go about what specific file changes do I need to make to get to that final product?
00:09:47.000 | And that sort of becomes a plan.
00:09:49.000 | And once you get to the plan, then you go to the implementation part.
00:09:52.000 | And that forms this loop of software development.
00:09:55.000 | And we call it inner loop.
00:09:56.000 | And we would like the AI to be helpful in all those aspects of that inner loop.
00:10:01.000 | And that's why we built co-pilot workspace.
00:10:04.000 | And mind you, like all Nix explorations, it did not start as co-pilot workspace.
00:10:08.000 | It started as individual explorations.
00:10:10.000 | For instance, we started to figure out, can we use natural language as a functional specification of programs?
00:10:16.000 | So there is a spec-lang exploration.
00:10:18.000 | In parallel, we were trying to figure out if we can improve the code completions by prompting the model with the runtime information.
00:10:26.000 | And all of those things combined and with the user feedback combined into this one bigger exploration called co-pilot workspace.
00:10:33.000 | And we were also talking to our users.
00:10:36.000 | Like, we wanted to talk to developers and we wanted to ask that, hey, we are building this thing.
00:10:40.000 | How would you like AI to support you?
00:10:42.000 | What are your major pain points?
00:10:44.000 | And a few things became very, very clear while talking to our users.
00:10:48.000 | Right?
00:10:49.000 | So first thing is that the most difficulty that people faced was getting started on a task.
00:10:54.000 | Like, how do I -- I know that an issue is assigned to me.
00:10:57.000 | How do I get started on it?
00:10:58.000 | Followed by, how do I trust the output of the AI?
00:11:01.000 | I don't trust it.
00:11:02.000 | And more importantly, they figured out that problem solving is what software development is about.
00:11:07.000 | And they would like to retain that problem solving aspects of it.
00:11:11.000 | And they would like the help of AI in the form of a thought partner or a sparring partner or a second brain which they can collaborate with to solve a problem.
00:11:19.000 | And lastly and most importantly, they would like to retain control.
00:11:23.000 | Developers are in control, not the other way around.
00:11:26.000 | And with this feedback, we build Copilot workspace.
00:11:31.000 | So what is it?
00:11:32.000 | It allows you to -- it simplifies getting started.
00:11:35.000 | So one-click proposal on your tasks.
00:11:38.000 | It has a built-in runtime that allows you to quickly verify what the code that has been provided by the AI.
00:11:45.000 | It has an environment which is built for iteration.
00:11:48.000 | So if you feel that AI is going in the wrong direction, you can just go and quickly correct it.
00:11:52.000 | And most importantly, it is designed for collaboration.
00:11:54.000 | So you can just share your code or your work as a part of the GitHub pull request.
00:11:59.000 | Or you can share your work or share your workspace with your colleagues if you're not comfortable with it.
00:12:05.000 | But let's -- enough talking.
00:12:08.000 | Let's just get into a demo about it, right?
00:12:11.000 | So this is Monospace, which is another GitHub exploration.
00:12:14.000 | So if we are to write code, let's write code in style.
00:12:17.000 | And these are the four -- is a family of Monospace fonts that has been released by GitHub.
00:12:21.000 | And this is a website that outlines a bunch of features of these fonts.
00:12:27.000 | And over here -- somewhere over here -- is this playground which says that --
00:12:32.000 | that here are how the syntax highlighting looks across different languages.
00:12:37.000 | Notice that it is missing Rust.
00:12:39.000 | And Rust appears to be the next cool thing that all the cool kids are doing.
00:12:43.000 | So we would like to update this Monospace website with a Rust example as well.
00:12:47.000 | So how do I get started?
00:12:49.000 | So I've created this issue.
00:12:52.000 | Or somebody has created this issue.
00:12:54.000 | It just happens to be me for the purpose of this demo that I would like to create --
00:12:58.000 | I would like to add a Rust example to the font playground.
00:13:01.000 | And I can just click this button over here.
00:13:05.000 | And it will open the Copilot workspace for me.
00:13:09.000 | And through the magic of caching, you can see that it quickly generates the specification --
00:13:14.000 | current specification and the proposed specification.
00:13:17.000 | Why caching?
00:13:19.000 | Because I had to finish this demo in time.
00:13:21.000 | But trust me, it's not a matter of hours.
00:13:23.000 | It does happen in a matter of minutes.
00:13:26.000 | Right?
00:13:27.000 | And for those of you who are interested, I would like to do a live demo for you
00:13:30.000 | in the Microsoft booth after this task.
00:13:32.000 | All right.
00:13:33.000 | So what is the current specification?
00:13:34.000 | It just goes and figures out, does the website have this playground that contains a Rust package?
00:13:41.000 | And it says it doesn't.
00:13:42.000 | And it goes to the target state.
00:13:44.000 | Where would the target -- what does the target state look like?
00:13:48.000 | And it says that, yes, the website will have the specific package for syntax highlighting.
00:13:52.000 | The website will have this package in package.json.
00:13:55.000 | And then I will update a bunch of other files.
00:13:58.000 | It looks nice.
00:13:59.000 | And I'll go and generate a plan for it.
00:14:01.000 | Again, through the magic of caching, a plan has been generated.
00:14:04.000 | And it will tell you that these three files need to be updated.
00:14:07.000 | And I will -- it appears that this seems to be at the right level of modality.
00:14:11.000 | Then I will go ahead and implement it.
00:14:13.000 | And yes, magic of caching again.
00:14:16.000 | What we see is the files that are over here.
00:14:19.000 | Now, this seems nice.
00:14:23.000 | And -- but what about the iterator part?
00:14:25.000 | What you can do is, at any given point of time, if you feel that something is not right,
00:14:28.000 | you can just go ahead and say that, okay, add Rust to the language mappings and say,
00:14:34.000 | add code documentation.
00:14:40.000 | And you can edit at any given point of time.
00:14:43.000 | And what you can also do is that you can edit via chat over here.
00:14:46.000 | And you can say that, hey, I want to edit this one specific location.
00:14:49.000 | How do I go about doing this?
00:14:52.000 | I'm not going to do this because it's going to go through the whole iteration loop.
00:14:55.000 | And then the illusion of the caching will break.
00:14:57.000 | And it will take a lot of time.
00:14:59.000 | But I would like to show that in live demos afterwards.
00:15:02.000 | But how do I trust whether this is, in fact, the right thing?
00:15:06.000 | So I will open up this integrated terminal.
00:15:09.000 | And I will say, install and run this repo.
00:15:18.000 | All right.
00:15:19.000 | So what's going to happen is that a suggestion is going to load.
00:15:24.000 | And apparently, not the right thing.
00:15:26.000 | But I can quickly go and edit it and say that, all right, this is the command that I'm specifically
00:15:32.000 | looking for.
00:15:33.000 | And I can go and run.
00:15:35.000 | Now, this will run this command in an actual terminal.
00:15:38.000 | And we'll see the output in some point of time.
00:15:41.000 | And you can see that, actually, this code does compile.
00:15:46.000 | What we also have is a preview.
00:15:49.000 | What we can do is open the live preview.
00:15:51.000 | I don't trust it.
00:15:53.000 | It will say that it's just going to be a second, but it takes longer than that while that loads.
00:15:58.000 | What are the other things?
00:15:59.000 | One of the things that you would say is that, hey, you wrote a very simple command in the terminal.
00:16:03.000 | You said npm.
00:16:04.000 | You could actually type that thing in the terminal.
00:16:06.000 | And yes, you're right.
00:16:07.000 | I can type that thing.
00:16:08.000 | But think about that in a mobile setting, when you can open Copilot workspace on your phone.
00:16:14.000 | It becomes very tedious to type those symbols, right?
00:16:17.000 | And if you have used the mobile keyboard, it's not very useful for that.
00:16:22.000 | So that's why we use this natural language way of writing these commands in the terminal.
00:16:28.000 | So that it can help you when you're on the go.
00:16:31.000 | It can synthesize those commands.
00:16:33.000 | And hopefully, the website has loaded.
00:16:35.000 | And there is a Rust example.
00:16:38.000 | Right?
00:16:39.000 | Cool.
00:16:40.000 | There was a demo.
00:16:43.000 | Thank you.
00:16:44.000 | We are working -- we are not stopping there.
00:16:49.000 | We are working on a bunch of these improvements.
00:16:50.000 | And I can talk about these improvements on one-on-one basis with you.
00:16:53.000 | And you already saw some of the improvements, like the runtime support to synthesize the terminal
00:16:58.000 | commands and faster file completions using -- to make the Copilot workspace better.
00:17:04.000 | But there are other next explorations that are also active.
00:17:07.000 | Like, how do we rethink the developer learning with AI?
00:17:09.000 | And how does the code review change if majority of the code that is now being written is by AI?
00:17:14.000 | So what does that mean?
00:17:15.000 | And some of these explorations will work out.
00:17:18.000 | And some of these explorations you will see as tech previews.
00:17:20.000 | And some of these explorations will kill because we don't know where they're going.
00:17:23.000 | So in summary, I'm saying that we do not know what the future of AI is.
00:17:29.000 | But what we know is explorations is the way to get it.
00:17:32.000 | And with all your help, we'll jointly explore the space so that we don't have to wait.
00:17:37.000 | Like electricity, we don't have to wait for 40 years to get to a place where -- to get to a place
00:17:42.000 | with software development where we enjoy the benefits of AI.
00:17:45.000 | You have been a lovely audience.
00:17:47.000 | That is my time.
00:17:48.000 | I really appreciate you.
00:17:49.000 | And if you have more questions, if you want to have live demos,
00:17:52.000 | I'm available in the Microsoft booth in, like, two salons over that side.
00:17:57.000 | Thank you so much.
00:17:58.000 | Thank you so much.
00:17:59.000 | Thank you.
00:18:00.000 | Thank you.
00:18:01.000 | Thank you.
00:18:02.000 | Thank you.
00:18:03.000 | Thank you.
00:18:04.000 | Thank you.
00:18:05.000 | Thank you.
00:18:06.000 | Thank you.
00:18:07.000 | Thank you.
00:18:08.000 | Thank you.
00:18:09.000 | Thank you.
00:18:10.000 | Thank you.
00:18:11.000 | Thank you.
00:18:12.000 | Thank you.
00:18:12.000 | Thank you.
00:18:13.000 | Thank you.
00:18:13.000 | you
00:18:15.060 | you