Harul Pandita Reviewer: Rhonda Jacobs My name is Rahul Pandita, and I am a researcher at GitHub Next. And today, we're going to talk about some of the GitHub Next explorations. Now, before we begin, who among you have heard of GitHub Next? Oh, cool. Quite a few of you. We'll make it go much easier and much faster.

All right. For those of you who don't know us, we are about 20 bunch of researchers, seniorish-level developers, and mostly tool builders who work outside of the regular product and report directly to our CEO. And that's by design. And our goal is to explore the future of software engineering like you all are doing in your day-to-day jobs.

And the reason for exploring that is that, like, once we do our explorations, we toss it on and we pass it on our learnings to the product and development teams so that they can build really compelling products like the co-pilot that you all have used, hopefully, at some point of time.

as an aside, for people who are following us on Twitter, I don't look anything like my picture over here. I'm the one in the green background. But we do have Devon in our team. He's not an automated AI. He's a very real person, and he looks exactly like the person on the top right corner on that slide.

Since we have gotten that out of the way, let's get back to the future of software engineering with regards to Gen AI. So here's what Andrew Ning, who single-handedly trained a whole generation of machine learning engineers, has to say about AI, that it's just as electricity. It's the new electricity.

It's going to transform the software development and almost every other field, just like electricity did 100 years ago. So what does that mean? Here's a picture of what a manufacturing facility looked like before electrification. There used to be a giant, mostly coal-powered steam turbine or steam engine located centrally, which used to turn these giant shafts, which would turn these auxiliary shafts, so forth and so on, and individual workers would connect to these shafts using the belt and pulley system.

Right? And these engines were like really, really huge. So it was the workers. The whole architecture of the factory was designed around this steam engine. And the whole workflow was around the steam engine. And it was the workers who were working around the technology, rather than the technology working for people.

Right? And along in 1880 came these electric motors. And they had the potential to revolutionize the manufacturing sector. Why? Because unlike steam engines or steam motors, they retained their efficiency when they were smaller. so you could basically redesign the entire factory floor plan. So you would think that, wow, this is great, and everyone would jump on this.

But it was not until 1920s where these became the mainstream. So early 1880s to late 1920s. What was happening for about these 40 years? What was happening was exploration and experimentation. People were trying to figure out how to use this technology, how to make it better, how to de-risk it to a point that the use of this technology becomes the norm rather than the exception.

And that's what we do at GitHub Next, right? Our charter is to explore the future of software engineering. And with the emphasis on the word explore, right? Because if we knew what the future of software engineering in context of AI looks like, we would just build it. That's more efficient.

But unfortunately, we do not. So what we have to resort to is exploration. We just try out different things, rapidly prototype, experiment, and figure out whether something works or not. And if it works, then we put it out in front of our customers or in users, and we learn from them, and then we finally transform into a product.

Oftentimes, an idea begins inside our next as a functional prototype, which goes through heavy dogfooding inside the next team. If it survives that, then we move on to the next level of dogfooding that is inside the company. If it survives that, then we move on to the next level, which is releasing it as a tech preview to the early adopters.

We learn from that. If it survives that, then it may have a chance to become a product like that, a product in the future. And we can kill or we can shelve any of these explorations at any point of time if we are not getting the right signal so that we can explore other areas.

We did that with the co-pilot. So, yes, co-pilot started off as a next experiment. And since that, we have created many other experiments like co-pilot for CLI, co-pilot voice, GitHub blocks, spec langs, so forth and so on. A lot of these have transformed into a product of their own.

So you can see some of them as GitHub product offerings. A lot of them have been absorbed into existing products. And you will see them as a part of the existing products. And a significant number of them have been shelved. We've learned what we learned from those experiments and figured out that this is not the right time for that kind of exploration or the exploration itself was flawed.

But we learned from them and we will keep that learning and use that in our next explorations. So there was an overview of GitHub Next. And today I'm going to talk about two specific explorations. One is the next edit suggestions in the co-pilot workspace that are currently active from GitHub Next's perspective.

And specifically, I'm going to talk about what their motivations was and how they came to be and what are the future plans for that. So, first off, co-pilot next edit suggestions. Right? So what if, it started off with this question, what if ghost text could be more intelligent? Right?

So we all know what co-pilot does. It provides you the code completions in your current context. Right? It's like really, really good at creating new code, but that's not what we all do. Right? We almost always edit existing code, which involves editing, adding, deleting lines at multiple locations in a program.

Right? What if ghost text was good at that as well? And that's what this exploration is. We call it next edit suggestion, which provides you suggestions not only at the current cursor level, but provides you suggestions what else needs to change in a program. But enough talking. Let's jump on to a demo.

Right? Here, I am going to add this parameter in this Python program. And the next edit suggestion automatically picks it up and says that, hey, you need to update your method definition. Once we update the method definition, it says that, hey, you need to add these arguments. And once that has been updated, then it will go back and say, hey, now the code document is not in line with what the code is actually doing.

And it goes ahead and edits that and updates that as well. And the same thing repeats when I add one more parameter. So that was Copilot next edit suggestions experiment. We are still not ready yet. We are still experimenting with a bunch of other stuff. Like, you know, is the ghost text completion the right modality for it?

Or do we need to figure out a different way of presenting those suggestions? What if the location of the next edit is not visible in the current viewport? Or what if the location is in a file that is not even open in an editor? Most importantly, we are also working on fine-tuning the models specifically for this use case.

The idea being that, like, if we want the next edit suggestions to be accurate and we want it to be very useful, then the suggestions needs to be on point. And once we are done with these further sub-explorations and we feel that it has gotten through our internal dog fooding standard, Next edit suggestions would be coming out either as a standalone tech preview from Next or as a part of an existing Next product sometime in your IDE in the next few months.

All right. So there was code completions. But let's move from the code completions to the task completions land. Why do we ask? Why move from the task completions? It just turns out that while code is, like, an important artifact that comes out of software development, but it's not the only artifact.

Software development involves this inner loop where you begin with a task. The idea is, like, what am I supposed to do? How am I? What is the specific thing that I'm trying to do? And followed by, how do I go about doing that thing? What are the frameworks that are at my disposal?

What are the programming languages that are at my disposal? What is the existing code that's there? How do I write a new code that is consistent with those codes? So that becomes a sort of a specification. And once you understand where you are, then you sort of try to decide, like, where am I going with it?

Like, how does the final product look like? Once you have zeroed in on that, then you go about what specific file changes do I need to make to get to that final product? And that sort of becomes a plan. And once you get to the plan, then you go to the implementation part.

And that forms this loop of software development. And we call it inner loop. And we would like the AI to be helpful in all those aspects of that inner loop. And that's why we built co-pilot workspace. And mind you, like all Nix explorations, it did not start as co-pilot workspace.

It started as individual explorations. For instance, we started to figure out, can we use natural language as a functional specification of programs? So there is a spec-lang exploration. In parallel, we were trying to figure out if we can improve the code completions by prompting the model with the runtime information.

And all of those things combined and with the user feedback combined into this one bigger exploration called co-pilot workspace. And we were also talking to our users. Like, we wanted to talk to developers and we wanted to ask that, hey, we are building this thing. How would you like AI to support you?

What are your major pain points? And a few things became very, very clear while talking to our users. Right? So first thing is that the most difficulty that people faced was getting started on a task. Like, how do I -- I know that an issue is assigned to me.

How do I get started on it? Followed by, how do I trust the output of the AI? I don't trust it. And more importantly, they figured out that problem solving is what software development is about. And they would like to retain that problem solving aspects of it. And they would like the help of AI in the form of a thought partner or a sparring partner or a second brain which they can collaborate with to solve a problem.

And lastly and most importantly, they would like to retain control. Developers are in control, not the other way around. And with this feedback, we build Copilot workspace. So what is it? It allows you to -- it simplifies getting started. So one-click proposal on your tasks. It has a built-in runtime that allows you to quickly verify what the code that has been provided by the AI.

It has an environment which is built for iteration. So if you feel that AI is going in the wrong direction, you can just go and quickly correct it. And most importantly, it is designed for collaboration. So you can just share your code or your work as a part of the GitHub pull request.

Or you can share your work or share your workspace with your colleagues if you're not comfortable with it. But let's -- enough talking. Let's just get into a demo about it, right? So this is Monospace, which is another GitHub exploration. So if we are to write code, let's write code in style.

And these are the four -- is a family of Monospace fonts that has been released by GitHub. And this is a website that outlines a bunch of features of these fonts. And over here -- somewhere over here -- is this playground which says that -- that here are how the syntax highlighting looks across different languages.

Notice that it is missing Rust. And Rust appears to be the next cool thing that all the cool kids are doing. So we would like to update this Monospace website with a Rust example as well. So how do I get started? So I've created this issue. Or somebody has created this issue.

It just happens to be me for the purpose of this demo that I would like to create -- I would like to add a Rust example to the font playground. And I can just click this button over here. And it will open the Copilot workspace for me. And through the magic of caching, you can see that it quickly generates the specification -- current specification and the proposed specification.

Why caching? Because I had to finish this demo in time. But trust me, it's not a matter of hours. It does happen in a matter of minutes. Right? And for those of you who are interested, I would like to do a live demo for you in the Microsoft booth after this task.

All right. So what is the current specification? It just goes and figures out, does the website have this playground that contains a Rust package? And it says it doesn't. And it goes to the target state. Where would the target -- what does the target state look like? And it says that, yes, the website will have the specific package for syntax highlighting.

The website will have this package in package.json. And then I will update a bunch of other files. It looks nice. And I'll go and generate a plan for it. Again, through the magic of caching, a plan has been generated. And it will tell you that these three files need to be updated.

And I will -- it appears that this seems to be at the right level of modality. Then I will go ahead and implement it. And yes, magic of caching again. What we see is the files that are over here. Now, this seems nice. And -- but what about the iterator part?

What you can do is, at any given point of time, if you feel that something is not right, you can just go ahead and say that, okay, add Rust to the language mappings and say, add code documentation. And you can edit at any given point of time. And what you can also do is that you can edit via chat over here.

And you can say that, hey, I want to edit this one specific location. How do I go about doing this? I'm not going to do this because it's going to go through the whole iteration loop. And then the illusion of the caching will break. And it will take a lot of time.

But I would like to show that in live demos afterwards. But how do I trust whether this is, in fact, the right thing? So I will open up this integrated terminal. And I will say, install and run this repo. All right. So what's going to happen is that a suggestion is going to load.

And apparently, not the right thing. But I can quickly go and edit it and say that, all right, this is the command that I'm specifically looking for. And I can go and run. Now, this will run this command in an actual terminal. And we'll see the output in some point of time.

And you can see that, actually, this code does compile. What we also have is a preview. What we can do is open the live preview. I don't trust it. It will say that it's just going to be a second, but it takes longer than that while that loads. What are the other things?

One of the things that you would say is that, hey, you wrote a very simple command in the terminal. You said npm. You could actually type that thing in the terminal. And yes, you're right. I can type that thing. But think about that in a mobile setting, when you can open Copilot workspace on your phone.

It becomes very tedious to type those symbols, right? And if you have used the mobile keyboard, it's not very useful for that. So that's why we use this natural language way of writing these commands in the terminal. So that it can help you when you're on the go. It can synthesize those commands.

And hopefully, the website has loaded. And there is a Rust example. Right? Cool. There was a demo. Thank you. We are working -- we are not stopping there. We are working on a bunch of these improvements. And I can talk about these improvements on one-on-one basis with you. And you already saw some of the improvements, like the runtime support to synthesize the terminal commands and faster file completions using -- to make the Copilot workspace better.

But there are other next explorations that are also active. Like, how do we rethink the developer learning with AI? And how does the code review change if majority of the code that is now being written is by AI? So what does that mean? And some of these explorations will work out.

And some of these explorations you will see as tech previews. And some of these explorations will kill because we don't know where they're going. So in summary, I'm saying that we do not know what the future of AI is. But what we know is explorations is the way to get it.

And with all your help, we'll jointly explore the space so that we don't have to wait. Like electricity, we don't have to wait for 40 years to get to a place where -- to get to a place with software development where we enjoy the benefits of AI. You have been a lovely audience.

That is my time. I really appreciate you. And if you have more questions, if you want to have live demos, I'm available in the Microsoft booth in, like, two salons over that side. Thank you so much. Thank you so much. Thank you. Thank you. Thank you. Thank you. Thank you.

Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. you you