The Code AI Maturity Model and What It Means For You: Ado Kukic
00:00:00.000 | Hello and welcome to my talk on how we're thinking about the levels of code AI. My name is Otto
00:00:21.280 | Kukic and I am the director of DevRel at Sourcegraph. At Sourcegraph, we're building Kodi,
00:00:27.460 | the only AI coding assistant that knows your entire code base. To help educate our customers
00:00:34.700 | and users, as well as shape our thinking of code AI, we've been using a concept that we call
00:00:40.820 | levels of code AI internally. These levels have really resonated with our community,
00:00:46.720 | so we wanted to publicize them and start a conversation with a broader developer community
00:00:52.240 | and we're better to do it than at the AI Engineer Summit. When we talk about code AI,
00:00:58.060 | we refer to software that builds software. Today, 92% of developers are using code AI tools,
00:01:06.580 | whereas this number was just 1% a year ago. Our founder and CEO, Quinn Slack, has shared a bold
00:01:14.360 | prediction that in 5 years, 99% of code will be written by AI. While we await that future,
00:01:22.320 | let's talk about how we see the levels of code AI today. We see 6 distinct levels across 3 different
00:01:30.040 | categories. Human-initiated, where humans are the primary coders, AI-initiated, where AI starts to take
00:01:38.900 | a proactive role in software development, and AI-led code where AI has full autonomy over a code base.
00:01:45.900 | We'll contrast these levels of code with the SAE levels of autonomy for vehicles. Let's dive in.
00:01:53.500 | At level 0, the developer writes all code manually without any AI assistance. The developer is responsible
00:02:02.520 | for writing, testing, and debugging a code base. AI does not generate or modify any part of the code base,
00:02:09.640 | but IDE features like symbol name completion can provide a bit of assistance. This level reflects
00:02:16.700 | the traditional software development process before introducing any AI assistance into the development
00:02:22.880 | workflow. A vehicle operating at level 0 is fully reliant on the human driver for acceleration,
00:02:29.960 | steering, braking, and everything in between. At level 1, the developer begins to use AI that can
00:02:37.540 | generate single lines or whole blocks of code based on developer intent. For example, a developer might
00:02:44.060 | write the signature of a function, and the AI will infer the context and generate the implementation details
00:02:50.580 | for said function. At level 1, the AI assistant has been trained on millions of lines of open source code
00:02:57.660 | code and can leverage this to provide superior completions based on the developer's guidance.
00:03:03.180 | SAE level 1 vehicles still require the full attention of the human driver, but offer features such as
00:03:11.180 | cruise control or lane centering that make driving an easier, safer, and more comfortable experience.
00:03:17.240 | SAE level 2, the AI coding assistant has superior understanding and context of the code base it is
00:03:26.400 | interacting with. Where at level 1, the context is broad and general, a level 2 AI coding assistant has
00:03:34.080 | specific context about the code base that it is working in. This allows the AI assistant to make better
00:03:40.540 | other suggestions for code completions. For example, if you are working in a Node.js code base and were using the
00:03:47.340 | Axiom library to handle HTTP requests, a level 2 AI assistant would provide autocomplete suggestions based on the
00:03:54.940 | Axiom library as opposed to a different node HTTP library like fetch or super agent. At SAE level 2, we get partial automation. The human driver is still in control and can override anything the car does at any time.
00:04:02.540 | At SAE level 2, we get partial automation. The human driver is still in control and can override
00:04:08.540 | anything the car does at any time, but features such as traffic-aware cruise control or automatic lane changes can make driving a much smoother experience.
00:04:18.540 | At level 3, the developer provides high-level requirements and the AI assistant delivers a code-based solution.
00:04:26.540 | The AI coding assistant goes beyond generating singular snippets of code to building out full components
00:04:32.460 | and even integrations with other pieces of software. Rather than writing the code themselves,
00:04:38.220 | a developer could instruct a level 3 code AI assistant to add a user authentication to an application that they are building,
00:04:45.580 | and the coding assistant would generate all of the code required. The coding assistant could then explain to the developer the code it wrote,
00:04:53.820 | how it works, and how it integrates with the rest of the application. SAE level 3 is also the first level where the vehicle itself takes on the primary
00:05:02.380 | role of driving, with the human driver being a fallback in case the vehicle cannot drive itself
00:05:08.140 | safely. The vehicle can perform most of the driving tasks, but may encounter situations where it cannot
00:05:14.620 | adequately perform these tasks, so it's forced to give control back to the human driver.
00:05:20.140 | At level 4, the code AI assistant can proactively handle coding tasks without developer oversight.
00:05:25.900 | Let's imagine a few scenarios where a level 4 code AI assistant would play a role.
00:05:32.140 | A level 4 capable code AI assistant could continuously monitor your code changes and autonomously submit PRs
00:05:39.900 | to ensure your documentation stays up to date. Even better, the coding assistant could monitor bug reports from customers and submit PRs to ensure your documentation stays up to date.
00:05:42.140 | Even better, the coding assistant could monitor bug reports from customers and submit PRs to fix the issues. The human developer could then simply review the pull requests and merge them.
00:05:54.140 | Level 4 SAE vehicles can perform virtually all driving tasks under specific conditions.
00:06:02.140 | For example, Waymo operates a fleet of fully automated self-driving taxis in cities where they have high-quality mapping data and can provide a safe driving experience for passengers without human drivers.
00:06:12.140 | A customer simply hails a Waymo taxi using a mobile app, provides a destination, and the vehicle is responsible for taking the passenger to their final destination without any additional human input.
00:06:30.140 | At level 5, the AI assistant requires minimal human guidance on cogeneration and is capable of handling the entire software development lifecycle.
00:06:40.140 | The developer provides high-level requirements and specifications. The AI then designs the architecture, writes production quality code, handles deployment, and continuously improves the code base.
00:06:54.140 | The developer's role is to validate that the end product meets the stated requirements, but the developer does not necessarily look at the generated code.
00:07:03.140 | The code AI assistant has complete autonomy to take code from concept to production.
00:07:10.140 | A self-driving car capable of level 5 driving automation can perform all driving tasks under all conditions, humans optional.
00:07:20.140 | The car is responsible for making all the decisions. At this level, a steering wheel or any ability for a human to override the car is unnecessary.
00:07:30.140 | So there you have it. The six levels of code AI, or at least how we're thinking about them at Sourcegraph.
00:07:36.140 | Do you agree? Disagree? We'd love to hear your thoughts. Find us at booth G5 and let's chat.
00:07:43.140 | And if you'd like to try Kodi for yourself, get it for your IDE of choice at Kodi.dev.
00:07:49.140 | Thank you and I'll see you on the show floor.
00:07:51.140 | Thank you and I'll see you on the floor.
00:07:52.140 | Thank you.
00:07:54.140 | We'll see you next time.