Large Language Models, Work and the Future of Jobs

LLMs
Work
GPT
OpenAI
Author

Lawrence Wu

Published

May 5, 2023

Last month, Allen Downey showcased the power of ChatGPT by using it to solve every problem in his Think Python books. As a result, he encouraged everyone who writes code to use LLM-assistance in their development. This inspired me to further explore the capabilities of GPT-4 and its potential effects on work and productivity.

Large Language Models (LLMs) like ChatGPT are already transforming the way we work. Even in my work as a data scientist, ChatGPT (actually GPT-4) has dramatically affected how I work and my daily tasks. Just a quick listing out of some of the prompts I’ve sent to GPT-4 in the last month:

187 packages can be updated.
27 updates are security updates.

And it responded with commands for how to update package lists, upgrade packages and do a distribution upgrade too

It’s difficult to quantify how much time GPT-4 has saved me, which it certainly has. I’d estimate it on average saves me about 1 hour of work per day. More than the time saved, the value of LLMs has been lowering the activation energy needed to get started. With data science and coding related prompts, I can arrive at answers much more directly and therefore quickly than trying to search Google and read StackOverflow answers. The code GPT-4 returns doesn’t always work the first time (zero-shot). In most cases, if an error is returned by that code, putting the error code back into the prompt will allow GPT-4 to generate correct code (one-shot, few-shot).

Along with Github Copilot, which is like autocomplete for code in an IDE like VS Code or PyCharm, I will likely never go back to coding without these assistants. They make me that much more productive. Though it’s also difficult to quantify how much more productive, 5%? 10%? 50%? There have been stories of people losing their jobs because of these technologies, but these are still relatively rare. I think the short-term impacts of these tools is making those that use them much more productive than those that don’t. Allen Downey wrote a post about LLM-assisted programming where he said:

Which brings me to what I think will be the most important skill for LLM-assisted programming: reading code. LLMs can generate code much faster than we can understand it, so the ability to read, understand, and check code will be critical.

The other skill that will become more important is meta-language, that is, the vocabulary we use to talk about programs. In my correlation in Elm example, I asked ChatGPT to “factor out the anonymous function”, and it new exactly what I meant. In general, it seems to understand the meta-language of programming well, so it will be useful if we can speak it.

I tend to agree with his points. Reading code is going to be critical. Also knowing the right terminology or “meta-language” as Downey calls it to prompt the LLM is also critical. For example, using Github Copilot I was working in a Jupyter Notebook preparing some data for a model that was in a dataframe. I wrote a comment “# pivot this from wide to long” and the LLM was able to generate the code to do exactly what I needed. This took 5 seconds instead of 30-60 seconds to Google and arrive at this SO answer. But one would need to know what terms like “pivot” and what “wide” and “long” data are (see Hadley Wickham’s Tidy Data paper to learn more).

Technological advancements inevitably change jobs and work dynamics. Some jobs may disappear, while others may evolve, and new ones will emerge. Although it’s difficult to predict the pace and extent of these changes, there already have been a few studies on the topic. But I would take these with a grain of salt given how difficult it is to forecast the future.

What is more valuable is waiting for real-world examples of how LLMs are affecting work. Here are some examples I’ve seen:

Second, and more consequentially, the AI era of computing has finally arrived. We’ve believed for many years that AI will give us new superpowers and completely transform knowledge work. And we’ve been building towards this future for a long time, as this year’s product pipeline will demonstrate.

In an ideal world, we’d simply shift people from one team to another. And we’ve done that wherever possible. However, our next stage of growth requires a different mix of skill sets, particularly in AI and early-stage product development. We’ve been bringing in great talent in these areas over the last couple years and we’ll need even more.

It’ll be certainly interesting to see how these technologies continue to evolve and how they affect work.