The annual State of AI report is one of the most cited documents in the field of artificial intelligence endorsed by the likes of Andrej Karpathy. Led by Nathan Benet this year, the original co-author Ian Hogarth is now one of the key players at the Bletchley Global AI Summit in November.

Yes, I read all 163 pages of this year's edition and thought that giving you just the highlights interspersed with developments from just the last few days like Eureka would be a good way of keeping you guys up to speed. But let's start out with a modality that the report didn't cover.

Notice on page 7 we have text, image, video, music, robot states and we'll cover all of those but what we don't have is thoughts. And why am I bringing that up you might ask? Well because a few days ago we had this from Meta: "Towards a real-time decoding of images from brain activity".

Now I have showcased thought to image and thought to text before on the channel but never real-time thought to image. It's not perfect but progress is astonishing and I think it points to the explosion of modalities that we are seeing at the moment in AI. Now if you're willing to sacrifice a real-time flow of images from the brain this is the state of the art using fMRI.

Much more accurate but not real-time. Now many of you might wonder if it works if someone imagines an image and apparently it doesn't work nearly as well. And also at the moment it tends not to be a real-time decoding. Also at the moment it tends not to work if the participants are distracted by other things.

As the paper says: "In other words the subject's consent is not only a legal but also a technical requirement for brain decoding". Back to the report and many pages are dedicated to covering GPT-4 but to be honest anyone who's followed this channel for any length of time will know a fair bit more than what is covered in these few slides.

So I'm going to skip to page 14 where I have a slight difference with one of the conclusions. They cite a paper which describes the false promise of imitating proprietary LLMs, large language models. Basically saying that you can imitate the style but not the content of a smarter model.

But I wish that they had cited Orca which despite being more than 10 times smaller than the original ChachiPT reaches according to the technical paper parity with ChachiPT on various benchmarks showing only a four-point gap in professional and academic examinations like the SAT, LSAT, GRE and GMAT. That's maybe why the information reported on Microsoft trying to substitute for GPT-4 with Orca.

Claiming to mimic the quality of OpenAI software at a fraction of the cost. On page 38 of the report they describe open-ended learning with large language models. Describing how they can explore and gain skills in a game like Minecraft. And they say that the best example of this kind of self-improvement and iterative prompting is Voyager.

Which is again GPT-4 getting better and better at Minecraft. But just after the report came out we got Eureka. And yes it deserves its own video but as Jim Phan, one of the key authors said, it's like Voyager in the space of physics simulations. And essentially what they did was feed the source code of the environment to GPT-4.

And then they asked it to write the code for the reward function. And then using the sheer power of parallelization inside Nvidia's Isaac Jim. They could then test out those reward functions. See which ones in simulation work the best. Thanks to GPU acceleration you could speed up reality a thousand x.

Each time the best grading systems would be used, refined upon and then tested again. All in simulation. Ultimately Eureka rewards outperformed expert human written reward functions on 83% of the tasks performed. The average margin was 52% and Eureka was able to learn even pin spinning tricks. Which are notoriously difficult even for CGI artists.

This marrying up of large language models and iterative feedback from the environment is probably the future of AI. As Shetal Shah from Microsoft said, the proverbial positive feedback loop of self-improvement might be just around the corner. That allows us to go beyond human-made training data and capabilities. And to those wondering, yes they are planning to connect it to a real robotic hand soon.

I was lucky enough to get early access to the report and one of the most convincing and amusing demonstrations was Eureka learning how to run efficiently. Back to the report though and if Eureka was all about simulated embodiment. Getting better with language feedback. How about language tasks being improved with robotics data?

As this page demonstrates the positive transfer seems to go both ways. Gaining vision data and embodiment or robotics data seems to make a model like Palm-E better at pure language tasks. And while the report went on to mention RT2, watchers of my channel will know about RT2X. That showed how data from a multiplicity of different robotic setups could complement each other and improve the base model.

The basic message across the board is the same. More data from more modalities in the same modality. And so when we get Eureka improving via feedback from GPT-4, it reminds me of a previous video where I talked about DALI-3 improving from feedback from GPT-4 vision. The same themes are reoccurring.

Improvements in one modality ricocheting across two others. And remember the original Palm language model and GPT-4 weren't even designed for this. I am genuinely curious about the kind of recursive improvement that might occur with a model designed to be multimodal from the start like Gemini. Speaking of robotics though, the report did mention something I missed on the channel.

That's the first time win for a robot in a competitive sport, which was first person view drone racing. Here is a glimpse of why the Swift system beat all the human champions. "The trajectories flown by the humans and Swift, we noticed that the autonomous system is more consistent across laps and is able to take tighter turns." "The trajectory is flown by the humans and swift, we noticed that the autonomous system is more consistent across laps and is able to take tighter turns." "This gives it a decisive advantage in a race." "Swift won multiple races against each of the human champions and achieved the fastest recorded race time." "This marks the first time that an autonomous mobile robot has achieved world champion level performance in a real world competitive sport." What might come next?

Well, these have received approval in China for fully autonomous flights, and the new model is expected to be launched in the next few months. Let me know in the comments if you would fly in one of these e-hang drones. At the moment I'm undecided, probably not right now, I'd have to see in action for a bit longer.

On page 58 the authors describe another year of progress in music generation. But rather than just describe this progress, I want you to hear it. I'm not sponsored by them, but I've been honestly very impressed by Suno AI. Check out a snippet of the following rap, all generated by Suno.

"A.I. explain, breaking through the mold, diving deep discovering secrets untold. Smarter than human, it's a whole new game. Unleashing power, a technological flame." If you like that one, I'm going to end the video with another sample. Very quickly though, the way to test it out is to join their discord, click on one of the chirp beaters, and then just type forward slash chirpbeaters.

Then you have to press enter again, for some reason, and up this will come. At the top you can put things like rap or classical, whatever you want. And then you can either enter 4-8 lines of your own lyrics or have ChatGPT do it for you. Honestly don't make more than around 8 lines because it degrades performance quite a lot.

Page 63 briefly describes the progress in medical performance of language models. And I do honestly think that was worth highlighting. I just wonder if MedPalm 2, when it got 85% used a smart GPT like framework. Because if it didn't, things like self consistency, chain of thought prompting, and reflection could boost the score even higher than 85%.

In other words, it is not out of the question that we could soon have in our pockets a model capable of outperforming all but the very best doctors at medical question answering. Speaking of medicine, apparently it's the fastest growing field in terms of mentions of AI within papers. And just look at the breadth of fields now applying AI to achieve research breakthroughs and the increase in volume.

And notice at the bottom how much mathematics is benefiting from this progress. And I bet many of you don't know about the automated prover and proof assistant GPT-F. One of the co-authors of that in 2020 was Ilya Sutskova of OpenAI. It was about proving mathematical theorems using a GPT model.

And again something many might not know is that that was superseded in 2022 by Meta. Their neural theorem prover solved 10 international math olympiad problems. That's 5 times more than any previous system. Their method was called "hypertree proof search" and is way too complicated for me to cover in this video.

And that's not even touching Alpha Tensor from Google DeepMind which improved upon decades old methods for matrix multiplication. That has direct ramifications for the usage of GPUs in modern day AI. Very quickly what they did to automate algorithmic discovery was to convert the finding of efficient algorithms for matrix multiplication into a single player game.

And I love this bit. They say this game is incredibly challenging. The number of possible algorithms to consider is much greater than the number of atoms in the universe. And compared to the game of Go, which remained a challenge for AI for decades, the number of possible moves at each step of our game is 30 orders of magnitude larger.

And it's the synergies that I'm interested in. The breakthroughs in one field that will then be applied to another. The art results in one modality that push forward another modality. And the number of people trying to do this is multiplying as well. Honestly, I hadn't even heard of Imbue.

Apparently they have 10,000 H100s which are the most cutting edge GPUs. I don't know how I missed an AI company with 5 figure numbers of H100 GPUs but here's the CEO describing their approach. At Imbue we're training large foundation models optimized for reasoning. On top of those models we build agents that we use to accelerate our own research.

And these do more than just output something. They also iterate and reflect and figure out what's the next step to do and then take that next step. We're starting with agents that code because it requires complex reasoning to be able to code well and because that's the work we do every day.

They seem to be flying completely under the radar with only about 300 subscribers on their channel. So to be honest, if we see the kind of manual dexterity that Eureka promises and a breakthrough in reasoning like Imbue and frankly every other company is working on. And if AI can start to crack international math olympiad problems and compose rap battles, it does start to become a genuine question of what can't they do.

I'm not saying that's true now, far from it, but in the not too distant future that may well be a very valid question. In the industry section on page 82 the authors describe the chip export bans that apply to H100s. Basically companies like Nvidia and Intel can't sell their cutting edge chips to China and select other countries.

And those companies got around it by creating models like the A100 which were just outside of the export restriction range. Well the report is already somewhat out of date in that respect. Just four days ago we hear that those chips are now also under export restrictions. Another development that's occurred since the publication of the report concerns copyright.

The report detailed how Microsoft has moved to reassure the users of its tools that the corporation will assume any use of the chip. So the report is that the company is now under the jurisdiction of the company that is responsible for the copyright claim. And of course every day we seem to be reading reports of one company or another getting sued as it relates to AI data.

But just a few days ago we had this, Google giving the same kind of indemnification. Basically if the use of one of its Gen AI products triggers any copyright claims, it, Google, will assume legal responsibility, it says. My only prediction is that this battle over data and copyright is not going anywhere.

So the report is that the company is now under the jurisdiction of the corporation that is responsible for the use of the chip. And that's why it's so important that the company is not under the jurisdiction of the corporation that it's not going anywhere. So the report is that the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the report is that the company is now under the jurisdiction of the corporation that it's not going anywhere. So the report is that the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere.

So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the corporation that it's not going anywhere. So the company is now under the jurisdiction of the