Back to Index

Pixel 6 AI explained | Lex Fridman


Chapters

0:0 ML demo on phone
1:13 Specs
2:52 Tensor - AI hardware
5:9 Benchmarks
6:32 AI software
8:14 Conclusion

Transcript

Here's the new Pixel 6 Pro from Google. You're now seeing the result of it running a computationally intensive neural network in real time that I put on there for testing purposes. It's using TensorFlow Lite and the new Tensor chip that is optimized for AI. I am unboxing this AI because as a robotics and AI person, I'm interested in seeing how innovation in AI hardware and software is increasingly taking over the smartphone space.

Also, unboxing AI makes me think of Pandora's box, the myth that serves as the metaphor for the mystery and the power of AI. I know this is just a phone, but let's pause for a second to think. This computer has over 200,000 times the processing power of the computer that first landed humans on the moon and over 2 million times the RAM.

We are engineering our way to superhuman intelligence one phone at a time. One small step for phone and soon enough, one giant leap for a hybrid of machine and mankind. Let's talk about the specs. Here's the comparison of the Pixel 6 to the Pixel 6 Pro across various specs, highlighting what to me are the key differences in yellow and in green, what are the key similarities.

Key differences, $300 in price. Also the Pixel 6 Pro has a slightly higher display, slightly higher resolution and a 120 hertz refresh rate versus the 90 hertz refresh rate. Though honestly, I've been using both phones for almost a week now and I don't feel any difference between them. The key similarity to me on the AI and the computational side is that they both have the same SoC system on a chip, the Google Tensor.

To me, the RAM and storage are very important, but once you get to a certain threshold, it really doesn't matter. 12 gigabytes feels the same as 8 gigabytes and the same goes for the storage. Both phones have a 50 megapixel wide angle. Now that produces a 12 megapixel image because Google combines the 2x2 pixel groups into single effective pixel to cut noise and improve color and dynamic range.

There's also the 12 megapixel ultra wide that's used to decrease noise for both the photos and the videos. And the Pro has a 48 megapixel telephoto lens, which results in one key difference, I think. It provides a 4x optical zoom plus a 20x digital zoom via machine learning via their super resolution algorithm.

The rest to me is pretty much the same. Both phones feel amazing in my hand, but of course, me being who I am, I care mostly about what's on the inside, and that's the Google Tensor. Now let's talk about the Tensor chip. My main flagship phone for machine learning applications this year has been the Samsung Galaxy S21 Ultra 5G, pictured here with the Pixel 6 and the Pixel 6 Pro.

The Galaxy Brain is powered by Snapdragon 888. The Pixel Brain is powered by the new Tensor chip. Both are truly amazing machines. I think AI innovation in both hardware and software will be what matters in flagship smartphones over the next decade. This is where the battle is. Let's now look at the details of the technical specs of the Tensor system on a chip and also the philosophical vision behind its architecture.

The key components of the architecture of the Tensor system on a chip are the CPUs, the GPU, ISP, TPU, Context Hub, and the Titan M2. Depending on the application, various components of this chip can be used at the same time, leading to what Google is calling heterogeneous computing. For the CPU, there's two big CPU cores with the Cortex-X1, there's two medium CPU cores with the A76, and there's four small CPU cores with the A55.

This is in contrast with the most common design for the flagships, which is one big Cortex-X1 and four and three medium A78 cores. It's funny that Google says that having one big CPU core is good for benchmarks but not good for the experience. It's funny because, as you'll see in the benchmarks, the Pixel 6 actually performs really well on the single core Geekbench 5 test.

Performance-wise, the benefit of having two Cortex-X1s is that you can distribute a thermal budget across them so there's less overheating on intensive tasks like 4K 60fps video. So in initial tests, there's a lot less overheating, so you'll be able to shoot video for much longer. Besides the CPUs, there's the GPU, there's an upgrade in that.

There's the ISP, Image Signal Processor, that's optimized for image and video processing in terms of machine learning. And then there's the more general machine learning engine, that's the Tensor Processing Unit. The Geekbench 6 Hub does ultra-low power ambient computing, and the Titan M2 does hardware security. Like I said, I've been using the Galaxy S21 Ultra with the Snapdragon 888 for many months now, and so it's nice to take a look at some benchmarks for the CPU, GPU, and NPU for these two flagship chips.

Now the big caveat here, as you probably know, is that benchmarks often don't reflect real-life performance so arguably they don't actually matter. But the main takeaway story here is that these are both amazing chips. In Geekbench 5 CPU benchmark, Pixel 6 outperforms the Galaxy S21 on single-core tests, and the Galaxy S21 outperforms Pixel 6 on the multi-core tests.

For the Geekbench machine learning benchmark, the Snapdragon wins on the CPU and the GPU, and the Pixel 6 wins on the NPU. The Geekbench machine learning benchmark, by the way, uses TensorFlow Lite. And there's also the AI Benchmark 4 that's specialized for machine learning. It runs a huge number of different neural networks on the devices, and there, once again, Pixel 6 far outperforms Galaxy S21.

In fact, it leads every other smartphone on the current AI Benchmark 4 leaderboard. Again, I think the takeaway here in terms of benchmarks is that Pixel 6 does well on machine learning tasks, and Snapdragon does well on CPU/GPU-centric tasks. But they're both, again, incredible machines. I think the important thing here is what does this heterogeneous computing enable in terms of software features?

And the Pixel 6 provides a huge number of seamlessly integrated machine learning algorithms, increasing the vibrancy of the color with the HDR+ for the images and HDR-Net for the video, improving the accuracy and the efficiency of the face detection, again, both for images and video. And then there's just a huge number of cool features like face unblur motion mode that adds blur to moving objects.

There's the magic eraser that's actually shown here on screen, where you can select certain parts of the object. They can be removed and then intelligently filled based on what the background is. Then for images, there's real tone that's looking at skin color, making sure this shows up looking great on photos.

Honestly, the video is where the fun is. Like I said, HDR-Net, that's an incredible use of neural networks. I actually personally think super resolution algorithms are one of the coolest applications of computer vision in terms of its maybe simplicity and usefulness and impact. And there's a huge number of applications outside of visual domain, so speech, automatic speech recognition.

You're talking about deployment of state-of-the-art ASR algorithms that pay attention to context, pauses, is able to do noise removal. And on the language side, there's neural machine translation. Obviously Google is taking natural language NLP really seriously in both the textual domain in speech, that's audio, and again, back to images and video.

This is incredible leveraging to do heterogeneous computing on AI hardware to enable all kinds of cool computational photography features. Okay, let's look at some takeaways. My two favorite AI chips for Android now are the Google Tensor and the Snapdragon 888. Time will tell which wins for which applications, but for now, competition in this space is great for everyone.

If you want me to talk about other AI systems or about running machine learning code on this and on other phones, let me know. I'll close with a quote from Eliezer Yudkowsky. "By far, the greatest danger of AI is that people conclude too early that they understand it." Thanks for watching, and hope to see you next time.

Bye. Eliezer Yudkowsky, Google Tensor, and Snapdragon 888 Page 2 of 10 Page 2 of 10 Page 2 of 10 Page 2 of 10 you you