Mark Zuckerberg: First Interview in the Metaverse

00:00:00.000 | The following is a conversation with Mark Zuckerberg

00:00:03.640 | inside the metaverse.

00:00:05.720 | Mark and I are hundreds of miles apart from each other

00:00:08.600 | in physical space, but it feels like we're in the same room

00:00:12.480 | because we're appeared to each other

00:00:13.960 | as photorealistic Kodak avatars in 3D with spatial audio.

00:00:18.960 | This technology is incredible.

00:00:21.760 | And I think it's the future of how human beings connect

00:00:24.440 | to each other in a deeply meaningful way on the internet.

00:00:28.460 | These avatars can capture many of the nuances

00:00:31.640 | of facial expressions that we use,

00:00:34.320 | we humans use to communicate emotion to each other.

00:00:38.200 | Now I just need to work on upgrading my emotion

00:00:42.200 | expressing capabilities of the underlying human.

00:00:45.260 | This is the Lex Friedman Podcast.

00:00:48.840 | And now dear friends, here's Mark Zuckerberg.

00:00:53.080 | (Mark sighs)

00:00:55.500 | (Mark laughs)

00:01:08.080 | - This is so great.

00:01:09.220 | Lighting change.

00:01:14.020 | Wow.

00:01:14.860 | - Oh yeah, we can put the light anywhere.

00:01:17.860 | - And it doesn't feel awkward to be really close to you.

00:01:22.300 | - No, it does.

00:01:23.140 | I actually moved you back a few feet

00:01:24.780 | before you got into the headset.

00:01:26.100 | You were like right here.

00:01:27.980 | - I don't know if people can see this,

00:01:30.680 | but this is incredible.

00:01:32.620 | The realism here is just incredible.

00:01:34.600 | Where am I?

00:01:36.960 | Where are you, Mark?

00:01:38.060 | Where are we?

00:01:39.780 | - You're in Austin, right?

00:01:41.180 | - No, I'm here in this place.

00:01:43.260 | We're shrouded by darkness with ultra realistic face

00:01:47.780 | and it just feels like we're in the same room.

00:01:51.580 | This is really the most incredible thing I've ever seen.

00:01:54.500 | And sorry to be in your personal space.

00:01:56.200 | - I mean--

00:01:57.100 | - We have done Jiu-Jitsu before.

00:01:58.380 | - Yeah, no, I was commenting to the team before

00:02:00.420 | that I feel like we've choked each other

00:02:04.720 | from further distances than it feels like we are right now.

00:02:08.140 | - I mean, this is just really incredible.

00:02:10.020 | I don't know how to describe it with words.

00:02:12.660 | It really feels like we're in the same room.

00:02:17.660 | It feels like the future.

00:02:19.980 | This is truly, truly incredible.

00:02:21.780 | I just wanted to take it in.

00:02:23.220 | I'm still getting used to it.

00:02:24.220 | It's like, it's you, it's really you,

00:02:27.420 | but you're not here with me, right?

00:02:30.460 | You're there wearing a headset and I'm wearing a headset.

00:02:34.460 | It's really, really incredible.

00:02:36.420 | So what, can you describe what it takes currently

00:02:40.740 | for us to appear so photorealistic to each other?

00:02:44.060 | - Yeah, so I mean, for background,

00:02:46.460 | we both did these scans for this research project

00:02:51.380 | that we have at Meta called Kodak Avatars.

00:02:54.460 | And the idea is that instead of actually,

00:02:58.500 | instead of our avatars being cartoony

00:03:00.740 | and instead of actually transmitting a video,

00:03:03.140 | what it does is we've sort of scanned ourselves

00:03:06.980 | in a lot of different expressions

00:03:09.460 | and we've built a computer model of sort of each

00:03:13.900 | of our faces and bodies and the different expressions

00:03:18.060 | that we make and collapse that into a Kodak

00:03:21.460 | that then when you have the headset on your head,

00:03:24.460 | it can, it sees your face, it sees your expression

00:03:28.060 | and it can basically send an encoded version

00:03:32.580 | of what you're supposed to look like over the wire.

00:03:34.300 | So in addition to being photorealistic,

00:03:37.260 | it's also actually much more bandwidth efficient

00:03:40.420 | than transmitting a full video

00:03:43.420 | or especially a 3D immersive video

00:03:45.780 | of a whole scene like this.

00:03:47.700 | - And it captures everything, like the flaws.

00:03:51.020 | Like to me, the subtleties of the human face,

00:03:54.260 | like even the flaws, that's like, that's all amazing.

00:03:57.500 | It makes you, it makes it so much more immersive.

00:04:00.500 | It makes you realize that like perfection isn't the thing

00:04:03.980 | that leads to immersion.

00:04:05.020 | It's like the little subtle flaws,

00:04:06.780 | like freckles and like variations in color

00:04:09.940 | and just-- - Yeah, wrinkles.

00:04:12.060 | - All stuff about noses. - Asymmetry.

00:04:14.060 | - Yeah, asymmetry.

00:04:15.380 | And just the different, like the corners of the eyes,

00:04:17.540 | like what your eyes do when you smile,

00:04:19.100 | all that kind of stuff.

00:04:20.380 | - Yeah, eyes are a huge part of it.

00:04:22.740 | Yeah, I mean, there's all the studies

00:04:23.900 | that most of communication, even when people are speaking,

00:04:28.140 | is not actually the words that they're saying, right?

00:04:30.500 | It's kind of the expression and all that.

00:04:32.420 | So, and we try to capture that with the kind of classical,

00:04:35.940 | expressive avatar system that we have.

00:04:39.380 | That's the kind of more cartoon designed one.

00:04:41.340 | You can kind of put those kinds of expressions

00:04:43.700 | on those faces as well.

00:04:44.860 | But there's obviously a certain realism

00:04:46.900 | that comes with delivering

00:04:48.380 | kind of this photorealistic experience that,

00:04:51.100 | I don't know, I just think it's really magical.

00:04:52.860 | I mean, this gets to kind of the core

00:04:54.580 | of what the vision around virtual and augmented reality is,

00:04:58.420 | of like delivering a sense of presence

00:05:00.060 | as if you're there together,

00:05:01.700 | no matter where you actually are in the world.

00:05:03.460 | And I mean, this experience, I think,

00:05:06.500 | is a good embodiment of that,

00:05:07.700 | where it's like, I mean, we're in two

00:05:09.100 | completely different states, halfway across the country,

00:05:11.780 | and it just like, you know,

00:05:13.260 | it looks like you're just sitting right in front of me.

00:05:14.780 | It's pretty wild.

00:05:16.980 | - Yeah, yeah, I can't, I'm almost getting emotional.

00:05:19.980 | It's like, it feels like a totally,

00:05:21.940 | it's a fundamentally new experience.

00:05:24.540 | Like for me to have this kind of conversation

00:05:26.380 | with loved ones, it would just change everything.

00:05:30.300 | Maybe just to elaborate, so I went to Pittsburgh

00:05:33.820 | and went through the whole scanning procedure,

00:05:35.820 | which has so much incredible technology,

00:05:39.060 | so software and hardware going on,

00:05:41.380 | but it is a lengthy process.

00:05:43.860 | So what's your vision for the future of this

00:05:47.540 | in terms of making this more accessible to people?

00:05:50.180 | - You know, it starts off with a small number of people

00:05:53.700 | doing these very detailed scans, right?

00:05:57.220 | Which is, that's the version that you did and that I did.

00:06:00.220 | And, you know, before there were a lot of people

00:06:02.180 | who we've done this kind of a scan for,

00:06:05.260 | we probably need to kind of over collect expressions

00:06:10.260 | when we're doing the scanning,

00:06:12.460 | because we haven't figured out how much

00:06:14.180 | we can reduce that down to a really streamlined process

00:06:17.980 | and extrapolate from the scans that have already been done.

00:06:21.780 | But, you know, the goal, and we have a project

00:06:23.820 | that's working on this already,

00:06:25.580 | is just to do a very quick scan with your cell phone,

00:06:29.100 | where you just take your phone,

00:06:30.500 | kind of wave it in front of your face

00:06:31.980 | for a couple of minutes, you know,

00:06:34.140 | say a few sentences, make a bunch of expressions,

00:06:37.660 | but overall have the whole process

00:06:40.300 | just be two to three minutes,

00:06:41.820 | and then produce something that's of the quality

00:06:43.620 | of what we have right now.

00:06:44.700 | So I think that that's one of the big challenges

00:06:47.140 | that remains, and right now we have the ability

00:06:49.180 | to do the scans if you have hours to sit for one.

00:06:53.340 | And with today's technology, I mean,

00:06:55.700 | you're using a meta headset that exists,

00:06:58.300 | it's a product that's kind of for sale now,

00:07:01.140 | you can drive these with that.

00:07:03.580 | But the production of these scans in a very efficient way

00:07:08.580 | is one of the last pieces that we still need to really nail.

00:07:13.420 | And then obviously there's all the experiences around it.

00:07:15.220 | I mean, right now we're kind of sitting in a dark room,

00:07:17.180 | which is familiar for your podcast.

00:07:21.140 | But I think part of the vision for this over time

00:07:24.220 | is not just having this be like a video call.

00:07:29.020 | I mean, that's fine, it's cool,

00:07:30.980 | or it feels like it's immersive,

00:07:32.500 | but you can do a video call on your phone.

00:07:35.860 | The thing that you can do in the metaverse

00:07:37.980 | that is different from what you can do on a phone

00:07:39.500 | is like doing stuff where you're physically there together

00:07:42.460 | and participating in things together.

00:07:44.140 | And we could play games like this,

00:07:46.580 | we could have meetings like this in the future.

00:07:50.220 | Once you get mixed reality and augmented reality,

00:07:54.420 | we could have Kodak avatars like this

00:07:56.620 | and go into a meeting and have some people physically there

00:07:58.780 | and have some people show up in this photorealistic form,

00:08:02.540 | superimposed on the physical environment.

00:08:05.300 | I think that stuff like that is gonna be super powerful.

00:08:07.660 | So we gotta still build out all those kind of applications

00:08:10.540 | and the use cases around it.

00:08:11.700 | But I don't know, I think it's gonna be a pretty wild

00:08:14.420 | next few years around this.

00:08:16.260 | - I mean, I'm actually almost at a loss of words.

00:08:18.860 | This is just so incredible.

00:08:20.500 | This is truly incredible.

00:08:21.700 | I hope that people watching this

00:08:23.540 | can get a glimpse of how incredible it is.

00:08:26.580 | It really feels like we're in the same room.

00:08:28.220 | Like there is that, I guess there's an uncanny valley

00:08:32.260 | that seems to have been crossed here.

00:08:34.300 | Like it looks like you.

00:08:36.860 | - Yeah, I mean, I think there's still a bunch of tuning

00:08:40.540 | that I think we'll wanna do

00:08:41.740 | where different people emote to different extents, right?

00:08:46.660 | So I think one of the big questions is,

00:08:49.780 | when you smile, how wide is your smile

00:08:53.300 | and how wide do you want your smile to be?

00:08:56.220 | And I think getting that to be tuned on a per person basis

00:08:59.860 | is gonna be one of the things

00:09:02.860 | that we're gonna need to figure out.

00:09:05.060 | You know, it's like, to what extent

00:09:06.900 | do you wanna give people control over that?

00:09:09.820 | Some people might prefer a version of themselves

00:09:14.020 | that's more emotive in their avatar

00:09:15.820 | than their actual faces.

00:09:17.220 | You know, so for example,

00:09:18.260 | I always get a lot of critique and shit

00:09:21.420 | for having like a relatively stiff expression.

00:09:26.420 | But, you know, I mean, I might feel pretty happy

00:09:29.700 | but just make a pretty small smile.

00:09:31.100 | So I mean, maybe, you know, for me,

00:09:33.220 | it's actually, you know, it's like,

00:09:34.740 | I'd wanna have my avatar really be able to better express

00:09:38.460 | like how I'm feeling than how I can do physically.

00:09:42.220 | So I think that there's a question

00:09:43.140 | about how you wanna tune that.

00:09:44.300 | But overall, yeah, I mean,

00:09:46.340 | we wanna start from the baseline

00:09:47.860 | of capturing how people actually emote

00:09:49.740 | and express themselves.

00:09:50.860 | And I mean, I think the initial version of this

00:09:53.140 | has been pretty impressive.

00:09:54.580 | And like you said, I do think we're kind of beyond

00:09:58.420 | the uncanny valley here where,

00:10:00.260 | and it does feel like you,

00:10:01.300 | it doesn't feel weird or anything like that.

00:10:05.060 | - I mean, that's gonna be the meme

00:10:06.660 | that the two most monotone people

00:10:08.780 | are in a metaverse together.

00:10:10.580 | But I think that actually makes it more difficult.

00:10:14.580 | Like the amazing thing here is that the subtleties

00:10:17.460 | of the expression of the eyes, you know,

00:10:20.260 | people say I'm monotone and emotionless, but I'm not.

00:10:22.700 | It's just this, maybe my expression of emotion

00:10:25.660 | is more subtle usually like with the eyes.

00:10:29.420 | And that's one of the things I've noticed

00:10:31.300 | is just how expressive the subtle movement

00:10:34.060 | of the corners of the eyes are

00:10:35.540 | in terms of displaying happiness or boredom

00:10:37.580 | or all that kind of stuff.

00:10:39.020 | - I am curious to see,

00:10:40.620 | just 'cause I've never done one of these before.

00:10:41.940 | I've never done a podcast as one of these Kodak avatars.

00:10:44.940 | And I'm curious to see what people think of it

00:10:49.300 | because one of the issues that we've had

00:10:51.460 | in some of the VR and mixed reality work

00:10:54.220 | is it tends to feel a lot more profound when you're in it

00:10:57.740 | than the 2D videos capturing the experience.

00:11:00.260 | So I think that this one, because it's photorealistic,

00:11:03.180 | may look kind of as amazing in 2D for people watching it

00:11:07.380 | as it feels, I think, to be in it.

00:11:09.780 | But we've certainly had this issue

00:11:12.780 | where a lot of the other things,

00:11:13.900 | just it's like you feel the sense of immersion

00:11:15.820 | when you're in it that doesn't quite translate

00:11:17.700 | to a 2D screen.

00:11:18.660 | But I don't know, I'm curious to see what people think.

00:11:21.660 | - Yeah, I'm curious to see if people could see that,

00:11:25.060 | like my heart is actually beating fast now.

00:11:27.580 | This is super interesting,

00:11:29.460 | like that such intimacy of conversation

00:11:33.300 | can be achieved remotely.

00:11:34.620 | There's been, I don't do remote podcasts for this reason.

00:11:38.540 | And this is like, breaks all of that.

00:11:41.780 | This feels like just an incredible transition

00:11:45.020 | to something else, to the different kind of communication.

00:11:47.760 | Breaks all barriers, like geographic, physical barriers.

00:11:51.000 | You mentioned, do you have a sense of timeline

00:11:54.840 | in terms of how many difficult things have to be solved

00:11:57.500 | to make this more accessible

00:11:59.980 | to like scanning with a smartphone?

00:12:02.580 | - Yeah, I mean, I think we'll probably roll this out

00:12:05.020 | progressively over time.

00:12:06.180 | So it's not going to be like we roll it out

00:12:07.420 | and one day everyone has a Kodak avatar.

00:12:10.100 | We want to get more people scanned and into the system.

00:12:13.540 | And then we want to start integrating it

00:12:16.620 | into each one of our apps, right?

00:12:18.240 | Making it so that, you know,

00:12:19.620 | I think that for a lot of the work style things,

00:12:22.420 | productivity, I think that this is going to make

00:12:24.060 | a ton of sense.

00:12:25.140 | In a lot of game environments, I mean, this could be fine,

00:12:27.460 | but games tend to have their own style, right?

00:12:29.340 | Where you almost want to fit more

00:12:31.060 | with the aesthetic style of the game.

00:12:34.660 | But I think for doing meetings,

00:12:36.340 | and one of the things that we get a lot of feedback

00:12:38.340 | on work rooms, where people are pretty blown away

00:12:41.540 | by the experience and this feeling that you can like

00:12:44.180 | be remote, but feel like you're physically there

00:12:47.420 | around a table with people.

00:12:48.900 | But then, you know, we get some feedback

00:12:50.660 | that people have a hard time with the fact that

00:12:52.660 | the avatars are so expressive and don't feel,

00:12:56.540 | you know, as realistic in that environment.

00:12:58.260 | So I think something like this

00:13:00.180 | could make a very big difference for those remote meetings.

00:13:02.820 | And especially with Quest 3 coming out,

00:13:04.820 | which is going to be the first mainstream

00:13:06.260 | mixed reality product, right?

00:13:07.500 | Where you're really taking digital,

00:13:09.220 | you know, expressions of either a person

00:13:12.580 | or objects and overlaying them on the physical world.

00:13:15.900 | I think the ability to do kind of remote meetings

00:13:19.460 | and things like that, where you're like,

00:13:21.980 | just remote hang sessions with friends.

00:13:23.780 | I mean, I think that that's going to be very exciting.

00:13:25.380 | So yeah, rolling it out over the next few years.

00:13:28.060 | It's not ready to be like a kind of mainstream product yet,

00:13:31.820 | but we just want to keep tuning in,

00:13:34.060 | keep getting more scans in there and keep, you know,

00:13:36.180 | and kind of rolling it out into more of the features.

00:13:37.900 | But yeah, I mean, definitely in the next few years,

00:13:41.940 | you'll be seeing a bunch more experiences like this.

00:13:44.740 | - Yeah, I would love to see some celebrities scanned

00:13:47.460 | and some non-celebrities.

00:13:49.060 | And just more people to experience this.

00:13:53.340 | I would love to see that.

00:13:54.180 | This is something that, I mean,

00:13:55.500 | on my mind is blown, I'm literally at a loss for words.

00:13:58.300 | 'Cause it's very difficult to just convey

00:14:02.380 | how incredible this is.

00:14:03.820 | How like, how I feel the emotion, how I feel the presence,

00:14:07.180 | how I feel like the subtleties of the emotion

00:14:09.860 | in terms of like work meetings or any kind of,

00:14:12.420 | in terms of podcasts, this is like, this is awesome.

00:14:14.580 | And I don't even need your arms or legs.

00:14:17.100 | Is that-- - Well, we got to get that.

00:14:19.460 | I mean, that's its own challenge.

00:14:22.420 | And part of the question is also, so you have the scan,

00:14:25.800 | then it takes a certain amount of compute to go drive that,

00:14:29.340 | both for the sensors on the headset and then rendering it.

00:14:33.580 | So one of the things that we're working through

00:14:35.980 | is what is the level of fidelity that is optimal, right?

00:14:39.180 | You can do the full body in kind of a codec

00:14:41.980 | and that can be quite intensive.

00:14:45.420 | But one of the things that we're thinking about is like,

00:14:48.980 | all right, maybe you can kind of stitch

00:14:51.820 | a somewhat lower fidelity version of your body,

00:14:54.420 | but still have the main kind of the major movements.

00:14:59.060 | But your face is really the thing

00:15:01.980 | that we have the most resolution on, right?

00:15:03.780 | In terms of being able to read and express emotions.

00:15:06.860 | I mean, like you said, if you move your eyebrows

00:15:10.220 | like a millimeter, I mean, that really changes the expression

00:15:12.900 | and what you're emoting.

00:15:14.260 | Whereas, I mean, moving your arm like an inch

00:15:19.260 | probably doesn't matter quite as much.

00:15:21.100 | So yeah, so I think that we do want to get all of that

00:15:23.940 | into here and that'll be some of the work

00:15:25.940 | over the next period as well.

00:15:27.900 | - So you mentioned Quest 3, that's coming out.

00:15:30.740 | I've gotten a chance to try that too.

00:15:31.980 | That's awesome.

00:15:33.140 | So the, how'd you pull off the mixed?

00:15:34.620 | So it's not just virtual reality, it's mixed reality.

00:15:37.420 | - Yeah, I mean, I think it's going to be,

00:15:38.380 | it's going to be the first mainstream mixed reality device.

00:15:42.180 | I mean, obviously we shipped Quest Pro last year,

00:15:45.340 | but it was $1,500.

00:15:46.900 | And part of what I'm super proud of is, you know,

00:15:50.500 | we try to innovate, not just on pushing the state of the art

00:15:53.700 | and delivering new capabilities,

00:15:54.940 | but making it so it can be available to everyone.

00:15:57.580 | And, you know, we have this and it's coming out, it's $500.

00:16:01.260 | And in some ways I think the mixed reality

00:16:04.340 | is actually better in Quest 3 than it was,

00:16:07.180 | than what we're using right now in Quest Pro.

00:16:09.180 | So, and I'm really proud of the team

00:16:10.980 | for being able to deliver that kind of an innovation

00:16:13.420 | and get it out.

00:16:14.260 | But, you know, some of this is just software,

00:16:18.100 | you tune over time and get to be better.

00:16:20.700 | Part of it is you put together a product

00:16:22.300 | and you figure out what are the bottlenecks

00:16:24.540 | in terms of making it a good experience.

00:16:26.100 | So we got the resolution for the mixed reality cameras

00:16:29.300 | and sensors to be multiple times better in Quest 3.

00:16:32.940 | And we just figured that that made a very big difference

00:16:35.180 | when we saw the experience that we were able

00:16:36.700 | to put together for Quest Pro.

00:16:38.220 | And part of it is also that, you know,

00:16:40.980 | Qualcomm just came out with their next generation chip set

00:16:43.940 | for VR and MR, that we worked with them

00:16:46.420 | on a kind of custom version of it.

00:16:49.100 | But that was available this year for Quest 3

00:16:51.620 | and it wasn't available in Quest Pro.

00:16:53.260 | So, you know, in a way, Quest 3,

00:16:54.980 | even though it's not the Pro product,

00:16:58.100 | actually has a stronger chip set in it

00:16:59.660 | than the Pro line at a third of the cost.

00:17:01.980 | So I'm really excited to get this in people's hands.

00:17:05.940 | It does all the VR stuff that Quest 2

00:17:08.340 | and the others have done too.

00:17:10.020 | It does it better because the display is better

00:17:12.620 | and the chip is better, so you'll get better graphics.

00:17:16.260 | It's 40% thinner, so it's more comfortable as well.

00:17:21.260 | But the MR is really the big capability shift.

00:17:24.100 | And part of what's exciting about the whole space right now

00:17:27.140 | is, you know, this isn't like smartphones

00:17:29.820 | where companies put out a new smartphone every year

00:17:32.060 | and you can almost barely tell the difference

00:17:33.580 | between that and the one the year before it.

00:17:36.260 | Now for this, each time we put out a new headset,

00:17:39.260 | it has like a major new capability.

00:17:41.380 | And the big one now is mixed reality.

00:17:44.580 | The ability to basically take digital representations

00:17:47.380 | of people or objects and superimpose them on the world.

00:17:51.900 | And basically, you know, I mean,

00:17:53.380 | there's one version of this is you're gonna kind of

00:17:56.980 | have these augments or holograms and experiences

00:18:01.140 | that you can kind of bring into your living room

00:18:03.260 | or a meeting space or office.

00:18:05.140 | Another thing that I just think is gonna be

00:18:08.140 | a much kind of simpler innovation

00:18:11.220 | is that there are a lot of VR experiences today

00:18:13.820 | that don't need to be fully immersive.

00:18:16.100 | And if you're playing a shooter game

00:18:17.940 | or you're doing a fitness experience,

00:18:20.500 | sometimes people get worried about swinging their arms

00:18:22.500 | around, like, am I gonna hit a lamp or something?

00:18:24.460 | You know, and am I gonna run into something?

00:18:28.500 | So having that in mixed reality actually is just

00:18:31.060 | a lot more comfortable for people, right?

00:18:32.860 | You kind of still get the immersion and the 3D experience

00:18:35.980 | and you can have an experience that just wouldn't be

00:18:38.260 | possible in the physical world alone,

00:18:40.220 | but by being anchored to and being able to see

00:18:41.980 | the physical world around you, it's like,

00:18:43.900 | it just feels so much safer and more secure.

00:18:46.180 | And I think a lot of people are really gonna enjoy that too.

00:18:48.260 | So yeah, I'm really excited to see how people use it.

00:18:50.740 | But yeah, Quest 3 coming out later this fall.

00:18:54.300 | - Yeah, and I got to experience it with other people

00:18:56.660 | sitting around and there's a lot of furniture.

00:18:58.420 | And so you get to see that furniture

00:18:59.820 | and get to see those people.

00:19:01.180 | And you get to see those people, like,

00:19:03.780 | enjoy the ridiculousness of you, like, swinging your arms.

00:19:06.820 | I mean, presumably they're friends of yours,

00:19:08.980 | even if they make fun of you,

00:19:10.900 | there's a lot of love behind that.

00:19:12.180 | And I got to experience that.

00:19:14.540 | That's a really fundamentally different experience

00:19:16.220 | than just pure VR, with zombies coming out of walls.

00:19:20.420 | - Yeah, it's like someone shooting at you

00:19:22.140 | and you hide behind your real couch

00:19:23.860 | in order to duck the fire, yeah.

00:19:26.300 | - It's incredible how it's all integrated,

00:19:27.820 | but also like subtle stuff, like,

00:19:30.100 | in a room with no windows, you can add windows to it.

00:19:33.020 | And you can look outside as the zombies run towards you,

00:19:35.780 | but like, it's still nice view outside, you know?

00:19:38.300 | - Yeah.

00:19:39.140 | - It's really, and so that's pulled off

00:19:41.300 | by having cameras on the outside of the headset

00:19:43.980 | that do the pass through.

00:19:46.220 | I think that technology is incredible,

00:19:47.660 | to do that on a small headset.

00:19:50.100 | - Yeah, and it's not just the cameras.

00:19:51.260 | You basically need to, you need multiple cameras

00:19:53.900 | to capture the different angles

00:19:55.700 | and sort of the three-dimensional space.

00:19:58.700 | And then it's a pretty complex compute problem,

00:20:01.700 | an AI problem to map that to your perspective, right?

00:20:05.020 | Because the cameras aren't exactly where your eyes are

00:20:06.860 | because no two people's eyes are,

00:20:08.340 | you're not going to be in exactly the same place.

00:20:10.060 | You kind of need to get that to line up

00:20:14.340 | and then do that basically in real time

00:20:16.620 | and then generate something that looks,

00:20:18.180 | that kind of feels natural.

00:20:21.180 | And then superimpose whatever digital objects

00:20:23.940 | you want to put there.

00:20:24.780 | So it's, yeah, it's a very interesting technical challenge.

00:20:28.140 | And I think we'll continue tuning this

00:20:30.700 | for the years to come as well,

00:20:32.140 | but I'm pretty excited to get this out

00:20:35.820 | because I think Quest 3 is going to be the first device

00:20:38.180 | like this with, that millions of people are going to get.

00:20:40.780 | That's mixed reality.

00:20:41.660 | And it's only when you have millions of people

00:20:43.740 | using something that you start getting

00:20:45.380 | the whole developer community really starting to experiment

00:20:48.700 | and build stuff,

00:20:49.580 | because now there are going to be people who actually use it.

00:20:52.820 | So I think we'll get,

00:20:53.780 | you know, we got some of that flywheel going with Quest Pro,

00:20:56.220 | but I think it'll really get accelerated

00:20:57.580 | once Quest 3 gets out there.

00:20:59.140 | So yeah, I'm pretty excited about this one.

00:21:01.820 | - Plus there's hand tracking without,

00:21:04.260 | so you don't need to have a control.

00:21:05.380 | So this camera, the cameras aren't just doing the pass through

00:21:08.380 | of the entire physical reality around you.

00:21:12.220 | It's also tracking the details of your hands

00:21:14.660 | in order to use that for like gesture recognition,

00:21:16.820 | this kind of stuff.

00:21:17.940 | Yeah, we've been able to get way further on hand recognition

00:21:21.380 | in a shorter period of time than I expected.

00:21:23.220 | So that's been pretty cool.

00:21:24.740 | I don't know, did you see the demo experience

00:21:27.540 | that we built around-

00:21:29.380 | - Piano?

00:21:30.220 | - Like, yeah, the piano, learning to play piano?

00:21:32.260 | - Yeah, it's incredible.

00:21:33.660 | You're basically playing piano on a table

00:21:35.420 | and it's that's without any controller

00:21:37.620 | and like how well it matches physical reality

00:21:41.260 | with no latency.

00:21:43.180 | And it's tracking your hands with no latency

00:21:46.420 | and it's tracking all the people around you

00:21:48.380 | with no latency, integrating physical reality

00:21:51.180 | and digital reality.

00:21:53.100 | Obviously that connects exactly to this Kodak avatar,

00:21:56.580 | which is in parallel, allows us to have ultra realistic

00:22:01.580 | copies of ourselves in this mixed reality.

00:22:05.100 | So it's all converging towards like an incredible

00:22:09.300 | digital experience in the metaverse.

00:22:11.820 | To me, obviously I love the intimacy of conversation.

00:22:14.180 | So even this is awesome.

00:22:16.100 | But do you have other ideas of what this unlocks

00:22:20.300 | of like something like Kodak avatar unlocks

00:22:22.980 | in terms of applications,

00:22:25.020 | in terms of things we're able to do?

00:22:28.460 | - Well, there's what you can do with avatars overall

00:22:31.500 | in terms of superimposing digital objects

00:22:34.020 | on the physical world.

00:22:35.260 | And then there's kind of psychologically,

00:22:37.940 | what does having photorealistic do?

00:22:39.740 | So I think we're moving towards a world where

00:22:44.900 | we're gonna have something that looks like normal glasses,

00:22:49.020 | where you can just see the physical world,

00:22:51.500 | but you will see holograms.

00:22:53.300 | And in that world, I think that they're gonna be

00:22:57.420 | not too far off, maybe by the end of this decade,

00:23:01.660 | we'll be living in a world where there are

00:23:03.220 | kind of as many holograms when you walk into a room

00:23:05.980 | as there are physical objects.

00:23:07.700 | And it really raises this interesting question about

00:23:11.020 | what are about,

00:23:14.980 | a lot of people have this phrase where they call

00:23:17.660 | the physical world, the real world.

00:23:19.540 | And I kind of think increasingly,

00:23:21.900 | and the physical world is super important,

00:23:23.860 | but I actually think the real world is the combination

00:23:26.460 | of the physical world and the digital worlds coming together.

00:23:29.140 | But until this technology, they were sort of separate.

00:23:33.900 | It's like you access the digital world through a screen.

00:23:36.620 | And maybe it's a small screen that you carry around

00:23:39.220 | or it's a bigger screen where you sit down at your desk

00:23:41.260 | and strap in for a long session,

00:23:43.300 | but they're kind of fundamentally divorced and disconnected.

00:23:47.100 | And I think part of what this technology is gonna do

00:23:49.860 | is bring those together into a single coherent experience

00:23:53.260 | of what the modern real world is,

00:23:54.900 | which is, it's gotta be physical

00:23:56.940 | because we're physical beings.

00:23:58.620 | So the physical world is always gonna be super important.

00:24:01.660 | But increasingly, I think a lot of the things

00:24:04.180 | that we kind of think of can be digital holograms.

00:24:08.620 | I mean, any screen that you have can be a hologram,

00:24:11.940 | any media, any book, art,

00:24:15.740 | can basically be just as effective as a hologram

00:24:18.940 | as a physical object, any game that you're playing,

00:24:21.540 | a board game or any kind of physical game, cards,

00:24:25.900 | ping pong, things like that.

00:24:27.140 | They're often a lot better as holograms

00:24:29.100 | 'cause you could just kind of snap your fingers

00:24:30.740 | and instantiate them and have them show up.

00:24:34.900 | It's like you have a ping pong table show up

00:24:36.580 | in your living room, but then you can snap your fingers

00:24:38.140 | and have it be gone.

00:24:39.620 | So that's super powerful.

00:24:41.740 | So I think that it's actually an amazing thought experiment

00:24:45.220 | of how many physical things we have today

00:24:49.100 | that could actually be better as interactive holograms.

00:24:52.860 | But then beyond that, I think the most important thing,

00:24:56.500 | obviously, is people.

00:24:57.620 | So the ability to have these mixed hangouts,

00:25:00.860 | whether they're social or meetings,

00:25:03.740 | where you show up to a conference room,

00:25:06.020 | you're wearing glasses or a headset in the very near term,

00:25:09.580 | but hopefully by, over the next five years, glasses or so.

00:25:13.300 | And you're there physically,

00:25:16.340 | some people are there physically,

00:25:18.540 | but other people are just there as holograms

00:25:20.740 | and it feels like it's them who are right there.

00:25:23.860 | And also, by the way, another thing

00:25:25.300 | that I think is gonna be fascinating

00:25:26.340 | about being able to blend together

00:25:28.060 | the digital and physical worlds in this way

00:25:31.980 | is we're also going to be able to embody

00:25:35.940 | AIs as well.

00:25:36.900 | So I think you'll also have meetings in the future

00:25:39.380 | where you're basically,

00:25:40.940 | maybe you're sitting there physically

00:25:42.940 | and then you have a couple of other people

00:25:44.740 | who are there as holograms.

00:25:46.300 | And then you have Bob, the AI,

00:25:48.300 | who's an engineer on your team, who's helping with things.

00:25:50.780 | And he can now be embodied as a realistic avatar as well

00:25:55.780 | and just join the meeting in that way.

00:25:58.660 | So I think that that's gonna be pretty compelling as well.

00:26:03.220 | So then, okay, so what can you do with photorealistic avatars

00:26:06.300 | compared to kind of the more expressive ones

00:26:09.740 | that we have today?

00:26:11.060 | Well, I think a lot of this actually comes down

00:26:13.700 | to acceptance of the technology.

00:26:17.300 | And because all of the stuff that we're doing,

00:26:21.020 | I mean, the motion of your eyebrows,

00:26:23.060 | the motion of your eyes, the cheeks and all of that,

00:26:26.620 | there's actually no reason why you couldn't do that

00:26:28.540 | on an expressive avatar too.

00:26:30.060 | I mean, it wouldn't look exactly like you,

00:26:32.100 | but you can make a cartoon version of yourself

00:26:34.620 | and still have it be almost as expressive.

00:26:38.260 | But I do think that there's this bridge

00:26:41.020 | between the current state of most of our interactions

00:26:45.020 | in the physical world and where we're getting in the future

00:26:47.780 | with this kind of hybrid physical and digital world,

00:26:51.420 | where I think it's gonna be a lot easier for people

00:26:54.420 | to kind of take some of these experiences seriously

00:26:59.420 | with the photorealistic avatars to start.

00:27:01.660 | And then I'm actually really curious

00:27:03.140 | to see where it goes longer term.

00:27:04.340 | I could see a world where people stick to the photorealistic

00:27:07.020 | and maybe they modify them to make them

00:27:09.420 | a little bit more interesting,

00:27:10.500 | but maybe fundamentally we like photorealistic things.

00:27:13.580 | But I can also see a world that once people get used

00:27:17.580 | to the photorealistic avatars

00:27:20.100 | and they get used to these experiences,

00:27:22.060 | that I actually think that there could be a world

00:27:24.900 | where people actually prefer being able to express themselves

00:27:30.460 | in kind of ways that aren't so tied

00:27:34.220 | to their physical reality.

00:27:35.780 | And so that's one of the things

00:27:37.420 | that I'm really curious about.

00:27:38.460 | And I don't know, in a bunch of our internal experiments

00:27:41.980 | on this, one of the things that I thought

00:27:44.380 | was psychologically pretty interesting

00:27:45.820 | is people have no issues blending photorealistic stuff

00:27:50.140 | and not.

00:27:50.980 | So we could have a, for this specific scene

00:27:53.460 | that we're in now, we happen to sort of being in a dark room.

00:27:58.420 | I think part of that aesthetic decision,

00:28:00.980 | I think was based on the way you like to do your podcasts.

00:28:03.060 | But we've done experiences like this

00:28:05.580 | where you have like a cartoony background,

00:28:11.140 | but photorealistic people who you're talking to.

00:28:13.780 | And we seem to, like people just seem to just think

00:28:17.620 | that that is completely normal, right?

00:28:19.100 | It doesn't bother you.

00:28:19.940 | It doesn't feel like it's weird.

00:28:21.500 | Another thing that we've experienced with

00:28:24.660 | is basically you have a photorealistic avatar

00:28:27.380 | that you're talking to, and then right next to them,

00:28:29.700 | you have an expressive kind of cartoon avatar.

00:28:32.300 | And that actually is pretty normal too, right?

00:28:35.460 | It's like, it's not that weird, right?

00:28:37.740 | To basically being interacting with different people

00:28:40.940 | in different modes like that.

00:28:41.940 | So I'm not sure.

00:28:43.380 | I think it'll be an interesting question

00:28:44.740 | to what extent these photorealistic avatars

00:28:47.220 | are like a key part of just transitioning

00:28:51.180 | from being comfortable in the physical world

00:28:52.740 | to this kind of new modern real world

00:28:55.580 | if that kind of includes both the digital and physical,

00:28:58.340 | or if this is like the long-term way that it stays.

00:29:01.100 | I mean, I think that there are gonna be uses

00:29:03.860 | for both the expressive and the photorealistic over time.

00:29:06.040 | I just don't know what the balance is gonna be.

00:29:08.020 | - Yeah, it's a really good,

00:29:09.340 | interesting philosophical question.

00:29:10.700 | But to me in the short term,

00:29:12.500 | the photorealistic is amazing to where I would prefer,

00:29:16.300 | like you said, the workroom,

00:29:17.700 | but like on a beach with a beer

00:29:19.940 | to see a buddy of mine remotely on a chair next to me

00:29:24.100 | drinking a beer.

00:29:25.980 | I mean, that, as realistic as possible,

00:29:28.820 | is an incredible experience.

00:29:30.340 | So I don't want any fake hats on him.

00:29:32.400 | I don't want any, just chilling with a friend,

00:29:35.280 | drinking beer, looking at the ocean

00:29:37.220 | while not being in the same place together.

00:29:39.220 | I mean, that experience is just,

00:29:42.820 | it's a fundamentally,

00:29:45.220 | it's just a high-quality experience of friendship.

00:29:49.420 | Whatever we seek in friendship,

00:29:50.620 | it seems to be present there

00:29:52.180 | in the same kind of realism I'm seeing right now.

00:29:54.820 | This is totally a game changer.

00:29:56.340 | So to me, this is,

00:29:57.420 | I can see myself sticking with this for a long time.

00:30:01.100 | - Yeah, and I mean, it's also, it's novel,

00:30:04.180 | and it's also a technological feat, right?

00:30:06.320 | It's like being able to pull this off is like,

00:30:08.220 | it's like a pretty impressive,

00:30:10.300 | and I think to some degree,

00:30:11.500 | it's just this kind of like awesome experience.

00:30:13.860 | - But I'm already, sorry to interrupt,

00:30:16.340 | I'm already forgetting that you're not real.

00:30:21.060 | Like this really, so it's novel.

00:30:23.900 | - It's just an avatar version of me.

00:30:26.300 | - That's a deep philosophical question, yes.

00:30:29.060 | - But I mean, but here's some of the,

00:30:30.180 | so I put this on this morning,

00:30:31.540 | and I was like, all right, like,

00:30:33.880 | it's like, okay, so my hair is a little shorter in this

00:30:37.360 | than my physical hair is right now.

00:30:39.100 | I probably need to go get a haircut.

00:30:41.220 | And like, and I actually,

00:30:42.940 | I did happen to shave this morning,

00:30:44.460 | but if I hadn't,

00:30:46.100 | I could still have this photorealistic avatar

00:30:48.180 | that is more cleanly shaven, right?

00:30:50.980 | Even if I'm a few days in physically.

00:30:55.220 | So I do think that they're gonna start to be

00:30:58.060 | these subtle questions that seep in

00:31:00.220 | where the avatar is realistic in the sense of,

00:31:04.340 | this is kind of what you looked like at the time of capture,

00:31:07.440 | but it's not necessarily temporarily accurate

00:31:10.580 | to exactly what you look like in this moment.

00:31:13.240 | And I think that they're gonna end up being

00:31:15.480 | a bunch of questions that come from that over time

00:31:20.420 | that I think are gonna be fascinating too.

00:31:22.380 | - You mean just like the nature of identity of who we are?

00:31:25.340 | Are we the people,

00:31:26.620 | you know how people do like summer beach body

00:31:29.820 | where the people will be, for the scan,

00:31:31.700 | they'll try to lose some weight and look their best

00:31:33.860 | and sexiest with the nice hair and everything like that.

00:31:36.660 | I mean, it does raise the question of,

00:31:41.240 | you know, if a lot of people are interacting

00:31:43.980 | with the digital version of ourselves,

00:31:45.780 | who are we really?

00:31:46.740 | Are we the entity driving the avatar?

00:31:50.060 | Are we the avatar?

00:31:52.060 | - Well, I mean, I think our physical bodies

00:31:53.820 | also fluctuate and change over time too.

00:31:55.820 | So I think there's a similar question of like,

00:31:58.560 | which version of that are we?

00:32:00.260 | Right, there's like the,

00:32:02.180 | I mean, and it's interesting identity question

00:32:04.500 | because all right, it's like, I don't know,

00:32:07.660 | it's like weight fluctuates or things like that.

00:32:10.500 | It's like, I think most people don't tend

00:32:12.300 | to think of themselves as the,

00:32:14.020 | I don't know, it's an interesting psychological question.

00:32:17.260 | Some, maybe some people,

00:32:18.660 | maybe a lot of people do think about themselves

00:32:20.220 | as the kind of worst version,

00:32:22.500 | but I think a lot of people probably think

00:32:24.580 | about themselves as the best version.

00:32:26.180 | And then it's like what you are on a day-to-day basis

00:32:29.220 | doesn't necessarily map to either of those.

00:32:33.500 | So I think that that's, yeah,

00:32:35.420 | there will definitely be a bunch of social scientists

00:32:39.180 | and folks who will have to, you know,

00:32:41.660 | and psychologists, really,

00:32:44.100 | there's gonna be a lot to understand

00:32:45.540 | about how our perception of ourselves and others

00:32:48.540 | has shifted from this.

00:32:51.380 | - Well, this might be a bit of a complicated

00:32:54.340 | and a dark question,

00:32:55.260 | but one of the first feelings I had experiencing this

00:32:59.420 | is I would love to talk to loved ones.

00:33:01.820 | And the next question I have is I would love to talk

00:33:04.340 | to people who are no longer here that are loved ones.

00:33:07.420 | So like, if you look into the future,

00:33:09.940 | is that something you think about,

00:33:11.340 | who people pass away,

00:33:12.860 | but they can still exist in the metaverse

00:33:14.780 | and you could still have, you know,

00:33:16.500 | talk to your father, talk to your grandfather

00:33:18.500 | and grandmother and a mother once they pass away.

00:33:22.060 | The power of that experience

00:33:25.020 | is one of the first things my mind jumped to

00:33:27.020 | 'cause it's like, this is so real.

00:33:29.620 | - Yeah, I think that there are a lot of norms

00:33:33.940 | and things that people have to figure out around that.

00:33:36.420 | There's probably some balance where, you know,

00:33:38.660 | if someone has lost a loved one and is grieving,

00:33:42.780 | there may be ways in which, you know,

00:33:46.580 | being able to interact or relive certain memories

00:33:49.980 | could be helpful,

00:33:51.300 | but then there's also probably an extent

00:33:52.940 | to which it could become unhealthy.

00:33:55.260 | And I mean, I'm not an expert in that.

00:33:57.500 | So I think we'd have to study that

00:33:59.500 | and understand it in more detail.

00:34:01.500 | We have, you know, a fair amount of experience

00:34:04.820 | with how to handle death and identity

00:34:08.340 | and people's digital content

00:34:09.860 | through social media already, unfortunately, right?

00:34:12.180 | Where there's, you know, unfortunately, you know,

00:34:15.500 | people who use our services die every day

00:34:18.900 | and their families, you know,

00:34:20.380 | often want to have access to their profiles.

00:34:22.540 | And we have whole protocols that we go through

00:34:24.900 | where there are certain parts of it

00:34:26.580 | that we try to memorialise

00:34:29.500 | so that way the family can get access to it.

00:34:32.500 | So that way the account doesn't just go away immediately.

00:34:34.900 | But then there are other things that are, you know,

00:34:36.460 | important kind of private things that that person has.

00:34:38.860 | Like we're not going to give the family access

00:34:40.180 | to someone's messages.

00:34:41.500 | You know, for example.

00:34:42.740 | So, yeah, I think that there's some best practices,

00:34:47.020 | I think, from the current digital world

00:34:49.380 | that we'll carry over.

00:34:50.420 | But yeah, I think that this will enable

00:34:53.540 | some different things.

00:34:55.100 | Another version of this is how this intersects with AIs,

00:34:58.820 | right, because, and one of the things

00:35:01.700 | that we're really focused on is,

00:35:05.980 | you know, we want the world to evolve in a way

00:35:08.580 | where there isn't like a single AI super intelligence,

00:35:11.740 | but where, you know, a lot of people are empowered

00:35:13.900 | by having AI tools to do their jobs

00:35:17.500 | and, you know, make their lives better.

00:35:19.180 | And if you're a creator, right,

00:35:21.180 | and if you run a podcast like you do,

00:35:25.580 | then you have a big community of people

00:35:28.020 | who are super interested to talk to you.

00:35:30.220 | I know you'd love to cultivate that community

00:35:33.380 | and you interact with them online

00:35:34.660 | outside of the podcast as well.

00:35:36.540 | But I mean, there's way more demand

00:35:38.500 | both to interact with you,

00:35:40.300 | and I'm sure you'd love to interact with the community more,

00:35:43.380 | but you just are limited by the number of hours in the day.

00:35:45.980 | So, you know, at some point, I think,

00:35:48.140 | making it so that you could build an AI version of yourself

00:35:53.140 | that could interact with people, you know,

00:35:56.140 | not after you die, but while you're here

00:35:58.100 | to help people kind of fulfill this desire

00:36:02.460 | to interact with you and your desire to build a community.

00:36:05.700 | And there's a lot of interesting questions around that.

00:36:10.580 | And, you know, that's obviously,

00:36:11.580 | it's not just in the metaverse.

00:36:13.060 | I think, you know, we'd want to make that work,

00:36:15.740 | you know, across all the messaging platforms,

00:36:18.220 | you know, WhatsApp and Messenger and Instagram Direct.

00:36:20.660 | But, you know, there's certainly, you know,

00:36:22.300 | a version of that where if you could have

00:36:24.740 | an avatar version of yourself in the metaverse

00:36:26.660 | that people can interact with,

00:36:27.780 | and you could define that sort of an AI version,

00:36:31.140 | where, you know, people know

00:36:32.260 | that they're interacting with an AI,

00:36:33.620 | that it's not, you know, the kind of physical version of you,

00:36:36.860 | but maybe that AI, even if they know it's an AI,

00:36:39.340 | is the next best thing, because they're probably

00:36:41.140 | not going to, you know, necessarily all get

00:36:42.660 | to interact with you directly.

00:36:45.140 | I think that that could be a really compelling experience.

00:36:48.220 | There's a lot of things that we need to get right about it,

00:36:51.660 | that, you know, we're not ready to release the version

00:36:55.260 | that a creator can kind of build a version of themselves yet,

00:36:58.020 | but we're starting to experiment with it

00:37:00.380 | in terms of releasing a number of AIs

00:37:03.060 | that people can interact with in different ways.

00:37:05.300 | And I think that that is also just going to be

00:37:08.460 | a very powerful, you know, set of capabilities

00:37:11.620 | that people have over time.

00:37:13.860 | - So you've made major strides in developing

00:37:16.580 | these early AI personalities with the idea

00:37:21.580 | where you can talk to them across the meta apps

00:37:24.420 | and have like interesting, unique kind of conversations.

00:37:29.260 | What, can you describe your vision there

00:37:31.100 | in these early strides and what are some

00:37:32.740 | technical challenges there?

00:37:34.740 | - Yeah, so, I mean, a lot of the vision comes from

00:37:37.820 | this idea that, you know, I don't think we necessarily

00:37:41.260 | want there to be like one big super intelligence.

00:37:44.060 | We want to empower everyone to both, you know,

00:37:46.900 | have more fun, accomplish their business goals,

00:37:49.260 | you know, just everything that they're trying to do.

00:37:52.340 | And, you know, we don't tend to have, you know,

00:37:54.500 | one person that we work with on everything.

00:37:56.180 | And I don't think in the future we're going to have,

00:37:58.060 | you know, one AI that we work with.

00:37:59.860 | I think you're going to want a variety of these.

00:38:03.660 | So there are a bunch of different uses.

00:38:07.580 | If some will be kind of more assistant oriented,

00:38:10.340 | there's a sort of the kind of plain and simple one

00:38:13.260 | that we're building is called just meta AI.

00:38:15.620 | It's simple, you can chat with it in any of your threads.

00:38:20.420 | It doesn't have a face, right?

00:38:21.820 | It's just kind of more vanilla and neutral

00:38:26.460 | and kind of factual, but it can help you

00:38:28.980 | with a bunch of stuff.

00:38:30.460 | Then there were a bunch of cases

00:38:32.060 | that are more kind of business oriented.

00:38:35.740 | So let's say you want to contact a small business,

00:38:38.500 | you know, similarly, you know,

00:38:41.060 | that business probably doesn't want to have to

00:38:43.180 | staff someone to man the phones.

00:38:45.460 | And you probably don't want to wait on the phone

00:38:46.700 | to talk to someone, but you're having someone

00:38:48.620 | who you can just like talk to in a natural way

00:38:50.500 | who can help you if you're having an issue with a product

00:38:53.300 | or if you want to make a reservation

00:38:55.260 | or if you want to buy something online.

00:38:58.260 | Having the ability to do that

00:38:59.820 | and have a natural conversation

00:39:01.620 | rather than navigate some website

00:39:03.060 | or have to call someone and wait on hold

00:39:05.060 | is gonna be really good, both for the businesses

00:39:07.580 | and for normal people who want to interact with businesses.

00:39:11.300 | So I think stuff like that makes sense.

00:39:13.260 | Then there are gonna be a bunch of use cases

00:39:15.660 | that I think are just fun, right?

00:39:18.420 | So I think people are gonna,

00:39:20.340 | I think that there will be AIs that can tell jokes.

00:39:23.700 | So you can put them into chat thread with friends.

00:39:25.660 | I mean, I think a lot of this,

00:39:26.500 | because we're like a social company, right?

00:39:28.260 | I mean, we're fundamentally around

00:39:30.660 | helping people connect in different ways.

00:39:32.740 | And part of what I'm excited about is

00:39:35.140 | how do you enable these kind of AIs

00:39:37.780 | to facilitate connection between two people or more,

00:39:42.260 | put them in a group chat,

00:39:43.580 | make the group chat more interesting

00:39:45.380 | around whatever your interests are,

00:39:47.540 | sports, fashion, trivia.

00:39:51.140 | - Video games.

00:39:52.340 | I love the idea of playing,

00:39:54.380 | I think you mentioned Baldur's Gate.

00:39:56.340 | An incredible game,

00:39:57.660 | just having an AI that you play together with.

00:40:01.580 | I mean, that seems like a small thing,

00:40:04.260 | but it could deeply enrich the gaming experience.

00:40:08.660 | - I do think that AIs will make the NPCs

00:40:11.060 | a lot better in games too.

00:40:12.260 | So that's a separate thing that I'm pretty excited about.

00:40:15.100 | But yeah, I mean, one of the AIs that we've built

00:40:19.820 | that just in our internal testing,

00:40:21.380 | people have loved the most is like a adventure,

00:40:24.820 | text-based, like a dungeon master.

00:40:29.260 | - Yeah, nice.

00:40:30.380 | - And I think part of what has been fun,

00:40:34.900 | and we talked about this a bit,

00:40:36.100 | but we've gotten some real kind of cultural figures

00:40:39.820 | to play a bunch of these folks

00:40:41.780 | and be the embodiment and the avatar of them.

00:40:43.940 | So Snoop Dogg is the dungeon master,

00:40:46.740 | which I think is just hilarious.

00:40:48.700 | - In terms of the next steps of,

00:40:52.140 | you mentioned Snoop, to create a Snoop AI,

00:40:55.460 | so basically AI personality replica,

00:40:58.500 | a copy, or not a copy, maybe inspired by Snoop.

00:41:03.500 | What are some of the technical challenges of that?

00:41:06.700 | What does that experience look like for Snoop

00:41:09.060 | to be able to create that AI?

00:41:11.140 | - So starting off, creating new personas is easier

00:41:16.140 | because it doesn't need to stick exactly

00:41:20.020 | to what that physical person would want,

00:41:23.420 | how they'd want to be represented.

00:41:24.900 | It's like it's just a new character that we created.

00:41:26.700 | So even though, so Snoop in that case is,

00:41:29.340 | he's basically an actor.

00:41:31.780 | He's playing the dungeon master,

00:41:33.420 | but it's not Snoop Dogg.

00:41:34.620 | Whoever the dungeon master is.

00:41:37.860 | If you want to actually make it so that

00:41:41.300 | you have an AI embodying a real creator,

00:41:45.260 | there's a whole set of things that you need to do

00:41:47.540 | to make sure that that AI is not going to say things

00:41:51.900 | that the creator doesn't want, right?

00:41:53.860 | And that the AI is going to know things

00:41:58.860 | and be able to represent things in the way

00:42:01.300 | that the creator would want,

00:42:03.140 | the way that the creator would know.

00:42:04.940 | So I think that it's less of a question around

00:42:11.580 | like having the avatar express them.

00:42:15.060 | I mean, that I think where,

00:42:16.780 | it's like, well, we have our kind of V1 of that

00:42:19.260 | that we'll release soon after Connect,

00:42:22.860 | but that'll get better over time.

00:42:24.780 | But a lot of this is really just about continuing

00:42:27.540 | to make the models for these AIs

00:42:29.900 | so that they're just more and more,

00:42:32.380 | I don't know, you could say like reliable

00:42:35.180 | or predictable in terms of what they'll communicate.

00:42:37.100 | So that way, when you want to create the Lex assistant AI

00:42:42.100 | that your community can talk to,

00:42:45.100 | you don't program them like normal computers,

00:42:49.220 | you're training them, they're AI models,

00:42:50.820 | not kind of normal computer programs,

00:42:53.820 | but you want to get it to be predictable enough

00:42:57.100 | so that way you can set some parameters for it.

00:42:59.820 | And even if it isn't perfect all the time,

00:43:03.860 | you want it to generally be able to stay

00:43:05.620 | within those bounds.

00:43:06.500 | So that's a lot of what I think we need to nail

00:43:11.100 | for the creators.

00:43:12.500 | And that's why that one's actually a much harder problem,

00:43:15.380 | I think, than starting with new characters

00:43:18.540 | that you're creating from scratch.

00:43:19.620 | So that one I think will probably start releasing

00:43:23.860 | sometime next year, not this year,

00:43:26.660 | but experimenting with existing characters

00:43:28.740 | and the assistant and games

00:43:30.620 | and a bunch of different personalities

00:43:32.140 | and experimenting with some small businesses.

00:43:36.020 | I think that that stuff will be ready to do this year

00:43:38.060 | and we're rolling it out basically right after Connect.

00:43:42.140 | - Yeah, I'm deeply entertained by the possibility

00:43:45.100 | of me sitting down with myself and saying,

00:43:46.860 | "Hey man, you need to stop the dad jokes," or whatever.

00:43:51.860 | - I think the idea of a podcast between you

00:43:54.980 | and AI assistant Lex podcast.

00:43:58.380 | - I mean, there's just even the experience

00:44:02.660 | of a Kodak Avatar, being able to freeze yourself,

00:44:05.500 | like basically first mimic yourself.

00:44:08.060 | So everything you do, you get to see yourself do it.

00:44:10.700 | That's a surreal experience.

00:44:12.340 | That feels like if I was like an ape looking in a mirror

00:44:15.740 | for the first time, realizing like, oh, that's you.

00:44:18.700 | But then freezing that and being able to look around

00:44:21.660 | like I'm looking at you.

00:44:23.700 | I don't know how to put it into words,

00:44:26.500 | but it just feels like a fundamentally new experience.

00:44:29.060 | Like I'm seeing maybe color for the first time.

00:44:32.300 | I'm experiencing a new way of seeing the world

00:44:37.260 | for the first time.

00:44:38.460 | Because it's physical reality, but it's digital.

00:44:42.140 | And realizing that that's possible, it's just,

00:44:46.100 | it's blowing my mind.

00:44:48.740 | It's just really exciting.

00:44:50.260 | 'Cause I lived most of my life before the internet

00:44:53.820 | and experiencing the internet,

00:44:55.540 | experiencing voice communication and video communication.

00:44:59.780 | You think like, well, there's a ceiling to this,

00:45:02.580 | but this is making me feel like, oh, there might not be.

00:45:05.740 | There might be that blend of physical reality

00:45:08.620 | and digital reality.

00:45:09.460 | It's actually what the future is.

00:45:12.380 | - Yeah, I think so.

00:45:13.220 | - It's a weird experience.

00:45:14.900 | It feels like the early days of a totally new way of living.

00:45:19.900 | And there's a lot of people that kind of complain,

00:45:22.060 | well, you know, the internet, that's not reality.

00:45:26.900 | You need to turn all that off and go in nature.

00:45:30.220 | But this feels like this will make those people happy,

00:45:33.780 | I feel like, 'cause it feels real.

00:45:36.020 | The flaws and everything.

00:45:37.500 | - Yeah, well, I mean, a big part of how we're trying

00:45:39.780 | to design these new computing products

00:45:43.260 | is that they should be physical.

00:45:45.020 | Right, I think that's a big part of the issue

00:45:46.660 | with computers and TVs and even phones is like,

00:45:50.980 | yeah, I mean, maybe you can interact with them

00:45:52.420 | in different places, but they're fundamentally,

00:45:54.500 | like, you're sitting, you're still.

00:45:56.660 | And I mean, people are just not meant to be that way.

00:45:59.100 | I mean, I think you and I have this shared passion

00:46:02.220 | for sports and martial arts and doing stuff like that.

00:46:05.340 | We're just moving around.

00:46:06.420 | It's like so much of what makes us people is like,

00:46:09.140 | you move around, we're not just like a brain in a tank.

00:46:12.420 | It's the where, the human experience is a physical one.

00:46:16.580 | And so it's not just about having the immersive expression

00:46:20.660 | of the digital world, it's about being able

00:46:22.500 | to really natively bring that together.

00:46:24.860 | And I do really think that the real world

00:46:28.580 | is this mix of the physical and the digital.

00:46:31.100 | Where the digital is, there's too much digital

00:46:33.140 | at this point for it to just be siloed to a small screen,

00:46:36.020 | but the physical is too important.

00:46:37.540 | So you don't want to just sit down all day long at a desk.

00:46:41.580 | So I think that this is, yeah,

00:46:44.980 | I do think that this is the future.

00:46:46.180 | This is, I think the kind of philosophical way

00:46:50.260 | that I would want the world to work in the future

00:46:52.060 | is a much more coherently blended physical and digital world.

00:46:56.060 | - There may be some difficult philosophical

00:46:59.620 | and unethical questions we have to figure out as a society.

00:47:03.060 | Maybe you can comment on this.

00:47:04.500 | So the metaverse seems to enable,

00:47:09.500 | sort of unlock a lot of experiences

00:47:11.700 | that we don't have in the physical world.

00:47:13.820 | And the question is like,

00:47:16.660 | what is and isn't allowed in the metaverse?

00:47:19.260 | You know, in video games, we allow all kinds of crazy stuff.

00:47:25.060 | And in physical reality, you know, a lot of that is illegal.

00:47:29.940 | So where's that line?

00:47:31.260 | Where's that gray area between video game

00:47:33.220 | and physical reality?

00:47:34.580 | Do you have a sense of that?

00:47:36.780 | - Well, I think, I mean, there are content policies

00:47:39.220 | and things like that, right?

00:47:40.340 | In terms of what people are allowed to create.

00:47:42.820 | But I mean, a lot of the rules around physical,

00:47:45.940 | I think you try to have a society

00:47:48.100 | that is as free as possible,

00:47:49.540 | meaning that people can do as much of what they want

00:47:52.100 | unless you're going to do damage to other people.

00:47:54.500 | And infringe on their rights.

00:47:57.780 | And the idea of damage is somewhat different

00:48:00.380 | in a digital environment.

00:48:02.420 | I mean, when I get into some world with my friends,

00:48:06.540 | the first thing we start doing is shooting each other,

00:48:08.580 | which obviously we would not do in the physical world

00:48:10.460 | 'cause you'd hurt each other.

00:48:12.740 | But in a game, that's like just, it's almost,

00:48:16.100 | you know, it's like just fun.

00:48:17.180 | And even like the lobby of a game, right?

00:48:20.500 | It's like, it's not even bearing on the game,

00:48:22.820 | which is kind of like a funny sort of humorous thing to do.

00:48:26.900 | So it's like, is that problematic?

00:48:28.500 | I don't think so because it's fundamentally,

00:48:30.340 | it's not, you're not causing harm in that world.

00:48:32.500 | So I think that the part of the question

00:48:36.060 | that I think we need to figure out

00:48:38.020 | is what are the ways where things could have been harmful

00:48:42.180 | in the physical world that will now be freed from that?

00:48:44.380 | And therefore there should be fewer restrictions

00:48:46.180 | in the digital world.

00:48:47.220 | And then there might be new ways

00:48:50.580 | in which there could be harm in the digital world

00:48:52.700 | that there weren't the case before.

00:48:54.140 | So there's more anonymity, right?

00:48:55.740 | It's, you know, when you show up to a restaurant

00:48:59.420 | or something, it's like all the norms

00:49:00.620 | where you pay the bill at the end,

00:49:02.620 | it's because, you know, you have one identity

00:49:06.060 | and, you know, if you stiff them,

00:49:08.820 | then like, you know, life is a repeat game

00:49:11.460 | and that's not gonna work out well for you.

00:49:13.300 | But in a digital world where you can be anonymous

00:49:16.220 | and show up in different ways,

00:49:18.620 | I think the incentive to act like a good citizen

00:49:21.380 | can be a lot less and that causes a lot of issues

00:49:23.660 | and toxic behavior.

00:49:24.660 | So that needs to get sorted out.

00:49:26.620 | So I think in terms of what is allowed,

00:49:30.060 | I think you wanna just look at what are the damages.

00:49:33.380 | But then there's also other things

00:49:35.140 | that are not related to kind of harm,

00:49:38.580 | less about what should be allowed

00:49:40.260 | and more about what will be possible

00:49:42.420 | that are more about the laws of physics, right?

00:49:44.220 | It's like, if you wanted to travel to see me in person,

00:49:48.300 | you'd have to get on a plane

00:49:50.740 | and that would like, you know, take a few hours to get here.

00:49:53.740 | Whereas, you know, we could just jump in a conference room

00:49:57.300 | and, you know, put on these headsets

00:49:59.260 | and we're basically teleported into a space

00:50:02.340 | where we're, you know, it feels like we're together.

00:50:05.260 | So that's a very novel experience

00:50:07.300 | that it breaks down some things

00:50:10.220 | that previously would have defied the laws of physics

00:50:13.140 | for what it would take to get together.

00:50:14.900 | And I think that that will create

00:50:16.180 | a lot of new opportunities, right?

00:50:17.820 | So, and one of the things that I'm curious about is,

00:50:21.220 | you know, there are all these debates right now

00:50:22.420 | about remote work or people being together.

00:50:25.820 | And, you know, I think this gets us a lot closer

00:50:28.260 | to being able to work physically in different places,

00:50:31.020 | but actually have it feel like we're together.

00:50:33.340 | So, you know, I think that the dream

00:50:35.460 | is that people will one day be able

00:50:37.700 | to just work wherever they want,

00:50:39.740 | but we'll have all the same opportunities

00:50:41.500 | because you'll be able to feel

00:50:42.340 | like you're physically together.

00:50:43.300 | I think we're not there today

00:50:44.900 | with just video conferencing

00:50:47.420 | and the basic technologies that we have.

00:50:49.020 | But I think part of the idea is that

00:50:50.580 | with something like this,

00:50:51.860 | over time you could get closer to that.

00:50:53.540 | And that would open up a lot of opportunities, right?

00:50:55.340 | Because then people could live physically where they want

00:50:58.260 | while still being able to get the benefits

00:51:00.780 | of being physically or kind of feeling like you're together

00:51:04.660 | with people at work,

00:51:05.540 | all the ways that that helps to build more culture

00:51:08.500 | and build better relationships and build trust,

00:51:11.300 | which I think are real issues

00:51:12.660 | that if you're not seeing people in person ever.

00:51:16.540 | So yeah, I don't know.

00:51:17.780 | I think it's going to be,

00:51:19.020 | it's very hard from first principles

00:51:20.940 | to think about all the implications

00:51:23.140 | of a technology like this and, you know,

00:51:27.580 | all the good and the things that you need to mitigate.

00:51:30.540 | So you try to do your best to kind of envision

00:51:33.260 | what things are going to be like

00:51:34.380 | and accentuate the things that they're going to be awesome

00:51:37.060 | and hopefully mitigate some of the downside things.

00:51:39.660 | But, you know, the reality is

00:51:41.620 | that we're going to be building this out one year at a time.

00:51:43.780 | It's going to take a while.

00:51:45.540 | So we're going to just get to see how it evolves

00:51:48.420 | and what developers and different folks do with it.

00:51:51.060 | - If you could comment,

00:51:53.940 | this might be a bit of a very specific technical question,

00:51:57.220 | but Lama 2 is incredible.

00:51:59.380 | It's the, you've released it recently.

00:52:02.340 | There's already been a lot of exciting developments

00:52:06.060 | around it.

00:52:06.980 | Is there, what's your sense about its release?

00:52:10.180 | And is there a Lama 3 in the future?

00:52:13.900 | - Yeah, I mean, I think on the last podcast

00:52:16.700 | that we did together,

00:52:17.860 | we were talking about the debate that we were having

00:52:19.780 | around open sourcing Lama 2.

00:52:21.700 | And I'm glad that we did.

00:52:25.260 | You know, I think at this point,

00:52:27.380 | there's the value of open sourcing

00:52:29.860 | a foundation model like Lama 2,

00:52:32.340 | is significantly greater than the risks.

00:52:36.460 | And in my view, I mean, we did,

00:52:37.820 | we spent a lot of time to get very rigorous assessment

00:52:41.020 | of that and red teaming it,

00:52:43.020 | but I'm very glad that we released Lama 2.

00:52:44.980 | I think the reception has been,

00:52:46.540 | it's just been really exciting

00:52:48.860 | to see how excited people have been about it.

00:52:52.580 | And it's gotten way more downloads and usage

00:52:56.580 | than I would have even expected.

00:52:58.660 | And I was pretty optimistic about it.

00:53:00.500 | So that's been great.

00:53:05.260 | Lama 3, I mean, there's always another model

00:53:10.260 | that we're training.

00:53:11.100 | So, I mean, it's, you know, for right now,

00:53:13.380 | you know, we built, we trained Lama 2

00:53:15.940 | and we released it as an open source model.

00:53:18.180 | And right now the priority is building that

00:53:21.340 | into a bunch of the consumer products,

00:53:23.620 | all the different AIs and a bunch of different products

00:53:28.620 | that we're basically building as consumer products.

00:53:30.740 | 'Cause Lama 2 by itself, it's not a consumer product, right?

00:53:33.060 | It's more of a piece of infrastructure

00:53:34.580 | that people could build things with.

00:53:36.860 | So that's been the big priority

00:53:38.860 | is kind of continuing to fine tune

00:53:41.300 | and kind of just get Lama 2

00:53:45.180 | and it's little, the branches that we built off of it

00:53:49.740 | ready for consumer products that hopefully, you know,

00:53:52.780 | hundreds of millions of people

00:53:53.940 | will enjoy using those products in billions one day.

00:53:58.380 | But yeah, I mean, we're also working

00:54:00.060 | on the future foundation models.

00:54:03.140 | And I don't have anything new or news on that.

00:54:08.140 | I don't know, you know,

00:54:09.620 | I don't know exactly when it's gonna be ready.

00:54:12.740 | I think just like we had a debate around Lama 2

00:54:15.700 | and open sourcing it,

00:54:16.860 | I think we'll need to have a similar debate

00:54:20.420 | and process to red team this

00:54:21.900 | and make sure that this is safe.

00:54:23.260 | But, and my hope is that we'll be able

00:54:25.060 | to open source this next version when it's ready too.

00:54:27.820 | But that's not, we're not, you know,

00:54:31.220 | close to doing that this month.

00:54:32.420 | I mean, this is, that's just,

00:54:33.940 | it's a thing that we're still somewhat early in working on.

00:54:37.940 | - Well, in general, thank you so much

00:54:39.500 | for open sourcing Lama 2 and for being transparent

00:54:41.820 | about all the exciting developments around AI.

00:54:45.780 | I feel like that's contributing

00:54:48.580 | to a really awesome conversation about where we go with AI.

00:54:51.860 | And obviously it's really interesting

00:54:54.140 | to see all the same kind of technology integrated

00:54:56.500 | into these personalized AI systems with the AI personas,

00:55:02.220 | which I think when you put it in people's hands

00:55:05.020 | and they get to have conversations with these AI personas,

00:55:08.140 | you'll get to see like interesting failure cases,

00:55:11.500 | like where the things are dumb

00:55:13.140 | or they go into weird directions

00:55:15.060 | or, and we get to learn as a society together,

00:55:17.300 | what's too far, what's interesting, what's fun,

00:55:21.300 | how much personalization is good, how much generic is good.

00:55:25.180 | And we get to learn all of this.

00:55:26.540 | And you probably don't know this yourself.

00:55:27.980 | Like we have to all figure it out by using it, right?

00:55:31.420 | - Yeah, I mean, part of what we're trying to do

00:55:32.740 | with the initial AI's launch is having a diversity

00:55:37.540 | of different use cases,

00:55:39.420 | just so that people can try different things.

00:55:41.340 | 'Cause I don't know what's gonna work.

00:55:42.380 | I mean, are people gonna like playing

00:55:44.260 | in the text-based adventure games?

00:55:45.860 | Are they going to like having a comedian

00:55:49.980 | who can add jokes to threads

00:55:53.380 | or they can want to interact with historical figures?

00:55:56.820 | You know, we made one of Jane Austen

00:55:59.060 | and one of Marcus Aurelius,

00:56:02.020 | and I'm curious to see how that goes.

00:56:04.140 | - I'm excited for both.

00:56:05.620 | As a big fan, I'm excited for both.

00:56:07.500 | I have conversations with them.

00:56:09.620 | I mean, yeah, that's, you know,

00:56:11.500 | and I am also excited to see, you know, the internet,

00:56:13.980 | I don't know if you heard, can get kind of weird

00:56:16.980 | and I applaud them for it.

00:56:18.100 | So I get to see it. - I've heard that, yeah.

00:56:19.300 | - Yeah, so it'd be nice to see how weird they take it,

00:56:22.940 | what kind of memes are generated from this.

00:56:24.940 | And I think all of it is,

00:56:26.900 | especially in this early stages of development,

00:56:29.700 | as we progress towards AGI,

00:56:31.620 | it's good to learn by playing with those systems

00:56:34.900 | and interacting with them at like a large scale,

00:56:37.420 | like you said.

00:56:38.540 | - Yeah, totally.

00:56:39.380 | I mean, that's why,

00:56:40.300 | once we're starting out with a set

00:56:42.380 | and then we're also working on this platform

00:56:46.140 | that we call AI Studio,

00:56:47.780 | that's gonna make it so that, you know,

00:56:49.460 | over time anyone will be able to create one of these AIs,

00:56:53.660 | almost like they create any other UGC content

00:56:56.860 | across the platform.

00:56:57.860 | So I'm excited about that.

00:56:59.340 | I think that to some degree,

00:57:00.980 | we're not gonna see the full potential of this

00:57:03.740 | until you just have the full creativity

00:57:06.740 | of the whole community being able to build stuff.

00:57:08.700 | But there's a lot of stuff that we need to get right.

00:57:11.780 | So I'm excited to take this in stages.

00:57:14.900 | I don't think anyone out there

00:57:17.780 | is really doing what we're doing here.

00:57:20.900 | I think that there are people

00:57:22.940 | who are doing kind of like fictional

00:57:24.460 | or consumer-oriented character type stuff,

00:57:26.700 | but the extent to which we're building it out

00:57:29.820 | with the avatars and expressiveness

00:57:34.820 | and making it so that they can interact

00:57:36.660 | across all of the different apps

00:57:39.260 | and they'll have profiles

00:57:41.700 | and we'll be able to engage people

00:57:43.660 | on Instagram and Facebook.

00:57:44.620 | I think it's just, it's gonna be really fun.

00:57:48.900 | - Well, I'm still,

00:57:50.700 | so we're talking about AI,

00:57:51.620 | but I'm still blown away this entire time

00:57:53.820 | that I'm talking to Mark Zuckerberg

00:57:56.300 | and you're not here,

00:57:57.540 | but you feel like you're here.

00:57:59.500 | I've done quite a few intimate conversations

00:58:02.780 | with people alone in a room,

00:58:04.260 | and this feels like that.

00:58:05.980 | So I keep forgetting for long stretches of time

00:58:08.500 | that we're not in the same room.

00:58:10.660 | And for me to imagine a future

00:58:13.100 | where I can, with a snap of a finger,

00:58:14.820 | do that with anyone in my life,

00:58:16.820 | the way we can just call right now

00:58:18.700 | and have this kind of shallow 2D experience,

00:58:21.940 | to have this experience

00:58:24.340 | like we're sitting next to each other,

00:58:26.180 | is like, I don't think I can,

00:58:29.340 | I don't think we can even imagine

00:58:31.140 | how that changes things,

00:58:33.140 | where you can immediately have intimate

00:58:36.340 | one-on-one conversations with anyone.

00:58:39.020 | That might, in a way,

00:58:40.500 | we might not even predict change civilization.

00:58:44.180 | - Well, I mean, this is a lot of the thesis

00:58:45.620 | behind the whole metaverse,

00:58:47.300 | is giving people the ability to feel

00:58:48.860 | like you're present with someone.

00:58:50.340 | I mean, this is the main thing

00:58:51.820 | I talk about all the time,

00:58:52.780 | but I do think that there's a lot to process about it.

00:58:56.100 | I mean, from my perspective,

00:58:57.540 | I mean, I'm definitely here.

00:58:59.340 | We're just not, we're not physically in the same place.

00:59:02.220 | It's not like you're not talking to an AI, right?

00:59:05.260 | So I think the thing that's novel

00:59:09.620 | is the ability to convey through technology

00:59:12.820 | a sense of almost physical presence.

00:59:16.860 | So the thing that is not physically real

00:59:19.940 | is us being in the same physical place,

00:59:23.900 | but kind of everything else is.

00:59:27.060 | And I think that that gets to this

00:59:29.100 | somewhat philosophical question

00:59:30.580 | about what is the nature of kind of the modern real world?

00:59:34.540 | And I just think that that's,

00:59:35.860 | it really is this combination of a physical world

00:59:40.340 | and the presence that we feel,

00:59:42.140 | but also being able to combine that

00:59:43.940 | with this increasingly rich and powerful

00:59:46.140 | and capable digital world that we have

00:59:48.460 | and all of the innovation that's getting created there.

00:59:52.380 | So I think it's super exciting

00:59:54.620 | because I mean, the digital world is just increasing

00:59:57.380 | in its capability and our ability to do awesome things,

01:00:01.660 | but the physical world is so profound.

01:00:03.260 | And that's a lot of what makes us human

01:00:05.020 | is that we're physical beings.

01:00:07.620 | So I don't think we want to run away from that

01:00:09.060 | and just spend all day on a screen.

01:00:10.780 | And that's like, you know,

01:00:12.260 | it's one of the reasons why I care so much

01:00:13.900 | about helping to shape and accelerate

01:00:16.860 | these future computing platforms.

01:00:18.220 | I just think this is so powerful.

01:00:19.820 | And it's, you know, even though the current version of this

01:00:23.340 | is like you're wearing a headset,

01:00:24.980 | I just think this is going to be by far the most human

01:00:29.060 | and social computing platform that has ever existed.

01:00:32.820 | And that's what makes me excited.

01:00:36.220 | - Yeah, I think just to linger

01:00:38.300 | on this kind of changing nature of reality,

01:00:40.820 | like of what is real,

01:00:43.220 | maybe shifting it towards the sort of consciousness.

01:00:48.220 | So what is real is the subjective experience

01:00:52.540 | of a thing that makes it feel real

01:00:54.860 | versus necessarily being in the same physical space.

01:00:58.140 | 'Cause it feels like we're in the same physical space.

01:01:01.100 | And that the conscious experience of it,

01:01:03.740 | that's probably what is real,

01:01:05.460 | not like that the space time, like the physics of it,

01:01:09.820 | like you're basically breaking physics

01:01:11.780 | and focusing on the consciousness.

01:01:13.900 | That's what's real.

01:01:15.220 | It's just whatever's going on inside my head.

01:01:17.460 | - But there are a lot of social and psychological things

01:01:19.900 | that go along with that experience

01:01:23.460 | that was previously only physical presence, right?

01:01:26.300 | I think that there's like an intimacy, a trust,

01:01:29.420 | you know, there's a level of communication

01:01:32.740 | because so much of communication is nonverbal

01:01:34.660 | and is based on expressions that you're kind of,

01:01:38.140 | you know, you're sharing with someone

01:01:40.460 | when you're in this kind of environment.

01:01:43.460 | And before those things would have only been possible,

01:01:46.060 | you know, had I gotten on a plane and flown to Austin

01:01:49.900 | and sat physically with you in the same place.

01:01:52.860 | So I think we're basically short-cutting

01:01:56.620 | those laws of physics and delivering the social

01:02:00.860 | and psychological benefits of being able to be present

01:02:04.180 | and feel like you're there with another person,

01:02:06.100 | which I think are real benefits to anyone in the world.

01:02:10.580 | And I think that that, like you said,

01:02:12.380 | I mean, I think that is going to be a very profound thing.

01:02:14.740 | And that a lot of that is,

01:02:16.620 | that's the promise of the metaverse and what, you know,

01:02:20.340 | why I just, why I think that that's the next frontier

01:02:24.420 | for what we're working on.

01:02:25.620 | You know, I started working on social networks

01:02:27.860 | when they were primarily text,

01:02:29.500 | where the first version of Facebook, your profile,

01:02:31.900 | you know, you had one photo and the rest of it

01:02:33.780 | was like lists of things that you were interested in.

01:02:36.180 | And then we kind of went through the period

01:02:38.060 | where we were doing photos.

01:02:39.660 | And now we're kind of in the period

01:02:41.500 | where most of the content is video,

01:02:43.580 | but there's a clear trend where, you know, over time,

01:02:47.300 | the way that we want to express ourselves

01:02:49.380 | and kind of get insight and content

01:02:52.940 | about the world around us gets increasingly just richer

01:02:56.140 | and more vivid.

01:02:57.460 | And I think the ability to be immersed

01:03:00.540 | and feel present with the people around you

01:03:03.060 | or the people who you care about is, from my perspective,

01:03:06.100 | clearly the next frontier.

01:03:08.140 | It just so happens that it's

01:03:09.980 | incredibly technologically difficult, right?

01:03:12.340 | It requires building up these new computing platforms

01:03:14.540 | and completely new software stacks to deliver that.

01:03:17.420 | But I mean, I kind of feel like

01:03:18.700 | that's what we're here to do as a company.

01:03:21.380 | - Well, I really love the connection

01:03:24.460 | you have through conversation.

01:03:26.260 | And so for me, this photo realism is really, really exciting.

01:03:29.620 | I'm really excited for this future.

01:03:32.580 | And thank you for building it.

01:03:35.900 | Thanks to you and thanks to the amazing Meta teams

01:03:38.740 | that I've met, the engineers,

01:03:40.980 | and just everybody I've met here.

01:03:42.660 | Thank you for helping to build this future.

01:03:46.740 | And thank you, Mark, for talking to me inside the Metaverse.

01:03:51.540 | This is blowing my mind.

01:03:52.940 | I can't quite express.

01:03:54.140 | I would love to measure my heart rate this whole time.

01:03:56.260 | Would be hilarious if you're actually

01:03:57.860 | like sitting on a beach right now.

01:04:00.660 | - I'm not, I'm in a conference room.

01:04:02.340 | - Okay, well, I'm at a beach and not wearing any pants.

01:04:05.860 | I'm really sorry about that for anyone else

01:04:08.140 | who's watching me in physical space.

01:04:09.940 | Anyway, thank you so much for talking today.

01:04:11.620 | This really blew my mind.

01:04:14.020 | It's one of the most incredible experiences of my life.

01:04:15.900 | So thank you for giving that to me.

01:04:17.180 | - Awesome, awesome.

01:04:18.020 | Glad you got to check it out.

01:04:19.100 | And it's always fun to talk.

01:04:21.380 | All right, I'll catch you soon.

01:04:22.740 | See ya. - See you later.

01:04:24.340 | This is so, so amazing, man.

01:04:26.700 | This is so amazing.

01:04:29.660 | (upbeat music)

01:04:32.260 | (upbeat music)

01:04:34.860 | [BLANK_AUDIO]

Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398

Chapters