Rohit Prasad: Deep Learning is Not Enough to Solve Reasoning | AI Podcast Clips
00:00:00.000 | - So deep learning has been at the core
00:00:03.360 | of a lot of this technology.
00:00:05.480 | Are you optimistic about the current deep learning
00:00:07.200 | approaches to solving the hardest aspects
00:00:09.560 | of what we're talking about?
00:00:11.300 | Or do you think there will come a time
00:00:13.440 | where new ideas need to,
00:00:15.280 | for the, you know, if we look at reasoning.
00:00:17.400 | So open AI, deep mind, a lot of folks are now
00:00:19.920 | starting to work in reasoning,
00:00:21.920 | trying to see how we can make neural networks reason.
00:00:24.640 | Do you see that new approaches need to be invented
00:00:28.560 | to take the next big leap?
00:00:31.400 | - Absolutely.
00:00:32.220 | I think there has to be a lot more investment
00:00:35.280 | and I think in many different ways.
00:00:37.440 | And there are these, I would say nuggets of research forming
00:00:41.000 | in a good way, like learning with less data
00:00:44.160 | or like zero short learning, one short learning.
00:00:47.720 | - And the active learning stuff you've talked about
00:00:49.480 | is incredible.
00:00:51.080 | - Yes, so transfer learning is also super critical,
00:00:53.720 | especially when you're thinking about applying knowledge
00:00:56.660 | from one task to another or one language to another, right?
00:01:00.120 | That's really ripe.
00:01:01.080 | So these are great pieces.
00:01:03.380 | Deep learning has been useful too.
00:01:04.880 | And now we are sort of matting deep learning
00:01:06.960 | with transfer learning and active learning, of course,
00:01:10.920 | that's more straightforward in terms of applying
00:01:13.520 | deep learning in an active learning setup.
00:01:15.040 | But I do think in terms of now looking
00:01:20.040 | into more reasoning based approaches is going to be key
00:01:24.780 | for our next wave of the technology.
00:01:27.520 | But there is a good news.
00:01:28.920 | The good news is that I think for keeping on
00:01:31.360 | to delight customers, that a lot of it can be done
00:01:33.800 | by prediction tasks.
00:01:35.960 | So, and so we haven't exhausted that.
00:01:38.760 | So we don't need to give up
00:01:42.560 | on the deep learning approaches for that.
00:01:45.400 | So that's just, I wanted to sort of point that out.
00:01:47.600 | - Creating a rich, fulfilling, amazing experience
00:01:50.680 | that makes Amazon a lot of money
00:01:52.320 | and a lot of everybody a lot of money,
00:01:54.480 | because it does awesome things.
00:01:56.320 | Deep learning is enough.
00:01:57.980 | The point--
00:01:59.180 | - I don't think, no, I wouldn't say deep learning is enough.
00:02:02.260 | I think for the purposes of Alexa
00:02:04.780 | and accomplish the task for customers,
00:02:06.500 | I'm saying there are still a lot of things we can do
00:02:10.280 | with prediction based approaches that do not reason.
00:02:13.220 | Right, I'm not saying that, and we haven't exhausted those,
00:02:16.700 | but for the kind of high utility experiences
00:02:20.540 | that I'm personally passionate about
00:02:22.340 | of what Alexa needs to do, reasoning has to be solved
00:02:26.460 | to the same extent as you can think
00:02:29.080 | of natural language understanding
00:02:31.640 | and speech recognition to the extent
00:02:33.560 | of understanding intents has been,
00:02:37.080 | how accurate it has become.
00:02:38.200 | But reasoning, we have very, very early days.
00:02:40.860 | - Let me ask that another way.
00:02:42.120 | How hard of a problem do you think that is?
00:02:44.840 | - Hardest of them.
00:02:45.880 | (laughing)
00:02:47.240 | I would say hardest of them, because again,
00:02:50.740 | the hypothesis space is really, really large.
00:02:55.740 | And when you go back in time, like you were saying,
00:02:58.460 | I want Alexa to remember more things,
00:03:01.300 | that once you go beyond a session of interaction,
00:03:04.580 | which is by session, I mean a time span, which is today,
00:03:08.860 | to versus remembering which restaurant I like.
00:03:11.420 | And then when I'm planning a night out to say,
00:03:13.740 | do you wanna go to the same restaurant?
00:03:15.780 | Now you're up the stakes big time.
00:03:17.980 | And this is where the reasoning dimension
00:03:21.080 | also goes way, way bigger.
00:03:22.980 | - So you think the space,
00:03:25.020 | we'll be elaborating that a little bit,
00:03:27.300 | just philosophically speaking, do you think,
00:03:30.260 | when you reason about trying to model
00:03:32.760 | what the goal of a person is in the context
00:03:36.340 | of interacting with Alexa, you think that space is huge?
00:03:39.380 | - It's huge, absolutely huge.
00:03:41.140 | - Do you think, so like another sort of devil's advocate
00:03:44.140 | would be that we human beings are really simple
00:03:46.820 | and we all want like just a small set of things.
00:03:49.580 | So do you think it's possible,
00:03:52.820 | 'cause we're not talking about
00:03:55.100 | a fulfilling general conversation,
00:03:57.340 | perhaps actually the Alexa prize
00:03:59.020 | is a little bit more after that.
00:04:01.420 | Creating a customer, like there's so many
00:04:04.180 | of the interactions, it feels like are clustered
00:04:09.140 | in groups that don't require general reasoning.
00:04:14.620 | - I think, yeah, you're right in terms of the head
00:04:17.420 | of the distribution of all the possible things
00:04:19.900 | customers may wanna accomplish.
00:04:21.820 | But the tail is long and it's diverse, right?
00:04:26.300 | So from that-- - There's many long tails.
00:04:29.380 | - So from that perspective,
00:04:31.600 | I think you have to solve that problem.
00:04:33.980 | Otherwise, and everyone's very different.
00:04:36.900 | Like, I mean, we see this already
00:04:38.500 | in terms of the skills, right?
00:04:40.420 | I mean, if you're an average surfer,
00:04:42.980 | which I am not, right?
00:04:45.060 | But somebody is asking Alexa about surfing conditions, right?
00:04:49.780 | And there's a skill that is there for them to get to, right?
00:04:53.620 | That tells you that the tail is massive,
00:04:55.980 | like in terms of like what kind of skills
00:04:58.860 | people have created, it's humongous in terms of it.
00:05:02.340 | And which means there are these diverse needs.
00:05:05.100 | And when you start looking at the combinations of these,
00:05:08.740 | right, even if you had pairs of skills
00:05:12.060 | and 90,000 choose two, it's still a big combination.
00:05:16.060 | So I'm saying there's a huge to do here now.
00:05:19.860 | And I think customers are wonderfully frustrated with things
00:05:24.860 | and they have to keep getting to do better things for them.
00:05:29.020 | - And they're not known to be super patient.
00:05:32.060 | So you have to-- - Do it fast.
00:05:33.660 | - You have to do it fast.
00:05:34.900 | (upbeat music)
00:05:37.500 | (upbeat music)
00:05:40.100 | (upbeat music)
00:05:42.700 | (upbeat music)
00:05:45.300 | (upbeat music)
00:05:47.900 | (upbeat music)
00:05:50.500 | [BLANK_AUDIO]