Stanford XCS224U: NLU I Presenting Your Research, Part 4: Giving Talks I Spring 2023
00:00:00.000 | Welcome back everyone.
00:00:06.200 | This is part four in our series on presenting your work.
00:00:09.000 | This is the final screencast in the series.
00:00:11.240 | We're going to be talking about giving talks,
00:00:13.120 | especially the talk that you might give
00:00:15.000 | at an NLP workshop or conference.
00:00:17.480 | I should say at the start that I'm going to give
00:00:19.800 | a lot of advice about how to give effective talks,
00:00:22.280 | but the only advice that I feel really confident in
00:00:25.640 | is that you should cultivate your own style as a speaker,
00:00:29.120 | and you should do that by giving talks and then really
00:00:32.320 | and truly reflecting on how it went,
00:00:34.480 | looking for ways you could do better,
00:00:36.280 | and then heading off in that direction.
00:00:38.240 | The precise direction that you go is highly individual.
00:00:41.040 | This will depend on your goals and your preferences
00:00:43.680 | and the style you're cultivating.
00:00:45.760 | But I can promise that if you do this act of self-reflection,
00:00:49.240 | you will become a better speaker.
00:00:51.120 | It can be painful, but it is very productive.
00:00:53.880 | This also means that the advice that you do here about giving
00:00:57.120 | effective talks should always be filtered
00:00:59.240 | through this highly personal thing that you're trying to do.
00:01:04.320 | That said, let's dive in.
00:01:05.640 | I'm going to offer a bunch of advice.
00:01:07.920 | The first piece of advice I think
00:01:09.400 | is pretty straightforward.
00:01:10.520 | For the kind of academic talks that we're talking about,
00:01:13.600 | it's a good default to assume that the talk structure should
00:01:16.520 | mirror the paper structure, but the talk
00:01:18.920 | needs to be much simpler.
00:01:20.560 | You absolutely cannot convey all the detail
00:01:22.840 | that you packed into a paper.
00:01:24.280 | It's just the wrong mode for that.
00:01:26.240 | So what you can really hope for is
00:01:27.680 | that you find a way of talking about the ideas that
00:01:30.000 | conveys their essence, teaches the core lessons,
00:01:33.520 | and entices people to actually invest in all of the details
00:01:37.000 | that are in your paper.
00:01:39.320 | To do that, just as with the paper,
00:01:40.760 | I think you should open with context setting.
00:01:43.160 | What problem are you solving?
00:01:44.840 | Why is it important?
00:01:46.400 | And what's been tried before?
00:01:47.560 | And why does it fall short?
00:01:49.000 | And if you do this context setting right,
00:01:51.040 | the audience will be really well set up
00:01:53.080 | to understand the essence of the talk.
00:01:55.280 | That's the middle part.
00:01:56.720 | You should do concrete details like what are the data, what
00:01:59.960 | are the metrics, things like that.
00:02:01.680 | What's the task structure?
00:02:03.000 | But really what you want to do is
00:02:04.560 | make sure that you convey the essence of your idea.
00:02:07.600 | What is your approach?
00:02:08.520 | If it's a model or a metric or a task structure or something
00:02:12.320 | else, this is where you really shine a light on those ideas.
00:02:15.440 | And that should feel like the essence of the talk,
00:02:17.600 | the ideas behind your work.
00:02:21.280 | Having done that, you can kind of
00:02:23.480 | add some quantitative results to support what you did.
00:02:26.760 | You can talk about ablation studies
00:02:28.720 | and do error analyses and so forth.
00:02:30.280 | It can kind of branch out from there,
00:02:32.560 | assuming that you've done the job of really conveying
00:02:34.960 | the approach.
00:02:36.240 | And then finally, at the end, you'll
00:02:38.040 | want to quickly summarize.
00:02:40.120 | But also think ahead to the future.
00:02:42.360 | This is your opportunity to convey to people
00:02:44.760 | how they might build on the ideas, what
00:02:47.000 | the important open questions are,
00:02:48.880 | and why the work was ultimately significant.
00:02:51.400 | [TYPING]
00:02:54.840 | Jeff Pullum has what he calls golden rules for giving talks.
00:02:58.120 | He says there are five of them, but there are actually six.
00:03:00.720 | I like all these rules.
00:03:01.760 | I especially like five and six, but I
00:03:03.440 | thought I would offer all of them to you.
00:03:05.800 | The first one is don't ever begin with an apology.
00:03:08.240 | I think this is reasonable.
00:03:09.360 | I think the concern here is that if you start
00:03:11.680 | by apologizing for how bad the talk is going to be because you
00:03:14.560 | didn't prepare or whatever, people might believe you.
00:03:18.200 | Their impression of you might be that, in fact, you're
00:03:20.480 | about to give a very bad talk, and that
00:03:22.600 | could cause them to check out.
00:03:23.880 | And it could certainly shape their opinions
00:03:26.320 | of the talk they do here.
00:03:28.000 | So start strong instead of starting with an apology.
00:03:32.160 | Don't ever underestimate the audience's intelligence.
00:03:34.920 | I think the spirit of this is that we've already
00:03:37.600 | baked in that we're going to give a simple version
00:03:39.680 | of the ideas we're trying to teach a core lesson.
00:03:42.160 | And with that in mind, you want to approach the audience
00:03:44.880 | as peers, people who might surprise you with new ideas,
00:03:48.080 | but certainly people who are equipped to understand
00:03:50.720 | what you're trying to do.
00:03:51.880 | And if you're in that mode, I think everyone
00:03:53.960 | will feel good and feel connected around what
00:03:56.760 | you're trying to do.
00:03:58.880 | Respect the time limits.
00:04:00.000 | This is nice and concrete, and this is absolutely crucial.
00:04:02.400 | It is tragic when people use up all their time doing context
00:04:06.080 | setting, and they never get to describe their idea.
00:04:09.920 | The way to respect the time limits
00:04:11.880 | is to do a lot of practice.
00:04:13.560 | I think there is no shortcut here.
00:04:15.480 | You should practice.
00:04:16.400 | You should watch yourself practicing,
00:04:18.440 | and you should find a way to fit the ideas in in a way that
00:04:21.560 | does what you want to do.
00:04:23.040 | If you don't practice, you are at serious risk
00:04:25.600 | of going way past the time limits.
00:04:28.840 | Don't survey the whole damn field.
00:04:30.400 | This aligns well with the advice I gave before.
00:04:33.000 | What you want to do is provide context for your ideas,
00:04:36.080 | but it should be just the context
00:04:37.640 | we need in order to understand the essence of what
00:04:39.920 | you've done.
00:04:40.760 | And if you cast the net too widely, people will get lost,
00:04:43.640 | and you might simply run out of time.
00:04:46.920 | And this one I really like.
00:04:48.560 | Remember that you're an advocate, not the defendant.
00:04:51.160 | The spirit of this is like you're a lawyer,
00:04:53.840 | and you're defending the ideas.
00:04:55.840 | You're invested in doing a good job,
00:04:57.720 | but you yourself are not on trial in any sense.
00:05:01.360 | And so this is like the right level of investment.
00:05:03.560 | It's not so personal, but you are, of course,
00:05:06.800 | trying to do the best thing that you can by the ideas
00:05:09.720 | that you're presenting.
00:05:11.600 | And then finally, expect questions that will floor you.
00:05:14.160 | On the assumption that you're talking to peers,
00:05:16.560 | very intelligent people, you've prepared
00:05:18.920 | for the discussion section.
00:05:20.080 | We'll talk about that a bit later.
00:05:21.520 | But even having done that, surprising things can happen.
00:05:24.640 | That's part of the reason that we're giving talks
00:05:26.720 | for the exploratory aspect, to learn new things.
00:05:29.720 | So you should expect questions that
00:05:32.080 | are surprising and hard to resolve,
00:05:34.560 | maybe questions that even terrify you.
00:05:36.840 | That should be considered part of the exploratory educational
00:05:41.200 | thing that you are doing.
00:05:44.000 | On top of all of that, I really like
00:05:46.000 | to return to Patrick Blackburn's fundamental insight
00:05:48.760 | about talks.
00:05:49.840 | I mentioned this before in the context of writing.
00:05:53.160 | Blackburn intended it to be about talks.
00:05:55.160 | It applies everywhere.
00:05:56.640 | He asks, where do good talks come from?
00:05:59.040 | And he says, honesty.
00:06:00.640 | A good talk should never stray far
00:06:02.520 | from simple, honest communication.
00:06:05.120 | I like to think of this in the context of education.
00:06:07.800 | The goal of the talk is not to broadcast ideas
00:06:11.120 | or to advertise, but rather to educate.
00:06:14.200 | And if you approach it from the perspective of genuinely trying
00:06:17.520 | to teach your ideas to your audience,
00:06:19.760 | I think you'll be in the right place.
00:06:21.640 | You'll connect with them.
00:06:23.000 | And they'll take those further steps
00:06:24.920 | of checking out the paper and more deeply understanding
00:06:28.160 | what you've done.
00:06:31.320 | People joke about PowerPoint.
00:06:33.320 | And they worry about the kind of distorting effects
00:06:36.160 | that it can have on the way we present our ideas.
00:06:38.800 | I think it's reasonable to be a bit worried.
00:06:41.720 | But obviously, I'm not too worried
00:06:43.160 | because this entire course is based in screencasts
00:06:46.040 | where I'm using slides, PowerPoint-style things.
00:06:49.920 | But we do need to be thoughtful about how we use this device.
00:06:53.440 | As you read about advice for talks,
00:06:56.680 | you see kind of two camps emerge, the minimalist
00:07:00.400 | and the comparative.
00:07:01.880 | The minimalist view is that your slide
00:07:03.800 | should be as spare as possible.
00:07:06.280 | The audience should spend most of the time
00:07:08.480 | listening to you and looking at you.
00:07:11.200 | Individual slides don't stay up for long.
00:07:13.640 | And they don't get used in more than one way.
00:07:15.960 | They're kind of just punctuation to the talk track.
00:07:19.320 | You could compare that with the comparative view.
00:07:21.960 | Your slide should be as full as possible
00:07:23.920 | without sacrificing clarity.
00:07:26.000 | Your talk should make it easy for people to spend time
00:07:28.360 | studying your slides.
00:07:30.080 | Individual slides stay up for a long time
00:07:32.440 | and get used to make multiple comparisons
00:07:35.160 | and establish numerous connections.
00:07:38.080 | I'll talk a little bit in a second
00:07:40.040 | about how I think these approaches compare to each other,
00:07:43.200 | how this relates to style and to your goals.
00:07:45.800 | The one thing I will say here now
00:07:48.920 | is that whatever your camp is, this slide
00:07:52.040 | is making a colossal mistake.
00:07:54.480 | The mistake it makes is that I put all the information up
00:07:57.520 | all at once.
00:07:58.280 | And I know that that means that for the most of the time
00:08:01.120 | that I was talking, a lot of you were reading around
00:08:04.000 | on the slide.
00:08:04.680 | And we got disconnected.
00:08:06.600 | What I should have done is show you
00:08:08.360 | the information in the order I presented it
00:08:10.920 | so that you were with me as I was giving my talk track.
00:08:14.400 | Whether the slide is simple or complex,
00:08:17.200 | I feel like that has to be good advice,
00:08:19.240 | because this is a lot about joint attention.
00:08:24.240 | But whether you're a minimalist or a comparativist
00:08:26.800 | is really going to be a personal matter.
00:08:29.320 | The minimalist view seems right to me for telling a story.
00:08:32.520 | It's often the best mode when time is of the essence
00:08:34.880 | and the audience is mainly there to learn at a high level
00:08:37.880 | about what your paper contains.
00:08:40.080 | Whereas the comparative view seems right to me for teaching.
00:08:43.760 | It's the closest slides come to a full, well-organized chalk
00:08:47.360 | board.
00:08:47.880 | Good advice for a chalk board is that you never
00:08:50.040 | erase things until you absolutely have to,
00:08:52.800 | because you might need them later.
00:08:54.600 | Same thing with a slide.
00:08:55.680 | If you can get a lot of information up there
00:08:57.800 | when it's still perspicuous, then you'll
00:08:59.560 | have lots of chances to do comparisons, answer questions,
00:09:02.960 | and fundamentally to educate.
00:09:06.080 | Find the style that works for you, though.
00:09:08.080 | As long as you think long and hard about what
00:09:10.040 | it will be like to listen to your talk
00:09:11.680 | and you make adjustments, you'll shine.
00:09:13.600 | You will become a better speaker in whatever mode
00:09:16.480 | you feel is a good way for you.
00:09:20.160 | Guiding audience attention is crucial, though.
00:09:22.440 | That's the joke that I made before about my over-full slide
00:09:25.440 | with no overlays.
00:09:26.960 | Use overlays to fill a slide while still keeping
00:09:30.000 | the audience with you.
00:09:31.000 | This is how you can be a comparativist while not losing
00:09:34.200 | the attention of the audience.
00:09:36.480 | Color used systematically to create distinctions
00:09:39.160 | is wonderful, as long as you're sure that the colors really
00:09:42.040 | do create contrast that will survive color blindness
00:09:45.720 | from your audience members and also really poor projectors,
00:09:48.920 | which can be quite a problem.
00:09:50.840 | Size is great to draw attention to things.
00:09:54.160 | Boxes, arrows, and other devices help people navigate plots,
00:09:58.160 | especially model diagrams, long prose statements,
00:10:01.320 | and so forth.
00:10:02.360 | It's incredibly helpful to put boxes
00:10:04.960 | around the part of a results table
00:10:07.160 | or the part of a model diagram that you are talking about
00:10:09.800 | in the order that you talk about it so that people know exactly
00:10:13.160 | where to look as you make your crucial points.
00:10:15.960 | These devices are absolutely wonderful.
00:10:18.160 | In the minimalist mode, I guess you would say,
00:10:20.520 | overlays, color, size, boxes, arrows.
00:10:24.760 | I honestly find this really hard to track
00:10:27.800 | a lot of the time.
00:10:28.840 | I understand that it's kind of dynamic feeling,
00:10:31.080 | but you can tell in the way I'm talking
00:10:33.000 | that I am more of a comparativist
00:10:34.520 | than a minimalist when it comes to slides.
00:10:38.440 | Let's wrap up with some more mundane things.
00:10:40.840 | Turn off notifications that might appear on your screen.
00:10:45.000 | If you get up in front of thousands of people
00:10:47.880 | at an ACL conference to give an important talk,
00:10:50.880 | it's your moment to shine.
00:10:52.080 | You do not want personal messages
00:10:54.120 | flashing by as notifications in the upper corner
00:10:57.200 | of your screen.
00:10:57.880 | Shut those down.
00:10:59.600 | Make sure your computer is out of power save mode
00:11:02.080 | so that the screen doesn't shut down while you're talking.
00:11:05.000 | The most tragic thing here is that you step away
00:11:07.520 | from the computer to make a point.
00:11:10.040 | You're doing that long enough that the screen shuts down,
00:11:12.800 | and then the projector won't come back,
00:11:15.080 | and you lose many minutes of your allotted time
00:11:18.080 | just trying to get the projector to cooperate.
00:11:20.640 | So sad.
00:11:21.640 | Just make sure your screen doesn't do that,
00:11:23.440 | and you'll be better off.
00:11:25.200 | Shut down running applications that might get in your way.
00:11:27.720 | They could eat up memory and cause problems,
00:11:29.760 | or they could just start to ask that you update them
00:11:32.920 | or some other needy thing right in the middle of the talk,
00:11:35.520 | and it could distract everyone.
00:11:36.920 | And you might not notice the little bouncing icon
00:11:39.360 | that everyone is now studying instead of listening to you.
00:11:43.960 | Make sure your desktop is clear of files and notes
00:11:46.320 | that you wouldn't want the world to see.
00:11:48.040 | Again, imagine you're up in front of thousands of people.
00:11:51.120 | They are definitely going to study
00:11:52.720 | whatever they can see of your desktop as they sit idly
00:11:55.600 | waiting for your talk to start
00:11:57.040 | or whatever else is happening.
00:11:58.720 | Make sure that's information
00:12:00.000 | that you want everyone to be looking at.
00:12:03.360 | And if you're using PowerPoint or Keynote or Google Slides
00:12:06.200 | or some specialized software
00:12:07.560 | that might depend on having net access,
00:12:09.760 | make a PDF backup so that it's there just in case.
00:12:13.400 | And really the ultimate in survival here,
00:12:16.400 | projectors can fail,
00:12:17.600 | so you should be prepared to give the talk
00:12:20.760 | without any slides.
00:12:22.120 | The audience will be on your side if you have to do this.
00:12:24.920 | It's a real opportunity to shine.
00:12:27.040 | Be ready to do it as a talk track alone.
00:12:31.560 | And then finally, the discussion period.
00:12:33.880 | This is often the most anxiety-producing part
00:12:36.200 | of giving any talk.
00:12:37.640 | It's very important.
00:12:39.520 | It should be the chance for the audience
00:12:41.600 | to gain a deeper understanding of your ideas.
00:12:44.280 | And when that is the spirit of the audience,
00:12:47.240 | this is wonderful.
00:12:48.640 | You really feel like you're engaged
00:12:50.280 | in a new kind of exploration and education.
00:12:53.720 | Sometimes, though, other things do happen.
00:12:56.760 | Hostile questioners, confused questioners,
00:12:59.880 | people who won't give up the floor,
00:13:01.680 | all of that other stuff that can be so anxiety-producing.
00:13:06.120 | You have to live with that.
00:13:07.840 | When people ask questions,
00:13:09.200 | my standard advice is to pause for a second
00:13:11.480 | before answering each one, if you can possibly help it.
00:13:15.040 | That will make sure that they're done talking,
00:13:17.040 | and it will also make you appear deliberative.
00:13:19.160 | And I think that's always good
00:13:20.360 | in terms of honoring the contributions people make.
00:13:23.360 | I grant that you've thought about the ideas forever.
00:13:25.880 | You might know what answer you want to give
00:13:27.680 | from word one from this person,
00:13:29.640 | but at least keep up the pretense
00:13:31.520 | that you are actively listening.
00:13:33.040 | You might be surprised,
00:13:34.680 | and also you want to honor the contributions of these people.
00:13:37.920 | Avoid saying, "I have no idea," and leaving it at that.
00:13:42.040 | You can probably get away with doing that once
00:13:44.520 | if it's a really blazing insight.
00:13:46.800 | But ideally, you say something like,
00:13:48.520 | "I have no idea, but let's think in this direction,"
00:13:51.680 | or, "Let's meditate on this possible extension,"
00:13:54.160 | or whatever it is to create a discourse again,
00:13:56.680 | 'cause that's, after all, the goal of these things.
00:13:59.360 | Most questions won't make total sense to you.
00:14:03.280 | You've thought about the ideas a lot.
00:14:04.920 | Your questioner may be entirely new to them,
00:14:07.000 | and they might be struggling as part of their question
00:14:09.640 | to figure out exactly how to come to grips with the ideas.
00:14:12.640 | So you have to work with that
00:14:13.920 | and find a way for this to feel productive.
00:14:16.720 | You'll be a hit if you can warp every question,
00:14:20.080 | whether confused or not, hostile or not,
00:14:22.440 | whatever it is, into one that makes sense
00:14:25.040 | and leaves everyone with the impression
00:14:26.640 | that the questioner raised an important issue.
00:14:29.400 | Then everyone will feel validated.
00:14:31.360 | You'll feel like you've connected with people,
00:14:33.440 | and they'll feel like they connected with you.
00:14:36.240 | Notice that I have called this the discussion period.
00:14:38.720 | It's often called the Q&A period.
00:14:41.280 | I think that's a misnomer.
00:14:42.880 | It's really not about question and answer.
00:14:44.960 | It's about discussion.
00:14:46.080 | And if you ease into that mode,
00:14:47.520 | I think you'll overall feel more satisfied
00:14:50.000 | by these experiences,
00:14:51.880 | recognizing, though, that sometimes
00:14:54.000 | they kind of do go off the rails.
00:14:55.840 | (upbeat music)
00:14:58.440 | you