Widening the evidence base for masks

00:00:00.000 | Hi there, my name is Jeremy Howard. I'm a data scientist at the University of San Francisco.

00:00:05.200 | I specialize mainly in medical data and recently I had the opportunity to present to a group of,

00:00:12.880 | I think entirely or nearly entirely epidemiologists at the World Health Organization who were looking

00:00:19.120 | at guidelines for mask wearing and I've had a lot of interest expressed in seeing the slides and the

00:00:25.760 | arguments that I put up to the World Health Organization in that so I thought I would share

00:00:30.080 | them with you here today. So I am part of an organization called Masks for All which is a

00:00:36.720 | group of volunteers. We're all scientists from many different areas of science, many different

00:00:42.880 | parts of the world and we've all spent a lot of time studying the science around community mask

00:00:49.760 | wearing and trying to communicate that to the public and to policymakers such as the World Health

00:00:54.400 | Organization. The World Health Organization has recently released some guidelines that have

00:01:03.520 | recommended using cloth masks in public or non-medical masks in public and we would actually

00:01:13.280 | like to see them go further with their recommendations and make them stronger and change them in some

00:01:18.240 | ways but perhaps more importantly one of the things I really focused on was a need to change

00:01:24.000 | the kind of evidence that epidemiologists tend to look at for public health interventions like

00:01:30.400 | mask wearing, governments and obviously in this case the World Health Organization. So the WHO

00:01:37.600 | actually sponsored a study which was what's called a meta-analysis where it's basically

00:01:46.800 | a study of other studies so this is the one here true at L which was published in the Lancet

00:01:52.400 | and what we realized is that actually this review which the WHO used quite heavily has

00:02:01.200 | a bunch of problems. The first most obvious ones is that outside of hospitals they only actually

00:02:06.480 | looked at three studies so that's what they call non-healthcare settings and in fact two of those

00:02:12.160 | studies had major problems. The first is that this one here Lao et al actually is not a study of

00:02:19.440 | community or household use of masks at all it was actually a study of the use of masks in hospitals

00:02:24.640 | during hospital visits. And then the second study with a big problem is Tuon et al which actually

00:02:34.320 | showed that in this case out of the nine groups that wore masks none of them got sick but it was

00:02:40.800 | so underpowered there were so few people in this study that the review used a number here that

00:02:47.600 | basically claimed that masks make you more likely to be infected not less likely even though

00:02:54.080 | actually nobody got sick who wore a mask in that study. Basically because there were so few infected

00:02:59.920 | people in the study overall it was kind of a statistical aberration as it were. So the only

00:03:05.040 | one which really studied community mask use was this one Wu et al which actually the number they

00:03:11.520 | used includes families where some people weren't wearing masks. So if you just actually include

00:03:17.520 | the ones where they were there was a only there was basically a 70% reduction in risk in the

00:03:24.640 | households that used masks. The study also looked at mask use in hospitals and this is the table

00:03:33.680 | that they included but actually only three of the studies in this case were of COVID-19 and in fact

00:03:40.880 | I should mention none of the studies in non-healthcare were of COVID-19 they were all of SARS 1.

00:03:46.720 | And the three healthcare studies of COVID-19 actually showed over 95% decrease in risk amongst

00:03:56.400 | those using masks in healthcare but they basically did the statistical analysis in a very strange way

00:04:03.520 | maybe not strange very common way but not really appropriate for this which is that they assumed

00:04:09.840 | that SARS, MERS and COVID-19 all had the same profile in terms of what do masks do and as a

00:04:17.680 | result they gave these three studies really low weights 0.9%, 0.9% and 1.7%. So even though the

00:04:26.800 | risk reduction from using masks for COVID-19 in all three cases was was huge those studies were

00:04:33.520 | basically ignored in this meta-analysis. There is one analysis actually of wearing masks in

00:04:43.040 | households for COVID-19 which is this one here and in this case there was a about an 80% reduction

00:04:51.440 | in risk in the households that wore masks so that's that's good you know it's a it's a pretty

00:04:58.160 | strong study it's not huge you know it's it we would love to see a bigger version of it but it's

00:05:04.240 | a it's a start. The problem is though that for all of these kinds of studies they're very hard

00:05:11.600 | to interpret because you know when you're trying to say what's the impact of wearing a mask on

00:05:17.920 | transmission if I wear a mask my hope is that I'm you know if I'm sick and I don't know it that I

00:05:23.920 | won't pass on that sickness to other people. So how do you know if me wearing a mask stopped

00:05:29.840 | other people from getting sick? You don't really right because people who come across me wearing

00:05:35.120 | my mask will have come across lots of other people as well so if they do get sick was it because a

00:05:40.240 | sickness got through my mask or was it because they came in contact with somebody else who was sick?

00:05:45.200 | Who knows right so the only way to study that properly really would be to actually have like

00:05:50.960 | 50 cities where everybody in those cities were told not to wear masks and 50 cities where everybody

00:05:56.320 | was told they have to wear masks and then get compliance to work and then compare them and that

00:06:01.920 | would be basically logistically and ethically impossible. So in fact the epidemiology and

00:06:09.040 | evidence-based healthcare communities have started to realize it well I shouldn't say started to

00:06:13.120 | actually for decades have realized this but have particularly been talking about this as being an

00:06:16.480 | issue for COVID-19 which is that population health interventions like tell everybody to wear a mask

00:06:22.720 | end up being implemented as natural experiments and their evaluation is more complex than a

00:06:28.560 | randomized controlled trial that's just the way it is you know we can't hope to find an RCT a

00:06:34.880 | randomized controlled trial for a public health intervention like this because it's just not

00:06:38.720 | ethical or logistically possible to do. So that was in BMG Journal of Epidemiology and Community

00:06:45.520 | Health in Pillow West Medicine Trish Greenhouse said something similar upstream preventative public

00:06:52.960 | health interventions aimed at supporting widespread and sustained behavior change across the whole

00:06:57.600 | population like masks rarely lend themselves to a controlled on versus off intervention design

00:07:05.280 | randomized controlled trial style design. So the reason I pointed this out to the World Health

00:07:10.160 | Organization was to kind of say look these are the kinds of studies that most epidemiologists

00:07:17.440 | are used to studying that you all want to study but they're just not going to happen

00:07:21.600 | for this right and so you actually have to look at a wider field of evidence.

00:07:26.400 | So this is what we did we being this group of 19 scientists including epidemiologists and

00:07:33.040 | biostatisticians and aerosol scientists and sociologists lots of different folks we all got

00:07:39.440 | together to study the evidence across a much wider evidence base than just traditional

00:07:46.080 | epidemiological evidence and so we ended up with I think 155 references might even be more now

00:07:52.960 | and that became the most viewed paper ever of any kind on preprints.org.

00:07:56.880 | And you know one of the things we looked at in this paper is that we now have a lot of these

00:08:05.440 | natural experiments which is that over 80% of the world now lives in regions that require

00:08:11.680 | masks. Furthermore in countries like for example in the US over 70% of the population has to wear

00:08:18.640 | masks in the US because of state or city specific mandates but back in early to mid-March only 10

00:08:27.200 | countries required masks. So this change means that we can actually look at kind of the before

00:08:32.720 | versus after effects of what happens when countries bring in or when in the case of

00:08:38.320 | the US for example states bring in mask requirements. So a paper from Loeffler et al looked at nearly

00:08:46.880 | 200 countries and you have to be very careful right because often when people bring in a

00:08:52.320 | mask mandate a country brings in a mask mandate they'll do a bunch of other things at around the

00:08:56.240 | same time they might have a lockdown at around that time they might have school closures at around

00:09:00.880 | that time. So statisticians have to use multivariate analysis to try to account for these

00:09:07.680 | confounders which Loeffler's paper for example did they looked at 10 different policy health

00:09:13.120 | interventions all the ones tracked by Oxford across as well as lots of demographic geographic

00:09:17.680 | and so forth confounders as well. And they found that countries that did not wear masks

00:09:24.720 | when you look at kind of the number of days since the start of the infection or the outbreak in that

00:09:28.720 | country after a couple of months that we had huge outbreaks normalized per million population

00:09:36.800 | countries where masks were required within 15 days of the initial infection had basically no

00:09:46.400 | infection at all and those that required them in the first 16 to 30 days had a bit of an outbreak

00:09:52.960 | but they quickly got it under control. So this is a super interesting data obviously and we see

00:10:00.080 | something very similar in a paper by Lewin Waby which is this one here which is that mask mandates

00:10:06.160 | in the US very similar result are associated with a 2 percentage point decrease in COVID-19 growth

00:10:15.520 | rate 2 percentage points a day that's enormous after 21 days after signing. So they guessed

00:10:23.840 | that by late May somewhere around 200 to 400,000 cases had been averted. So these are often

00:10:32.480 | called ecological studies and they're super unpopular amongst many in the epidemiological

00:10:42.320 | community there's this idea of this thing called the ecological fallacy which many people misinterpret

00:10:49.040 | as meaning you can't generate any understanding of population behavior or the underlying individual

00:10:54.880 | behavior of those populations by looking at population data. But actually data scientists

00:10:59.520 | and statisticians and machine learning folks have actually been studying exactly that for many,

00:11:04.560 | many decades and in fact there's another paper called the fallacy of the ecological fallacy

00:11:09.760 | which is to say basically the ecological fallacy is this kind of grab bag of multiple different

00:11:14.720 | issues which for which there are ways to deal with them. So we should be very careful not to

00:11:20.000 | throw away these vitally important studies it's vitally important information because it's really

00:11:24.560 | the only kind of data direct data that we can get about the impact of these population health

00:11:30.480 | interventions. We can however get lab data and in some ways this is kind of the strongest and most

00:11:38.080 | obvious because you can actually literally see here's an example of somebody speaking

00:11:44.640 | and this researcher when they're speaking you can see that little tiny droplets come flying out of

00:11:50.080 | their mouth and thanks to the use of this laser scattering chamber we can actually see them.

00:11:54.080 | Normally you can't they're too small to see these can be around 10 microns in size in terms of what's

00:12:00.720 | visualized here and you can see here they're holding a paper towel over their face just a

00:12:06.160 | normal paper towel and when they do almost all of the droplets are blocked it looks like there's two

00:12:12.560 | got through. Now of course you can see he's holding it quite carefully here which obviously

00:12:19.120 | if you're using a paper towel or a cloth mask you want it to be as nicely tightly fitting as possible

00:12:24.320 | but the basic idea is that yeah you know as you would expect a physical barrier blocks this

00:12:32.560 | physical thing which is the droplets that transmit the virus so we can very directly see

00:12:38.240 | the way a mask does in fact stop this vector of transmission this key vector of transmission.

00:12:46.480 | So why do we focus on speech droplets? Well the reason we focus on speech droplets is

00:12:52.960 | this is a respiratory infection so how does this respiratory infection get transmitted through

00:13:00.080 | stuff coming out of your mouth and nose. One way for that to happen is sneezes

00:13:04.320 | and we really haven't looked at sneezes much because that is a symptom and so people who

00:13:08.880 | are symptomatic hopefully are staying home. Another is breathing but breathing is mainly

00:13:16.720 | going to bring up drop very tiny droplets from your lower respiratory tract which actually it

00:13:22.400 | turns out is something where the disease kind of appears later on again after you have symptoms.

00:13:29.360 | So what we mainly care about is the upper respiratory tract viral shedding and this

00:13:36.400 | comes out in speech droplets so this is why we're particularly interested in speech droplets because

00:13:42.880 | these are the ones that people who are walking around don't even know they're sick are rejecting

00:13:49.600 | from their mouth. So one of the key things to realize this is not just for speech droplets but

00:13:58.080 | particularly so is that on average a wet particle with one variant will be about 27 microns across

00:14:05.680 | which is pretty giant compared to the size of the virus but what happens is after it comes out of

00:14:13.040 | your mouth it rapidly evaporates down to in this case it would be down to about 5 microns in size.

00:14:21.200 | There's a whole range of droplet sizes but the key is that they will evaporate and become smaller

00:14:26.320 | and so when they come out of your mouth they're moving straight forward straight into the fabric

00:14:30.880 | they're pretty big the atmosphere inside your mask is very humid because of your breathing

00:14:37.360 | and so it hits then this large droplet hits the fabric of your mask and soaks into it so it

00:14:44.000 | doesn't go through and then once it becomes much smaller after evaporation if you're not

00:14:49.840 | wearing a mask it can float in the air and as it does so it now also doesn't have that direct

00:14:56.240 | trajectory so it's also much easier for it to squeeze through the gaps above or around your mask

00:15:02.800 | so it's much easier to do what's called source control which is blocking it on the way out and

00:15:07.120 | this is why we focus on this idea my mask protects you your mask protects me. Now some aerosol some

00:15:18.240 | tiny droplets maybe from breathing are also important in transmission we don't really know

00:15:23.680 | for sure we suspect maybe not so much for the folks that don't have symptoms yet but even if

00:15:29.360 | the breathing droplets are important as well we actually know from a study that the basic cloth

00:15:36.560 | masks are actually about the most effective at stopping both the the direct coming through the

00:15:45.040 | front and through coming out the sides breath because cloth masks can often be like larger

00:15:51.040 | cover more of the face and there also can be more absorbent so you know we shouldn't assume

00:15:57.120 | that cloth masks are less effective than other masks or most importantly we shouldn't assume

00:16:04.000 | that they're not effective at all the lab data we have suggests they are and also remember that

00:16:10.640 | most of those countries that now require masks which covers the vast majority of the world's

00:16:15.520 | population aren't using medical masks because they're not available so the results we're seeing

00:16:20.560 | in practice are from people who are largely wearing simple unfitted masks. So how long does

00:16:29.200 | that droplet cloud stay in the air for well this study showed that when you again look at this

00:16:35.840 | laser scattering chamber and you look at the speech droplets that come out they after coming

00:16:42.400 | out of the mouth at minute zero after about 10 minutes about half of them have fallen to the

00:16:48.960 | ground and after 20 minutes maybe three quarters have but you can see there's still a large droplet

00:16:54.880 | cloud even 20 minutes after somebody speaks so these droplets if they're not blocked on the way

00:17:00.800 | out they can actually hang around for quite a long time. A fluid dynamics study found something

00:17:08.960 | pretty similar about the effect of masks which is that if you don't wear a mask your droplet cloud

00:17:14.720 | can go out beyond two meters where else with a mask it really is well contained you can see with

00:17:22.480 | a mask a little bit of it goes backwards but the actual radius of the cloud is much smaller.

00:17:30.080 | So how much is this impact of the size of the rejection in size of the droplet cloud

00:17:38.080 | so a study in physics of fluids found that the amount of reduction in size of the droplet cloud

00:17:43.360 | depends a lot on what kind of mask you're wearing all kinds of mask at least made it at least

00:17:50.480 | half the radius but a stitched mask with cotton was the most effective in terms of getting it

00:17:57.360 | down to just 2.5 inches in this study. So one of the interesting things is that since a mask can

00:18:05.520 | make the droplet cloud smaller that means that social distancing becomes more helpful becomes

00:18:10.720 | more important because you're not as likely to infect somebody six feet away. Another reason

00:18:15.920 | that there's a great impact on social distancing is that a study found that people who didn't wear

00:18:22.320 | masks people tended to stay closer to them than people who did wear masks. So in other words when

00:18:30.400 | you see somebody wearing a mask there's this conscious or unconscious thing going on where

00:18:34.160 | people actually keep their distance away from you more. I guess maybe it's this like reminder of

00:18:40.480 | like oh this is a pandemic there's somebody in a mask you know I should I should keep her a healthy

00:18:45.760 | distance. So that's interesting because some folks have claimed including the WHO that wearing a mask

00:18:56.400 | could reduce to less could result in less social distancing but the data we have doesn't support

00:19:02.960 | that at all it seems populations that wear masks have dramatically less transmission it seems that

00:19:09.440 | the effect on social distancing is much better because of the reduced droplet cloud size

00:19:14.080 | and that people may even keep their distance more. In terms of children which is particularly

00:19:21.360 | important right now schools seem like a particularly important place to wear masks it's it's somewhere

00:19:27.360 | where people talk a lot and so therefore a lot of speech droplets in places like Sweden where they

00:19:34.880 | didn't close the schools kids had just as high an infection rate in in seroprevalence studies as

00:19:41.520 | adults did so it seems like when you don't close schools kids become infected often and then they

00:19:47.360 | go on to pass on those infections to adults even though the kids themselves rarely get seriously

00:19:52.480 | sick it seems that they just as often or maybe nearly just as often pass on that sickness to

00:19:59.680 | adults and so for example in Victoria the largest outbreak in the whole state came from a school

00:20:07.600 | in Israel there was massive outbreaks around the country both Victoria and Israel are both places

00:20:12.880 | where COVID-19 was almost under control and they reopened schools and things got totally out of

00:20:19.520 | control and in fact you know it when you look at the viral loads in kids they tend to be pretty

00:20:26.320 | similar to the viral loads in in adults. So one of the things that we talked about is is this idea

00:20:35.520 | of likely harms the World Health Organization currently has a list of likely harms under the

00:20:40.720 | section around community mask use. So some of the likely harms that we've heard about a lot and seen

00:20:47.440 | in the World Health Organization's guidelines don't really seem likely at all in fact some of them

00:20:52.800 | seem like likely benefits rather than likely harms and for example there's a claim that there could

00:20:58.000 | be a increased risk of self-contamination but actually it's much more likely you would see a

00:21:02.720 | decreased risk a mouth covers the mouth and nose making it much harder to touch them and so there

00:21:09.360 | has been some studies of this it does need more research but the evidence we have suggests that

00:21:14.320 | maybe this is a benefit rather than a harm. There's also a claim in the guidelines that there's a risk

00:21:20.640 | of droplet transmission and it splashes to the eyes which is true of course but that is not the

00:21:28.160 | result of mask wearing that's just like oh also guys cover up our eyes. So it's kind of strange

00:21:33.840 | to me that there are these that you know official health bodies like the WHO include these likely

00:21:41.520 | harms when they're not likely or harms necessarily or related to wearing a mask it's it's really odd

00:21:50.400 | to I don't really know what's going on here. There's a claim often including the WHO guidelines

00:21:57.600 | that there could be a likely to be a false sensor security but there's actually no data supporting

00:22:04.240 | that and in fact it was claimed over a hundred years ago and it's never been demonstrated. So

00:22:12.080 | this is just this kind of speculative well maybe it might happen. So I think we've got to be very

00:22:19.360 | careful of when you see somebody saying here are some likely harms is to say like do we have any

00:22:23.920 | evidence of that is there any data to support that is there any lab evidence of that or is this just

00:22:28.800 | an assertion which somebody's just made up and could could the opposite be equally true.

00:22:34.800 | The other issue with the WHO guidelines is that it it lists some suggested types or a particular

00:22:45.840 | type of mask based on a table of mask filtration levels and actually the the advice is based on

00:22:53.920 | studies which don't use the right kind of approach. In particular they look at particle sizes of 72

00:23:03.920 | nanometers. Now that's very very small very small and much smaller than the five microns we talked

00:23:10.240 | about and in fact it's it's very hard to even come up with a droplet smaller than one micron

00:23:17.840 | that can both hold together you know at all and actually contain a very on a virus particle.

00:23:24.400 | So the particle size that's being studied but in the WHO table is way smaller than we would expect

00:23:32.080 | to ever need to filter in practice particularly for source control. So when it's coming out

00:23:37.760 | as we discussed it's larger than when it's coming in. So you know that's a much smaller

00:23:45.600 | particle than we actually think you should be looking at. Furthermore they used a flow rate

00:23:51.840 | so the amount of particle you know the amount of gas going through the machine that they were

00:23:57.040 | testing with was was much higher than you would have in real life when you're kind of

00:24:04.240 | sitting or walking slowly or something like this. So in fact the the particle sizes and flow rates

00:24:10.320 | they used are based on the testing criteria for masks as protective equipment for health care

00:24:19.120 | workers which is like worst case situation not really that appropriate for for community use.

00:24:25.360 | Also they didn't actually test the best materials and so there there are actually tests of materials

00:24:33.760 | that that you can buy from Amazon or the local shop or whatever which are orders of magnitude

00:24:40.800 | more effective than the ones that that they talked about and even recommended in their guidelines.

00:24:45.360 | So I think it's a waste that they're not actually pointing out that there are much better materials

00:24:51.520 | available. They also didn't touch on the best designs for ensuring a good fit. Again I think

00:24:59.680 | that's a real shame for example this rubber mask brace or this approach I'm wearing here of just

00:25:05.920 | three rubber bands were shown to surpass the N95 fit test so the kind of health care medical worker

00:25:16.560 | fit test in in most situations that this study looked at. So there are a lot of crafters now

00:25:31.920 | working with you know great materials and great designs for example on Etsy if you look for face

00:25:37.840 | masks with a moldable nose wire and an insertable filter where you can put these really great

00:25:42.960 | filter materials there's over 69,000 different shops where you can buy these from kind of DIY

00:25:52.160 | crafters. So you know we I think it's good it would be great if public health bodies can

00:25:59.280 | try and provide this information about what are the best approaches and and realize that they are

00:26:04.560 | available. Okay so that's it for my little summary of what we talked about and what I presented to

00:26:14.720 | the World Health Organization on community mask wearing and I hope you found that helpful. Thanks

00:26:20.640 | for watching.

Widening the evidence base for masks

Chapters