- What is correlation? What is it, so probability of something happening is something, but then there's a bunch of things happening. And sometimes they happen together, sometimes not. They're independent or not. So how do you think about correlation of things? - Correlation occurs when two things vary together over a very long time.

There's one way of measuring it. Or when you have a bunch of variables that they all vary cohesively. Then we call it, we have a correlation here. And usually when we think about correlation, we really think causally. Things cannot be correlated unless there is a reason for them to vary together.

Why should they vary together? If they don't see each other, why should they vary together? - So underlying it somewhere is causation. - Yes. Hidden in our intuition, there is a notion of causation because we cannot grasp any other logic except causation. - And how does conditional probability differ from causation?

So what is conditional probability? - Conditional probability, how things vary, when one of them stays the same. Now staying the same means that I have chosen to look only at those incidents where the guy has the same value as the previous one. It's my choice as an experimenter. So things that are not correlated before could become correlated.

Like for instance, if I have two coins which are uncorrelated, okay, and I choose only those flippings experiments in which a bell rings, and the bell rings when at least one of them is a tail, okay, then suddenly I see correlation between the two coins because I only look at the cases where the bell rang.

You see, it's my design, with my ignorance essentially, with my audacity to ignore certain incidents, I suddenly create a correlation where it doesn't exist physically. - Right, so that's, you just outlined one of the flaws of observing the world and trying to infer something from the math about the world from looking at the correlation.

- I don't look at it as a flaw, the world works like that. But the flaws comes if we try to impose causal logic on correlation, it doesn't work too well. - I mean, but that's exactly what we do. That's what, that has been the majority of science, is you-- - The majority of naive science.

Statisticians know it, statisticians know that if you condition on a third variable, then you can destroy or create correlations among two other variables. They know it, it's in the data. It's nothing surprising, that's why they all dismiss the Simpson paradox, ah, we know it. They don't know anything about it.

- Well, there's disciplines like psychology where all the variables are hard to account for. And so, oftentimes, there's a leap between correlation to causation. You're imposing-- - What do you mean, a leap? Who is trying to get causation from correlation? No one. - Not, you're not proving causation, but you're sort of discussing it, implying, sort of hypothesizing with our ability to-- - Which discipline you have in mind?

I'll tell you if they are obsolete, or if they are outdated, or they're about to get outdated. - Yes, yes. - Tell me which one you have in mind. - Oh, psychology. - Psychology, what, is it SEM, Structural Equations? - No, no, I was thinking of applied psychology studying, for example, we work with human behavior in semi-autonomous vehicles, how people behave.

And you have to conduct these studies of people driving cars. - Everything starts with a question, what is the research question? - What is the research question? The research question, do people fall asleep when the car is driving itself? - Do they fall asleep, or do they tend to fall asleep more frequently-- - More frequently.

- Than with a car not driving itself? - Not driving itself. - That's a good question, okay. - And so, you measure, you put people in the car, because it's real world, you can't conduct an experiment where you control everything. - Why can't you-- - You could. - Turn the automatic module on and off?

- Because it's on-road public, I mean, there's aspects to it that's unethical, because it's testing on public roads. So you can only use vehicle, they have to, the people, the drivers themselves have to make that choice themselves, and so they regulate that. So you just observe when they drive it autonomously and when they don't.

- But maybe they turn it off when they're very tired. - Yeah, that kind of thing, but you don't know those. - So you have now uncontrolled experiment. - Uncontrolled experiment. - We call it observational study, and we form the correlation, detected, we have to infer causal relationship, whether it was the automatic piece that caused them to fall asleep, or, so that is an issue that is about 120 years old.

I should only go 100 years old, okay? Oh, maybe it's not, actually I should say it's 2000 years old because we have this experiment by Daniel, about the Babylonian king that wanted the exile, the people from Israel that were taken in exile to Babylon to serve the king, he wanted to serve them king's food, which was meat, and Daniel, as a good Jew, couldn't eat non-kosher food, so he asked them to eat vegetarian food, but the king overseer said, "I'm sorry, "but if the king sees that your performance "falls below that of other kids, "he's going to kill me." Daniel said, "Let's make an experiment.

"Let's take four of us from Jerusalem, "give us vegetarian food, "let's take the other guys to eat the king's food, "and in about a week's time, we'll test our performance." And you know the answer, of course he did the experiment, and they were so much better than the others, and the king nominated them to super position in his case.

So it was the first experiment, yes. So there was a very simple, it's also the same research questions. We want to know if vegetarian food assists or obstructs your mental ability. And okay, so the question is very old. Even Democritus said, if I could discover one cause of things, I would rather discover one cause than be a king of Persia.

The task of discovering causes was in the mind of ancient people from many, many years ago. But the mathematics of doing that was only developed in the 1920s. So science has left us orphaned. Science has not provided us with the mathematics to capture the idea of X causes Y, and Y does not cause X.

Because all the questions of physics are symmetrical, algebraic. The equality sign goes both ways. (air whooshing) (silence) (silence) (silence) (silence) (silence) (silence)