# Bayes Rule I wrote an article about this [here](https://www.nathanieldake.com/Machine_Learning/08-Bayesian_Machine_Learning-01-Bayesian-Inference.html#2.1-Bayes-Theorem). But the main idea is most easily illustrated with the cancer example. Say we take a test that is meant to detect cancer. We are told that the test has an 80% probability of detecting cancer if you have it. You then receive a positive test result. What is the probability of having cancer? The answer is *not* 80%. That is simply the probability of detecting cancer, **conditioned** on you having it. But of course there is also the chance that you don't have it! All this is measuring is: $P(\text{positive test} \mid \text{cancer})$ But, that isn't what we want to know! We want to know: $P(\text{cancer} \mid \text{positive test})$ For that we must use **Bayes Rule**: $P(A \mid B) = \frac{P(B \mid A) P(A)} {P(B)}$ And in this case that is: $P(\text{cancer} \mid \text{positive test}) = \frac{P(\text{positive test} \mid \text{cancer}) P(\text{cancer})} {P(\text{positive test})}$ So clearly there are two pieces additional pieces of information that we must take into account here! We must know the unconditional probability of someone having cancer, $P(\text{cancer})$, as well as the unconditional probability of getting a positive test, $P(\text{positive test})$. Let's say the unconditional probability of cancer is 1% (i.e. 1% of the total population has cancer). Now, how do we determine the probability of having a positive test? Well, what are the ways in which a positive test can occur? Well, you can get a positive test and have cancer, or not have cancer, hence: $P(\text{positive test}) = P(\text{positive test} \cap \text{cancer}) + P(\text{positive test} \cap \text{no cancer})$ $P(\text{positive test}) = P(\text{positive test} \mid \text{cancer}) P(\text{cancer}) + P(\text{positive test} \mid \text{no cancer}) P(\text{no cancer})$ This will clearly give a different result depending on the above probabilities. > The most important thing to keep in mind is remembering how we need to be aware of if we have *conditioned* on something and forgot to take that into account! ### 3b1b the heart of bayes theorem  The image above is incredibly useful. We can think of Bayes rules as follows: 1. We start with all possibilities: the number of farmers and the number of librarians. 2. We then update to only include those that fit the evidence. 3. We then divide the particular case we are looking for (i.e. Librarian = 1) by all possibilities that fit the evidence.  em,%20the%20geometry%20of%20changing%20beliefs%206-6%20screenshot.png)   ### 3b1b medical testing paradox [A great intuition that Grant shares is](https://youtu.be/lG4VkPoG3ko?t=441): > You should not think of tests as determining if you have a disease. You should not think of them as determining *your chances of having* a disease. Tests **update** *your chances of having* a disease. The *accuracy* of a test (see *specificity* and *sensitivity* in [Accuracy Precision Recall F1](Accuracy%20Precision%20Recall%20F1.md)) can be thought of as *the strength of the update* to our prior state of knowledge:  One of the things that makes test statistics confusing is that their are at least 4 numbers associated with them:  Thankfully, if we want to pull out a single number to focus on, we can pull out the sensitivity and the fall positive rate: In other words, how much more likely are you to see the positive test result given you have cancer vs. if you don't?  A simple rule of thumb is that if you have a small prior, in order to update it you simply multiply it by the **Bayes Factor**:  A way to make this equation exact is to talk about **odds**, where we take the ratio of the number of positive's vs the number of negatives:   --- Date: 20220203 Links to: Tags: References: * [Nathaniel Dake Blog](https://www.nathanieldake.com/Machine_Learning/08-Bayesian_Machine_Learning-01-Bayesian-Inference.html#2.1-Bayes-Theorem) * [The medical test paradox, and redesigning Bayes' rule - YouTube](https://www.youtube.com/watch?v=lG4VkPoG3ko) * [Bayes theorem, the geometry of changing beliefs - YouTube](https://www.youtube.com/watch?v=HZGCoVF3YvM&t=3s) * [The quick proof of Bayes' theorem - YouTube](https://www.youtube.com/watch?v=U_85TaXbeIo&t=0s)