Página de pruebas 4

Applying Bayes' Theorem - Example 1

https://stats.stackexchange.com/questions/66079/naive-bayes-classifier-gives-a-probability-greater-than-1

Let's say that we are training a Span classifier.

We need information about the frequency of words in spam or ham (non-spam) emails. Let's assume these numbers:

In this first example, we will assume that we know the frequency of only one word (Viagra) in spam or non-spam emails. Let's assume these numbers:

	Viagra		Money		Groceries		Unsubscribe
	Yes	No	Yes	No	Yes	No	Yes	No	Total
Spam	4/20	16/20	10/20	10/20	0/20	20/20	12/20	8/20	20
Ham	1/80	79/80	14/80	66/80	8/80	72/80	23/80	57/80	80
Total	5/100	95/100	24/100	76/100	8/100	92/100	35/100	65/100	100

${\mathbf {P(Viagra|Spam)}}$		(Likelihood)	:	The probability that a spam message contains the term $Viagra$	$\rightarrow$	$4/20=0.20=20\%$
${\mathbf {P(Viagra)}}$		(Marginal likelihood)	:	The probability that the word $Viagra$ appeared in any email (spam or ham)	$\rightarrow$	$5/100=0.05=5\%$
${\mathbf {P(Spam)}}$		(Prior probability)	:	The probability that an email is Spam	$\rightarrow$	$20/100=0.20=20\%$
${\mathbf {P(Spam|Viagra)}}$		(Posterior probability)	:	The probability that an email is Spam given that contain the word $Viagra$	$\rightarrow$	${\frac {P(Viagra|spam)P(spam)}{P(Viagra)}}={\frac {0.2\times 0.2}{0.05}}=80\%$
The probability that a message is spam, given that it contains the word "Viagra" is ${\mathbf {80\%}}$. Therefore, any message containing this term should be filtered.

Applying Bayes' Theorem - Example 2

• Let's extend our spam filter by adding a few additional terms to be monitored: "money", "groceries", and "unsubscribe".
• We will assume that the Naïve Bayes learner was trained by constructing a likelihood table for the appearance of these four words in 100 emails, as shown in the following table:

As new messages are received, the posterior probability must be calculated to determine whether the messages are more likely to be spam or ham, given the likelihood of the words found in the message text.

We can define the problem as shown in the equation below, which captures the probability that a message is spam, given that the words 'Viagra' and Unsubscribe are present and that the words 'Money' and 'Groceries' are not.

$P(Spam|Viagre\cap Money\cap Groceries\cap Unsubscribe)={\frac {P(Viagra\cap Money\cap Groceries\cap Unsubscribe|spam)P(spam)}{P(Viagra\cap Money\cap Groceries\cap Unsubscribe)}}$

For a number of reasons, this is computationally difficult to solve. As additional features are added, tremendous amounts of memory are needed to store probabilities for all of the possible intersecting events. Therefore, Class-Conditional independence can be assumed to simplify the problem.

Class-Conditional independence

The work becomes much easier if we can exploit the fact that Naïve Bayes assumes independence among events. Specifically, Naïve Bayes assumes class-conditional independence, which means that events are independent so long as they are conditioned on the same class value.

Assuming conditional independence allows us to simplify the equation using the probability rule for independent events $P(A\cap B)=P(A)\times P(B)$. This results in a much easier-to-compute formulation:

$P(Spam|Viagre\cap Money\cap Groceries\cap Unsubscribe)={\frac {P(Viagra|Spam)\cdot P(Money|Spam)\cdot P(Groceries|Spam)\cdot P(Unsubscribe|Spam)\cdot P(spam)}{P(Viagra)\cdot P(Money)\cdot P(Groceries)\cdot P(Unsubscribe)}}$

For example, suppose that a message contains the terms Viagra and Unsubscribe, but does not contain either Money or Groceries:

$P(Spam|Viagre\cap Unsubscribe)={\frac {P(Viagra|Spam)\cdot P(Unsubscribe|Spam)\cdot P(spam)}{P(Viagra)\cdot P(Unsubscribe)}}={\frac {{\frac {4}{20}}\cdot {\frac {12}{20}}\cdot {\frac {20}{100}}}{{\frac {5}{100}}\cdot {\frac {35}{100}}}}=$

The presentation shows this example this way. I think there are mistakes in this presentation:

Using the values in the likelihood table, we can start filling numbers in these equations. Because the denominatero si the same in both cases, it can be ignored for now. The overall likelihood of spam is then:

${\frac {4}{20}}\cdot {\frac {10}{20}}\cdot {\frac {20}{20}}\cdot {\frac {12}{20}}\cdot {\frac {20}{100}}=0.012$

While the likelihood of ham given the occurrence of these words is:

${\frac {1}{80}}\cdot {\frac {60}{80}}\cdot {\frac {72}{80}}\cdot {\frac {23}{80}}\cdot {\frac {80}{100}}=0.002$

Because 0.012/0.002 = 6, we can say that this message is six times more likely to be spam than ham. However, to convert these numbers to probabilities, we need one last step.

The probability of spam is equal to the likelihood that the message is spam divided by the likelihood that the message is either spam or ham:

${\frac {0.012}{(0.012+0.002)}}=0.857$

The probability that the message is spam is 0.857. As this is over the threshold of 0.5, the message is classified as spam.