Probability Conditional - Revision history

Admin: /* Exercises */

2013-08-09T10:59:22Z

‎Exercises

Admin: /* Additional resources */

2013-08-09T10:58:12Z

‎Additional resources

Admin at 10:56, 9 August 2013

2013-08-09T10:56:46Z

Admin: /* Exercises */

2013-08-09T10:16:45Z

‎Exercises

Admin at 10:16, 9 August 2013

2013-08-09T10:16:13Z

Admin: /* Conditional Probability */

2013-08-09T09:58:51Z

‎Conditional Probability

Admin: Created page with " = Conditional Probability = An important consideration in the development of probability is that of ''conditional probability''. This refers to the calculation of updating ..."

2013-08-09T09:57:26Z

Created page with " = Conditional Probability = An important consideration in the development of probability is that of ''conditional probability''. This refers to the calculation of updating ..."

New page

= Conditional Probability =

An important consideration in the development of probability is that of ''conditional probability''. This refers to the calculation of updating probabilities in the light of revealed information. For example, insurance companies nearly always set their home contents insurance premiums on the basis of the postcode in which the home is located. That is to say, insurance companies believe the risk depends upon the location; i.e., the probability of property crime is assessed conditional upon the location of the property. (A similar calculation is made to set car insurance premiums.) As a result, the premiums for two identical households located in different parts of the country can differ substantially.

* In general, the probability of an event, <math>E</math>, occurring ''given'' that an event, <math>F</math>, has occurred is called the ''conditional probability'' of <math>E</math> given <math>F</math> and is denoted <math>\Pr (E|F)</math>.

As another example, it has been well documented that the ability of a new born baby to survive is closely associated with its birth-weight. A birth-weight of less than 1500''g'' is regarded as dangerously low. Consider <math>E=</math> ''birth weight of a baby is less than'' 1500''g'', <math>F=</math> ''mother smoked during pregnancy''; then evidence as to whether <math>\Pr(E|F)>\Pr (E|\bar{F})</math> is of considerable interest.

As a preliminary to the main development, consider the simple experiment of rolling a fair die and observing the number of dots on the upturned face. Then <math>S=\left\{ 1,2,3,4,5,6\right\} </math> and define events, <math>E=\left\{4\right\} </math> and <math>F=\left\{ 4,5,6\right\} ;</math> we are interested in <math>\Pr \left(E|F\right)</math>. To work this out we take <math>F</math> as known. Given this knowledge the sample space becomes restricted to simply <math>\left\{ 4,5,6\right\} </math> and, given no other information, each of these <math>3</math> outcome remains equally likely. So the required event, <math>4</math>, is just one of three equally likely outcomes. It therefore seems reasonable that <math>\Pr (E|F)=\frac{1}{3}</math>.

We shall now develop this idea more fully, using Venn Diagrams with the implied notion of area giving probability. Consider an abstract sample space, denoted by <math>S</math>, with events <math>E\subset S,\,\,F\subset S</math>. This is illustrated in the following Figure. Eventually we will want to construct the conditional probability, <math>\Pr \left( E|F\right)</math>. Sticking with the above example that could be the probability that ''a child is underweight'', given that ''the mother is a smoker''. Two important areas used in the construction of this conditional probability are highlighted as <math>\mathbf{a}</math> and <math>\mathbf{b}</math>:

[[File:ProbCond_venn1.jpg|frameless|600px]]

In general, it is useful to think of <math>\Pr (E)</math> as <math>\frac{area\left( E\right)}{area\left( S\right) }</math>; and similarly for <math>\Pr (F)</math>. The <math>\Pr (E\cap F)</math> could equally be thought of as <math>\frac{area\left( a\right)}{area\left( S\right) }</math>. With this in mind, consider what happens if we are now told that <math>F</math> has occurred. Incorporating this information implies that the effective sample space becomes restricted to <math>S^{*}=F</math>, since <math>F</math> now defines what can happen. This now covers the sample area <math>a+b.</math> On this new, restricted, sample space an outcome in <math>E</math> can only be observed if that outcome also belongs to <math>F</math>, the restricted sample space <math>S^*</math>. And this only occurs in area <math>a</math> which corresponds to the event <math>E\cap F</math>. Thus the event of interest ''now'' is <math>E^{*}=E\cap F,</math> as defined on the ''restricted ''sample space of <math>S^{*}=F</math>.

In order to proceed with the construction of the conditional probability, <math>\Pr \left( E|F\right)</math>, let <math>area(S)=z</math>. Then, since the ratio of the area of the event of interest to that of the sample space gives probability, we have (on this restricted sample space):

<math>\begin{aligned}
\Pr (E|F) &=&\frac{area\left( E\cap F\right) }{area\left( F\right) } \\
&=&\frac{a}{a+b} \\
&=&\frac{a/z}{\left( a+b\right) /z} \\
&=&\frac{\Pr \left( E\cap F\right) }{\Pr \left( F\right) },\end{aligned}</math>

We have shown, for this example how a conditional probability can be expressed as a function of the joint probability <math>\Pr \left( E\cap F\right)</math> and the marginal probability <math>\Pr \left( F\right)</math>. This is a profound result and should be formulated in more general terms:

<ul>
<li>The probability that <math>E</math> occurs, given that <math>F</math> is known to have occurred, gives the '''conditional probability''' of <math>E</math> given <math>F</math>. This is denoted <math>Pr(E|F)</math> and is calculated as
<math>\Pr (E|F)=\frac{\Pr (E\cap F)}{\Pr (F)}</math>
and from the axioms of probability will generate a number lying between 0 and 1, since <math>\Pr (F)\geq \Pr (E\cap F)\geq 0.</math></li>
<li>''Example: ''A Manufacturer of electrical components knows that the probability is 0.8 that an order will be ready for shipment on time and it is 0.6 that it will also be delivered on time. What is the probability that such an order will be delivered on time given that it was ready for shipment on time?
Let <math>R=</math> READY, <math>D=</math> DELIVERED ON TIME. <math>Pr(R)=0.8,Pr(R\cap D)=0.6.</math> From this we need to calculate <math>Pr(D|R),</math> using the above formula. This gives, <math>Pr(D|R)=Pr(R\cap D)/Pr(R)=6/8,\,\,</math>or<math>\,\,75\%</math>.</li></ul>

If we re-arrange the above formula for conditional probability, we obtain the so-called ''multiplication rule of probability ''for ''intersections'' of events:

== Multiplication rule of probability ==

The multiplication rule of probability can be stated as follows:

* <math>\Pr (E\cap F)=\Pr (E|F)\times \Pr (F)</math>

Note that for any two events, <math>E</math> and <math>F,</math> <math>(E\cap F)</math> and <math>(E\cap \bar{F})</math> are mutually exclusive with <math>E=(E\cap F)\cup (E\cap \bar{F})</math>; this has been seen before. So the ''addition rule'' and ''multiplication rule'' of probability together give:

<math>\begin{aligned}
\Pr (E) &=&\Pr (E\cap F)+\Pr (E\cap \bar{F}) \\
&=&\Pr (E|F)\times \Pr (F)+\Pr (E|\bar{F})\times \Pr (\bar{F}).\end{aligned}</math>

This is an extremely important and useful result, in practice, as we shall see shortly.

== Statistical Independence ==

If the knowledge that <math>F</math> has occurred does NOT alter our probability assessment of <math>E</math>, then <math>E</math> and <math>F</math> are said to be (statistically) ''independent''. In this sense, <math>F</math> carries no information about <math>E</math>.

<ul>
<li>Formally, <math>E</math> and <math>F</math> are '''independent''' events if and only if
<math>Pr(E|F)=Pr(E)</math>
which, in turn is true'' if and only if ''
<math>Pr(E\cap F)=Pr(E)\times Pr(F).</math></li></ul>

This concept of independence is of enormous importance in practice. Consider the case of lung cancer and its connection to smoking (apologies to all smokers for being picked upon here). The first connection between smoking and lung cancer was made in the 1920s. However, for many decades after the tobacco industry spend a lot of money and effort to convince people that there was no connection between the two. In other words they claimed that the two events are ''independent'', or <math>Pr(Cancer|Smoking)=Pr(Cancer|\cap{Smoking})=Pr(Cancer)</math>. It was then the task of epidemiologists to show otherwise. This was famously and comprehensively achieved by the [http://en.wikipedia.org/wiki/British_Doctors_Study ''British Doctors Study''].

== Bayes’ Theorem ==

One area where conditional probability is extremely important is that of clinical trials - testing the power of a diagnostic test to detect the presence of a particular disease. Suppose, then, that a new test is being developed and let <math>P=</math> ‘''test positive''’ and <math>D=</math> ‘''presence of disease''’, but where the results from applying the diagnostic test can never be wholly reliable. From the point of view of our previous discussion on conditional probability, we would of course require that <math>\Pr \left( P|D\right)</math> to be large; i.e., the test should be effective at detecting the disease. However, if you think about, this is not necessarily the probability that we might be interested in from a diagnosis point of view. Rather, we should be more interested in <math>\Pr \left( D|P\right)</math>, the probability of correct diagnosis, and require this to be large (with, presumably, <math>\Pr (D|\bar{P})</math> being small). Here, what we are trying to attach a probability to is a possible ‘cause’. The observed outcome is a positive test result (<math>P</math>), but the presence or non-presence of the disease is what is of interest and this is uncertain. <math>\Pr (D|P)</math> asks the question ‘''what is the probability that it is the presence of the disease which caused the positive test result''’? (Another recent news-worthy example would be the effect of exposure to depleted uranium on Gulf and Balkan war veterans. Given the presence of lymph, lung or brain cancer in such individuals (<math>P</math>), how likely is that the cause was exposure to depleted uranium weapons (<math>D</math>)? Firstly, is <math>\Pr \left( D|P\right) </math> high or low? Secondly, might there being something else (<math>F</math>) which could offer a “better” explanation, such that <math>\Pr \left( F|P\right) >\Pr \left( D|F\right) </math> ?)

The situation is depicted in the following Figure, in which there are two possible ‘states’ in the population: <math>D</math> (depicted by the lighter shaded area covering the left portion of the sample space) and <math>\bar{D}.</math> It must be that <math>D\cup \bar{D}=S,</math> since any individual in the population either has the disease or does not. The event of an observed positive test result is denoted by the closed loop, <math>P</math>. (Notice that the shading in the diagram is relatively darker where <math>P</math> intersects with <math>D</math>.)

[[File:ProbCond_venn2.jpg|frameless|600px]]

To investigate how we might construct the required probability, <math>\Pr \left(D|P\right)</math>, proceed as follows:

<math>\begin{aligned}
\Pr \left( D|P\right) &=&\frac{\Pr \left( D\cap P\right) }{\Pr (P)} \\
&=&\frac{\Pr (D\cap P)}{\Pr (P\cap D)+\Pr (P\cap \bar{D})},\end{aligned}</math>

since <math>P=(P\cap D)\cup (P\cap \bar{D}),</math> and these are mutually exclusive. From the multiplication rule of probability, <math>\Pr \left( P\cap D\right) =\Pr(P|D)\times \Pr (D),</math> and similarly for <math>\Pr \left( P\cap \bar{D}\right)</math>. Thus

<math>\Pr \left( D|P\right) =\frac{\Pr \left( P|D\right) \times \Pr \left(D\right) }{\Pr \left( P|D\right) \times \Pr \left( D\right) +\Pr (P|\bar{D})\times \Pr \left( \bar{D}\right) },</math>

which may be convenient to work with since <math>\Pr \left( P|D\right) </math> and <math>\Pr\left( P|\bar{D}\right) </math> can be estimated from clinical trials and <math>\Pr\left( D\right) </math> estimated from recent historical survey data.

This sort of calculation (assigning probabilities to possible causes of observed events) is an example of ''Bayes’ Theorem''. Of course, we may have to consider more than two possible causes, and the construction of the appropriate probabilities is as follows.

<ol>
<li>Consider a sample space, <math>S</math>, where <math>E\subset S</math> and <math>A,B,C</math> are three mutually exclusive events (possible causes), defined on <math>S</math>, such that <math>S=A\cup B\cup C</math>. In such a situation, <math>A,B</math> and <math>C</math> are said to form a '''partition''' of <math>S</math>. '''Bayes’ Theorem''' states that:
<math>\Pr (A|E)=\frac{\Pr (E|A)\times \Pr (A)}{\left\{ \Pr (E|A)\times \Pr(A)\right\} +\left\{ \Pr (E|B)\times \Pr (B)\right\} +\left\{ \Pr(E|C)\times \Pr (C)\right\} }.</math></li>
<li>And, more generally, consider a sample space, <math>S</math>, where <math>E\subset S</math> and <math>F_{1},F_{2},...,F_{k}</math> are <math>k</math> mutually exclusive events (possible causes), which form a partition of <math>S:S=\bigcup_{j=1}^{k}F_{j}</math>. '''Bayes’ Theorem''' then states that:
<math>\Pr (F_{j}|E)=\frac{\Pr (E|F_{j})\times \Pr (F_{j})}{\sum_{s=1}^{k}\left\{\Pr (E|F_{s})\times \Pr (F_{s})\right\} }.</math></li></ol>

From the above formula, you should be able to satisfy yourself that <math>\sum_{j=1}^{k}\Pr \left( F_{j}|E\right) =1.</math> If this is not at first clear, consider case (1) and show that <math>\Pr \left( A|E\right) +\Pr \left(
B|E\right) +\Pr \left( C|E\right) =1.</math> The reason for this is that since <math>A</math>, <math>B</math> and <math>C</math> form a partition of <math>S,</math> they must also form a partition of any event <math>E\subset S.</math> In the above conditional probabilities, we are regarding <math>E</math> as the restricted sample space and therefore the probabilities assigned the mutually exclusive events <math>\left( A,B,C\right) </math> which ''cover'' this (restricted) sample space, <math>E</math>, must sum to 1.

<ul>
<li>''Example'': Box A contains 2 red balls. Box B contains 1 red and 1 white ball. Box A and Box B are identical. If a box is selected at random and one ball is withdrawn from it, what is the probability that the selected box was number 1 if the ball withdrawn from it turns out to be red?
Let <math>A</math> be the event of selecting Box A and <math>R</math> the event of drawing a red ball. Require <math>Pr(A|R)</math>.</li></ul>

<math>Pr(A|R)=Pr(A\cap R)/Pr(R);</math>

<math>Pr(A\cap R)=Pr(A)Pr(R|A)=(1/2)\times 1=1/2.</math>

And,

<math>\begin{aligned}
Pr(R) &=&Pr(A\cap R)+Pr(\bar{A}\cap R) \\
&=&\Pr (A)\times \Pr (R|A)\,\,\,\,+\,\,\,\,\Pr (\bar{A})\times \Pr (R|\bar{A}) \\
&=&(1/2)\,\,\,\,+\,\,\,\,(1/2)\times (1/2) \\
&=&3/4.\end{aligned}</math>

Therefore, <math>\Pr (A|R)=(1/2)/(3/4)=2/3</math>.

= Footnotes =

@@ Line 126: / Line 126: @@
 = Exercises =
-You can find examples related to these topics here: [[Probability<sub>C</sub>onditional<sub>E</sub>xercises]].
+You can find examples related to these topics here: [[Probability_Conditional_Exercises]].
 = Footnotes =

← Older revision		Revision as of 10:58, 9 August 2013
Line 67:		Line 67:

	This concept of independence is of enormous importance in practice. Consider the case of lung cancer and its connection to smoking (apologies to all smokers for being picked upon here). The first connection between smoking and lung cancer was made in the 1920s. However, for many decades after the tobacco industry spend a lot of money and effort to convince people that there was no connection between the two. In other words they claimed that the two events are ''independent'', or <math>Pr(Cancer\|Smoking)=Pr(Cancer\|\cap{Smoking})=Pr(Cancer)</math>. It was then the task of epidemiologists to show otherwise. This was famously and comprehensively achieved by the [http://en.wikipedia.org/wiki/British_Doctors_Study ''British Doctors Study''].		This concept of independence is of enormous importance in practice. Consider the case of lung cancer and its connection to smoking (apologies to all smokers for being picked upon here). The first connection between smoking and lung cancer was made in the 1920s. However, for many decades after the tobacco industry spend a lot of money and effort to convince people that there was no connection between the two. In other words they claimed that the two events are ''independent'', or <math>Pr(Cancer\|Smoking)=Pr(Cancer\|\cap{Smoking})=Pr(Cancer)</math>. It was then the task of epidemiologists to show otherwise. This was famously and comprehensively achieved by the [http://en.wikipedia.org/wiki/British_Doctors_Study ''British Doctors Study''].
−
−	~~=== Additional resources ===~~
−
−	~~Khan Academy~~
−
−	* A different example that intuitively leads to Bayes Formula [https://www.khanacademy.org/math/probability/independent-dependent-probability/dependent_probability/v/introduction-to-dependent-probability]

	== Bayes’ Theorem ==		== Bayes’ Theorem ==

@@ Line 113: / Line 113: @@
 = Exercises =
-You can find examples related to these topics here: [[Probability_Conditional_Examples]].
+You can find examples related to these topics here: [[Probability_Conditional_Exercises]].
 = Footnotes =

← Older revision		Revision as of 10:16, 9 August 2013
Line 110:		Line 110:

	Therefore, <math>\Pr (A\|R)=(1/2)/(3/4)=2/3</math>.		Therefore, <math>\Pr (A\|R)=(1/2)/(3/4)=2/3</math>.
		+
		+	= Exercises =
		+
		+	You can find examples related to these topics here: [[Probability_Conditional_Examples]].

	= Footnotes =		= Footnotes =

@@ Line 49: / Line 49: @@
 This is an extremely important and useful result, in practice, as we shall see shortly.
 == Statistical Independence ==
@@ Line 61: / Line 67: @@
 This concept of independence is of enormous importance in practice. Consider the case of lung cancer and its connection to smoking (apologies to all smokers for being picked upon here). The first connection between smoking and lung cancer was made in the 1920s. However, for many decades after the tobacco industry spend a lot of money and effort to convince people that there was no connection between the two. In other words they claimed that the two events are ''independent'', or <math>Pr(Cancer|Smoking)=Pr(Cancer|\cap{Smoking})=Pr(Cancer)</math>. It was then the task of epidemiologists to show otherwise. This was famously and comprehensively achieved by the [http://en.wikipedia.org/wiki/British_Doctors_Study ''British Doctors Study''].
 == Bayes’ Theorem ==
@@ Line 110: / Line 122: @@
 Therefore, <math>\Pr (A|R)=(1/2)/(3/4)=2/3</math>.
 = Exercises =
-You can find examples related to these topics here: [[Probability_Conditional_Exercises]].
+You can find examples related to these topics here: [[Probability<sub>C</sub>onditional<sub>E</sub>xercises]].
 = Footnotes =

@@ Line 13: / Line 13: @@
 We shall now develop this idea more fully, using Venn Diagrams with the implied notion of area giving probability. Consider an abstract sample space, denoted by <math>S</math>, with events <math>E\subset S,\,\,F\subset S</math>. This is illustrated in the following Figure. Eventually we will want to construct the conditional probability, <math>\Pr \left( E|F\right)</math>. Sticking with the above example that could be the probability that ''a child is underweight'', given that ''the mother is a smoker''. Two important areas used in the construction of this conditional probability are highlighted as <math>\mathbf{a}</math> and <math>\mathbf{b}</math>:
-[[File:ProbCond_venn1.jpg|frameless|600px]]
+[[File:ProbCond_venn1.jpg|frameless|400px]]
 In general, it is useful to think of <math>\Pr (E)</math> as <math>\frac{area\left( E\right)}{area\left( S\right) }</math>; and similarly for <math>\Pr (F)</math>. The <math>\Pr (E\cap F)</math> could equally be thought of as <math>\frac{area\left( a\right)}{area\left( S\right) }</math>. With this in mind, consider what happens if we are now told that <math>F</math> has occurred. Incorporating this information implies that the effective sample space becomes restricted to <math>S^{*}=F</math>, since <math>F</math> now defines what can happen. This now covers the sample area <math>a+b.</math> On this new, restricted, sample space an outcome in <math>E</math> can only be observed if that outcome also belongs to <math>F</math>, the restricted sample space <math>S^*</math>. And this only occurs in area <math>a</math> which corresponds to the event <math>E\cap F</math>. Thus the event of interest ''now'' is <math>E^{*}=E\cap F,</math> as defined on the ''restricted ''sample space of <math>S^{*}=F</math>.
@@ Line 68: / Line 68: @@
 The situation is depicted in the following Figure, in which there are two possible ‘states’ in the population: <math>D</math> (depicted by the lighter shaded area covering the left portion of the sample space) and <math>\bar{D}.</math> It must be that <math>D\cup \bar{D}=S,</math> since any individual in the population either has the disease or does not. The event of an observed positive test result is denoted by the closed loop, <math>P</math>. (Notice that the shading in the diagram is relatively darker where <math>P</math> intersects with <math>D</math>.)
-[[File:ProbCond_venn2.jpg|frameless|600px]]
+[[File:ProbCond_venn2.jpg|frameless|400px]]
 To investigate how we might construct the required probability, <math>\Pr \left(D|P\right)</math>, proceed as follows: