Joint Probability Distributions

Joint Probability Distributions

The objective of statistics is to learn about population characteristics: this was first mentioned in the Introductory Section. An EXPERIMENT is then any process which generates data. It is easy to imagine circumstances where an experiment (say a few interviewers asking couples walking down the road) generates observations, for example the weekly income of husbands and the weekly income of wives in a particular population of husbands and wives. One possible use of such sample data is to investigate the relationship between the observed values of the two variables. In the Section Correlation and Regression we discussed how to use correlation and regression to summarise the extent and nature of any linear relationship between these observed values. It has to be said that the discussion of relationships between variables in this section does but scratch the surface of a very large topic. Subsequent courses in Econometrics take the analysis of relationships between two or more variables much further.

If these two pieces of information generated by the experiment are considered to be the values of two random variables defined on the SAMPLE SPACE of an experiment (see this [[Probability_Intro | Section]]), then the discussion of random variables and probability distributions needs to be extended to the multivariate case.

Let [math]X[/math] and [math]Y[/math] be the two random variables: for simplicity, they are considered to be discrete random variables. The outcome of the experiment is a pair of values [math]\left( x,y\right)[/math]. The probability of this outcome is a joint probability which can be denoted

[math]\Pr \left( X=x\cap Y=y\right) ,[/math]

emphasising the analogy with the probability of a joint event [math]\Pr \left( A\cap B\right)[/math], or, more usually, by

[math]\Pr \left( X=x,Y=y\right) .[/math]

• The collection of these probabilities, for all possible combinations of [math]x[/math] and [math]y[/math], is the joint probability distribution of [math]X[/math] and [math]Y, [/math] denoted

  [math]p\left( x,y\right) =\Pr \left( X=x,Y=y\right) .[/math]

• The Axioms of Probability discussed in this Section carry over to imply

  [math]0\leqslant p\left( x,y\right) \leqslant 1,[/math]

  [math]\sum_{x}\sum_{y}p\left( x,y\right) =1,[/math]

• where the sum is over all [math]\left( x,y\right)[/math] combinations.

Examples

Example 1

Let [math]H[/math] and [math]W[/math] be the random variables representing the population of weekly incomes of husbands and wives, respectively, in some country. There are only three possible weekly incomes, £0, £100 or £ 200. The joint probability distribution of [math]H[/math] and [math]W[/math] is represented as a table:

		[math]H[/math]
Probabilities		[math]\pounds\ 0[/math]	[math]\pounds\ 1[/math]	[math]\pounds\ 2[/math]
Values of [math]W:[/math]	£  [math]0[/math]	[math]0.05[/math]	[math]0.15[/math]	[math]0.10[/math]
	[math]\pounds\ 1[/math]	[math]0.10[/math]	[math]0.10[/math]	[math]0.30[/math]
	[math]\pounds\ 2[/math]	[math]0.05[/math]	[math]0.05[/math]	[math]0.10[/math]

Then we can read off, for example, that

[math]\Pr \left( H=0,W=0\right) =0.05,[/math]

or that in this population, [math]5\%[/math] of husbands and wives have each a zero weekly income.

In this example, the nature of the experiment underlying the population data is not explicitly stated. However, in the next example, the experiment is described, the random variables defined in relation to the experiment, and their probability distribution deduced directly.

Example 2

Consider the following simple version of a lottery. Players in the lottery choose one number between [math]1[/math] and [math]5[/math], whilst a machine selects the lottery winners by randomly selecting one of five balls (numbered [math]1[/math] to [math]5[/math]). Any player whose chosen number coincides with the number on the ball is a winner. Whilst the machine selects one ball at random (so that each ball has an [math]0.2[/math] chance of selection), players of the lottery have “lucky” and “unlucky” numbers. Therefore they choose numbers not randomly, but with the following probabilities:

Number chosen by player	Probability of being chosen
[math]1[/math]	[math]0.40[/math]
[math]2[/math]	[math]0.20[/math]
[math]3[/math]	[math]0.05[/math]
[math]4[/math]	[math]0.10[/math]
[math]5[/math]	[math]0.25[/math]
	[math]1.0[/math]

Let [math]X[/math] denote the number chosen by a player and [math]Y[/math] the number chosen by the machine. If they are assumed to be independent events, then for each possible value of [math]X[/math] and [math]Y[/math], we will have

[math]\Pr \left( X\cap Y\right) =\Pr \left( X\right) \Pr \left( Y\right) .[/math]

The table above gives the probabilities for [math]X[/math], and [math]\Pr \left( Y = y\right) =0.2[/math] for all [math]y=1,...,5[/math], so that a table can be drawn up displaying the joint distribution [math]p\left( x,y\right) :[/math]

Probabilities
	Selected by machine
Chosen by player	[math]\ 1\ [/math]	[math]\ 2\ [/math]	[math]\ 3\ [/math]	[math]\ 4\ [/math]	[math]\ 5\ [/math]	Row Total
[math]1[/math]	[math]0.08[/math]	[math]0.08[/math]	[math]0.08[/math]	[math]0.08[/math]	[math]0.08[/math]	[math]0.40[/math]
[math]2[/math]	[math]0.04[/math]	[math]0.04[/math]	[math]0.04[/math]	[math]0.04[/math]	[math]0.04[/math]	[math]0.20[/math]
[math]3[/math]	[math]0.01[/math]	[math]0.01[/math]	[math]0.01[/math]	[math]0.01[/math]	[math]0.01[/math]	[math]0.05[/math]
[math]4[/math]	[math]0.02[/math]	[math]0.02[/math]	[math]0.02[/math]	[math]0.02[/math]	[math]0.02[/math]	[math]0.10[/math]
[math]5[/math]	[math]0.05[/math]	[math]0.05[/math]	[math]0.05[/math]	[math]0.05[/math]	[math]0.05[/math]	[math]0.25[/math]
Column Total	[math]0.20[/math]	[math]0.20[/math]	[math]0.20[/math]	[math]0.20[/math]	[math]0.20[/math]	[math]1.00[/math]

The general question of independence in joint probability distributions will be discussed later in the section. <>

= Footnotes =