distribution; continuous variables)
9. Bivariate joint normal distributions
that is,
• the standard deviation more or less corresponds to the length of a vector,
• the variance more or less corresponds to the squared length of a vector,
• the covariance more or less corresponds to the scalar product of the two vectors, and
• the correlation coefficient corresponds to the cosine of the angle between the two vectors; so
• being uncorrelated corresponds to the orthogonality of the two vectors.
Question: what does the equality in the proof of 7.1 correspond to?
9. Bivariate joint normal distributions
1: the bivariate standard normal distribution:
1.1: the distribution of the bivariate random variable with component variables X and Y is bivariate standard normal, if it has a joint density
Let X and Y both be univariate standard normal variables and let them be independent. Then these two, taken together as components of a vector variable, make a bivariate standard normal variable. (As X and Y are independent, their two-dimensional, joint density can be obtained as the product of their respective univariate densities.) Then
• the conditional densities (or the intersections of the joint density with planes of the kind x=x0, or with planes of the kind y=y0) are all univariate standard normal densities;
• the joint density is, as if a univariate normal density7 would have been rotated around the z-axis;
(Explanation: the nominator in the exponent is the distance squared of point (x;y) from the origin, so the points with a common value of the joint distribution fulfil the equality x2+y2=c with some non-negative c – these points are on a circle with center (x=0,y=0). Therefore the level sets of the joint distribution are circle-shaped with center (x=0,y=0).)
• the level sets of the joint distribution are concentric circles;
• the covariance matrix of the joint distribution is : the two-dimension identity matrix.
1.2: Likewise, the joint distribution the component variables X1,X2,...,Xn make is an n-dimensional normal distribution if the Xi s are independent, univariate standard normal variables.
(The k-dimensional conditonal distributions will be k-dimensional standard normal distributions here, the level sets n-dimensional spheres.)
2: bivariate normal distributions:
7more exactly, the 3-dimensional graph of the joint density is such, as if the 2-dimensional graph of a univariate normal density with zero expected value would have been rotated around the z-axis.
A bivariate normal distribution is obtained, making a linear transformation A on the bivariate standard normal variable . The distribution of the resulting random vector variable is called bivariate normal.
Less exactly (an example):
1. 1st step: make the vector variable extend 3-folds in the x-direction and compress 2-folds in the y-direction (that is, its x-coordinate is multiplied by 3, its y-coordinate is multiplied by 0.50). The density of the resulting vector variable is no longer rotationally symmetric: the graph is elongated in the x direction. The level sets will be concentric ellipses with their center in the origin, their length of their long axis is 6 times the length of the short axis; the long axes lie on the x-axis of the two-dimensional plane; the short axes on the y-axis.
2. 2nd step: rotate the plane e.g. to the left with angle α – the joint density rotates with it, so the elongation now is not parallel with the x-axis but oblique; the long axes of the level-set ellipses likewise are making an angle of α with the x-axis, the short axes are making an angle of α with the y-axis.
The first step corresponds to a multiplication with the matrix ;the second step corresponds to a multiplication with the matrix The two, taken together, corresponds to a multiplication with the
matrix .
Remark: two-dimensional normal distributions are obtained this way only if the linear transformation A is such that does not map the two-dimensional space into one dimension. (That is, linear tansformations with nonzero determinants, transformations with rank 2, regular transformations).
E.g., the matrix would map every point onto the x-axis;
would map every point onto the y-axis;
and a multiplication with the matrix maps every point to points with coordinates x=y, so these points all lie on the line x=y , that is, in one dimension.
If these transformations were used in the derivation AZ=X of the vector variable X, all values of X would lie on one single line, that is, in one dimension, so the distribution of X would not be a bivariate (two-dimensional) normal.
2.1.2: statements(1) – conditional distributions, conditional expectations and conditional variances with bivariate normal distributions:
With a bivariate normal distribution,
a. all conditional distributions are univariate normal distributions;
b. all marginal distributions are univariate normal distributions;
c. the level sets are concentric ellipses;
further:
(d') all conditional distributions (Y|X=x) have the same variance, that is, the conditional variances var(Y|X=x0
are all equal regardless of x0;
(d") all conditional distributions (X|Y=y) have the same variance, that is, the conditional variances var(X|Y=y0) are all equal regardless of y0 azonosak;
(e') the conditional expectation function E(Y| X=x) is a linear function of x;
(e") the conditional expectation function E(X| Y=y) is a linear function of y.
(f') the slope of E(Y| X) is (f") the slope of E(X| Y) is consequence.1 :
From the above (a),(d),(e) and (f) it follows that
if X and Y are the component variables of a bivariate normal distribution then X and Y are independent <=> E(Y| X) is constant <=> cov(X,Y)=0,
and
X and Y are independent <=> E(X| Y) is constant <=> cov(X,Y)=0.
The conditional distributions Y|X=x0 are all univariate normal distributions with equal variances, they can differ in their expected values only. These expected values – the values E(Y|X=x) – all lie on a linear function of x, on a line ax+b . If the covariance equals zero
=> the slope of the line ax+b equals zero
=> the conditional expectations E(Y|X=x) are all equal
=> the conditional distributions Y|X=x0 are all identical.
In short, with joint normal distributions, the uncorrelatedness of the component variables is a sufficient condition for their independence.
consequence.2 : (e') and (e") mean that with joint normal distributions the conditional expectation function E(Y|X) =x and E(X|Y)=y can be found among the linear functions: that is, with linear regression.
Remark: the line of the conditional expectation function (the line of the linear regression) is not identical to the line of longer axis of the level sets – slants less.
The regression line estimating y from x-values lies between the line of the long axes and the x-axis; the regression line estimating x from y-values lies between the line of the long axes and the y-axis. So the line estimating x from y and the line estimating y from x are not the same.
2.1.3, statements (2): calculations with the defining transformation matrix A:
Let be bivariate standard normal; let the bivariate normal be derived from Z with the transformation X=AZ (multiplying X with the 2x2 matrix A). Then
a. (a) the covariance matrix D of the new variable X is D = A AT .
(This follows from the statements concerning variances and covariances of linear combinations of variables in 6.5.)
Consequence: variances and covariances of component variables of a bivariate normal X are easily obtained, if we know the defining transformation matrix A.
b. the joint density of the new X derived this way is
(22.8) with the coefficients bi,j being the elements of corresponding indices of matrix [ b ]i,j = (A -1 )T A-1 .
2.1.4:
All the way up to now it has been assumed in (9) that the expected value of the vector variables in question is zero E(X1)=0 and E(X2)=0. In general, E(X1)=m1 and E(X2)=m2. Then
• the level sets are concentric ellipses with their common center in (m1,m2)
• the lines of the conditional expectation functions are crossing in (m1,m2).
That is, the joint distribution we have talked about by now is translated, in the geometrical sense, from the origin to (m1,m2).
So the joint density is
(22.9) 2.1.5,an alternative formula for the joint density:
the distribution of X and Y is bivariate normal if their joint density
(22.10 ) with m1 and m2 denoting the expected values of X1 és X2, d1 and d2 denoting the standard deviations of X1 és X2, and r denoting their correlation coefficient.
Readings
[bib_39] Probability and Statistical Inference. R Bartoszynski and M Niewiadomska-Bugaj. Copyright © 1996.
John Wiley & Sons, New York, Chichester, Brisbane, Toronto, Singapore. Chapter 7.