 FACTOID # 7: The top five best educated states are all in the Northeast.

 Home Encyclopedia Statistics States A-Z Flags Maps FAQ About

 WHAT'S NEW

SEARCH ALL

Search encyclopedia, statistics and forums:

(* = Graphable)

Encyclopedia > Covariance matrix

In statistics and probability theory, the covariance matrix is a matrix of covariances between elements of a vector. It is the natural generalization to higher dimensions, of the concept of the variance of a scalar-valued random variable. Statistics is a type of data analysis which practice includes the planning, summarizing, and interpreting of observations of a system possibly followed by predicting or forecasting of future events based on a mathematical model of the system being observed. ... Probability theory is the mathematical study of probability. ... In probability theory and statistics, the covariance between two real-valued random variables X and Y, with expected values and is defined as: where E is the expected value. ... In probability theory and statistics, the variance of a random variable is a measure of its statistical dispersion, indicating how far from the expected value its values typically are. ... Scalar is a concept that has meaning in mathematics, physics, and computing. ... A random variable can be thought of as the numeric result of operating a non-deterministic mechanism or performing a non-deterministic experiment to generate a random result. ...

If X is a column vector with n scalar random variable components, and μk is the expected value of the kth element of X, i.e., μk = E(Xk), then the covariance matrix is defined as: Scalar is a concept that has meaning in mathematics, physics, and computing. ... In probability theory (and especially gambling), the expected value (or mathematical expectation) of a random variable is the sum of the probability of each possible outcome of the experiment multiplied by its payoff (value). Thus, it represents the average amount one expects to win per bet if bets with identical...

The (i,j) element is the covariance between Xi and Xj. In probability theory and statistics, the covariance between two real-valued random variables X and Y, with expected values and is defined as: where E is the expected value. ...

This concept generalizes to higher dimensions the concept of variance of a scalar-valued random variable X, defined as In probability theory and statistics, the variance of a random variable is a measure of its statistical dispersion, indicating how far from the expected value its values typically are. ...

where μ = E(X).

## Conflicting nomenclatures and notations GA_googleFillSlot("encyclopedia_square");

Nomenclatures differ. Some statisticians, following the probabilist William Feller, call this matrix the variance of the random vector X, because it is the natural generalization to higher dimensions of the 1-dimensional variance. Others call it the covariance matrix, because it is the matrix of covariances between the scalar components of the vector X. Unfortunately, several different conventions jar to some degree with each other: William (Vilim) Feller (July 7, 1906 - January 14, 1970) was a Croatian-American mathematician specializing in probability theory. ... In probability theory and statistics, the covariance between two real-valued random variables X and Y, with expected values and is defined as: where E is the expected value. ...

Standard notation:

Also standard notation (unfortunately conflicting with the above):

Also standard notation:

(the "cross-covariance" between two random vectors)

The first two of these usages conflict with each other. The first and third are in perfect harmony. The first notation is found in William Feller's universally admired two-volume book on probability. William (Vilim) Feller (July 7, 1906 - January 14, 1970) was a Croatian-American mathematician specializing in probability theory. ...

## Properties

For and the following basic properties apply:

1. If p = q,
2. If and are independent, then

where and are a random vectors, is a random vector, is vector, and are matrices.

This covariance matrix (though very simple) is a very useful tool in many very different areas. From it a transformation matrix can be derived that allows one to completely decorrelate the data or, from a different point of view, to find an optimal basis for representing the data in a compact way. This is called principal components analysis (PCA) in statistics and Karhunen-Loève transform (KL-transform) in image processing. In linear algebra, linear transformations can be represented by matrices. ... In statistics, principal components analysis (PCA) is a technique that can be used to simplify a dataset; more formally it is a linear transformation that chooses a new coordinate system for the data set such that the greatest variance by any projection of the data set comes to lie on... Statistics is a type of data analysis which practice includes the planning, summarizing, and interpreting of observations of a system possibly followed by predicting or forecasting of future events based on a mathematical model of the system being observed. ... This article needs to be cleaned up to conform to a higher standard of quality. ...

## Complex random vectors

The variance of a complex scalar-valued random variable with expected value μ is conventionally defined using complex conjugation: In mathematics, the complex numbers are an extension of the real numbers by the inclusion of the imaginary unit i, satisfying . ... In mathematics, the complex conjugate of a complex number is given by changing the sign of the imaginary part. ...

where the complex conjugate of a complex number z is denoted z * .

If Z is a column-vector of complex-valued random variables, then we take the conjugate transpose by both transposing and conjugating, getting a square matrix: In mathematics, the conjugate transpose or adjoint of an m-by-n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking the transpose and then taking the complex conjugate of each entry. ...

where Z * denotes the conjugate transpose, which is applicable to the scalar case since the transpose of a scalar is still a scalar.

## Estimation

The derivation of the maximum-likelihood estimator of the covariance matrix of a multivariate normal distribution is perhaps surprisingly subtle. It involves the spectral theorem and the reason why it can be better to view a scalar as the trace of a matrix than as a mere scalar. See estimation of covariance matrices. In probability theory and statistics, a multivariate normal distribution, also sometimes called a multivariate Gaussian distribution (in honor of Carl Friedrich Gauss, who was not the first to write about the normal distribution) is a specific probability distribution. ... In mathematics, particularly linear algebra and functional analysis, the spectral theorem is a collection of results about linear operators or about matrices. ... Scalar is a concept that has meaning in mathematics, physics, and computing. ... In multivariate statistics, the importance of the Wishart distribution stems in part from the fact that it is the probability distribution of the maximum likelihood estimator of the covariance matrix of a multivariate normal distribution. ...

## External references Results from FactBites:

 APT - The APT Approach (2247 words) To estimate portfolio risk irrespective of the covariance matrix is folly. The relation is embedded in the covariance matrix. Their meaning is that they span the covariance matrix best; simply put, they reflect the actual covariance between all stocks, irrespective of what real world variable may have led to such covariance at any one point.
 Positive Definite, Positive Semidefinite Covariance Matrix (996 words) Positive semidefinite matrices are important in probability theory because covariance matrices are always positive semidefiniteso all properties of positive semidefinite matrices are properties of covariance matrices. Because a correlation matrix is essentially a normalized covariance matrix, results apply equally to them. In this sense, a singular covariance matrix indicates that at least one component of a random vector is extraneous.
More results at FactBites »

Share your thoughts, questions and commentary here
Press Releases | Feeds | Contact