FACTOID # 17: Though Rhode Island is the smallest state in total area, it has the longest official name: The State of Rhode Island and Providence Plantations.
 
 Home   Encyclopedia   Statistics   States A-Z   Flags   Maps   FAQ   About 
   
 
WHAT'S NEW
RELATED ARTICLES
People who viewed "Estimator" also viewed:
 

SEARCH ALL

FACTS & STATISTICS    Advanced view

Search encyclopedia, statistics and forums:

 

 

(* = Graphable)

 

 


Encyclopedia > Estimator

In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data. Many different estimators are possible for any given parameter. Some criterion is used to choose between the estimators, although it is often the case that a criterion cannot be used to clearly pick one estimator over another. To estimate a parameter of interest (e.g., a population mean, a binomial proportion, a difference between two population means, or a ratio of two population standard deviation), the usual procedure is as follows: A graph of a Normal bell curve showing statistics used in educational assessment and comparing various grading methods. ... Graph of example function, The mathematical concept of a function expresses the intuitive idea of deterministic dependence between two quantities, one of which is viewed as primary (the independent variable, argument of the function, or its input) and the other as secondary (the value of the function, or output). A... The factual accuracy of this article is disputed. ... In mathematics, a set can be thought of as any collection of distinct objects considered as a whole. ...

  1. Select a random sample from the population of interest.
  2. Calculate the point estimate of the parameter.
  3. Calculate a measure of its variability, often a confidence interval.
  4. Associate with this estimate a measure of variability.

There are two types of estimators: point estimators and interval estimators. In statistics, a confidence interval (CI) for a population parameter is an interval between two numbers with an associated probability p which is generated from a random sample of an underlying population, such that if the sampling was repeated numerous times and the confidence interval recalculated from each sample according... In statistics, point estimation involves the use of sample data to calculate a single value which is to serve as a best guess for an unknown (fixed or random) population parameter. ... In statistics, interval estimation is the use of sample data to calculate an interval of possible (or probable) values of an unknown population parameter. ...

Contents

Point estimators

Suppose widehat{theta} is an estimator of a parameter theta . That is, widehat{theta} is a function that maps each sample s to its sample estimate widehat{theta}(s) .

  1. For a given sample s , the error of the estimator widehat{theta} is defined as widehat{theta}(s) - theta, where widehat{theta}(s) is the estimate for sample  s, and theta is the parameter being estimated. Note that the error depends not only on the estimator (the estimation formula or procedure), but on the sample.
  2. The mean squared error of widehat{theta} is defined as the expected value (probabilty-weighted average, over all samples) of the squared errors; that is, operatorname{MSE}(widehat{theta}) = operatorname{E}[(widehat{theta} - theta)^2]. It is used to indicate how far, on average, the collection of estimates are from the single parameter being estimated. Consider the following analogy. Suppose the parameter is the bull's-eye of a target, the estimator is the process of shooting arrows at the target, and the individual arrows are estimates. Then high MSE means the average distance of the arrows from the target is high, and low MSE means the average distance from the target is low. The arrows may or may not be clustered. For example, even if all arrows hit the same point, yet grossly miss the target, the MSE is still relatively large. Note, however, that if the MSE is relatively low, then the arrows are likely more highly clustered (than highly dispersed).
  3. For a given sample s , the sampling deviation of the estimator widehat{theta} is defined as widehat{theta}(s) - operatorname{E}( widehat{theta} ), where widehat{theta}(s) is the estimate for sample  s, and operatorname{E}( widehat{theta} ) is the expected value of the estimator. Note that the sampling deviation depends not only on the estimator, but on the sample.
  4. The variance of widehat{theta} is simply the expected value of the squared sampling deviations; that is, operatorname{var}(widehat{theta}) = operatorname{E}[(widehat{theta} - operatorname{E}(widehat{theta}) )^2]. It is used to indicate how far, on average, the collection of estimates are from the expected value of the estimates. Note the difference between MSE and variance. If the parameter is the bull's-eye of a target, and the arrows are estimates, then a relatively high variance means the arrows are dispersed, and a relatively low variance means the arrows are clustered. Some things to note: even if the variance is low, the cluster of arrows may still be far off-target, and even if the variance is high, the diffuse collection of arrows may still be unbiased. Finally, note that even if all arrows grossly miss the target, if they nevertheless all hit the same point, the variance is zero.
  5. The bias of widehat{theta} is defined as B(widehat{theta}) = operatorname{E}(widehat{theta}) - theta. It is the distance between the average of the collection of estimates, and the single parameter being estimated. It also is the expected value of the error, since operatorname{E}(widehat{theta}) - theta = operatorname{E}(widehat{theta} - theta ). If the parameter is the bull's-eye of a target, and the arrows are estimates, then a relatively high absolute value for the bias means the average position of the arrows is off-target, and a relatively low absolute bias means the average position of the arrows is on target. They may be dispersed, or may be clustered. The relationship between bias and variance is analogous to the relationship between accuracy and precision.
  6. widehat{theta} is an unbiased estimator of theta if and only if B(widehat{theta}) = 0, for all values of θ in the parameter space or, equivalently, if and only if operatorname{E}_theta(widehat{theta}) remains equal to θ regardless of the value of θ. Note that bias is a property of the estimator, not of the estimate. Often, people refer to a "biased estimate" or an "unbiased estimate," but they really are talking about an "estimate from a biased estimator," or an "estimate from an unbiased estimator." Also, people often confuse the "error" of a single estimate with the "bias" of an estimator. Just because the error for one estimate is large, does not mean the estimator is biased. In fact, even if all estimates have astronomical absolute values for their errors, if the expected value of the error is zero, the estimator is unbiased. Also, just because an estimator is biased, does not preclude the error of an estimate from being zero (we may have gotten lucky). The ideal situation, of course, is to have an unbiased estimator with low variance, and also try to limit the number of samples where the error is extreme (that is, have few outliers). Yet unbiasedness is not essential. Often, if we permit just a little bias, then we can find an estimator with lower MSE and/or fewer outlier sample estimates.
  7. The MSE, variance, and bias, are related: operatorname{MSE}(widehat{theta}) = operatorname{var}(widehattheta) + (B(widehat{theta}))^2,
i.e. mean squared error = variance + square of bias.

The standard deviation of an estimator of θ (the square root of the variance), or an estimate of the standard deviation of an estimator of θ, is called the standard error of θ. In statistics the mean squared error of an estimator T of an unobservable parameter θ is i. ... In probability theory and statistics, the variance of a random variable (or somewhat more precisely, of a probability distribution) is a measure of its statistical dispersion, indicating how its possible values are spread around the expected value. ... In statistics, the difference between an estimators expected value and the true value of the parameter being estimated is called the bias. ... In the fields of science, engineering, industry and statistics, accuracy is the degree of conformity of a measured or calculated quantity to its actual (true) value. ... In statistics, the difference between an estimators expected value and the true value of the parameter being estimated is called the bias. ... It has been suggested that this article or section be merged with Logical biconditional. ... In probability and statistics, the standard deviation of a probability distribution, random variable, or population or multiset of values is a measure of the spread of its values. ... In mathematics, a square root of a number x is a number r such that , or in words, a number r whose square (the result of multiplying the number by itself) is x. ... The standard error of a method of measurement or estimate is the estimated standard deviation of the error in that method. ...


Consistency

A consistent estimator is an estimator that converges in probability to the quantity being estimated as the sample size grows. In probability theory, there exist several different notions of convergence of random variables. ... Sample size, usually designated N, is the number of repeated measurements in a statistical sample. ...


An estimator tn (where n is the sample size) is a consistent estimator for parameter θ if and only if, for all ε > 0, no matter how small, we have Sample size, usually designated N, is the number of repeated measurements in a statistical sample. ... The factual accuracy of this article is disputed. ...

lim_{ntoinfty}Prleft{ left| t_n-thetaright|<epsilon right}=1.

It is called strongly consistent, if it converges almost surely to the true value. In probability theory, there exist several different notions of convergence of random variables. ...


Asymptotic normality

An asymptotically normal estimator is a consistent estimator whose distribution around the true parameter θ approaches a normal distribution with standard deviation shrinking in proportion to 1/sqrt{n} as the sample size n grows. Using &# 0;ghtarrow{D} to denote convergence in distribution, tn is asymptotically normal if The normal distribution, also called Gaussian distribution by scientists (named after Carl Friedrich Gauss due to his rigorous application of the distribution to astronomical data (Havil, 2003)), is a continuous probability distribution of great importance in many fields. ... In probability theory, there exist several different notions of convergence of random variables. ...

sqrt{n}(t_n - theta) &# 0;ghtarrow{D} N(0,V),

for some V, which is called the asymptotic variance of the estimator.


Asymptotic normality is proven using a central limit theorem. A central limit theorem is any of a set of weak-convergence results in probability theory. ...


Efficiency

The quality of an estimator is generally judged by its mean squared error. In statistics, efficiency is one measure of desirability of an estimator. ... In statistics the mean squared error of an estimator T of an unobservable parameter θ is i. ...


However, occasionally one chooses the unbiased estimator with the lowest variance. Efficient estimators are those that have the lowest possible variance among all unbiased estimators. In some cases, a biased estimator may have a uniformly smaller mean squared error than does any unbiased estimator, so one should not make too much of this concept. For that and other reasons, it is sometimes preferable not to limit oneself to unbiased estimators; see estimator bias. Concerning such "best unbiased estimators", see also Cramér-Rao bound, Gauss-Markov theorem, Lehmann-Scheffé theorem, Rao-Blackwell theorem. In statistics, efficiency is one measure of desirability of an estimator. ... In statistics, the Cramér-Rao bound (CRB) or Cramér-Rao lower bound (CRLB), named in honor of Harald Cramér and Calyampudi Radhakrishna Rao, expresses a lower bound on the variance of estimators of a deterministic parameter. ... In statistics, the difference between an estimators expected value and the true value of the parameter being estimated is called the bias. ... In statistics, the Cramér-Rao bound (CRB) or Cramér-Rao lower bound (CRLB), named in honor of Harald Cramér and Calyampudi Radhakrishna Rao, expresses a lower bound on the variance of estimators of a deterministic parameter. ... This article is not about Gauss-Markov processes. ... In statistics, the Lehmann-Scheffé theorem states the any estimator that is complete, sufficient, and unbiased is the unique best unbiased estimator of its expectation. ... In statistics, the Rao-Blackwell theorem describes a technique that can transform an absurdly crude estimator into an estimator that is optimal by the mean-squared-error criterion or any of a variety of similar criteria. ...


Robustness

See: Robust estimator, Robust statistics Robust statistics provides an alternative approach to classical statistical methods. ... Robust statistics provides an alternative approach to classical statistical methods. ...


Other properties

Sometimes, estimators should satisfy further restrictions (restricted estimators) - eg, one might require an estimated probability to be between zero and one, or an estimated variance to be nonnegative. Sometimes this conflicts with the requirement of unbiasedness, see the example in estimator bias concerning the estimation of the exponent of minus twice lambda based on a sample of size one from the Poisson distribution with mean lambda. In statistics, the difference between an estimators expected value and the true value of the parameter being estimated is called the bias. ...


See also

Estimation theory is a branch of statistics and signal processing that deals with estimating the values of parameters based on measured/empirical data. ... Maximum likelihood estimation (MLE) is a popular statistical method used to make inferences about parameters of the underlying probability distribution from a given data set. ... In statistics, the method of moments is a method of estimation of population parameters such as mean, variance, median, etc. ... The generalized method of moments is a very general statistical method for obtaining estimates of parameters of statistical models. ... In statistics, the Cramér-Rao bound (CRB) or Cramér-Rao lower bound (CRLB), named in honor of Harald Cramér and Calyampudi Radhakrishna Rao, expresses a lower bound on the variance of estimators of a deterministic parameter. ... Minimum mean-square error (MMSE) relates to an estimator having estimates with the minimum mean squared error possible. ... In statistics, the method of maximum a posteriori (MAP, or posterior mode) estimation can be used to obtain a point estimate of an unobserved quantity on the basis of empirical data. ... In statistics, and more specifically in estimation theory, a minimum-variance unbiased estimator (MVUE or MVU estimator) is an unbiased estimator of parameters, whose variance is minimized for all values of the parameters. ... This article is not about Gauss-Markov processes. ... In statistics the mean squared error of an estimator T of an unobservable parameter θ is i. ... In statistics, the difference between an estimators expected value and the true value of the parameter being estimated is called the bias. ... Result of particle filtering (red line) based on observed data generated from the blue line ( Much larger image) Particle filter methods, also known as Sequential Monte Carlo (SMC), are sophisticated model estimation techniques based on simulation. ... Markov chain Monte Carlo (MCMC) methods (which include random walk Monte Carlo methods) are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its stationary distribution. ... The Kalman filter is an efficient recursive filter that estimates the state of a dynamic system from a series of incomplete and noisy measurements. ... The Wiener filter is a filter proposed by Norbert Wiener during the 1940s and published [1]. // Description Unlike the typical filtering theory of designing a filter for a desired frequency response the Wiener filter approaches filtering from a different angle. ...

External links

  • A maths course on estimators
  • Fundamentals on Estimation Theory

  Results from FactBites:
 
estimator: Definition and Much More from Answers.com (616 words)
In statistics, an estimator is a function of the known sample data that is used to estimate an unknown population parameter; an estimate is the result from the actual application of the function to a particular set of data.
The standard deviation of an estimator of θ (the square root of the variance), or an estimate of the standard deviation of an estimator of θ, is called the standard error of θ.
A consistent estimator is an estimator that converges in probability to the quantity being estimated as the sample size grows.
PlanetMath: estimator (78 words)
In addition, sample median, sample mode, sample trimmed mean are all estimators of
This is version 3 of estimator, born on 2004-12-09, modified 2005-08-03.
Object id is 6549, canonical name is Estimator.
  More results at FactBites »

 
 

COMMENTARY     


Share your thoughts, questions and commentary here
Your name
Your comments

Want to know more?
Search encyclopedia, statistics and forums:

 


Press Releases |  Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms, 1022, m