FACTOID # 30: If Alaska were its own country, it would be the 26th largest in total area, slightly larger than Iran.
 
 Home   Encyclopedia   Statistics   States A-Z   Flags   Maps   FAQ   About 
   
 
WHAT'S NEW
 

SEARCH ALL

FACTS & STATISTICS    Advanced view

Search encyclopedia, statistics and forums:

 

 

(* = Graphable)

 

 


Encyclopedia > Bayesian probability

Bayesian probability is an interpretation of probability suggested by Bayesian theory, which holds that the concept of probability can be defined as the degree to which a person believes a proposition. Bayesian theory also suggests that Bayes' theorem can be used as a rule to infer or update the degree of belief in light of new information. Probability is the chance that something is likely to happen or be the case. ... This article is about the word proposition as it is used in logic, philosophy, and linguistics. ... Bayes theorem (also known as Bayes rule or Bayes law) is a result in probability theory, which relates the conditional and marginal probability distributions of random variables. ...

Contents

History

Thomas Bayes. (The correct identification of this portrait has been questioned.)

Bayesian theory and Bayesian probability are named after Thomas Bayes (1702 — 1761), who proved a special case of what is now called Bayes' theorem. The term Bayesian, however, came into use only around 1950, and it is not clear that Bayes would have endorsed the very broad interpretation of probability that is associated with his name. Laplace proved a more general version of Bayes' theorem and used it to solve problems in celestial mechanics, medical statistics and, by some accounts, even jurisprudence. Laplace, however, didn't consider this general theorem to be important for probability theory. He instead adhered to the classical definition of probability. PD image of Rev. ... PD image of Rev. ... Thomas Bayes (c. ... Bayes theorem (also known as Bayes rule or Bayes law) is a result in probability theory, which relates the conditional and marginal probability distributions of random variables. ... Year 1950 (MCML) was a common year starting on Sunday (link will display the full calendar) of the Gregorian calendar. ... To meet Wikipedias quality standards, this article or section may require cleanup. ... Philosophers of law ask what is law? and what should it be? Jurisprudence is the theory and philosophy of law. ... The classical definition of probability is identified with the works of Pierre Simon Laplace. ...


Frank P. Ramsey in The Foundations of Mathematics (1931) first proposed using subjective belief as a way of interpreting probability. Ramsey saw this interpretation as a complement to the frequency interpretation of probability, which was more established and accepted at the time. The statistician Bruno de Finetti in 1937 adopted Ramsey's view as an alternative to the frequency interpretation of probability. L. J. Savage expanded the idea in The Foundations of Statistics (1954). Frank Plumpton Ramsey (February 22, 1903 – January 19, 1930) was a British mathematician who, in addition to mathematics, made significant contributions in philosophy and economics. ... Statistical regularity has motivated the development of the relative frequency concept of probability. ... Bruno de Finetti (Innsbruck, June 13, 1906 - Rome, July 20, 1985) was an Italian probabilist and statistician, noted for the operational subjective conception of probability. ... Leonard Jimmie Savage (20 November 1917 - 1 November 1971) was a US mathematician and statistician. ...


Formal attempts have been made to define and apply the intuitive notion of a "degree of belief". The most common interpretation is based on betting: a degree of belief is reflected in the odds and stakes that the subject is willing to bet on the proposition at hand. The term gambling has had many different meanings depending on the cultural and historical context in which it is used. ...


On the Bayesian interpretation, the theorems of probability relate to the rationality of partial belief in the way that the theorems of logic are traditionally seen to relate to the rationality of full belief. Degrees of belief should not be regarded as extensions of the truth-values (true and false) but rather as extensions of the attitudes of belief and disbelief. Truth and falsity are metaphysical notions, while belief and disbelief are epistemic (or doxastic) ones. Metaphysical may refer to: Metaphysics, a branch of philosophy dealing with the ultimate nature of reality; or The Metaphysical poets, a poetic school from seventeenth century England who correspond with baroque period in European literature. ... 1. ...


The Bayesian approach has been explored by Harold Jeffreys, Richard T. Cox, Edwin Jaynes and I. J. Good. Other well-known proponents of Bayesian probability have included John Maynard Keynes and B.O. Koopman, and many philosophers of the 20th century. Sir Harold Jeffreys (22 April 1891 – 18 March 1989) was a mathematician, statistician, geophysicist, and astronomer. ... Richard Threlkeld Cox (1898 - May 2, 1991) was a professor of physics at Johns Hopkins University, known for Coxs theorem relating to the foundations of probability. ... Edwin Thompson Jaynes (July 5, 1922 – April 30, 1998) was Wayman Crow Distinguished Professor of Physics at Washington University in St. ... Irving John (Jack) Good (born 9 December 1916) is a British statistician who worked also as a cryptographer and developer of the Colossus computer at Bletchley Park. ... This article does not cite any references or sources. ... Bernard O. Koopman (1900–1981) was a French-born American mathematician, known for his work in operations research. ...


Varieties

The terms subjective probability, personal probability, epistemic probability and logical probability describe some of the schools of thought which are customarily called "Bayesian". These overlap but there are differences of emphasis. Some of the people mentioned here would not call themselves Bayesians.


Bayesian probability is supposed to measure the degree of belief an individual has in an uncertain proposition, and is in that respect subjective. Some people who call themselves Bayesians do not accept this subjectivity. The chief exponents of this objectivist school were Edwin Thompson Jaynes and Harold Jeffreys. Perhaps the main objectivist Bayesian now living is James Berger of Duke University. Jose Bernardo and others accept some degree of subjectivity but believe a need exists for "reference priors" in many practical situations. Edwin Thompson Jaynes (July 5, 1922 – April 30, 1998) was Wayman Crow Distinguished Professor of Physics at Washington University in St. ... Sir Harold Jeffreys (22 April 1891 – 18 March 1989) was a mathematician, statistician, geophysicist, and astronomer. ...


Advocates of logical (or objective epistemic) probability, such as Harold Jeffreys, Rudolf Carnap, Richard Threlkeld Cox and Edwin Jaynes, hope to codify techniques whereby any two persons having the same information relevant to the truth of an uncertain proposition would calculate the same probability. Such probabilities are not relative to the person but to the epistemic situation, and thus lie somewhere between subjective and objective. However, the methods proposed are controversial. Critics challenge the claim that there are grounds for preferring one degree of belief over another in the absence of information about the facts to which those beliefs refer. Another problem is that the techniques developed so far are inadequate for dealing with realistic cases. Sir Harold Jeffreys (22 April 1891 – 18 March 1989) was a mathematician, statistician, geophysicist, and astronomer. ... Rudolf Carnap (May 18, 1891, Ronsdorf, Germany – September 14, 1970, Santa Monica, California) was an influential philosopher who was active in central Europe before 1935 and in the United States thereafter. ... Richard Threlkeld Cox (1898 - May 2, 1991) was a professor of physics at Johns Hopkins University, known for Coxs theorem relating to the foundations of probability. ... Edwin Thompson Jaynes (July 5, 1922 – April 30, 1998) was Wayman Crow Distinguished Professor of Physics at Washington University in St. ...


Relationship to frequency probability

Bayesian probability - sometimes called credence (i.e. degree of belief) - contrasts with frequency probability, in which probability is derived from observed frequencies in defined distributions or proportions in populations. Statistical regularity has motivated the development of the relative frequency concept of probability. ...


The theory of statistics and probability using frequency probability was developed by R.A. Fisher, Egon Pearson and Jerzy Neyman during the first half of the 20th century. A. N. Kolmogorov also used frequency probability to lay the mathematical foundation of probability in measure theory via the Lebesgue integral in Foundations of the Theory of Probability (1933). Savage, Koopman, Abraham Wald and others have developed Bayesian probability since 1950. Statistical regularity has motivated the development of the relative frequency concept of probability. ... Sir Ronald Fisher Sir Ronald Aylmer Fisher, FRS (February 17, 1890–July 29, 1962) was an extraordinarily talented evolutionary biologist, geneticist and statistician. ... Egon Sharpe Pearson (11 August 1895 — 12 June 1980) a son of Karl Pearson, was like his father, a British statistician, and succeeded him as professor of statistics at University College London and as editor of the journal Biometrika. ... Jerzy Neyman (April 16, 1894, in Bendery, Moldova – August 5, 1981, in Oakland, California) was a Polish mathematician. ... Andrey Nikolaevich Kolmogorov (Андре́й Никола́евич Колмого́ров) (kahl-mah-GAW-raff) (April 25, 1903 in Tambov - October 20, 1987 in Moscow) was a Russian mathematician... The integral can be interpreted as the area under a curve. ... Abraham Wald (October 31, 1902 Kolozsvár, Hungary (now Cluj, Romania) - December 13, 1950 India) was a mathematician who contributed to decision theory, geometry, and econometrics, and founded the field of statistical sequential analysis. ...


The difference between Bayesian and Frequentist interpretations of probability has important consequences in statistical practice. For example, when comparing two hypotheses using the same data, the theory of hypothesis tests, which is based on the frequency interpretation of probability, allows the rejection or non-rejection of one model/hypothesis (the 'null' hypothesis) based on the probability of mistakenly inferring that the data support the other model/hypothesis more. The probability of making such a mistake, called a Type I error, requires the consideration of hypothetical data sets derived from the same data source that are more extreme than the data actually observed. This approach allows the inference that 'either the two hypotheses are different or the observed data are a misleading set'. In contrast, Bayesian methods condition on the data actually observed, and are therefore able to assign posterior probabilities to any number of hypotheses directly. The requirement to assign probabilities to the parameters of models representing each hypothesis is the cost of this more direct approach. One may be faced with the problem of making a definite decision with respect to an uncertain hypothesis which is known only through its observable consequences. ... In statistics, a null hypothesis is a hypothesis set up to be nullified or refuted in order to support an alternative hypothesis. ... In statistical hypothesis testing, a Type I error consists of rejecting a null hypothesis that is true, in other words finding a result to have statistical significance when this has in fact happened by chance. ...


Applications

Since the 1950s, Bayesian theory and Bayesian probability have been widely applied through Cox's theorem, Jaynes' principle of maximum entropy and the Dutch book argument. In many applications, Bayesian methods are more general and appear to give better results than frequency probability. Bayes factors have also been applied with Occam's Razor. See Bayesian inference and Bayes' theorem for mathematical applications. Coxs theorem, named after the physicist Richard Threlkeld Cox, is a derivation of the laws of probability theory from a certain set of postulates. ... The principle of maximum entropy is a method for analyzing the available information in order to determine a unique epistemic probability distribution. ... In gambling a Dutch book or lock is a set of odds and bets which guarantees a profit, no matter what the outcome of the gamble. ... Statistical regularity has motivated the development of the relative frequency concept of probability. ... In statistics, the use of Bayes factors is a Bayesian alternative to classical hypothesis testing. ... William of Ockham Occams razor (sometimes spelled Ockhams razor) is a principle attributed to the 14th-century English logician and Franciscan friar William of Ockham. ... Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true. ... Bayes theorem (also known as Bayes rule or Bayes law) is a result in probability theory, which relates the conditional and marginal probability distributions of random variables. ...


Some regard Bayesian inference as an application of the scientific method because updating probabilities through Bayesian inference requires one to start with initial beliefs about different hypotheses, to collect new information (for example, by conducting an experiment), and then to adjust the original beliefs in the light of the new information. Adjusting original beliefs could mean (coming closer to) accepting or rejecting the original hypotheses. Scientific method is a body of techniques for investigating phenomena and acquiring new knowledge, as well as for correcting and integrating previous knowledge. ... Look up Hypothesis in Wiktionary, the free dictionary. ... In the scientific method, an experiment (Latin: ex-+-periri, of (or from) trying), is a set of actions concerning phenomena. ...


Bayesian techniques have recently been applied to filter spam e-mail. A Bayesian spam filter uses a reference set of e-mails to define what is originally believed to be spam. After the reference has been defined, the filter then uses the characteristics in the reference to define new messages as either spam or legitimate e-mail. New e-mail messages act as new information, and if mistakes in the definitions of spam and legitimate e-mail are identified by the user, this new information updates the information in the original reference set of e-mails with the hope that future definitions are more accurate. See Bayesian inference and Bayesian filtering. E-mail spam is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ... Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true. ... Bayesian filtering is the process of using Bayesian statistical methods to classify documents into categories. ...


Probabilities of probabilities

One criticism levelled at the Bayesian probability interpretation has been that a single probability assignment cannot convey how well grounded the belief is—i.e., how much evidence one has. Consider the following situations: The scientific method or process is fundamental to the scientific investigation and acquisition of new knowledge based upon physical evidence. ...

  1. You have a box with white and black balls, but no knowledge as to the quantities
  2. You have a box from which you have drawn n balls, half black and the rest white
  3. You have a box and you know that there are the same number of white and black balls

The Bayesian probability of the next ball drawn being black is 0.5 in all three cases. Keynes called this the problem of the "weight of evidence". One approach is to reflect difference in evidential support by assigning probabilities to these probabilities (so-called metaprobabilities) in the following manner: John Maynard Keynes John Maynard Keynes [ˈkeɪns], 1st Baron Keynes of Tilton (June 5, 1883 - April 21, 1946) was an English economist, whose radical ideas had a major impact on modern economic and political thought. ...

1. You have a box with white and black balls, but no knowledge as to the quantities
Letting θ = p represent the statement that the probability of the next ball being black is p, a Bayesian might assign a uniform Beta prior distribution:
forall theta in [0,1]
P(theta) = Beta(alpha_B=1,alpha_W=1) = frac{Gamma(alpha_B + alpha_W)}{Gamma(alpha_B)Gamma(alpha_W)}theta^{alpha_B-1}(1-theta)^{alpha_W-1} = frac{Gamma(2)}{Gamma(1)Gamma(1)}theta^0(1-theta)^0=1
Assuming that the ball drawing is modelled as a binomial sampling distribution, the posterior distribution, P(θ | m,n), after drawing m additional black balls and n white balls is still a Beta distribution, with parameters αB = 1 + m, αW = 1 + n. An intuitive interpretation of the parameters of a Beta distribution is that of imagined counts for the two events. For more information, see Beta distribution.
2. You have a box from which you have drawn N balls, half black and the rest white
Letting θ = p represent the statement that the probability of the next ball being black is p, a Bayesian might assign a Beta prior distribution, Β(N / 2 + 1,N / 2 + 1). The maximum aposteriori estimate (MAP estimate) of θ is theta_{MAP}=frac{N/2+1}{N+2}, precisely Laplace's rule of succession.
3. You have a box and you know that there are the same number of white and black balls
In this case a Bayesian would define the prior probability Pleft(thetaright)=deltaleft(theta - frac{1}{2}right).

Other Bayesians have argued that probabilities need not be precise numbers. In probability theory and statistics, the beta distribution is a continuous probability distribution with the probability density function (pdf) defined on the interval [0, 1]: where α and β are parameters that must be greater than zero and B is the beta function. ... It has been suggested that this article or section be merged with MAP estimator. ... In probability theory, the rule of succession is a formula introduced in the 18th century by Pierre-Simon Laplace in the course of treating the sunrise problem. ...


Because there is no room for metaprobabilities on the frequency interpretation, frequentists have had to find different ways of representing difference of evidential support. Cedric Smith and Arthur Dempster each developed a theory of upper and lower probabilities. Glenn Shafer developed Dempster's theory further, and it is now known as Dempster-Shafer theory. Cedric Austen Bardell Smith (February 5, 1917, – January 16, 2002) was a British statistician and geneticist. ... Upper and lower probabilities are representations of imprecise probability. ... Glenn Shafer authored A Mathematical Theory of Evidence which was published by the Princeton University Press in 1976. ... The Dempster-Shafer theory is a mathematical theory of evidence [SH76] based on belief functions and plausible reasoning, which is used to combine separate pieces of information (evidence) to calculate the probability of an event. ...


Controversy

A quite different interpretation of the term probable has been developed by frequentists. In the "frequency probability" interpretation, what are probable are not propositions entertained by believers, but events considered as members of collectives to which the tools of statistical analysis can be applied. Statistical regularity has motivated the development of the relative frequency concept of probability. ...


The Bayesian interpretation of probability allows probabilities to be assigned to all propositions (or, in some formulations, to the events signified by those propositions) in any reference class of events, independent of whether the events can be interpreted as having a relative frequency in repeated trials. Controversially, Bayesian probability assignments are relative to a (possibly hypothetical) rational subject: it is therefore not inconsistent to be able to assign different probabilities to the same proposition by Bayesian methods based on the same observed data. Differences arise either because different models of the data generating process are used or because of different prior probability assignments to model parameters. Differences due to model differences are possible in both Frequentist and Bayesian analyses, but differences due to prior probabilities assignments are distinctive to Bayesian analyses. Such probabilities are sometimes called 'personal probabilities', although there may be no particular individual to whom they belong. Frequentists argue that this makes Bayesian analyses subjective, in the negative sense of 'not determined from the data'. Bayesians typically reply that differences due to alternative models of the data generating process are equally subjective, and that such model choices are also (ideally) chosen prior to analysis of the data, by the analyst. Since the analyst is also a natural person to assign prior probabilities, the two subjective inputs can be seen as deriving from the same source. In a series of observations, or trials, the relative frequency of occurrence of an event is calculated as: The of an event over a long series of trials is the conceptual foundation of the frequency interpretation of probability. ...


Although there is no reason why different interpretations (senses) of a word cannot be used in different contexts, there is a history of antagonism between Bayesians and frequentists, with the latter often rejecting the Bayesian interpretation as ill-grounded. The groups have also disagreed about which of the two senses reflects what is commonly meant by the term 'probable'. More importantly, the groups have agreed that Bayesian and Frequentist analyses answer genuinely different questions, but disagreed about which class of question it is more important to answer in scientific and engineering contexts.


See also

The word probability has been used in a variety of ways since it was first coined in relation to games of chance. ... Statistical regularity has motivated the development of the relative frequency concept of probability. ... This article or section does not adequately cite its references or sources. ... Inference is the act or process of deriving a conclusion based solely on what one already knows. ... Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true. ... The Doomsday argument (DA) is a probabilistic argument that claims to predict the future lifetime of the human race given only an estimate of the total number of humans born so far. ... In physics the Maximum entropy school of thermodynamics (or more colloquially, the MaxEnt school of thermodynamics), initiated with two papers published in the Physical Review by Edwin T. Jaynes in 1957, views statistical mechanics as an inference process: a specific application of inference techniques rooted in information theory, which relate... // Philosophy of mathematics is the branch of philosophy that studies the philosophical assumptions, foundations, and implications of mathematics. ...

External links and references


  Results from FactBites:
 
BIPS: Bayesian Inference for the Physical Sciences (2872 words)
From Laplace to SN 1987A: Bayesian inference in astrophysics (Loredo 1990)
From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics (1990)
The Promise of Bayesian Inference for Astrophysics (1992)
Bayesian inference: Information from Answers.com (3621 words)
Bayesian inference is statistical inference in which evidence or observations are used to update or to newly infer the probability that a hypothesis may be true.
Bayesian inference uses aspects of the scientific method, which involves collecting evidence that is meant to be consistent or inconsistent with a given hypothesis.
Bayesian inference can be used in a court setting by an individual juror to coherently accumulate the evidence for and against the guilt of the defendant, and to see whether, in totality, it meets their personal threshold for 'beyond a reasonable doubt'.
  More results at FactBites »

 
 

COMMENTARY     


Share your thoughts, questions and commentary here
Your name
Your comments

Want to know more?
Search encyclopedia, statistics and forums:

 


Press Releases |  Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms, 1022, m