PR_Chapter02_BayesianDecisionTheory

大数学家 Thomas Bayes (1702-1761), London, England.
© 庞彦伟, Tianjin University

第2章 第2页
DIP
§2.1 Introduction
Prior to the birth of Thomas Bayes, the proud parents, Mr Joshua Bayes and Mrs. Ann Carpenter Bayes, had 11 daughters (Anne, Rebecca, Mary, etc). While Mrs. Bayes was pregnant with Thomas, she REALLY, REALLY wanted a son. So they went to the local seer, who placed her hands on Mrs. Bayes‘ stomach and pronounced that without a doubt, the next baby would be a boy. Well, Mrs. Bayes really, really believed that this next baby would be a boy. So when the baby actually arrived, the actual physical evidence that the baby was a girl was not strong enough to overcome her prior (ahem) belief that the baby would be a boy, and so Joshua and Anne named their new baby daughter Thomas and raised her to be the son they had always wanted. （Jokes may be）
重要性:
研究内容:
Quantifying the tradeoffs between various classification decisions using probability and the costs that accompany such decisions.
前提
The decision problem is posed in probabilistic terms, and that all of the relevant probability values are known.
Decision rule based on priori probability
The only information we are allowed to use is the value of prior probabilities. Decide ω1, if P(ω1)>P(ω2); Decide ω2, if P(ω1)<=P(ω2); This rule makes sense if we to judge one fish, but if we to judge many fish, using this rule repeatedly many seem a bit strange. After all, we would always make the same decision even though we know that both types of fish will appear.
Prior
The prior probabilities reflect our prior knowledge of how likely we are to get a sea bass or salmon before the fish actually appears. It might depend upon the time of year or the choice of fishing area. There is some a priori probability P(ω1) that the next fish is sea bass, and some priori probability P(ω2) that the fish is salmon.
© 庞彦伟, Tianjin University

第2章 第3页
DIP
§2.1ቤተ መጻሕፍቲ ባይዱIntroduction
Bayesian decision theory is a fundamental statistical approach to the problem of pattern classification
问题：给定特征x, 判断该特征隶属于类ω1还是ω2
Decision based on px(x|ω) and p(ω) and feature value x
How does this measurement influence our attitude concerning the true state of nature (i.e. the category of the fish)? Bayes Theorem can be derived from the Product Rule of probability p (ω j , x) = p(ω j | x) p( x) = p ( x | ω j ) p(ω j )
You could be Bayesian if your observation are less important than your prior
大数学家 Thomas Bayes (1702-1761), London, England.
/~b rani/isyebayes/jokes.html
问题：给定特征x, 判断该特征隶属于类ω1还是ω2
Class-conditional probability density px(x|ω)
Different fish will yield different lightness readings, and we express this value this variability in probabilistic terms. px(x|ω) is the probability density function for x given that the state of nature is ω. Then the difference between px(x|ω1) and px(x|ω2) describes the difference in lightness between populations of sea bass and salmon.
© 庞彦伟, Tianjin University 第2章 第5页
DIP
§2.1 Introduction
问题：给定特征x, 判断该特征隶属于类ω1还是ω2
Prior
The prior probabilities reflect our prior knowledge of how likely we are to get a sea bass or salmon before the fish actually appears. It might depend upon the time of year or the choice of fishing area. There is some a priori probability P(ω1) that the next fish is sea bass, and some priori probability P(ω2) that the fish is salmon.
p (ω j | x) =
where
p ( x | ω j ) p (ω j ) p( x)
j
=
likelihood × prior = posterior evidence
j
p ( x ) = ∑ p ( w j , x) = ∑ p ( x | w j ) p ( w j )
Bayes formula shows that by observing the value of x we can convert the prior probablity P(ωj) to the a posteriori probatility p(ω|x)
Class-conditional probability density px(x|ω)
Feature value (e.g. lightness)
© 庞彦伟, Tianjin University 第2章 第7页
DIP
§2.1 Introduction
Statistical Pattern recognition vs. 算命
Even fortune-teller needs to see your palm or face
© 庞彦伟, Tianjin University 第2章 第6页
DIP
§2.1 Introduction
DIP
统计模式识别
第二章 贝叶斯决策理论 (Bayesian Decision Theory)
天津大学电子信息工程学院 庞彦伟 (Yanwei Pang) 2009年4月28日 年 月 日
DIP
§2.1 Introduction
However, some researchers still argue that concentrating on inference for model parameters is misguided and uses unobservable, theoretical quantities. Due to this skepticism, some are reluctant to fully support the Bayesian approach and philosophy. Even though Bayes was not highly recognized for his mathematical work during his life, he was elected a Fellow of the Royal Society in 1742. At that time he had no published works on mathematics, indeed none were published in his lifetime under his own name, the article on fluxions referred to above was published anonymously