exponential

All points in Θ1−α have a higher density than any point outside the regions. Often requires iterative solution: Find points such that p(θ|Y ) > h Find probability of that set Adjust h until reach desired coverage may not have symmetric tail areas multimodal posterior may not have an interval
L(β) ∝ β n exp(−β yi )
One Parameter Models – p. 7/2
Gamma Distribution
Z ∼ Gamma(a, b) with mean a/b and variance a/b2 with density ba a−1 f (z) = exp(−zb) z Γ(a) z>0
f (y|β) = β exp(−yβ) L(β; y1 , . . . , yn ) =
i
y>0
β exp (−yi β) yi β)
i
(Likelihood)
= β n exp (−
Look at plot of L(β) for
yi = 336.
One Parameter Models – p. 4/2
Distribution of β
Identify normalizing constant to make this a density for β or recognize the form of the distribution
p(β|y1 , . . . , yn ) = cβ n exp(−β yi )
> qgamma(.025, 10, 336) [1] 0.01427199 > qgamma(.975, 10, 336) [1] 0.05084763
One Parameter Models – p. 15/2
Examples of Conjugate Families
Data Distributions Prior/Posterior Exponential Gamma Gamma Poisson Beta Binomial Normal Normal (known variance) Normal (unknown mean/variance) Normal-Gamma Always available in exponential families
P yi )
(1) (2) (3) (4)
yi )
“Uniform” is a limiting case with a = 1 and b → 0 “default” prior is p(β) ∝ 1/β – a “Gamma(0,0)”
One Parameter Models – p. 12/2
One Parameter Models – p. 10/2
Uniform Prior
“Uniform” prior p(β) = 1 in exponential example is not a proper distribution; although the posterior distribution is a proper distribution. “Formal Bayes” posterior distribution obtained as a limit of a proper Bayes procedure. Be very careful with improper prior distributions, they may not lead to proper posterior distributions!
One Parameter Models – p. 8/2
Examples of Gamma Distributions
0.20
a = 1, r = 1/5 a = 2, r = 1/5 a = 2, r = 1/2
Density
0.05
0.10
0.15
0
2
4 x
6
8
10
One Parameter Models – p. 9/2
One Parameter Models
September 22, 2010
Reading: Hoff Chapter 3
One Parameter Models – p. 1/2
Highest Posterior Density Regions
Find
Θ1−α = {θ : p(θ | Y ) ≥ hα } such that P (θ ∈ Θ1−α | Y ) = 1−α
In this parameterization b is a rate parameter.
rgamma(n, shape=a, scale=s) # mean = shape*scale rgamma(n, shape=a, rate=r) # mean = shape/rate
Mode is at (a − 1)/b if a > 1 and at 0 if a ≤ 1
One Parameter Models – p. 16/2
Exponential Family
One parameter exponential family is expressed as
p(y | θ) = h(y)c(θ) exp(θt(y))
where θ is the natural parameter of the exponential family and t(y) is the sufficient statistic Likelihood: L(θ) ∝ c(θ)n exp(θ n
Likelihood function
l.exp = function(theta, y) { n = length(y) sumy = sum(y) l = thetaˆn*exp(-sumy*theta) return(l) } Vectorized: can be used to evaluate L(θ) at multiple values ot θ rather than using a loop beta = seq(.00001, .25, length=1000) l.exp(beta, y)
where
∞
c = 1/
0
β n exp(−β
yi )dβ yi )
Looks like a Gamma(a = n + 1, b =
normalized likelihood has same shape has likelihood same form that would be obtained using Bayes Theorem with a “uniform” prior
One Parameter Models – p. 11/2
Gamma Prior/Posterior Distributions
Prior and Posterior distributions are in the same family
β ∼ G(a, b) ba a−1 −βb n −β P yi p(β | Y ) ∝ β e β e Γ(a) ∝ β a+n−1 e−β(b+ β | Y ∼ G(a + n, b +
One Parameter Models – p. 5/2
Plot of Likelihood
Exponential Likelihood
2.5e−20 L(β) 0.0e+00 0.00 5.0e−21 1.0e−20 1.5e−20 2.0e−20
0.05
0.10 β
0.15
0.20
0.25
One Parameter Models – p. 6/2
One Parameter Models – p. 3/2
Exponential Example: Rate β
The time between accidents modeled with an exponential distribution with a rate of β accidents per day. n = 10 observations: 1.5 15.0 60.3 30.5 2.8 56.4 27.0 6.4 110.7 25.4
Posterior Quantities
posterior distribution G(10, 336)
ˆ posterior mode β = 0.0268
posterior mean E(β|Y ) = 10/336 = 0.0298 Interval Estimates – Probability Intervals or Credible Regions Highest Posterior Density region [0.012, 0.049] Equal Tail area [0.014, 0.051] After observing the data, we believe there is a 95% chance of [0.01, 0.05] accidents per day (or 1 to 5 accidents per every 100 days)
One Parameter Models – p. 13/2
Highest Posterior Density Region
95% Highest Posterior Density [0.012, 0.05]