异方差层次模型中异方差的最优收缩估计

Optimal One??
SURE estimate
Stein’s unbiased risk estimate (SURE): With sum of squared error loss
for shrinkage estimator
,
an unbiased estimate of its risk
and
Real data: Baseball
Brown (2008) collects statistics of 567 MLB players of the 2005 season
Task: Given first half season performance, predict the second half season
James-Stein estimator
Shrinkage est.
Emp. Bayes est.
Hierarchical model: a natural way to arrive at it
The picture
Efron and Morris (1977)
Heteroscedastic Hierarchical Model
Optimality For any and the corresponding we always have
and
Regularity conditions
The oracle property
Oracle parameter: parameter that minimizes the loss
Optimal Shrinkage Estimation in Heteroscedastic Hierarchical Models
Samuel Kou Department of Statistics Harvard University Joint with Lawrence Brown and Xianchao Xie
SURE estimate
An unbiased estimate of the risk
is
SURE estimate:
where
Risk properties
SUREM(λ, μ) is uniformly close to
:
Can be strengthened to loss function:
Shrinkage Estimate
Ever since Stein’s groundbreaking work, shrinkage estimation not only revolutionized statistics, but also has wide-ranging applications in science and engineering.
Task: Given first half season performance, predict the second half season
Simulation Results
Case 1: Ideal empirical Bayes setup:
risk
EB-ML EB-MM SURE (par)
SURE est.
3
Real data: Baseball
Brown (2008) collects statistics of 567 MLB players of the 2005 season
Task: Given first half season performance, predict the second half season
Semi-parametric SURE estimate where
Risk properties
SUREM(b, μ) is uniformly close to
:
L1
Optimality
For any shrinkage estimator
with
and MON, we always have
So far, shrink toward origin as we put
Can shrink toward the grand mean In general, starting from
Bayes estimate
Determine λ and μ from the data?
Oracle property Asymptotically optimal
Competitive numerical performance
Acknowledgement
Larry Brown Xianchao Xie
NSF, NIH
Thank you!
5
SURE (Semi-par)
Par. SURE
Semi-parametric SURE p
Case 2:
Case 3:
Shrinkage factor risk
risk
Par. SURE
Semi-parametric SURE p
Par. SURE Semi-parametric SURE
Question: can one do better? Yes, if we move on to a larger class:
Semi-parametric
Semi-parametric SURE estimate
An unbiased estimate of the risk is
Semi-parametric SURE estimate
The properties of the SURE shrinkage estimate does not involve the prior distribution
However, the general form of motivated from normal prior
Distinct variances (e.g., different accuracy, sample sizes)
Shrinkage estimate
How to obtain ? Could use empirical Bayes MLE empirical Bayes MM Generalized James-Stein … …
TotalTSoqtaulaSrequEarrreorEarrftoerron the atrhcesipnrotrpaonrstfioonrms ation
Semi-parametric SURE
Real data: Baseball
Brown (2008) collects statistics of 567 MLB players of the 2005 season
p
4
Case 4:
risk
Par. SURE
p
Semi-parametric SURE
Conclusion
We consider shrinkage estimate for heteroscedastic hierarchical Models
Propose SURE estimate: Par. and semi-par. Good theoretical properties of SURE estimate
depends on each realization. gives the theoretical limit for any shrinkage estimator This theoretical limit is achieved by the SURE estimate:
General shrinkage estimate
2
Risk properties (cont.)
Optimality
Hale Waihona Puke For anyand corresponding
we always have
and
The oracle property

Oracle
(
λ~paOrLa,m~eOtLe)rs:
arg
min
l
(θ,
θˆ

,
)
,
depends on each realization. gives the theoretical limit for any shrinkage estimator This theoretical limit is achieved by the SURE estimate:
is
SURE estimate:
Risk properties of SURE estimate
SURE(λ) is uniformly close to
:
L2
In fact, can strengthen to the loss function: L2
1
Risk properties of SURE estimate (cont.)