lec5 Overdispersion in logistic regression

Lecture5:Overdispersion in logistic regressionClaudia CzadoTU M¨u nchenOverview•Definition of overdispersion•Detection of overdispersion•Modeling of overdispersionOverdispersion in logistic regressionCollett(2003),Chapter6Logistic model:Y i∼bin(n i,p i)independentp i=e x t iβ/(1+e x t iβ)⇒E(Y i)=n i p i V ar(Y i)=n i p i(1−p i)If one assumes that p i is correctly modeled,but the observed variance is larger or smaller than the expected variance from the logistic model given by n i p i(1−p i), one speaks of under or overdispersion.In application one often observes only overdispersion,so we concentrate on modeling overdispersion.How to detect overdispersion?If the logistic model is correct the asymptotic distribution of the residual deviance D∼χ2n−p.Therefore D>n−p=E(χ2n−p)can indicate overdispersion. Warning:D>n−p can also be the result of-missing covariates and/or interaction terms;-negligence of non linear effects;-wrong link function;-existance of large outliers;-binary data or n i small.One has to exclude these reasons through EDA and regression diagnostics.Residual deviance for binary logistic models Collett(2003)shows that the residual deviance for binary logistic models canbe written asD=−2ni=1(ˆp i lnˆp i1−ˆp i+ln(1−ˆp i)),whereˆp i=e x i tˆβ/(1+e x i tˆβ).This is independent of Y i,therefore not useful to assess goodness offit.Need to group data to use residual deviance as goodness offit measure.Reasons for overdispersion Overdispersion can be explained by-variation among the success probabilities or-correlation between the binary responsesBoth reasons are the same,since variation leads to correlation and vice versa. But for interpretative reasons one explanation might be more reasonable than the other.Variation among the success probabilitiesIf groups of experimental units are observed under the same conditions,the success probabilities may vary from group to group.Example:The default probabilities of a group of creditors with same conditions can vary from bank to bank.Reasons for this can be not measured or imprecisely measured covariates that make groups differ with respect to their default probabilities.Correlation among binary responsesLet Y i=n ij=1R ij R ij=1success0otherwiseP(R ij=1)=p i⇒V ar(Y i)=n ij=1V ar(R ij)p i(1−p i)+n ij=1k=jCov(R ij,R ik)=0=n i p i(1−p i)=binomial varianceY i has not a binomial distribution.Examples:-same patient is observed over time-all units are from the same family or litter(cluster effects)Modeling of variability among successprobabilitiesWilliams(1982)Y i=Number of successes in n i trials with random success probability v i, i=1,...,nAssume E(v i)=p i V ar(v i)=φp i(1−p i),φ≥0unknown scale parameter.Note:V ar(v i)=0if p i=0or1v i∈(0,1)is unobserved or latent random variableConditional expectation and variance of Y i:E(Y i|v i)=n i v iV ar(Y i|v i)=n i v i(1−v i)SinceE(Y)=E X(E(Y|X))V ar(Y)=E X(V ar(Y|X))+V ar X(E(Y|X)), the unconditional expectation and variance isE(Y i)=E vi (E(Y i|v i))=E vi(n i v i)=n i p iV ar(Y i)=E vi (n i v i(1−v i))+V ar vi(n i v i)=n i[E vi (v i)−E vi(v2i)]+n2iφp i(1−p i)=n i(p i−φp i(1−p i)−p2i)+n2iφp i(1−p i) =n i p i(1−p i)[1+(n i−1)φ]Remarks-φ=0⇒no overdispersion-φ>0⇒overdispersion if n i>1-n i=1(Bernoulli data)⇒no information aboutφavailable,this model is not usefulModelling of correlation among the binaryresponsesY i=n ij=1R ij,R ij=1success0otherwiseP(R ij=1)=p i⇒E(Y i)=n i p ibut Cor(R ij,R ik)=δk=j⇒Cov(R ij,R ik)=δV ar(R ij)V ar(R ik)=δp i(1−p i)⇒V ar(Y i)=n ij=1V ar(R ij)+n ij=1k=jCov(R ij,R ik)=n i p i(1−p i)+n i(n i−1)[δp i(1−p i)] =n i p i(1−p i)[1+(n i−1)δ]Remarks-δ=0⇒no overdispersion-δ>0⇒overdispersion if n i>1δ<0⇒underdispersion.-Since we need1+(n i−1)δ>0δcannot be too small.For n i→∞⇒δ≥0.-Unconditional mean and variance are the same ifδ≥0for both approaches, therefore we cannot distinguish between both approachesEstimation ofφY i|v i∼bin(n i,v i)E(v i)=p i V ar(v i)=φp i(1−p i)i=1,...,g Special case n i=n∀iV ar(Y i)=np i(1−p i)[1+(n−1)φ]σ2heterogenity factorOne can show thatE(χ2)≈(g−p)[1+(n−1)φ]=(g−p)σ2where p=number of parameters in the largest model to be considered andχ2=gi=1(y i−nˆp i)2nˆp i(1−ˆp i).⇒ˆσ2=χ2g−p⇒ˆφ=ˆσ2−1n−1Estimation ofβremains the sameAnalysis of deviance when variability among the success probabilities are presentmodel df deviance covariates1ν1D1x i1,...,x iν12ν2D2x i1,...,x iν1,x i(ν1+1),...,x iν20ν0D0x i1,...,x iνFor Y i|v i∼bin(n i,v i),i=1,...,g.Since E(χ2)≈σ2(g−p)we expectχ2a∼σ2χ2g−p and D a∼χ2a∼σ2χ2g−p χ2Statistic distribution⇒(D1−D2)/(ν2−ν1)D0/ν0a∼χ2ν2−ν1χ2νa∼Fν2−ν1,ν0→no change to ordinary caseEstimated standard errors in overdispersed modelsse0(ˆβj),se(ˆβj)=ˆσ·wherese0(ˆβj)=estimated standard error in the model without overdispersion This holds since V ar(Y i)=σ2n i p i(1−p i)and in both cases we have EY i=p i.Beta-Binomial models v i=latent success probability∈(0,1)v i∼Beta(a i,b i)f(v i)=1B(a i,b i)v a i−1i(1−v i)b i−1,a i,b i>0densityB(a,b)=1x a−1(1−x)b−1dx−Beta functionE(v i)=a ia i+b i=:p iV ar(v i)=a i b i(a i+b i)2(a i+b i+1)=p i(1−p i)/[a i+b i+1]=p i(1−p i)τiτi:=1a i+b i+1If a i>1,b i>1∀i we have unimodality and V ar(v i)<p i(1−p i)13.Ifτi=τ,the beta binomial model is equivalent to the model with variability among success probabilities withφ=τ<13(⇒more restrictive).(Marginal)likelihoodl(β)=ni=11f(y i|v i)f(v i)dv i=ni=1n iy i1B(a i,b i)v y ii(1−v i)n i−y i v a i−1i(1−v i)b i−1dv iwhere p i=e x t iβ/(1+e x t iβ)p i=a ia i+b i=ni=1n iy iB(y i+a i,n i−y i+b i)B(a i,b i)needs to be maximized to determine MLE ofβ. Remark:no standard software existsRandom effects in logistic regression Let v i=latent success probability with E(v i)=p ilogv i1−v i=x tiβ+δi“random effect”δi measures missing or measured imprecisely covariates.When an intercept is included we can assume E(δi)=0.Further assumeδi i.i.d.with V ar(δi)=σ2δLet Z i i.i.d.with E(Z i)=0and V ar(Z i)=1⇒δi D=γZ i withγ=σ2δ≥0Thereforelogv i1−v i=x tiβ+γZ iRemark:this model can also be used for binary regression dataEstimation in logistic regression with randomeffectsIf Z i∼N(0,1)i.i.d.the joint likelihood forβ,γ,Z i is given byL(β,γ,Z)=ni=1n iy iv y ii(1−v i)n i−y i=ni=1n iy iexp{x t iβ+γZ i}y i[1+exp{x t iβ+γZ i}]n ip+1+n parametersToo many parameters,therefore maximize marginal likelihood L(β,γ):=R nL(β,γ,Z)f(Z)d Z=ni=1n iy i∞−∞exp{x t iβ+γZ i}y i[1+exp{x t iβ+γZ i}]n i1√2πe−12Z2i dZ iThis can only be determined numerically.One approach is to use a Gauss-Hermite approximation given by∞−∞f(u)e−u2du≈mj=1c j f(s j)for known c j and s j(see tables in Abramowitz and Stegun(1972)). m≈20is often sufficient.Remarks for using random effects-no standard software for maximization-one can also use a non normal random effect-extension to several random effects are possible.Maximization over high dim.integrals might require Markov Chain Monte Carlo(MCMC)methods -random effects might be correlated in time or space,when time series or spatial data considered.ReferencesAbramowitz,M.and I. A.Stegun(1972).Handbook of mathematical functions with formulas,graphs,and mathematical tables.10th printing, with corr.John Wiley&Sons.Collett,D.(2003).Modelling binary data(2nd edition).London:Chapman &Hall.Williams,D.(1982).Extra binomial variation in logistic linear models.Applied Statistics31,144–148.。