House Money Effects in Public Good Experiments Comment

House Money Effects in Public Good Experiments: CommentbyGlenn W. Harrison†April 2006Forthcoming, Experimental Economics, 2006Abstract. We reconsider evidence from experiments that claim to show that using“house money” in standard public goods experiments has no effect on behavior. Weshow that it does have an effect when one examines the data using appropriatestatistical methods that consider individual-level responses and account for the errorstructure of the panel data.Keywords: Public goods, experimental economicsJEL Codes: D7, C92† Department of Economics, College of Business Administration, University of Central Florida, USA. E-mail: gharrison@. I am grateful for comments from two referees and an editor. All data and statistical code are available for public access at the ExLab Digital Library located at .Clark [2002] considers the effect of providing “house money” on voluntary contributions to a public good. He concludes that it has no effect. If one examines the individual-level data using appropriate panel estimation techniques, one draws a different inference from his experiments: that there is a significant effect on behavior from using house money. The experiments he designed were ideal for the purpose, and were well executed. The deeper problem is that unconditional non-parametric tests are inappropriate for this inferential task. The statistical analysis of data generated with repetitions of the task needs to account for possible correlation of responses at the individual level and over time. What statistical methods should be used if one does not want to assume some specific model of learning? This note suggests some answers to this larger methodological question, encouraging the use of panel econometric methods and discouraging the use of unconditional non-parametric methods, while addressing the substantive question posed by Clark [2002] about the effect of house money.1. The Experimental DesignThe experimental design is simple. Fifteen subjects in each session participated in 10 rounds of the standard public goods game. The subjects were randomly assigned to one of 3 groups of 5, so that reputation effects could be mitigated.1 Five sessions with the house money treatment were conducted with 75 subjects, and five sessions with the “own money” treatment were conducted with another 75 subjects drawn from the same population.2All subjects were asked to bring at least $8 to the experiment, and were told they could avoid losing any money with certainty by appropriate strategy choice. The own money treatment involved1 But not eliminated. See Botelho et al. [2005].2 Cherry, Kroll and Shogren [2005] examine similar issues in the context of a a one-round public goods experiment. They find no evidence of a house money effect on the single round of contributions.subjects initially giving this $8 to the experimenter at the start of the experiment, for use in the experiment. The subject could gain as little as 40 cents per round, by contributing all of their 80 token endowment to the group investment and being in a group where nobody else contributed to the group investment fund. Or they could earn as much as $2 per round if everyone contributed to the group investment fund. If everyone followed the dominant strategy, and contributed nothing to the group fund, everyone would earn 80 cents per round. Thus the total earnings over all rounds in the own money treatment could range between $16 and a loss of $4, after netting out the initial $8 stake paid to the experimenter.In the house money treatment no mention was made of the $8 the subject was asked to bring, and earnings over all rounds could range between $4 and $24.Subjects in the own money treatment were actually given an $8 “participation fee” after all decisions were made. This procedure was followed to be consistent with the recruiting announcement, which unfortunately mentioned the expected range of earnings: the subjects were promised that they would be paid “an average amount of $12-$16, but that the actual amount will vary due to decision-making.” More common practice is to avoid mentioning specific, numerical target earnings, since it could lead subjects to have certain expectations as to which strategy to adopt.2. Statistical AnalysisClark [2002] reports the average contributions in each round under each treatment, as shown in Figure 1. Despite some differences, the averages look quite close. To avoid relying on eyeball inferences, he then undertakes a formal statistical analysis of these data (p.228/9):In a strangers design participants have the potential to influence all other participantsin their session over the course of the ten decision rounds. As a result, eachindividual’s contribution may not be independent of others’ decisions for any but thefirst decision round. Thus, to compare overall contributions, each session shall providea single unit of observation, namely the average contribution of fifteen subjects overten rounds. [A Mann-Whitney test and a t-test are then applied to these summarystatistics, viewed as samples of 5 and 5 for each treatment.] (...) Unfortunately, usingsessions as single observations results in a small sample size that would make itdifficult to detect differences even if they did exist. A second approach is to use eachindividual’s contribution decision in the first round, on the basis that it will beindependent of interactions with other subjects. [Again, Mann-Whitney and t-tests areapplied to the samples of 75 and 75 in each treatment.] (...) In sum, the statistical testsconfirm the impression from figure 1 [identical to our Figure 1] that subjects did notdiffer significantly in their contribution towards the public good when the moneybeing used came from their own pockets rather than the experimenter’s.Thus the data analysis does not examine the complete data set, which consists of a panel of 10 observations for each of 150 subjects, assigned randomly to 2 treatments.The first analysis in fact is undertaken on statistics of the actual observations, the session-specific means. It is easy to imagine circumstances in which these statistics do not reflect the effect of the treatments. For example, what if the treatments affect the median but not the mean? Or the variance and not the mean? Or if the mean difference in early rounds is offset by an opposite mean difference in later rounds?The second analysis only looks at the first round of behavior. As Figure 1 shows, the first round is not representative of the difference across all rounds. Nor would we, in general, expect it to be representative.Neither problem derives from the use of simple non-parametric tests, but from the use of inappropriate metrics of evaluation. The basic unit of observation here is the choice by a subject in a particular round, over a number of rounds, and across sessions and a treatment. Any aggregation across rounds and subjects discards information, by definition. And any analysis if only one round similarly discards information. As it happens, however, non-parametric procedures are not able to easily condition on the information needed to examine the complete set of data.33 Pagan and Ullah [1999] offer an excellent review of the landscape of non-parametric and semi-parametric econometrics. They are honest about the strengths and weaknesses of the field (e.g., p. 2, 340-341), and correctly view them as complementary to parametric tools.4One might imagine a number of latent learning processes that would generate such intra-panelcorrelation (e.g., Camerer [2003; ch.6]).5 I am grateful to Jeremy Clark for making his data available. All statistical analyses can be implemented with version 8 (or higher) of Stata , documented in StataCorp [2003a][2003b].Inspection of the raw, individual data provides some important clues about the statisticalissues to be aware of. Figure 2 shows kernel density estimates of the distribution of contributions,pooled over all rounds and sessions. These distributions appear to be similar for the two treatments,but of course the pooling could be masking significant differences. The main feature of the data evident from Table 2 is the “spike” of contributions at 0, which is the free riding prediction.Figure 3 provides an illustration of choices for 10 players drawn from each treatment. These are literally the first 10 players in the data for each treatment, and should not be presumed to represent treatment effects. What is apparent from inspection of Figure 3, and similar charts for all players, is a strong temporal correlation of responses for a given player. It is not apparent, unless one uses spectral glasses, that there is any particular structure to the temporal correlation. Intuitively, one might expect that there would be more variation in earlier rounds, as subjects evaluate their best options, and then less variation in later rounds as subjects settle into a choice that they are happy with. Thus one might expect less (first-order) autocorrelation in the earlier rounds and more in later rounds, suggesting caution with respect to a statistical specification that presumes one autocorrelation for all rounds.4Figure 4 puts these two dimensions of the raw data together: breaking out the subjects that were pure free riders from those making some positive contribution, in both cases over each round.We see the main hypothesis emerge from Figure 4. It appears that the house money had a major effect on the fraction of free riders, and no clear effect on the levels of positive contributions.A panel analysis of these data allows a systematic examination of these possible differences between individual subject responses across the treatments, allowing for various forms ofautocorrelation.5 We first differentiate between zero contributions and positive contributions,suggested by Figures 2 and 4, using a hurdle specification. This posits a minimally parametric latent process of decision-making: that subjects decide if they are going to contribute nothing or something, and then if they decided to contribute something they determine what level their contribution should be at. Thus the treatment of interest, the provision of house money, can have separate effects on either decision.There are several statistical procedures that can be used to control for panel effects associated with the subject (“unobserved individual effects”) and with the passage of time (intra-panel autocorrelation), both of which seem to be important to control for in this context. We use population-averaged estimation methods known as Generalized Estimating Equations (GEE). These methods grew out of the General Linearized Models (GLM) tradition initiated by Nelder and Wedderburn [1972], primarily due to an insight from Liang and Zeger [1986].6 Excellent reviews of GLM and GEE, stressing their connections, can be found in Hardin and Hilbe [2001][2003], respectively.The relationship between the population-average approach and the random effects (and fixed effects) approach is summarized by Hardin and Hilbe [2003; p.49]:There are two classifications of models [...] for addressing the panel structure of data.A population-averaged model is one which includes the within-panel dependence byaveraging effects over all panels. A subject-specific model is one which addresses thewithin-panel dependence by introducing specific panel-level random components.A population-averaged model, also known as a marginal model, is obtainedthrough introducing a parameterization for a panel-level covariance. The panel-levelcovariance (or correlation) is then estimated by averaging across information from allof the panels. A subject-specific model is obtained through the introduction of a paneleffect. While this implies a panel-level covariance, each panel effect is estimated usinginformation only from the specific panel. Fixed-effects and random-effects models aresubject specific.Population-averaged estimates are relatively popular in epidemiology and some areas of operations6 Their insight was that a “working correlation matrix” that characterizes intra-panel dependence can be viewed as a symmetric matrix of ancillary parameters to be estimated as part of the GLM.7In fact, in some well-known special cases the two are formally identical from a numericalperspective. For example, see StataCorp [2003b; p.68/69]. The GEE estimates require, amongst other things,that the mean effects and the correlation structure be correctly specified in order for the estimates to be valid.Under the full set of assumptions needed for the random effects specification, GEE estimates are not the most efficient, but they are likely to be more efficient than pooled estimators that ignore the intra-panel correlation (e.g., Wooldridge [2002; p.487]). On the other hand, GEE does not require the zero-correlation assumption of the random effects specification that is the focus of Hausman test and that is often rejected.Thus it provides a complementary approach.8 Conditional on contributing, average contributions were roughly 32 tokens.research, perhaps due to the earlier popularity of GLM in those fields.In many practical settings the population-averaged estimates will be identical to those obtained with a subject-specific estimation procedure, such as random effects specification.7 However, the interpretation of the coefficients is very different. Consider the effect of a dummy variable for the house money treatment on the probability of making a positive contribution. A subject specific estimate would be interpreted as the effect of a change in the treatment on the subject, holding all other characteristics constant. The population-averaged estimate would instead be interpreted as the effect on an average subject in one treatment compared to the average subject in the other treatment.In fact, for many inferential purposes that is what one is interested in, but the differences must be kept in mind.The results indicate a significant effect of the house money treatment on the propensity to completely free ride, but virtually no effect on the extent of a contribution once one has decided to contribute something. In each case we control for session effects with binary dummy variables for each session, as well as a binary dummy for the treatment of interest. No information on individual demographic characteristics were collected. The house money effect is estimated to reduce theprobability of providing something by 8.2 percentage points, compared to a pooled average of 73%.8This estimated effect is statistically significantly different from zero (p -value = 0.010), and the 95%confidence interval for the estimated effect is between 14.4 and 2.0 percentage points. Intra-panel correlation of the binary choice to contribute or not varies over the life of the experiment. In the9A comparable “hurdle” specification with the one-period data from Cherry, Kroll and Shogren[2005] confirms their conclusion from an ordinary least squares regression on contributions (Table 2, p. 362)that there is no effect of house money in their experiments. I am grateful to Todd Cherry for providing their raw data.10 It also creates a parallel danger that we “let the story speak,” by just telling a convincing and novel story to knit some of the raw data together, dotting the statistical i with some unconditional test that does not evaluate the complete data at the most detailed level of observation.11 For example, the discussion of Davis and Holt [1993; p.525-552], which has influenced generations of experimental economists. The emphasis on non-parametric tests, and concern with simple parametric assumptions, also derives from earlier oral traditions of the Economic Science Association meetings. The Bonn School repeatedly stressed the dangers of naive parametric procedures whenever they were used.earlier rounds it is around 0.5 and 0.6, and is eventually 0.80 between choices in rounds 9 and 10.Conditional contributions are estimated to be only 0.15 of a token lower with house money, and this effect is not statistically significant (p -value = 0.95). Again, there is evidence of an increase in intra-period correlation of conditional contributions, from roughly 0.5 in the earliest rounds to 0.7 and 0.8in later rounds. There is also evidence of large and statistically significant session effects on the decision to contribute at all and the conditional contribution level. These effects may be controlling for unobserved demographics effects correlated with the session.93. ConclusionThere is much to be said for letting the raw data speak. After all, this is one of the reasons that many economists were drawn to experimental methods (Smith [1982; p.928ff.]).10 But weexperimenters are now trying to ask more of our designs, and many of these questions require that we condition on demographics and explicit parametric processes. The sample sizes collected in most experiments do not come close to those needed for randomization to treatment to be statistically reliable, so sample differences in individual demographic characteristics have to be allowed for before we can draw reliable inferences about treatments. This problem becomes particularly acute when interactions between treatments and individual characteristics are allowed. Sadly, the familiar non-parametric and semi-parametric tests that we grew up using 11 do not easily translate into conditionalestimators. This problem is compounded further when the theories that we are using require careful statements of latent behavior if they are to be meaningfully evaluated and tested. In the present case study, an example is the decomposition of behavior between those that contribute nothing and those that contribute something. In other applications we have to be concerned with the stochastic structure of the latent process itself if we are to make proper sense of observed data.12 Inferences in these settings demand that we model behavior at the individual level and account for error structures explicitly. Doing so in the present case demonstrates that there is evidence of a statistically significant house money effect on public good contributions.ReferencesBallinger, T. Parker, and Wilcox, Nathaniel T., “Decisions, Error and Heterogeneity,” Economic Journal, 107, July 1997, 1090-1105.Botelho, Anabela; Harrison, Glenn W.; Pinto, Lígia M. Costa and Rutström, Elisabet E.,“Testing Static Game Theory with Dynamic Experiments: A Case Study of Public Goods,” WorkingPaper05-25, Department of Economics, College of Business Administration, University ofCentral Florida, 2005.Camerer, Colin, Behavioral Game Theory (Princeton, NJ: Princeton University Press, 2003).Cherry, Todd L.; Kroll, Stephan, and Shogren, Jason F., “The Impact of Endowment Heterogeneity and Origin on Public Good Contributions: Evidence from the Lab,” Journal of Economic Behavior & Organization, 57, 2005, 357-365.Clark, Jeremy, “House Money Effects in Public Good Experiments,” Experimental Economics, 5(3), December 2002, 223-231.Davis, Douglas D., and Holt, Charles A., Experimental Economics (Princeton, NJ: Princeton University Press, 1993).Hardin, James, and Hilbe, Joseph, Generalized Linear Models and Extensions (College Station, TX: Stata Corporation, 2001).Hardin, James, and Hilbe, Joseph, Generalized Estimating Equations (Boca Raton: Chapman &12 For example, Ballinger and Wilcox [1997], Hey [2005] and Wilcox [2006].Hall/CRC, 2003).Hey, John D., “Why We Should Not Be Silent About Noise,” Experimental Economics, 8(4), December 2005, 325-345.Liang, K-Y., and Zeger, S.L., “Longitudinal Data Analysis Using Generalized Linear Models,”Biometrika, 73, 1986, 13-22.Nelder, J.A., and Wedderburn, R.W.M., “Generalized Linear Models,” Journal of the Royal Statistical Society – Series A, 135(3), 1972, 370-384.Pagan, Adrian, and Ullah, Aman, Nonparametric Econometrics (New York: Cambridge University Press, 1999).Smith, Vernon L., “Microeconomic Systems As An Experimental Science,” American Economic Review, 72(5), December 1982, 923-955.StataCorp, Stata Statistical Software: Release 8.0 (College Station, TX: Stata Corporation, 2003a). StataCorp, Stata Cross-Sectional Time-Series Reference Manual: Release 8.0 (College Station, TX: Stata Corporation, 2003b).Wilcox, Nathaniel T., “Theories of Learning in Games and Heterogeneity Bias,” Econometrica, 74, 2006 forthcoming.Wooldridge, Jeffrey M., Econometric Analysis of Cross Section and Panel Data (Cambridge, MA: MIT Press, 2002).。