How a summer school made me even more confused

Arjen De WitBy Arjen de Wit / Reading Time: 4 minutes

 

As every motivated PhD I’m happy to attend conferences and courses every once in a while. Thanks to a grant from the VU-Graduate School of Social Sciences I was able to attend a four-week seminar on quantitative methods by the ICPSR Summer Program in Ann Arbor, Michigan. I hoped that this program would solve the causality issues I had in my research question. The contrary is true: it raised even more questions.

  1. You should use panel data! When using survey data it is better to study the same respondents over time in order to test whether changes in X are followed by changes in Y.This can make you a bit more certain of causal relations. So I used data from the Giving in the Netherlands Panel Study (GINPS) including 1,902 people surveyed over multiple years. In this study participants report their donations to 17 of the largest charities in the Netherlands, like World Nature Fund or the Salvation Army. From annual reports we know how much those organizations receive from different governmental subsidies. This allows me to compute how subsidies to an organization in a certain year are correlated with private donations in the following year.

But then the confusion came in. These are only 17 charities, are my results the same when I exclude one of these organizations? Can we expect effects to be the same for international aid organizations as for charities working in the field of health care?

  1. You should include fixed effects! Fixed effects account for variables that don’t change over time, which allows you to look only at the effect of variables that do change over time. For example, some organizations receive both more subsidies and more donations just because they are bigger organizations. An analysis that includes fixed effects for organizations rules out the effect of organization size.

But there is the confusion again. Should I use fixed effects for individuals or for organizations? A person’s gender or other individual characteristics can disturb the effect, as well as an organization’s size, sector or age. Or should I use fixed effects for each unique combination of individual and organization?

  1. You should do Tobit regression! Because most people don’t donate to all 17 organizations in the sample there are a lot of cases scoring 0 on the dependent variable. Linear regression is not appropriate in that case. Tobit regression, I was told, includes both the likelihood of scoring higher than 0 and the linear distribution of valid donations in one estimation.

Confusion! Are the decision whether or not you donate and the decision on how much you donate the same thing? Are non-donors motivated by the same considerations as donors?

  1. You shouldn’t do Tobit with fixed effects! The ‘incidental parameters problem’ means that fixed effects can make the estimation of a binary outcome variable (donating or not donating) biased. In other words, Tobit and fixed effects are not always good friends.

So shouldn’t I use Tobit? Or shouldn’t I do fixed effects? Or is there another way to account for this problem?

I went to the ICPSR summer school to get answers on the causality issues I had with my research question. Do higher government subsidies lead to lower charitable donations, is it the other way around, or is there another variable that causes both subsidies and donations? The summer school provided answers but those answers confused me even further. More difficult methods come with more difficult problems, and that’s how researchers keep on struggling with their analyses until they come up with answers that are the best they can get to.
_____________

Arjen de Wit is a PhD candidate at the Center for Philanthropic Studies, where his research concerns the question to what extent government support affects volunteering and charitable giving. He also works for ProDemos, House for Democracy and the Rule of Law, and writes for his personal blog www.arjendewit.nl.

One thought on “How a summer school made me even more confused

  1. Pingback: Doubting with the stars – why doubt is actually constructive for your project | Socializing Science

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s