Note: In preparation for the results announcement by DonorsChoose, this series is meant to carve up different issues raised by my work on the DonorsChoose Data and address them directly and more fully. You can find the original announcement and report at Predicting Success on DonorsChoose.org.
Creating the algorithm involved running standard statistical analyses which are typically used for description. In the original report, I only mention the sociological readings of the data I analyzed. Here, I want to do a fuller sociological interpretation.
If, like most people, you don’t know how to read a regression table (the appendices in the report), first focus on the sign of the numbers (are they positive or negative). The price variable is negative, thus, the probability of success decreases for every additional dollar requested (it’s actually a logged price, so it’s not that simple, but that’s the very basic interpretation). The male variable is positive, thus male teachers have a higher probability of success on DonorsChoose than female teachers. As for the actual numbers, they are log odds (a result of the particular statistical model). To make them make sense, just exponentiate them (e^number). Thus, the male .18360 becomes 1.20, meaning that men are 1.20 times more likely than women to succeed.
Maximizing Impact? The number of students impacted, whether the project would be used by future students, and if the project would be used for essential instruction all had no effect on project success. These seem to be the most direct measures of impact and it makes me wonder how concerned donors are with maximizing their impact. In the report, I summarize donors’ as motivated by notions of deservingness, not impact, because of this. However, this could be due to the fact that this information is at the very bottom of project pages and, except for essential/enrichment, you cannot search by these features. Donors might care, but not be aware that this information is available or might not be able to use it effectively.
Causation and Correlation: In describing my results in the paper, I talk ambiguously about the link between the significant variables and the probability of success. While I maintain that, if a teacher posts a proposal for books, they will have a higher actual probability than if that same teacher requested technology, this could be read as a causal claim though it is not. I do however, claim that receiving matching funds or donations through a giving page do cause an increase in probability. The reasoning is fundamentally theoretical. These matching funds and giving pages are real actions that, according to the statistics, significantly affect the chances of success.
The Power of Giving Pages: The incredible significance of giving pages (increasing the likelihood of success by more than 50% in some cases) is very interesting theoretically. I believe that active giving pages are sites of highly engaged donors who not only fund projects (as opposed to the 90% of visitors to DonorsChoose who do not), but actively recruit and motivate other donors. This direct appeal by a highly engaged collection of people is how social movements operate and what sociologists mean when we talk about the power of social networks to create social capital and motivate collective action.
Market Effectiveness: Most often, people talk about markets in terms of efficiency. I believe the first measure of a market should be effectiveness. The first question should be whether or not a market is providing the goods it is supposed to to the participants who want them. For DonorsChoose, are they connecting donors to the projects that they want to fund? My belief is that they are pretty effective based on the fact that most of their searchable terms (poverty level, subject area, etc.) were also significant variables. A significant number of donors care whether they’re donating to a music program or a soccer team. They care whether the money will fund a class library or a new smart board. Despite the insignificance of the impact measures (which may be due more to their lack of integration in the search system), DonorsChoose allows donors to choose projects based on the dimensions they care about. There could be a reverse causation here, a market making dimension where donors come to see subject area or resource type as important only because DonorsChoose offers them as potential search options. This does not negate the inference however that they are significant, only where the preferences come from.
Market Change: In the first version of my report, I stated that the positivity of the Year variable meant that the DonorsChoose market was becoming more effective over time because the probability of success increased every year. I retract that, because Year was actually not so generally positive across the models. There are a couple of possible explanations for the declining measures, following simple supply and demand. I think the most probable explanation would be an increasing number of projects posted. With more projects being posted every year, if the number of donors does not increase proportionately as well, then the overall rate of success is likely to go down. The supply of donor funds has not kept up with project demand. I find the opposite side of supply/demand, a decreasing amount of donor funds, less probable given the growth of activity on DonorsChoose, though it could be driven by the current economic environment.
These are some of the surely multitudinous implications that can be drawn from my research here. Please feel free to ask questions or posit your own interpretation of what’s going on in the comments section below.