Research Fugue: Measuring Power in Political Campaigns

I’ve been working on a project inspired by the Center for Investigative Reporting and moderated by Kaggle.  I used a network analysis of the movement of money between campaign committees to measure the extent to which different campaigns and different committees were more or less independent, controlling, or broadly influential.  It turns out that corporations have the most broadly influential committees while the most seasoned congressional candidates are the most independent.  However, when you look at the committees that are the most controlling or dependent, things get a bit interesting.  You can download the report and raw results at Influence, Control, Dependent, and Independent.  The code will be up soon.

Doing Program Evaluation Scientifically

I was inspired to write this post after reflecting on James Boutin’s series of posts critiquing the construction and use of data in schools.  There are a lot of ways to screw up evaluations, beginning with misguided initial theories, terrible instrument design, and inept analysis and interpretation.  In this post, I’m not going to tell you all of the ways you can fail and how to succeed.  There are too many for a single post.  Instead, I want to provide the big picture process for doing evaluation scientifically so that you know what you should be getting into when you decide to evaluate.

Evaluation has two components – assessing the causal processes and developing the monitoring system (i.e. benchmarks) to continually assess them.  The causal assessment tells you what about your program and what about your operating environment are influencing your outcomes.  It allows you to say something like, “participation in our interview-skills training program increases the probability of employment by 25%, but the lack of access to public transportation decreases our clients’ probability by 30%.”  The benchmarks allow you to keep track of these influential variables and outcomes and detect any changes or problems with the program.  They allow you to say “over the past year, 50 clients have participated in our interview skills training, but 40 did not have access to public transportation.”  These two pieces of information can play a very influential role in getting city government to expand train or bus routes in your direction or increased funding to pay for bus passes.

My suggestion for a general strategy is to perform a causal analysis once every five or ten years and use the findings to select which benchmarks to track.  This 5-10 year interval is a heuristic.  Some programs operate in very dynamic environments that change quickly relative to other programs.  The more dynamic your environment and the more changes you make to your program, the more often you will have to redo the causal analysis.  In the example above, a new bus line might change the interview program dynamics in several, indirect ways: more clients may come from new areas changing group dynamics, while better access to other resources like a public library or health facilities may improve the job chances of participants but not because of your program.

Causal Evaluation:   Assessing causal relationships is not only the most important part of evaluation, but also the most difficult and most susceptible to bias, misinterpretation, and generally terrible research.  That is why I strongly advise hiring an expert, typically someone who has at least a master’s level training in appropriate research methodologies.  Causal inference involves the highest standards of social sciences research and requires some of the most sophisticated methods we’ve developed (which is why I describe this approach as doing evaluation “scientifically”).  In essence, I suggest paying the $10,000-$50,000 (or more for larger, more complex programs and organizations) once every five to ten years to hire a well-qualified contract researcher or consultant.  My earlier post “Researching With Nonprofits” goes a bit into what this process might be like.  Even better would be to hire one full-time, but I won’t get into the difficulties with financing operating costs.

The most important part of putting the causal evaluation together is the program logic model (for entrepreneurs, this is why you must make one).  Writing out the logic model gives you an explicit understanding of what you believe are the most important processes determining your program’s outcomes and is the starting point for designing the analysis.  Depending on how much data you can gather and to what extent you’re able to randomly select clients to participate in programs, you can expect several waves of data collection or possibly one big one.  Large amounts of data allow for several sophisticated analyses that provide evidence for causal inference.  Small datasets require multiple measures over time to both gather enough data and add temporal variables that help support causal inference.  So, if you’re a small organization or the program is small, you can expect waves of data collection lasting for a period of time determined by the turnover in your program.

So what do you get for your investment?  It depends on the results.  If the study fails to find any significant causal connections and there’s nothing wrong with the data, then a full program review is in order since your program logic model has not received empirical validation.  This is the difference between benchmarks and a causal analysis and why benchmarks are not useful in themselves.  For the interview skills program example, benchmarks would say “40 clients used the service and 30 received a job offer.”  Great, right?  Nope.  The causal analysis concludes that those 30 people would have gotten those jobs without the training.

Benchmarks tell you what’s happening.  The causal analysis can tell you whether you should take credit for it.  The overall goal then is to get the causal part right and then ride on the results for as long as the causal dynamics remain stable.

Benchmarking: If the study succeeds in isolating key causal relationships, then those variables become benchmarks.  To go back to the interview skills program example, if you find that, say, access to transportation, client’s education-level, or involvement in other programs all affect the probability of receiving a job offer, then you collect that information, put it into a spreadsheet, and monitor the changes.  So, if the rate of job offers decreases, you can look and find that your client base in the last cycle was less educated or less involved in the rest of your programs.  Thus, you can say that the program is working with more disadvantaged clients and that you need to do more to get clients involved in other programs.   Hopefully, you can see how this might inspire confidence among your staff and board and encourage donors to open their wallets.

Long-Term Planning:  The basic feature of planning evaluations over time is understanding the dynamics in your environment.  As mentioned above, programs not only have their own dynamics which may change over time, but they also operate within dynamic environments, the causal processes of which will change.  I see three indicators of when a new causal analysis might be necessary.  First, front-line staff and program managers can recognize when dynamics are changing.  Changes in client demographic, new complaints about new issues, or decreasing contact with potential employers can each indicate new dynamics entering the program.  Second, changes in benchmarks can indicate underlying changes in the causal dynamic.  For example, in the interview skills program, if job offers decline, and none of the other measures change correspondingly, it might be time to do another causal analysis.  Finally, dynamics will likely change when you substantively alter your programs.  If you redesign your program to include resume writing or professional writing, dynamics associated with writing like immigration status, race, and education will likely influence how well clients write in your programs and, if the writing component has an impact, the rate of job offers.

Lastly, I would like to take note of current national and sector-level governments, organizations, and thinkers pushing for accountability.  While I believe that data-informed program development and evaluation is the way to go, there isn’t a one-size fits all approach to developing good data and the capacity of organizations to do their own high-standard evaluations represents probably the single biggest barrier to accountability.  Anyone can do research, but to do good research by social scientific standards requires specific training in hypothesis testing, data collection design, and data analysis.  If the accountability movement wants to succeed, it needs to develop the financial and technical resources necessary for organizations to develop this capacity.

The Diminishing Power of the Public, Part 1: Nonprofits as Privatization

This is the first in a series of posts on privatization, the decline of public power, and its implications for democracy and the provision of public and social goods.

A common argument among globalization’s flattening earth theorists is the assertion that state power is being eclipsed by capital mobility, international governmental organizations, immigration, and innovations in transportation and communication.  Here, I want to walk through a counter-argument I’m thinking about.  Historically speaking, state autonomy was actually diminished by democratization.  The more proper question is whether or not public power, engendered by democratic processes and public accountability, is diminishing.  I argue that public power is significantly diminishing, at least in the U.S., and being replaced by a multitude of private powers.  The major forms of this privatization are the outsourcing of responsibility for the provision of public and social goods, the encroachment of private organizations on these goods’ provision, and the privatization of public funds.  In this first part, I want to introduce the question of the declining power of the public and elaborate my first argument: that the provision of public and social goods are being outsourced to private corporations, particularly nonprofits.

First, there’s an ambiguity in the idea of state power.  For globalization researchers, the decline in state power is the declining ability of the state to determine its own policies.  The primary driver for many is global capital flight in which, if states choose anti-capitalist policies, multinational corporations will pick up and move.  Hence, states are forced to dismantle welfare, minimize taxation, and deregulate.  While I would agree that state policy is being influenced by global capital markets, I believe that this conception of state power as policy autonomy obscures what state autonomy actually is.  I argue that states generally are less and less autonomous the more democratic they become.  Democratic states are significantly less autonomous because they are fundamentally beholden to the voters, interest groups, and other public groups that shape elections, policy making, and program implementation.  In essence, the decline of state autonomy has already happened for democracies.

The more pertinent change in state power over the past four decades, best exemplified by the U.S., is the increasingly private control of state money and programmatic responsibility.  This is a broader definition of privatization, which typically refers to governments contracting public enterprises like waste management and parking meters out to for-profit companies.  I define privatization as the private control and responsibility for public resources and programs.  Of course, privatization comes with political overtones and I do not mean to take sides as to whether these trends are better or worse for providing public and social goods.  I only mean to hypothesize about its relationship to public power.

Nonprofits as Privatization:  Prime examples of private responsibility for public programs are nonprofits and traditional privatization initiatives.  Some may be surprised to consider nonprofits as a form of privatization, but they are, in fact, privately-operated corporations (that’s what the “c” in 501(c)(3) stands for).  What is categorically significant about this form of privatization is that the implementation of publicly determined programs is not democratically accountable in the same ways as public programs.  Charter schools are a perfect example of the nonprofit form of privatization.  We elect the school boards who oversee our public school systems.  We do not elect the CEO’s who run charter school management corporations.  Some may think this is a specious distinction since charters are overseen by school boards or other state offices (hence they are still “public”).  But, two important differences should be noted.  First, charter schools are granted exemptions from some of the (democratically chosen) rules and regulations governing public schools.  Secondly, the oversight process is at arms length compared with traditional public schools.

The potential implications of nonprofit privatization are surely more numerous than I’ve come up with, but here are some key points.  First, this privatization likely leads to more innovation, at minimum because of sheer organizational diversity and competition for funding.  This diversity cuts both ways, in that some organizations will be much less effective and potentially harmful while others wildly successful.  The key is the competitive mechanisms which ensure that the ineffective fail and the effective survive.  This gets me to my second implication.  The arms-length relationship between democratic oversight and program implementation problematizes the oversight process because inspection and grant reporting, rather than direct management and public reporting, ensure compliance.  While direct management is no panacea for good governance (think state-run institutions for people with mental illness), an annual inspection has little hope of doing better.  This, I believe is the source for the accountability movement in the third sector.

Third, it allows public programs to tap into a broader range of private resources, particularly foundations (this is more apparent in social services like homeless shelters and services for people with developmental disabilities, than education).  The access to private wealth for public and social programs is a double-edged sword.  On the one hand, the depth of private, philanthropic pocketbooks is enormous.  While there are some policy areas that have long thrived on public and private funding (health, education, research, the arts), other areas like mental illness, job re-training, and homelessness have much more fragmented funding histories that have been positively transformed through the development of the third sector.  On the other hand, it has enabled the retrenchment of the state and the decline in public funding for publicly initiated programs.  Access to private resources did not necessarily cause state budgets to continue to be scaled back, but the ability of social and public services to access private wealth has certainly prevented widespread failure in the nonprofit marketplace in the face of declining public funding.

Finally, this privatization may have shifted the onus of civic engagement into professionalized volunteerism and under-informed philanthropy, rather than political action or democratic civic organizing.  This point goes back to the shift in public provision of services from benevolent associations (like the Elks) to nonprofits.  Before the post-WWII era, public and civic resources circulated through communities via politically active civic groups with regular meetings and democratically elected leadership.  There was a marriage of long-term civic engagement, political activism, and community self-help.  Those days are long gone, replaced by short-term, hyper-circumscribed volunteerism in the professional machinery of an albeit virtuously intending corporation.  Individual philanthropy, rather than being donations to your civic group’s democratically-controlled community pot, are determined by friendship networks (“the ask”), entertainment (galas, concerts, and the like), and emotional appeals.  This represents an information poor market driven by social convenience and an appealing narrative, rather than long-term social relationships, systematic knowledge, and democratic control over the use of donor funds.  It should come as no surprise that nonprofit leaders like Sean Stannard-Stockton and nouveau-riche philanthropists like Bill Gates and Pierre Omidyar are so interested in treating philanthropy as a form of investment.  There is wide-spread concern that the philanthropic marketplace is driven by emotions and convenience (and institutionalized traditions among old-school foundations) rather than impact.  As for volunteers and donors, they’ll have to get their democratic community elsewhere.

In conclusion, the increasing amount of private control over public resources and responsibilities, which I’ve broadened to include nonprofits, has significant, if morally ambiguous, consequences.  This shift, broadly speaking, represents a significant decline in the power of the public to control the provision of public and social services.  This nonprofit form of privatization is not, as some may argue, a capitalist take-over of the public sector because the nonprofit sector is categorically not capitalistic (though it is a marketplace).  Other forms of the declining power of the public, however, are capitalistic as I explain in the next post on the encroachment of private enterprises on public services.

Researching With Nonprofits

There seems to be a growing integration of higher education with real world experience.  While internships are the most historically common form, other types of university-community collaboration are emerging with success.  Take campusCATALYST for example.  They are a graduate student mentored, undergraduate consulting outfit for nonprofits.  Their recent projects include a fundraising and marketing plan for an at-risk youth program and designing a product line to provide an independent line of income.  Consulting projects like these are gaining in popularity in business schools and mirror an important part of professional academics.   But, there’s also a growing number of programs for students in the social sciences to develop their research skills by helping nonprofits.  I wanted to share a bit of my experience researching with nonprofits to illustrate how these programs work and what value they can provide to nonprofits and students.

As an undergraduate, I was a part of the Public Service Research Program (PSRP) at Appalachian State University.  App State, like a number of other universities, imported the program from Stanford’s Public Service Scholars Program.  Basically, I spent a year, under the mentorship of a professor, working with a volunteer office to figure out why so few men volunteered there.  The staff and I put together a couple of hypotheses and created a survey to test them.  It turned out that men and women responded strongly to recruiters of the same gender.  Thus, since the office only had a few men, it was more difficult to recruit more men in proportion to women.

Currently, I volunteer my time working on program evaluation with the Neighborhood Writing Alliance.  We set up some basic outcomes we thought were important, given their mission statement and my knowledge of social science research, wrote up a survey, and analyzed the results.  Now, the NWA knows how engagement in their writing work shops connect with their writers’ satisfaction with the program; how engaged writers are in other organizations, their local communities, and politics; and writers’ perceptions of fairness, trust, and self-confidence.  All of this goes into their grant-writing, reports to their board, and program development.

These experiences illustrate the positive potential of combining social scientific education with nonprofits’ research needs.  As a student, these projects have challenged me to put my understanding of social trends and processes to the test.  The most important lesson I learned was getting the input of the people I worked with.  Social scientists do not have a monopoly on knowledge.  Nonprofit staff have good ideas about what’s going on in their organizations and their input is crucial to creating realistic hypotheses and data collection methods.  On the academic side, these experiences have pushed me to explain plainly what sociology is and says and learn about different areas of research like volunteerism and, thanks to an NWA intern, a niche on assessing community empowerment at the boundary between sustainability and psychology.  Finally, and most practically, it taught me to manage a complicated, multi-stakeholder research project.

For nonprofits, the benefits can be just as enormous.  Social scientific methods and research are at the heart of the growing demand for program evaluation and impact assessment.  Further, I’ve found that nonprofits have an incredible range of interesting and very relevant questions from staff training and retention to local food security and health trends.  A good student can do the background research and collaborate in constructing a workable research strategy with the result being a broader, research-based perspective on the problem and, hopefully, an answer (research is never guaranteed to provide actionable results).

Given that 85% of undergraduates in the social sciences go on to work in nonprofits, corporations, and the government; instituting a PSRP kind of program, I believe, not only pushes students’ engagement with the practice of social research, but prepares them to actually use it in their lives after university.  As for nonprofits, such a broad engagement would, among other things, help cover the accountability funding gap where grant-makers ask for evaluations, but won’t pay for the overhead cost of a researcher.

Finally, if you’re a nonprofit looking to find a research volunteer or intern, contact the nearest university’s civic engagement or volunteer office and see if they have a such a program.  If not, encourage them to look into one.  At the very least, they will post your opening and channel an interested student your way.

DonorsChoose Supplement Part 3: Market Corrections

Note: In preparation for the results announcement by DonorsChoose, this series is meant to carve up different issues raised by my work on the DonorsChoose Data and address them directly and more fully.  You can find the original announcement and report at Predicting Success on DonorsChoose.org.

If, based on my findings, we believe that there are some deserving projects that are being unwarrantably disadvantaged, by say teacher gender or metropolitan location or even the state of origin, there are a couple of ways we can use the algorithm to change the dynamics of the market and test the efficacy of those changes.  My philosophy behind this is that, given that the DonorChoose market is biased towards urban schools, for example, if we don’t believe urban schools are any more deserving than suburban or rural schools (see my discussion on deservingness for why this might be true), then I would call that systematic under-valuation in the market.  Using the algorithm, we can test potential correctives for that.

The first intervention was actually suggested to me by Jonathan Eyler-Werve.  He suggested that search pages could weight the search results based on whether they were urban/suburban/rural or by state, for example, such that under-valued projects could be found earlier.  Technically speaking, this random sort would be weighted, such that more rural and suburban projects, randomly selected from those returned by a user’s search, would show up earlier.  So, say that you’re looking to help out a music project that’s coming down to the wire.  You might not care whether it’s urban or suburban, but, as things stand now, the higher number of urban projects in the system means that roughly 60% of the project’s you’ll see will be urban.  You’ll more likely donate to an urban school just by sheer roll of the dice.  With this weighted, random sort, the search results will balance out the proportion of urban, suburban, and rural projects.  Of course, this would not apply to searches that explicitly ask for urban, suburban, or rural projects.  Testing the impact of these corrections would involve re-running the analysis that produced this model on the post-implementation rates of success and seeing whether the significance of the urban/suburban/rural variables decreased.  If the significance decreases, then the bias has decreased.

Another form of market correction, and one which I mentioned in the report, would be allowing donors to see a project’s chances of success or sort their results by them.  This directly informs donors of the value given to these projects by the market and let’s donors decide if the project is really deserving of a 30% chance.  Thus, a donor could look at two similar projects, like two music programs in Chicago, and know that one has a 60% chance of success and the other an 80% chance.  If the donor thinks the first one is actually more deserving, they might be more motivated to donate to it to try and help its chances.  They may even start a giving page around it.  This approach is a donor-driven market correction in which donor’s can use their own set of preferences to determine if the 60% project is really less deserving than the 80% project.

Monitoring the effect of this implementation would involve re-running the model after this has been implemented and testing any changes in the probability of projects.  Thus, if the original algorithm predicted a 60% probability of success and the post-implementation data shows some projects going to 80%, we can see which variables in those projects correlate with the increase in probability.  If we find, for example, that projects posted by female teachers increase in probability, then we can infer that donors are correcting for the existing gender bias.  The same goes for any variable measured.

Finally, offline strategic initiatives can be developed to target under-valued projects.  For example, a foundation focusing on rural development may be very interested in trying to build support on DonorsChoose for rural projects.  Most importantly, this research provides justification for this strategy, in that rural schools are less likely to reach project completion.  Thus, such a foundation might be convinced to offer matching funds to rural projects or distribute gift cards to rural areas to raise awareness of DonorsChoose and build the rural donor pool.  The same goes for under-engaged states.  I’m not sure what the retention rate of gift cards is, though it could easily be figured out from the data provided for this competition.  In the case of a matching funds initiative, assessing the impact would involve the first method mentioned above, seeing whether significance of the rural variable decreased during the fund period.  As for recruiting new donors through gift cards, not only can we assess how many people used the gift cards, but, using the second method mentioned above, we can estimate the lasting effect of the initiative.

If you have any other ideas of how this prediction algorithm might be used to improve the DonorsChoose market, please feel free to discuss it in the comments section below.

DonorsChoose Supplement Part 2: Deservingness

Note: In preparation for the results announcement by DonorsChoose, this series is meant to carve up different issues raised by my work on the DonorsChoose Data and address them directly and more fully.  You can find the original announcement and report at Predicting Success on DonorsChoose.org.

In the report, I argue that donors choose what they perceive to be the most deserving projects.  I explicated this a little in my first supplement, but want to refocus on it here to talk about what deservingness may be and how we might be able to reflect on exactly what deservingness should be.  First though, I dispel the idea of interpreting the data as a straight-forward measure of deservingness.

I found a number of categories that negatively affect the probability of success.  For example, suburban and rural schools, and projects for non-high school students are less likely to reach project completion.  This does not necessarily mean that donors perceive urban high schools as more deserving.  It may be due to the fact that donors are more likely to be parents of urban high school children donating to their child’s school.  From the data provided by DonorsChoose, this hypothesis is untestable.  Thus, my talking about the findings in terms of deservingness is based on an informed, but not fully justified assumption that the findings do in fact, to some extent, come from donor’s notions of deservingness.  Essentially, in light of the fact that there are alternative explanations of the findings, I still believe that some donors see urban high schools as more deserving.

The findings of the report allow us to ask ourselves who is more deserving.  Obviously, the effect of gender, urban/suburban/rural, and subject area are fertile grounds for discussion.  Gender presents a unique challenge, first and foremost, because there is a real question of whether or not donor’s consciously see male teachers as more deserving.  While there is plenty of literature on the glass escalator effect for men in typically female dominated workplaces, whether or not this applies to DonorsChoose is not testable with the data.  It could be that male teachers write different types of essays that donors find more persuasive or ask for different types of resources within given categories.  In essence, the gender variable could be capturing the effect of something else that I didn’t take into account.  This becomes less likely the more, other variables I control for (and I controlled for a lot of them).  Given the previous research on this issue and the number of controls I introduced, I am inclined to believe that the gender effect is actually attributable to a gender bias.  The question then is, is that okay?  Are we comfortable with the idea that male teachers are more deserving than their female counterparts?  I would say not.  How could we fix that?  I’ll elaborate that in the next supplement on strategies for correcting valuations in the DonorsChoose market.

The salience of the urban/suburban/rural distinction on project success raises two issues.  The first harkens back to my earlier discussion of alternative interpretations to deservingness.  I’m inclined to believe that there are more urban donors (by simple dint of population).  The challenge then is to reach more rural and suburban donors.  I’m also inclined to believe that non-local donors,  those who are not fully tied to donating to their local schools, are biased towards urban schools because of the recent education reform movement’s emphasis on urban school reform.  Urban schools are the locus of heated debate about educational innovation (i.e. the charter school movement) and public school reform (i.e. the accountability movement).  On DonorsChoose, I believe this translates into the perception among some donors that urban schools are especially wracked with problems and that their students are especially disadvantaged.  (I also believe this translates into the preference for charter schools that I found).  Educational research is not my area, so I’m not well-versed in the research on whether or not urban schools are more or less disadvantaged than suburban or rural schools (when you control for local property values, parents education, and the like).  Thus, I can’t make an informed opinion as to whether this perception of urban deservingness is justified.  But, if you buy my argument, now that we have evidence that urban schools are preferred among some donors, we can find out if that perception is warranted, and if necessary, try to change it.

Finally, the differential preferences for donations to different subject areas ranked very roughly from most preferred to least: arts, sports, STEM, social science, special education, applied, and language and writing.  There are ways to obtain a more accurate ranking (though this one is likely to be pretty close), but I only want to use this as an example.  If we assume deservingness, we could read these rankings as reflecting a sentiment in which donors believe some subjects are already disproportionately under-funded (i.e. arts and after-school athletics programs), but still adhere to our present emphasis on the value of math and science education (the STEM fields).  But, now that we know how these subjects stand, we can ask, should more be done to promote special education or, applied or language projects?  This would be a good selling point for getting a foundation to fund a campaign for literature projects.  In addition, we can ask ourselves whether or not we should be distinguishing by subject area.  Is a piano for a music program more deserving than microscopes for a biology class?  I believe that when a donor searches through DonorsChoose, they are more likely to say to themselves, “I want to fund the arts because the government won’t,” than say,  “The piano is more important than the microscope.”  In essence, I believe the bias for different subject areas is applied at the beginning, rather than at the final choice between different concrete options.  However, the outcome is the same, the piano is more likely to be funded than the microscope and we have to decide if that’s a worthwhile value judgement.

I’ve gone through just some of the findings to introduce their implications for DonorsChoose and for us as a philanthropic society judging schools and programs based on an idea of deservingness.  In my next post, I’ll discuss some ways we can use the prediction algorithm to fix any biases in the market that we find unjustifiable.

DonorsChoose Summary Part 1: Sociological Insights

Note: In preparation for the results announcement by DonorsChoose, this series is meant to carve up different issues raised by my work on the DonorsChoose Data and address them directly and more fully.  You can find the original announcement and report at Predicting Success on DonorsChoose.org.

Creating the algorithm involved running standard statistical analyses which are typically used for description.  In the original report, I only mention the sociological readings of the data I analyzed.  Here, I want to do a fuller sociological interpretation.

If, like most people, you don’t know how to read a regression table (the appendices in the report), first focus on the sign of the numbers (are they positive or negative).   The price variable is negative, thus, the probability of success decreases for every additional dollar requested (it’s actually a logged price, so it’s not that simple, but that’s the very basic interpretation).  The male variable is positive, thus male teachers have a higher probability of success on DonorsChoose than female teachers.    As for the actual numbers, they are log odds (a result of the particular statistical model).  To make them make sense, just exponentiate them (e^number).  Thus, the male .18360 becomes 1.20, meaning that men are 1.20 times more likely than women to succeed.

Maximizing Impact? The number of students impacted, whether the project would be used by future students, and if the project would be used for essential instruction all had no effect on project success.  These seem to be the most direct measures of impact and it makes me wonder how concerned donors are with maximizing their impact.  In the report, I summarize donors’ as motivated by notions of deservingness, not impact, because of this.  However, this could be due to the fact that this information is at the very bottom of project pages and, except for essential/enrichment, you cannot search by these features.  Donors might care, but not be aware that this information is available or might not be able to use it effectively.

Causation and Correlation: In describing my results in the paper, I talk ambiguously about the link between the significant variables and the probability of success.  While I maintain that, if a teacher posts a proposal for books, they will have a higher actual probability than if that same teacher requested technology, this could be read as a causal claim though it is not.  I do however, claim that receiving matching funds or donations through a giving page do cause an increase in probability.  The reasoning is fundamentally theoretical.  These matching funds and giving pages are real actions that, according to the statistics,  significantly affect the chances of success.

The Power of Giving Pages: The incredible significance of giving pages (increasing the likelihood of success by more than 50% in some cases) is very interesting theoretically.  I believe that active giving pages are sites of highly engaged donors who not only fund projects (as opposed to the 90% of visitors to DonorsChoose who do not), but actively recruit and motivate other donors.  This direct appeal by a highly engaged collection of people is how social movements operate and what sociologists mean when we talk about the power of social networks to create social capital and motivate collective action.

Market Effectiveness: Most often, people talk about markets in terms of efficiency.  I believe the first measure of a market should be effectiveness.  The first question should be whether or not a market is providing the goods it is supposed to to the participants who want them.  For DonorsChoose, are they connecting donors to the projects that they want to fund?  My belief is that they are pretty effective based on the fact that most of their searchable terms (poverty level, subject area, etc.) were also significant variables.  A significant number of donors care whether they’re donating to a music program or a soccer team.  They care whether the money will fund a class library or a new smart board.  Despite the insignificance of the impact measures (which may be due more to their lack of integration in the search system), DonorsChoose allows donors to choose projects based on the dimensions they care about.  There could be a reverse causation here, a market making dimension where donors come to see subject area or resource type as important only because DonorsChoose offers them as potential search options.  This does not negate the inference however that they are significant, only where the preferences come from.

Market Change: In the first version of my report, I stated that the positivity of the Year variable meant that the DonorsChoose market was becoming more effective over time because the probability of success increased every year.  I retract that, because Year was actually not so generally positive across the models.  There are a couple of possible explanations for the declining measures, following simple supply and demand.  I think the most probable explanation would be an increasing number of projects posted.  With more projects being posted every year, if the number of donors does not increase proportionately as well, then the overall rate of success is likely to go down.  The supply of donor funds has not kept up with project demand.  I find the opposite side of supply/demand, a decreasing amount of donor funds, less probable given the growth of activity on DonorsChoose, though it could be driven by the current economic environment.

These are some of the surely multitudinous implications that can be drawn from my research here.  Please feel free to ask questions or posit your own interpretation of what’s going on in the comments section below.

The Revolutionary Potential of Social Enterprise

Over the past forty years, we’ve become accustomed to the legal, economic, organizational, and moral distinctions between for profit and non-profit enterprises.  For-profits, like McDonald’s and Proctor and Gamble, provide goods and services in exchange for money which then gets distributed to workers, owners, investors, and the like.  Non-profits, like Feeding America or the Salvation Army, provide free or nominally priced services to those who likely could not afford them otherwise.  Their income is through the generosity of individuals, philanthropists, and governments/taxpayers and spent to pay employees and subsidize these services’ cost to clients.  Money left over may be socked away for a rainy day or invested into expansion.  There are no investors or shareholders in the for-profit sense (nonprofits still take out bank loans and other lines of interest-bearing credit).  This institutionalized distinction between for-profit and nonprofit, I believe, is becoming incoherent and social enterprise demonstrates the revolutionary potential.

First, a hypothetical.  What if McDonald’s wanted to become a nonprofit, what would it take?  Under the IRS definition of a 501(c)(3) it would need to be operated for an exempt purpose whose income does not “inure” to controlling individuals or shareholders.  I may be wrong, but a $1 McDouble seems like a charitable price for feeding those in poverty.  The key difference I believe is the “inuring” of profit.  So, McDonald’s could re-privatize its ownership, change certain lines of investment capital, rework its executive benefits and viola! (I ignore the political limits because they are less relevant to my point here).  In fact, Panera Bread now has a self-sustaining nonprofit arm integrated with its restaurants.

What social enterprise has demonstrated is that you don’t have to give away things for free to be a nonprofit.  This is an inversion of what business students say – “You can do well and do good.”  This is the revolutionary potential of social enterprise.  Many for-profit business could qualify as charitable.  Many charities could turn a profit and still be providing a social good.  The line between socially beneficial and business is being recognized as transient because there are not many activities that could not be considered “charitable.”  The distinction we’re accustomed to is the artifact of a custom whereby services for those in need were organized by nonprofits.  For-profit entrepreneurs and executives are only now realizing that, for the most part, the only thing preventing them from being a nonprofit is “inuring” profit.  The biggest disadvantage to filing as a nonprofit is the access to investment capital.  Hence comes the L3C designation.

L3C stands for a low-profit limited liability company.  Essentially, they are for-profit companies that, for providing a social good or service, can accept return-bearing investments from traditional nonprofit sources, like foundations and governments, (called “program related investments“) but cannot have profit as a “significant purpose.”  While the initial rationale for the L3C was to enable would-be nonprofit organizations to gain more (traditionally capitalist-like) investment, it can go both ways.  Would be for-profit companies (who happen to provide a charitable service) can adopt the L3C as a sign of their ethical commitment to consumers. (What operations and rules define profit as a non-”significant purpose” is left wide-open (4th paragraph), hence I only assert that the L3C is an ethical signal, rather than an operational restriction)

Capitalism would be completely different if the business community recognized that what many of them produce could be considered charitable in a legal sense.  Imagine a world where McDonald’s, Walmart, and Coke are nonprofits (or L3C’s. I want to address the question of capital access in a later discussion).  They already provide cheap food, clothing, potable water, and other essential items to billions of people.  Would you buy a burger from a nonprofit McDonald’s or a for-profit Burger King?  Would Walmart clothes still be made in sweatshops?  How silly does the ideology of shareholder value sound now?

There is no clear push for this radical of a restructuring of capitalism.  But, the emergence of new energizing strategies in business, social enterprise in particular, indicates that Americans are seriously challenging our assumptions that conducting business is ultimately just for profit and providing services to those in need is just charity.  How far we could take it seems pretty revolutionary.

Predicting Success on DonorsChoose

Using statistical regression methods, I constructed an algorithm that predicts the probability of success for almost every project on DonorsChoose.org within 2-7%.  This is the report on how I constructed the prediction models and evaluated the results along with a couple of ways that it can be implemented in the DC system: Predicting Project Completion on DonorsChoose.org (pdf).  The video below is my introduction to the algorithm and what it could do for DC.  I was hoping to have an app developed as a part of this analysis, so that people could at least see predictions generated in real time, but it’s still a little above my skill level and I needed to focus on improving the algorithm.  If you want to calculate your own probabilities, see the appendices in the report for the necessary values and variables along with an example.  I also have a python script that will compute probabilities for live projects on DonorsChoose.org if you’re savvy like that.  I have a zipped copy of it along with an unusual dependency package (nltk_contrib) that is used to compute the readability scores.

Business, Civics, and Entrepreneurship

My research and engagement in the Chicago community seem to be converging on a nexus embedding business and civil society, or the economic and nonprofit sectors.  In the academy, this research is emerging around the concepts of social capital/social networks, social movements and mobilization, and social innovation/entrepreneurship.  In the Chicago civic scene, this convergence is occurring around the tech community and tech startups, the infusion of technological innovation in local organizations, and the first waves of post-dot com, CSR business students.  While I doubt this nexus is wholly novel, I want to pull out the strands to offer up some possible connections between what social scientists are thinking and what community activists are doing.

Social Capital, Networks, and Chicago’s Tech Community:  The basic idea behind social capital and social networks was originally worked out by James Coleman.  While network research originated in the 1940′s, the concept of social capital crystallized thought about how the structure of social networks explains innovation, diffusion, norm enforcement, and resource flows.  Social capital essentially refers to the particular assets a person or group of people have due to the way they’re connected with each other and to other groups.  For example, two groups with overlapping membership are more likely to share information and innovation than groups that are not connected (called “structural folds“)

New technologies are ubiquitously useful in organizations (from marketing, communications, organizational learning, etc.),  social life (keeping up with friends, meeting new people, finding a spouse, etc.), and community organizing (collaboration, advocacy, new forms of protest, etc.).  In theory, this should mean that tech human capital (an individual’s technical know-how) should be almost infinitely transferable across networks. Basically, a tech person could be a contributing member of any network.  Thus, the continual rate of innovation in the tech sector should encourage a high rate of social change in all sectors as everyone builds a website, creates a Facebook page, sends their donors newsletters, and so on.

The tech sector in Chicago is a dynamic, surprisingly dense social network of career professionals, entrepreneurs, and technophiles whose substantive interests extend beyond Perl and BuddyPress to monitoring global corruption, increasing connections within the nonprofit field, and offering people collective coupons.  This diversity and the transferability of tech human capital indicate a relatively diffuse network structure connecting techies across these different groups, yet the widespread implementation and experimentation with technologies, new businesses, and initiatives indicates that this diffuse network is still highly mobilized.  This gets to my second thread.

Social Movements and Technological Implementation: The social movement literature in the social sciences was revived by the civil rights moment which deeply affected a generation of scholars.  The results of this inquiry have been a highly profitable framework centered around the concepts of resource mobilization, framing, and opportunity structures.  More recently, social movement outcomes, non-state-directed movements, individual motivation, and emotions have been added to the mix (among others).  This toolkit of concepts and empirical research has been so profitable in fact, that the concepts have attracted the attention of sociologists in other sub-fields, particularly economic sociology who are applying the concepts to business innovation and organizational change.

To innovators and civically engaged business professionals, this direction should come as no surprise.  Accomplishing change, whether implementing new supply chain controls or obtaining subsidies for a new plant from the state, very often seems like a social movement.  More generally, Rao’s book Market Rebels reveals some two hundred pages of economic mobilizations from computer clubs to AAA (the automobile association) that brought computers and cars to every home in the America.  Being a leader, even a business leader, means mobilizing resources, framing your strategy, and seizing opportunities.

In Chicago, and I’ll stick to the tech community, there is a social movement pushing technological innovations to anyone who will listen.  The goals range from online donor best practices and consulting to re-envisioning the Magnificent Mile with augmented reality.  The outcomes have been a mass experimentation with new tech mobilizations (think the Pepsi Challenge and its role in brand-building).  Doubtless, Chicago is not special among major metropolitan areas in this mobilization.  In our consciousness, new technology is an experiment leading inexorably to some future that we create and that we must be part of.  This experiment knows no sector (remember infinite transferability) and always has a theory of what the world could and should look like.

Social Entrepreneurship and New MBA’s: Social entrepreneurship is a new concept, maybe two decades old, that’s become integrated into a new domain along with social innovation (maybe a century old?) and social enterprise (maybe fifty years old).  This domain, both academically and practically, integrates other concepts that cross sectors like privatization, public entrepreneurship, public-private partnership, and venture philanthropy.  The academic-practical integration has probably occurred because the social entrepreneurship/innovation domain has been actively developed in business schools.  While I cannot verify this genealogy yet (I’m working on it), it very likely that the domain resonated with the corporate social responsibility movement that was already resonating in 1990′s, ethically vacuous business schools.  Then, the tech bubble burst.

To demonstrate this conjunction, let me point out the Stanford Center for Social Innovation. It was created in the late 90′s in Stanford’s business school as a joint project between business faculty and Silicon Valley moguls.  Silicon Valley was also the origin of Venture Philanthropy (though Rockefeller III officially coined the term).  The tech boom and bust of the late 90′s was the pinnacle of the first public wave of experiments with new technology.  It brought with it new values (tech savviness, innovation, being the next Silicon Valley) and a new vision of reality (a mature Web 1.0) that picked up on older values and practices and propelled them forward.  The burst bubble was a trauma, like 9/11, that altered not only our collective vision of reality, but of our potential.  New technology was going to bring us a new world, but there is no panacea.

Corporate social responsibility took a serious hold in business school communities (particularly in business ethics) and theories of the role of business in the 90′s.  This renewed discussion of corporate responsibilities beyond shareholder value created fertile ground for the growing discussion over social innovation and social entrepreneurship which emerged as a distinct domain in the mid and late 90′s.  In the early 2000′s, the ever-expanding role of new technologies defining world-changing innovation along with a new discussion over responsibility seems to have created an existential revisioning of doing business which matched innovation and responsibility.  Hence, Harvard started sending MBA’s to developing countries to work with social enterprises, businesses providing essential goods like clean water and sustainable irrigation.  Assuming a two-year program, these socially conscious, tech MBA’s have been graduating since 2004 or 2005.

In Chicago, these MBAs have teamed up with older generations of social entrepreneurs and civic leaders and created a number of civic groups mirroring these new trends, often with names that use the phrases social entrepreneurs, social enterprise, and social innovation.  (you can typically find your local group through Meetup).  These economic activists are creating footholds of practice and organization that connects like-minded activists (business, techie, and otherwise) to pursue socially-conscious innovations.  In this, we see an example of how all of this ties together – technology, social networks, and social movements.  New tech is a mobilizing force that has become diffuse throughout our social networks, yet still remains coherent.  The business of technology, embedded in these civic networks and creating new ones, are a distinct driving force linking the civic and economic in a mutually interested, mutually active experimentation oriented by a socially conscious (yet still mutually suspicious) vision of the future.

Conclusion: The academic and practical convergence of the civic and economic through a number of topics and new groups, while not thoroughly historically novel, will drive change in both.  Business may have turned a corner in the marriage of social and innovation.  Civic organizations may be turning a corner in adapting to the extent and level of innovation and uncertainty.  Academics will likely soon face the challenge of synthesizing social capital/network and social movement research with the ever-present interaction between businesses and civic organizations now most apparent in the problems surrounding social entrepreneurship, social enterprise, and the new technological revolution.

As for the government, this is America, the most a-governmental of all advanced countries.  The government, I believe, does play a role around this.  But, it appears that the business/civic-government interface is occurring on different grounds (remember privatization and public-private partnerships).  There are thematic similarities, but I believe these are very different domains of debate.  There are significant implications for the role (or lack thereof) of government, but I’ll reserve that for other, possible articles.