In search of: Best practice for code repositories?
I was asked by a colleague about organized efforts within the economics community to develop or support repositories of code for research. Her experience was with the astrophysics world which apparently has several and she was wondering what could be learned from another academic community. So I asked a non-random sample of technical economists with whom I work, and then expanded the question to cover all of social sciences and posed the question to the IASSIST community.
In a nutshell, the answer seems to be “nope, nothing organized across the profession” – even with the profession very broadly defined. The general consensus for both the economics world and the more general social science community was that there was some chaos mixed with a little schizophrenia. I was told there are there are instances of such repositories, but they were described to me as “isolated attempts” such as this one by Volker Wieland: http://www.macromodelbase.com/. Some folks mentioned repositories that were package or language based such as R modules or SAS code from the SAS-L list or online at sascommunity.org.
Many people pointed out that there are more repositories being associated with journals so that authors can (or are required to) submit their data and code when submitting a paper for publication. Several responses touched on this issue of replication, which is the impetus for most journal requirements, including one that pointed out a “replication archive” at Yale (http://isps.yale.edu/research/data). I was also pointed to an interested paper that questions whether such archives promote replicable research (http://www.pages.drexel.edu/~bdm25/cje.pdf) but that’s a discussion for another post.
By far, the most common reference I received was for the repositories associated with RePEc (Research Papers in Economics) which offers a broad range of services to the economic research community. There you’ll find the IDEAS site (http://ideas.repec.org/) and the QM&RBC site with code for Dynamic General Equilibrium models (http://dge.repec.org/) both run by the St. Louis Fed.
I also heard from support folks who had tried to build a code repository for their departments and were disappointed by the lack of enthusiasm for the project. The general consensus is that economists would love to leverage other people’s code but don’t want to give away their proprietary models. They should know there is no such thing as a free lunch!
I did hear that project specific repositories were found to be useful but I think of those as collaboration tools rather than a dissemination platform. That said, one economist did end his email to me with the following plea: “lots of authors provide code on their websites, but there is no authoritative host. Will you start one please?”