Like it or not the academic world is slowly (or perhaps not that slowly) moving towards being judged by how highly cited our work is. Actually, it is not a bad measure as if you publish something and nobody ever reads it, or refers to it (i.e. cites it in their own work), then was their any point in publishing that research?
The problem is, of course, that it might takes years (perhaps decades; or even more) for somebody else to recognise the value in a particular piece of research. The other problem is that once scientists know that they are being evaluated in this way, they find ways to maximise their citations (they are typically quite clever people!).
The next research assessment that will be carried out in the UK (called the Research Excellence Framework (REF)) will using citations as a major way to measure the impact of the research that is carried out by individuals, research groups and universities.
If you are interested in the REF take a look at http://www.hefce.ac.uk/Research/ref
At the moment, in the UK, this area is the subject of heated debate in the scientific community and this will be the case for the foreseeable future, especially when you bear in mind that the last research assessment (Research Assessment Exercise (RAE)) has only just been completed (the results were announced in December 2008 – http://www.rae.ac.uk).
In my opinion, there's a huge gap between the software industry and academic software research.
I believe that a lot of academic research papers and source code end up in a closet and never get used in the real world.
Much of that source code is probably useful, but is simply not available.
I believe research sponsors should force academic software research to open source the source code from day one under the BSD license (or possibly the ASL license so the research entity gets credit for its work).
Although I am in favor of a measurement of research papers, I do not think that a citation count in other research papers is optimal. Those papers could be unused in the software industry as well and/or probably written by the same research group. A webpage or download count from trustworthy resources (for example SourceForge for downloads) could be an – although not perfect I admit – alternative.
The huge gap between the software industry and academic software research is in both directions. Not only does the software industry use very little of the research papers, also the research source codes use relatively little (although that is changing faster) of the industry standard tools such as log4j/slf4j, maven, findbugs/pmd/checkstyle, junit, subversion/git, bugzilla/jira, UML, docbook manual, …
This is a personal opinion of course 🙂
I fully agree that there is a wide gap between academic software and commercial software. Part of the problem is that the commercial software have to write robust software and also get it working.
The scientists also have to get the software working, but it does not have to be of commercial strength.
I also agree that there is a wide gap between academia and industry. Both sides have totally different goals. We (scientists) want to publish papers, whereas industry has to deliver solutions whilst keeping an eye on the bottom line.
In fact, one of our research "interests" is in trying to bridge this gap between "theory and practice" and I may blog more on this soon.