It’s the data stupid (“The Fourth Paradigm”)
I stumbled across this New York Times article in my RSS feeds this AM regarding a Microsoft research endeavor/book titled “The Fourth Paradigm”.
I’ve skimmed TFP so far, and have highlighted a few sections I want to read in depth, and it looks quite good.
Essentially, it explores today’s data volumes in the science realm relative to those who must make sense of it all. Make no mistake, there’s a deluge. Now that I’ve been back in the science/quasi-academic realm at a NOAA data center for a while, I can certainly attest to there being more data available than people know what to do with. In fact, some scientists even have to make determinations of which level 0 data to delete in some cases, because they simply lack the space to store it all. This is a scary fact, since processing theories and algorithms evolve constantly, and if you no longer have your level 0 data, you can’t refine or evolve your models as effectively (if at all). This is bad for science in the long run, and this book hits on these topics, among others.
It’s available for free in PDF form from Microsoft, so what are you waiting for ?