MapReduce becoming ubiquitous ? => Collaborative Map-Reduce in the Browser
As an academic exercise, myself and a colleague at Sun are currently implementing a MapReduce client for iPhone which has a server component, and a number of other clients we’ve written in various programming languages (currently Python, Ruby and Perl). Our hope is to statistically prove iPhone and similar devices can be useful in a larger MapReduce exercise to augment beefier hardware in the data center,
specifically for processing massive amounts of textual information for the purpose of search and data mining.
Assuming you have enough mobile device toting participants of course…
In the course of our implementation we’ve come across a few other interesting ideas in this
space, such as this one which creates a MapReduce client in Javascript to farm compute to website visitors, which is very neat.
Also, Amazon recently announced elastic MapReduce.
More and more lately I’ve been sensing that MapReduce is seeping into software developer’s common knowledge, which is a good thing.
There is a class of problem for which it’s extremely applicable, but like anything, it is NOT a silver bullet, nor
the solution to every massive scale data processing problem. Alas, as is too common, it seems to be the next big shiny thing…
After its completion, I’ll provide additional details and possibly some source code on our iPhone MapReduce project.