Tuesday, July 7, 2009

Wolfram Alpha Review

Wolfram Alpha was released with great fanfare recently. Their overarching objective is very broad:

"Wolfram|Alpha is the first step in an ambitious, long-term project to make all systematic knowledge immediately computable by anyone". [http://www.wolframalpha.com/]

This can be compared to Google’s corporate statement:

"To organize the world's information and make it universally accessible and useful." [http://www.google.com/corporate/]

Many of the reviews deal with making side-by-side comparisons with results from different search engines. Even if Wolfram|Alpha succeeds in returning better results for a limited subset of data, it will fail to make a relevant fraction of “all systematic knowledge immediately computable” because doing so requires expert humans:

“… as the physicist sat, exhausted, immersed in the minutiae of food science. On the computer screen before him were raw tables of information from the U.S. Department of Agriculture, containing data on 7,000 foods, from blackberries to beef. He and a four-person team were "curating" the data, readying it for a new kind of online search.” [http://www.technologyreview.com/web/22834/]

Remember the old Yahoo where everything was placed into categories by humans? This is a much simpler problem than data “curation” described above. In fact, the act of organizing data and preparing it for systematic computation is a significant part of what a researcher does. Every field of science has their own way of organizing data and this requires specialists to organize and validate it.

The fundamental problem with Wolfram|Alpha is that it requires so much human intervention. What will happen a few years from now when all of the USDA’s data tables are updated or their data base format changes? Wolfram|Alpha needs to find and pay a team of experts to update and validate the new data.

My assessment of this fundamental flaw in Wolfram|Alpha’s approach is primarily based on what I know is required to deal with satellite orbit data. You can find satellite orbit data displayed in a comprehensive and clear manner at Wolfram|Alpha [http://www01.wolframalpha.com/input/?i=NOAA+15], which on the surface seems much more usable than what is available elsewhere [Start at http://sscweb.gsfc.nasa.gov/cgi-bin/sscweb/Query.cgi and then work your way through the menu to get equivalent data]. When a new satellite becomes available, who will update Wolfram|Alpha? And why would they be motivated to do it? If I have a detailed technical question about the implementation of an orbit calculation, who do I ask? If, 5-10 years in the future we see 100s of small satellites launched per year, who is going to update the Wofram|Alpha database?

The most basic problem is that if they are going to keep up with every special data set from every scientific subspecialty that has data that is computable, they need a community that will contribute. Wolfram’s approach is not community oriented, however. If I have an article about physics that could fit in http://scienceworld.wolfram.com/ or Wikipedia, I would choose Wikipedia. Many would justify this choice by saying that Wolfram Research is a corporation, and why should I give free contributions to a corporation if I don’t get anything in return? I have a more pragmatic reason; Wolfram Research is a corporation, and corporations come and go. When they go, their intellectual assets tend to follow. Releasing intellectual assets from a company that is dying or being swallowed requires money, which is not typically plentiful for a company in such a state.

- Bob Weigel

No comments:

Post a Comment