Tuesday, November 1, 2011

IBM's Watson bests Harvard, MIT; New challenge ahead in medicine

By Lori Valigra, Mass High Tech correspondent

Watson, IBM Corp.’s 90-server computer, beat Harvard Business School and MIT Sloan School of Management teams handily in a Jeopardy! match Monday afternoon, at times answering questions so quickly it left the best and the brightest contestants speechless. But IBM has much broader goals for the machine, including as an aid in medical diagnosis.

In introducing Watson to the packed auditorium at Harvard Business School, Bernard Meyerson, IBM’s vice president of innovation, asked the students to imagine having an assistant that is very smart, thinks very fast, works all the time, and doesn’t ever complain. Yet, four years ago, “the odds of this working were pretty much zero,” he said.

“This exceeded our expectations in how it stimulated people’s imaginations,” David Ferrucci, an IBM fellow and Watson’s principal investigator, said of the reaction after the company focused for four years to get Watson ready for a televised Jeopardy! match last Feb. 14 against Brad Rutter and Ken Jennings, the greatest champions in the show’s history.

But moving Watson beyond an ability to play chess, which has a finite, mathematically well-defined search space, and on to analyze human language, which is grounded in human cognition, was a tough challenge. The automatic question answering in Jeopardy!, not too different from the battery of questions a physician asks a human, required the computer to handle a broad/open domain and complex language, and to respond with high precision, accurate confidence, and high speed, Ferrucci said.

“Massive parallelism is a key enabler,” he said. “The computer generates many hypothesis, collects a wide range of evidence, and balances the combined confidences of more than 100 different algorithms.” To move from only 45 percent precision in December 2006 to a high of about 95 percent in November of 2010, IBM ran more than 8,000 documented experiments over four years.

There were growing pains early on. For example, when asked to respond to a question in the category New York Times headlines, “An exclamation point was warranted for the end of this! in 1918,” instead of saying “World War I,” Watson responded with “a sentence.”

But with a barrage of simulated Jeopardy! games, including 170 at IBM Research’s headquarters hosted by Todd Crain, who also took on Alex Trebek’s role at the business school challenge, the computer learned from its mistakes. “That’s about 11,000 individual clues,” Crain said of the mock games held among former jeopardy contestants and Watson. The computer learned from natural language content, gaining confidence that its answers were correct.

“To apply this technology to healthcare, we need to provide evidence about why the answer has confidence,” said Ferrucci. “For diagnostics, we need to gather and score evidence, such as patient background and symptoms, and weigh it.”

A big difference from the Jeopardy! queries is that with the game show, the computer only needs to look at one question at a time. “In healthcare you must input a huge graph of different factors,” he said. “We’ll have to tackle the multidimensional input problem. The data we have now are very optimistic, but there’s a lot of work to do.”

The ability of computers to learn from users has changed fundamentally over the past several decades, said Alfred Spector, vice president of research at Google, during a morning symposium at MIT on Watson and the future of technology.

Rodney Brooks, MIT professor emeritus, said four things are needed to solve issues with robotics: the robots must have the visual object recognition of a two-year-old, the spoken language recognition of a four-year-old, the manual dexterity of a six-year-old, and the social understanding of an eight-year-old.

“Machines will get better,” Brooks said when answering a question about the “singularity movement,” the creation of a super intelligence that some think will center on computers. “What’s starting to happen is bringing machines into our bodies, including cochlear implants, to change the way we hear.” Future retinal implants might also help humans see infrared and navigate better through traffic. “We’ll merge and couple machines to us for the foreseeable future,” he added.

Brooks also is the founder of both iRobot and Heartland Robotics. Heartland, still in stealth mode in Boston’s Innovation District, to date has raised $32 million and has 40 employees, Brooks said. The company is funded by Highland Capital Partners, Sigma Partners, Charles River Ventures, and Bezos Expeditions, the personal investment company of Jeff Bezos. When asked by Mass High Tech when the company would emerge from stealth mode, Brooks declined to comment.

Google’s Spector said he believes that in the future, people will work with machines to do things neither could do alone.

But in the meantime, the Harvard and MIT students saw how “human” their computer rival could be. In the final category of Jeopardy!, Americana, they were asked, “Finding the spot for this memorial caused its creator to say, ‘America will march along that skyline.’” Watson, taking the logical approach, bet the full amount its nearest competitor, Harvard, could bet plus $1, to win the game with $53,601. Harvard followed with $42,399, though if it had bet the entire amount allowed it would have come in at $53,600. MIT was last with $100.