The full disclosure of IBM Watson's answer process

See Watson in jeopardy. The amazing performance of the preliminaries, have you ever thought about what it would be like to see Watson understand the problem and give the answer? Does Watson really have the same ability to think as humans do? Or is it just that Watson is a better search engine than Google?

Now the shell nets to his answer to the first question for example, to explain the whole process of Watson's work.

Moderator: Ken, you are standing at number 1th, please select the question.

Ken: I'm a little nervous about this on the TV show ... "Beauty digs me Up", 200 dollars.

* Note: Chicks Dig Me, meaning the question of female archaeologists.

The topic appears on the big screen, and the host starts to read the question aloud

Kathleen Kenyon ' s excavation of this city mentioned in Joshua shows that the walls had been.

Then Waston began to act.

look at the problem--Identify the text of the problem

Watson photographed the text on the screen through the camera, then made an optical identification, and got the text of the topic stored in words rather than images:

Kathleen Kenyon's excavation of the city mentioned in Joshua shows that the city's walls have been repaired 17 times.

Understanding the problem--extracting the basic information such as the proper nouns in the text

The first basic information to be extracted includes names of people, place names, and time. Waston They are extracted, compared with the database, and then the results of the knowledge classification tag.

In this question, the first thing Watson has to do after reading a question is to extract all the initials in the title and search the database, and the search task will be split into separate tasks by Watson and sent to each processor thread.

A total of three initials in the title, including Kathleen Kenyon These two words together. So Watson would have issued 4 search tasks: Search the database for Kathleen,kenyon,kathleen Kenyon and Joshua respectively.

Search results are as follows:

Kathleen	Names	/
Kenyon	Name of person/university/City	/
Kathleen Kenyon	Names	Archaeologist female
Joshua	Book (chapter) name/Names	The Book of the Bible and Joshua

Analyze the problem--extract the information from the structure of words, such as the part of speech and the relation of Action connection.

Since the proper nouns for search are often more than one label, then Watson will need to help sift through the proper nouns by the nature and relationship of the other words given in the question to get their more precise meaning.

In this step, Watson searches for every word and the word near it, which is a lot of work. But by using conjunctions, adverbs, and sentence structures, and so on, many redundant searches can be reduced.

For example, mentioned,shows,had,been,repaired these 5 words have the possibility of becoming a predicate verb. Data comparisons revealed that had and been were only repaired tenses, so the alternative verbs were reduced to 3. According to the word before and after that, judge that this is a clause, then repaired out of the competition. Finally, judging by the phrase mentioned in Joshua, mentioned is not the predicate verb of the whole sentence. Then we get the grammatical structure of the topic: the subject-predicate structure, and the "shows" is the predicate.

At the same time, it can not be overlooked that the problem of jeopardy is a declarative sentence, is a description of the answer. So through the key word this city, the answer should be a town.

Moreover the topic classifies for "The beautiful woman digs up me", but the question in the dry only Kathleen Kenyon is the female archaeologist, then judges the real meaningful noun should be Kathleen Kenyon (hereafter is called KK).

Mentioned in Joshua explains that the answer should be mentioned in Joshua, and that it can be judged by its categorical label that Joshua should be a book rather than a person.

In addition, through this step, Watson will also get the general information in the topic and the relationship between the answer. Like what:

KK ' s excavation of this city. KK in the answer City did archaeological excavation work

The walls had been repaired the city's walls have been repaired 17 times

KK ... shows ...... 17times walls have been repaired 17 times. This information is from the discovery of KK

In this step, each problem is decomposed into several child problems to solve. If you can't get the answer to a child problem directly, the problem is broken down into several child problems and sometimes you need to introduce a new problem to get the answer to a question. Until you get all the answers.

This is a "fan out" of the structure, can be understood as a "divide and conquer" strategy. In addition, every problem split may have several different approaches, so Watson tries all the different divisions at the same time. In fact, a common problem that contains 20 words may eventually be decomposed into tens of thousands of child problems.

answering questions-searching for their own stored structural and unstructured data

Because the title was "KK in the answer City did archaeological excavation work" This information, so Watson will search its own database of KK data (because it has been organized into a certain structure, so called structural data), see if you can find relevant information. The information of Kathleen Kenyon as shown in the following table:

Gender	Woman
Occupational	Archaeologists
Major achievements	Caesaria (Jericho) city was excavated in 1952-1958. The City of Jerusalem (Jerusalem) was excavated in 1961-1967.

In addition, Watson stores a large amount of encyclopedias, paper documents, and other articles stored in the form of information, and they also have relevant information. But since the data has not been collated and organized (known as unstructured data), Watson must obtain information through a very laborious text search. This process will also use the previous use of the extraction of proper nouns, parts of speech, text structure and so on.

By "The answer city in the Book of Joshua (Joshua)," The clue that the search for the full text of Joshua and determine which nouns are the name of the city is necessary. In addition, KK once published archaeological papers should also be included in the scope of the search.

The Book of Joshua mentions at least 3 cities, Jerusalem, Caesaria and Raj. Of course, one of KK's papers may have mentioned that the walls of Jericho had been repaired 17 times.

The possibility of analyzing pun messages

Because of the fuzziness and two meanings of human language, it is usually necessary for Watson to analyze the meaning of the topic to further increase the correctness of the understanding. This is often done in contrast to the stored, potentially ambiguous, expression libraries in the database. This is the hardest step, and is the least confident step for Watson.

Fortunately, there is no pun in this topic.

the right possibility to judge each answer

Watson uses hundreds of algorithms to evaluate possible answers, including the type and nature of the answer, the timing of the answer, the correct location, the part of speech, the grammatical structure, and so on.

All the cities mentioned in the book of Joshua are the alternative answer database. It mentions at least the three cities of Caesaria, Jerusalem and Raj, and the relevant information is as follows:

City	Related information
Caesaria	KK once discovered Jericho. KK mentioned in a document that the walls of Jericho had been repaired 17 times
Jerusalem	KK once discovered Jerusalem.
Raj did not	Find the records of KK's discovery of Raj.

The result of the judgment that Watson produced:

Caesaria (Jericho): 97%

Jerusalem (JERUSALEM): 42%

Raj (lachish): 7%

The highest correct answer, if the correct degree is higher than 51% to emit signal driver actuator, press the key.

I don't know if you noticed, where Watson is, there is a device on the top of the answer button that is dedicated to the key. Even supercomputers, you need to press the key in the physical world to answer. This should be jeopardy. The producer's request.

The correctness of Jericho is as high as 97%, above 51%. So in the end of the moderator's description of the moment, Watson issued a signal to the button device, press the buzzer switch, grabbed the right to answer.

answer the question-speech synthesis, issue format speech

Due to jeopardy. The rules of the game are given in the form of answers, and answers are given in the form of questions. So Watson needs to answer questions in a format such as "What is xxx" or "Who's is xxx".

Watson judged that Jericho was here a place rather than a name, so the answer should be "What is Jericho." ”。 Then, Watson only needs a speech synthesis engine to convert the text of the answer to voice and play it out.

Source: http://www.guokr.com/watson/article/6643/