ITA Annual Fall Meeting September 2014 The International Technology Alliance in Network and Information Sciences The Analysis Game TA6/P4 David Mott, IBM UK The "Analysis Game" Used by Prof Don Shemanski, PSU, as part of a analysts training course Similar to experiments being considered byJon Bakdash Includes a elimination style of puzzle that requires logical reasoning Includes components that require interpretation We formulated this in CE as an informal evaluation our technologies in assisting human problem solving Seven girls live on the same street. Each of the girls has a pet and lives in a different colored house. Using the following clues, determine who lives in each house and the pet that lives with them. The monkey lives between the dog and the lizard. Cassie does not own a cuddly pet but she lives next door to someone who does and that person lives in a green house. A Solution Domain model girl, house, pet owns, lives with next to, two doors down if ( the girl G lives in the house H and owns the pet P ) then ( the pet P lives in the house H ). Problem solving strategy Reasoning by elimination If only one place left for Brenda to go then that is where she is Negative concepts to represent constraints the person Jane cannot live with the person Brenda Use of descriptive terms to handle UNKNOWN entities: brendas_pet the person Brenda owns the pet brendas_pet and lives in the house brendas_house. Rules: if ( the pet P is a furry animal ) then ( the pet P cannot be the pet cassies_pet ). Identification of unknowns and propagation of properties the pet brendas_pet is the same as the pet monkey the pet brendas_pet lives in the house orange_house -> the pet monkey lives in the house orange_house Form of variable binding Working with Word Documents All information is contained in a Microsoft Word document Context, problem statement, domain model, comments, answers Styles define the rules, concepts and facts Macros send information to ERG and CE reasoning system, and receive information into tables Use Microsoft Word document to express CE queries in an answer grid Journey to the solution Most important discovery was that the way that the man and machine could collaborate Did not want the machine just to provide the answer as difficult to understand why More natural, human, way is to engage in an iterative journey of discovery involving Partial formulation Running of the reasoning Review of results so far Construction of new ways to conceptualise, or revisions to existing concepts Added components to CE in order to model Problem Solving itself formulation reasoning Domain Model & Givens Inferences Problem Statement Solution Initial Concepts Revised Concepts Inconsistencies Assumptions New Givens Rationale Interpretation needed Cuddly pets Is a bunny cuddly? A dog? A fish? We chose only those that are definitely cuddly Furry pets More scientifically validated, so all pets assigned a value Two doors down Does this mean one door in between or two doors in between? Left and right There are no relations that talk about left or right, so there are two possible solutions, which are mirror images At one point there were two possibilities for the location of an object, but it did not matter, so there isnt really a choice But in reality might need to know which actual house? Some of these are deliberately introduced into the class Could make an assumption? Misformulations Originally the houses were used to define the locations of the girls and pets the house h1 is next to the house h2. But this was confusing as the houses were themselves variable in their location! Thus we defined a plot that has a fixed location, with spatial relations: the plot p1 is a left end plot. the plot p2 is immediately right of the plot p1. Other things could be located on plots: the girl Brenda is located in the plot p3. the house orange_house is located on the plot p3. This gave a more stable mental model of space! In a more complex problem (zebra problem) some constraints seemed to be rules: The Old Gold smoker owns snails. Originally this was formulated as: if ( the person P smokes the cigarette brand Old Gold ) then ( the person P owns the pet snails ). [ and vice versa] But this did not make the person visible on the answer grid, so was in effect invisible to the human, so we formulated it as an actor: the person old_gold_smoker owns the pet snails. This was much easier to understand, and worked just as well! This is creating new concepts DURING problem solving Maintaining consistency of rules During the exploration process, new rules were added, BUT Some rules were missed, some duplicated It isdifficult to maintain consistency In the more difficult (zebra) puzzle, matrices of rules were constructed in Word: where A = the person X drinks the beverage Y, B = the person X is located on the plot Y etc Based on logical principles (all possible conbinations of A, B, C, -A, -B, and C) covered in table Allowed duplicates and missing rules to be detected Making choices In the zebra puzzle we got to a point where no new information seemed to be available, but for many variables there were just two possibilities System found the Englishman could either drink orange juice or milk. Manually tried orange juice and continued the analysis System found inconsistency, no possible choice for another variable. We then tried milk, and continued the analysis This led to only one choice for each variable, and hence the solution Therefore this is the ONLY solution We could have done this by an assumption, and by having the system automatically explore the choices (but we didnt) It would be worth trying this out It would be worth working with the argumentation technology to see how this could be automated Rationale Rationale was used (to some extent) to check apparent errors of reasoning Now we are reviewing rationale to see if an explanation of inferences can be given Simple rationale is easy: But complex rationale, needs story telling the furry animal cat is located on the left end plot p1. Rationale Graph Proof Table Complex rationale This may well not be! Sameas We can explode the reasoning We are trying to explain why Brenda is located in plot 3. Brenda is located on plot 3 because she is in the orange house, which is located on plot 3. the orange house is on plot 3 because it is two doors down from the green house which is known to be on plot 1. Brenda is in the orange house because her pet is in the orange house, and pets live with their owners. Her pet is in the orange house because her pet is located on plot 3, which is the orange house. This step seems redundant as the colour seems irrelevant to Brenda being on plot 3.. Better to say.. Brenda is located on plot 3 because her pet is located in plot 3, and the pet live with their owners. In order to work out that brendas pet is on plot 3 we must first work out the locations of two other pets, the fish and the dog. First the dog. We know that Cassie lives next to the green house, which is on plot 1, the end house. So cassie must be on plot 2 and so must her pet. We also know that cassie is allergic to fur, so cassies pet cannot be the dog, and so the dog cannot be on plot 2. But we know that the dog is next to the orange house, which is on plot 3, so the dog must be on either plot 2 or plot 4. the neighbours of plot p3. But we just discovered that the dog is not on plot 2, so it must be on plot 4. Now for the fish. We know that Fiona owns the fish, since she owns a pet with the same first letter of the name, and there is only one pet that starts with f. We also known that Fiona lives in the pink house, so the fish lives in the pink house too. But we have also discovered that the pink house is on plot 5. This is because the pink house is two doors down from the parrot, and the parrot is on plot 7, because we know the parrot lives in the purple house, and we know that the purple house is on plot 7. Since plot 7 is an end plot, two doors down from plot 7 can only be plot 5, and since the pink house is two doors down from plot 7 it must be on plot 5. As the fish lives in the pink house, it must be on plot 5 as well. So the dog is on plot 4 and the fish is on plot 5. We have some more information about Brendas pet. Since Fiona owns the fish, Brendas pet cannot be the fish, and so Brendas pet cannot be in the same plot as the fish. We also know that Brenda lives next to the dog, so Brendas pet lives next to the dog too.. Thus we have the dog on plot 4, the fish on plot 5, Brendas pet being next to the dog, but not being on plot 5 because the fish is on plot 5. So Brendas pet must be on plot 3. qed And try an explanation! Conclusion We have managed to solve some quite complex and hard problems using a combination of CE and human reasoning Microsoft Word has been very useful as a means of managing and organising the information We are discovering some principles for how human and machine could collaborate using CE as the common language We have added new ways to formulate problems in CE An informal example for evaluation of CE technology, similar to that considered by Job Bakdash Interest has been shown by: Prof Shemanski David Alberts Jon Bakdash CNL14 conference Acknowledgement This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.