SOLVINGSUDOKUPUZZLESWITHTHECOUGAARAGENTARCHITECTUREbyMichaelRayEmeryAthesissubmittedinpartialfulllmentoftherequirementsforthedegreeofMasterofScienceinComputerScienceMONTANASTATEUNIVERSITYBozeman,MontanaJuly2007c COPYRIGHTbyMichaelRayEmery2007AllRightsReservediiAPPROVALofathesissubmittedbyMichaelRayEmeryThisthesishasbeenreadbyeachmemberofthethesiscommitteeandhasbeenfoundtobesatisfactoryregardingcontent, Englishusage, format, citations, biblio-graphic style, and consistency, and is ready for submission to the Division of GraduateEducation.Dr.JohnPaxtonApprovedfortheDepartmentofComputerScienceDr.MichaelOudshoornApprovedfortheDivisionofGraduateEducationDr.CarlA.FoxiiiSTATEMENTOFPERMISSIONTOUSEInpresentingthisthesisinpartial fulllmentoftherequirementsforamastersdegreeatMontanaStateUniversity,IagreethattheLibraryshallmakeitavailabletoborrowersunderrulesoftheLibrary.IfIhaveindicatedmyintentiontocopyrightthisthesisbyincludingacopyrightnoticepage, copyingisallowableonlyforscholarlypurposes, consistentwithfairuse as prescribed in the U. S. Copyright Law. Requests for permission for extendedquotationfromorreproductionof thisthesisinwholeorinpartsmaybegrantedonlybythecopyrightholder.MichaelEmeryJuly,2007ivACKNOWLEDGEMENTSI deeplythankDr. JohnPaxton, myadvisor, for providingdirectiontomyresearch,reviewingmywork,andeditingthisthesis;mywife,Ginny,forherendlesslove, support, encouragement, and thesis editing; SRI International for their exibilityinallowingme totake 2weeks oworktocomplete this thesis; the Meyers forprovidinghotmealsandaplacetostayduringmynal pushforcompletion; andGod,withwhom,allthingsarepossible.vTABLEOFCONTENTSLISTOFTABLES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiLISTOFFIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiABSTRACT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix1. PROBLEMSTATEMENT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. RELATEDWORK. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3IntroductiontoSudoku . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3SolvingSudoku . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3ComputerAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4HumanTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53. COUGAAR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6TheDecisiontoUseCougaar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6AlternativeAgentArchitectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7TheCougaarProblemDomain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7CougaarComponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8TheCommunity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8TheNode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8TheAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Inter-AgentCommunication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104. THEAGENTAPPROACHTOSUDOKU. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11KeyObjects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14TheCellAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Starting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16RollingBack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17TheBoardMasterAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Starting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19DetectingaStalledPuzzle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20viGuessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20RollingBack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21IllustrativeExample. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215. EMPIRICALANALYSIS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26TestSets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27CougaarEnvironment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27ExperimentsonVeriedSudokuPuzzles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Backtracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28DLX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30RandomChoice,RandomWorstAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31RandomChoice,RandomAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33RandomChoice,RandomBestAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37ExperimentsonRandomSudokuPuzzles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Backtracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38DLX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39RandomChoice,RandomWorstAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41RandomChoice,RandomAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42RandomChoice,RandomBestAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46FinalAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466. FUTUREWORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48RoleoftheBoardMaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Pre-guessAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48GuessTechniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48PerformanceMeasurement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49FaultTolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49PuzzleSubtleties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50REFERENCESCITED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51viiLISTOFTABLESTable Page1. BacktrackingExperimentOne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292. DLXExperimentOne. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313. RandomWorstAgentExperimentOne. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324. RandomAgentExperimentOne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345. RandomBestAgentExperimentOne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366. BacktrackingExperimentTwo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397. DLXExperimentTwo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408. RandomWorstAgentExperimentTwo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429. RandomAgentExperimentTwo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4410. RandomBestAgentExperimentTwo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4511. TimeLimitandCommunicationFailures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46viiiLISTOFFIGURESFigure Page1. ASudokuPuzzleanditsSolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42. InternalAgentStructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93. CommunicationPointsforAgentA00. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114. InternalStructureofaCellAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155. InternalStructureoftheBoardMasterAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196. ExampleSudokuPuzzle: CellAgentsatStartup. . . . . . . . . . . . . . . . . . . . . . . . . 227. ExampleSudokuPuzzle: CellAgentsAfterRoundOne . . . . . . . . . . . . . . . . . 238. ExampleSudokuPuzzle: CellAgentsAfterRoundTwo. . . . . . . . . . . . . . . . 239. ExampleSudokuPuzzle: CellAgentsAfterRoundThree . . . . . . . . . . . . . . . 2410. ExampleSudokuPuzzle: CellAgentsAfterGuess . . . . . . . . . . . . . . . . . . . . . . . 2411. ExampleSudokuPuzzle: CellAgentsAtCompletion. . . . . . . . . . . . . . . . . . . . 2512. ExperimentOneTimeResultsforBacktracking. . . . . . . . . . . . . . . . . . . . . . . . . . 2913. ExperimentOneGuessResultsforBacktracking . . . . . . . . . . . . . . . . . . . . . . . . . 2914. ExperimentOneTimeResultsforDLX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3015. ExperimentOneGuessResultsforDLX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3116. ExperimentOneTimeResultsforRandomWorstAgent. . . . . . . . . . . . . . . . 3217. ExperimentOneGuessResultsforRandomWorstAgent . . . . . . . . . . . . . . . 3218. ExperimentOneMessageResultsforRandomWorstAgent. . . . . . . . . . . . 3219. ExperimentOneTimeResultsforRandomAgent. . . . . . . . . . . . . . . . . . . . . . . 33ix20. ExperimentOneGuessResultsforRandomAgent. . . . . . . . . . . . . . . . . . . . . . 3421. ExperimentOneMessageResultsforRandomAgent . . . . . . . . . . . . . . . . . . . . 3422. ExperimentOneTimeResultsforRandomBestAgent. . . . . . . . . . . . . . . . . 3523. ExperimentOneGuessResultsforRandomBestAgent. . . . . . . . . . . . . . . . . 3624. ExperimentOneMessageResultsforRandomBestAgent . . . . . . . . . . . . . . 3625. ExperimentTwoTimeResultsforBacktracking. . . . . . . . . . . . . . . . . . . . . . . . . 3826. ExperimentTwoGuessResultsforBacktracking. . . . . . . . . . . . . . . . . . . . . . . . 3827. ExperimentTwoTimeResultsforDLX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3928. ExperimentTwoGuessResultsforDLX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4029. ExperimentTwoTimeResultsforRandomWorstAgent. . . . . . . . . . . . . . . 4130. ExperimentTwoGuessResultsforRandomWorstAgent. . . . . . . . . . . . . . . 4131. ExperimentTwoMessageResultsforRandomWorstAgent . . . . . . . . . . . . 4132. ExperimentTwoTimeResultsforRandomAgent . . . . . . . . . . . . . . . . . . . . . . . 4333. ExperimentTwoGuessResultsforRandomAgent . . . . . . . . . . . . . . . . . . . . . . 4334. ExperimentTwoMessageResultsforRandomAgent . . . . . . . . . . . . . . . . . . . 4335. ExperimentTwoTimeResultsforRandomBestAgent . . . . . . . . . . . . . . . . . 4436. ExperimentTwoGuessResultsforRandomBestAgent. . . . . . . . . . . . . . . . 4537. ExperimentTwoMessageResultsforRandomBestAgent . . . . . . . . . . . . . . 45xABSTRACTThe Cougaar distributed agent architecture, originally a DARPA-funded researchproject, provides aplatformfor developingdistributedagent systems. Testingitsabilitytosolvecomplexproblemsusingalargenumberof agentsinaninterestingtopicofresearch.Inthis thesis, the Cougaar distributedagent architecture is studiedfromthestandpointofSudoku. Throughanalysisandexperimentation,insightisgainedintoboth the properties and weaknesses of Cougaar. Cougaars performance when solvingSudoku puzzles is then compared with other Sudoku solving techniques. The Cougaaragent approachsolves Sudokupuzzles inahuman-like fashion, reachingsolutionsusingbothanalysisandguessing.CougaarisshowntobecapableofsolvingSudokupuzzlesinadistributedagentarchitecture. Although not as fast as traditional techniques, the Cougaar distributedagentapproachisabletoprovidesolutionstoSudokupuzzlesusingfewerguesses.Additionally, solvingSudokupuzzles withCougaar exposedsomereliabilityissuesanddemonstratedtheoverheadrequiredforcommunication. TheseresultsopenthewayforadditionalstudyofCougaar.1PROBLEMSTATEMENTThe purpose of this thesis is to study the Cougaar distributed agent architecturebyapplyingittotheSudokuproblemandtounderstandsomeofthepropertiesandweaknesses of Cougaar. Cougaar provides astableplatformfor distributedagentwork by handling many of the basic requirements for an agent system, such as agent-to-agentcommunicationandservicediscovery.Sudokuis aninterestingproblemtosolvewiththeCougaar distributedagentarchitecture. EachsquareontheSudokuboardcanbetreatedasanautonomousagent. These agents then communicate with neighbors to gather enough informationto solve the Sudoku puzzle as a community. This distribution creates a large numberof agentsandprovidesanenvironmenttostudyCougaar. Traditional, singleCPUapproachestosolvingSudokupuzzlesarealsodiscussed. Theseprovidedabaselineforcomparisonintheexperiments.Each agent in the Cougaar system only had a limited amount of analytical power.Moredicult Sudokupuzzles requiredanadditional agent tomakeguesses whenneeded. Three Sudoku problem solving methods were tested: a random choice from arandomworstagent,arandomchoicefromanyrandomagent,andarandomchoicefromarandombestagent. Here, bestindicatesanagentisfromthesetof agentswiththefewestpossibilities,andworstisthesetwiththemostpossibilities.2Two sets of Sudoku puzzles were tested. The rst consisted of actual Sudoku puz-zlesfromWebSudoku.com[1]. Thesecondwasrandomlygenerated. Forbothsets,the single CPU approaches performed faster and more reliably than the Cougaar dis-tributedagentSudokuproblemsolvingtechniques. AlthoughtheCougaarSudokuproblemsolvingtechniquesweremuchslower, theyconsistentlysolvedeachSudokupuzzlewithsignicantlyfewer guesses. Thetestingshowedthat theCougaar dis-tributedagentarchitectureiscapableofbeingappliedtosolveSudokupuzzles.During testing, Cougaar was shown to have some reliability issues. Agents wouldbecome unresponsive and would no longer receive messages sent to them. Restartingtheagentwastheonlywaytoreturnthesystemtoaworkingstate. Additionally,messagesbetweenfunctional agentswouldnotbedelivered. Theoverheadrequiredto send messages among agents and ensure the delivery of commands and informationgreatlyoutweighedanyworkdonebyindividual agentsfocusedtowardsolvingtheSudokupuzzleitself. Thesendingsshowsomeof theweaknessesof Cougaarandopenthewayforadditionalresearch.3RELATEDWORKIntroductiontoSudokuSudokuisoftenthoughttohaveoriginatedinJapan, duetothenameanditspopularity asa Nikoli-publishedpuzzlegameinJapan sincethe1980s[2]. However,Dell Magazines previouslypublishedit under thenameNumber Placeinthelate1970s[3]. ThegamealsohasrootsinLatinSquares, agamethoughttohavebeeninventedbyLeonhardEuler inthe1700s [4]. Regardless of origin, thegamehasrecently become internationally popular and can be found in a number of mainstreamnewspapers.SolvingSudokuSudoku is a number logic game where the goal is to ll a square grid with numbersrelatingtothesizeofthepuzzle. Thetypical Sudokupuzzleisaninebyninegridthat must be lled using the numbers one through nine. The grid is divided into rows,columns, andsubsquares. Asubsquareisannbynsquare, wherenisthesquareroot of the number of columns or rows. For a standard Sudoku puzzle, n is three. Allstandard Sudoku puzzle will be already partially lled in (Figure 1, left puzzle). Theinformationprovidedbythesepreexistingnumbersdeterminesthegeneraldicultylevel of the puzzle. The goal is to arrange numbers on the grid so that every numberappearsonlyonceineachrow,column,andsubsquare(Figure1,rightpuzzle).4Figure1. ASudokuPuzzleanditsSolution.AproperlyformedSudokupuzzlehasonlyonecorrectsolution. Showingthatacandidate Sudoku puzzle has more than one solution was proved to be NP-Completeby Yato andSeta [5]. The total number of properly formedSudokupuzzles is6670903752021072936960,asenumeratedbyFelgenhauerandJarvis[6]. Algorithmsfor solving Sudoku generally follow two styles: rapid computer algorithms and humantechniques.ComputerAlgorithmsPopularrapidsolversarebasedonbacktrackinganddancinglinks. RapidstyleSudokusolversarepreferredforverifyinglargesetsof randomlygeneratedSudokupuzzlesduetotheireaseofimplementationandquickresults. Backtracking,acom-montechniqueinarticialintelligence,isa renedbruteforcesearchthat exploresaconstrainedsetofpossibilitiesuntilreachingasolution[7]. DonaldKnuthsuggested5Dancing Links (DLX) as an implementation of his Algorithm X which solves the exactcoverproblem[8].SinceSudokuisasubsetoftheexactcoverproblem,DLXprovidesaquickandecient Sudoku solver. Each space is treated as a vertex on the Sudoku grid, yielding81vertices. ThedrivingforcebehindDLXistheconceptthatnodesinacircular,doubly-linkedlistcanbequicklyinsertedorremoved. DLXoperatesonamatrixofpossibledecisionswhereeachcolumnandrowisacircular, doubly-linkedlist. Theway DLX inserts and removes nodes from the matrix was the inspiration for the nameDancingLinks.HumanTechniquesHumanstylesolvingisgenerallyusedasawaytoestimatethedicultylevelof apuzzle. Humansolvers commonlyemploytechniques suchas cross-hatching,counting, andmarkingup[9]. Cross-hatchingisascanningtechniquethatusestheprocess of eliminationtoidentifylines whereanumber must be. Countingis theoppositeofcross-hatchingbecauseiteliminatesnumbersthatcannotbeassignedtoacell. Markingupis usedafter applyingcross-hatchingandcounting. Acell ismarkedupbylistingallpossibleremainingvalidnumbers. Afterallcellshavebeenmarkedup,theycanbecomparedandanalyzed.6COUGAARIntroductionCOGnitive Agent Architecture (Cougaar) is a Java-based agent architecture [10].Cougaar is capable of handling large-scale distributed applications and was developedaspart of thesolutionto the DARPA UltraLog project,a distributedlogisticsappli-cationcomprisedofmorethan1000agentsdistributedover100physicallydispersedhosts[11]. Aprimarygoal of UltraLogwastomaintaincommunicationamongallhostsevenwith40%ofthecommunicationnetworkdisabledbyasymetricalattacks.Theresultingsystemiscompletelyopen-source. DuetothesizeandcomplexityofCougaar, onlythecomponentsusedforthisresearchwill bediscussed. Foramorecompletedescription,seetheCougaarDevelopersGuide[12].TheDecisiontoUseCougaarThedecisiontoinvestigateCougaar, insteadof anotheragentarchitecture, wasbasedontheauthorspreviousworkwithCougaar. AgrantbytheTransportationSecurity Administration [13] to Rocky Mountain Agile Virtual Enterprises at MontanaTech[14]fundedresearchanddevelopmentofanairportsecuritysystemdistributedacrossmultipleagentsusingCougaar. Theauthorworkedonapartof theprojectdevotedtodooraccesscontrol usingdistributedagents. Apaperpresentedatthe7Computational Intelligence conference in Calgary in 2005 provides details of the dooraccesscontrolprojectandhowitwastested[15].AlternativeAgentArchitecturesAlthoughtheCougaardistributedagentarchitectureisthefocusof thisthesis,other alternativeagent architectures areavailable. TheOpenAgent Architecture(OAA), developedbySRIInternational, primarilyfocusesontheconceptofadele-gated computing model where agents are assigned tasks by a controller agent based ontheir capabilities [16]. The Common Object Request Broker Architecture (CORBA),developed by OMG, is a widely used platform that provides the infrastructure and ar-chitecture needed to develop a distributed agent architecture [17]. Jini, an open sourceproject supported by Sun Microsystems [18] and SolutionsIQ [19], is a service orientedarchitecturethatcansupportintelligentagentsystems[20]. TheReusableEnviron-ment for Task-StructuredIntelligent NetworkedAgents (RETSINA), developedatCarnegieMellon,isaresearchplatformforstudying agentarchitecturerelatedprob-lems such as reliable communication and agent reuse [21]. Finally, The KnowledgeableAgent-oriented System (KAoS), developed by Boeing, provides an agent architecturethatattemptstoaddressthecommondistributedagentarchitectureproblemcausedbyalackof semanticsandlackof extensibilityforagentcommunicationlanguages[22].8TheCougaarProblemDomainCougaarissuitedforuseinawiderangeof distributedoragentrelatedprob-lems. Any autonomous group of robots could utilize an agent architecture to providesoftwareinfrastructurefortaskssuchascommunicationandtaskdelegation. Theadaptive logistics required for inventory management dispersed across multiple phys-ical locations would be another problem suited to an agent architecture. Other prob-lemsthatcouldbeexploredwithanagentarchitecturesuchasCougaararesecuritysystems,datamining,loadbalancing,andcerticateauthorities.CougaarComponentsTheCommunityCougaarsupportsthesegmentingofagentsintocommunities[23]. Acommunityis composed of logically related agents. An agent can belong to an unlimited numberof communities. It mayjoinor leave acommunityas its goals change. Fromacommunicationstandpoint, thisfunctionallowsasinglemessagetobeaddressedtoanentirecommunitywithoutregardtoacommunityscurrentmembership.TheNodeANodeis asingleJavaVirtual Machine[24] that maycontainandmaintainmultipleagents[12]. Generally, Nodessharea1:1correspondencewithaCPU. Forexample, anodemaycontainall agentsrequiringaccesstoarestrictedresource. AnodecanbetreatedlogicallyasacommunityrestrictedtoasingleCPU.Anodeis9similartoacommunitybecauseanagentononenodecanmovetoanother. Anodeprovides several services to its agents, such as message routing, logging, and resourceaccess.TheAgentEachCougaaragentiscomposedofplugins, ablackboard, andsupportservices(Figure2). Thepluginsprovidethebehaviorofanagent. Theycommunicatewithotherplugins, agents, andcommunitiesusingtheblackboard. Theblackboardsup-ports the publish/subscribe semantic for object handling [25]. A plugin indicates thetypesofobjectitisinterestedinbyregisteringasubscriptionwiththeblackboard.After an object is published to the blackboard, the agent checks all subscriptions andactivatesrelevantplugins. Theplugincanthenretrievetheobjectandperformanytasks. Anypluginmaypublishobjects totheblackboard. Plugins normallyonlyactivatewhenoneoftheirsubscriptionsistriggered.Figure2. InternalAgentStructure.10Inter-AgentCommunicationAgents communicate witheachother usingabuilt-in, asynchronous message-passing protocol called the Message Transport Service (MTS) [26]. The MTS providesan API for sending messages to agents by name. For solving Sudoku puzzles, messagestaketheformofarelay. Arelaycombinessendandreceiveblackboardinterfacestosharemessages betweenagents. Sourceobject dataappears ontheblackboardofthetargetagentandcanreturntothesourceagentsblackboardasareply[27]. Awhitepagesservice,similartoDNS,performsaddressresolutionfortheagents[28].Additionalnodescanrunthewhitepagesservicetoprovideredundancy.11THEAGENTAPPROACHTOSUDOKUTheagentapproachtoSudokuisbasedondividingtheworkloadevenlyamongmany agents. One agent is assigned to each of the 81 cells on the board. A nal agent,the board master, has the task of maintaining a complete picture of the boards stateandmakesanydecisionsthatrequireaknowledgeofeachagentsstatus. Eachcellagentisonlyallowedtocommunicatedirectlywiththeboardmasterandthecellagentsinitsrow, column, andthreebythreesquare. Forexample, cell agentA00communicateswith20othercellagentsandtheboardmaster(Figure3).Figure3. CommunicationPointsforAgentA00.12KeyObjectsTherearethreekeyobjectsthatplayanimportantroleinhowagentsrelatetoeachotherandhowdecisionsaremade: thenumber,thestatus,andthecommand.NumberTheNumberobjectisapairofintegers. Therstrepresentsapossiblevalueonthe Sudoku board, ranging from one to nine. The second is a guess ID that indicatesthe point at which the Number was removed as a candidate. An active Number is anyNumberobjectthathasnotbeeneliminatedoruniquelyselectedasthechoiceforaparticularcell. ActiveNumbersaredesignatedbyaguessIDof-1. Forexample, acellmighthavethefollowingun-eliminatedchoices: 1,5,and7. TheactiveNumberassociatedwith7wouldbe[7,-1]. If itwereeliminatedasacandidateforthecellaftertherstguesswasmade, itwouldbecome[7,1]. Thepurposeof storingtheguessIDalongwiththecandidatenumberisdiscussedintheRollbacksubsectionofTheBoardMasterAgentsection.StatusThe Status object represents all important external information about a cell agentto other cell agents and the board master. Depending on the situation, a Status objectmaycontainanyorallofthefollowingelements:Thecellagentsname. Forexample,thetop-leftcellsnameisA00andthebottomrightsisA88.13AlistofthenineNumberobjects,reectingtheircurrentstatus.ThemessageID. Thisisanincreasingnumberthatisneverreset. Themes-sageIDisusedtoeliminateout-of-ordermessages, onlythelatestmessageisimportant.TheresetID.Thisisanincreasingnumberthatisneverreset. Afterapuzzleis completed, every cell agent must be reset to its starting conguration for thenext puzzle. A cell agent with the wrong Reset ID is clearly still referencing anoldpuzzleandcanbecorrectedbytheboardmaster.TheguessID.Thisisavaryingnumberthatresetsoncompletionofapuzzle.The guess ID indicates the cell agents understanding of the current guess level.ThisisusedincombinationwiththecommandIDtoensurethatcell agentsarefollowingtheguessandroll-backsequencecorrectly.ThecommandID.Thisisanincreasingnumberthatresetsoncompletionofapuzzle. SincetheguessIDcanincreaseordecrease, thecommandIDisusedtoguaranteethatthecellagenthasrolledbackappropriatelybeforeanalyzingdatafromthecurrentguessID.Thechangecounter. ThisisusedinternallybytheboardmastertocountthenumberofconsecutivetimesacellagenthassentthesameStatus.Neighbor completeness. This is set bythecell agent toindicatethat it hasreceivedStatusobjectsfromall of itsneighbors. Theboardmasterwill notsignalforsolvingtobeginuntilallcellagentsreportthisastrue.14A Status contains almost all relevant data for a cell agent and is capable of producingguessesbasedonthecurrentlyactiveNumbers. Thereforewhenmultipleguessesaremadeforthesamecell,itwillnotproducethesameguesstwice.CommandTheCommandobjectistheonlymessagepassedfromtheboardmastertothecellagents. Theboardmasterusesvetypesofcommands:Startup: BeforeacellagentcanbeginitspartinsolvingaSudokupuzzle,theboardmastermusttellitwhetherhasastartingvalueornot.Reset: Afterapuzzleiscomplete,everycellagentissentthesameresetcom-mand. ThecommandcontainsanIDtobestoredbythecellagents.Guess: Whenevertheboardmastermakesaguess, itsendsthiscommandtothechosencell agent, alongwiththeguessedvalueforthatcell agentandanincreasedguessID.Rollback: If thepuzzleentersaninvalidstate, theboardmasterinstructsallcellagentstorollbacktotheguessIDprovided.Report: Theboardmasterinstructsthecellagenttorespondwithitscurrentstatus.TheCellAgentFora9x9Sudokugrid, thereare81total cell agents, oneforeachspaceontheSudokuboard. Eachisonlyawareoftheneighborsinitsrow,column,andthreeby15threesquare. Thecellagentsarecompletelyunawareoftheoverallboardstatus. AcellagentsonlygoalistoreduceitssetofactiveNumberstoone. Asaresult, thesystem exhibits an emergent behavior by solving the Sudoku puzzle. In this case, theemergent behavior is known formally as a weak emergence because the new property(the completion of a Sudoku puzzle) comes directly from the interaction of individualelements (the cell agents) [29]. Arrangement of the cell agent within Cougaar is showninFigure4.Figure4. InternalStructureofaCellAgent.16StartingBeforestartingworkonaSudokupuzzle, all cell agentsneedtobecommuni-catingsuccessfullywiththeirneighbors. Eachcellagentisexplicitlyawareofothercell agentsinitsrow, column, andthreebythreesquare. Communicationlistsarecomposed at startup based on the cell agents name. Each cell agent sends a messageto the other cell agents on its lists. For eciency, cell agents appearing on more thanonelistonlyreceiveonemessage. Acell agenthastwoopportunitiestoreceivearesponse. First,whenthetargetcellagentstartsandsendsmessagesofitsown,andsecond,whenitrespondstoarequest. Topreventendlessrespond-respondlooping,eachcell agentkeepsalistof othercellsitknowstoberesponsive. Whenthelistcontainsallcellsacellagentneedstocommunicatewith, itsendsaStatusmessagetotheboardmasterindicatingitisreadytobegin. Atthisstage, all cellsconsider1-9tobevalidchoices.SolvingWork on the puzzle begins when the board master sends replies to the rst roundof Status messages. If this response indicates the cell agent is constrained to a specicnumber,thecellagentsendstheupdatedStatustoitsneighbors(row,column,andthreebythreesquare). Asothercell agentsreceivethesemessages, theydeactivatethe appropriate Number in their local Status. The cell agent then sends the updatedStatustoall itsneighbors. Thesetwotypesof Statusmessages(onewithasingleactiveNumber, onewithmany) allows for twotypes of analysis. Therst is the17eliminationofallchoicesbutonefora speciccell. Thesecondtypeiswhenthecellagent compares the local Status to the received Status from its neighbors. By processofelimination,acellagentmayconstrainitselftoasingleactiveNumberifnoothercellagentsinitsrow,column,orthreebythreesquareshowthatNumberasactive.RollingBackWhenacell agentreportstotheboardmasterwithaduplicatevaluealreadycontainedinitsrow,column,orthreebythreesquare,theentirepuzzlehasenteredaninvalidstate. Theboardmasterissuesarollbacktorestoretheboardtoastatewhere a solution is still possible. Also, if a cell agent reports that it has eliminated allcandidateNumbersasvalidchoices,arollbackisneeded. Theboardmastertriggersarollbackbysendingarollbackcommandtoall cell agents. Eachcell agentthenrevertsitself tothestatebeforethemostrecentguesswasmade. Describinghowguessesandrollbacksarehandledatthecellagentlevelisbestshownbyexample.AcellagentsNumberlist,beforestartup:[1,-1][2,-1][3,-1][4,-1][5,-1][6,-1][7,-1][8,-1][9,-1]AconstrainedcellagentsNumberlist(constrainedto5),afterstartup:[1,0][2,0][3,0][4,0][5,-1][6,0][7,0][8,0][9,0]Acell agentsNumberlistwith3, 5, and6ascandidates, beforetheboardmastermakesaguess:[1,0][2,0][3,-1][4,0][5,-1][6,-1][7,0][8,0][9,0]18The same cell agents Number list, after the board master makes a guess (ID 1) whichsetsaneighborto3:[1,0][2,0][3,1][4,0][5,-1][6,-1][7,0][8,0][9,0]The same cell agents Number list, after the board master makes a guess (ID 2) whichsetsaneighborto5:[1,0][2,0][3,1][4,0][5,2][6,-1][7,0][8,0][9,0]Now the cell agent is constrained to 6. If the board master issues a rollback for guessID 2, the cell agent reverts the Number list by reactivating any Number with a guessIDof 2orhigher. TheboardmastercanissuemultiplesimultaneousrollbacksbyusingalowerguessID.ThesamecellagentsNumberlist,afterrollbackwithguessID2:[1,0][2,0][3,1][4,0][5,-1][6,-1][7,0][8,0][9,0]CellagentsconstrainedbeforeagivenguessIDhavenorollbackworktodo.TheBoardMasterAgentThe board master agent is responsible for keeping the system synchronized, hand-ingoutstartingandguessvalues,rollingbackguesses,andtrackingtheoverallstateoftheboard. TheboardmasteragentonlyreceivesStatusobjectmessagesfromthecell agents, but thestateof theboarddetermineshowthemessageswill beused.ArrangementoftheboardmasteragentwithinCougaarisshowninFigure5.19Figure5. InternalStructureoftheBoardMasterAgent.StartingThe board master creates a list of cell agents as they request their startup valuesin order to ensure that all cell agents are online. When the list reaches 81 (for a ninebynineSudokuboard), theboardmasteragentsendsall startingvaluestothecellagents. If acell agentissupposedtobeconstrainedtoasinglestartingvalue, butreportsalistof several values, theboardmasteragentsendsthestartupmessageagain.SolvingAs cell agents become constrained, they send status messages to the board masterthatreecttheirconstraints. Theboardmasterkeepsa2Darrayofall cell agentsstatus. A timer periodically requires all cell agents to report to the board master withtheir Status. The timer only triggers if there have been no Status reports for a periodof time, set at the start of the system. The slowest CPU in the system determines the20necessary wait time, to allow for late coming messages. If the cell agents are reportingupdatestotheboardmaster,thetimerwillnottriggertoavoidunnecessarytrac.DetectingaStalledPuzzleIfall cell agentsarerespondingwithunchangingStatus, noneareabletomakefurtherrestrictionstotheirnumberlists,andtheboardisconsideredstalled. Whentheboardhasstalled,theboardmasterstartsguessing.GuessingTheboardmasterusesseveral methodstochoosewhichcellagentshouldmakeaguess.1. RandomGuessfromaRandomAgent: TheboardmasterrandomlyselectsacellagentfromthesetofallcellagentswithanyactiveNumbers. ThisisthemostbasicSudokuproblemsolvingmethod.2. Random Guess from a Random Best Agent: The board master randomly selectsa cell agent from the set of cell agents with the lowest amount of active Numbers.3. RandomGuess fromaRandomWorst Agent: The boardmaster randomlyselects a cell agent from the set of cell agents with the highest amount of activeNumbers.Oncetheboardmasterhasselectedacellagenttomakeaguess,thecellagentsStatusisaddedtoaguessstack. ThesizeofthestackistheboardsguessID.Each21Status provides amethodtoreturnpossibleguesses until noneremain. Oncetheboard master has exhausted all guesses for a cell agent without reaching a completedpuzzle state, the cell agents Status is removed from the guess stack. The next Statuson the stack will provide the next guess. Whenever a guess leads to an invalid boardstate, the board master issues a rollback that reverts the board to its state before thecurrentguess.RollingBackAll cell agents perform rollbacks if needed and respond with a Status that reectsthenewguessID.Afterallcellagentshavereportedback,theboardmasterselectsthenextguessfromtheStatusontopoftheguessstack. Ifnomoreguessesremainfor agivenStatus, it issuesanotherrollbackbeforemovingtothenextStatusonthestack. Oncetheboardmasterndsasuitableguess, itsendstheguesstothecorrespondingcell agent andsolvingresumes. Byprocess of elimination, therstchosencell agentmustyieldaguessthatleadstoacorrectsolution. If theboardmasterexhaustsallguessesforthiscellagent,theSudokupuzzlehasnosolution.CompletionOncethesystemhassolvedtheboardcorrectly,allcellagentsbecomedormant.Iftheboardmasterisloadedwithanotherpuzzle, itsendsaresetcommandtothecellagents,andthesystemreturnstotheStartupstage.22IllustrativeExampleAlthoughall testinganddesignis centeredaroundthestandardninebynineSudoku puzzle, it is more insightful to use a smaller four by four Sudoku puzzle as anexample. TheguessingmethodusedwillbeRandomGuessfromaRandomAgent.Throughoutthisexample,italicsindicatechangesbetweenpreviousandcurrentg-ures. At startup, A11 and A30 are constrained to 2. A23 is constrained to 1. All cellagents not provided an initial constraint have all active Numbers, as shown in Figure6.Figure6. ExampleSudokuPuzzle: CellAgentsatStartup.Immediatelyfollowingstartup, non-constrainedcell agents determinewhichoftheir active Numbers can be deactivated. 7 shows this change in all active cell agentsexceptA02.Now that all cell agents have an accurate representation of their neighbors,anal-ysiscanoccur. Forthis,eachcellagentscansitsneighborsactiveNumbersagainstitsown. IfaNumberisactiveforthecurrentcellagent,butnotforanyofthatcell23Figure7. ExampleSudokuPuzzle: CellAgentsAfterRoundOne.agents neighbors in a row, column, or two by two square, the current cell agent is theonlyremainingcandidate. Forthisexample,A23usesthisprocesstoconstrainitselfto2(Figure8).Figure8. ExampleSudokuPuzzle: CellAgentsAfterRoundTwo.Asecondroundofmessagesandanalysisyieldsonemoreconstrainedcellagent,A03to2(Figure9).Atthispoint,nofurtherworkcanbedonebythecellagentswiththeircurrentanalysis tools. The puzzle will not move closer to a solution without a guess from the24Figure9. ExampleSudokuPuzzle: CellAgentsAfterRoundThree.boardmaster. SelectingarandomNumberfromarandomagent, theboardmasterchooses3fromagentA10(Figure10).Figure10. ExampleSudokuPuzzle: CellAgentsAfterGuess.Afteranalroundofmessages,everycellagenthasconstraineditselftoasingleactiveNumber,andtheSudokupuzzlehasbeensuccessfullysolved(Figure11).25Figure11. ExampleSudokuPuzzle: CellAgentsAtCompletion.26EMPIRICALANALYSISMethodologyMetricsBecauseit isdicult tomeasurespaceinadistributedagent-basedarchitecture,onlytimeandguesseswereconsideredinthisstudy. Forbacktracking,therawtimetosolvetheproblemwasmeasured, andeachrecursivecall wastreatedasaguess.ForDLX,onlythemeasurementoftimecouldbeaccuratelycomparedtotheotherSudokusolvingtechniques. WorkdonebyDLXwasmeasuredbyrecordingthere-moval oradditionof linkstothematrix. Forconsistency, eachof thesechangestothematrixwasreferredtoasaguess.For the Cougaar approach, measurements includedthe rawtime tosolve theproblem, the number of messages passed, and the number of guesses made. Althoughmessage counts cannot be eectively compared to backtracking or DLX, they provideausefulmetricforcomparingSudokuproblemsolvingstrategieswithintheCougaarapproach.EachSudokusolverhad10recordedattemptsateverypuzzle. Oneachpuzzle,averages and standard deviations of the metrics were recorded. For backtracking andDLX, this applied to time and guesses. For the Cougaar Sudoku system, this appliedtotime,guesses,andmessages.27TestSetsTwotestsetswereusedtomeasuretheperformanceoftheAgent-basedSudokusolver. Therstconsistedof veriedSudokupuzzles, andthesecondconsistedofrandomlygenerated,unveriedpuzzles. AveriedSudokupuzzlehasbeenshowntoonly have one solution. All of the veried Sudoku puzzles used for testing were takenfromWebSudoku.com. Tenpuzzlesweretakenfromeachof theavailabledicultylevels, Easy, Medium, Hard, andEvil, yielding40total puzzles for testing. Forreference,thedicultylevelsareclassiedbyWebSudoku.comasfollows:Thedicultyof apuzzleisrelatedtothedepthof thinkingrequired. AneasySudokucanbesolvedusingmoresimplelogic, whereasanevil puzzleneeds deeper analysis. Obviouslythere is alsovariationbetweendierentpuzzles, soevenwithinalevel somepuzzlesmayseemharderthanothers.[1]A Sudoku puzzle was selected at random to provide the base puzzle for generatingadditional puzzles. The Sudokupuzzle starts inacompletedstate andcells areblankedatrandomingroupsofnine. Thisisrepeated, savingtheresultingSudokupuzzleateachstep, until theentireSudokupuzzleisblank. TheentirereductionfromacompletedSudokupuzzletoablankonewasrepeatedfourtimestoyieldatotal of forty puzzles. These randomly reduced Sudoku puzzles will be referred to bythe number of blanks they contain. For example, the 0 puzzle contains no blank cellsandthe81puzzleiscompletelyblank.28CougaarEnvironmentOneofCougaarskeyfeaturesistheabilitytoconnectagentsregardlessoftheirlocation on a network. During development, the agent system was tested on networksof uptovedesktopcomputers. For theexperiments, 82agents ranonasingleworkstation. The GUI proved an eective tool for developing, testing, and debuggingthe agents on single boards but was impractical for any large scale analysis. To enablerepetitive batch-processing for large numbers of puzzles with varying decision makingconstraints, a separate board master was developed. At the Cougaar level, the secondboard master is identical to the rst. In the place of a GUI, all commands came fromthecommandlineandall resultswereloggedinseparatelesaccordingtopuzzlenumber, date, and time. Finally, several scripts were used to compile the results intomeaningfulformats.ExperimentsonVeriedSudokuPuzzlesEach of the Sudoku puzzles used in this section were veried by WebSudoku.comtoonlyhaveonesolution. Duringthetestingof theCougaaragentapproach, oneormoreCougaaragentsoccasionallybecameunresponsive. ThisseeminglyhappensatrandomandappearstobeamessagingbreakdownwithinCougaarwhileunderhighmessagingloads. Eachof thetimesthisoccurred, testinghadtoberestartedmanually.Backtracking29Figure12. ExperimentOneTimeResultsforBacktracking.Figure13. ExperimentOneGuessResultsforBacktracking.Table1. BacktrackingExperimentOne.Time GuessesAverage Average Easy 0.16 0.51 816.70 837.78Medium 0.79 1.12 4027.90 4067.09Hard 2.06 3.10 8320.50 7308.03Evil 12.99 11.79 34694.30 32137.3230The backtracking solver successfully found a solution for all of the Sudoku puzzlesinthis test set. For all but the Evil puzzles, backtrackingwas able tosolve theSudokupuzzleswithin10millisecondsusingalownumberof guesses. Formostofthe puzzles, backtracking reached a solution in an amount of time that was too smalltobemeasuredbythetestingcomputer. Figure12andFigure13showastrongcorrelationbetweentimetosolveandguessesmade. Thisisespeciallyapparentinthe results of Evil 1: 40milliseconds andover 100,000guesses. This correlationbetweentimesandguessesexistsforallofthesolvingmethodstested.DLXFigure14. ExperimentOneTimeResultsforDLX.TheDLXsolver successfullyfoundasolutionfor all of theSudokupuzzles inthis test set. DLXtooklonger thanbacktrackinginalmost all cases. Anotableexception is DLXs overall time on the Evil puzzles (Table 2), which was very close tobacktracking (Table 1). This islikely attributable to DLXbeing a more complicatedalgorithmthanstraightforwardbacktracking. For theeasier Sudokupuzzles, this31Figure15. ExperimentOneGuessResultsforDLX.Table2. DLXExperimentOne.Time GuessesAverage Average Easy 4.67 2.63 949.50 998.27Medium 5.32 5.17 6163.30 8679.90Hard 8.27 9.05 29154.10 38082.75Evil 12.77 5.75 38513.40 26903.39slowed DLX down when compared to backtracking. On the more dicult Evil puzzles,thesamecomplexityallowedittoquicklysolvethepuzzles. Backtracking, ontheotherhand,experiencedasignicanttimeandguessincreasewhencomparedtotheotherthreecategoriesof puzzle. NotethatalthoughFigure14scalesintonegativetime, negative values are only shown for the sake of graphing the standard deviation.Thisoccursthroughouttheresultsandshouldbeignored.RandomChoice,RandomWorstAgentRandomChoicewithRandomWorstAgent(RandomWorst)istherstof theCougaarSudokuproblemsolvingmethods. RandomWorstexperiencedamessaging32Figure16. ExperimentOneTimeResultsforRandomWorstAgent.Figure17. ExperimentOneGuessResultsforRandomWorstAgent.Figure18. ExperimentOneMessageResultsforRandomWorstAgent.Table3. RandomWorstAgentExperimentOne.Time Guesses MessagesAverage Average Average Easy 1753.34 236.50 0.00 0.00 4.75 0.33Medium 1891.06 186.36 0.00 0.00 5.29 0.31Hard 3296.10 1987.08 0.81 1.40 8.00 3.60Evil 12221.46 6806.97 7.09 5.26 26.90 15.1733failure on four occasions: three Evil boards and one Medium board. Easy and Mediumrequired no guessing, so interesting results do not occur until the second half of HardandallofEvil. InsparselypopulatedinitialSudokupuzzles,RandomWorstisverylikely to make an incorrect guess early on. Figure 17 shows that Evil 8 was particularlydicultforthisSudokuproblemsolvingmethod. Thelargedeviationfromaverageindicatesthatsomeattemptsmadegoodguesses, whileothersmadeverybadones.Backtracking and DLX greatly outperformed Random Worst with regard to time, butRandomWorstusedfewerguessesinall cases. ThisisattributabletotheanalysistheAgent Methodperforms at eachstep. It is interestingtonotethat themostdicult puzzle for Random worst (Evil 8) was dierent from the most dicult puzzlefor backtracking (Evil 1) and DLX (Evil 5). Clearly,subtleties in these puzzles favoronesolvingmethodoveranother. FindingthespecicreasonsforthisismentionedintheFutureWorkchapter.RandomChoice,RandomAgentFigure19. ExperimentOneTimeResultsforRandomAgent.34Figure20. ExperimentOneGuessResultsforRandomAgent.Figure21. ExperimentOneMessageResultsforRandomAgent.Table4. RandomAgentExperimentOne.Time Guesses MessagesAverage Average Average Easy 1603.27 241.87 0.00 0.00 4.48 0.32Medium 1862.89 192.81 0.00 0.00 5.16 0.28Hard 3102.60 1627.65 0.63 1.04 7.44 2.75Evil 13156.62 11521.59 7.65 8.43 27.16 23.7335Random Choice with Random Agent (Random) failed to solve the Sudoku puzzleon two Evil boards. Random performed very similarly to Random Worst. Of particu-lar interest is Randoms average number of guesses (Figure 20) compared to RandomWorstsaveragenumberofguesses(Figure17)onthethird-to-lastEvilboard. Thedierence of 10 guesses shows that in some cases, guessing from a smaller set of agents(Random Worst) could reach a solution more quickly than guessing from a larger set ofagents. Compared to backtracking, Randoms performance is very similar to RandomWorsts.RandomChoice,RandomBestAgentFigure22. ExperimentOneTimeResultsforRandomBestAgent.RandomChoice withRandomBest Agent (RandomBest) failedtosolve theSudokupuzzleontwooccasions: oneEvilboardandoneHardboard. RandomBestpredictably shows the best overall performance of the three Cougaar Sudoku problemsolving methods. It consistently outperforms the other two Cougaar Sudoku problemsolvingmethodsforeverymetric. OnEvil8puzzle,RandomBestmakesthefewest36Figure23. ExperimentOneGuessResultsforRandomBestAgent.Figure24. ExperimentOneMessageResultsforRandomBestAgent.Table5. RandomBestAgentExperimentOne.Time Guesses MessagesAverage Average Average Easy 1703.64 223.13 0.00 0.00 4.65 0.30Medium 2009.71 177.80 0.00 0.00 5.42 0.30Hard 2955.28 1184.41 0.44 0.72 7.01 1.71Evil 10090.55 5006.66 5.34 3.56 19.77 9.0437guesses. Thishierarchy(RandomBest,RandomWorst,Random)indicatesthattheCougaarSudokuproblemsolvingmethodsmightperformbetterwithsmallersetsofagentstoguess from. Comparedtobacktracking, RandomBestss performanceisverysimilartotheothertwoCougaarSudokuproblemsolvingmethods.ConclusionsBacktrackingandDLXweresubstantiallyquickerthananyof theCougaarSu-doku problem solving methods. All of the Cougaar Sudoku problem solving methodsrequiredfewer guesses toreachasolutionthanbacktracking. RandomandRan-domWorst performed very similarly. RandomBest waspredictably theclear winneramongtheCougaarmethods.ExperimentsonRandomSudokuPuzzlesIntheprevioussection,theveriedSudokupuzzleswereallknowntohaveonlyonesolution. Inthissection, theSudokupuzzlesmighthaveoneormoresolutions.Forexample,aninitiallyemptyninebynineSudokupuzzlehasaverylargenumberof solutions. Due to the large number of unconstrained cells in some of these Sudokupuzzles, it is possible for the random Sudoku problem solving methods to take a verylongtime. Atimelimitof veminuteswassetforeachsolvertoreachasolutionbefore moving on to the next Sudoku puzzle. For each of the Cougaar Sudoku problemsolvingtypestested, thistimelimitfailureismentionedalongwithanyfailuresin38messagecommunication. Thefullsetofpuzzlesinthissectionwillbereferredtobythenumberofblankstheycontain: 0,9,18,27,36,45,54,63,72,and81.BacktrackingFigure25. ExperimentTwoTimeResultsforBacktracking.Figure26. ExperimentTwoGuessResultsforBacktracking.Thebacktrackingsolversuccessfullyfoundasolutionforall oftheSudokupuz-zles inthis test set. Figure 25andFigure 26showthat none of the randomly-reducedSudokupuzzlesposedasmuchsolvingdicultyforbacktrackingasdidtheEasy/Medium/Hard/Evil Sudoku puzzles (Figure 12 and Figure 13). In Table 6, the39Table6. BacktrackingExperimentTwo.Time GuessesAverage Average 0 0.00 0.00 81.00 0.009 0.00 0.00 81.00 0.0018 0.00 0.00 82.25 1.8927 0.00 0.00 93.00 15.7736 0.00 0.00 159.75 103.0345 0.00 0.00 744.50 682.6954 2.38 2.74 7260.50 2775.5363 3.52 3.45 5523.75 8100.8472 0.00 0.00 506.25 191.3781 0.38 0.75 391.00 0.000puzzleresultsshowtheminimumnumberofrecursivecallsbacktrackingperformson an already-completed Sudoku puzzle. It is interesting that only the 45, 54, and 63puzzlescausedbacktrackingtodoanywork. Theseresultsindicatethatbacktrack-ingisextremelyeectiveatsolvingover-constrainedandunder-constrainedSudokupuzzles.DLXFigure27. ExperimentTwoTimeResultsforDLX.40Figure28. ExperimentTwoGuessResultsforDLX.Table7. DLXExperimentTwo.Time GuessesAverage Average 0 3.12 3.05 0.00 0.009 5.05 2.86 9.00 0.0018 3.15 1.27 19.00 1.1527 3.17 1.27 34.50 11.1236 3.92 2.74 95.00 55.7345 1.98 1.97 492.00 445.5254 3.07 1.76 1240.50 696.2763 3.47 4.02 576.50 712.9372 5.50 3.70 565.00 507.2481 3.40 1.46 701.00 0.00The DLX solver successfully found a solution for all of the Sudoku puzzles in thistest set. DLX took longer to solve each puzzle (Figure 27) than backtracking (Figure25). InTable7, the0puzzleresultsshowthatDLXperformssomeanalysisof apuzzlebeforedoinganywork. Puzzles0,9,18,and27onTable7showthatDLXiscapableofmakingaboutonechangetothematrix(guess)perblankontheboard,whenthechoicesarefromasucientlyconstrainedset.41RandomChoice,RandomWorstAgentFigure29. ExperimentTwoTimeResultsforRandomWorstAgent.Figure30. ExperimentTwoGuessResultsforRandomWorstAgent.Figure31. ExperimentTwoMessageResultsforRandomWorstAgent.42Table8. RandomWorstAgentExperimentTwo.Time Guesses MessagesAverage Average Average 0 354.72 143.03 0.00 0.00 0.51 0.339 1081.80 52.67 0.00 0.00 3.16 0.0918 1148.40 204.42 0.00 0.00 2.98 0.3127 1351.92 120.73 0.00 0.00 3.73 0.2036 2981.22 997.17 0.75 0.50 5.82 1.1045 3853.52 1449.05 1.10 0.84 7.51 2.1154 12155.10 2051.22 7.08 1.68 21.71 4.6363 21971.80 610.34 13.60 0.49 36.34 1.6072 40208.98 10824.08 26.49 8.03 65.88 22.6981 71155.77 34313.10 40.96 11.32 98.74 32.79RandomWorstfailedtosolvetheSudokupuzzleonsevenoccasions: twicefor9, threetimesfor72, andtwicefor81. Additionally, 24attemptswerehaltedafterreachingthetimelimit. Foreachofthesefailures,thepuzzleswerefromthe63,72,and81sets. Unlikebacktracking, thenumberof guessesmadebyRandomWorst(Figure 30) did not peak at the 53 and 63 puzzles but increased steadily throughout.Theexactreasonforthisisunknown, butthesteadyincreaseisacommonthemethroughouttheCougaarrandomSudokuproblemsolvingtechniques.RandomChoice,RandomAgentRandomfailedtosolvetheSudokupuzzleonfouroccasions. Additionally, nineattempts were halted after reaching the time limit. For each of these failures, puzzleswere from the 63, 72, and 81 sets. The overall performance of Random (Table 9) wasslightlybetterthanRandomWorst(Table8)foramajorityofsets. Thisispossibly43Figure32. ExperimentTwoTimeResultsforRandomAgent.Figure33. ExperimentTwoGuessResultsforRandomAgent.Figure34. ExperimentTwoMessageResultsforRandomAgent.44Table9. RandomAgentExperimentTwo.Time Guesses MessagesAverage Average Average 0 304.12 102.64 0.00 0.00 0.59 0.339 1064.60 133.90 0.00 0.00 3.12 0.1418 1082.86 91.45 0.00 0.00 2.91 0.1127 1307.31 164.90 0.00 0.00 3.72 0.2236 3032.95 1104.55 0.75 0.50 5.78 1.2745 3803.89 1555.45 1.12 0.83 7.35 1.8754 13299.98 2327.06 7.60 1.71 21.71 3.8363 22461.45 367.69 13.93 0.35 35.75 0.8072 36548.60 3036.95 23.39 2.12 54.41 5.9781 52775.97 3917.06 34.11 2.27 76.10 7.33causedbyRandoms access toslightlybetter agents toguess fromthanRandomWorst.RandomChoice,RandomBestAgentFigure35. ExperimentTwoTimeResultsforRandomBestAgent.RandomBestfailedtosolvetheSudokupuzzleonfouroccasions. Noattemptswerehaltedduetoreachingthetimelimit. RandomBestsuccessfullysolvedeachof theSudokupuzzlesinthistestsetwithin80seconds. RandomBestappearsto45Figure36. ExperimentTwoGuessResultsforRandomBestAgent.Figure37. ExperimentTwoMessageResultsforRandomBestAgent.Table10. RandomBestAgentExperimentTwo.Time Guesses MessagesAverage Average Average 0 263.83 79.57 0.00 0.00 0.35 0.189 1020.47 23.97 0.00 0.00 2.96 0.0818 1020.03 79.94 0.00 0.00 2.79 0.0927 1207.97 92.48 0.00 0.00 3.47 0.1236 2924.35 874.62 0.75 0.50 5.69 1.0745 3991.12 1594.27 1.15 0.86 7.45 1.9254 15594.60 2055.84 8.90 1.47 23.61 2.6763 33836.05 2551.76 21.25 1.64 48.10 3.3572 54349.30 729.20 34.95 0.21 74.99 0.6781 70580.57 1807.07 45.60 1.20 95.33 2.4046performmorepoorly(Table10)thanRandomWorst(Table8)andRandom(Table9), butthesetablestonottakeintoconsiderationattemptsprematurelyendedbyreachingthetimelimit.Table11. TimeLimitandCommunicationFailures.TimeLimitedFailures CommunicationFailuresRandomWorst 24 7Random 9 4RandomBest 0 4Table 11 shows that Random Best has a better overall performance than RandomWorstorRandom,oncetime-limitedfailuresaretakenintoconsideration. Inthesetof solutions excluding time-limited failures, Table 8 shows that selecting guesses fromthesetofleast-constrainedcellagentstendstoleadtosolutionswithfewerguesses.ConclusionsAswiththeEasy/Medium/Hard/Evil Sudokutestset, backtrackingandDLXreached a solution much more quickly than any of the random methods. The Cougaarrandommethods requiredsignicantlyfewer guesses thanbacktracking. RandomBestwasagainthesuperiorrandomSudokuproblemsolvingmethod,butonlyaftertime-limitedfailureswereconsidered.47FinalAnalysisThesetests clearlyshowthebruteforcesolvingmethodusedbybacktrackingwas the fastest. Random Best consistently produced solutions using the least numberofguesses. TheperformanceofRandomWorstandRandomBestontherandomlygeneratedSudokupuzzlesindicatedthatagoodguessingstrategycouldbedevisedbyrestrictingthesetof agentsavailableforguessing. Althoughseveral of RandomWorstsattemptsexceededthetimelimit, therest werecompletedwithrelativelyfew guesses. Diering performances of DLX, backtracking, and the Cougaar approachshowed that certain Sudoku puzzles were more suitable for a particular solving methodthan others. However, the exact cause of this was not apparent. Overall, the CougaardistributedagentapproachdemonstrateditselfcapableofsolvingallSudokupuzzlestested.TheCougaarapproachshowedweaknessinoneareaspecically. Occasionally,messageswouldnotbedeliveredtotheintendedagent, orwouldarrivelaterthanusual. To compensate for this, time delays and the resending of messages was needed.In some cases, an agent would cease to receive messages entirely. Only restarting thenode containing the agent could restore communication with the agent. These factorscausedtheoverheadneededformessagepassingtogreatlyoutweighthetimespentactuallyexecutingcodeattheagentlevel. Additional researchisneededtosearchforasolutiontotheseproblems.48FUTUREWORKRoleoftheBoardMasterCurrently, the boardmaster fullls several roles inthe system. It loads newboards,logsresults,keepscellagentssynchronized,providesinterfacesforusercon-trol, and handles guessing. To create a more interesting environment for study, someof these tasks could be shifted from the board master to cell agents or communities ofcell agents. For example, the cell agents could be individually responsible for makingsuretheirneighborsaresynchronizedwitheachother. Communitiesof cell agentscouldhandlestalldetection,guessing,androllingback.Pre-guessAnalysisSomeSudokufansareoftheopinionthatanywell-formedSudokupuzzlecanbesolvedwithoutguessing. Thecurrentimplementationof theAgentMethodresortstoguessingearlyon. Certainlytheboardmasterorindividual cell agentscoulddomoreanalysistodelayoreliminatetheneedforaguess. Certaintypesofanalysis,suchaslookingatgroupsofrelatedcellswhichsharethesamen-choices, wouldbeparticularlysuitedtoanagent-basedsolver.49GuessTechniquesThe potential for solutions with fewer guesses was demonstrated for the RandomWorst method. This could be explored by changing the random number selected fromarandomworstagenttosomexedselection, suchasrstorlast. UndocumentedtestsonacompletelyemptySudokupuzzleseemtoindicatethatarstchoicefromarandomworstagentleadstoasolutionwithfewerguessesthananyoftherandomSudokuproblemsolvingmethodsdiscussedinthispaper. Apossiblecauseofthisisthepotentialguesssizereductionresultingfromtakingtherstorlastguess.PerformanceMeasurementAs mentioned in the Measurable Results section, measuring the time required bytheAgentMethodtosolveaSudokupuzzlecurrentlydoesnotyieldausefulmetricfor measuring performance. Time spent waiting for additional messages quickly over-whelmstimespentactuallysolvingthepuzzle. Separatingtherecordedworktimesfromtherecordedwaittimesmaypotentiallyyieldmeaningful datathatcouldbecomparedtootherSudokusolvingprograms.FaultToleranceFaulttoleranceiscritical tothesuccessofanyagentsystemwherecommunica-tionsmaybelostorindividual agentsmaybecomeunresponsive. MessagesendinginCougaarisgenerallyveryfaulttolerant. Theboardmasterkeepstrackof what50itexpectstohearfromeachcell agent, sothatalostmessagecanbere-sentoranout-of-sync cell agent can be reset. In what is likely to be a Cougaar implementationissue,arandomagent maybecomeunresponsive. Theagentwillcontinuetooperatebutcannolongersendorreceivemessages. Whenthisproblemoccurs, thesystemcompletelyfails. Everyagentmustberesponsiveforasolutiontobereached. Im-plementingafunctionwheretheboardmasterdetectsdeadcellagentsandrestartsthemcouldbeapotential solutiontotheproblem. AnothersolutionwouldbetouseanenvironmentwhereeachcellagentresidesinaseparateCougaarinstance,sothatanagentsentireCougaarinstancecouldberestarted. Fortheexperiments,all82 agents were in the same Cougaar instance therefore restarting the instance meantrestartingtheentiretestsequence.PuzzleSubtletiesTheexperiments showedthat specicSudokupuzzles weremoreeasilysolvedbyoneSudokuproblemsolvingtechniqueover another. For example, Evil 1wasmostdicultforbacktrackingwhileEvil 5wasmostdicultforDLX. Theexactreasons causing a Sudoku puzzle to be more easily solved using one technique insteadofanotherwouldbeanexcellenttopicforfurtherresearch.51REFERENCESCITED[1] Websudoku.Availableat: http://www.websudoku.com.[2] Puzzlecommunicationnikoli.Availableat: http://www.nikoli.co.jp/en/.[3] Dellmagazines.Availableat: http://www.dellmagazines.com/.[4] M.Kraitchik. Mathematical Recreations,page178. W.W.Norton,1942.[5] T.YatoandT.Seta. Complexityandcompletenessofndinganothersolutionand its application to puzzles.IEICE TRANSACTIONS on Fundamentals ofElectronics, CommunicationsandComputerSciences, E86-A(5):10521060,2003.[6] BertramFelgenhauer andFrazer Jarvis. Mathematics of sudokui. Preprint,2006.[7] RobertoTamassiaandMichael Goodrich. AlgorithmDesign, pages 627631.PWSPublishingCompany,2002.[8] DonaldKnuth. Dancinglinks. Millennial Perspectives inComputer Science,pages187214,2000.[9] Peter Gordon and Frank Longo.Mensa Guide to Solving Sudoku.Sterling, 2006.[10] AaronHelsinger, Michael Thome, andToddWright. Cougaar: Ascalable, dis-tributed multi-agent architecture. In IEEEInternationalConferenceonSys-tems,Man,andCybernetics,pages19101917,2004.[11] Michael Thome, ToddWright, andSebastienRosset. Ultralog: Thecougaarcomponentmodel. Technicalreport,BBNTechnologies,2004.[12] BBNTechnologies. Cougaardevelopers guide. Technicalreport,BBNTechnolo-gies,2004.[13] Thetransportationsecurityadministration.Availableat: http://www.tsa.gov/.52[14] Rockymountainagilevirtualenterprisesatmontanatech.Availableat: http://www.mtech.edu/.[15] Michael Emery, JohnPaxton, andRichardDonovan. Applicationandtestingof a cougaar agent-based architecture. In ComputationalIntelligence: Multi-AgentSystems,pages153157,2005.[16] PhilipCohen, AdamCheyer, MichelleWang, andSoonChoel Baeg. Anopenagentarchitecture. InAAAISpringSymposium,pages18,1994.[17] JonCurrey. Real-timecorbatheoryandpractice: Astandards-basedapproachtothedevelopmentof distributedreal-timesystems. InEmbeddedSystemsConferenceSanJose,2000.[18] Sunmicrosystems.Availableat: http://www.sun.com/.[19] Solutionsiq.Availableat: http://www.solutionsiq.com/.[20] FrankSommers. Dynamicclusteringwithjini technology. ArtimaDeveloper,2006.[21] K. Sycara, M. Paolucci, M. vanVelsen, andJ. A. Giampapa. Theretsinamasinfrastructure. AutonomousAgentsandMAS,7(1,2),2003.[22] JereyBradshaw, Stewart Duteld, Pete Benoit, andJohnWoolley. KAoS:TowardanIndustrial-strengthOpenAgentArchitecture, pages 375418. MITPress,1997.[23] RonaldSnyderandDouglasMacKenzie. Murdoch: Publish/subscribetaskallo-cationforheterogeneousagents. InProceedingsOpenCougaar, pages143147,2004.[24] TimLindholmand Frank Yellin. The Java Virtual Machine Specication.Addison-Wesley,1999.[25] Brian Gerkey and Maja Mataric. Murdoch: Publish/subscribe task allocation forheterogeneous agents. In ProceedingsofAutonomousAgents, pages 203204,2000.53[26] TheFoundationforIntelligentPhysical Agents. Fipaagentmessagetransportservicespecication. Technicalreport,TheFoundationforIntelligentPhysi-calAgents,2001.[27] BBNTechnologies. Cougaar architecture document. Technical report, BBNTechnologies,2004.[28] ToddWright. Namingservices inmulti-agent systems: Adesignfor agent-basedwhite pages. InProceedings 3rdInternational Joint Conference onAutonomousAgentsandMultiagentSystems,pages14781479,2004.[29] MarkBedau. Weakemergence. Philosophical Perspectives: Mind, Causation,andWorld,11:375399,1997.