Sourcing and Matching: Artificial Intelligence vs. Human Cognition

86
Sourcing and Matching Glen Cathey V.P. Recrui3ng, Kforce

description

Glen Cathey's presentation from Sourcecon 2010.

Transcript of Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Page 1: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Sourcing and Matching Glen Cathey V.P. Recrui3ng, Kforce 

Page 2: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

What’s the big deal? 

Page 3: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Hidden Talent Pools can be at least  30‐40% of each database/source 

Page 4: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

“When every business has free and ubiquitous data, the ability to understand it and extract value from it becomes the complimentary scarce factor. It leads to intelligence, and the intelligent business is the successful business, regardless of its size. Data is the sword of the 21st century, those who wield it well, the Samurai.” 

                        ‐Jonathan Rosenberg, SVP, Product Management @ Google 

Page 5: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Finding people is easy… 

Page 6: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Finding the right people IS NOT! 

Page 7: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Stop wasting time trying to create difficult and complex Boolean search strings 

  Let “intelligent search and match applications” do the work for you 

  A single query will give you the results you need ‐ no more re‐querying, no more waste of time! 

Page 8: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

How do they really work? 

Page 9: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Intuit experience by context = resume parsing 

  Parsing breaks down and extracts resume information  

Page 10: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Well developed ontologies and taxonomies   Hierarchical 

Page 11: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Synonymous taxonomy   Programmer, Software Engineer, Developer   Tax Manager, Manager of Tax   CSR, Customer Service Representative   Ruby on Rails, RoR, Rails, Ruby   Oracle Financials, Oracle Applications, e‐Business Suite, etc. 

Page 12: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Some applications use complex statistical methods in an attempt to "understand" language and relationships between words 

Page 13: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Keywords with the same or similar meanings in a natural language sense tend to be "close" in units of Google distance, while words with dissimilar meanings tend to be farther apart 

Page 14: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Non‐interactive and unsupervised machine learning technique seeking to automatically analyze and define relationships between words and concepts 

Page 15: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The design and development of algorithms that allow computers to evolve behaviors based on empirical data 

  A major focus is to automatically learn to recognize complex patterns and make intelligent decisions based on data 

Page 16: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Aims to classify data (patterns) based either on a priori knowledge or on statistical information extracted from the patterns   Ex. Computer‐aided‐diagnosis, spam/non‐spam 

Page 17: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Finds approximate matches to a pattern in a string 

  Useful for word and phrase variations and misspellings 

Page 18: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

PROS and CONS 

Page 19: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Can lessen or eliminate the need for recruiters to have deep and specialized knowledge within an industry or skill set 

  Reduce and even eliminate time spent on research 

  Reduce time to find relevant matches 

Page 20: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Go beyond literal, identical lexical matching 

  Levels the playing field 

  Can make an inexperienced person look like a sourcing wizard 

Page 21: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Good for simple/straightforward positions, where title matching works well and there is a low volume and variety of keywords 

  Good for a high volume of unchanging hiring needs 

  Good for teams with low search/sourcing capability  

Page 22: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Removes thought from the talent identification  process 

  Danger of eliminating the need for recruiters to understand what they’re searching for 

  Information technology, healthcare, and other sectors/verticals can create pose serious challenges 

Page 23: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Apps find some people, miss or bury others 

  Level the playing field 

  Different users find and miss the same people 

Page 24: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Belief that one search finds all of the best candidates is intrinsically flawed 

  Best candidates are not necessarily the ones with the “best” resumes 

  Can favor keyword rich resumes  

Page 25: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

 Sr. Storage & Systems Administrator.  The current environment is SAP & ORACLE on EMC storage and Unix Operating Systems. The current production database size is 9TB and total storage capacity in the SAN environment is more then 200TB spread across EMC frames like DMX3 & DMX2000, DMX800, 8830, 8730, 8430, CX3 & IBM shark F20 & 800.  

 All SAN storage is connected through the Mcdata 140 & 6064 director level switches and CISCO MDS 9000. Currently 8 switches are configured in the environment. 

 Primarily administering and managing SAN storage and secondary all high‐end servers.  Configured all the EMC storage, creating Symdevices, converting Symdevices, and creating Metas, allocating/Unallocating to FA, Configuring BCV and SRDF using the Symconfigure manager & ECC 5.2. Creating zoning on MCDATA Switches using ECC as well as EFC connectrix manager software. 

Page 26: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

 Storage Area Network (SAN) Engineer (Infrastructure Specialist)        In this position I design, develop and test numerous storage solutions utilizing Network Appliance and EMC products for the largest enterprise active directory environment in the world (NMCI). I am responsible for meeting life cycle deliverables and the creation of engineering documentation for use during implementation. I also participate in the engineering of incentive "proof of concept" projects for potential corporate revenue for the next fiscal year. Some of these projects include Enterprise Quota management of file services and Distributed File Service (DFS) utilizing Windows 2003 R2 Server. This position requires interfacing with numerous vendors while maintaining and developing bleeding edge SAN technology for a fast paced environment servicing the Navy and Marine Corps customers ensuring we provide quality services to meet their proposed requirements. 

Page 27: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Pre‐built taxonomies are limited in their completeness 

  Taxonomies are only as good as who created them 

  Cannot “think outside of the box” 

Page 28: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Related words may not be relevant  

Page 29: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Match primarily on titles and skill terms 

  Some applications rank results favoring recent employment duration 

  May not be legal/OFCCP compliant 

Page 30: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Apps are not aware of who they didn’t find 

  Apps don’t “know” what you’re looking for 

  Applications are not truly intelligent 

Page 31: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 32: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The capability of a machine to imitate intelligent human behavior  

  Artificial = humanly contrived 

Source: Merriam‐Webster.com 

Page 33: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The ability to learn or understand or to deal with new or trying situations: REASON; also: the skilled use of reason  

  The ability to apply knowledge to manipulate one’s environment or to think abstractly 

Source: Merriam‐Webster.com 

Page 34: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The power of comprehending, inferring, or thinking specifically in orderly rational ways  

  Infer = to derive as a conclusion from facts or premises  

Source: Merriam‐Webster.com 

Page 35: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Dr. Michio Kaku   Theoretical physicist and futurist specializing in string field theory 

  Currently working on completing Einstein's dream of a unified field theory 

  What are his thoughts on AI? 

Page 36: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 37: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  “…pattern recognition and common sense are the two most difficult, unsolved problems in artificial intelligence theory. Pattern recognition means the ability to see, hear, and to understand what you are seeing and understand what you are hearing. Common sense means your ability to make sense out of the world, which even children can perform.” 

                     ‐ Dr. Michio Kaku 

Page 38: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The job market of the future will be “dominated by jobs involving common sense (e.g. leadership, judgment, entertainment, art, analysis, creativity) and pattern recognition (e.g. vision and non‐repetitive jobs).  Jobs like brokers, tellers, agents, low level accountants and jobs involving inventory and repetition will be eliminated.”                      ‐ Dr. Michio Kaku 

Page 39: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Search and match applications do not possess any true cognitive power 

  Cognitive: involving conscious intellectual activity (as thinking, reasoning, or remembering) 

Source: Merriam‐Webster.com 

Page 40: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Matching apps do not have the dynamic ability to learn, understand and instantly relate new concepts and through direct experience and observation 

  They depend on taxonomies, statistical models, or semantic clustering to “understand” relationships and concepts 

Page 41: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The human mind naturally organizes its knowledge of the world  Carolinas HealthCare System, Charlo=e, NC  Infec3on Preven3onist 1997‐present 

 Responsible for all aspects of infec3on preven3on and control for an 800+bed hospital. Uses science‐based research to perform infec3on preven3on.  Conducts all aspects of surveillance, data analysis, and presents data to interdisciplinary teams, including the Infec&on Control Commi=ee. Uses computer‐based sta3s3cal analysis to present data.  Serves as an educa3onal resource for staff, physicians, pa3ents, and the community for infec3ous and communicable diseases.  

Page 42: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 43: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  To process natural language, applications use complex statistical methods in an attempt to make sense of human language 

  To computers, sentences can be highly ambiguous, yielding hundreds or thousands of possible analyses  

Page 44: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Humans automatically process and understand language, regardless of sentence length or complexity, ambiguity, incorrect grammar, etc. 

  We can udnretsnad any msseed up stnecene as lnog as the lsat and frsit lteetrs of wdros are in the crrcoet plaecs 

Page 45: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Human sourcers can deduce potential experience, even with contradictory evidence or the absence of information 

Page 46: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Applications are not “aware” of the issues associated with searching resumes  

  Human sourcers can specifically target Hidden Talent Pools 

Page 47: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

How do you target candidates your searches CAN’T find? 

Page 48: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Well developed taxonomies, semantically generated query clouds and matching algorithms 

Page 49: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Think + Perform Research   For every title or keyword you are thinking of 

using in your search, ask: 1. Would ALL candidates mention those titles and 

keywords? 2. How many ways could the experience you’re 

searching for be expressed?  

Page 50: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Global Experience 

What search terms might you use if you are looking for people with global experience? 

Page 51: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Global, international, foreign, multinational, worldwide 

  Europe, European, EU, EMEA, Asia, Asia‐Pac, Pacific Rim, South America, Latin America, Americas, CALA (Caribbean and Latin America), Middle East 

  Canada, Japan, China, Russia, India, UK, United Kingdom, etc. 

  Countries, Offshore, Overseas 

Page 52: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

How do you target candidates your searches DON’T find? 

Page 53: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 54: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

Run Multiple Searches   Start with maximum qualifications 

  Use the NOT operator to systematically filter through mutually exclusive result sets 

  End with minimum qualifications 

Page 55: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 56: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Required: A,B,C 

  Explicitly desired: D,E 

  Implicitly desired: F 

Page 57: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

1.  A and B and C and D and E and F 

2.  A and B and C and D and E and NOT F 

3.  A and B and C and D and NOT E and F 

4.  A and B and C and NOT D and E and F 

5.  A and B and C and NOT D and NOT E and F 

6.  A and B and C and D and NOT E and NOT F 

7.  A and B and C and NOT D and E and NOT F 

8.  A and B and C and NOT D and NOT E and NOT F 

Page 58: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 59: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  A mix of “man and machine,” integrating human knowledge and expertise into computer systems 

  Essentially ‐ the best of both worlds:    Autopilot   Manual override 

Page 60: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  AI/semantic matching engine 

  Taxonomies built by human SMEs that are continually modified and improved specifically for the organization 

Page 61: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Resume/profile parsing 

  White box relevance weighting 

  Searchable tagging 

Page 62: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Standard and extended Boolean in full text and field‐based search 

  AND, OR, NOT  

  Configurable proximity 

  Variable term weighting 

Page 63: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Lucene and dtSearch are text search engines that support configurable proximity and term weighting  

Page 64: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Resume Mirror 

  Burning Glass 

  Sovren 

Page 65: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

“Society has reached the point where one can push a button and immediately be deluged with…information. This is all very convenient, of course, but if one is not careful there is a danger of losing the ability to think.” 

   ‐ Eiji Toyoda, Former President of Toyota Motor Corp 

Page 66: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Information requires analysis  

  Matching apps move/retrieve information, but only PEOPLE can analyze and interpret for relevance 

Page 67: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Sourcers and recruiters need technology that can enable their productivity 

  Intelligent search and match apps are not a replacement for creative, curious, analytical, investigative people  

  Do not seek to automate that which you do not understand and cannot accomplish manually! 

Page 68: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

“Computers move information, people do the work” 

   ‐ Jeffrey Liker, Author of The Toyota Way 

Page 69: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 70: Sourcing and Matching: Artificial Intelligence vs. Human Cognition
Page 71: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  “The holy grail of search is to understand what the user wants. Then you’re not matching words; you’re matching meaning.”  

           – Amit Singhal  

  Can applications ever really know what we’re looking for?  

Page 72: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  The study of meaning, inherent at the levels of words, phrases, and sentences 

  5 Levels of semantic search 

Page 73: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

1.  Skill words/title association, variants, and misspellings 

  Director of business development, business development director, etc. 

  JDE, JD Edwards, etc.   10Q = SEC reporting   SAP = ERP   JMPC, JP Morgan, JPMorganChase 

Page 74: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

2.  Contextual   Specifically where words, titles and skills are 

mentioned in resumes    Summary, education, recent work experience…   Education vs. address (Harvard Ave.)  

Page 75: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

3.  Grammatical, natural language search 

  3 full life cycle SAP R/3 implementations 

  Carry out wound (pressure ulcer) assessment, recommend treatment… 

  SOX compliancy weekly internal auditing 

  Perform investment performance and attribution analysis 

Page 76: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  support* near (CEO or CFO or CTO or CIO or "C‐Level" or chief*) 

Page 77: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  config* near juniper near router* 

Page 78: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  “created access database”~7 

Page 79: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

4.  Implied skills, experience and responsibilities 

  Data center migrations  possibility of SAN, and possibly EMC specifically  

  Working at Equant  possibility of PeopleSoft experience 

Page 80: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

5.  Human‐reviewed and classified  

  Candidate records (resumes, profiles, etc.) that have been identified, analyzed and labeled 

  Tagging 

Page 81: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Using human capital data for talent discovery and identification  

  5 Levels  

Page 82: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Straight lexical matching of job titles and required skill terms  

  Deep understanding of positions not necessary   Buzzword bingo   Superficial match   “Once and done”    Easily outsourced and automated 

Page 83: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Beyond literal lexical matching  

  Involves:   Interpretive analysis of the need (explicit/implicit)   Synonymous terms and concepts    The analysis of the relevance of the initial search 

results and the adaptive process of learning from the results to creatively refine searches   

  Awareness of candidates excluded/not found 

Page 84: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Searching for and identifying what isn’t explicitly mentioned 

  Involves:   Understanding of intrinsic limitations of 

resumes and profiles   Skill that can only be developed over time from 

observation and experience  

Page 85: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Searching for responsibilities and capabilities, not keywords  

  Involves:   Semantic search – specifically targeting what 

people DO, not just what they SAY 

Page 86: Sourcing and Matching: Artificial Intelligence vs. Human Cognition

  Searching for the “wrong” people to find the “right” people  

  Involves:   Targeting under/overqualified professionals    Targeting people who likely work with or know 

the professionals you need