Develop  Your  Analysts,  and  They’ll  Pay  for  Themselves        

 Develop  Your  Analysts,  and  They’ll  Pay  for  Themselves  [00:01]    [Peter  Monaco]  Hello  everyone  and  welcome  to  our  webinar.    So   I’m  here  with  Russ,  he’s  here,  and  going   to  help  me  answer  some  questions  and  then  also  fire  me  if  I  say  something  stupid.        

 Today’s  Agenda  [00:13]    So   to   start,   let’s  go  over   the  agenda.     There’s  a   lot  of  ways  obviously   to   cover   this   topic  and  we’ll   be   talking   about   five   –   culture,   the   processes   that   your   analysts   can   be   a   part   of,   the  different   skillsets   that   your   analysts   need,   the   type   of   involvement   that   they   need   in   your  organization,  and  finally  the  barriers,  basically  getting  in  and  out  of  the  way  for  them.      

 Poll  Question  #1  If  you  could  hire  only  one  analyst,  would  you  pick:  a)  A  domain  expert;  b)  A  data  wizard;  or  c)  A  visualization  guru  [00:46]    But   before  we  begin,   I  wanted   to   ask   one   poll   question.     If   you   could   hire   only   one   analyst,  would  you  pick  a  domain  expert,  a  data  wizard,  or  a  visualization  guru?    [Facilitator]  Alright.     Peter,  we’ve  got  poll   question  up   right  now.     So,   if   you   could  hire  only  one  analyst,  would  you  pick  a  domain  expert,  a  data  wizard,  or  a  visualization  guru?    We’ll  leave  that  open  for  a  few  moments  to  give  everyone  that’s  on  the  line  a  chance  to  respond.    Alright.    Let’s  go  ahead  and  close  that  poll  and  show  the  results.    Well  Peter,  it  looks  like  we’ve  got   21   percent   responded   a   domain   expert,   54   percent   responded   a   data   wizard,   and   24  percent  responded  a  visualization  guru.        [Peter  Monaco]  That’s   actually   really   interesting.     So   we   have   the   chance   to   ask   the   same   question   at   the  Healthcare  Analytics  Summit.    And  the  results  there  were  a  little  bit  different  and  swung  more  in   the   direction   of   a   domain   expert.     And   so,   I   think   this   kind   of   gets   that   there   is   a   lot   of  

different  ways  where  we  think  we  can  stuff  the  perfect  analyst.    And  I  hope  to  cover  some  ways  in  which  we  can  develop  analysts  across  all  three  of  these  areas.      

 What  do  we  mean  by  “analyst”?  [02:11]    But   first,   we   should   talk   about   what   do   we   mean   actually   by   an   analyst.     There   are   really  multiple  terms  that  encompass  these  same  skill  sets.    We  can  talk  about  them  in  terms  of  being  architects,   business   intelligence  developers,   analysts   obviously,   and   the  one   I   relate   to  most,  being   nerds.     But   they   all   revolve   around   the   same   basic   skillsets,   that   is   talking   to   and  manipulating  data  using  SQL,  ETL,  modeling,  but  then  also  analyzing  and  communicating  those  results   through   some   visualization   tool,   Excel,   or   domain   expertise.       We   will   use   the   term  “analyst”   to   describe   all   of   these.     So   really   think   about  more   the   skills   versus   the   titles   and  what  it  means  in  your  organization.      

 Culture  An  environment  fostering  improvement  [03:09]    So  with  that,  let’s  talk  about  how  they  fit  in  to  your  culture.          

 Motivating  with  value  and  purpose  [03:14]    One  of  the  things  that  we  realize   is   that  you  need  to  motivate  them  with  value  and  purpose.    There  is  this  incredible  talk  on  what  motivates  us  that  was  given  by  Dan  Pink  in  a  TED  talk,  and  he  talks  about  what  are  the  things  that  actually  motivate  someone.    And  money  is  what  most  of  us  think  of,  but  not  in  the  way  that  you  think.    It’s  not  pay  us  millions  but  pay  us  enough  to  take  money  off  the  table.    Get  them  thinking  about  the  actual  work  versus  the  pay.    It’s  a  reflection  of  how  much  analysts  are  valued  in  your  organization,  it  helps  keep  good  analysts  around.    And  we’re  not  talking  again  paying  in  the  99th  percentile  of  the  pay  range  but  something  closer  to  60  to  75  percent.    They  are  not  trying  to  pay  them  as  little  as  possible  but  just  enough  so  that  they  are  focused  more  on  the  work.    With  that,  he  also  talks  about  these  two  motivators  –  the  first  one  being  autonomy,  the  desire  to  be  felt  directed;  the  next  one  mastery,  the  urge  to  get  better  at  stuff;  and  finally  purpose.    We’ll  talk  more  later  on  autonomy  and  mastery  but  now  I  want  to  focus  on  purpose  and  how  it  rolls  into  the  organization.      

 Purpose  [04:42]    It’s  this  idea  from  Steve  Jobs  that  gives  us  some  insight  into  how  we  need  to  be  thinking  about  an  analyst’s   role.    “It  doesn’t  make   sense   to  hire   smart  people  and   tell   them  what   to  do,  but  hiring   smart   people   to   tell   us  what   to   do.”    Well   I’m   not   saying   the   analysts   should   rule   the  organization.     But   if   you   think   about   where   they   sit   in   your   org,   analysts   create   analytics.    They’re   looking  into  the  data  that’s  being  generated  from  the  organization.    But  they  need  to  actually   be   leveraged   by   people   who   are   ready,   able,   and   eager   to   use   it.     It   needs   to   be  adopted.          

 Analytics  –  Best  Practices  –  Adoption  –  Outcomes  Improvement  [05:25]    And   if   you   think   about   where   these   analysts   sit   in   terms   of   outcomes   improvement,   this   is  where  they  can  start  to  derive  that  purpose.    The  best  analytics  and  the  best  best  practices  are  really   nothing   without   that   adoption,   without   those   people   who   are   going   to   use   those  analytics   on   the   frontlines   to   change   behavior   and   therefore   improve   outcomes   –   because  outcomes  improvement  will  stall  without  this  adoption.      

 The  role  of  an  analyst  is  outcomes  improvement  not  “building  a  dashboard”  [05:53]    So   surround  your  analysts  with  people  who  are  going   to  use   the  value   they  provide  because  really  the  role  of  an  analyst  is  outcomes  improvement,  not  “building  a  dashboard”.    I’ve  worked  with  nursing  directors  who  have  sat  down  with  me  for  hours  to  really  explain  their  passion  for  the  work,  why  it  matters,  and  why  they  are  so  excited  to  use  data  to  change.    So  you  need  to  surround  your  analysts  with  people  who  are  data-­‐driven.      

 Processes    Fail  fast  [06:20]    So   once   you’ve   established   the   culture,   there   are   some   processes   that   could   really   stunt   or  delay  development  of  analysts.          

 Poll  Question  #2    If   data   surfaced   showing   a   provider’s   dismal   performance,   how   comfortable  would   you   be  presenting  the  data  in  a  meeting  with  that  provider  present?  [06:30]    But  first,  another  poll  question.    So  if  data  surfaced  showing  a  provider’s  dismal  performance,  how  comfortable  would  you  be  presenting  the  data  in  a  meeting  with  that  provider  present?    [Facilitator]  Alright.    We’ve  got  that  poll  launched.    If  data  surfaced  showing  provider’s  poor  performance,  how  comfortable  would  you  be  presenting  the  data   in  a  meeting  with  that  provider  present?    Would  you  be  very  comfortable,  slightly  comfortable,   it  depends  on  the  provider,   I’d  hide  the  provider  names,  or  I’d  call  in  sick?    So  we’ll   leave   that   open   for   a   few  moments   and   also   remind   everyone   that   if   you   have   any  questions  or  comments,  please  be  sure  to  use  the  questions  pane  in  your  control  panel.        Alright.    Let’s  go  ahead  and  close  this  poll  and  see  how  folks  have  responded.    

Alright.     It   looks   like  29  percent  said  they  would  be  very  comfortable,  18  percent  said  slightly  comfortable,  34  percent  said  depends  on  the  provider,  18  percent  I’d  hide  the  provider  names,  and  1  percent  did  say  they’d  call  in  sick.          

 Poll  Question  #2  [07:33]    [Peter  Monaco]  That   seems   to   really   match   up   well   with   the   same   question   we   asked   at   the   Healthcare  Analytics  Summit.    So  we  asked  that  same  question  in  terms  of  2  percent  of  the  person  actually  attending,  HAS  and  the  analyst.    And  what  we  saw  is  that  those  attending  may  not  have  been  analysts.    So  I  think  it  was  a  less  heavy  audience  in  terms  of  the  analyst  position,  would  be  more  comfortable  than  the  actual  analyst   in  providing  this   information   in  the  meeting.    And   I   think  from  this,  you  can  see  that   it’s  really  kind  of  hitting  this   in  terms  of  comfortability,  whether  a  good  chunk,  almost  a  third,  are  very  comfortable,  but  then  there’s  almost  50  percent  where  it  really  depends.          

 Analytics  are  still  scary  [08:23]    And  I  think  that  means  analytics  are  still  scary,  even  in  mature  organizations.    So  there’s  some  things  to  remember.    One,  you  need  a  baseline.    You  need  to  measure  if  you  want  to  improve.    So  you  need  that  starting  point.    The  next  is  that  analysts  are  only  the  intermediaries  between  the  data  and  the  people  creating  it.    They’re  the  “messengers”.    And  the  last  one,  which  I’ll  then  emphasize  most  is  that  errors  are  a  part  of  this  process.    So  to  kind  of  get  at  that  poll  question.    It’s  why  we  use  factors  of  agile  development  and  it’s  something  that  we  use  as  really  a  trouble  detector.    We  want  to  have  full  scheme  into  trouble  so  that  in  the  context  of  finding  truth  in  the  data,  bad  news  is  good  news.      

 Failing  Slow  [09:15]    If  something  is  going  to  fail,  you  want  it  to  fail  as  quickly  as  people,  so  that  we  can  get  closer  and  closer  to  not  the  right  answer  but  the  best  answer.    So   one   way   to   think   about   it   is   the   opposite.     So   if   we   talk   about   failing   slow,   with   tight  boundaries   of   failure,   with   implied   consequences,   if   you   fail,   it   changes   the   process   from   a  learning  opportunity  to  a   long  slow  progress,  where  really   interest  and  momentum  go  to  die.    Do   it   right   the  first   time  sends  the  wrong  message  to  your  analyst  and  your  organization  and  says  we  can’t  experiment,  so  we  can’t  learn  from  our  mistakes  and  we  can’t  deviate  from  some  plan  that  was  originated  before  we  had  data  to  support  it.    So  what  happens  when  we  champion  failing  fast?      

 Failing  Fast  [10:14]    When  you  give  your  analysts  some  flexibility  to  really  hone  their  problem  solving  skills  and  use  them   and   you   describe   failure   as   something   else,   then   suddenly   the   processes   shifted   from  getting  the  right  answer  to  getting  the  best  answer  and  that  those  involved  have  collaborated  together  from  the  start  and  actually  achieved  something  as  a  team,  rather  on  deciding  logic  or  rules   without   first   seeing   data.     And   really   you’re   robbing   yourself   if   you’re   an   analyst   and  you’re  robbing  your  organization  of  opportunities  to  develop.      

 The  answer  you  find  through  failure  is  usually  different  and  more  useful  Than  the  one  you  find  avoiding  it  [10:56]    And  what  we’ve  seen  is  really  interesting  is  that  the  answer  you  find  through  failure  is  usually  different  and  more  useful  than  the  ones  you’ll  find  in  avoiding  it.    So,  some  examples  of  these  are  showing  a  dashboard  of  data  at  the  very  first  meeting,  if  you  have  data  to  support  it.    Giving  stakeholders  something  to  actually  respond  to  versus  thinking  theoretically  about  how  to  solve  the  problem.    Give  them  something  to  hate,  something  to  criticize.    Otherwise,  you’re  going  to  spend   weeks   of   defining   a   cohort   or   the   best   way   to   calculate   or   measure   and   over   time  everyone  will  start  to  lose  interest  and  probably  stop  showing  up  to  the  meeting.    The  shot  in  the  dark  helps  everyone  realize  that  it  doesn’t  have  to  be  perfect  to  get  started.    And  really  if  you  are  avoiding  surfacing  something,  that’s  a  pretty  good  indicator  that  it  means  it’s  the  first  thing   that   needs   to  be   surfaced.     If   it’s   someone   that   you’re   afraid   to   invite   to   a  meeting,   it  probably  means  they  are  the  first  person  you  should  invite  to  the  meeting.      

 Skillsets    Cultivate  the  right  skills  [11:53]    So,  what   are   the   skillsets   that   need   to   be   developing   in   your   analysts,   now   that   you   cannot  define  the  culture  and  some  processes  that  will  enable  them  to  develop  more  quickly.      

 Does  this  look  familiar?  [12:07]    As  I  walk  through  this  little  animation,  I  want  you  to  see  if  this  resonates  with  any  of  you.    You  have  an  architect  working  on  something,  a  BI  developer  waiting  for  something,  an  end  user  not  providing  feedback  until  the  product  is  actually  polished  and  finished  and  everyone  is  okay  with  it.          

 Starting  knowledge  by  role…  [12:29]    Everyone   is   really   waiting   for   this   waterfall   to   reach   them   rather   than   iterating   and  communicating  along  the  way.    Not  only  that,  but  we    found  that  oftentimes  each  silo  may  start  thinking  of  another  person  as  lazy  or  incompetent  simply  because  they  don’t  understand  what  each  other  is  doing.    This  might  be  how  people  are   thought  of   in  your  organization   in   terms  of   their   skillsets.    We  have  a  data  architect,  an  outcomes  analyst,  a  Business   Intelligence.    There  might  be  more  or  less  with  different  roles.    But  really  they’re  kind  of  siloed  into  their  area  and  their  specialty.    And  what   we’ve   recommended   is   that   over   the   long   term,   you   encourage   skillsets   across   the  spectrum.    This  can  include  areas  like  predictive  analytics  and  statistics  sheet  where  you  start  to  actually   build   out   and   encourage   being   aware   of   those   skills.     Your   members   might   have  preferences  and  weaknesses  but  they  should  be  able  to  handle  and  talk  about  each  area  with  familiarity.    And  I’m  not  saying  to  build  an  expert  across  everything  because  that  is  impossible  and   it’s   not   scalable.     But   really   to   encourage   understanding   of   each   area.     So   that’s   great,  Peter,   you   want   us   to   hire   an   expert.     But   let’s   talk   about   really   where   to   start   with   this   –  because  it  might  not  mean  a  single  person  but  a  team.      

 Questions  to  start…  [13:59]    So  what  are  the  skills  of  your  analysts?    Take  stock  of  the  skillsets  and  their  proficiency.    You  can  do  this  for  individual  analysts  but  also  do  it  for  your  team.    Does  your  team  cover  the  basis  in  all  of   these   areas   that   are   going   to  make   them   a  well-­‐honed  machine   in   terms   of   building   out  analytics.        Just   as   important   is   where   does   this   team   “sit”?     So,   when   I   worked   on   a   project   with   a  coworker   or   I   know   other   individuals   in   our   organization,   when   we’re   working   together   on  something,   we   sit   next   to   each   other   but   the   emphasis   is   on   building   a   data   model   or   a  visualization.     We   get   together   multiple   times   during   the   day   and   are   conferring   with   each  other,   building   out   the   back   and   front   end   simultaneously,   letting   each   partner   inform   the  other.     And   at   least   weekly,   sometimes   even   daily,   we   talk   to   stakeholders   and   users   for  feedback.        This  might  be   impossible  for  some  team  setups   if   they’re  remote  for  this  notion  of  “sit”.    But  really  you  want  to  make  them  function  as  much  like  a  single  person  that’s  possible.    So  having  daily   stand-­‐ups,   having   daily   calls,   having  more   room   sessions  with   pizza   and   energy   drinks.    These  are  all  very  good  things.        

 

 Why  a  “generalist”?  [15:24]    So  why  do  we  encourage  this  “generalist”?    There’s  a  few  things.    So,  I  know  I  get  a  lot  of  job  satisfaction   and   the   fact   that   I’m   able   to   shift   my   work   across   the   spectrum   and   in   a   self-­‐directed   way.     If   you   remember,   that   autonomy,   that   ability   to   be   self-­‐directed   is   deeply  satisfying.    But  maybe  more   importantly,   that   I   can  actually   start   to  walk  on  data  model,   the  functionality,  and  visualizations  and  match  those  to  a  user  request.        So  users  have  their  requirements,  their  ideas  for  what  they  need  to  solve  the  problem.    And  the  more  I’ve  been  able  to  build  out  skillsets  across  the  spectrum,  the  easier  it’s  been  to  translate  those  into  analytics  or  certain  visualizations  that  would  best  facilitate  that  answer.    And  we  all  know  that  depending  on  the  BI  tool,  the  data  model  used  for  each  of  these  could  be  drastically  different.    But  knowing  that  being  aware  of  that  will  help  me  realize  what  type  of  data  model  I  need  to  build  out  visualizations,  and  then  even  down  to  the  databases  and  schemas  that  I  can  get  a  feel  for  and  an  idea  of  the  actual  data  I  need  to  support  both.          

 [16:49]    We  know  that  all  of  this  can  be  at  last  in  translation  and  that’s  why  we  talk  about  making  it  as  seamless  as  possible  –  because  I  think  there’s  a  lot  of  examples  across  the  industries  where  that  complexity  of  simply  a  user  requirement  could  be  translated  in  so  many  different  ways.    And  I  think  especially   in  healthcare,   there’s   the  complexity   to  caring   for  patients   that   is  not  kind  to  hand  off.    And  on  a  smaller  scale,  the  same  thing  could  be  said  for  building  an  analytic  solution.    So  you  want  to  make  this  as  seamless  as  possible.      

 Building  Skillsets  [17:25]    So  let’s  talk  more  about  the  actual  skillsets  that  we  need  to  build  out.    So  how  do  we  build  out  areas  of  weakness?    Well,  one  of  the  first  things  you  can  actually  do,  and  this  might  come  into  hiring,  but  there  should  be  soft  skills  that  drive  your  culture.    I  think  I  can  safely  say  I  was  hired  purely   for   soft   skills   not   because   of   any   technical   prowess   or   domain   expertise.     I   had   no  healthcare   experience,   a   few   years   of   SQL   in   a   much   simpler   environment,   and   I   perhaps  stressed   the   truth   in   terms   of   having   visualization   experience.     But   think   of   soft   skills   as   a  support  services   for  more  technical  skillsets.    At  Health  Catalyst,  we  kind  of  encompass  these  three  of  smart,  hardworking,  and  humble,  and  they  drive  a   lot  of  our  hiring  physicians  to  the  point  where  that  will  turn  down  candidates  who  might  be  extremely  strong  in  one  area  but  lack  these  soft  skills.        In  terms  of  the  actual  hard  skillsets,  inch  your  way  towards  usefulness.    So  if  you’re  the  analyst,  you   need   to   define   the   resources   for   basic   training,   and   if   you’re   not   the   analyst,   build   out  those   resources   for   them.     So   you  need   to   inch   them  along.     So  having   this  basic   training  of  people   listings   going   towards  dummy  projects  or   exercises   and   training  modules   for   them   to  engage  in  on  the  way,  and  finally  times  where  they  actually  get  to  apply  it  to  real  projects.    This  could  be  something  like  having  them  rebuild  the  product  that’s  already  there,  having  them  walk  

through  the  model  and  the  visualization  and  go  through  that  full  spectrum,  have  them  redo  it,  or   have   them   take   the   first   shot   in   the   dark   on   visualization   or   building   a   metric   for   a  dashboard.     The   thing   you   always   need   to   remember   is   keeping   that   balance   between  shadowing  and  mentorship  and  best  practices  with  actually   letting  them  hone  their  skills   into  fire,   letting   them  drive   something   potentially   slightly   before   they’re   comfortable   doing   it,   so  that  they  can  start  to  really  hone  out  those  skills.    And  again,  we’re  talking  here  about  learning  opportunities  down  to  failure.    So  you  need  to  give  them  chances  to  fail  and  chances  to  learn.      

 Visual  Story  Telling  [20:02]    One  area,  probably  back  to  getting  a  special  emphasis  for  is  especially  due  to  its  minimization  in  our  poll  at  the  Healthcare  Analytics  Summit  and  then  also  in  the  one  we  just  went  through,  is  visual  story  telling.    So,  I  think  one  of  the  reasons  why  it’s  minimized  often  is  because  it  is  really  easy  to  make  things  hard  to  understand.    This  can  make  you  think  this  visualization  stuff  is  really  overrated.    To  you  that’s  not  helping.        I  would  like  you  to  think  of  this  map,  where  every  piece  of  information  available  is  on  the  map.    But  really  if  you  think  about  it,  do  you  need  to  know  the  land’s  features?    Do  you  need  to  know  these   town   names   that   have   nothing   to   do  with   the   actual   railway   stops?     Do   you   need   to  

know,  because  the  railways  are  drawn  to  the  exact  scale  and  exact  curvature?    No.    So  it  was  actually  redesigned  to  help  with  its  single  purpose.      

 Visual  story  telling  It  is  easy  to  make  things  hard  to  understand  [21:06]    Which  stop  do  I  get  off?     It’s  clean  and  it’s  not  the  scale,  because  the  scale  didn’t  matter.     Its  points  are  equidistant  because  that’s  you  need  to  know  how  many  stops  there,  not  how  far  part  they  are.    And  just  one  river  for  reference,  not  all  the  rivers.    

 Visual  Story  Telling  [21:27]    And  there’s  a  few  things  that  in  the  vein  of  that,  that  we  would  recommend  reading.    And  there  are  these  two  books  by  Edward  Tufte  and  Stephen  Few,  that  with  these  tools,  you’ll  be  able  to  really  design  simple  tools  with  profound  insights.      

 The  most  important  question    Should  I  use  a  pie  chart?  [21:44]    And  perhaps  answer  the  most  important  question  of  should  you  use  a  pie  chart?    To  which  I  will  borrow  an  answer  from  David  Thorne  who  is  a  famous  comedy  writer.    The  answer  is  no,  you  should  not.    And  I’m  mostly  kidding  because  it  can  make  sense  with  some  instances.    But  with  those  two  books,  you  will  actually  be  able  to  figure  out  when  those  instances  will  make  sense  and  when  you  should  utilize  this  chart.    Everyone  loves  to  hate  on  the  pie  chart.      

 Involvement  Don’t  minimize  the  analyst  role  [22:14]    Another  thing.    So  now  that  you  have  your  analysts  and  you  started  to  build  out  the  skillset  and  surround  them  with  the  culture  and  the  processes,  how  do  you  involve  them?    At  what  point  in  the  process  do  you  involve  them  in  building  out  analytic  solution?      

 Think  of  furnishing  a  room…[22:33]    So,   I’d   like   to  have  everyone  on   the  call   think  about   furnishing  a   room.    Do  you  ever   try  and  throw  all  of  the  furniture  in  the  same  room?    By  thinking,  oh  we  need  a  couch  and  make  a  toilet  and  a  sink  and  a  fridge  and  a  chair  and  a  lamp  and  the  list  goes  on.    And  then  trying  and  make  it  all  fit,  as  if  it  could  accomplish  the  purpose  simply  by  having  all  of  the  stuff  in  there.    Or  do  you  design   a   room   that   perhaps   looks   incredible   but   that   no   one   uses   because   it’s   completely  dysfunctional?    I  think  most  people  decide  what  the  purposes  of  the  room  is  fill  it  with  things  to  accomplish   that   purpose.     So   that  may   be   as   simple   as   a   powder   room   right   for   the   kitchen  where   everyone   entering   it   has   the   exact   same   purpose,   or   as   complex   as  maybe   a   kitchen  where  there  are  multiple  ways  to  solve  this  problem  and  it’s  highly  individualized.          

 But  sometimes  we  ask  the  same  thing  of  our  analysts…  [23:35]    If   you   think   about   that   analogy,   we   sometimes   ask   analysts   a   similar   question.     Here   is   this  spreadsheet,  here  is  the  metric  list,  build  us  a  dashboard.    But  really  the  development  of  your  analysts   depends   on   them   flexing   their   problem   solving   skills   and   applying   them   to   your  problems,  not  a  checklist  of  metrics  that  they  are  building  out.    

 Awesome  dashboard  v1  [24:03]    So  what  can  happen  is  that  I  think  all  of  us  have  been  here  at  some  point,  where  you  come  to  the   first  meeting  with  your  awesome  dashboard  v1  with  everything  on   it.    At   the  Healthcare  Analytics   Summit,   we   were   able   to   kind   of   carry   out   this   exercise   and   this   was   actually   a  strategy  where  maybe   it’s   some  weird   tetris   problem.     I   know   I’ve  used   this   technique  quite  often.    We  tried  to  kill  all  the  white  space  on  a  dashboard.    It’s  not  recommended.          

 Awesome  Dashboard  v2  [24:37]    Or  if  they  come  out  from  a  different  vein  and  think  that  they  maybe  need  to  make  something  that’s   aesthetically   pleasing   or   elegant   or   maybe   just   a   feet   of   technical   prowess   but   not  necessarily  useful.      

 Awesome  Dashboard  v3  [24:50]    The  sweet  spot   is  when  the  act   is  tailored  to  the  problem  and   it   is  designed  to  solve,  not  the  metrics  in  it.    So  whether  that’s  something  brutal  in  its  simplicity,  where  it’s  maybe  a  single  one  chart,  or  if  they  glorify  spreadsheet.    As  long  as  you  designed  it  with  a  purpose  in  mind  and  with  the   end  users   informing   that   purpose   on   how   they   use   it,   it’s   going   to   be   a   lot   better.     And  we’re  not  saying  that  failing  is  bad,  but  we  want  to  minimize  avoidable  failures  due  to  lack  of  a  metrics  filter  and  provide  context  and  direction,  not  step-­‐by-­‐step  instructions,  so  that  the  first  failure  is  much  closer.    In  fact,  if  you  think  about  it,  it  would  be  much  easier  if  the  metrics  were  the  only  metrics  you  asked  them  to  develop  in  the  first  place.        So  how  do  we  decide  what  is  truly  important,  even  though  everyone  wants  everything  on  the  same  sheet?    You  build  filters  into  the  process.          

 What  metrics  are  you  dealing  with?  [25:59]    So  what   type  of  metrics  are  you  dealing  with?    We  are   talking  about  perhaps  accountability.    The  metrics  have  to  be  right.    Think  here  of  regulatory  measures.    For  research,  you  might  get  into  the  –  well  it  would  be  nice  to  know,  or  just  in  case,  let’s  bring  in  this  metric.    Something  to  explore.    But  for  improvement,  you  only  need  just  enough  data.    Decide  on  the  purpose  of  your  analytics   application   and   give   it   that   single   track.     In   another   sense,   thinking   about  improvement.    Don’t  spend  weeks  on  logic  to  capture  that  remaining  5  percent  of  patients.    If  95  percent  of  the  data  directionally  captures  the  process  you  are  trying  to  improve,  then  move  on.    In  fact,  in  most  cases,  you’ll  find  there’s  probably  less  than  95  percent.    So  if  you  designed  your   dashboard   around   improving   a   process,   don’t   confuse   accounting   for   100   percent   of  patients  with  improving  that  process.          

 Involve,  don’t  minimize  [27:08]    The  other   filter,   so  we’ve   talked  about  kind  of   filtering  yourself   in   terms  of  what  metrics  you  include.     But   the   other   filter   needs   to   happen   right   up   front   and   it’s   with   how   analysts   are  involved  in  the  process.    So  our  first  poll  question,  it  has  kind  of  highlighted  that  domain,  and  really  understanding  the  data  in  that  domain  is  key.    So  we  need  to  not  isolate  analysts  from  it.    Involve  them  in  the  very  first  meeting   the  metrics   are   being   decided,  when   the  problem   is   being   hashed  out   because  what  that  does  is  it  builds  curiosity,  it  builds  passion,  and  that’s’  where  value  is  really  realized.    So,  one  of  the  techniques  –  I’ll  frame  it  this  way.    So  writers  use  a  framework  developed  by  an  author  named  Randy  Ingermanson  called  the  “Snowflake  Method”  and  they  start  with  a  single  sentence  that  describes  the  plot  in  the  book.    And  from  that  same  sentence,  they  break  it  down  into  three  sentences  that  maybe  cover  with  three  main  plot  points  of  the  book.    And  then  from  those   three  or   four  sentences,   they  build   that   to   three  or   four  paragraphs.    They  start   to  get  into  the  characters.    And  those  three  or  four  paragraphs  get  into  the  three  or  four  pages.    And  by   the  time  we  are  done,  you  actually  have  seams  that   link  all   the  way  back  up  to   the  single  purpose.      

 And   so,   we’d   like   to   describe   something   similar,   where   you   start   with   the   goal   of   your  application.    Then  you  follow  with  the  different  aims  that  you  are  trying  to  accomplish,  how  are  we  going  to  accomplish  this  goal.    And  from  there,  metrics  to  support  those  aims.    So  going  all  the  way  from  goal  to  metric.    And  finally,  how  do  you  use  these  metrics  to  accomplish  the  aims?    With  user   stories   that   start   from  how  someone   is  actually  going   to   leverage   these  metrics   to  accomplish   these   goals   and   aims.     So,   not   only   will   this   context   the   way   you   build   out   this  designer  on  metrics,  not  only  will  it  filter  out  things  but  it  will  give  the  analyst  an  understanding  of  user  needs  as  they  use  their  dashboard.          

 Example  sepsis  “snowflake]  [29:41]    So  let’s  touch  on  a  few  examples.    Let’s  think  in  the  area  of  sepsis.    We  have  a  goal  of  perhaps  improving  early  recognition  of  sepsis   in  the  emergency  department.    From  there,  we  create  a  first   aim   of   increasing   compliance   for   routine   sepsis   screening   protocol.     And   further   down  metrics,  so  talk  about  compliance  by  location  and  provider  and  maybe  shift,  a  time  from  arrival  to  triage,  and  mortality  rate  of  screened  versus  unscreened.    The  point  of  this  is  that  it  gives  you  something  that’s   limited   in  scope.    The  functionality  that’s  described  here   is  not  huge  but   it’s  easily  translated  into  a  shot  in  the  dark.    The  context  and  purpose  are  maintained  and  especially  

the   analyst  was   present   to   soak   in   the   passion   and   experience   of   those   providers  who  were  present  in  the  meeting.          

 Example  Revenue  Cycle  User  Story  [30:43]    Another   transition   to   user   story   in   a   different   area   –   so  maybe   your   revenue   cycle,  where   a  typical   layout   for  a  user  story   is  usually   ‘As  a’   ‘I  want’   ‘So  that’.    So   for  example,  as  a  patient  access  manager,  ensuring  patient  registration,  authorization  and  verification,  so  that  there  are  no  delays  in  billing,  loss  of  revenue  or  patient  dissatisfaction.    You  get  the  feeling  for  a  user  how  a  user  will  actually  sit  down  and  interact  with  the  dashboard.    The   metrics   are   called   out   specifically   but   they   are   implied,   as   is   the   functionality   of   the  dashboard  and  what  metrics  are  important  to  see  next  to  one  another.        

 Do  you  want  metrics  or  smart  people  using  data  to  think  about    Your  problems?  [31:33]    The   really   important   point   here   is   that   the   emphasis   is   taken   off   metric   building   and   onto  solving   the   problem   –   because   in   the   end,   you   don’t  want  metrics.     You  want   smart   people  using   data   to   think   about   your   problems.     One   example   we’ve   seen   of   this   is   if   there   is   an  organization  who  has  decided  to  build  a  new  wing.    And  once  they’ve  reframed  the  question  of  instead  of  building  out  this  new  wing,  what  is  the  problem,  why  are  we  even  thinking  about  this  in  the  first  place.    And  giving  the  analyst  that  context  versus  validating  some  decision  they  have  already   made,   they   actually   reframed   the   question   and   got   a   different   answer   where   they  needed  to  only  keep  their  clinics  open  a  little  bit  longer  and  that  alone  saved  them  millions  of  dollars  and  again  the  analysts  paid  for  themselves  just  by  allowing  them  to  see  the  context  of  a  problem  and  use  their  skills  that  have  been  developed  to  solve  it.      

 Barriers  Getting  in  and  out  of  the  way  [32:40]    The   last   thing   that  we’re   going   to   talk   about   here   is   to   really   close   the   loop   and   talk   about  things  that  might  get  in  the  way  of  an  analyst  developer.          

 Barriers  to  data  [32:52]    So  there  are  going  to  be  barriers   to  data.    So   first,  provide  your  analysts  with   full  access  to  a  testing  environment.    Sometimes  they  are  limited  in  their  effectiveness  because  there  are  some  technical   barrier   restricting  access   to   the  EDW  or   to   some   source.     So   give   your   analysts   the  opportunity   to  actually  build,  break,  and   re-­‐build  data   sets  within  your  data  warehouse.    We  need   a   sandbox   to   store   anything   and   everything   that   we   find   useful   and   self-­‐serve   that  curiosity  and  passion  that  we’re  trying  to  build.    Again,  remember  autonomy  and  mastery.    We  want  to  be  self-­‐directed  in  how  we  do  things  but  then  also  get  better  at  the  things  that  we  think  are  going  to  help  us  in  our  job.        The   next   one,   surprise,   they   need   an   EDW,   and   really   with   good   reason.     So,   we   had   the  opportunity  again  to  ask  a  poll  question  to  someone  else  to  another  group  of  people  and  the  amount  of  time  that   is  spent  actually  gathering  data  versus  analyzing   it,  almost  80  percent  of  analysts  spend  60  to  80  percent  of   their   time  hunting  and  gathering  data.    At   that  point,  you  might  as  well  not  call  them  analysts.          

 Barriers  to  data  [34:24]    So,  how  hard  can  it  be  to  gather  data  for  analytics?    In  a  weak  analytic  system,  very  hard.    So  they  are  spending  time  hunting  for  data,  gathering,  compiling,  running,  distributing,  all  of  these  non-­‐value-­‐add  tasks.        Having  a  strong  analytic  system  which  hinges  upon  having  an  EDW  shifts  an  analyst  from  non-­‐value-­‐add   to   value-­‐add.    And   if   you’re   thinking   about  developing   the   analyst   and   getting   the  right  skills  cultivated,  you  help  them  to  get  really  good  at  the  value-­‐add,  not  sending  out  PDS.      

 Other  barriers  [35:10]    Here  are  some  other  barriers  that  might  prohibit  and  prevent  analysts  from  developing.    One  of  them  is  access  to  people.    So  not  only  do  they  have  access  to  the  data  they  need,  but  the  actual  people  they  need  to  get  the  job  done  –  are  they  in  brainstorming  meetings,  are  they  friends,  so  to  speak,  with  the  frontlines,  so  will  use  what  they  build.        You  also  would  ask   if  you  kill  any  “ghosties”  that  are  haunting  your  analyst.    So  there  may  be  some  potentially  mental  barriers  where  they  have  reports   that   they  have  spent  most  of   their  time  gathering,   compiling   and   running.    And   that  now   if   you   think   about   that  previous   slide,  their   job   is  being  shifted  from  that  report  running  to  actually   interpreting  data,  and  they  may  feel  that  their  job  is  being  replaced.    But  really  if  you  think  about  it,  we’re  simply  reframing  their  work  into  something  that’s  more  value-­‐add  than  ever  before.    The  next  idea,  and  this  one  is  really  important,  is  work  prioritization.    This  is  where  the  armor  is  dumb  if  you  are  someone  who  is  an  advocate  of  the  analyst.    And  if  you  are  the  analyst,  these  need  to  be  done  for  you,  where  you  will  start  to  become  useful,  especially  to  develop  skills  that  are  going  to  make  you  into  a  critical  piece  of  outcomes  improvement,  and  people  will  start  to  clamor   for   your   services.     So,   you   need   to  make   sure   that   they   are  managing   that   claimer.    Prioritize   for   them  and  be   a   barrier   between   them   to  people  who  might   prevent   them   from  

getting  work  done.    Maintain  an  agreement  backlog,  whether  that’s  through  some  Cloud-­‐based  tool  or  spreadsheet,  and  provide  that  single  source  of   truth   for  what   they  should  be  working  on.        The  last  barrier  I’ll  speak  of  is  obviously  a  technology  barrier.    So,  as  we  play  in  databases  and  more  testing  and  running  queries,   this   is  going   to  utilize  a   lot  of   resources.    But   there’s  been  studies  that  have  found  that   it  only  takes  a  few  seconds  of  waiting  before  someone  becomes  disengaged  and  actually  have   to   re-­‐engage  with   that  work,   and   it’s   something   in   the  area  of  three   or   four   seconds.     And   they   actually   have   to   spend   closer   to   seven   seconds   to   get   re-­‐engaged.     And   that   inefficiency   could   actually   lead   you   to   think,   oops,   I   need  more   analysts,  versus  actually  solving  a  technology  problem.    So  you  spend  money  on  a  faster  server  or  money  on  four  new  analysts.        But  also  in  another  sense,  giving  your  analyst  a  voice  into  the  tools  that  are  going  to  facilitate  their  job.    Let  them  test,  let  them  play,  and  let  their  vote  actually  count  in  what  gets  deployed  to  them.    One  way  we’ve  seen  this  happen  is  through  analyst  actually  getting  new  computers  yearly   that   then   got   deprecated   into   roles   that   actually   didn’t   need   a   computational   heavy  computer.     So   don’t   limit   their   effectiveness   simply   by   the   resource   that   you   provided   them  with.          

 Lessons  Learned  [38:41]    So  here  are  some  lessons  that  I  hope  you’ve  taken  away  from  this  presentation  today.    One  is  that   in   your   culture,   the   analyst’s   role   is   not   to   build   dashboards   but   to   get   outcomes  improvement.    The  second  is  that  the  process  is  to  encourage  failing  fast  in  that  first  meeting.    The  skillsets   that  really  you  want,  perhaps  your  team   is  a  generalist   team  that   they  can  carry  out   the   full   spectrum  of   analytics   through   encouraging   that   in   a   single   person   but   then   also  making  sure  your  team  is  well  rounded.    The  involvement  of  applications  of  analysts  upfront  so  that  your  applications  –  we  all   know  that   they’re  not   just  a   technical  problem.     If   they  were,  then  we  wouldn’t  have  this  webinar.    This  is  the  hard  thing.    And  involving  your  analysts  in  each  of  these  processes  is  what  really  helps  them  develop.    And  lastly,  eliminating  barriers  so  that  it’s  easy  for  them  to  do  their  job.      

 Thank  you  [39:51]    Thank  you.      

 

How   interested   are   you   in   someone   from   Health   Catalyst   contacting   you   to   schedule   a  demonstration?  [39:53]    [Facilitator]  Alright.    Thank  you,  Peter.    Before  we  start  with  the  Q&A  session,  we  do  want  to  say  that  our  webinars  are  meant  to  be  educational  about  various  aspects  of  our  giant  industry,  particularly  from   a   data   warehousing   analytics   perspective.    We   have   had  many   requests,   however,   for  more   information   about   what   Health   Catalyst   does   and   what   our   products   are.     If   you   are  interested  in  having  someone  from  Health  Catalyst  reach  out  to  you  to  talk  more  about  Health  Catalyst,  please  answer  this  poll  question.    And  while  this  poll  question  is  up,  we’re  just  going  to  pause  this  for  a  bit  and  then  we’re  going  to  shift  around  and  make  sure  Russ  is  in  here,  so  he  can  help  to  answer  some  questions.    It’s  getting  closer  here.    Now,  also  while  this  is  up,  also  we  have  had  quite  a  few  requests  for  the  presentation  slides.    I  would   like   to   remind   everyone   that  we  will   be   providing   the   slides   along  with   the   recorded  presentation  and  the  results  of  the  poll  questions  shortly  after  the  event.        So   let’s   go  ahead  and  get   to   the   first  question.    But   first,  we’ve  got   a  nice   comment,   saying,  “Excellent  information.    I  look  forward  to  applying  this  knowledge  to  my  organization.”      

QUESTIONS   ANSWERS  What   presentation   tool   do   you   recommend   for  dashboards?    

[Russ  Staheli]  I  think  really  it  depends  on  the  goal  of  the  application  again.     So   if   it’s   potentially   a   more   self-­‐served  environment,  matching  that  request  and  functionality  with  the  tool.    We’ve  seen  a  lot  of  utilization  across  a  lot   of   tools   from   Excel   to   Tableau   to   QlikView.     So  really   tailoring   that   to   what   the   user   request   is   and  what  the  purpose  of  the  application  is.        And  as  you  become  a  data-­‐driven  organization,  I  think  it’s   wise   to   know   that   there   is   going   to   be   multiple  venues   for   these   dashboards   to   present   it   back   to  them.    And  one  of  the  things  that  is  very  important  to  us   as   an   organization   is   identification   and   adoption.    So   trying   to   see   how   many   users   we   can   get   using  these   tools   and   creating   a   culture   of   data   in   your  system   and   bringing   that   to   a   single   type   of   a   tool,  generally  you  can’t  get   the  same   level  of  adoption  as  you  would  if  you  were  to  enable  end  users  to  get  their  data  to  multiple  schools.    

Are   there   typical   org   charts   used   by   healthcare  systems  for  an  analytics  team?    

You  know,  we  put  a  couple  of  blogs  out  related  to  this  same   question   and   there’s   generally   two   patterns  which   I   think  all  of  us  would  recognize  that  are  being  centralized   and   that   are  being  decentralized.    And  as  we  had  opportunities   to   interact  with  multiple  health  

systems  and  see  both  of  those  strategies  at  play,  one  thing  that  we  don’t  think  works  is  if  you  go  completely  to   one   end   of   the   spectrum   or   the   other   end   of   the  spectrum.     What   we   found   to   be   most   effective   is  centralizing   certain   skillsets   that   are   very   specialized  and  decentralizing  more  of   the  common  skillsets   that  are   more   you   pick   with   just   across   the   organization.    So   it’s   a   little   bit   of   both   but   it   would   be   interesting  hearing  your   feedback  as  well  as   to  ways   that  you’ve  seen  to  best  organize  these  teams.    

What  questions  do  you  ask  to  hire  for  soft  skills?    

[Peter  Monaco]  So,   I   know   personally   in   a   few   interviews,   I’ll   take  humility,   for   example.    We’ll   ask   when   they’ve   been  part  of  a  project  that  failed,  that  didn’t  go  as  planned,  and   how   they   actually   responded   to   that   and   how  they   leveraged   that   learning   into   their   next   thing.    Because   I   think   one   of   the   important   things   about  humility,  and  I  think  it  was  emphasized  if  you  were  at  the   Healthcare   Analytics   Summit,   is   that   humility  depends  on  setting  a  goal  that  may  be  kind  of  they  call  it  big  hairy  and  audacious.    And  so,  seeing  times  when  and   seeing   repetitive   examples   of   someone   who   set  goals   that   might   have   been   out   of   reach   but   always  strove   to   payment.     That’s’   a   few   ways   that   we’ve  asked  in  our  interviews  to  give  that  questions.        [Russ  Staheli]  And  we  most   definitely   did   not   only   interview   based  on  soft  skills.    There  was  our  filtering  process  and  we  have  an  opportunity  to  interview  a  lot  of  candidates  as  a   growing   organization   for   a   time   hiring   about   three  analysts  in  this  broad  definition  a  week.    We  did  have  to   come   up   with   some   structure   and   some   specific  questions   that   we   would   ask   both   to   do   technical  assessment   and   soft   skills   assessment.     I   think   there  would   be   ways   for   us   to   share   some   of   that   if   you  want   to   reach   out.     I’m   not   sure   what’s   the   best  process  for  them  to  reach  back  out  to  us.    [Facilitator]  Well   you   can   reach   out   through   the   webinars   at  HealthCatalyst.com   in   email   address.     That’s   one   of  the  best  ways  and  all  that  will  come  through  me  and  I  can  make  sure   that   I   could  pass  on   to  you,  Russ,  and  you,  Peter,  to  respond  to  that.        I  personally  see  the  soft  skills  as  being  so  important  in  terms  of  you  talked  about  having  access  to  people.    If  you   don’t   have,   how   do   you   build   more   and   more  access  to  people  and  if  you’re  given  initial  access,  how  do   you   keep   that   access   to  people   and  being   able   to  

not   only   explain   in   a   straightforward   way   what   the  data   means   but   also   to   be   able   to   keep   those  relationships,   so   that   you   can   continue   to   explain  those  things.    Those  things  are  very  important  skillset.      

As  a  domain  expert  who  wants  to  migrate  into  a  data  analyst   role,   what   would   be   the   first   three   things   I  could  do  on  my  own  to  prepare  for  such  a  role?    

[Russ  Staheli]  So,   wow,   I   think   Peter   and   I   might   have   different  opinions  here  and  I’d  love  to  take  a  little  bit  more  time  to   think   that   through.     Some  of   the   things   that   come  to  mind  are  the  most  ubiquitous  skillsets   that  we  see  across   our   analysts.     One   of   those   being   SQL.     You  know,  as  much  as  we  try  to  create  and  have  tools  or  as  many   tools   as   there   are   in   the   industry   that   try   to  create  a  layer  for  action  above  that,  it  feels  the  core  to  the   skillsets   in   all   of   our   roles   here   at   Catalyst,  understanding   how   to   manipulate   the   data.     And  those  books  that  Peter  suggested  in  his  slide  deck  are  another   great   resource   that   has   helped   individuals  start  to  understand  how  do  they  tell  a  story  with  data,  not   just   how   do   they   interact  with   data,   but  what   is  the  best  way  to  represent  data  back  to  users.        There  are  other,  of  course,  user  groups  that  you  could  join   with   individuals   that   have   a   similar   passion   and  I’m  not  sure,  Peter,  if  you  want  to  jump  in  here.        [Peter  Monaco]  No,   I   was   reading   these   questions   as   they   come   in.    But   yeah,   so   I   think   you   have   a   unique   advantage  potentially   by   having   that   domain   expertise   because  you  can   then  start   to   see  what  you  would   like  out  of  the  data.    So  with  that  domain  expertise,  start  to  self-­‐serve   those   using   books   that’s   kind   of   the   Catalyst  there.     So,   using   the   SQL   training   to   get   up   the   data  you’d   like   and   then   the   visualization   books   are  agnostic   to   a   visualization   tool,   and   really   that’s  something  that  can  translate  across  tools  and  tools  are  getting   more   and   more   self-­‐served   these   days.     So  being  able  to  understand  what  it  takes  to  make  a  great  visualization,   I   think   those   two   books   are   a   great  starting  point  for  that.    

How   likely   is   it   that   analysts   want   to   develop   as   a  generalist?    I  typically  see  them  gravitate  to  a  specific  skillset.    

There  are   two  bands.    There  are   two  different  camps  here   for   sure.     Of   the   individuals   that  we   have   hired  over  the  past  three  or  four  years,  which  I  think  would  represent  probably  about  70   to  80  different  analysts.    So   I   think  we  have  a  sizeable  end  to   look  at.     I  would  say  the  vast  majority  or  the  majority,  may  not  be  the  vast  majority,  but  the  majority  have  actually  been  very  interested   in   learning  and  expanding   their   skillsets.     I  would   maybe   say   60   to   70   percent   of   those  

individuals.        There  are  definitely  the  groups  that  want  to  focus  very  deeply  on  a  specific  area,  and  those  are  very  valuable  individuals  as  well.    And  a  balance  of  both  generalists  and   specialists   are   very   valuable.     We   have   a   team  here  at  Catalyst  that  sort  of  is  where  the  specialists  all  gather   together,   and   it’s   amazing   to   watch   how   the  rest   of   the   company   leverages   that,   and   they   come  down   and   ask   the   question,   the   deeper,   deeper  question,  “Hey,  can  you  help  me  with  this  data  model?    Hey,  can  you  help  me  with  this  visualization?    Can  you  help  me  with   this   complex   SQL   statement   that  we’re  having  performance  issues  on?”        So  shying  away  from  having  those  types  of  individuals  on  the  team  is  not  what  we’re  trying  to  suggest.    Just  that   there   is   a   large   percentage   of   these   individuals  that   could   be   very   effective   as   generalists,   and   that  minimizing   those   hand-­‐offs   as   you’re   solving   a   user’s  problem  has   shown  great   efficiencies   and  a  way   that  have  enabled  our  analysts  to  take  in  themselves.        

Can   you   explain   the   difference   between   a   data  architect  and  a  domain  expert?    

[Russ  Staheli]  Yeah.    So,  we  start  to  get  into  some  of  this  vocabulary  stuff   and   in   our   industry,   it’s   still   pretty   mushy.     I  remember   I   worked   at   Intermountain   Healthcare   for  about  six  years  and  my  title  was  an  analyst.    Yet,  daily  I  was  doing  what  these  people  at  Intermountain  would  call   an   architect   role.     I   was   doing   that   daily.     I   was  down   in   the   weeds   data   modeling,   identifying   data  from  source   systems  and  understanding  how   I  would  merge  that  data  together  to  tell  a  complete  story.        So  for  us,  the  way  that  we’ve  usually  used  those  terms  internally,   and   again,   these   are   defined   in   different  ways  and  shared  where  you  go,  is  that  the  architect  is  more  of  an  individual  that  is  doing  data  modeling,  that  is  looking  at  very  specific  data  tables  and  schemas  and  bringing   them   together   and   having   those   data  elements  interact  with  each  other.        Whereas   a   domain   expert  may   not   necessarily   know  the  technical  pieces.    This  is  more  of  an  individual  that  understands   the   content   that  would   be   found  within  those   tables,   say,   yeah,   it’s   their   discharge   plan,   or  they  might   know   a   lot   about   the  ADP   records.    Or   is  there,  you  know,  a  cardiovascular,  they’ll  know  about  their  modules,   it’s   in   their   EMR   system   that   supports  that.     When   we   talk   about   domain   experts   here   at  Catalyst,   we’re   not   specifically   identifying   a   technical  skillset.    More   of   an   expertise   and   knowledge   of   the  

specific  content  area.        Anything  you’d  add  there,  Peter?    [Peter  Monaco]  No.    That  like  covered  it.  

What   aspects   of   SQL   are  most   important?     DDL   or   a  DML  complex  queries  like  Aggregation?  

[Peter  Monaco]  You  know  what,  it  really  depends  on,  I  would  say,  the  complexity  of   the  problem  you’re   trying   to   solve   and  really  how  many  sources,  how  close  the  source  system  matches   the  output  you’re   looking   for.     So,  as  you’re  trying   to   pull   stuff   from  multiple   places   that  may   be  fragmented,  you  might  have  to  get  more  complex  with  the  queries  you’re  writing  to  normalize   it  and  clean  it  versus  something  where  data  might  be  brought  into  a  single  EDW  that  has  more  common  linkages,  that  that  complexity   might   be   able   to   be   shifted.     But   we’ve  seen  cases  where,  as  everyone  talks  about,   there   is  a  lot   of   different   ways   to   define   a   cohort   of   patients.    And   so,   that   type   of   thing   is   something   where   that  complexity  won’t  go  away  completely  but  it’s  the  best  you  can  get  in  terms  of  getting  source  systems  into  an  EDW  will  help  in  that  respect.    [Russ  Staheli]  Yeah,   I   would   say   as   the   maturity   of   health   systems  grows,  we  find  that  in  that  they  start  to  use  tools  like  Enterprise  Data  Warehouses   in   the  way.    We  see   the  need   for   all   of   our   analysts   to   have   DDL   knowledge  going  away.    They’re  slowing  down.    It’s  not  nearly  as  critical   in   the   day-­‐to-­‐day   life   as   DML   or   Data  Manipulation   Language.     And   so,  we   do   see   that   the  majority   of   our   analysts,   once   there   is   a   mature  underlying   structure   readily   available   and   great   hard  work   and   almost   great   things   already   set   up,   you’re  able   to   focus   more   on   the   business   problem   and  writing   the   code   that   would   solve   the   business  problem  or  address  the  business  problem  than  you  are  at   writing   the   technical   pieces   to   ensure   that   those  problems  run  in  an  efficient  manner,  etc.,  etc.    

 [Facilitator]  Alright.    Well  we  have  a  question  about  the  titles  of  the  two  books  that  you  mentioned,  the  two  visualization  books  that  you’ve  mentioned.    We  do  have  those  on  the  slides  we’ll  hand  out.     I  actually   have   those   up.     One   is   the   second   edition,   The   Visual   Display   of   Quantitative  Information   and   the   other   is   Information   Dashboard   Design:     The   Effective   Visual  Communication  of  Data.    But  the  titles  of  both  those  along  with  their  authors  will  be  included  in  the  slide  that  we  will  be  distributing  to  everyone.    

[Russ  Staheli]  Yeah.    For  those  that  enjoy  the  theory  of  visual  design  and  want  to  understand  the  principles,  the  guiding  principles  that  sit  behind  why  you  wouldn’t  use  a  pie  chart,  for  example,  pick  up  the  Edward  Tufte  book  which  is  The  Visualization  Display  of  Quantitative  Information.    Or  any  of  his  books.    He  has  some  great  other  books  as  well.    For  those  that  are  more  just  needing  day  in,  day  out  practical  guidance  as  to,  okay,   I’m  showing  this  kind  of  data,   I  want  to  see  what  the  best  way  to  solve  this  problem  and  just  have  the  answer  for  them,  Stephen  Few’s  literature  is  more  closely  related  to  that.    [Facilitator]  Thanks  Russ.    We’ve  got  a   few  more  questions.     It   looks   like  we’ve  got   time   for  at   least   two  more  questions.          

QUESTIONS   ANSWERS  How  would   you   go   about   teaching   someone   SQL   for  the   strong   background   in   requirement   gathering   and  manipulating  dashboards   that   is   not   as   strong   in   SQL  how  they  fit  in  the  application  tool?    

[Russ  Staheli]  There  are  definitely  an  alignment  with  passion,  where  we   don’t   in   our  model   force   people   to   go   down   the  generalist’s  path  and  there’s  generally  an  interest  that  kind  of  pushes  people  down,  and  we   just  enable  that  and   make   it   easy   for   them   to   go   after   that   passion.    And  what  I  had  found,  and  I’m  not  sure  if  this  is  across  the  board,  Peter,   and   I’d  want  you   to  get   your   thing,  but  what  I  had  found  with  our  team  members  is  those  that  have  the  passion,  we  assign  the  project  that  kind  of   just  pushes   them   into   the   fire  more   than  anything  else.     They   have   the   opportunity,   they   have   support  from  their  team  members  around  that  are  stronger  in  SQL  but  there  isn’t  a  specific  book  that  we  have  them  pick  up.    We  pair  them  with  someone  and  have  them  do  a  little  paired  programming  for  a  part  of  it  but  also  let   them   run   on   their   own   a   little   bit.     Back   to   our  concept   of   failing   fast   and   allowing   people   to,   you  know,   creating   that   culture   of   failure   not   being   a  problem,  not  being  something  bad.    It’s  generally  how  we  see  that  happening  and  it’s  been  fairly  natural  and  it’s   been   something   that   as   people   have   been  passionate,  they’ve  been  able  to  grow  into  the  skillsets  for  the  most  part.    There  are  some  that  definitely  have  the  passion  but  just  will  never  have  the  technical  jobs  and   there’s  discussions   that  need   to   take  place   there  but   for   the  most  part,  we’ve  seen  them  come  up  the  speed   through   giving   them   real   life   projects   and  pairing   them  up  with   someone   that  has  a   strength   in  that  area.    [Peter  Monaco]  And   really   I   think   that   matches   well   with   my  experience   at   Catalyst   and   it   began,  when   I  was   first  hired,   I   know   a   little   bit   of   SQL   but   a   different   –   it’s  

actually   mySQL,   and   so   a   different   structure   and   a  much   more   complex   environment   here.     So   I  remember   being   inserted   into   a   project   that  demanded   domain   expertise   and   visualization  expertise,  of  which   I  had  none.    And  that   insertion  of  being   surrounded   by   people   who   have   strength   in  those  skills,  it’s  really  where  I  failed  multiple  times  in  a  single  day,  where  things  I  ran  did  not  work,  I  didn’t  get  the   right   answer.     And   being   able   to   bounce   that  across   to   someone   rather   than   being   scared   about  divulging  information  that  I  don’t  know  a  lot  and  being  able   to   have   resources   to   help  me   out   there,   I   think  that  goes  well  with  more  than  anything  but  especially  SQL.    

Anything  else  in  terms  of  a  client  experience  that  you  can  share  where  failing  fast  led  to  analyst  success?    

[Peter  Monaco]  Yeah.    I  was  going  to  say,  how  many  do  they  want?    So  I   know   more   recently   we   worked   with   clients   who  didn’t   actually   want   the   results   shown   to   providers  until   they  were   sure   that   everything  was  up   to   snap.    And   so,   it’s   actually   a   couple   of   months   before   that  was  shown.    And  so  this  is  perhaps  an  example  of  not  failing  fast,  where  at  the  end  of  those  couple  months,  they   showed   the   results   to   the   providers   and   it   was  way   off,   according   to   them.     And   what   was   realized  through   that   experience   is   that   if   they   had   just  included   the  providers   from   the  very   first  meeting   to  hash   out   what   their   requirements   were   for   those  metrics,   there   would   have   been   a   lot   simpler  processes,  something  that  maybe  would  delay  two  or  three  months  because  they  were  afraid  of  that  failure  could  have  been  shortened  to  simply  a  month  because  everyone  was  in  sync.        And  that’s  really  where  the  failing  fast  comes  in  for  me  –   is   whether   I’m   building   a   data   model   and   I’m   not  sure   how   it’s   going   to   fit   in   with   the   visualization,  rather  than  conjecture,  actually  build  the  visualization  on  top  of   it.    See  how  well   it  works.    And  if   it  doesn’t  work,   build   back   to   the   backend.     Constantly  reiterating  between   that   continuum  of  building  out  a  data   structure   to  visualization,   to  actually  having  end  users  use  it,  and  having  each  one  inform  the  other.    

 [Facilitator]  Alright.    Well   thank  you  so  much.    We’ve  reached  the  top  of   the  hour.    So   I  would   like  to   let  everybody  know,  shortly  after  this  webinar,  you  will  receive  an  email  with  links  to  the  recording  of  the  webinar,  the  presentation  slides,  and  the  poll  question  results.    Also,  please  look  forward  to  the  transcript  notification  we  will  send  you  once  it  is  ready.        

On  behalf  of  Peter  Monaco,  Russ  Staheli,  as  well  as  the  rest  of  us  here  at  Health  Catalyst,  thank  you  for  joining  us  today.    This  webinar  is  now  concluded.                      

[END  OF  TRANSCRIPT]