CCLTracker  Framework    

Monitoring  users  learning  and  ac7vity    in  web  based  ci7zen  science  projects  

Jose  Luis  Fernandez-­‐Marquez  

Joseluis.fernandez@unige.ch  

Outline  

•  Mo7va7on  •  Background  (Google  Analy7cs)  •  CCLTracker  Framework  

•  Conclusions  •  Experiment  and  demo  

Mo7va7on  

   Bounce  rate,  pages  views,  avg.  7me  per  session,  etc…  might  not  be  relevant  to  measure  user  engagement,  or  relevant  in  our  ci7zen  science  project.    

Google  Analy7cs  

•  Audience  •  Acquisi7on  •  Behaviour  

Audience:  Par7cipa7on  16.000  sessions  8.000  users  108  countries  –  90  languages  Avg.  of  400  sessions  per  day  

Audience:  Who?  

+  browser,  OS,    and  mobile  or  PC  connec7on  

Acquisi7on:  where  do  they  come  from?  

Acquisi7on:  where  do  they  come  from?  

Acquisi7on:  Evalua7ng  dissemina7on  ac7vi7es  

Itc.ua  1st  Release  

Twi^er  Reddit  Post  

Behavior  

Segmenta7on,  crossing  data  and  adding  filters  

•  Give  me  previous  stats  focusing  on  visitors  coming  from  reddit.  

•  Give  me  previous  stats  focusing  on  female  visitors,  between  18  and  24  years,  linux  users,  living  in  Switzerland.    

•  Infinite  number  of  possible  combina7ons.    

Segmenta7on,  crossing  data  and  adding  filters  

•  Top  10  countries  by  Female  users.  •  Top  10  referrals  (reddit,  facebook,  …)  gathering  female  users.  

•  Top  10  referrals  gathering  visitors  who  immediately  run  away  from  the  site.    

Why  do  we  need  anything  else?  

Bounce  rate,  pages  views,  and  avg.  4me  per  session  might  not  be  relevant  for  a  ci7zen  science  project.    – We  need  to  know  the  ac7ons  performed  in  the  site  to  measure  par7cipants  contribu7on.  

– We  need  to  be  able  to  make  public  analy7c  data.    – We  need  to  be  able  to  create  advance  data  aggrega7on.  I.e.  clustering  analysis,  advance  engagement  func7ons.    

CCLTracker  framework  

CCLTrackerJS Library Google

Analytics

Google Tag Manager

RGA Library R

Monitoring Storing, aggregating, reporting

Advance Aggregation and reporting

Google Super Proxy

CCLTracker  events  

-­‐ Is  the  user  scrolling  down  on  the  web  site  (0%,  25%,50%,75%100%)  -­‐   is  the  user  clicking  new  projects,  about,  forum,  etc?  

CCLtracker  events  

CCLTracker  events  

Segmenta7on,  crossing  data  and  filtering  

•  Stats  only  for  par7cipants  who  has  properly  installed  the  CERN  VM.  (segmenta7on)  

•  Comparing  two  segments  (e.g.  all  sessions  vs  sessions  running  the  virtual  machine.    

Segmenta7on,  crossing  data  and  filtering  

Segmenta7on,  crossing  data  and  filtering  

•  Most  common  web  API  errors  by  browser  (crossing  data,  filtering)  

Segmenta7on,  crossing  data  and  filtering  

•  Top  more  ac7ve  users.    

Segmenta7on,  crossing  data  and  filtering  

•  All  ac7ons  by  a  given  user    

Advance  Data  Aggrega7on  

1 2 3 4 5 6 7 8 9 10 12

Number of days

% of us

ers

05

1015

2025

3035

29.58

17.63

9.5210.18

7.38

2.883.875.31

3.321.84

4.064.43

•  Engagement  

Advance  Data  Aggrega7on  

Advance  Aggrega7on  

Visitors  per  age  group  

Visitors  successful  running  VM  per  age  group    

0  

10  

20  

30  

40  

50  

60  

18-­‐24   25-­‐34   35-­‐44   45-­‐54   55-­‐64   >65  

%  of  session

s  

Users'  age  

User  engagement  per  age  group  

Advance  Data  Aggrega7on  

Desired  Flow  of  ac7ons  

Advance  Data  Aggrega7on  

Why  do  we  want  all  this  data?  

•  Increase  the  number  of  par7cipants  •  Increase  par7cipants’  engagement.  

•  Improve  the  navigability,  and  accessibility  of  the  website.    

•  Improve  users’  learning  experience.  Are  users  improving  the  quality  of  their  contribu7ons?  

What  did  we  learn  using  CCLTracker?  

•  Transla7on  to  different  languages  is  important  to  reach  a  large  audience.  E.g.  Russian  referral.  

•  Addressing  technology  sec7ons  in  newspapers.  

•  Web  site  naviga7on  is  not  trivial.    

•  Low  engagement.    

•  Low  female  par7cipa7on.  

2nd  CERN  Public  Compu7ng    Challenge  

•  Increase  the  number  of  par7cipants:  – Addressing  female  par7cipa7on.  –  Transla7ng  the  website  to  different  languages.  –  Pos7ng  the  informa7on  in  data  hubs,  scien7fic  sec7ons  of  newspapers,  etc…    

•  Segmen7ng  data  by  different  level  of  engagements:  –  Visitors  who  do  not  run  the  VM.    –  Visitors  who  compute  10  jobs,  20  jobs,  …  n  jobs.  –  Visitors  who  compute  for  1  h,  5  h,  …  n  hours.    –  Visitors  who  par7cipate  over  the  whole  challenge…  

“top  10%  of  contributors  responsible  for  almost  80%  of  total  classifica7ons.”

Open  ques7on