Talk at IS-EUD '11

of 16 /16
End-users Productivity in Model-based Spreadsheets: An Empirical Study Laura Beckwith acome Cunha Jo˜ ao P. Fernandes Jo˜ ao Saraiva HCIResearcher, Denmark, [email protected] Universidade do Minho, Portugal, {jacome,jpaulo,jas}@di.uminho.pt Universidade do Porto, Portugal, [email protected] IS-EUD 2011 June 6 – 10 acome Cunha (UMinho) IS-EUD 2011 1 / 16

Embed Size (px)

description

 

Transcript of Talk at IS-EUD '11

  • 1. End-users Productivity in Model-based Spreadsheets: An Empirical StudyLaura Beckwith Jcome Cunha a Joo P. Fernandes a Joo Saraiva a HCIResearcher, Denmark, [email protected] Universidade do Minho, Portugal, {jacome,jpaulo,jas}@di.uminho.pt Universidade do Porto, Portugal, [email protected] IS-EUD 2011 June 6 10 Jcome Cunha (UMinho) a IS-EUD 2011 1 / 16
  • 2. Agenda1 Introduction2 Model-based Spreadsheets3 Study Design4 Eectiveness5 Eciency6 Conclusions Jcome Cunha (UMinho) a IS-EUD 2011 2 / 16
  • 3. Introduction Spreadsheets are widely used But they are error-prone And many, many spreadsheets contain errors To improve this situation, much research has been done Model-Driven Engineering is a promising approach We have created a set of dierent model-based spreadsheets Need to evaluate their impact when used by end users Jcome Cunha (UMinho) a IS-EUD 2011 3 / 16
  • 4. Original Spreadsheet - original Tabular spreadsheet Represents a project management system Jcome Cunha (UMinho) a IS-EUD 2011 4 / 16
  • 5. Refactored Spreadsheet - refactored Each entity separated in a dierent table Tables are separated by empty columns Jcome Cunha (UMinho) a IS-EUD 2011 5 / 16
  • 6. Visual Spreadsheet - visual Similar to original Combo boxes help user introducing data Jcome Cunha (UMinho) a IS-EUD 2011 6 / 16
  • 7. Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets versus the original unmodied one?RQ2 Are end users more ecient using the model-based spreadsheets?RQ3 Do particular models lead to fewer errors in particular tasks? Jcome Cunha (UMinho) a IS-EUD 2011 7 / 16
  • 8. Study Design Background questionnaire, followed by a list of tasks and a post questionnaire No initial tutorial was given to the participants 38 participants: 25 females and 11 males, all students Tasks for inserting and editing data and computing statistics 3 dierent spreadsheet applications (projects, dishes and properties) 3 dierent models (original, refactored, visual) Each participant received 3 spreadsheets, each of a dierent application/model Jcome Cunha (UMinho) a IS-EUD 2011 8 / 16
  • 9. Global Eectiveness original refactored visual dishes 86% 76% 78% projects 73% 68% 78% properties 75% 64% 62% Globally, models do not seem very competitive Further analyses are necessary Jcome Cunha (UMinho) a IS-EUD 2011 9 / 16
  • 10. Eectiveness: Inserting Data original refactored visual dishes 91% 90% 81% projects 76% 60% 75% properties 86% 67% 68% Models always lose But very similar to original Jcome Cunha (UMinho) a IS-EUD 2011 10 / 16
  • 11. Eectiveness: Editing Data original refactored visual dishes 91% 82% 82% projects 54% 62% 50% properties 65% 98% 48% refactored in general was better Because there is less data to edit Jcome Cunha (UMinho) a IS-EUD 2011 11 / 16
  • 12. Eectiveness: Computing Statistics original refactored visual dishes 52% 37% 57% projects 19% 76% 13% properties 44% 57% 51% Again, refactored was better And again, less data do analyze Jcome Cunha (UMinho) a IS-EUD 2011 12 / 16
  • 13. Eciency original refactored visual dishes 35 32 28 projects 39 40 41 properties 37 36 40 Models are competitive In further analyzes we discovered that users took more time understanding model, specially refactored Jcome Cunha (UMinho) a IS-EUD 2011 13 / 16
  • 14. Answers to Research QuestionsRQ1 Do end users introduce fewer errors when they use one of themodel-based spreadsheets versus the original unmodied one? We believe there is potential for model-based spreadsheets But a deep knowledge seems to be necessary to be eectiveRQ2 Are end users more ecient using model-based spreadsheets? Model-based spreadsheet can help in some cases We believe better knowledge about models could improve this resultRQ3 Do particular models lead to fewer errors in particular tasks? For inserting data, this was not the case The models helped editing data and in computing statistics Jcome Cunha (UMinho) a IS-EUD 2011 14 / 16
  • 15. Conclusions We did not gave a tutorial to participants We believe better results could be achieve with initial training Nevertheless, our model-based spreadsheet are competitive Better for editing and statistics We plan to run a new study similar to this, but with an initial tutorial Jcome Cunha (UMinho) a IS-EUD 2011 15 / 16
  • 16. Thanks for your attention.Jcome Cunha (UMinho) a IS-EUD 2011 16 / 16