Reducing digestible tidbits from meaty stock:

Satisfying varied tastes with an attractive instructional assessment menu

Erika Bennett, Jennie Simning, & Kim StaleyCapella University Library

LOEX 2011

How are you turning your data into persuasion?

1) The Hoarder

How are you turning your data into persuasion?

2) The Philosopher

Meanwhile we have hungry people waiting...

How are you turning your data into persuasion?

3) Data informs your Actions• You pose questions that data can answer• You gather• You analyze statistically• You propose actions• You carry out and test the actions• Everybody knows what important measures

you collect and why.

So this session is for you if…

• You collect streams of data that never see the light of day.

• There is data you wish you had quickly at hand.

• You struggle reconciling decision-makers’ demands with instructional best practices.

Objectives1. Prepping the Menu: Who is our audience

and what are trying to give them?2. Equipment: Why scorecards and dashboards?3. From the Pantry: What do we have already?4. Preparing the meal: Gathering measures &

metrics and creating scorecards and dashboards.

5. Plating and Garnish: Using technology for attractive dashboard displays & infographics.

About Us

• Capella is a regionally accredited online university

• 39,900+ students• Avg. age is 39• Instruction Librarian

Team• In-house library, 2007

BusinessEducation

Prepping Our Menu

The Big Trends

• In IL assessment– Moving toward larger scale value assessments

• In business assessment– Balanced assessments of business performance

• Significant parallels!

Similar development cyclesIL Assessment (ILIAC)

1. Review learning goals2. Identify learning outcomes3. Create learning activities4. Enact learning activities5. Gather data6. Interpret & analyze data7. Communicate data, enact

decisions, & seek improvement

Balanced Scorecards

1. Identify perspectives & strategic themes

2. Identify objectives & causal links3. Create metrics & measures4. Develop targets & goals5. Gather data6. Interpret & analyze data7. Communicate data, enact

decisions, & seek improvement

Equipment

Assessment Tools

• Scorecards • Dashboards• Often confused…

Scorecards

What are Scorecards?-Tool for managing performance-Report card on the organization’s performance.-Key objectives -Measures & Metrics-Goals or targets

Scorecard example

From Dagan, 2007

Scorecard example

http://www.metricus.jp/index.php/itil/metricus-modules/modules-for-itil-best-practices/request-fulfillment-management.html

Scorecard example

http://tandyjacque.narod.ru/balanced-scorecard-example.html

Dashboards

What are Dashboards?-Tool for monitoring performance

-Summarize scorecard data-Snap-shot view of most important metrics.

-Often graphical, like gauges on the dashboard of your car.

Dashboard example

Dashboard example

http://www.spiderstrategies.com/blog/index.php/2009/10/stunning-dashboards-from-scott/

Beyond the Stereotype

• Proactive tools• Flexible tools

From the Pantry

Shared ResponsibilityNarrow “Old world” Library Instruction

1. Focusing a Topic2. Sources: Background3. Searching the Catalog4. Search Terms5. Searching Periodicals6. ID-ing Reference Sources7. Citation & References8. Library Classification9. Etc.

Shared ResponsibilityContemporary IL Instruction

1. Determine Info Need2. Research Methods3. Scholarly Communications4. Information Technology5. Evaluating Info Sources6. ID-ing Reference Sources7. Economic, political, legal and

cultural contexts8. Evaluating info content9. Etc.

Our Data Network

Strategy Alignment

Strategy Map

A clear illustration of long-term strategy

Strategy Map

A clear illustration of long-term strategy

Strategy Mapping

http://www.moe.edu.bn/image/image_gallery?img_id=31478

Classic Perspectives:1. Customer2. Financial3. Internal Processes4. Learning & Growth

Strategy Mapping

Our Perspectives:1. Learner2. Financial3. Internal Processes4. Learning & Growth

Strategy Mapping

Our Perspectives:1. Learner2. National Standards3. Internal Processes4. Learning & Growth

Strategy Mapping

Our Perspectives:1. Learner2. National Standards3. Internal Processes4. Learning & Growth

Strategy Mapping

Our Perspectives:1. Learner2. National Standards3. Internal Stakeholders4. Learning & Growth

Our Strategy Map

Our Strategy Map

• Learner• National• Internal• Growth

Perspectives

Our Strategy Map

• Learner• National• Internal• Growth

Strategic Themes

Strategic Themes

• Memorable and motivational names of 3-5 words.

• Defensible with good execution.

Our Strategy Map

• Learner• National• Internal• Growth

Lifelong Learning Program IntegrationGreat Teaching Assessment as Learning

Strategic ThemesPe

rspe

ctive

s Initiatives

WE

Preparing the Meal: Scorecards & Metrics

Instruction Strategic Scorecard

• Key Performance Indicators (KPIs)• Targets and Benchmarks• SWAG approach

From: http://www.osu.edu/academicplan/2006scorecard.pdf

Academic Scorecard Example

Metric Review Questions• What action will I take if I see significant change in this

metric?flatlines and spikes

• Will that action control the process in the way needed?

• Will measuring this metric change the behavior and action (learning) as I want?

Raw Data

Dealing with the Raw Data

Our Solution:• Operational

Scorecards with Dashboard components

Ramekins of Data

• Built a relevant 2011 sheet for each category/ venue:

– Webinars– Self-directed

Tutorials– LibGuides

– First Course Experience

– Collaboratives– Residencies

Plating & Garnish

Plating & Garnish

• Executive Dashboards– Revisit

definition/tips/samples?

Dashboard example

http://www.spiderstrategies.com/blog/index.php/2009/10/stunning-dashboards-from-scott/

Analytical Add-ons

• Bissant SparkMaker• BonaVista Microcharts

• Columns• Minipies • Bullet Graphs

Add-ons

http://www.vertex42.com/ExcelArticles/sparklines.html

• Sparklines • Scatter Plots• Whiskers

Add-ons

What if, for YOU…

EXCEL EXCELLENCE

Plating & Garnish

• New Technologies– Cloud/Freeware options• Bittle: Reporting and dashboards• Style Scope free edition: Dashboards• Active Dashboard: Dashboards

– Infographics for specific situations• Creately: Diagrams and collaboration• Stat Planet: Interactive maps and graphs• Tufte’s VUE: Visual Understanding Environment

http:

//bl

og.s

ocra

to.c

om/w

p-co

nten

t/up

load

s/20

11/0

4/st

agge

ring_

educ

ation

_inf

ogra

phic

1.jp

g

Wrap Up/Q&A

Bibliography from HandoutBalanced Scorecards for Libraries & Higher Education:

• Matthews, J. R. (2008). Scorecards for results: A guide for developing a library balanced scorecard. Westport, Conn: Libraries Unlimited.

• Lyddon, J., & McComb, B. (2008). Strategic reporting tool: Balanced scorecards in

higher education. Journal of Applied Research in the Community College, 15(2), 163. Retrieved from http://www.ncccrp.org/

• Bielavitz, T. (2010). The balanced scorecard: A systemic model for evaluation and assessment of learning outcomes? Evidence Based Library & Information Practice, 5(2), 35-46. Retrieved from http://ejournals.library.ualberta.ca/index.php/EBLIP

• Lyddon, J., & McComb, B. (2008). Strategic reporting tool: Balanced scorecards in

higher education. Journal of Applied Research in the Community College, 15(2), 163. Retrieved from http://www.ncccrp.org/

Bibliography from HandoutScorecards and Dashboards:

• Person, R. (2009). Balanced scorecards and operational dashboards with Microsoft Excel. Indianapolis, IN: Wiley.

• Dagan, B. (2007). Dashboards and scorecards aid in performance management and

monitoring. Natural Gas & Electricity, 24(2), 23-27. Retrieved from http://www.wiley.com/WileyCDA/WileyTitle/productCd-GAS.html

• Hursman, A. (2010). Diagnosis: Dashboredom. Information Management, 20(6), 14-17. Retrieved from http://www.sourcemedia.com

• Schiff, C. (2008). Three things you should know about dashboards. DM Review, 18(6), 29-29. Retrieved from http://www.information-management.com

• Butler, L. M. (2007). Warning lights. Connection: The Journal of the New England Board of Higher Education, 21(5), 31. Retrieved from http://www.nebhe.org/

Bibliography from HandoutInformation Literacy Assessment: • Gilchrist, D., & Zald, A. (2008). Instruction & program design through assessment. In C. N. Cox, & E. B. Lindsay (Eds.),

Information Literacy Instruction Handbook (pp. 164-192). Chicago: Association of College and Research Libraries. • Oakleaf, M. (2009). The information literacy instruction assessment cycle: A guide for increasing student learning and

improving librarian instructional skills. Journal of Documentation, 65(4), 539-560. doi:10.1108/00220410910970249 • Oakleaf, M. (2011). Are they learning? Are we? Learning outcomes and the academic library. Library Quarterly, 81(1),

61-82. Retrieved from http://www.journals.uchicago.edu/toc/lq/current • Oakleaf, M., & Kaske, N. (2009). Guiding questions for assessing information literacy in higher education. Portal:

Libraries and the Academy, 9(2), 273-286. Retrieved from doi:10.1353/pla.0.0046 • Poll, R., & Payne, P. (2006). Impact measures for libraries and information services. Library Hi Tech, 24(4), 547-562.

doi:10.1108/07378830610715419 • Oakleaf, M. (2009). Writing Information Literacy Assessment Plans: A guide to best practice. Communications in

Information Literacy, 3(2), 80-89. Retrieved from http://www.comminfolit.org/index.php/cil

• Orcutt, D. (Ed.). (2010). Library data: Empowering practice and persuasion. Santa Barbara, CA: Libraries Unlimited.

• Thompson, G. B. (2002). Information literacy accreditation mandates: What they mean for faculty and librarians. Library Trends, 51(2), 218. Retrieved from http://www.press.jhu.edu/journals/library_trends/

Bibliography from HandoutFree Infographic creation software:• Infographic software links compiled by Melodie Brewer:

http://melodiebrewer.com/data%20visualization.html• Creately: http://creately.com/• Tableau: http://www.tableausoftware.com/public/• Hohli: http://charts.hohli.com/• Stat Planet: http://www.sacmeq.org/statplanet/• Visual Understanding Environment (VUE): http://vue.tufts.edu/ Free statistical software: • List of links: http://www.freestatistics.info/index.php• Many Eyes: http://www-958.ibm.com/software/data/cognos/manyeyes/

Additional Capella program-focused Pubs & Presentations

• Staley, K., Brothen, E. & Bennett, E. ACRL Virtual – 2011; “When Nontraditional is the Norm: Shifting the Instruction Paradigm for Adult Students.” Link 1 & Link 2.

• Bennett, E. & Brothen, E (2010). Citation Analyses as a Prioritization Tool for Instruction Program Development. Journal of Library Administration. 50(5/6).

• Bennett, E., & Simning, J. (2010). Embedded Librarians and Reference Traffic: A Quantitative Analysis. Journal of Library Administration. 50(5/6).

• Sollien, J., Pohlman, J. & Waitz, E. ACRL 2009 “Challenges for Distance Students and Distance Librarians: Taking Advantage of the Online Environment” Link 2.

• ARLD Day 2009 “Reference Data in Action: Explore the Possibilities.”• Brothen, E., Berg, S., & Bennett, E. (2009). Maximizing the value of reference

data: A case study. In D. Orcutt (Ed.), Library data: Empowering practice and persuasion (Chapter 11). Santa Barbara, CA: Libraries Unlimited.

• Veal, R., & Bennett, E. (2009). The virtual library liaison: A case study at an online university. Journal of Library Administration, 49(1/2), 161-170.

• Bennett, E. (July 2008). eLearning: Libraries increasingly help steer the ship. Elsevier’s Library’s Connect Newsletter: v6(3).