TUNING USA:Utah’s Experience

William Evenson, Ph.D.Utah System of Higher Education

NAFSA2

Bologna Process and U.S. HE

• What parts of the Bologna Process could help U.S. HE?

• Structural changes– Degrees - ECTS - Diploma

Supplement• Quality changes within accepted (new

or old) structures– Qualifications/Degrees frameworks– Tuning by disciplines

5/29/2011

NAFSA3

U.S. Higher Education• Faculty autonomy• Institutional autonomy• General Education• Transfer articulation• Accountability• Transparency

5/29/2011

NAFSA4

Needs of U.S. HE• Improved quality, participation,

degree attainment• Greater attention to needs of

stakeholders outside the academy• Better communication about

student learning with students, alumni, faculty, advisors, librarians, employers, legislatures, parents

5/29/2011

NAFSA5

U.S. Higher Education (2)• Tuning and Degree Qualifications

Profiles are elements of Bologna that can help U.S. HE move forward

• Utah’s experience with Tuning, while still underway, has resulted in great benefits

5/29/2011

NAFSA6

Lumina Foundation’s Tuning USA Pilot Project

• Tuning– What does a student need to know,

understand, and be able to do to qualify for a degree in X?

• Focused on the quality, accountability, and transparency questions

• Faculty driven; set aside for now any structural changes

5/29/2011

NAFSA7

The Quality Question

Quality = Student Learning

Tuning focuses on student learning rather than on seat time in classes, number of classes, etc. Student outcomes rather than faculty inputs.

5/29/2011

NAFSA8

Tuning USA Pilot Project

• Three states – MN, IN, UT• Seven disciplines – history (2),

biology, chemistry, graphic design, physics, elementary education

• Started April 2009; first phase concluded December 2009

• Teams produced competencies, surveys, reflection on feasibility and process, suggestions for next steps

5/29/2011

NAFSA9

Utah’s Experience

• More than a decade of activities prepared Utah for Bologna-type initiatives

5/29/2011

NAFSA10

Utah’s Experience (2)• Majors’ Meetings – all eight

institutions in the Utah System of Higher Education, 37+ disciplines, annually

• Annual Conference on “What Is An Educated Person?” – faculty from all USHE institutions

5/29/2011

NAFSA11

Utah’s Experience (3)• Essential Learning Outcomes for

general education• “Tuning”

5/29/2011

NAFSA12

Utah’s Experience (4)• 8 state institutions

–2 research universities–4 comprehensive or regional

universities–2 community colleges

• 2 disciplines: history, physics–“AS/AA”, BS, (MS, Doctoral)

5/29/2011

NAFSA13

Utah’s Experience (5)• Faculty representatives from each

institution on each discipline team• Student representatives on each

team – reality check! & student focus groups

• Team members nominated by their departments and institutions

5/29/2011

NAFSA14

Define the Discipline• Our work began with in-depth

discussion of our joint discipline– How do we define it?– What are the essential competencies?– What competencies are taught in other

departments? In ours?– The central discipline, not

specializations at this point

5/29/2011

NAFSA15

Utah’s Experience (6)• Several sessions of discussion (3-4) were

required before teams took ownership– Understanding the process and how the

elements fit together– Understanding that Tuning is NOT

standardization!

• Faculty teams then agreed fairly quickly on common sets of discipline competencies – via faculty judgments!

5/29/2011

NAFSA16

General CompetenciesExamples of highly ranked general

competencies:• Oral and written communication• Abstract thinking, analysis and synthesis• Essential knowledge and understanding

of academic subjects and profession• Reasoned decision-making• Capacity to learn and update learning

5/29/2011

NAFSA17

General Competencies (2)Some lower ranked examples:• Able to work in an international

context• Able to motivate people and move

toward common goals• Commitment to environmental

conservation• Initiative, spirit of enterprise• Commitment to workplace safety

5/29/2011

NAFSA18

Discipline Competencies: HistoryExample• Historical Thinking

– recognizes the "pastness" of the past (awareness of continuity and change over extended time spans)

– recognizes the complex nature of past experience (a command of comparative perspectives, which may include the ability to compare the histories of different countries, societies, or cultures)

– recognizes the complex and problematic nature of the historical record (appreciation of the complexity of reconstructing the past, the problematic and varied nature of historical evidence)

5/29/2011

NAFSA19

Discipline Competencies: Physics7 Categories of Competencies• Understanding the Nature of Science and Nature

of Physics• Mathematical Skills, Modeling Skills, and Problem

Solving Skills• Understanding Physics Concepts• Laboratory Skills• Scientific Communication Ability (written, oral,

and visual communication)• Computational and Information Skills• Physics Research

5/29/2011

NAFSA20

Competencies to LOs• With competencies, the teams set

about defining learning outcomes (LOs) at the associate’s, bachelor’s, and master’s levels

• Common sets of competencies and agreed LOs do NOT prescribe how different institutions bring students to achieve the LOs – no prescription of curriculum or pedagogy, but exchange ideas

5/29/2011

NAFSA21

The “Ratchet Principle”

Ratchet up levels at which competencies are achieved from Associate’s to Bachelor’s to Master’s to Doctoral level

5/29/2011

NAFSA22

Concrete Examples

Physicists used examples to concretize the ratcheting of expectations

5/29/2011

NAFSA23

Levels of SophisticationAt each of the levels of sophistication

described below, students will be able to demonstrate the following: 1. Ability to identify physical laws by name and to provide definitions of important terms related to the physical laws2. Understanding of the meaning of physical laws and knowledge of their general formulas3. Ability to apply the general formulas or

concepts to specific limited situations

5/29/2011

NAFSA24

Levels of Sophistication (2)4. Ability to design or describe experiments that could test a specific

formula5. Understanding of the limits of validity of general formulas and the domains of validity of physical theories6. Understanding how empirical science functions, i.e. the supremacy of experiment and observation in establishing physical theory 5/29/2011

NAFSA25

Levels of Sophistication (3)7. Ability to apply physical laws across

different subdisciplines of physics and appreciation of common threads8. Ability to construct specific formulas

for specific situations from established general formulas9. Understanding of general physical

principles outside the context of their mathematical expression

5/29/2011

NAFSA26

Levels of Sophistication (4)10. Ability to construct mathematical

models from general principles without reference to other specific, limited-use formulas11. Ability to teach effectively and see

where common pitfalls in understanding occur

5/29/2011

NAFSA27

Ratcheting “Organizing a Physical Problem”

The AS-level student should demonstrate the ability to• Identify the physical principles that underlie a problem

from the introductory physics curriculum– Identify the relevant physical laws and know their names,

e.g. Coulomb’s law or Gauss’s law– Know the definitions of important terms or symbols in the

relevant physical laws– . . . .

• Express the meaning of the relevant physical laws or principles in words

• Draw appropriate schematic diagrams showing relationships among the elements of the problem

• etc.

5/29/2011

NAFSA28

Ratcheting “Organizing a Physical Problem” (2)

The BS-level student should demonstrate the ability to

• Do everything on the AS list, but for the more sophis-ticated problems in the BS curriculum. In addition, the BS-level student should be able to:

• Suggest experimental tests of the validity of the model embodied in the problem as the student has set it up

• Specialize general formulas for specific problems• Set up problems in more complicated geometries,

e.g. two- and three-dimensional problems or curvilinear coordinates

• Estimate the order of magnitude of expected results for problems involving multiple physical concepts

• etc.5/29/2011

NAFSA29

Ratcheting “Organizing a Physical Problem” (3)

The MS-level student should demonstrate the ability to• Do everything on the BS-level student list, but for the

still more sophisticated problems in the MS curriculum. In addition, the MS-level student should be able to:

• Set up problems combining several subfields of physics, e.g. mechanics and electricity & magnetism

• Teach problem organization and solving effectively to AS- and BS-level students

• Incorporate advanced mathematics (e.g., complex analysis, group theory) into problem solving

• etc.

5/29/2011

NAFSA30

Learning Outcome ExampleStudents shall demonstrate• Ability to organize problems by

identifying physical principles, identifying relevant vs. irrelevant quantities, and making appropriate diagrams

5/29/2011

NAFSA31

Consultations• Faculty consult stakeholders, which

builds credibility with policymakers and the public and informs the faculty about concerns of employers, alumni, etc.

• Ultimately, faculty define the discipline; consultations do not dictate our programs but keep us grounded in the realities of our context

5/29/2011

NAFSA32

Consultations (2)• Teams surveyed stakeholders about

general competencies– students– recent alumni– faculty– employers

• Surveyed other faculty in the disciplines about discipline competencies

5/29/2011

NAFSA33

Consultations (3)• Focus groups are superior to surveys• Consultations are an important part

of the process but present challenges of survey or focus group design, administration and analysis

5/29/2011

NAFSA34

Example Survey ResultsGeneral Competencies: Technical Employers

• Top Five Priorities (in descending order)– Able to work in a team– Oral and written communication– Able to identify, pose and resolve problems– Determination and perseverance in tasks and responsibilities– Able to plan and manage time

• Bottom Four Priorities– Shows awareness of equal opportunities and gender issues– Appreciation of and respect for ethnic, cultural and other diversity– Social responsibility and civic awareness– Commitment to environmental conservation

5/29/2011

NAFSA35

Example Focus Group Results

Top Skills Desired by History/Archives/Museums Employers

• Ability to research• Good communication (written and oral)• Critical thinking• Ability to organize• Passion for the subject• Knowledge of the subject area

5/29/2011

NAFSA36

Example from Focus Groups

Traits Desired of New Employees

• Personality• Qualifications• Experience• Training in specific areas• Customer service skills• Ability to manage a project

5/29/2011

NAFSA37

Other Elements of Tuning• Map employability

– Alumni surveys– Professional organizations

• Draft degree profiles– Institution by institution – what are

your particular strengths in bringing students to achieve the LOs

5/29/2011

NAFSA38

Evaluation of Tuning Process• Utah Tuners were generally

enthusiastic about–How Tuning shifts the focus from

inputs (lengths of a learning experience, type of institution, etc.) to what a person with a degree in the discipline actually understands and is able to do

5/29/2011

NAFSA39

Evaluation of Tuning Process (2)• Utah Tuners were also enthusiastic

about– Added transparency– Clear system of accountability with

assessment of LOs– Better match between educational

programs and needs of labor market for knowledge, skills and other competencies

5/29/2011

NAFSA40

What Does Tuning Add?• Discussions about student learning

across different institutions and sectors (to achieve appropriate consistency)

• Meaningful relationships among faculty members from various institutions and sectors, sharing experience and ideas

5/29/2011

NAFSA41

What Does Tuning Add? (2)

• Makes implicit expectations explicit• Increased attention to general

competencies – in addition to the discipline-specific competencies that most professional organizations focus on

5/29/2011

NAFSA42

What Does Tuning Add? (3)

• Led by faculty and a defense against accountability from above

• Involves employers/alumni/advisors as well as faculty/students in thinking about what degrees represent

5/29/2011

NAFSA43

What Does Tuning Add? (4)

• Facilitates the transfer of credit and degrees

• Aligns expectations across sectors and institutions

• Allows validation of non-traditional learning

5/29/2011