Fall 2007 Semester Program Assessment Report - ALTERNATE · Web viewAccepting 3 units of computer...

Program Assessment Report

ASSESSMENT REPORTINGAY 2010-2011

Date submitted: __2/23/2011__________

Degree Program(s): Minor, BA, BS, MA, Applied MA Department: Economics

Department Chair: Lydia Ortega Phone: 4-5400

Report Prepared by: Matthew Holian Phone: 4-1371

Next Self-Study due : Fall 2011 E-mail: [email protected]

Note: Schedule is posted at: http://www.sjsu.edu/ugs/programplanning/

ARCHIVAL INFORMATION

Location: DMH 131 Person to Contact: Matthew Holian 4-1371

(Bldg/Room #) (Name) (Phone)

Every five years, the Department of Economics assesses each of it's five SLO's: 1.1, 1.2, 1.3, 1.4, & 2.1, which are separate (though related) for both our undergraduate and graduate programs. The undergraduate and graduate SLOs 1.2 were assessed in Fall 2010.

1.2 – undergraduate - core principles of macroeconomicsMacroeconomic Hallmarks

Comparative Advantage (specialization and the gains from trade; globalization)· Macroeconomic Measures (real versus nominal calculations; components and concept of GDP;

components and concept of unemployment figures; calculation of inflation)· Macroeconomic Models (circular flow; monetary and fiscal policy; the market for loanable funds

and interest rate determination; the demand and supply of money and price level determination)

1.2 – graduate - the economy as a social and cultural system

To be knowledgeable about the economy as a social and cultural system with complex interactions within markets and between firms, governments, households and other organizations, at the local and global levels

of 27

mailto:[email protected]


Initial Evidence of Student Learning:

Undergraduate programs

As part of our program review, in Fall 2005 faculty in the department assessed performance of our Principles of Macroeconomics students. We implemented a national test, the TUSE (Test of Understanding College Economics.) Although there were problems with the assessment instrument, the results largely confirmed what faculty had long suspected: students were not achieving high enough performance levels in the area of Principles of Macroeconomics. Therefore, our assessment and curriculum committees have focused efforts over the last five years on ways of improving performance in this area. While the department has not ignored its other goals, SLO1.2 (Principles of Macroeconomics) is among our most important.

Graduate Programs

Assessment energies over the last few years have primarily been directed at the undergraduate programs. The last documented discussion about graduate programs occurred in from Spring 2006; the Assessment Report submitted that year is attached at the bottom of this report. Among the topics discussed at that time was poor performance on what were then newly instituted comprehensive exams. Also at that time, the faculty agreed to post past comprehensive exams to the Department website to help students prepare.

Change(s) to Curriculum or Pedagogy:

Undergraduate Programs

In Fall 2006, the Department implemented a major curricular change related to SLO1.2, in the form of moving our Principles of Macroeconomics (Econ 1A) courses from three to four credit units. The rationale behind this change was that students were underperforming because of a lack of studying. By adding a fourth unit, instructors would be able to develop online assignments, and these would count for between 20 and 25 percent of students’ grades.

At the same time, the Department began to consider making other strategic changes, in the form of updating and rewriting our SLOs. The rationale behind these discussions was to focus our attention on a few core areas. Reiterating the same key concepts across classes should serve to reinforce learning. This has lead to a focus on “hallmarks” for principles courses.

Finally, the Department has continued to modify and improve its data collection processes vis-à-vis assessment. Our most recent efforts are explained in the next section.

Graduate Programs

As noted in the Spring 2006 Assessment Report, the Comprehensive Exam replaced the Oral Exam, at the same time that admissions criteria were raised. The Department followed through with the idea of posting previous comprehensive exams online (see http://www.sjsu.edu/economics/graduate_students/comprehensive_exams/).

Evidence of Student Learning after Change.

of 27

http://www.sjsu.edu/economics/graduate_students/comprehensive_exams/


Undergraduate Programs

While we don’t have comparable data from before the curricular move to four-unit principles courses, it is likely that the average amount of time students spent working on homework was very low. However after the move, we are able to say with a good deal of precision how much time our students are spending on homework. This is because the online homework programs that instructors have implemented in their courses allow the instructor to observe, for each student, how much time he or she has spent working on homework per week. In informal discussion, many of the faculty estimate that the average time students spend working on homework has increased by orders of magnitude.

To assess the undergraduate SLO 1.2, the department conducted a ten question survey prior to final exams. Details on survey methodology is available from Professor Holian by request. We were able to achieve about a 67% response rate, reaching about 200 students enrolled in Econ 1A. One of the new features this year's assessment survey provides is a baseline for external comparison. In particular, the survey's three "test of knowledge" questions were taken from two previously published surveys, which provide data on average scores in various reference populations. The remaining seven survey questions are on gender, major, race and ethnicity, income, first-generation status, and previous experience with college economics.

The tables below show average scores for the entire sample and selected subsamples. It is noteworthy that correct response rates are greater than 90% on two questions, and above the external benchmarks on all questions. Overall, the survey suggests we are achieving SLO 1.2 with a fairly high degree of success for the undergraduate program, but that there is room for improvement, primarily in the area of macro models.

Two findings from the multiple regression analysis reported in Table 2 below are worthy of noting. First, having principles of macro before principles of micro does not seem to help students do better; this surprising obtained in both SP09 and FA09. However, in the results presented here, students who previously had principles of micro performed better, and this was marginally significant in one specification. These results suggest we should consider switching the numbering of principles of microeconomics (from 1B to 1A), and principles of macroeconomics (from 1A to 1B).

The second noteworthy findings is that the multiple regression analysis suggests there is a statistically significant gender-based achievement gap, but no achievement gaps due to race, ethnicity, income or foreign-born status.

The three questions we asked in this survey are shown below, and the correct answers follow.

1. Which one of the following is the most widely used measure of inflation?

of 27


a. The Consumer Price Index.b. The Index of Leading Economic Indicators.c. The prime rate.d. The Federal Funds rate.

2. What must the government do to reduce high inflation?

a. Increase both spending and the money supply.b. Decrease both spending and the money supply.c. Decrease spending and increase the money supply.d. Increase spending and decrease the money supply.

3. Which of the following occurs when one country trades wheat to another country in exchange for oil?

a. Both countries gain.b. Both countries lose.c. The country that trades wheat gains, the country that trades oil loses.d. The country that trades oil gains, the country that trades wheat loses.

4. What is your gender?

a. Femaleb. Male

5. What is your major or intended major?

a. Economicsb. Other major in the College of Social Sciencec. Other major in the College of Businessd. Other

6. Are you Hispanic?

a. Yesb. No

7. Which of the following best represents your race or ethnicity?

a. Asianb. Blackc. Caucasian (White)d. Hispanice. Other

8. What is the best estimate of your income, or if someone can claim you as a dependent on their tax returns, your family’s income, in 2010?

a. Less than 24,999b. $25,000 to 49,999c. $50,000 to 74,999d. $75,000 to 100,000e. more than $100,000

9. Were both of your parents born outside of the United States?

a. Yesb. No

10. You are taking this exam because you are currently enrolled in macroeconomics; have you ever had, or are you currently taking a college course in microeconomics?

a. Yesb. NoCorrect Answers, SLO Hallmark Matching, and Question Sources (Benchmarking)

of 27


Macroeconomic Measures Hallmark

Which one of the following is the most widely used measure of inflation?a. The Consumer Price Index.b. The Index of Leading Economic Indicators.c. The prime rate.d. The Federal Funds rate.

Correct answer: (A) The Consumer Price Index.Gen public, 35%, HS seniors, 35%, College Seniors 56%

Macroeconomic Models Hallmark

What must the government do to reduce high inflation?a. Increase both spending and the money supply.b. Decrease both spending and the money supply.c. Decrease spending and increase the money supply.d. Increase spending and decrease the money supply.

Correct answer: (B). 30 % Answered Correctly

Comparative Advantage Hallmark

Which of the following occurs when one country trades wheat to another country in exchange for oil?a. Both countries gain.b. Both countries lose.c. The country that trades wheat gains, the country that trades oil loses.d. The country that trades oil gains, the country that trades wheat loses.

Answer: (A). 70 % Answered Correctly

Sources for correct answers:

The first question comes from http://ecedweb.unomaha.edu/ecedweek/quiz.htmThe last two questions, and the rationale for the correct answers, are based on the "Test of Economic Literacy," Second Edition, Examiner's Manual, by John C. Soper and William Walstad, Joint Council on Economic Education, 1987. The reported % answered correctly come from the Economic Literarcy Survey: http://www.minneapolisfed.org/publications_papers/pub_display.cfm?id=3580

of 27

http://www.minneapolisfed.org/publications_papers/pub_display.cfm?id=3580

http://ecedweb.unomaha.edu/ecedweek/quiz.htm


Table 1 Summary Statistics

Variable Obs Mean Std. Dev. Min Maxq1_correct 196 0.918367 0.274505 0 1q2_correct 196 0.372449 0.484695 0 1q3_correct 196 0.908163 0.289535 0 1num_cor 196 2.19898 0.653353 0 3male 194 0.530928 0.500334 0 1econ_major 195 0.020513 0.142111 0 1other_COSS_major 195 0.020513 0.142111 0 1other_COB_major 195 0.820513 0.384748 0 1other_major 195 0.138462 0.346273 0 1hispanic_y_or_n 187 0.187166 0.391092 0 1asian 196 0.397959 0.49073 0 1black 196 0.05102 0.220603 0 1caucasian 196 0.265306 0.442627 0 1hispanic 196 0.168367 0.37515 0 1race_other 196 0.081633 0.274505 0 1income_L_24999 189 0.253968 0.436436 0 1income_25000_49999 189 0.190476 0.39372 0 1income_50000_74999 189 0.222222 0.416844 0 1income_75000_100000 189 0.174603 0.380636 0 1income_G_100000 189 0.15873 0.366395 0 1previous_micro 190 0.6 0.491192 0 1

Tabulation of Test of Knowledge Question Responses

answer | Freq(Q1) Freq(Q2) Freq(Q3) ----------+----------------------------------- 1 | 180 17 178 2 | 4 73 3 3 | 2 29 12 4 | 10 77 3----------+-----------------------------------

of 27


Multiple Regression Analysis

Table 2 Regression Results (Ordinary Least Squares)

VARIABLES numcor numcor numcor

male 0.236** 0.190** 0.232**(0.10) (0.09) (0.10)

asian -0.14 -0.0986(0.18) (0.18)

black -0.345 -0.233(0.26) (0.26)

caucasian 0.197 0.179(0.19) (0.19)

hispanic 0.173 0.212(0.20) (0.20)

econ_major 0.45 0.374 0.446(0.34) (0.34) (0.34)

other_COSS_major -0.447 -0.571 -0.529(0.46) (0.47) (0.47)

other_COB_major 0.138 0.0909 0.1(0.14) (0.14) (0.14)

foreignparents 0.125 0.0181 0.12(0.12) (0.10) (0.12)

previous_micro 0.152 0.182* 0.151(0.10) (0.10) (0.10)

income_L_24999 -0.201 -0.212(0.14) (0.14)

income_25000_49999 0.153 0.0604(0.15) (0.15)

income_75000_100000 -0.0245 -0.0538(0.15) (0.15)

income_G_100000 0.196 0.126(0.15) (0.16)

Constant 1.780*** 1.894*** 1.823***(0.24) (0.19) (0.26)

Observations 186 186 186R-squared 0.125 0.112 0.155

Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1

of 27


The table below summarizes the methodology we will use to assess our Undergraduate SLOs.

Course-to-SLO Map, Undergraduate Program

SLO Brief Description Method of Assessment1.1 Principles of Microeconomics Perform Assessment / Achievement Gap Survey

(1B)1.2 Principles of Macroeconomics Perform Assessment / Achievement Gap Survey

(1A)1.3 Analytical methods Perform Assessment / Achievement Gap Survey

(101 for BA program & minor; 103 for BS program)1.4 Fields:

1. International Economics2. Financial Economics3. Policy Economics 4. Quantitative Methods

Perform Four Assessment / Achievement Gap Surveys 1.) 112, 136 & 1582.) 135, 137A, 137B, 139 3.) 121, 132, 141, 151, 1664.) 103, 104, 138 (Fields 1-4 assessed for BA program; Field 4 emphasized for BS program)

2.1 How to apply core economic theory and reasoning

Faculty Focus Group(Faculty bring examples of undergraduate student writing, e.g. class papers, to retreat, and use a rubric to assess student writing)

Graduate Program

The table below summarizes the methodology that we will use to assess our Graduate SLOs.

Course-to-SLO Map, Graduate Program

SLO Brief Description Method of Assessment1.1 Basic economic models Analyze results from comprehensive exam, section I

(104 and 201)1.2 Economic interdependencies Analyze results from comprehensive exam, section II

(202 and 235)1.3 Research methods Analyze results from comprehensive exam, section III

(205A)1.4 Fields: 1. International,

2. Finance, 3. Policy, and 4. Quantitative Methods

Analyze comprehensive exams, sections II & III (1. 158, 212; 2. 202, 235; 3. 132 or 232, 121 or 221; 4. 103 or 203)

2.1 Conduct research Analyze final reports submitted in Policy Analysis Workshop (205B)

of 27


To assess graduate SLO 1.2, we analyzed scores on section II of the Fall 2010 comprehensive exam, and specifically the Econ 202 (macroeconomic theory) and 235 (money and banking) questions. The questions asked over this period can be found at the following link: http://www.sjsu.edu/economics/graduate_students/comprehensive_exams/

The comprehensive exam results for questions 2A (ECON 202) and 2B (ECON 235) are summarized in the tables below.

Fall 2008 - Fall 2010 (covers five exam periods)Questions A and B combined results

Number ofFull Passes 16Marginal Passes 17Fails 8 (19.5%)Total section II question attempts

64

Spring 2006 - Spring 2008 (covers five exam periods)Question A and B combined results

Number ofFull Passes 30Marginal Passes 32Fails 27 (30.3%)Total section II question attempts

107

A comparison of the results presented in the two tables above show that our students have been passing SLO 1.2 questions at a higher rate in the more recent period. Obviously there are many factors to consider, but it is possible that the decrease in the rate of failure were due to improvements in student learning, and possibly as a result of giving students access to previous versions of the exams. Or the stricter admission standards may have lead to a sample selection effect.

of 27

http://www.sjsu.edu/economics/graduate_students/comprehensive_exams/


Assessment Reporting Spring 2009 – Spring 2010

Program InformationDegree Program(s): BA and BS Department: Economics



Student Learning Outcome (SLO)

Core principles of microeconomics, namely:

Incentives Matter (law of demand; law of supply; rational decision makers weight marginal costs versus marginal benefits; the power of self-interest)

Opportunity Costs (sunk costs; production possibilities; the free-lunch fallacy; tradeoffs in consumption and production; gains from interpersonal and international trade; comparative advantage)

Supply and Demand (understanding the S&D model as a representation of individual choices in exchange based on individual preferences, knowledge and circumstances; ability to examine current events using S&D tools; movement versus shift; welfare analysis.)

Evidence for Need: As part of our program review, in Fall 2005 faculty in the department assessed performance of our Principles of Microeconomics students. We implemented a national test, the TUSE (Test of Understanding College Economics.) Although there were problems with the assessment instrument,1 the results largely confirmed what faculty had long suspected: that students were not achieving high enough performance levels in the area of Principles of Microeconomics. Therefore, our assessment and curriculum committees have focused efforts over the last five years on ways of improving performance in this area. While the department has not ignored its other goals, SLO1 (Principles of Microeconomics) is arguably our most important SLO; it provides the basis for much of the subsequent work in our BA and BS programs, is a GE (general education) course, and in addition is a required course for students in the College of Business.

Changes to Curriculum or Pedagogy: In Fall 2006, the Department implemented a major curricular change related to SLO1, in the form of moving our Principles of Microeconomics (Econ 1B) courses from three to four credit

1 See Fall 2008 Assessment Report, on file at: http://www.sjsu.edu/ugs/docs/assess_report_08fall/06-Economics-BA.doc

of 27

http://www.sjsu.edu/ugs/docs/assess_report_08fall/06-Economics-BA.doc

http://www.sjsu.edu/ugs/docs/assess_report_08fall/06-Economics-BA.doc


units. The rationale behind this change was that students were underperforming because of a lack of studying. By adding a fourth unit, instructors would be able to develop online assignments, and these would count for between 20 and 25 percent of students’ grades.

At the same time, the Department began to consider making other strategic changes, in the form of updating and rewriting our SLOs. The rationale behind these discussions was to focus our attention on a few core areas. For example, most students take not only Econ 1B from the Department, but also Econ 1A (Principles of Macroeconomics). The curriculum of Econ 1A also covers some of the same material as does Econ 1B, and so reiterating the same key concepts in both classes should serve to reinforce learning.

Finally, the Department has continued to modify and improve its data collection processes vis-à-vis assessment. Our most recent efforts are explained in detail in the report titled, “A New Tool for Integrating Statistical Analysis, Socioeconomic and Demographic Data, and Program Assessment.”2

Evidence for Impact:

Given the changes to our assessment data collection processes mentioned above, we were not able to conduct before and after analyses of student performance data. However, our qualitative assessment, which consisted largely of structured faculty discussions and information sharing sessions, suggests that the move to four-unit classes has been very effective.

While we don’t have hard data, before the move, it is likely that the average amount of time students spent working on homework was very low. However after the move, we are able to say with a good deal of precision how much time our students are spending on homework. This is because the online homework programs that instructors have implemented in their courses allow the instructor to observe, for each student, how much time he or she has spent working on homework per week. Most faculty estimate that now, the average time students spend working on homework has at least doubled, and in some cases has increased by a factor of three or four.

At the same time, our new assessment data collection procedures suggest that there is still room for improvement in the area of SLO1.3 The Department remains focused on further improving our students’ performance in this area. Future initiatives will include assessing the effectiveness of our online modules, and working to further increase our faculty’s focus on a common set of core principles.

ASSESSMENT REPORT, FALL 2009

2 This report is on file at: http://www.sjsu.edu/ugs/docs/assess_report_10spr/COSS/Holian_Assessment_Instrument.pdf.3 These data are summarized in the report titled, “A New Tool for Integrating Statistical Analysis, Socioeconomic and Demographic Data, and Program Assessment.”

of 27

http://www.sjsu.edu/ugs/docs/assess_report_10spr/COSS/Holian_Assessment_Instrument.pdf


Degree Program(s): BA, BS Department: Economics



E-mail: [email protected]

Next Program Review: 2010-2011



Location: Person to Contact:Economics Department DMH Matt Holian 924-1371

Does the information (e.g., Mission, Goals, and/or Learning Outcomes) posted on the web (see, http://www.sjsu.edu/ugs/assessment/programs/ ) for this program need to be updated?

If yes, please submit changes to [email protected]

SCHEDULE OF ASSESSMENT ACTIVITIES*Please complete the schedule of assessment activities below by listing all program Student Learning Outcomes (SLOs) by number down the left column and indicating when data were/will be collected (C) and when they were/will be discussed (D) by your faculty. You can also schedule/track program changes resulting from your assessment activities by indicating an “I” (implemented changes) where relevant. This schedule is meant to be fluid; providing a proposed schedule for future assessment while at the same time, providing a record of your efforts as the program planning cycle progresses.

↓Semester after Program Review Semester before next Program Review↓

SLOs F05 S06 F06 S07 F07 S08 F08 S09 F09 S101 C D D I I I D C D,C, I D,C, I2 C D D I I I D C D,C, I D,C, I3 C D D I I I D C D,C, I D,C, I4 C D D I I I D C D,C, I D,C, I5 C D D I I I D C D,C, I D,C, I67

C D D I I I D C D,C, I D,C, I7 C D D I I I D C D,C, I D,C, I8 C D D I I I D C D,C, I D,C, I

*Note: This template is based on a five-year program planning cycle. If your program planning follows another cycle (e.g., based on accreditation), please feel free to add (or subtract) columns as necessary.

Introduction

Our SLOs have been written. This was part of an overhaul of the department’s entire strategic management process. The major changes were to drastically reduce the number of SLOs to facilitate

of 27

NoYes

http://www.sjsu.edu/ugs/assessment/programs/



complete program assessment in a five-year cycle. In addition, the mission and goals were streamlined. Finally, the mission, goals and SLOs were written in a way such that the SLOs now flow from the mission and goals.

1. Core Principles of Microeconomics

1.1 Data Collection:

Spring 2009. In cooperation with the Office of Institutional Research (OIR), we assessed students in all of our principles of microeconomics class. Results of this survey are provided in detail in an attached report. The focus on this survey was 1.) overall performance, and 2.) breakdown by race.

1.2 What have you learned about this Student Learning Outcome?Spring 2009. Our students are still not performing at an acceptable level. In addition, we potentially hae large achievement gaps. However, future studies will attempt to control for gender and income effects.

1.3 Action Item:

We need to continue to emphasize the three hallmarks for microeconomics in our classes. This can be accomplished by ensuring that each faculty member lists them on his or her syllabus

Disseminate the results of survey to faculty members.

1.4 Results of Action Items

1. Scores will improve2. Faculty will focus greater attention to the diversity of learning styles among our students

2. Core Principles of Macroeconomics

No activity to report

of 27


3 Statistical Interpretation

We plan to ask instructors teaching: ECON 003: Economic Statistics; ECON 103: Econometrics to discuss assessment, and to provide a paragraph for this report.

4 Core Principles from Specialist Areas


Our field classes are broken into four areas. We will begin our new data collection and analysis process by collecting and analyzing the grades students receive in each of the following classes:

Area I, International Economics

ECON 112: Economic Development; (Haight)ECON 136: International Economics; (Haight)

Area II, Financial Economics

ECON 135: Money & Banking' (HummelECON 137A: Fundamentals of Corp. Finance; ECON 137B: Topics in Corp. Finance; ECON 139: Principles of Investment

Area III, Public Policy Economics

ECON 121: Industrial Organization, ECON 132: Public Finance; ECON 141: Law & Economics; ECON 151: Labor Economics; ECON 166: Urban Economics

Area IV, Quantitative Economics

ECON 103: Econometrics, ECON 104: Mathematical Methods for Economics ,

of 27

http://info.sjsu.edu/web-dbgen/catalog/courses/ECON104.html








http://info.sjsu.edu/web-dbgen/catalog/courses/ECON137B.html

http://info.sjsu.edu/web-dbgen/catalog/courses/ECON137A.html







5 Analytical Methods and Model-Based Argument


We are developing a question to test SLO 5 on the exit exam, which we plan to administer online through the OIR.

5.2 What have you learned about these Student Learning Outcomes?Fall 2008. We have little hard data on SLO 5. Casual empiricism and faculty discussion suggests that our students are very good at analyzing the costs and benefits of projects, and enter or return to the workplace with highly developed analytical skills.

5.3 Action Item

We need to develop and implement the exit exam. After assessing this SLO, we may consider shifting some focus to certain areas, if we find that our students show any deficits.


Fall 2008. Teaching and learning will improve and classroom content will better match student needs and institutional goals.

6 Relationship between Verbal, Mathematical, Statistical and Graphical Ideas

6.1 Data Collection:We are developing a question to test SLO 6 on the exit exam, which we plan to administer online through the OIR.


6.3 Action Item We need to develop and implement the exit exam. After assessing this SLO, we may consider

shifting some focus to certain areas, if we find that our students show any deficits.

6.4 Results of Action ItemsFall 2008. Teaching and learning will improve and classroom content will better match student needs and institutional goals.

of 27


7 Applying Economic Theory to Various Social Settings



7.3 Action Item



8 Evaluating Government Policy




8.3 Action Item



of 27


ASSESSMENT REPORT, FALL 2008

Degree Program(s): BA, BS Department: Economics



E-mail: [email protected]

Next Program Review: 2010-2011



Location: Person to Contact:Economics Department DMH Matt Holian 924-1371

Does the information (e.g., Mission, Goals, and/or Learning Outcomes) posted on the web (see, http://www.sjsu.edu/ugs/assessment/programs/ ) for this program need to be updated?

If yes, please submit changes to [email protected]

SCHEDULE OF ASSESSMENT ACTIVITIES*Please complete the schedule of assessment activities below by listing all program Student Learning Outcomes (SLOs) by number down the left column and indicating when data were/will be collected (C) and when they were/will be discussed (D) by your faculty. You can also schedule/track program changes resulting from your assessment activities by indicating an “I” (implemented changes) where relevant. This schedule is meant to be fluid; providing a proposed schedule for future assessment while at the same time, providing a record of your efforts as the program planning cycle progresses.

↓Semester after Program Review Semester before next Program Review↓

SLOs F05 S06 F06 S07 F07 S08 F08 S09 F09 S101 C D D I I I D C D,C, I D,C, I2 C D D I I I D C D,C, I D,C, I3 C D D I I I D C D,C, I D,C, I4 C D D I I I D C D,C, I D,C, I5 C D D I I I D C D,C, I D,C, I67

C D D I I I D C D,C, I D,C, I7 C D D I I I D C D,C, I D,C, I8 C D D I I I D C D,C, I D,C, I

*Note: This template is based on a five-year program planning cycle. If your program planning follows another cycle (e.g., based on accreditation), please feel free to add (or subtract) columns as necessary.

Introduction

of 27

NoYes

http://www.sjsu.edu/ugs/assessment/programs/



Our SLOs have been written. This was part of an overhaul of the department’s entire strategic management process. The major changes were to drastically reduce the number of SLOs to facilitate complete program assessment in a five-year cycle. In addition, the mission and goals were streamlined. Finally, the mission, goals and SLOs were written in a way such that the SLOs now flow from the mission and goals.

3. Core Principles of Microeconomics


Spring 2009. In cooperation with the Office of Institutional Research (OIR), we assessed students in all of our principles of microeconomics class. Results of this survey are provided in detail in an attached report. The focus on this survey was 1.) overall performance, and 2.) breakdown by race.

1.2 What have you learned about this Student Learning Outcome?Spring 2009. Our students are still not performing at an acceptable level. In addition, we potentially hae large achievement gaps. However, future studies will attempt to control for gender and income effects.

1.3 Action Item:

We need to continue to emphasize the three hallmarks for microeconomics in our classes. This can be accomplished by ensuring that each faculty member lists them on his or her syllabus

Disseminate the results of survey to faculty members.


3. Scores will improve4. Faculty will focus greater attention to the diversity of learning styles among our students

of 27


4. Core Principles of Macroeconomics

No activity to report

3 Statistical Interpretation

We plan to ask instructors teaching: ECON 003: Economic Statistics; ECON 103: Econometrics to discuss assessment, and to provide a paragraph for this report.

4 Core Principles from Specialist Areas


Our field classes are broken into four areas. We will begin our new data collection and analysis process by collecting and analyzing the grades students receive in each of the following classes:

Area I, International Economics

ECON 112: Economic Development; (Haight)ECON 136: International Economics; (Haight)

Area II, Financial Economics

ECON 135: Money & Banking' (HuECON 137A: Fundamentals of Corp. Finance; ECON 137B: Topics in Corp. Finance; ECON 139: Principles of Investment

Area III, Public Policy Economics

ECON 121: Industrial Organization, ECON 132: Public Finance; ECON 141: Law & Economics; ECON 151: Labor Economics; ECON 166: Urban Economics

Area IV, Quantitative Economics

ECON 103: Econometrics, ECON 104: Mathematical Methods for Economics ,

of 27









http://info.sjsu.edu/web-dbgen/catalog/courses/ECON137B.html

http://info.sjsu.edu/web-dbgen/catalog/courses/ECON137A.html







5 Analytical Methods and Model-Based Argument




5.3 Action Item



Fall 2008. Teaching and learning will improve and classroom content will better match student needs and institutional goals.

6 Relationship between Verbal, Mathematical, Statistical and Graphical Ideas



6.3 Action Item We need to develop and implement the exit exam. After assessing this SLO, we may consider

shifting some focus to certain areas, if we find that our students show any deficits.


of 27


7 Applying Economic Theory to Various Social Settings



7.3 Action Item



8 Evaluating Government Policy




8.3 Action Item



of 27


DEPARTMENT OF ECONOMICSASSESSMENT REPORT

SPRING 2006

Undergraduate Program Changes Related To Assessment

The Department’s process of faculty feedback on student learning has led to substantial changes, most notably, the addition of an online lab to introductory courses. This development was in response to widespread faculty dissatisfaction with student learning in the introductory courses. In discussions, faculty narrowed the problem to “How do we get students to do more independent work.” Faculty members had taken a variety of approaches involving more frequent tests, reviews, and short assignments designed to motivate students to work outside of the classroom. Unfortunately, those assignments required grading. Cursory grading (e.g. a check mark if the assignment was turned-in) resulted in lackadaisical student effort. Rigorous grading meant that fewer assignments could be distributed and returned in a timely manner. More assignments meant that each individual assignment counted for a smaller fraction of the student’s grade.

Some obvious solutions—student assistants to do grading or to conduct required tutorials were considered but quickly dismissed because there are no resources for student assistants to do grading or facilities to have physical labs for tutorials. There are over 800 students in introductory courses each semester. The solution presented itself in a company (APLIA) that had been trying to meet with the Chair about their online labs in economics. In 2001 – 2002 Professors Ortega and Stringham conducted extensive testing and evaluation of the software from a pedagogical and student-use perspective. In Fall 2002 the faculty reviewed a presentation made by APLIA and passed a detailed proposal weighing the pros and cons of incorporating a one-unit online lab in introductory courses. Once the proposal passed the Department, there was a challenging set of implementation hurdles. The online labs have been running since Fall 2003. A Spring 2004 focus group discussion by faculty revealed that it would take several more semesters of use before it would be appropriate to concretely evaluating the impact of this change on learning. It was a major curriculum change that required time for each faculty member to learn and incorporate in their teaching.

In Fall 2005 the Department conducted national pre- and post- testing in introductory courses. This data, in conjunction with identified learning objectives, will be used in 2006-2007 when the Department evaluates the learning effectiveness of online labs. It is not anticipated that the online labs will be eliminated, rather that a discussion of best practices, perhaps technical support to incorporate the labs into classroom discussion, and presentations by other newly developed competing companies will assist the Department in developing a model for blending technology with traditional chalk and talk lectures.

Other Assessment Changes: Similar faculty feedback about poor performance among economics majors in economic content and writing, and among high school teachers led to the following program changes: (Note: Most of the changes listed below became effective in Fall 2003. The online Writing Support page was considered in Spring 2005 and should be effective Spring 2006.)

THE DEPARTMENT NO LONGER ACCEPTS 3 UNITS OF COMPUTER SPREADSHEET AS A SUBSTITUTE FOR AN ECONOMICS ELECTIVE, BUT ACCEPTS 3 UNITS OF BASIC ACCOUNTING. Accepting 3 units of computer spreadsheet or database management for one econ elective was a very old condition that began with accepting 3 units of programming. Students are increasingly familiar with word processing, spreadsheets, and networking. Students have very strong private incentives to learn these methods. Alumni surveyed in the last review indicated that accounting was highly desirable/beneficial for economics majors.

of 27


THE DEPARTMENT MOVED 3 UNITS FROM THE ANY-COURSE-ELECTIVE CATEGORY TO THE ECONOMICS-COURSE-ELECTIVE CATEGORY. The Department concluded that further training in economics was required hence the increase in economics course electives. The Department concluded that the B.A. and B.S. still presented ample opportunities for interdisciplinary learning with the remaining 28 and 24 any course elective units.

The Department renumbered Econ 103A to be a lower division course Econ 3. ECON 103A COVERS THE MATERIAL CONVENTIONALLY TAUGHT IN LOWER DIVISION COURSES. WE ALLOW STUDENTS TO TRANSFER 3 UNITS OF LOWER DIVISION STATISTICS WITH THE PROVISION THAT THEY MAKE-UP THE MISSING UNIT OF LAB.

THE DEPARTMENT DESIGNATED ECON 109 TO PROVIDE INSERVICE AND PRE-SERVICE TRAINING FOR HIGH SCHOOL TEACHERS. The Center for Economic Education concludes that one course, directly targeted at the interests and issues facing current and future teachers will be more effective than traditional Econ 1A and 1B courses. Economic majors cannot count Econ 109 in the major but can count it towards a double major.

THE DEPARTMENT RESPONDED TO DEFICIENCIES IN WRITING IN ECON 1B (GENERAL EDUCATION COURSE) AND ACROSS THE DISCIPLINE BY INSTITUTING A WEB PAGE OF ONLINE WRITING SUPPORT. The use and effectiveness of this proposal remains to be evaluated in the next review period. In brief, the web page consists of online resources (power-point, online self-testing) to practice writing. Faculty will receive a handbook of the lessons developed by category (e.g., Subject-Verb agreement). The handbook will also include a variety of short, multiple-choice and short-answer tests corresponding to each online category. Faculty will have the option of referring students for self-study of specific assignments based on evaluation of writing and then assessing the work with a brief test.

The Department continues its evaluation of the undergraduate program and in Fall 2005, developed a formal set of learning goals and objectives, mapped these goals into each course, and developed priorities for assessment.

STUDENT LEARNING GOALS

1. To stimulate students intellectually through the study of Economics and to lead them to appreciate its applicability to a range of problems and its relevance in a variety of contexts.

2. To develop familiarity with the major facts and institutions of the United States economy, other national and regional economies and the world economy.

3. To develop a fundamental understanding of modern economic theory, of its scope and limitations, and of its applications in the interpretation of facts and the formulation of policy.

4. To equip students with basic but relevant mathematical and statistical skills.

5. To develop in students the ability to apply the knowledge and skills they have acquired to the solution of theoretical and applied problems in economics, and an appreciation of the appropriate level of abstraction in the construction of models used as solution tools.

6. To develop in students, through the study of economics, a range of transferable skills that will be of value in employment and self-employment.

of 27


7. To generate in students an appreciation of the economic dimension of wider social and political issues.

The faculty developed seven learning objectives based on the above learning goals. These seven learning objectives were mapped into each undergraduate course. For assessment purposes the seven learning objectives can be grouped into three proficiencies adapted Hansen’s “What Knowledge is Most Worth Knowing for Economics Majors?” American Economics Review, May 1986.

LEARNING OBJECTIVES

PROFICIENCY 1: DISPLAYING COMMAND OF EXISTING ECONOMIC KNOWLEDGE

1. Explain theoretical concepts and demonstrate how they can be use in microeconomic theory

2. Explain theoretical concepts and demonstrate how they can be use in macroeconomic theory

3. Demonstrate how to interpret statistics and apply probability methods in situations involving risk and in quantitative evaluation of economic data.

4. Explain core principals and demonstrate how they can be use from specialized areas in economics (e.g., labor, industrial organization, economic development).

PROFICIENCY 2: DISPLAYING THE ABILITY TO DRAW OUT EXISTING ECONOMICS KNOWLEDGE

5. Identify the relationships between verbal, graphical, mathematical and statistical representations of economic ideas and analysis.

6. Demonstrate how to apply core economic theory and reasoning to topics of practical interest to governments, firms, households and other institutional and social groups.

PROFICIENCY 3: UTILIZING ECONOMIC KNOWLEDGE TO EXPLAIN ECONOMIC ISSUES

7. Demonstrate how to discuss, analyze, and write about economic policy issues and to assess the performance of the U.S. and other economies.

METHODS FOR ASSESSING LEARNING OBJECTIVES

Proficiency 1 is traditionally evaluated by classroom examinations that require the explanation of theoretical concepts and the demonstration of how they can be used across introductory, intermediate, and advanced economics courses). Information from examination performance has been the Department’s main input for the graduate and undergraduate program changes already instituted. The Department will continue using

of 27


regular examinations to evaluate Proficiency 1 but will also add a new component—a nationwide comparative examination. The Nationwide Comparison assessment method, uses nationally produced tests and provides information on how SJSU Economics students are learning relative to others. The high cost (monetary and time) however, means that this assessment method can be conducted only periodically, approximately every three to four years.

The Nationwide Comparison Assessment method was adopted for implementation Fall

2005. The Department used TUCE (Test of Understanding College Economics) a

nationally produced and normed test. This assessment allows the Department to do a

direct, national comparison of learning objectives across introductory economics

students and to perform an indirect, in-house comparison of learning across upper

division majors. Students enrolled in Intermediate Micro and Macroeconomics will also

take the pre- and posttests and be compared to benchmark learning by principles

student at SJSU and nationwide. This assessment method allows the Department to

make evaluations of valued-added for students who go through the major relative to

students who complete only the introductory courses.

The Department’s focus for assessment for 2006-2007 will be on identifying curricular

changes associated with the results of this Nationwide Comparison particularly as it

pertains to the impact of the online labs on learning objectives 1, 2, and 4..

Proficiency 2 is traditionally evaluated by written assignments and classroom presentations. For example, students may be asked to analyze how economic concepts are used by the popular press (introductory and upper division courses) and to read and interpret published professional economic analyses (upper division courses). One method that other departments have used to assess these assignments is the collection and review of poor, good, and excellent writing examples. This assessment method has been used and abandoned because the method fails to draw out “more effective” conclusions from the collected data than from the instructor’s summative, subjective assessment. That is, an assessment of this assessment method of writing assignments is that it does not produce better or different information than the instructor’s summary of student learning and it comes with high costs—particularly, the rapid inundation of data for processing and storage.

For the time being, the Department decided to institute regularly scheduled faculty discussion as the method of assessing Proficiency 2. This method acknowledges the frequent, yet often informal assessment that currently occurs. It also acknowledges the effectiveness and role of faculty assessment discussions in the Department’s recent, substantive and constructive curriculum changes. One to two

of 27


hours of the first faculty meeting of each academic year will be devoted to discussion of learning based on the objectives for each course. Institutionalization of this feedback mechanism coupled with the establishment of learning goals and objectives, allows the Department faculty to constructively evaluate the link between curricular changes with improved student learning.

Note, there is overlap in assessment mechanisms of learning goals. For example, issues in objective 5, listed under Proficiency 2 are evaluated in Proficiency 1 examinations. Objective 5 is placed in Proficiency 2 area to highlight the importance of understanding the relationships between verbal, graphical, mathematical and statistical representations of economic ideas and analysis in the ability to draw out existing economic knowledge. In other words, the student should be proficient in “translating” economic concepts from common parlance, graph, or statistics, into formal economic parlance.

While assessment of Proficiency 2 will not be a priority, in part because it has already produced substantial changes, it will not be entirely neglected because formalized discussion of learning objectives has been instituted for each fall semester.

Proficiency 3 has been evaluated using oral examinations of graduating seniors. At one university, at least two faculty members questioned students on the student’s research paper. This approach was abandoned, for some of the same reasons the Department abandoned oral comprehensive exams: high cost of faculty time and the inability to effectively evaluate economic knowledge because of student anxiety. The university, a small liberal arts college, switched to oral exams based on a student paper asking for reflection on student learning objectives. Preliminary evidence, suggest that students are more relaxed on discussion of reflections and the information may be useful for assessment-based changes. Given the Department’s size, individualized oral examination is not feasible. Instead, the Department is considering institution of a Senior Exit Survey. Assessment of Proficiency 3 is still in the development stage with expected design and implementation in Fall 2009.

The Department is considering the following uses of an exit survey: ask students, submitting graduation forms to complete an online survey; after 5 years send the survey electronically to alumni for a follow-up assessment. Students would provide a “permanent” email address when they complete the survey the first time. This second attempt would allow for evaluation of perceptions of student learning over time and within a work context. This is a work in progress, but the Department has provided a rough sample of questions that would be asked on the exit survey.

Graduate Program Changes Related To Assessment

During 2002 the Department curriculum committee completed an extensive review of the M.A. program. The most significant change in the Graduate program also resulted from dissatisfaction with the method of assessing student learning. Formerly, the Department used an oral exam for the program’s culminating experience. In Fall 2003 the Department moved from oral comprehensive exams to written comprehensive exams. This change has more evenly distributed the workload for the graduate program across all professors who now submit and grade questions. Previously, the work of oral exams fell heavily on a few professors who were more readily available. The change has also increased the quality of the program and expected performance of graduate students, because exams are graded anonymously and because students, mainly international students, are not unduly hindered by public speaking anxiety.

of 27


The Department continues its evaluation of the graduate program and made the following changes at the Fall 2005 faculty retreat. They are based on faculty findings of student learning and on student feedback regarding the comprehensive exams.

Faculty voted to increase the requirements for admission to the MA program. Effective Fall 2006, the requirements will be as follows:

At least a four year B.A. or B.S. degree in any discipline. A 3.0 minimum GPA on the most recent 60 semester units (or 90 quarter units) of coursework. Applicants with a GPA of 2.5 - 2.99 will be considered on a space available basis. These

applicants must submit GRE scores of at least 1000 including a minimum of 550 on the verbal, plus 3.0 or higher (scale of 0-6.0) on the Analytical Writing Section of the General Test).

In addition, International Applicants must have a TOEFL of 575 (paper version) or 225 (computer version) or provide records showing that the primary language of undergraduate instruction was in English.

The faculty adopted limited use of GRE scores to allow students with a poor record to demonstrate ability for the program. Previously, such students had to take courses at community colleges to change the grade point average of the last 60 units. The faculty also agreed to post past comprehensive exams to the Department website to help students better prepare for the exams.

In the next five years, the Department will develop better methods for tracking the quality of applicants—their University, degree, and whether they were admitted as classified or conditionally classified. The Department will also track the number of accepted applicants who actually enroll. The goal of better tracking of graduate students has two objectives: to gauge the Department’s performance over time as measured by reputations and demand, and to track resources, particularly from international and out-of-state fees.

The following summary of the characteristics of applicants from the Spring 2006 pool serves as a benchmark for future comparisons of graduate program applicants is from the Spring 2006 pool. In general, the quality of applicants is approving. More applicants enter the program as classified students because they majored in economics, double majored or minored in economics. Still the program is open to a variety of majors including: Political Science, Marketing, Applied Mathematics, Management, International Relations, MIS, History, Communications, Finance, Chemical Engineering, Bio Engineering, Geography, Agricultural Economics and there is even an applicant with a Juris doctorate.

The Department always gets a large number of SJSU students—nine in Spring 2006, but applicants are also coming from other notable universities. From California, the Department had applicants from the following universities: UC San Diego, UC Davis (5) UCLA, Stanford University, and Santa Clara University (3). Other United States applications came from: University of Massachusetts; Hampden-Sydney College, Virginia; University of Texas, Dallas; University of Notre Dame, Indiana; Holy Names University; Lamar University; and Loyola University, Louisiana. International applications came from: India (3) University of Madras, Osmania University, and University of Jaipur; from Doshisha University, Japan; Nanjing University, China; Asian University, Thailand; University of Ibadan, Nigeria; and the University of Kuwait.

of 27

Fall 2007 Semester Program Assessment Report - ALTERNATE · Web viewAccepting 3 units of computer...

Documents

Transcript of Fall 2007 Semester Program Assessment Report - ALTERNATE · Web viewAccepting 3 units of computer...