Assessment of Engineering Faculty Performance in the Developing Academically

Autonomous Environment - VIT, Pune, India - A Case Study

Prof. Ashutosh Marathe

Dean - Quality Assurance Management Representative - ISO 9001:2008

Vishwakarma Institute of Technology (An Autonomous Institute Affiliated to University of Pun e)

Pune, India ashutosh.marathe@vit.edu

Abstract- The necessity of Documenting and quantifying the accountability of faculty in Higher Education Institutes (HEI) is gathering momentum in countries all around the globe. In India, where Academic Autonomy is slowing spreading its wings outside the Indian Institute of Technologies, National Institute of Technologies and Government Engineering colleges, into the group of Private unaided Engineering Institutions, the need is felt to develop a rubric for assessing faculty performance in a Quantitative manner. Addressing to that need, various attributes are identified which are essential for assimilating a 'complete faculty performance'. This paper details out these efforts and its outcomes in one of the leading Private Engineering Autonomous Institute in the academically progressive western part of the country. The teaching performance of the faculty along with academic, co-curricular, extension and research activities is quantified into a credit based assessment system (CBAS). This system is run under Quality Management Systems adopted at the Institute. This faculty performance assessment is observed to have helped in identifying better performing as well as poor

performing faculty. Also the targets for the subsequent Academic year can be set up with clarity and transparency

Index Terms- Higher Education Institutes (HEI); Faculty

Performance; Academic Development; Co-curricular and

Extra-curricular contribution; Research contribution;

performance Points (PP); Performance index (PI) Weightage based Indexing (WBI); Credit Based Faculty

Assessment

I. INTRODUCTION

The Indian Higher Education System is a matured and established system. The strong emphasis on basics of Mathematics and Sciences at school level has traditionally assisted the Engineers to comprehend the Engineering concepts effectively. The contribution of Indian Engineers in Academics as well in Industry all across the globe is well recognized and respected. The traditional methods of teaching - learning with an emphasis on classroom teaching supplemented by laboratory practices have been the backbone. Yet over the last 2 decades, with globalization and emerging opportunities, the number of opportunities for able and competent Engineers has increased many folds. Many authors such as [1] have commented about this issue from a similar if not identical perspective.

Also with the advent of ICT (Information and Communication Technology) based practices, the modern

978-1-4673-5261-1/13/$31.00 ©2013 IEEE

teachers also have had to equip themselves to suit the needs of the new generation of Engineering students. While the knowledge management infrastructure through the use of ICT has grown, we still need to rethink of conceptual network, research agenda, strategies and models that are more adaptive and responsive [2]

In order to ensure that the local, National and Global need of engineers is met with in sufficient numbers, there has been a planned growth in the number of Higher Education Institutes (HE!) [3].

The role of the Engineering teacher is also changing into a facilitator rather than only a classroom or laboratory advisor.

For academically competent Institute such as the one considered here, Academic autonomy is granted. The Academic decisions can be taken autonomously by the Institute. Hence after the absorption of the admitted students at the First Year of Undergraduate level, the Institute designs its curriculum, sets up its own Question papers and carries out assessment and awards grades to the students at the end of 4 years of Undergraduate Engineering course. The student performance is mapped in terms of a Cumulative Performance Index (CPI) on a scale of 0 to 10. While executing this entire activity, the role of a faculty in the multi-faceted development of student is very critical. Hence as a quality conscious and progressive Institute, specific efforts are taken to monitor and evaluate the faculty performance in the campus on the basis of a novel assessment mechanism. The application of quantificational method helps to improve the effect of performance evaluation of staff members in the Institute [4].Authors in [5] share a similar opinion that a well-structured performance appraisal system that is based on quantifiable objectives & standards can lead to achieving a high level of accountability. According to the Author of this FIE conference paper, this is the blending of various appraisal areas for the holistic development of the faculty, thereby consequently the student.

II. IDENTIFIED AREAS IN THE ASSESSMENT SCHEME

The modern faculty in a progressive and developing Institute is expected to contribute in Quality Teaching as well as other essential areas. For an Institute with around 4000 students, running 23 programs (UG, PG and PhD combined) the appointed faculty is expected to commit and support the technical growth of the student as well as Institute growth.

Various elements of effective performance appraisal were identified and exploration of domains and dimensions of faculty performance was made in [6]. The major domains commonly identified were teaching, service & research along with faculty development and administration in some cases in the same paper.

With these broad set of ideas in the Assessment scheme the following domain areas suitable for the Institute working culture as shown in Fig. 1 are considered.

Fig. [ - Areas considered for Assessment Scheme

A. Classroom Performance of the faculty - on the basis of the Staff Classroom performance feedback given by the students across the entire Academic year, the quantified performance rating called Performance Index (PI) is calculated.

B. Teaching and Learning Activities - The faculty involvement in Teaching learning, the adherence to academic deadline, amendment of course contents etc. along with efforts taken for slow learners are considered.

C. Faculty Attributes - An Engineering Faculty is a highly [ntellectual, sensitive human being. While teaching the courses, keeping abreast with emerging technical areas is equally necessary for the faculty. The faculty as an individual, faculty playing the role as a colleague is as important in the faculty growth as well as in sustenance of the quality of the deliverables. This fact is also mapped in Faculty attributes.

D. Academic Extension Activities -Design and development of in-house projects, laboratory setups, development of Additional supplementary Course material, providing additional learning opportunities beyond the syllabus to students is as important. The liaison of the faculty with Industry around is vital, for the experiential learning of the students. [n that regards, efforts taken by the faculty to provide Internship opportunities, or sandwich training or arrangement of a useful [ndustrial visit is considered under these activities.

E. Co-curricular Contributions - The typical organizational skills for scheduling and executing Conferences, Workshops, Short Term Training Programs for faculty, refresher courses for Industry, Certificate Courses for community around as a service to Society are all encouraged and recorded. Work carried out on the platforms of technical bodies such as IEEE, SAE, ASME, ISA etc. is considered. The Entrepreneurship initiatives taken by faculty are weighed highly.

Besides this the faculty is expected to contribute in various statutory or incidental committees at Department level or the Institute level. Based on the nature of work, corresponding 'credits' are given to the faculty.

F. Research Contribution - Although the Institute is predominantly an Undergraduate Teaching Institute, a lot of emphasis is given on Research initiatives and significant outcomes are expected from senior faculty involved in research on National and [nternational forums. Various areas of research were discussed during the evolution of the format and considered worthy to be included. [t is seen that not only does it help in quantification of the performances, they act as clues for junior faculty

Typical areas such as Publications, Consultancy, Patents, Technology Transfer, Innovations, Research Guidance etc. were considered. The assigned credits and other details are provided in the later part of the paper.

The domain areas D & E are equally important as highlighted by author in [7]

III. IMPLEMENTATION DETAILS

Area A - Classroom Performance of the faculty : This area is of significant interest amongst academicians.

The authors in [8] enlisted the common strengths and weaknesses amongst faculty, and also presented implications for continuous quality improvement.

In the chosen Institute, Faculty Feedback on the basis of regularity, delivery of basic concepts, ability of explanation of applications, communication skills, blackboard & multimedia presentation, interactiveness, class control etc. is carried out. The students rate the faculty performance on a scale of 0 to 10.

Such feedback is compiled on a question wise basis for the entire division . The faculty performance for the entire Institute faculty is ranked for each question. Performances above average and those below average are awarded positive and negative Performance points (PP) .The group of faculty in a certain band are identified on the basis of Gaussian distribution.

The ranked performances are obtained and graded for all questions. However based on the significance and impact of each question a Weightage based Indexing (WBI) is done. E.g. a greater scalar multiplier is used for question based on Communication skill, presentation and delivery of concepts while the weightage for regularity is less. Once the scaling is done, an overall performance Index is calculated.

A sample table for obtaining Performance Index as a resultant is as shown in Fig. 2

Sr. Initia[ PP PP Ql *1.25 Q2*1.5 PI

No. For For (a) (b) (a) + Ql Q2 b)

1 SJ 2 -1 2.5 -l.5 1 2 MK -3 -4 -3.75 -6 -9.75

3 MP 6 8 7.5 12 19.5 4 AM 8 7 10 10.5 20.5

Fig. 2 - Sample Performance Index Calculation

This feedback activity is carried out 4 times in the academic year. The performances of all 4 feedbacks are combined and the final Performance Index is calculated. The entire activity is shown in the flow chart as shown in fig. 3

PI Calculation for One Feedback(n)

Staff Feedback collection and average score calculation (based on inputs by all students)

Ranking of score for the question (group for ranking- scores of all faculty)

Award of 'Score (i.e. PP)' on the basis of average and relative position

Compilation of such scores for all questions

Multiplication of each such score by relevant 'Weight age factor'

Summation of outcomes of multiplication to form Performance Index (PI)

Cumulative Performance Calculation (n+1In+2/n+3)

Staff Feedback Collection and Average Score Calculation (based on inputs by all students)

for feedback n+ 1

Average of Performance for the question of feedback (n) and (n+ 1)

� Fig. 3 - Flowchart - calculation of Performance Index

Areas B to F -

For Other areas , formats as shown in fig. 4 are designed. Based on the significance of the parameters credits are assigned.

fig. 4 (a) Area B : Teaching and Learning Activities

Sr. Parameter Credits No.

I 100 % conduction of assigned teaching 10 load

2 Adherence to the paper setting and 4 assessment deadlines (End Sem. Exam,

Mid Sem. Exam, Class Test, etc.)

3 Course file and its regular updation 2

4 Release and assessment of Home 2 Assignment as per schedule

5 Extra efforts for slow learners 2

Total Number of Credits 20

Performance Index (PI) =

(SI XCI + ..... +Ss x Cs) / 20

fig. 4 (b) Area C: Faculty Attributes

Sr. Parameter Credits No.

1 Teaching performance 4 2 Technical knowledge of subjects 4

3 Discipline 2

4 Regularity & punctuality 2

5 Attitude towards colleagues 2

6 Attitude towards students 2

7 Motivation towards work 2

8 Awareness of technical education sector 2

9 Willingness to accept responsibility and 2 take decisions

10 Willingness to contribute besides office 2 hours as per need

11 Willingness to learn 2

12 Ability to do and get good quality work 2 done

1 3 Behavioral aspects 2

Total Number of Credits 30

Performance Index (PI) =

(SI XCI + ..... +S13 x C13) / 30

fig 4 (c) Area D· Academic Extension Activities Fig 4 (e) Area F· Research Activities

Sr. Parameter Credits Sr. Parameter Credits

No. No.

Research Paper Publication I Syllabus enrichment by providing 2

I Journals 101 additional resources to students

Credits to be augmented as : Publication 2 Use of participatory and innovative teaching 2

1 credit per 0.5 impact factor of the - learning methodologies

journal, 3 Contribution towards theory and laboratory 2 1 course

2 credits if the appraised faculty is course development

the first author 4 Contribution towards GP 1 PO 1 SO 1 OE 1 2 /course 2 International Conference 8 1

Elective course development Credits to be augmented as 1 credit Publication

5 Conduct of Continuing Education Program 4 1 if the appraised faculty is the.first program author

6 Design and development of Diploma 4 /course 3 National Conference 1 Magazine 4 1 courses articles Publication

7 Design & development of Certificate 2 /course 4 Reviewer 4 1 Review courses Book Publication

8 Development of quality in-house projects 2 /project I Published by International Publisher 10 1 book 9 Academic resource person 2 1 session Credits to be augmented as 2

10 Creation of industry exposure opportunities for students credits if the appraised faculty is a) Internship 2 1 group sole author

b) Sandwich training 4 1 2 Published by National Publisher 5 1 book student which has ISBN 1 ISSN Number

c) Industrial visit 2 1 visit Credits to be augmented as 1 credit

11 Attending faculty development program 2 1 week if the appraised faculty is sole

12 Attending industrial training of minimum 6 1 month author

one month's duration Research Publication

1 Sponsored Projects

Fig. 4 (d)Area E : Co-Curricular Contributions a) Amount Sanctioned 3 / 0.1 Million Rs.

Sr. Parameter Credits b) Amount Utilized 3 / 0.2 No. (Per Activity) Million Rs.

1 Organizing Short Term 10 for Co-ordinator 1 2 Consultancy Projects - consultancy 3 / 0.1 Training Programs week worth Million Rs. (Minimum 1 week) 2 for team members 1 3 Completed Projects (Acceptance 3 / 0.1

week from funding agency) - credits to be Million Rs. 2 Organizing Workshops 1 for Co-ordinator 1 day earned on the basis of the cost of the

3 Organizing conferences project

a) International 10 for Co-ordinator 4 Projects Outcome (Patent A warded 3 for Team Members 1 Technology Transfer 1 Innovative

b) National 6 for Co-ordinator Product)

2 for Team Members a) International 10

3 Organizing guest lecture 1 1 lecture b) National 6

4 Services to community 2 for Co-ordinator i.R.G. (Internal Revenue Generation)

1 for Team Members 1 I Self sustained activities 3 credit per

day Rs.25000 institute

share Development of Laboratory Experimental Setup

1 As a part of curriculum development 4/ experimental

set-up

2 As a Joint Venture with companies 6/ setup

1.

IV. RESULTS

Classroom Performance of faculty

A tabular mapping of the same on the basis of various areas is done as under. Each column represents a parameter and each entry is the Performance Point (PP) given to the faculty

Staff Regularity Delivery Communication Presentation

AH -2 8 7 9

GN 8 8 9 I

IK 6 7 8 7

DD -6 -6 -5 -4

SG 9 \ -\ -2

In the actual database, 196 faculty for the current Academic year are considered.

Variance is calculated for all faculty to highlight the abnormality rank in a certain area.

Important conclusions such as the following can be drawn i) for faculty AH - the performance is excellent except the Regularity. ii) for faculty GN -presentation skills need

. to

be improved. iii) faculty IK has a steady performance IV) faculty DD has a steady but poor performance v) faculty SG has only got excellent regularity

In the actual database applying the method for 196 faculty members , conclusions are derived such as

i) No. of faculty requiring communication skill training -22

ii) No. of faculty requiring inputs for improving presentation skills - 15

iii) No. of faculty needing subject specific technical training - 1 8

The feedback is given by around 4000 students every time, hence covers out individual biases towards faculty.

In case of credit based activities, listed in Results pt. No.2, 3 & 4, the faculty self Appraisal is followed by appraisal by mentor and Head of Department. This averages out self bias if any. The claims made by faculty are verified through documentation support.

2. Teaching - learning activity & Faculty Attributes -

As can be seen from the table listed above, the performance of the faculty is calculated using a Performance Index (PI) , while the faculty earns 20 and 30 credits resp. in these 2 areas.

The faculty is expected to perform on all parameters in the Teaching - learning areas.

The scores in the two areas mentioned above are recorded for all 196 faculty.

3. Academic Extension Activities and co-curricular contributions These are open ended areas for faculty to earn credits. Based on the requirements of the Department or the Institute, as well as individual interests and abilities, the faculty members tend to involve themselves in a variety of Academic Extension activities, as well as in

many co-curricular contributory areas.

4. Research Activities As shown in the table, a faculty is encouraged to be

involved in a number of research activities. The Institute offers the faculty all basic support desired

to contribute the research activities.

V. OUTCOMES OF CYCLE AT THE END OF ACADEMIC YEAR

In the recently concluded Academic Year, this activity has resulted in the following outcomes shown here on Department wise basis as well as in terms of overall Institute performance

1. Classroom Performance of faculty -Average Institute Performance Index - 1 3.155 The Indices for Departments are as shown below in Fig. 5

25

20

15

10

5

o

f--

f--

f--

-

- f-

f-- t--

Fig. 5 - Performance Indices for Institute and Departments

Head of Department of Instrumentation and Electronics are informed to emphasize more on improvement of Quality of Teaching - Learning.

The faculty is deputed for Pedagogical training and various classroom performance improvement workshops.

A number of in-house activities are also carried out by well appraised faculty of the Institute.

2. Teaching and Learning Activities (on a scale of 0 to 10)

9.20

9.00

8.80

8.60

8.40

8.20

3.

9.20 9.00 8.80 8.60 8.40 8.20 8.00 7.80

Faculty attributes (on a scale of 0 to 10)

4. Academic Extension activities

30.00 25.00 20.00 15.00 10.00

5.00 0.00

5. Co-curricular Contributions

6. Research Activities

30.00 25.00 20.00 15.00 10.00

5.00 0.00

Cadre wise Research Contributions

35

30

25

20

15

10

5

o

l.

2.

3.

4.

5.

Professor Associate

Professor

VI. BENEFITS

Asst. Professors

The planning and benchmarking in various areas or across the Departments is possible through this exhaustive analysis The Institute implements ISO 9001:2008 Quality Management System. It helped streamline the entire compilation of outcomes at individual as well as Institute level The preparation of documentary evidence for the National Accrediting bodies such as National Board of Accreditation (NBA) and National Assessment and Accreditation council (NAAC) is automatically done as a benefit of this activity. Cadre wise targets - especially in the area of Research are given to Professors and Associate Professors in the subsequent areas. Based on the identified areas of involvement of faculty, the Department and Institute activity plan is prepared.

VII. FUTURE PLANS

Some approaches such as 'Format Concept Analysis (FCA) as used by [9] are being studied for a more systematized approach. Effective conversion of the observations when collected over the span of 3 years can be made into defining the short range and long range goals for the Departments and the Institute.

VIII. REFERENCES

I . Saleh M. " Challenges in Engineering Education - A view towards the future', in 3rd IEEE International conference on e-Iearning in Industrial Electronics, ICELIE 2009

2. W.Omona, T.Van Der Wiede, Jude Lubega, " Using ICT to enhance Knowledge Management in higher education: A conceptual framework and research agenda", International Journal of Education and Development using Information and communication Technology, (IJEDICT), 2010, vol.6, Issue 4 pp 83-101. Ref. ijedict.dec.uwi.edu/include/getdoc.php?id=4316 Accessed : 8th April 201 3

3. " Higher Education in India at a glance" Brochure released by University Grants Commission (UGC) , http://oldwebsite. ugc.ac.inipubIHEglance20 12.pdf

4. Xiang He, Tang Guo, " Research on quantificational performance evaluation of staff members in Institutions", 2nd International conference on Information Science and Engineering (lCISE), 20 I 0, pp 6067-6070

5. A.Gilani, KJaravi, A.Angelis," Fostering a culture of accountability through a performance appraisal system", Healthcare Management Forum, vol. 1 8, Issue 1, 2005, pp 35-38

6. J. Dienemann, C. Shaffer," Faculty Performance Apprisal Systems: Procedures & Criteria", Journal of Professional Nursing, vol. 8, No. 3, 1992, pp 148 -154

7. H.Golob," The impact of teachers' Professional Development on the results of pupils at national assessment of knowledge", Elsevier Procedia - Social and behavioral sciences, vol.47, 2012, pp 1648-1 654

8. Z. Wolf, P. Bender et. ai, " Strengths and weaknesses of faculty teaching performance reported by Undergraduate and Graduate nursing students: A descriptive study", Journal of Professional Nursing, vol. 20, Issue 2, March 2004, pp 11 8-128

9. C. Aswani Kumar, K.Sumangali," Performance Evaluation of employees of an organization using Format Concept Analysis", International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME), 2012, pp 94 -98