JEEReformProposal_RK

8/8/2019 JEEReformProposal_RK

1/11

Version 09 July 2010 Page 1 of 11

IIT JEE Reform Proposal : A Summary1.Welcome Steps : On Plus Two

Common Plus 2 curriculum and Common Question paper base/format Suggestion : Include Aptitude Test in Plus 2 curriculum to minimize coaching.

2.Deficiencies of Acharya Committees Alternative No details of how to normalize the marks of Plus Two; in the absence of this

crucial input, Acharya Committees alternative is for Academic Use only.

Instead of the premise to have a single examination, the alternative hasmultiplicity of examinations One Plus Two, Three NATs, and Add-on(s).

Increases coachings one coaching each for Plus 2, NAT and Add-on(s); noattempt to increase the attendance in schools

Tremendously increases the stress level of a candidate by spreadingexaminations throughout the year

Increases eligibility cutoffs beyond 90% from the existing 60% in Plus 2 A subjective evaluation of Add-on is error-prone and open to manipulation Final selection is based on a one-time manual evaluation. Widen rural-urban divide as well as gender bias. Is silent on many issues involving large scale bungling in the past JEEs. Acharya committee had no discussion with any real stake holders students,

parents and teachers; Reports did not have any suggestion given by IIT faculty.

3.Salient Features of the Another Alternative as Proposed A single examination for admissions to all (IITs, IISERs, NITs, IIITs, states and

private) with a single merit list (all India and state ranks) for each category.

Single examination is conducted with 4 subjects -- PCMA for engineering andPCBA for medicals -- having equal weights; add-ons for architecture, design.

Instead of having the above as a one-time examination, take an aggregate ofthree sittings of the examination conducted thrice over a fortnight/month.

Examination to be of multiple choice objective type with complete andbalanced coverage of Plus 2 curriculum.

Increase the spread of marks by proper setting of questions with multi-levelevaluation (a.k.a. differential grading) in place of the existing binary.

Integrated admission counseling(with sliding) to avoid wastage of seats. Student with UID is given two chances as of now, additional chance for rurals. With transparency, e.g., sample question, instructions, answer keys, evaluation

scheme, cutoff decision, filled-in seats etc. to be disclosed at appropriate time.

Add safeguards, e.g., ORS filling by Pen, carbon copy for self evaluation, usetrusted softwares and ethics in all operations; eliminate manual operations.

Change the mindset from feudal to facilitator. Attendto all errors/disputesbefore admission counseling, encourage involvement of real stakeholders.


2/11


IIT JEE Reform Proposal(As An Alternative to the Alternative to IIT-JEE, AIEEE and State JEEs,

Placed in the Special Senate Meeting of IIT Kharagpur on 02 July 2010)

Rajeev KumarDept. Computer Science & Engineering, IIT Kharagpur

In reference to my oral submission, in the Special Senate Meeting of the Institute, thatthe recommended alternative to IIT JEE et al. as proposed by Four IIT DirectorsCommittee, headed by Professor D. Acharya, do not meet the objectives as stated inthe said report, and consequent to the directions of the Chairman, Senate, IITKharagpur, I submit the IIT JEE Reform Proposal, herein, as an alternative to IIT JEE,AIEEE and state JEEs.

I call the Alternative as proposed by Four IIT Directors Committee as DirectorsAlternative in rest of this document. This report is organized into the followingSections. Section I is on the welcome steps regarding proposed reforms in Plus Two.Section II highlights the inherent contradictions. Section III is on desirable features ofthe entrance examination as identified by the Directors Committee. Section IV is onDirectors Alternative and Section V is on major issues on which the DirectorsAlternative is completely silent. Additionally, I have interspersed my views,comments, proposals, issues and challenges at appropriate places for betterreadability. Finally, an alternative proposal by me to the existing IIT JEE, AIEEE andState JEEs is included in the Last Section VI.

I : Welcome Steps

The following Plus 2 Reforms are welcome, however, there are some concerns:

S.No.

Plus 2 Reforms Concerns

1. Common PCMB curriculum across all boards Feasible

2. Common base of the question-paper/instruction-format across all boards

Feasible

3. Unique identity of a student to be ensured bythe UID project

Feasible

4. Conducting online NAT This will further widen theexisting urban/rural divide.

5. Availability of final results across the boardsby the prescribed deadlines

Doubtful, due to unforeseen localdisturbances etc., though this isindispensable to the success of theDirectors Alternative

6. Conduct of free and fair examination acrossall boards

Extremely doubtful due to localconsiderations, favoritism and


3/11


influence, etc.;

My Submissions : To circumvent the above concerns,

1. NAT should be made online as well as offline, by use of pen and paper, tominimize the urban/rural divide.

2. Addition of another test at Plus 2 level will further stress an already stressedPlus 2 candidate. There will be tremendous stress on appearing three times inNAT during one year in addition to study for the Plus 2.

3. Therefore, it is recommended that NAT should be made part of the Plus 2uniform curriculum. Such a scheme will have PCMA in place of PCM, PCBA inplace of PCB. Such a scheme will have psychological advantages to de-stress.

4. With the above proposal, there is no need to have an X-factor weighted CWP. Incalculating a candidates school level performance, all the four components outof five, PCMBA, will get equal weight of 0.25 each.

5. A model/system whose success is dependent on achieving Items, 5 and 6, asenvisioned above cannot be a practical.

II : Inherent Contradictions, Extraneous Factors, UnrelatedData having Zero-Effort

S.No.

Extracts from Directors Report Concerns

1. Consultations were held with thestakeholders

Consultations were held withChairmen/Directors/VCs/Secretarieswho CANNOT be called stakeholders;they are the organizers, unlike the realstakeholders, who are candidates/parents/school-level teachers whohave at stake their careers.

Additionally, to reach to realstakeholders and to enable their

participation thereby enhancing theacceptability of whole scheme, theCommittee should have invitedsuggestions/feedback/commentsonline/offline through issuance ofPublic Notice in leading newspapers(as is done by the ParliamentaryCommittees), in the matters of PublicInterest.

2. State of Tamil Nadu abolished JEE This is not relevant due to the facts that

(i) TN has many more engineeringseats than the aspirants, and (ii) Most of


4/11


the candidates admitted are from theTN board(s).

3. Conclusions drawn from theperformance analysis of JEEs as

presented in Section 6 of DirectorsReport

This section does not contribute muchuntil complete data and quantitative

results are presented. Correlationanalysis including certain conclusionssuch as poor correlation between AIRand CGPA from 2nd year onwardsdoes not have any relevance. This ismainly due to many extraneous factors.Thus, the conclusions drawn, in thissection, have no relevant basis.

4. Focus shifted from +2 Scienceeducation in School to Coachingfor

JEE

With the Directors Alternative,importance of coaching is

strengthened further. Now, a studentwill go for 3 types of coaching one forPlus 2, second for NAT and third forAdd-on tests.

5. School attendance has become acasualty

With the Directors Alternative in place,a student will go to Plus Two coachingrather than attending the school.

6. Coaching classes concentrate onteaching student tricks

IIT JEEs question paper setters areresponsible (elaborated later)

7. Students suffer from burn-outsyndrome

Extraneous factors are responsible,

8. JEEs are urban centric Directors Alternative will furtherwiden the gap due to increasedimportance of coaching

9. Girl students fare worse than boys As above and not addressed any where

10. Dearth of quality institutions Not within the purview

11. Increase in number of students This is bound to happen in a SocialInclusive Model, which runs on PublicExchequer, and must be based ongrowth for all

III : Desirable Features as identified by Directors Committee

S.No.

Desirable Features Concerns

1. Decision based on one-time test

needs to be examined

Directors Alternative is still a one-time

test; one-time for Plus 2 and one time forthe most crucial Add-on test; only NAT


5/11


is best out of three, which has only 30%weightage and thats too in qualifyingphase

2. Students must be relieved of the

pressure of multiple JEEs

In fact, Directors Alternative is for

multiplicity of tests. For IIT-alone, therewould be 1 + 3 + 1 = 5 tests. In the sameway, many more high-in-demandinstitutes (e.g., better NITs, IIITs, manygood state and private colleges) will optto have their own add-on tests.

3. Influence of coachingfor JEE needsto be minimized

In the new set up, a candidate has to join3 coaching(one each for Plus 2, NATand Add-on) in place of a singlecoaching

4. Urban-rural and gender-bias has tobe eliminated

In fact, with increased coaching effect,gaps for both will widen

5. Objective type of examinationlends itself to undue influence ofcoaching

Not necessarily; it is due to wrongsetting of question paper as wellunbalanced coverage of the topics. Alsodue to unnecessarily silent race between

JEE question setters and coaching.(elaborated later)

6. JEE, especially the IIT-JEE, havebecome a huge money spinningactivity

This is contributed mainly due to lack ofadequate transparency in theprocesses/practices in the actual conductof past IIT-JEEs. Also attributed toessentially random nature of selectionsin the past JEEs (elaborated later)

IV : Directors Alternative

S.

No. Alternative Concerns

1. NAT should not require extensivepreparation and coaching

Any time in a year Best of 3 attempts

India is flooded with coaching forAptitude tests (for CAT, GMAT, GREetc.). Since there are three possibleattempts spread over a year, NAT willtake a heavy toll on school levelexamination.

2. Normalized Plus 2 score There does not exist any formula tonormalize effectively. Whosoever used

normalization of marks in the past,including IIT-GATE, UPSC-CS, and


6/11


BITS, has discarded normalization.

3. Composite Weighted Performance(CWP)

Success of CWP solely depends on theNormalized Plus 2 score, which cannotbe done. Therefore, it is futile to talk for

CWP.4. Add-on test for candidates three

times the seatsMajor problems

It will indirectly make Plus 2 cutoffabove 90%;

Add-on is based on subjectiveevaluation which is error-prone,biased to examiner, slow, open tomanipulation and cannot scale.

Many institutes will start opting foradd-on tests.

V : Major Issues on which Directors Alternative is Silent

1. Normalization: Success of CWP in Directors Alternative solely depends onnormalizing the Plus Two Score. There does not exist any statistical formula simple or complex - which can effectively neutralize the variability quotient. Sincethere are tens of Board, across the country, having hugely varying population ofcandidates (in the range of a few thousands to a few millions), normalization of

Plus 2 score CANNOT be done.

Due to large variation in population, percentile is clearly ruled out.

Here, I would like to present three case studies First is the case of BITS PIlaniwho used the normalized scores for a few decades, ultimately they discardednormalization by observing that in spite of normalizing the Plus 2 score, more than70% of the total candidates admitted in BITS were from a single board.

UPSC used normalized score across a range of elective subjects in their CivilServices Preliminary and Main examinations for the past quarter a century.Unable to meaningfully normalize, they are under so much fear that they did not

disclose their normalization procedure in spite of receiving directions and flakfrom CIC. Now, instead of disclosing their normalization procedure, they havecome up with their new proposal for Civil Services (Preliminary) in which theydiscarded all those components which need normalization.

Third example is of IITs themselves. After self-realization that the GATE-Percentile does not tell much about the candidates performance, IITs introducedin early 2000s, a concept of Normalized GATE Score across the disciplines. In a fewyears, IITs had their own realization that neither Percentile nor the NormalizedScore is of much worth. Consequent to which IITs started revealing THE RAW

GATE SCORE, which they could hide since GATE was introduced, approximatelythree decades ago.


7/11


In the simplest term, normalization cannot be done, therefore, much hyped,CWP is of no use. A merit list as proposed in Directors Alternative, if donebased on CWP, wouldnt be of any use.

2. Issues with Question Paper Setting :-- The major problem with IIT JEE, in thecurrent form lies with improper question setting. I include a few of the samplerexcerpts taken from the commentary of JEE Mathematics question paper of therenowned JEE mathematician, Professor KD Joshi of IIT Bombay, who whilepresenting a commentary on JEE 2008 Mathematics question paper included that:

we have also pointed out how the questions could have been made moreinteresting and challenging even within the framework of a completelyobjective type testing,

one of the reasons for the dearth of interesting questions seems to be that thepaper-setters have either simply omitted or only given a lip service to many

topics where there is an opportunity to ask interesting problems.

there are many avoidable instances of such duplications. these repetitions have made the papers heavily dominated by calculus and

coordinate geometry. A rough count of marks shows that out of the combined163 marks of the two papers, calculus takes more than 50 marks andcoordinates geometry takes even more. These are the areas where the mediocrestudents are more comfortable. With nearly two thirds of the marks catering tothese areas, mediocre students who can compute fast and without makingnumerical mistakes must have had an easier time.

it does not appear that the paper-setters took an over-all view of both thepapers together. Had they done so, the duplicate and triplicate appearances ofthe same concepts would have been noticed and after dropping some of them,room could have been made for areas which have received little or norepresentation.

the greatest disappointment came from the so-called comprehension questions. but even after making an allowance for such inherent human fallibility, the sad

conclusion is that JEE 2008 stands out for the mistakes in it.

There is a long list of such comments.

3. Question paper, Model answers and Evaluation:-- Professor KD Joshi In hiscommentary stated that But as far as model answers are concerned, there ishardly any reason to hide them from the eyes of those who may be victimizedbecause of mistakes in them. It will be highly desirable if the model answersprepared by the paper-setters and scrutinized by JEE experts are made open topublic scrutiny before freezing.

4. De-Stress:-- The first and foremost objective of an examination should be to de-stress a candidate. The Directors Alternative will unnecessarily put a candidate

under tremendous stress mainly due to the following two factors (i) multiplicityof examinations, and (ii) a huge cutoff (may be much above 90% in Plus 2 andNAT) for a candidate to qualify to sit in Add-on Test.


8/11


5. Promoting coaching:-- It is no secret that there had been a race between JEEquestion paper setters and coaching. In the process, question papers became theun-necessary causality which forces JEE aspirants to look for coaching.

On the other hand, attempts of some of the JEE coaching (e.g., Super30 of Patna)

have been lauded by many including Prime Minister, Minister HRD, ChiefMinisters of many states and national and international media. Very recently,PACE Mumbai coaching was accorded a status of Science School. Also, it is wellknown that many of the science schools across the country do not have adequatescience teachers. Such coachings are being recognized as value additions to sciencelearning among school-going students.

Its time for IIT-JEE to keep their house in order by setting proper and balancedquestions from and within the Plus 2 syllabi.

Further, there are many acts of IIT-JEEs, which promoted coaching. For example,

coachings display model answers on same day of the examination, and thus boosttheir image in public, while IITs never did or did in next 3-4 months; JEE 2010 isthe exception. Candidates login to their web-sites and thus become soft target.Similarly IITs do not disclose marks at the time of result declaration. Essentiallyblack-boxed nature of JEEs had been the real promoter.

6. Transparency:-- In spite of large scale JEE bungling in the past (from JEE 2006 to JEE 2010), Directors Alternative is completely silent on measures for inculcatinggreater transparency in the whole process of examination including admissioncounseling.

7. Accountability:-- The experience of JEE 2006 to JEE 2010 has firmly establishedthat JEEs were/are-being conducted with zero accountability. None of theblunder/bungling could ever be explained by any of the IIT-JEEs executive, rather,all efforts were made to Change the Rules of the Game after the Game was Overand/or take a legal shield and delay the matter to make them infructuous. All thepublicly known blunders remained unexplained yet.

VI : My Alternative Proposal

1. Best is to have a single examination (replacing IIT-JEE, AIEEE and all otherstates JEEs) by which a student (securing a UID) can have a score and an allIndia as well as State rank. A single (all India level and state level each) merit listshould be prepared by which one can seek admission in one of the engineeringcolleges, including IITs, IISERs, NITs, IIITs, states and others.

2. Instead of having the above one-time examination, a candidate should appear inthree sittings of the same identical examination conducted thrice over afortnight/month. An aggregated score over three sittings of the examination isconsidered for the final ranking.

3. The single test is conducted with four subjects of PCMA (Physics, Chemistry,Mathematics and Aptitude). Test has equal weights for all the four.


9/11


4. There would be add-on tests for additional skills needed, for example, forArchitecture, Town & Planning, Design etc. There should not be any additionaltest to test PCMA/PCMB, which is already tested by the single test.

5. Examination should have the Common Plus 2 curriculum. Thus, school leveland entrance examination have the same uniform curriculum and a studentcannot afford to ignore Plus 2. It is proposed that the testing of the knowledge ofAptitude should be included in Plus Two common curriculum/examination.

6. Present IIT-JEEs have a heavily skewed marks distribution. There is essentiallya bimodal marks distribution, a few hundred candidates score decent marks foraround 10000 seats; most others are in the other. Therefore, scoring of marks formost such students can be contributed to random selection of choices.

The main reason for this is that the questions are unnecessarily complex;questions do not test much of analytical or conceptual knowledge; questions do

not have proper coverage.Therefore, the care should be taken for setting correct, unambiguous and

within the syllabi Multiple Choice Questions (MCQs). Set proper and balancedquestions of Plus Two standard; this will reduce dependence on coaching;students will concentrate for school level examination.

7. However, since a binary grading of MCQs does not test much of the skills of astudent. Introduce multi-level grading to test intelligence as is tested in longquestion-answer format. Each option can be assigned different marks, in positiveor negative, based on its closeness to the correct solution.

Such a scheme, which is not based on binary evaluation, will yield theevaluation quality akin to testing long question-answer format. This scheme willmake dispersion of marks wider and improve selection quality drastically. Thisstrategy will take care of wrong setting of questions, if any, without making itopen.

8. A student, having a UID, is given two chances as it existed now. UID will ensurethat none can abuse the system by availing multiple chances in JEEs by repeatingPlus 2 examination.

However, candidates from rural background can be given one extra

chance to minimize the urban-rural divide and ultimately make SocialInconclusive model.

Hopefully, the gender bias may be reduced implicitly by the above reforms.

9. Currently JEEs are conducted too early, on second Sunday of April. Students donot get time after Plus 2 exam, thereby, they do not concentrate much on Plus 2.Having the same curriculum of Plus 2 and deferring the examination by a periodof 4-6 weeks, a candidate can do additional preparation needed for JEE. Yet,there would be enough time left for result and admission.

10.Conduct examination with complete transparency. For example,Disclose model answers after the examination on the same day (current

practice is to disclose model answers after 3-4 months of examination when


10/11


all admissions are completed; JEE 2010 is the exception); invite public to sendfeedback regarding correctness within a week and then freeze the answers forfinal evaluation. This is the practice used in some states, e.g., CET inKarnataka,

Disclose complete information at appropriate time, e.g., disclose modelsolutions on the day of examination; disclose marks and ORS with resultdeclaration so that errors, if any, can be corrected immediately.

Make counseling fully on-line and publish filled-in and vacant seats on day-to-day basis. Prepare a wait list during admission counseling and fill/upgradeseats by sliding. Setdeadline for students to join, else allot those seats to wait-listed candidates. It would be best if JEE, AIEEE and all other counseling arecombined. A UID can provide a perfect solution.

Make all decisions, including cutoff decisions, open.Make question format along with the Instructions public. Perhaps, there existed

no other examination in which question/instruction format is not known apriori. If this had been public, the blunders of ambiguous instructions wouldnot have happened.

11.Print question paper in English and Hindi both; the left page of the questionbooklet may be for Hindi and the right-side for English. Print marks of eachquestion (positive and negative both) along with the question. Such printing ofthe question papers, will eliminate many of the printing-errors as noticed in JEE2010 question papers.

12.Additional safeguards should be taken in terms of marking ORS by black ballpen. Backside of ORS should be coated with a carbon film to ascertain anytampering and a carbon duplicate of the ORS can be retained by the candidate asa proof. A student is given a carbon copy of his/her for self-evaluation of thescore.

13.Safeguards should be taken to ensure trustworthiness of all the software used inJEE operations along with soft data integrity.

14.Improve ethics. Do not repeat same set of people for question setting, computerexperts and JEE administration. The current practice of empowering JEE

Chairman, with the raw and coded data must be abandoned; personal identity ofthe candidate should not be revealed. Randomly allocate examination centre.Take finger prints at the time of application, examination, counseling andadmission. Strengthen vigilance

15.Change the mindset from feudal to facilitator. Amend the declarationto have achoice of not waiving the right to challenge irregularities without having anybearing on candidature with an explicit disclaimer that not waiving the right isnot a disqualification. Correct the errors, if any, instantly rather than taking alegal shield and/or changing the rules after the examination is over. Settledisputes, in house, during the period from result declaration to counseling. All

records must be kept till disposal of disputes.


11/11


16.Encourage involvements of real stakeholders candidates, parents, school-levelteachers in decision making of JEEs. Attend to errors rather then repeating allis well though it is universally known, at times, that none is well

* * *

Acknowledgements :: The author gratefully acknowledges receiving inputs,comments, suggestions and feedbacks, while working on many versions of theproposals in the past four years, from many stakeholders including school-levelstudents, their parents, school-level teachers, IIT and non-IIT faculty (from India andabroad), IIT students and alumni, professionals, social and media persons, RTIactivists, public watch-dogs and NGOs.

* * *

JEEReformProposal_RK

Documents

Transcript of JEEReformProposal_RK