Chapter 16 Document and Voice Examination Document Examination.
Online Examination _Advocacy Document
-
Upload
kallol-saha -
Category
Documents
-
view
234 -
download
2
Transcript of Online Examination _Advocacy Document
Sri Krishna Institute of Public AdministrationInnovation, Planning & Management Unit
Presents a Case Study On
ONLINE EXAMINATIONWider Choice, Easier Selection
Introduction
Remember the Glorious days of Board Examination. Year around hard labor, used to
be managed by a huge Institutional set up. There were Great Recruitment Boards
conducting examination all across the country. From the perspective of recruiter this
was a costly and tedious affair. Added were the responsibility of maintaining
confidentiality and holding volumetric records for future reference. Thanks to computer
and internet we have online Examination System as an option.
For years, Administrative Training Institutes (ATI) like other Government Agencies
have followed the conventional system of conducting examination. The entire selection
procedure took around six months. At the outset, the vacant positions were advertised in
at least four different newspapers. One month time was given to the aspiring candidate
to apply. Then in one week time scrutiny of the applications were done and a list of
shortly listed candidates was to be prepared. Thereafter the shortlisted candidates were
informed the date of exam by post.
Before the examination a number of preparations like framing of question papers, selection
of examination center etc. were done. On the examination day a huge number of staff was
deputed to conduct the examination. Papers were checked in a time span of one to two
months. Results were declared followed by the interview call letters. The next fortnight was
spent in conducting the interviews. The interviews were held over a period of time as per the
number of eligible candidates. Post interviews, Declaration of final results would also take
time. If there is any trivial mistake, depending upon situation, 3-6 months additional time
may be required.
However, advent of online examination have opened up possibilities for those who otherwise
did not have a choice. A testing put online is efficient, adaptable and prudent. For
mentors/educators, they don't need to save the time to check the test in light of the fact that
the test outcomes can be reviewed consequently. For test-takers, they can take the online test
at their own favored time. For associations, testing online spares a ton of expense to utilize
teacher architects and print a great many test papers.
As the times progresses, online test will most of use by applicants many online portals are
available on web which provide the online test and mock test and thousands of other benefits
for students such like: they provide the online test series at just affordable cost as compare to
market rates, Who created huge amounts of helpful items for us to make online tests, now test
creators assume an imperative part in enhancing the efficiency of making an online test.
While in principle such developments should be adorned by various agencies, Government
system often fails to see a value in the proposition. It is in this context, Online Examination at
SKIPA demands special mention for this effort.
The Case: Process & Methodology
The online Examination was an initiative to select suitable human resource for Innovation,
Planning and Monitoring Unit (IPMU) at SKIPA. The IPMU has been sanctioned by the
Government of Jharkhand on 25.03.2015 to function under the control of Center for
Innovation in Public Service (CIPS) at Hyderabad. IPMU works to study, document, pilot
and Scale Up innovations in Public Service Delivery System.
NotificationBesides the traditional way of newspapers, online advertisements were made on the websites
of SKIPA and CIPS. The advertisement along with link to Course wise Term of Reference
was also promoted through Facebook Pages, Google Groups, LinkedIn, and Twitter. 351
candidates across the world applied for three (3) positions of Research Analyst (RA),
Program Manager (PM) and Program Officer (PO).
ApplicationOnline application formats were provided to the candidates applying along with facility for
Resume uploade in Pdf format. Online and telephonic support was made available through
dedicated Help desk. On successful completion of application process an auto genarated E-
mail was sent as Acknowledgement receipt to the concerned candidate. Log in credentials to
participate in the Examination were sent through E- Mail and SMS.
ScreeningA dedicated help desk was created at SKIPA to screen applications online and guide
candidates suitably. Individual instructions were made available to the candidates to address
their querries and technical assistance during Form Fill up. To nullify chances of Favoritism,
automated Screening of applications based on specific parameters was conducted without
manual interference were adopted
Development of QuestionnaireThe questions were developed in a highly secured manner, with three different sets, from
which one was randomly chosen for uploading in the online question format on the morning
of the day of examination. The entire process was designed in a manner which was analysis
friendly through data mining, since the entire database of candidates was tabulated. CIPS ,
Hyderabad were responsible to conduct the examination.
ExaminationThe Examination was conducted on 15th September, 2016. 351 candidates from 14 Indian
States and 5 countries participated in the Examination (Vide List 1). Computer Laboratory at
SKIPA was used as a Test center at for willing candidates at Ranchi. Candidates have option
to participate in the Examination from the comfort of their home. The entire selection
procedure was outsourced to an independent organisation.
Figure: Participant Location were automatically recorded during Online Examinationinitiative by IPMU, SKIPA. Ensuring internet connectivity and availability of desktops forcandidates attending examination at the center and developing a mechanism to checkplagiarism and legitimacy of the candidate were key challenges in front of SKIPA Team.
Result : Checks and Balances
Evaluation of papers were made on the very date of Examination conducted by Faculty
members of SKIPA. The team was able to conduct the online examination and recruit the
team members of IPMU within a period of two months. The selected candidates have been
inducted and are working at IPMU. The system has capability of conducting Instructor
assisted Evaluation and uses data analytics module to correlate result. Some examples of
correlation study is given below:
A. Overall Relationship between three Evaluators for the post of State Program Officer
(SPO): According to this result, Marks similarity between Evaluator 1 & Evaluator 2 is
70.5%, Evaluator 1 & Evaluator 3 is 76.3%, and Evaluator 2 & Evaluator 3 is 96.1%.
B. Overall Relationship between three Evaluators for the post of Project Officer (PO):
According to this result, Marks similarity between Evaluator 1 & Evaluator 2 is 60.9%,
Evaluator 1 & Evaluator 3 is 71.2%, and Evaluator 2 & Evaluator 3 is 91.8%.
C. Overall Relationship between three Evaluators for the post of RA: According to this
result, Marks similarity between Evaluator 1 & Evaluator 2 is 64.9%, Evaluator 1 &
Evaluator 3 is 70.8%, and Evaluator 2 & Evaluator 3 is 95.6%.
Check points:
Pre Examination Provision for more than one question paper- three sets of paper for three
positions was kept in fray for the exercise. Care were taken to Frame questions that cannot be
searched through Google and other websites for answers (analysis based rather than subject
based). Timer Enabled with username and password bounded question set for each candidate
was ensured. Real Time Technical Assistance through support desk was arranged.
Post Examination Circulation of Answer Sheets for evaluation on the same day (by email),
Saved a lot of time and effort as compared to sending answer sheets through post. Since data
is collected, collated and dispatched for evaluation digitally without human intervention,
there was no loss of data. Answer sheets were encoded with watermark to avoid tampering of
data using Non-editable data format and converted into Password protected PDF documents.
Plagiarism check for answer sheets with details of web documents was conducted. IP and
location based information on geographical spread of candidates appearing for online test
were tracked on real time basis during the examination. The data was stored for future
reference and comparisons in subsequent years.
Impact
The online exam conducted by SKIPA for the selection of IPMU team members was one of
the first of its kind in Government Sector and Mission bodies in Jharkhand. The entire
process of recruitment was complicated with major challenges like Voluminous Application,
chances of Favoritism, Screening of applications based on specific parameters.
1. Automated Screening of applications without manual interference leading to no scope for
favoritism were adopted.
2. There was minimal Displacement of applicants owing to the online nature of the
examination ensuring higher participation and lesser Cost / Resource involvement.
3. It was possible to answer the examination from any corner of the world. Besides a large
number of applicants across the country, candidates from countries outside India could
attend the examination.
4. This also proved to be an environmental friendly means that reduced the need for travel at
various ends and also helped in paper saving.
5. The system appears to be more secure for management user records and more reliable to
work at any conditions.
6. There is no cost involved in printing or distributing of question papers. Transport and
logistic expenses for conducting examination in several geographical locations is saved.
7. It saves the travelling time involved for both the examinee and the examiner. It alsolessens the time and cost required for filling application and posting of application. Underthis system panel of question setters and examiner are independent of geographicallocation.
8. Assured Cyber security provisions enhances Confidentiality and minimize chances of
paper leaks. It expands the scope of conducting examination. At the same time
examination could be conducted across different geographical location. It gives
opportunity to more and more physically challenged candidates to participate in the exam.
The IPMU track record shown that, candidates from 28 Different State and countries have
applied for the positions. For candidates an estimated 4108 Hours of travel time were
saved. Government could save an estimated expenditure of approximately 15 lakhs rupees
to be provided otherwise in terms of Travel Allowance (AC III tier) and manpower
deployment. The entire selection process for 351 candidates was completed within a period of
Two (2) months. A similar recruitment process at UMC (Urban Management Cell),
Jharkhand conducted during the same period following traditional methods including offline
examination and Manual Communication cum Evaluation took Six (6) months for completion
to recruit from 35 candidates.
Way forward
Online technologies can provide useful learning and assessment opportunities for practical
skills when they are developed in a manner that promotes the obvious and coherent
relationship between subject matter, performance expectations, student demonstrations of
practical skills and feedback.
This is particularly important in the relationship between the formative and summative
assessment expectations associated with practical skills development and assessment.
Resources and requirements of the protocol promoted activities and behaviors may result in
notable gains in candidate’s examination quality.
Number of candidates appearing in government examination outnumbers participants in any
other form of exercise in India. Online Examination Connected with Common Service Centre
and Mobile based applications has potential to minimize cost and maximize quality.
Government agencies, including Jharkhand Public Service Commission and Human
Resource Department, Government of Jharkhand may try this on priority basis.
At a Glance: Key Advantages for Online Examination
Cost Effective A. Substantial cost reduction in the expenses of conduction an examB. No Printing or Distribution Expenses for question papers.
C. No transport and logistics expenses while conducting examsacross several geographically scattered locations.
Secured D. Maintain confidentiality and avoid paper leaks.
E. Change question papers at the last minute by pulling questionsfrom the Question Bank.
F. Randomize sequence of questions for each examinee: No twoexaminees will see the same question at the same time.
G. No data loss even in situation of power and internet failure.
Expands Scope H. Create multiple exams on multiple subjects for multiple courses.
I. Create exams in multiple languages and overcome the languageconstraints.
J. Conduct exams in several geographically scattered locations atthe same time.
K. Panel of Questions Setters and Examiners can be independent ofgeographical location.
User
Convenience
A. Users can take exams as per their convenience. Exam can beconfigured for 24X7 availability.
B. User can view the exam results immediately after the exam.
C. Option to display the feedback for correct answers.
Analysis D. Store data for comparison in subsequent years.
E. Customizable Reports to help you retrieve and analyze thedesired statistic data.
Go Green F. You will be saving a substantial amount of paper by using onlinemode of exams. Prevent the use of paper and save the planet.
Glimpses
Figure above shows Online Examination at Ranchi Test Center, SKIPA.Based on success of this initiative, SKIPA conducted an Online Exam ofComputer Basics on 28.09.2015 for the office staffs and the result cameout on the same date of examination. Figure below shows the same testin process.
Participant List for Online Examination Conducted by SKIPA
Sl No. Name of Country/ State Name of Cities Number of Candidates
1 Karnataka Bangalore 34
2 Delhi Delhi 30
3 Maharashtra Mumbai, Pune , Sholapur 20
4 Andhra Pradesh Hyderabad 11
5 West Bengal Kolkata 11
6 Jharkhand Ranchi, Dhanbad, Bokaro 9
7 Haryana Gurgaon, Faridabad, Kaul 6
8 Bihar Patna, BeguSarai 5
9 Rajasthan Jaipur 4
10 United State (US) San Bruno, Mountain Viw 3
12 Thailand Chanthaburi 1
13 Kerala Thana 1
14 Gujarat Himatnagar 1
16 Orissa Bhubaneswar, Cuttack, Kharial 3
17 Uttar Pradesh Allahabad, Ghaziabad 2
21 Maharashtra Kolhapur 1
23 Punjab Patiala 1
24 Uttarakhand Kundan 1
25 Madhya Pradesh Jabalpur, Bhopal 2
26 Karnataka Belgaum 1
27 Telengana Telengana 1
28 Andhra Pradesh Machilipatnam 1
Total 351
Concept:Shri Sudhir Prasad, Director General, SKIPA
Editorial Team:
Hiralal Mandal, Additional DirectorAnil Kumar, Joint DirectorSanjay Ambasth, Joint DirectorDr. Sanjay Singh, Joint DirectorUsha Dwivedi, Deputy Director
Assisted By:Suman Kumari, Ratnakar Tiwari, Anuj Kumar Sinha, Pabitra Sahu
Consulting Editor: Kallol Saha
Background Research by:
BADLAO INSTITUTE OF TRAINING AND MANAGEMENTV.I.P. Road, Arsande, Kanke, Tel: 0651- 2450372, E-maill: [email protected],Website:www.bitmedu.org, Ranchi-834004, Jharkhand, India
Published by
Shri Krishna Institute of Public AdministrationAdministrative Training Institute, Govt. of JharkhandMears Road, Ranchi-834008, Jharkhand,FAX: 0651-2283806 | Phone: 2283804Email:[email protected], [email protected],Web site: http://skipa.nic.in