Stan/Eval Testing (SET) Checklists and Process Flow Charts ... PEX 5.13 Stan/Eval Testing (SET)...

PEX 5.13 Stan/Eval Testing (SET) Checklists

1

Stan/Eval Testing (SET) Checklists and Process Flow Charts Current As Of: 11/18/16

The checklists contained in Section I of this document focus on frequently used Stan Eval Testing documentation activities and provide step-by-step instructions for completing these activities. As a general rule, they are organized in the following manner: first, activities required to set up the Testing module for a unit, then activities used to create and administer tests, and finally activities used to oversee and manage the units' testing program. Each checklist begins with three key pieces of background information: Purpose of the checklist; those Permissions or Roles in the Users & Permissions module which will allow a user to execute the steps in the checklist; and the role which is typically the Primary User of the checklist. Throughout the checklists, the user will see grey shaded areas labeled, “TTPs.” TTPs are tips, techniques and procedures provided for the user to better understand the activities being executed, to learn a trick of the trade, or to emphasize Air Force or MAJCOM policy guidance specific to the activity. Section II provides overarching process flow charts, which are designed to help a user understand or remember how to execute routine Stan Eval Testing processes using PEX. Those activities which are executed within PEX (as opposed to actions external to PEX) are indicated by unshaded rectangles and refer the user to the appropriate checklist to execute the specific activity (e.g., SET 3-1 refers to SET Checklist 3-1).

MAJCOM POCs, PEX ISR SME POCs and PEX ISR files (checklists, instructional

videos, CT HQ files, SEM Eval Criteria files, Qual Training Courses, and CFETPs) are

all located on the PEX website at:

https://home.pexmain.com

Questions and concerns regarding MAJCOM or mission set policies and guidance can

be directed to the appropriate MAJCOM POC(s) listed on the PEX website.

For general PEX questions and troubleshooting assistance, please contact: [email protected]

mailto:[email protected]


2

Summary of Changes

Description Date

Updated PEX ISR SME info 9/21/16

Updated PEX help desk info 11/18/16


3

Table of Contents

SECTION I: SET Module Checklists ............................................................................... 4

1 SET Module Set-Up .................................................................................................. 4

1.1 Testing Admin – Setup Tab ................................................................................ 4

2 Creating Exams ........................................................................................................ 6

2.1 Import QDBs ...................................................................................................... 6

2.2 Create or Edit QDBs .......................................................................................... 6

2.3 Export QDBs ...................................................................................................... 7

2.4 Add Questions to a QDB .................................................................................... 8

2.5 Edit Questions in a QDB .................................................................................... 9

2.6 Copy Questions from one QDB to Another ........................................................ 9

2.7 Create (Add) An Exam ..................................................................................... 10

3 Assigning and Un-assigning Exams to Personnel .................................................. 12

3.1 Assigning Exams .............................................................................................. 12

3.2 Un-assigning Exams ........................................................................................ 13

4 Taking, Grading, and Reviewing Exams ................................................................. 15

4.1 Taking and Reviewing an Exam ....................................................................... 15

4.2 Testing Admin Manual Grade Review and Adjustment .................................... 16

4.3 Making and Taking a Practice Exam ................................................................ 17

4.4 Entering an Off-Line Exam ............................................................................... 17

5 Testing Program Management ............................................................................... 18

5.1 View Assigned Exams ...................................................................................... 18

5.2 View Completed Exams ................................................................................... 18

5.3 Generate Testing Report .................................................................................. 19

5.4 Query for Trends (High-Miss Questions) .......................................................... 20

6 Using PEX to Administer Classified Exams ............................................................ 21

6.1 Bubble Sheet TTP explanation ......................................................................... 21

6.2 Download and Import Bubble Sheet QDB ........................................................ 21

6.3 Edit / Add Questions to the Bubble Sheet QDB ............................................... 21

6.4 Create Exam using Bubble Sheet QDB ........................................................... 21

Section II: Module Process Flow Charts ....................................................................... 22

SET Flow Chart 1: Creating Exams ........................................................................... 22

SET Flow Chart 2: Assigning, Grading, and Reviewing Exams ................................. 22

SET Flow Chart 3: Testing Program Management .................................................... 22


4

SECTION I: SET Module Checklists

1 SET Module Set-Up

1.1 Testing Admin – Setup Tab

PURPOSE: Allow personnel who create, administer and take exams to automate as much of

the task as possible. Trainers can create and manage tests that are part of MQT completion requirements (e.g., AOR Threats closed/open book tests, Vis Recce tests). Stan Eval personnel can create and manage the exams required for MSN and specialized evaluations.

Note: Be aware that Testing Admin permission allows access to the Setup tab as well as all exams and functionality within the PEX SE Testing module. SE Admin permissions do not allow access to the Stan Eval Testing module.

Permissions/Roles Able to Execute This Checklist: ⁻ PEX Admin ⁻ Testing Admin

Primary User(s) of this Checklist: ⁻ Testing Admin

Step Navigation Action

1 1. Login to PEX as a user with Testing Administrator Permissions

2 Main Menu -> Stan Eval -> Testing Admin -> Setup Tab

1. Select your “Squadron”.

3 Under “Subjects, References, & Media” link

1. Add “Subject” and/or “Reference”: a. Type in the desired subject or reference. b. Click the applicable “Add” button.

Note: Subjects can be associated with questions later and can be helpful if you want to ensure subjects are covered on an exam. References can also be assigned to questions later and make it easier to review all the questions associated with a reference document when it is updated. The “Assign Media” section will be used later—Media must be uploaded prior to being assigned, see steps 2 and 3.

4 Click “Upload/Delete Media” link

1. Click “Browse” to open the folder where your media files are stored.

2. Select the media file you wish to upload. Current formats that can be uploaded: .jpeg, .jpg, .bmp

3. Enter a description in “Media Description” (required field). 4. Click “Upload” button. Tip: Each picture needs to be uploaded individually. Recommend the test administrator upload all media first before creating questions for the exam.


5

5 Click “Subjects, References, & Media” link

5. Click “Assign Media” button 6. In the Webpage dialog: 7. Check the box of the media you wish to assign. 8. Selecting the Yes radio button next to “Show Media” will allow

you to see the picture that was uploaded. 9. Click “Assign Selected Media” button. 10. After the media is assigned and appears in the “Media

Description” column, you may edit the Media Description by clicking “Edit”.

11. Click “Save” to save changes.

6 Click “Trending Preferences” link

12. Select “Squadron”. 13. Set “Minimum Sample Size”. This is the minimum number of

times that a question needs to be answered in order to be considered a trend item.

14. Set “Threshold”. This is the minimum percentage of misses that must be met before being considered a trend item.

15. Set “Alert Minimum Sample Size”. This is the minimum sample size required before you’re alerted to an item that is close to becoming a trend item.

16. Set “Alert Threshold”. This is the minimum percentage of misses required before alerting you to a question that is close to becoming a trend item.

17. Click “Save Preferences”.

7 Click “QDB Source & Exam Types”

18. Select “Squadron”. 19. Enter “QDB Source”. HHQ and Local are usually sufficient. 20. Click “Add QDB Source” 21. Enter “Exam Type”. Closed Book and Open Book are usually

sufficient. 22. If “Exam Type” is a requisite which corresponds to an eval type,

click “Requisite” check box. 23. Select “Phase” from drop down arrow. Normally “Ground” is

used. 24. Click “Add Exam Type”.

8 Click “Proctors” link 25. Select “Squadron” Proctor(s) are to be assigned to. 26. Locate and click on the checkbox next to Short Name for each

Proctor. 27. Click “Save” button at bottom.


6

2 Creating Exams

2.1 Import QDBs

PURPOSE: A Question Data Bank (QDB) can be Imported which makes it possible for another unit or your HHQ to share their QDBs with you. Permissions/Roles Able to Execute This Checklist: ⁻ Admin ⁻ Testing Admin



1 Stan Eval -> Testing Admin -> “QDB” Tab

1. Click “Import QDBs” link 2. Select the “Squadron” to import the QDB to.

2 Import QDB 1. Click “Browse” button to find the QDB you want to import. 2. Note: Only files with a .qdbx extension can be imported. 3. Associate imported “Subjects”, “References” and “Media” with

existing ones. This will prevent duplication resulting from minor spelling differences.

4. Click “Import” to complete.

2.2 Create or Edit QDBs

PURPOSE: Follow the steps of this table to create QDBs. After you create a QDB, then you will populate the QDB with questions. Creating QDBs will allow you to manage and organize questions by subject and reference. Taking the time to have thorough information about each of the QDBs you create will make your testing materials easier to manage.

TTP: Creating a QDB allows the testing administrator to create required exams for completing training courses, qualification exams, and Stan Eval requisite exams. Providing more information in the description and being precise in the association with MDSs and subjects will help with management of testing materials. You may want to start with creating a QDB for each MQT knowledge event. For example, you may want to have a QDB for AOR Threats, MDS Weapons Systems Academics, PR Academics, FP Academics, Vis Recce and RAD. As you develop and populate your master question files you may want to further refine your QDBs. For example, you may want to refine AOR Threats and create separate QDBs which cover SAM, AAA, Aircraft, IED, Ground, Naval and Cyber threats. Organizing your QDBs will assist in being able to create exams which are able to focus on evaluating specific areas of interest.

Permissions/Roles Able to Execute This Checklist: ⁻ Admin ⁻ Testing Admin




1. Click “List QDBs” link. 2. Select the “Squadron” you want to create the QDB for.


7

2 Click “Create QDB” button at the bottom of the page to create a new QDB -or- To Edit a QDB’s info, click the “Details” link for the desired QDB.

3. If editing an existing QDB and all fields are grayed out, click the Edit button at bottom of screen

4. Enter “QDB Name” 5. Check “Public QDB” if desired. This option allows any user to

create practice exams using the questions in this QDB. 6. Fill in the description. 7. Choose a QDB Source from the dropdown menu 8. Check applicable MDS’s (optional). 9. Check applicable “Crew Positions” that the QDB will apply to.

(At least one Crew Position must be assigned) 10. Check the “Subjects” and “References” that apply to the QDB. 11. Click “Save QDB” to complete.

Tip: If you realize you need to list additional subjects or references, you must go back to the Setup->Subject, References, & Media tab and add them to the database before you can apply them to the QDB.

2.3 Export QDBs

PURPOSE: Exporting a QDB allows units to share their QDBs with other units or for a HHQ to share a standardized QDB with its units. Note: The QDB should be populated with questions prior to exporting and must contain questions before it can be used to create an exam. Note: If exporting a QDB that contains pictures or other media, Any file above a certain size will cause the entire QDB export.to fail. This is a known PEX issue.





1. Click “List QDBs” link. 2. Select the “Squadron” from which to select the QDB.

2 Export QDB 3. Click “Details” button to the right of the QDB desired to be exported.

4. Click “Export QDB” at bottom of QDB information displayed. 5. Choose “Save” and the file will most likely save in your

Downloads folder. You can also click the down arrow beside Save and choose Save As to save the file wherever you’d like.


8

2.4 Add Questions to a QDB

PURPOSE: Creating or adding questions to a QDB is necessary to populate the QDB. You can import questions into a QDB from another QDB or create new questions as needed.




1 Stan Eval -> Testing Admin -> “Questions” Tab

1. Select “Squadron”.

2 On the “List Question” screen

2. Select “QDB” to which you want to add questions. 3. A list of questions the QDB contains will appear. If it is a new

QDB, you will need to add or create questions. 4. Click on the “Create Question” button at bottom of page.

3 In the “Create Question” screen

5. Question numbers are generated automatically and sequentially. The question number may be changed if desired.

6. Choose “Question Type” by clicking on the radio button. o Multi = Multiple Choice question o True/False question o Fill In = Fill in the blank question. Be aware that for these,

the answer the examinee gives must exactly match the answer you type for PEX to auto-grade the examinee’s answer as correct. There is a way for a reviewer to give credit for a correct answer that doesn’t match exactly before a test is finalized though.

7. Version:

o “Never Include”--will not be included on any exams. o “Question Bad”--no longer trended or marked wrong on

exams. o “Always Include”--always used in PEX generated exams.

8. Fill in the remaining fields with the applicable information. Subject and Reference info can be useful later when reference docs are updated and the QDB needs to be reviewed.

Note: Ensure questions are properly classified. This ensures compliance if QDB is imported to classified network.

9. Type question in “Text” box. 10. Type possible answers in “Answer Text” fields. 11. Click on radio button to mark “Correct Answer”. 12. Click floppy disk icon to import images if desired.

o Click radio button to select image. Note: Images must be imported into the database already. Refer to Checklist 1.4 for procedure to import images

13. Click “Save &New” to continue adding questions

Or 14. Click “Save” to finish.


9

2.5 Edit Questions in a QDB

PURPOSE: It is necessary to edit questions in a QDB from time to time. The question may have had the wrong answer when it was originally created or a reference document may be updated which could change an answer or meaning of a question.




1 Stan Eval->Testing Admin -> “Questions” Tab

1. Click the “List Questions” Link:

2 “List Questions” screen 2. Select the “Squadron” 3. Select “QDB” you want to edit. A list of questions the QDB

contains will appear. 4. Click on “Details” link for the question you wish to edit.

3 “Edit Question” screen

5. Click “Edit” button on the bottom of the screen 6. Edit the fields as desired. Grey fields cannot be edited. 7. When you change a field in the question details, PEX will ask

whether the change you made is substantive. 8. Answer “No” if the meaning or intent of the question has not

changed (Ex: fixing a minor typo). If there is anyone who has an incomplete exam that has already answered this question, PEX will keep their answer.

9. Answer “Yes” if the meaning of the question changed and the answer will be removed from any open exams and the examinee will have to answer the question again.

10. Click “Save” to finish.

2.6 Copy Questions from one QDB to Another

PURPOSE: Copying questions between QDBs may be helpful in consolidating and reorganizing questions depending on how you want to have them divided among QDBs. You do not need to treat each QDB as if it is its own test. The Exam Tab will allow you to create exams by pulling questions from any QDBs you specify.




1 Stan Eval->Testing Admin -> “Questions” Tab

1. Click “Copy Questions” link


10

2 On the “Copy Questions” screen in the “Source QDB” section

2. Select the “Squadron” from which you want to copy questions. 3. Select “QDB” you want to copy the questions from. 4. A list of questions the QDB contains will appear. 5. Click on square in front of each question you want to select to

copy.

3 In the “Target QDB” section

6. Select the squadron to which you want to copy the questions. 7. Select the QDB to which you want the questions copied 8. Click “Copy Selected Questions”. When done you should get a

message that the questions were successfully copied.

2.7 Create (Add) An Exam

PURPOSE: This checklist will how to create an exam in PEX. To create exams you must first have completed the previous checklists and created at least one QDB from which to pull questions.




1 Stan Eval->Testing Admin -> “Exam” Tab

1. Under “List Exams” Link: 2. Select “Squadron”. 3. Click “Create Exam” button at bottom.

2 Exam Creation screen

4. Enter an “Exam Name”

5. Enter the desired “Number of Questions” 6. Enter the “Pass/Fail Percentage” – This is normally 85 (IAW 14-

202). 7. Check “Auto-generate Questions” if desired. If you do not check

this, you will pick each individual question on the test. If you do check this, PEX will auto-generate a test for you.

8. If Auto-generate is selected, select the appropriate radio button which will tell PEX to either pull any question from the QDBs you select or to use additional criteria to pick questions.


11

continued 9. Select “Exam Type” with drop down arrow. Usually this will be either Open Book or Closed Book.

10. “Proctored” exam is selected by default. Uncheck it if the examinee can take the test unsupervised.

11. Check the “No Go” box if not taking or failing the exam should cause the person to turn red on the Go/No-Go board. If the individual holds multiple crew positions the Stan Eval ball for the other crew positions will not turn red unless the failed exam was also a No Go exam for the other crew positions.

12. Check the "Repeat" box to enable options to set the exam to be re-assigned at specific intervals

o When the "Repeat" box is checked a "Repeat Exam Options" window appears. Select the start date the exam will be initially assigned, the interval the exam will be re-assigned, the date or number of occurrences the exam will no longer be re-assigned (if there is one), and when the exam is to be completed after being assigned or re-assigned.

continued 13. “Last Reviewed” defaults to current date. 14. Enter Instructions for the examinee. 15. Enter POC email and phone number. 16. Select “MDS(s)” and “Crew Position(s)” applicable to exam, at

least one MDS and Crew Position must be selected. Note: In PEX 5.12, failing an exam that is associated to an MDS/CP will cause the person to turn red or yellow on the Go/No-Go board (depending choices made in SEM->Preferences->Eval Rules grid)

17. Check box next to QDB(s) that questions are to be selected from.

18. Click “Next”

3 If “Auto-generate Questions” was selected:

19. Enter the number of questions from the selected QDB(s) to be included in the exam.

20. Click “Save Exam” button if you want the exam to contain the same questions for all examinees –or-- Click “Save Exam Template” if you want the exam to contain different random questions for each examinee.

4 If “Auto-generate Questions” was NOT selected:

selected at the top of the screen. 21. Check the box next to all the questions you wish to include in the

exam. 22. Click the “Add Questions” button. The bottom pane now shows “Selected Questions”. In the upper right you will see how many questions you have left to select. 23. To select questions from another QDB, click the “Next QDB”

button. 24. Continue to select questions and add them until the number of

questions requirement has been met. 25. Click the “Next” button at bottom of screen.


12

5 Stan Eval->Testing Admin -> “Exam” Tab -> List Exams -> Details -> View Questions

If you created a static exam, either by using the Auto-generated option and selecting “Save Exam”, or by not using an Auto-generated test, you may still change the questions on the test. 26. Add or Delete questions using the appropriate button at bottom

of the screen. It is possible to end up with more or fewer questions than desired if you modify the exam on this screen.

27. Click on the “Order Questions” link in the center of the screen. You can reorder questions randomly, by subject or by references (the default is random).

28. To order by subject or references, click on the appropriate radio button

29. Click on the desired subject or reference and use the up and down arrows to move the associated questions up or down within the exam.

30. Click OK. The questions should now appear in the new order on the Exam Display page. 31. Click “Finish Creating Exam”

3 Assigning and Un-assigning Exams to Personnel

3.1 Assigning Exams

PURPOSE: This checklist will walk you through assigning exams to personnel. Before doing this you must have created the exams using the previous checklist.




1 Stan Eval->Testing Admin -> “Exam” Tab -> “Assign Exams” link

1. Select “Assignment Type” by clicking on the appropriate radio button.

o “Person to Exam(s)” 2. –or-

o “Exam to People”

2 If “Person to Exam(s)” selected:

Under “Personnel”: 3. Select “Squadron”. That displays a list of squadron personnel. 4. Click “Short Name” of person to assign exam to. Under “Available Exam(s)”: 5. Check the box to select the exam. 6. Click “Assign” button to assign the exam. 7. Click “OK” to acknowledge the message box. If the exam is a repeating exam it will only show up in the lower right pane under "Repeat Exam(s)" until the date the date configured to be initially assigned. To immediately assign the exam: 8. Check the box to select the repeat exam. □ Click "Assign Now" button to assign the exam. □ Click "OK: to acknowledge the message box. 9. Click "Close" at the "Test are assigned" message box.


13

3 If “Exam to People” selected:

Under “Exams”: 10. Select “Squadron”. That displays a list of available exams. 11. Select “MDS”.

o “Crew Position” Filter can also be used. 12. Click “Exam Name” to select the exam. Under “Non-assigned Personnel”: 13. Select “Squadron”. That displays personnel that the exam HAS

NOT been assigned to. 14. Click check box to select each person. 15. Click “Assign” button to assign the exam to people checked. 16. Click “OK” to acknowledge the message box.

4 To assign due date(s): After the Exam is assigned, a “Tests are assigned—Webpage Dialog” window will pop up. 17. Click “Edit” to adjust the Due Date. Date defaults to the day the

exam was assigned. o Select Month/Day/Year from the drop downs.

18. –or- o Click the “Calendar” icon. o Click “Save” to set the new due date.

When an “Exam” is assigned to multiple people, the same due date can be assigned to the remaining individuals. 19. Click the Close button PEX will now auto generate an e-mail to the personnel you just assigned exams to as a reminder/notification.

3.2 Un-assigning Exams

PURPOSE: This checklist will give you the steps to un-assign exams when necessary.




1 Stan Eval->Testing Admin -> “Exam” Tab -> “Unassign Exams” link Use these steps to un-assign a single exam from one or multiple individuals

Under “Exam Name”: 1. Select “Exam” to un-assign Under “Assigned Personnel”: 2. Check the box next to each individual to un-assign the exam from. 3. Click “Unassign” button. Popup success notice will appear.

4. Click “OK” button to acknowledge and close the message box. Page will refresh and exam will no longer be assigned to person


14

2 Click “Assign Exams” link Use to un-assign an exam or multiple exams from an individual

Under “Personnel”: 5. Select “Short Name” of person to un-assign exam from. Ensure

their row is now highlighted in yellow. Under “Assigned Exam: individuals name” 6. Check the box to select the exam(s) that will be un-assigned. 7. Click “Unassign” button to un-assign the exam. 8. Click “OK” to acknowledge that you want to unassign the exam.

After clicking OK the exam should no longer show under Assigned Exams.


15

4 Taking, Grading, and Reviewing Exams

4.1 Taking and Reviewing an Exam

PURPOSE: This checklist will show how to take an exam in PEX

Permissions/Roles Able to Execute This Checklist: ⁻ PEX Admin ⁻ Testing Admin ⁻ Basic User

Primary User(s) of this Checklist: ⁻ Basic User


1 My Page (The page that displays when you first login to PEX)

There are two ways to get to “Testing” module: 1. Click “TEST” tab near top right of My Page,

-Or- 2. On PEX Main menu, click the Stan Eval section. 3. Click “Testing”.

2 “Assigned Exam(s)” Link:

4. Click “Exam Name” to select it. 5. Click “Take Exam” button.

Note: For a proctored exam, the proctor will have to select his or her name from the dropdown list and enter their PEX security identifier before the exam may be taken. Note: As questions are answered the question number will turn green. At this stage green only means that the question has been answered. 6. If the last question does not turn green when answered, click

the Back button and then the next button and the last question number should turn green.

7. Click “Complete Exam” button when done. PEX will auto-grade the exam and give you a score, however the testing administrator is able to review the test and give credit for answers that were auto-graded as incorrect if appropriate.

3 To review an exam (after completion): 8. Click “Exam Name” to select it. 9. Click “Review Exam” button.

4 “Review Exam”—Webpage Dialog

In the row of question numbers at the top of the window, correct answers will now be “green”, incorrect “red” 10. Review the incorrect answers – the correct answer will not be

given 11. Click “X” in upper right corner to exit review.


16

4.2 Testing Admin Manual Grade Review and Adjustment

PURPOSE: This checklist will show Testing Administrators how to review and exam and adjust grades if necessary.




1 Stan Eval->Testing Admin -> “Overview” Tab -> “Completed Exams” link

1. Choose a date range that will include the exam(s) to review 2. Check the box next to the squadron(s) desired. 3. Click “Go” button. Exams will appear in the main window.

4. Click “Review” link to review the exam. If credit should be given for any answer that was auto-graded as incorrect, proceed to the next step. This is most often encountered with fill in the blank tests.

2 5. Click the “Adjust” link 6. Check the “Accept as Correct” box for the question(s) that should

be counted as correct. 7. Click the “Complete Exam” button. 8. Click OK to verify you want to complete the exam. The grade will

now be changed.


17

4.3 Making and Taking a Practice Exam

PURPOSE: As a PEX user, you can create practice exams from public QDBs. If the testing administrator has not designated a QDB as “public,” you will not be able pull questions from the QDB to create a practice exam.

Permissions/Roles Able to Execute This Checklist: ⁻ PEX Admin ⁻ Testing Admin ⁻ All users

Primary User(s) of this Checklist: ⁻ All users


1 Stan Eval -> Testing -> “Practice Exam(s)” Link:

To take a practice exam: 1. 2. 3. 4. select “Crew Position”. 5. to pull questions from. 6. 7.

exam. 8. 9. one. 10.

Correct answer is green, incorrect is red.

4.4 Entering an Off-Line Exam

PURPOSE: Testing admins can input test answers from a hardcopy answer sheet. This will allow PEX to grade the exam and trend answers even if PEX is not used when the examinee takes the exam


Primary User(s) of this Checklist: ⁻ All users


1 Stan Eval -> Testing Admin -> Overview Tab -> “Assigned Exam(s)” Link:

1. Find the row with the Person and Exam Name for which you want to enter the answers

2. Click the “Enter Off-Line Exam” link


18

2 Off-Line Exam – webpage dialog

3. Enter the answers from the hardcopy answer sheet. Check “Answer not given” if appropriate.

4. Click Complete Exam when finished PEX will auto-grade the exam and answers will be trended

5 Testing Program Management

5.1 View Assigned Exams

PURPOSE: This checklist shows how to view all currently assigned exams for your unit’s personnel.




1 Stan Eval->Testing Admin -> “Overview” Tab -> Assigned Exams link

1. Under “Assigned Exams”, check the Intel unit(s) you wish to view to see all the assigned exams.

2. Click “Go”

o The tabular listing of personnel and their exam information will appear in the window. 3. Click “Excel” to export the information to an .xls file if desired. You

will then be prompted to save the file to your computer

l.

5.2 View Completed Exams

PURPOSE: This checklist will give you the steps to view completed exams for a specified time period in your unit




1 Main Menu -> Stan Eval -> Testing Admin -> Overview Tab -> Completed Exams link

1. Choose the start and end dates for the timeframe you want to see. 2. Select the squadron(s) 3. Click “Go”

o The tabular listing of personnel and their exam information will appear in the window. 4. Click “Excel” to export the information to a .xls file if desired From this screen, you can also review and adjust answers to exams. Refer to checklists 4.1 and 4.2 for those procedures.


19

5.3 Generate Testing Report

PURPOSE: This checklist will give a testing administrator the steps to generate a report that shows all exams given during a selected time period.




1 Main Menu -> Stan Eval -> Reports

1. Click “Testing Report” tab. 2. Under “Criteria Selection”: 3. Select “begin/end” dates for the report. 4. Select the “Squadron(s)”. 5. Select “MDS’s”. 6. Select “Qual Code”. 7. Select “CP”. 8. Select the output format--PDF or Excel by clicking the appropriate

button.


20

5.4 Query for Trends (High-Miss Questions)

PURPOSE: Trending will allow you to quickly identify high-miss questions and allow you to either improve the questions or improve your training program.




1 Stan Eval->Testing Admin -> “Trending” Tab

You can select trending criteria by QDB or by exam 1. Select the appropriate link (by QDB or by exam) 2. Select “Squadron”. 3. If trending by QDB, Select “MDS” you want to Trend. 4. You may also use the “Filter” link to apply additional criteria to

trending.

2 5. If trending by QDB: Check the box next to the QDB(s) you want to trend

6. If trending by Exam: Check the box next to the exam(s) you want to trend 3 In “Personnel” block 7. You must use a Personnel Filter in your query. If there are no previously created any Personnel Filters in your drop down list, refer to the Basic User checklists for procedures to create a filter.

4 In “Exam Criteria” block 8. Select the desired date range, “Start/End”.

The threshold and sample size defaults are set in the Setup tab but can be changed here if desired. 9. Set “Threshold” for what is considered a high-miss question 10. Set “Alert Threshold”. Usually this is close to, but not quite high-

miss 11. Set “Min. Sample Size”. If a question was asked fewer than this

number of times on a test, it will not be trended. 12. Set “Alert Min. Sample Size”. This number will usually be lower

than the Min Sample Size and is used to alert you to items that are close to being trends.

5 13. Click “View Results” button at bottom of screen to view results of

trend query.

You can export the results by clicking on the “Export To Excel” button at the bottom of the screen.


21

6 Using PEX to Administer Classified Exams

6.1 Bubble Sheet TTP explanation

The Bubble Sheet TTP uses an unclassified QDB that an examinee can use to input answers to questions from a hard copy classified source such as an Master Question File (MQF). The advantages of using this TTP include the ability to require an exam proctor (if a proctor is needed), electronically schedule and notify the examinee (via e-mail and examinee’s PEX My Page), trend results by exam or question, randomize order of questions within the exam, and automatically link requisite exam to an individual’s evaluation. Using PEX SET ensures compliance with AFI 14-202, Vol. 2 and AFI 14-2(MDS) Vol. 2.

6.2 Download and Import Bubble Sheet QDB


1 https://cs1.eis.af.mil/sites/pex/default.aspx

1. Download the Bubble Sheet QDB from the PEX support webpage: https://cs1.eis.af.mil/sites/pex/default.aspx

2. Use the “Import QDB” checklist or “Create QDB” checklist to build the bubble sheet QDB from scratch if desired.

3. Name the QDB, “Bubble Sheet”. If you need multiple bubble sheet QDBs you will want to clearly name them so you know what MQF or section of the MQF they go with.

6.3 Edit / Add Questions to the Bubble Sheet QDB


1 Refer to checklist 2.5 for procedure to edit questions in the QDB Refer to checklist 2.4 if you are building QDB from scratch and adding questions

1. Edit the questions in the QDB so the answers match the answers in the MQF.

2. Ensure that the questions are kept generic. For example: Question Text= “Question 1 from MQF” and Answer Text= “Answer A”

6.4 Create Exam using Bubble Sheet QDB


1 Refer to checklist 2.7 to create the exam

1. Name the exam or exam template appropriately; i.e., CENTCOM AOR FAM, Vis Recce Exam, MSN Knowledge Exam, etc…

2. Ensure you use the correct QDB as the source for questions

https://cs1.eis.af.mil/sites/pex/default.aspx




22

Section II: Module Process Flow Charts

SET Flow Chart 1: Creating Exams

SET Flow Chart 2: Assigning, Grading, and Reviewing Exams

SET Flow Chart 3: Testing Program Management

Stan/Eval Testing (SET) Checklists and Process Flow Charts ... PEX 5.13 Stan/Eval Testing (SET)...

Documents

Transcript of Stan/Eval Testing (SET) Checklists and Process Flow Charts ... PEX 5.13 Stan/Eval Testing (SET)...