Nebraska Assessment System RFP

Nebraska Assessment System RFPRFP Number NDE 08-449-01

Questions and AnswersNo additional questions are clarifications are permitted. However, NDE will attempt to clarify any conflicting responses.

RFP pg

Section Text in question

Question

21 1.VV Nebraska Technology Access Standards.

Does the online reporting system require access to the NDE Portal user management system? If so, is the system set up to pass user characteristics such as those for districts, and school access to the online reporting system Answer: The online system does not require access to the NDE Portal.

28 Implementation Plan table

II. Test Development

Do test specifications document exist (i.e., test design, blueprints, passage specifications, item specifications)?Answer: The test specifications are still under development and will be available in the Fall of 2008.

28 Implementation Plan table

Social Studies For costing purposes, should contractors assume time to work with the NDE in establishing Social Studies standards?Answer: No

30 Technology Capacity

Technology plans

Each year, districts and ESU’s that plan to participate in the e-rate program….submit a technology plan ….

Does the state have a statewide survey for systems in use at the school level for bidders to consider regarding online assessment capabilities? For Example, In each school, what platforms, operating systems, etc are in place? Appendix A does not specify software systems such as Windows NT, 2000, etc.Answer: A current accurate (audited) database of information about district hardware and software is not available.

30 Inclusiveness of Assessments

Pilot Test Please confirm if it is NDE’s intention to have a large print and Braille version of field test form(s).

Answer Yes, it is NDE’s intention to have a large print and Braille version of the field test form(s). An exact count is not available for large print and Braille. The following provides the information we do have. Nebraska has 228 visually impaired and 12 deaf/blind students ages 3 to 21.

30 4.a. Technology Capacity

The Department is in the process of securing proposals for a decision support system to enable users at various levels to access data and generate reports using data in the NSSRS.

Is there any expectation that the vendor solution will need to access and/or integrate with this system? If so, for what operational reasons?Answer: No, the decision support system will only access data/interface with the NDE.

30

37,

4 Technical Approach

4 a.d.i.

State Demographics

Additional information about Nebraska’s districts, schools and students, including enrollment data by grade level, can be found ….at the link provided

Contractor will ship test materials directly to schools.

Should contractors assume only Public schools will be involved in this

1

RFP pg


Question

assessment? Could the NDE please confirm number of schools/bldgs for costing purposes?Answer: Only public schools will be involved in this assessment. A chart of the enrollment at all grade levels and the number of districts and schools is provided at the end of these responses.

The data provided at the link is for 2006/2007 and shows counts for grades 9, 10, 11 and 12. For costing purposes, should contractors use an average of these student counts?Answer: That would be a way to estimate the number of students at one grade in the high school.

For costing purposes, should contractors assume any percent increase in student counts year to year starting with 06/07 data?Answer: As the chart shows, the student enrollment counts have been fairly stable.

Could the state provide student counts by grade and year to assure all contractors are using the same quantities? For example, the RFP states 175,000 students. If you add the total of Public schools grades 3-8 and the average of grades 9-12 from the 06/07 data it totals less than 150,000 students.Answer: See chart at the end of these responses. The 175,000 was an estimate.

31 Technology Use

508 requirements

…the contractor should provide specific examples of how technology will be applied to support the assessment system including meeting the requirements of accessibility in section 508 (29 U.S.C 794d) of the Rehabilitation Act …

In addition to the online assessments being accessible to students with disabilities, please clarify if PDFs of ancillary materials prepared for posting on NDE websites must be tagged for 508 compliance.

For costing purposes, please clarify if the computerized information system itself must be accessible to NDE users with disabilities consistent with Nebraska Technology Access Standards as described at: http://www.nitc.ne.gov/standards/accessibility/tacfinal.html Answer: Yes

31 B. Scope of work All proposals should be developed based on an estimate of 25% of the students participating online and 75% of the students using the paper/pencil tests.

For costing purposes, should contractor’s assume that each school is ‘all or nothing’ in terms of online vs. paper & pencil administration. (Reporting mandates are for student, by district.) Or should contractor’s assume mixed-mode administration within a building (school) Answer: Contractor should assume that each school will be all online or all paper/pencil.

32 1.a.iii Management Meetings

Please confirm if it is NDE’s intention that the contractor’s project manager be onsite for face-to-face weekly meetings during the first year of the contract.Answer: The RFP allows the bidder to propose video-conferencing or other options.

33 2. Assessment System Development

Items and tasks will include both

Please confirm that the item bank will include items for modified assessments. Has NDE already developed items based on modified academic achievement standards? If not, will the contractor be required to

2

http://www.nitc.ne.gov/standards/accessibility/tacfinal.html

RFP pg


Question

regular and modified assessments

develop modified items for the 2% solution?

Please explain what type of results of locally scored alternate assessments must be collected. Would this be results of individual assessments or summary, performance level category information?Answer: The modified assessments will be supplied by NDE for inclusion in the item bank and identified as such. Modified assessments are for the “2% solution”.

The results of the locally scored alternate assessments will consist of one score per subject for each student. (no subscores)

33 2.a.i Item Bank Please provide the number of items in the existing pool that have been field tested or not field tested by subject/grade, the number of reading passages per grade.Answer: Nebraska has no existing item bank. Items for the item bank are under development.

33 2.a.i Item Bank Do reading passages in NDE’s item bank require copyright permissions? If yes, for costing purposes, what percentage of passages should be assumed to require copyright permissions?Answer: NDE will provide reading passages for the item bank and will obtain any required permissions or copyrights.

33 2.a.i Item Bank What is the file format of items, passages, and art that NDE would provide to the contractor? What is the native file format (Word, Access, Illustrator, Photoshop, etc.)? Are only hardcopy available?Answer: The items will be delivered as WORD or EXCEL files – Office 2003 or 2007 as preferred.

33 2.a.ii Item bank Please clarify if NDE has existing items to embed on operational test forms during the term of this contract. Do Reading items that would be field tested in spring 2009 already exist?

Has the existing item bank for Reading been aligned to the current state standards (posted in June 2008)? Answer: Nebraska has no existing items. Items for the Spring 2009 field test are under development. NDE is building its initial item bank and all items are aligned to the revised state content standards.

33 2.a.iii Item Bank Multiple formats required for presenting assessment items. Should the contractor assume one of the multiple formats is for a breach form? makeup form?, other? What is the purpose for multiple formats?Answer: Multiple forms means two sets of tests. Example: Form A and B. Whereas, multiple formats is used in discussing the presentation/format for assessment items. Multiple formats for Reading might include selected response, passages, comprehension charts, graphics, etc..

33 2 Assessment System Development

Items and tasks will include both regular and modified assessments.

Please further define “modified assessments”. Is this the same as ‘alternate’ assessments (for challenged students) that will be created outside the scope of this RFP? Answer: The modified assessments are for the “2% solution” and are in addition to alternate assessments. Alternate assessments are not included in this RFP except for the requirement to be able to collect student results on alternate assessments.

33 2.aiii. Item Bank The contractor will develop multiple formats for presenting assessment

3

RFP pg


Question

items. Please further define expectation, particularly for the online version of the assessment. Is this something different than the standard student accommodations provided for challenged users?Answer: Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for reading..

33 2.a.iv Practice Tests The contractor will develop a unique practice test for each subject and grade level….would this practice test have to have similar accommodations to the regular test (e.g. Large Print, Braille)?Answer: Practice tests are online.

33 2. big. e). Keep the need to scroll down to a minimum.

Please expand on this subjective requirement? Is there a set item length that you have in mind? Answer: There is no set item length.

33 2.b.i. h) ’Online Assessments

Capability to click anywhere near the boundaries of choice to select a response.

The way the online assessment is designed, we have found that this behavior causes far more user error than any benefit gained. (The buttons are fairly large, and when you increase a buffer around them, the buffers tend to almost touch…so you have many instances of user accidentally clicking an unintended choice.) Is this a requirement you want to retain? Answer: Thank you for this information. Bidders may provide this rationale for not addressing this.

33, 3740

24.b8

Reporting Alternate Assessment Scores

Section 2. specifies that although the item bank will not include alternate assessments, the system “must provide a means for collecting the results of the locally scored alternate assessments. Section 4.b. on page 37 states that the online system “must allow for teacher or local recording of the results of alternate assessments.” And Section 8 on page 40 stipulates that “Reports must include results from the alternate assessments entered by the district.”

What kinds of alternate assessment scores (and how many scores per student) will be collected, and what kind of information must be included in the reports?

Answer: Alternate assessment scores will be one score per subject for each student (no subscores).

34 2.c Paper/pencil assessments

For costing purposes, how many NDE staff or other participants will attend the one-day style guide meeting?Answer: Plan on three NDE staff.

35 3.b.ii Student Id and tracking

The response should also discuss options for collecting information to generate student labels or computer access identification for students that move after the data files have been provided.

Based on student migration data, can the NDE estimate the number of students per grade that may require a second set of labels?Answer: Based on our experience with the Statewide writing test, we can estimate that this would involved 2-3% of the population included in the assessment.

35 3.c.ii. Identifying Districts

The contractor will propose a web-based system…and to collect or confirm information provided by the state such as contact information.

4

RFP pg


Question

For whom is the contact information? Contact information for student, or NDE users (administrators, teachers, etc.)?Answer: NDE will provide district level administrator contact information. The contractor should plan on collecting/verifying other district/school level contact information as needed.

36 3.d.ii Large print For costing purposes, what is the required number of large print test booklets to print per grade Answer: An exact count is not available for large print and Braille. The following provides the information we do have. Nebraska has 228 visually impaired and 12 deaf/blind students ages 3 to 21.

37 3.e.v.i. Each year the contractor will support the NDE in the production of up to three reports…

Does the NDE have examples of these reports from prior years? Could they be shared with the bidder’s?

Answer: NDE does not have copies of prior year state-generated reports.

39 7.b.i Equating Please confirm if NDE is seeking contractors’ advice on a test design that ensures year-to-year equating. Answer: Yes

40 8.a.i. Reporting of Results

‘The web based reporting system must enable NDE access...’

Do you need/want teacher access to the Classroom Roster report?Answer: Yes

41 IV.B.8.a.i.b Reporting How will “Classroom” be identified to provide the “Classroom Roster and Classroom Summary reports? Is there a classroom data value the state can provide to track classrooms?Answer: The NDE does not collect data at the classroom level, not even in the NSSRS. It is assumed that the contractor’s generated district reports would need to identify classroom summary data.

41 8.a. v & vi Reporting Provide a database

Please elaborate. Do you mean access to the information/data we capture and store to generate reports through a web interface? Access by a dba to our system? Availability of a static output file? “Provide…confidential database,” – provide for you to host? Answer: Yes, the database referenced in 8.a. is the information/data the assessment system would capture at assessment time and then supply as a file to NDE. Exchange of data will be by file. NDE will hold the longitudinal data for student records. The assessment system will hold the “point-in-time” data from the assessments needed to generate reports.

41 IV.B.8.vi Reporting Will the database requested here need to include the teacher associated with that student?Answer: It is assumed that the contractor’s generated district reports would need to identify classroom summary data. NDE does not collect data that would identify the teacher associated with a student.

42 IV.B.8.x Reporting Does the data analysis option need to provide reporting for additional tests other than the NeSA? Is the NSSRS set up to pass data to other systems on an ongoing basis? Is there a reference for the file record layout for data

5

RFP pg


Question

transferred from the NSSRS?Answer: The data analysis needs to include the teacher entered results for the alternate assessments since the alternate assessments are not a part of NeSA. The NSSRS is not set up to pass data to another system automatically at this time. Updated files can be made available on a timetable established jointly by contractor and NDE. The file record information on the NSSRS is at: http://www.nde.state.ne.us/nssrs/

42 8.a.viii Released items

Please provide information about the number of items that NDE wishes to release each year. Does NDE’s current item bank include sufficient items for creating the required number of forms, practice tests, and allow for releasing items?Answer: The number of items released in the initial year will be minimal but subject to increases over time.

42 8.a.viii Released items

Please confirm that sample student work as part of released sample test items applies only to constructed response items, which are not a required part of this scope of work.Answer: Correct.

42 8.a.xi Spanish language versions

Please clarify which reports or ancillary products require Spanish versions.

Answer: Student Progress reports for parents. Other ancillary products to be determined jointly by NDE and contractor.

General Question Will it meet the requirements to assume user credentials can be downloaded from the NDE portal or will there be a web service for system integration?Answer: NDE will provide district level administrator contact information. The contractor should plan on collecting/verifying other district/school level contact information as needed.

General Question Does the NDE have a Budget range in mind for this RFP?Answer: The Nebraska legislature approved this fiscal note to support LB1157: http://uniweb.legislature.ne.gov/FloorDocs/Current/PDF/FN/LB1157

General Question The RFP makes several references to the Table of Test Specifications (to be supplied by NDE.) Can NDE please make this document available to bidders in order for us to develop fully articulated test designs and costs, which are comparable from all bidders?Answer: As stated in the RFP, the NDE will not be making the Table of Test Specifications available to bidders at this time since it is not yet finalized. The Table of Test Specifications will be available by Fall 2008.

General Question Bearing in mind that we will have electronic copies, how long will we need to warehouse the paper forms of students’ answer documents after they have been scanned?Answer: Paper forms are disposed of during the first two weeks of January following the year in which the tests are administered.

General Question How soon after they are inventoried and any outstanding issues resolved, can test booklets be destroyed? Does the NDE require that the test booklets be warehoused for a specific period of time?Answer: Paper forms are disposed of during the first two weeks of January following the year in which the tests are administered.

General Question Is there any reason that prevents the contractor from hiring Nebraska teachers to write additional questions in case we need to increase the size of the item bank?

6

http://uniweb.legislature.ne.gov/FloorDocs/Current/PDF/FN/LB1157

http://www.nde.state.ne.us/nssrs/

RFP pg


Question

Answer: NDE will provide the items for the item bank.

Pgs 1 and 45 For how many years should bidders provide cost information – for just the initial two-year contract (October 2008 through June 2010) or for five years (October 2008 through June 2013)?Answer: Form A is to be completed for each fiscal year.

Pg 5 Section H Submission of proposals

How many copies of the separate Cost Proposal should be submitted?

Answer: Bidders should provide one original and five copies of the cost proposal as well.

Pg 5 Section H. Submission of proposals

For the package containing the information that the bidder wishes the State to withhold from the public, how many copies of that package should bidders provide?Answer: Bidders should provide one original and five copies.

Pgs 28-29 Section – Implementation Work Plan

Please clarify what is meant by “Verification Studies” following the second operational year of each of the subject areas.Answer: This would be validation studies.

Pg 30 Section – Inclusiveness of Assessments

The RFP states that policies regarding inclusiveness of assessments also apply to field tests. Should bidders plan to provide Braille, Large-print, and ELL accommodations for each of the field tests?Answer: Yes

Pg 31 Section- Technology Capacity Who is the contractor for the State Report Card and AYP calculations?Answer: Quilogy

Pg 32 Section – Scope of Work (introduction)

The RFP states that bidders should assume 25% of students participating online and 75% of students using paper/pencil tests. Over the five-year implementation plan, could these percentages change (i.e. a higher percentage of students participating on line vs. paper?) If so, please provide percentages for bidders to use for the entirety of the contract.Answer: NDE does anticipate the percentages will change over the years with increases on the online side. This assumption was to support equitable evaluation of proposals. Bidders may propose other ratios and rationale for years following the initial year.

Pgs 33-34 Section B1 Item Bank Should bidders plan for any committees to review items for content, bias, etc? If so, please provide specifications for committees (number of committee members, costs to be covered by the contractor, honoraria rates, etc)Answer: No

Pg 34 Section B1.c. What is NDE expecting for multiple formats for presenting assessment items in the item bank?Answer: Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for the Reading test.

Pg 34 Section B1.d. Will NDE be supplying items to be used on the practice tests?Answer: Practice test items should come from the item bank.

Pg 34 Section B1.d. How should the practice tests be delivered? Posted to the Web? Via the online test system? Printed copies? (if printed, please provide quantities for bidders to use in costing)Answer: Online

Pg 34 Section B3.a. Does NDE have a preference for test booklet/answer document design (i.e.

7

RFP pg


Question

scan able test booklets where students respond directly in the book, or separate test booklets and scan able answer documents?)Answer: Scannable answer documents

Pg 35 Section B5.a. Does NDE have a preference regarding timing of the Spring assessments (i.e., which month, length of test window?Answer: Not yet determined.

Pg 36 Section C4.b. Please provide quantities for large-print versions by grade.Answer: An exact count is not available for large print and Braille. The following provides the information we do have. Nebraska has 228 visually impaired and 12 deaf/blind students ages 3 to 21.

Pg 38 and 45 Sections D3.a. and H2.a For the test administration workshops and reporting workshops, would NDE be open to having some or all of those workshops conducted via WebEx (or similar technology?)Answer: NDE would be open to using technology to provide workshops as long as it is interactive.

Pg 39 Section E1.a The RFP indicates that the contractor is responsible for scanning or imaging all ancillary materials. Which ancillary materials does NDE anticipate needing to be scanned?Answer: To be determined jointly by NDE and contractor.

Pg 40 Section G3.b. The RFP states “The contractor will produce item statistics for all operational items. The contractor’s response must include a description of the item statistics that should be generated to assist in the evaluation of these items including a discussion of the appropriate statistics for multiple-choice items if this is included as an option.” Please confirm that the last part should read “appropriate statistics for constructed-response items if this is included as an option.”Answer: Yes, the last part should read “appropriate statistics for constructed-response items if this is included as an option.

Pg 41 Section H1.a. When are Parent/Guardian reports due in districts? When are summary reports due (via web release?)Answer: Parent/Guardian reports should be available by the end of the school year. Electronic reports of data for district staff should be available within two weeks, if possible.

Pg 45 Section C Payment Schedule Does a proposed payment schedule need to be included in the proposal?Answer: The contractor should propose a schedule of deliverables and a payment amount associated with each deliverable.

General Question Will the State respond to questions as they are received? Or will all answers to questions be posted on July 21? If the latter, will the State consider granting an extension to the proposal due date given the fact that bidders will not be able to accurately gather cost information in some areas until questions are answered?Answer: The answers are posted on July 21.

Section II.L. Evaluation Would NDE please provide information on the points available under each item in Section II.L. Evaluation?Answer: NDE is not required to do so.

General Question For estimating purposes, approximately how many operational and field test items does NDE anticipate will be on each test?Answer: NDE estimates approximately 60 operational test items. The number of field test items to be determined.

8

RFP pg


Question

General Question Approximately how many items will be available for the pilot test in each grade/subject?Answer: To be determined.

Pg 33 Section IV.A. Section 4.C4.d In how many languages does NDE need to support testing?Answer: Spanish is most common language after English.

General Question Approximately how many items does NDE expect to release each year in each grade/subject?Answer: The number of items released in the initial year will be minimal but subject to growth over time.

Pg 33 Section B Will the NDE be providing both the regular and the modified items and tasks? YesHow will these be distinguished? Can the NDE clarify what the “tasks” will consist of? Answer: Items will be tagged as modified or regular assessments.

Pg 33 Section B1. How many items and tasks per grade are expected to be delivered to the vendor in the initial year for each NeSA content-area field test?Answer: To be determined.

How many items and tasks per grade are expected to be provided each year thereafter?Answer: To be determined.

Will these NDE-supplied items be provided in multiple formats? Yes If so, would the NDE be able to approximate a proportional mix of item types?Answer: Items are still under development.

What is the format for delivering items and tasks (e.g., MS Word)?Answer: MS WORD or EXCEL (2003 or 2007 as preferred)

When will NeSA-R items and tasks be available for import into the electronic item bank?Answer: Winter 2009

What percentage overage is being applied to allow for item/task attrition due to poor field-test performance? To be determined

Could items and tasks be written directly into the vendor’s electronic item bank system? – This is optional.

Can the NDE clarify the meaning of “multiple formats for presenting assessment items?”Answer: For Reading, this might include comprehension charts, passage, graphics, selected response, etc.

Will the NDE provide additional items and tasks to develop the unique practice tests, or will items and tasks come from the field-test administrations?Answer: Practice test items will come from the item bank.

What is the mode of release of the practice test (i.e. PDF, online, paper, posted to website, etc.?)Answer: online

Pgs 34-35 Section B4 Would the NDE please describe its intention for developing “at least two

9

RFP pg


Question

Content of Test Forms forms of tests...per year per subject?” That is, what is the purpose of the two test forms? Will these forms represent different operational forms (with different sets of operational items), or will they be differentiated by unique sets of embedded field-test items/tasks?Answer: The two test forms are to be used in each operational testing. They should be equated but can include some of the same items.

Can one of the test forms be reused the following year if it is not needed?Answer: Yes

General Question When will the Table of Test Specifications be available for the NeSA-R? Is the total number of items/tasks per test form known?Answer: Fall 2008; NDE estimates 60 test items/tasks per subject area.

General Question For any permissioned information (i.e., art, passages, graphs, web pages, etc.), will the cost of the permissions be the vendor’s responsibility or will the NDE cover all permissions when items and tasks are delivered? If the vendor is responsible for the permissions, what are the expected percentages for commissioned versus permissioned reading passages for the NeSA-R?Answer: NDE will cover the cost of permissions or copyrights for items that it delivers to the contractor.

General Question What are the expected percentages for passage-dependent versus stand-alone items and tasks for the NeSA-R?Answer: Most will be passage dependent

General Question Will test items/tasks/passages be released? If so, what percentage of the items/tasks/passages by grade and content area will be released, and what is the frequency of release?Answer: Release of items will be minimal at first.

General Question If items will be released, is the vendor responsible for that deliverable? If so, in what format should that be provided?Answer: Electronic format

General Question Does the NDE expect a delivery of the item bank at the end of the contract?Answer: Yes

General Question Does the NDE anticipate delivering the initial field test in each content area on paper only?Answer: No – online and paper/pencil versions; field tests items embedded in operational tests after the initial year.

General Question What is the required delivery date for score reports after testing?Answer: Electronic score reports should be available to teachers in at least two weeks.

General Question What is the anticipated test window for the NeSA?Answer: Spring

General Question For the field-testing, what will the n-count be for each subject area?Answer: Bidder should propose an option for the n-count for field testing.

Pg 11 Q1.E. Ownership of Information and Data

Section E. “Ownership of Information and Data” of the RFP, discusses the contractor’s responsibility for obtaining permissions for copyrighted material that is used in the NeSA. What is the anticipated cost estimate for these fees?Answer: NDE will obtain permission and cover costs associated with any

10

RFP pg


Question

copyrighted material used in the development of the test items.

Pg 20 Q2 Limitation of Liability The statement that, “There will be no limitation to the Contractor’s liability under the contract...” Please clarify if liability is monetary. If so, it would stand to reason that some cap would need to be defined or negotiated upon contract award. Please comment.Answer: This is the State’s standard language and practice. Liability is not limited to monetary concerns nor to the value of the contract.

Pg 28 Q3 Implementation Work Plan Is the term “Full Implementation” synonymous with “operational test administration?”

Are the terms “field test” and “pilot test” (pg. 30 Inclusiveness) used synonymously for the purposes of this contract? If “no”, where does the NDE anticipate that the pilot will fit into the work plan on page 28?

Please provide the anticipated student count at each grade.Answer: Full implementation means operational test administration. While field test and pilot test have been used somewhat interchangeably in this proposal, the Implementation Plan for LB 1157 calls the first test, Spring 2009, a pilot. Field testing will need to be conducted prior to the initial operational test in each subject area. Test items will need to be embedded in subsequent years’ operational tests as needed. Student enrollment by grade information is provided at the end of these responses.

Pg 30 Q4 Assessment Systems Three groups are mentioned that will have input on the state’s new assessment: the National Advisory Committee, the governor’s State Technical Advisory Committee, and the Statewide Assessment Advisory Group.Please clarify the contractor’s role and responsibilities relative to these three committees and the number of days that should be allocated for contractor attendance at each of the state technical committee and state advisory committee meetings?Answer: The contractor is not required to be present at the meetings of these groups but may attend and may include the cost of attending in the bid. The contractor will assist NDE in meeting advisory group recommendations. Therefore, attendance at the meetings will be helpful.

Pg 30 Q5 Assessment Systems The RFP states that “the contractor’s response must discuss how the development of items and test forms will address this issue.” Elsewhere it is implied that the contractor has no item development role on the NeSA. Please clarify.Answer: Contractor does not develop items unless selecting and specifying this activity as an option. See introduction to “B”.

Pg 33 Q6. Assessment System Development

The RFP states, “The system must provide a means for collecting the results of the locally scored alternate assessments.”

Is “The System” in this reference referring to the NDE NSSRS or the new Assessment Delivery System?

Please clarify the contractor’s responsibilities in regards to the alternate assessment, i.e., if the contractor is required to collect results, please provide specifications for the information to be collected.Answer: The “system” in reference is the NeSA assessment delivery system. For the alternate assessment, the contractor must provide a way to collect results – one score (no subscores) per subject for each student. Less than 1% of Nebraska students take an alternate assessment.

11

RFP pg


Question

Pg 33 Q7 Item Bank How many useful items exist currently in the bank for Reading, Mathematics, and Science at each grade level?Answer: There is no existing item bank. Items are under development.

Are all existing items in the item bank aligned to the revised content standards or is contractor expected to conduct an item alignment?Answer: Items are being aligned to the revised content standards.

Are all relevant items characteristics logged/tracked in the item bank, i.e., Difficulty, DOK, key, Last appearance, Item Status, etc?Answer: Relevant information will be provided for each test item – the specifics to be determined.

When will items be provided to the contractor for test form development?Answer: The Reading items will be available by Winter 2009.

Is it the contractor’s responsibility to edit items from the item bank prior to test form construction, (field tests and operational tests?) If ‘yes’, what additional reviews should be included?Answer: No

In what format (e.g., MS Word, Access, etc.) will items and associated data be delivered to the contractor?Answer: Items and associated data will be in MS WORD or EXCEL files (2003 or 2007 as preferred).

Pg 34 Q8. Item Bank RFP outlines, “The contractor will develop multiple formats for presenting assessment items.” Does NDE have any desired formats?Answer: No

Does the NDE have a preference for a combined-subject test booklet versus separate subject area test booklets? Does the NDE have a preference for consumable test booklets or answer documents?Answer: NDE prefers scannable answer documents. NDE has not yet determined a preference for combined subject vs separate subject area test booklets.

Pg 34 Q9 Paper/Pencil Assessments Does NDE prefer the contractor to develop a customized style guide for the NeSA or is there an existing style guide that has been used for previous item/test development?Answer: There is no existing style guide.

Pg 35 Q10 B4 Content of Test Forms How many items are to be field tested in each subject per year and how many field test slots exist in each form?Answer: To be determined.

Pg 35 Q10 B4.b We assume this includes “per grade-level tested”, i.e., two forms of the Operational test (A & B) per year, per subject, per grade-level tested. Is this a correct assumption?Answer: Yes

What is the anticipated item refresh rate per year?Answer: To be determined.

Please provide testing time limitations by subject area and grade (i.e., number of minutes).Answer: To be determined.

When will the test blueprint and item specifications be available to the

12

RFP pg


Question

contractor?Answer: Table of Test Specifications available in Fall of 2008

Pg 36 Q11. Paper/Pencil Tests What are the print/color specifications for the test and ancillary documents? (Does state prefer four colors on white paper or just color cover with text consisting of black ink on white paper).

Please provide estimated page counts for test forms and ancillary materials.Answer: Ancillary documents in black ink on white paper. Print/color specifications for test is not determined but anticipate that some color will be needed for assessments that include graphics like charts, etc.

Pg 36 Q12. C4.a Paper/Pencil Tests RFP outlines, “Note that while the 10% figure is used as the basis for estimates throughout the RFP, very small schools will receive a fixed number of extra materials.”

Can the NDE define a “small school” and provide us with a definition of “fixed?”

Will the “fixed” amount vary depending on student population differences within the definition of “small school?”Answer: Nebraska still has some very small schools, enrollment of fewer than 10 students in all the grades assessed. These schools might not need 10% extra materials.

Pg 36 Q13 C4.b Paper/Pencil Tests How many large-print assessments per grade level will be required?Answer: An exact count is not available for large print and Braille. The following provides the information we do have. Nebraska has 228 visually impaired and 12 deaf/blind students ages 3 to 21.

Pg 36 Q14. C4.d Paper/Pencil Tests ELL Accommodations

Does the Department expect the contractor to propose accommodated forms for ELL students as an optional item or as part of the basic BID?

Please provide estimated numbers of ELL students by grade.Answer: Accommodated forms for ELL Students is an optional item. The latest count of all English Language Learner students is:Grades 3-8: 7,890 ELL studentsGrade 11: 614 ELL studentsThe closest estimate we have is that 10% of the ELL students are Spanish speaking.

Pg 37 Q15 D.1 Online Administration Please provide detail on the criteria used for selecting 25% of the students that will take the online test? (i.e., Will students be selected based upon District, School, Grade, Classroom, or other criteria?)

Should the contractor plan to provide paper and pencil test forms as backup for the 25% of students who will take the test online?Answer: NDE does anticipate the percentages will change over the years with increases on the online side. This assumption was to support equitable evaluation of proposals. Bidders may propose other ratios and rationale for years following the initial year.

NDE does not anticipate the need for paper/pencil test forms as a backup for the 25% of students who will take the test online.

Q16 Does NDE anticipate the need for Professional Development/Informational workshops for teachers, administrators, and/or students prior to implementation of the NeSA, excluding the administrator training workshops?

13

RFP pg


Question

Answer: NDE anticipates the Administrator Training and Reporting Workshops would not include students but would include building and central office staff.

Pg 40 Q17. G3.b Item Evaluation Should the last sentence say “constructed response” rather than “multiple-choice?” Please clarify.

Answer: Yes, the last part should read “appropriate statistics for constructed-response items if this is included as an option. The introduction to Part B says:

Nebraska wishes to pursue constructed response and machine scored assessments as part of the assessment system. Although constructed response assessments are not a requirement of this proposal, the NDE will consider vendor’s potential for providing this in the future as part of the proposal review. If including this option, the cost should be identified and included as a separate item in the budget.

Pg 41 Q18 H.1 Reporting of Results-Parent/Guardian Report

Please provide the estimated page counts and the print color specifications for the Parent/Guardian Report. (Does State prefer color print on white paper or black ink on white paper or some other combination?Answer: Black ink on white paper. Estimated page counts for Parent/Guardian Reports should be based on student enrollment in the grades tested.

Pg 43 Q19. H.1 J Reporting RFP outlines, “As a separate cost item, the contractor’s response must include costs for providing the State, each district and school with software to enable them to analyze and produce reports from their student-level data. The contractor’s response must include a detailed description of the proposed software.”

Can the NDE provide further information with regard to the level of analysis and reporting they would require from this product.Answer: NDE anticipates that, at a minimum, the district staff (teachers and administrators) should be able to analyze data at the student and classroom level (disaggregated) but only for their district.

Pg 43 Q20 H2 – Reporting Support How many participants are expected at each of the ten, half-day Reporting Workshops?Is contractor responsible for all travel reimbursements, substitutes, snacks, other?Is the contractor responsible for securing the location for these Reporting Workshops?Does the State pre-register the participants or is the contractor responsible for pre-registration?Are the workshops to be scheduled all at one location or multiple locations? If multiple, how many locations and where?

Administrator and Reporting Training:NDE anticipates that meetings will be held in various locations across the State. NDE will assist contractor in securing meeting locations. Contractor is responsible for any costs for the meeting rooms and any refreshments provided. For these meetings, participants will be responsible for their own travel costs, substitutes, etc. The contractor will be responsible for pre-registration, materials, etc. Contractor should plan on an average of 75 participants at each meeting.

Pg 43 Q21 I Standard Setting Please clarify the contractor’s responsibilities for the standard-setting meetings. E.g., securing facilities, inviting attendees, stipends, honorariums, travel, meals, hotel, etc.

14

RFP pg


Question

Please provide the State’s reimbursement rates for: stipends, substitutes, meals, mileage, and lodging.

Is the contractor responsible for securing the location for standard setting sessions?Answer: Costs of the standard setting session for each subject are to be included in the cost proposal. NDE will assist with making arrangements for meeting room(s). Contractor is responsible for determining the number of participants. NDE will assist with identifying appropriate individuals to participate.Average daily stipend for teachers is $150/day. Plan on similar amount for substitute pay. Mileage – current rate $.585 per mile, Lodging – estimate $150; the State meal reimbursement is $25.00 per day.

General Question What is the state mileage reimbursement rate? .585 cents/mile

What are the state rates for meal reimbursement? $25.00/dayGeneral Question In order to allow bidders to develop innovative, responsive proposals that

fully incorporate the NDE’s answers to bidder questions, would the NDE be willing to extend the due date for proposals to allow more time for bidders to adjust their solutions in response to the NDE’s answers to questions?Answer: As of this day, we have not decided to make any changes in the Schedule.

Pg 6 L. Evaluation Would the NDE please provide more detail about the evaluation process, specifically,

1. What is the maximum number of points possible for each of the four main categories and each sub-category?

2. Will the NDE weight any of the categories differently, or will they all be of equal importance?

3. What is the maximum cumulative score?4. What is the minimum score a bidder must receive to be considered

technically qualified?Answer: NDE is not required to provide this information.

Pg 24 IV. Proposal Instructions The RFP states that “bidders should identify the subdivisions of ‘Project Description and Scope of Work’ clearly in their proposals.” Please list which sections of the RFP are to be included in the “Project Description” subdivision.Answer: The RFP must only address the following. The Technical Approach (#4) of the Technical Proposal includes all that is needed for Project Description.

TECHNICAL PROPOSAL The Technical Proposal shall consist of four (4) sections:

1. The SIGNED “State of Nebraska Request for Proposal for Contractual Services” form;

2. Executive Summary;

3. Corporate Overview; and

4. Technical Approach.

Pg 26 Section 3.h.i.e Please clarify the NDE’s expectations regarding the “originally scheduled completion date and budget, as well as the actual (or currently planned) completion date and actual (or currently planned) budget.” Is this meant to compare an original bid/contract amount vs. an amended amount that may include scope changes?Answer: Include any amended amounts and scope changes, particularly

15

RFP pg


Question

related to completion dates.

Pg 30 Section 5. Technical Approach, State Demographics

Would the NDE please provide a break-out by grade for the approximately 175,000 students to be tested? What population growth rate does the NDE anticipate from year-to-year?Answer: Enrollment chart provided at end of these responses. It also provides longitudinal data.

Pg 32 Scope of Work Does the NDE anticipate the number of online testers to increase over the course of the contract? If so, will the NDE please provide an estimated percentage of students participating in online vs. paper/pencil assessments through 2013?Answer: NDE does anticipate the percentages will change over the years with increases on the online side. This assumption was to support equitable evaluation of proposals. Bidders may propose other ratios and rationale for years following the initial year.

Pg 33 A3 Technical Policy Issues, a and b

Please confirm that the contractor is responsible for only travel costs to attend these meetings.Answer: Yes

Pg 33 B. Assessment System Development

The RFP states that “Nebraska wishes to pursue constructed response and machine scored assessments as part of the assessment system.” Is the reference to “machine scored assessments” meant to differentiate multiple-choice items from constructed-response items?Answer: Yes

Pg 33 B1.a “The contractor will accept from NDE, items and tasks for each subject to be assessed for the creation of the item bank.” Should bidders assume that the items delivered by the NDE are those from which the Reading, Math, and Science field-test forms will be constructed? Approximately how many items per content area (Reading, Math, and Science) will be delivered to the contractor?Answer: Field tests will be developed from the items supplied by NDE. Total number of items to be delivered will be determined jointly by NDE and the contractor.

Pg 34 B1.c The RFP states that “The contractor will develop multiple formats for presenting assessment items.” Please clarify what is meant by multiple formats.Answer: Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for Reading.

Pg 34 B2.a What is classified as an “acceptable range of screen resolutions?”Answer: NDE has no prior experience in this and will rely on bidders proposed options.

Pg 35 B4.b The RFP states that “The contractor will develop at least two forms of the tests (i.e., Form A and B) per year per subject.” Will NDE supply a test blueprint/test specifications document?

Should bidders assume that the two forms per subject per year are administered in equal quantities or is one a breach form?Answer: NDE will supply the Table of Test Specifications. The two forms per subject per year are to be administered in equal quantities.

Pg 36 C4.b How many large-print booklets should bidders budget for per content area per grade?Answer: It is NDE’s intention to have a large print and Braille version of the

16

RFP pg


Question

field test form(s). An exact count is not available for large print and Braille. The following provides the information we do have. Nebraska has 228 visually impaired and 12 deaf/blind students ages 3 to 21.

Pg 36 C4.d The RFP states that “The contractor’s response should discuss issues involved in the various approaches to accommodating English language learners...” Which languages are the predominant first languages of ELL students in Nebraska? For costing purposes, for how many ELL accommodated versions should bidders budget?Answer: The latest count of all English Language Learner students is:Grades 3-8: 7,890 ELL studentsGrade 11: 614 ELL studentsThe closest estimate we have is that 10% of the ELL students are Spanish speaking.

Pg 37 C5.f “Each year the contractor will support NDE in the production of up to three reports related to the tests...” Should bidders assume that the three reports correspond to the three content areas, Reading, Math, and Science, in which tests will be developed?Answer: Yes

Pg 38 D3.a For costing purposes, please specify which costs related to the test administration workshops are the contractor’s responsibility.

Will the NDE be responsible for the cost of meeting space and any meals/snacks for participants?

Approximately how many district personnel will be in attendance at each of these meetings?Answer: Administrator and Reporting Training:NDE anticipates that meetings will be held in various locations across the State. NDE will assist contractor in securing meeting locations. Contractor is responsible for any costs for the meeting rooms and any refreshments provided. For these meetings, participants will be responsible for their own travel costs, substitutes, etc. The contractor will be responsible for pre-registration, materials, etc. Contractor should plan on an average of 75 participants at each meeting.

Pg 40 G3.b The RFP states that “The contractor’s response must include a description of the item statistics that should be generated to assist in the evaluation of these items including a discussion of the appropriate statistics for multiple-choice items if this is included as an option.” Did the NDE mean “constructed response items” in b) as was noted in G3.a)?Answer: Yes

Pg 41 H1 Please specify dates by which reports must be delivered to the districts and state.Answer: Electronic score reports should be available to teachers within two weeks after the testing window. Parent Progress reports should be available by the end of the school year.

Pg 43 H1.k For costing purposes, please specify how many Spanish language versions of the Parent/Guardian report templates and interpretive materials bidder should budget.Answer: The latest count of all English Language Learner students is:Grades 3-8: 7,890 ELL studentsGrade 11: 614 ELL studentsThe closest estimate we have is that 10% of the ELL students are Spanish speaking.

17

RFP pg


Question

Pg 43 H2.a For costing purposes, please specify which costs related to the reporting workshops are the contractor’s responsibility.

Will the NDE be responsible for the cost of meeting space and any meals/snacks for participants?

Approximately how many district personnel will be in attendance at each of these meetings?Answer: Administrator and Reporting Training:NDE anticipates that meetings will be held in various locations across the State. NDE will assist contractor in securing meeting locations. Contractor is responsible for any costs for the meeting rooms and any refreshments provided. For these meetings, participants will be responsible for their own travel costs, substitutes, etc. The contractor will be responsible for pre-registration, materials, etc. Contractor should plan on an average of 75 participants at each meeting.

Pg 43 H3 The text for H3 states that “The contractor will bear the costs to perform [the retrieval of student work].” The text for H3. A) states that “The cost for this service will be charged upon request and should not be included in the budget.” Please clarify.Answer: The proposal did not ask this as a separate budget item because we cannot anticipate the frequency of it occurring.

Pg 44 I1.c Would NDE please provide the following information? Daily stipend amount Average daily substitute reimbursement The average number of round-trip miles that each meeting

participant will travel Additional costs contractor will be responsible for (e.g., mileage,

accommodations, meals)Is the contractor responsible for selecting meeting facilities and any costs related to conducting the meeting (e.g. meeting room rental, snacks/meals)?Answer: Costs of the standard setting session for each subject are to be included in the cost proposal. NDE will assist with making arrangements for meeting room(s). Contractor is responsible for determining the number of participants. NDE will assist with identifying appropriate individuals to participate.Average daily stipend for teachers is $150/day. Plan on similar amount for substitute pay. Mileage – current rate $.585 per mile, Lodging – estimate $150; the State meal reimbursement is $25.00 per day.

Pg 45 and 46 B. Cost Proposal Requirements and Form A

a) Please confirm that bidders are to provide first-year costs for a 9-month year that runs from October 1, 2008 to June 30, 2009.

b) Please confirm that the five years for which bidders are to provide costs are as follows:

1. October 1, 2008-June 30, 20092. July 1, 2009-June 30, 20103. July 1, 2010-June 30, 20114. July 1, 2011-June 30, 20125. July 1, 2012-June 30, 2013

Answer: Yes

Pg 45 C. Payment Schedule Please clarify when the proposed payment schedule must be delivered.Answer: With the proposal

General Question If available, please provide the projected budget amount for the contract.Answer: The Nebraska legislature approved this fiscal note to support

18

RFP pg


Question

LB1157: http://uniweb.legislature.ne.gov/FloorDocs/Current/PDF/FN/LB1157

General Question Would NDE be open to conducting field testing (stand alone or embedded, ro both) only online as a cost saving measure? Or should bidders plan to conduct field testing both online and on paper?Answer: Plan for online and paper/pencil field testing

Pg 28 “The State intends to use the expertise and experience of the educators in the State to participate, to the maximum extent possible...” What are the State’s requirements regarding Nebraska educator participation?Answer: NDE will use Nebraska educators in developing the test items. The contractor should plan on involving Nebraska educators in the Standards setting.

Pg 29 Please clarify “Documentation of Standards of Revision.”Answer: Not included in this proposal.

Pg 29 Please clarify “Documentation of Test Development.”Answer: Not included in this proposal.

Pg 30, Inclusiveness of Assessments Would you please distinguish between pilot testing and field testing.Answer: Pilot and field testing are synonymous

Pg 32, Paragraph 3

Pg 42, Section H1.i

Since technical documentation is a critical requirement, can the NDE explain the difference between what they require for technical documentation vs. what is to be included in the Technical Report?Answer: Technical documentation of the processes used which may be beyond what is included in the Technical Report.

Pg 33, Section A2a5 Please describe the contents of the Table of Test Specifications.Answer: The Table of Test Specifications is under development.

Pg 33, Section A3b Does the narrative imply that the vendor will be attending two sets of two meetings for a total of four meetings per year?Answer: Yes but attendance is “as requested”. The contractor is not required to be present at the meetings of these groups but may attend and may include the cost of attending in the bid. The contractor will assist NDE in meeting advisory group recommendations. Therefore, attendance at the meetings will be helpful.

Pg 33, Section B The RFP indicates that items and tasks will include both regular and modified assessments. Please provide more detail about NDE’s expectation for the development of modified assessments. Is this for the 2% population? Is development and administration of separate modified assessments part of the scope of this contract?Answer: NDE will provide the modified assessments which are for the 2% population. The administration of these assessments is included in this contract.

Pg 33, Section B Please define what is meant by “machine scored assessments.”Answer: Multiple choice responses

Pg 33, Section B1 How many test items will NDE deliver for inclusion in the item bank by subject, grade, and type and when prior to item selection/forms construction will the vendor receive the test items?Answer: To be determined jointly by NDE and contractor.

Pg 33, Section B1 Will the test items provided by NDE be of final print-ready quality or will they require fact checking / editing / proofreading / reformatting by the vendor?Answer: Items will not required fact checking, editing proofreading, etc by

19

http://uniweb.legislature.ne.gov/FloorDocs/Current/PDF/FN/LB1157

RFP pg


Question

the vendor.

Pg 33, Section B1 What will be the format of the test items received from NDE (e.g., hard copy, electronic)? If items and their artwork are provided in electronic format, will native files be provided? What software will be used to create these files (e.g., MS Word, XML, InDesign, Adobe Illustrator)?Answer: MS WORD or EXCEL files (Office 2003 or 2007 as preferred)

Pg 33, Section B1 Will all reading passages and item artwork be commissioned material, owned by NDE, or will copyrighted material be used requiring permission from the authors prior to use? Will the vendor or NDE be responsible for securing permissions of any copyrighted materials?Answer: NDE will cover the cost of permissions or copyrights for items that it delivers to the contractor.

Pg 33, Section B1a Will items be proofed and ready to import into the item bank each year for the embedded field test?Answer: Yes

Pg 33, Section B1.b Should bidders include embedded field test items in operational forms every year, including 2013?Answer: Yes

Pg 34, Section Bld Please identify number of items of each practice test and will practice test items be drawn from existing NDE items or be newly developed?Answer: Number of items not determined yet, but items should be drawn from the item bank.

Pg 35, Section B4B How many items does NDE anticipate will be on each test per content per grade?Answer: NDE anticipates about 60 items on each test.

Pg 36, Section C4 Large-print test materials have been successfully developed using one of two standard approaches:

1. Photographic or electronic enlargement of the standard test materials without style and formatting adjustments other than size of the printed material.

2. Typeset documents using content from standard test booklet that uses different font, styles, layout, and format along with minor modifications to artwork and item format to address various item-specific readability issues.

Does the NDE have a preference as to which one of the two generic large-print formats described above should be used?Answer: No preference

Pg 40, Section G1.b What is meant by a student’s “composite score?” Since scores are being reported for each subject, can we assume that composit means total score?Answer: Composite means total score.

Pg 42, Section H 1 H What number of items from the operational test does NDE anticipate releasing as sample items?Answer: The number of items released in the initial year will be minimal but subject to increases over time.

Pg 43, Section I1 Is it the contractor’s responsibility to develop the Performance Level Descriptors prior to standard setting?Answer: NDE will develop the Performance Level Descriptors.

General Question Please provide information on the Test Administration window each spring.

20

RFP pg


Question

Are there particular dates already established or planned?Does NDE expect the field test window to be the same length as the full implementation window?Answer: No dates are set, proposed Spring administrationLength of field test window to be determined

General Question In Year 1, does NDE have a preference for the number of project management meetings that the contractor should budget to attend in person at the NDE, and the number that can be attended by other means (e.g., video conferencing)? Will these meetings be led by the NDE project lead or by the contractor’s Project Director?Answer: The meetings can be face-to-face or other means as jointly agreed upon by NDE and the contractor. The contractor’s Project Director will take the lead.

Item Bank 2.a.i. In what format will NDE provide the items and tasks to the contractor, e.g. hard copy, .pdf, Word, XML? Do the items already exist in a web-based rendering and if so, what is that format?Answer: MS WORD or EXCEL files (Office 2003 or 2007 as preferred)

General Question Can a sample item and task be provided for each content area?Answer: No

General Question Please provide information regarding the number and type of items that NDE will provide to the contractor, by grade and content area.Answer: To be determined.

General Question Does the item bank include more items than will be needed for the initial test forms? Does it include items that will be field tested by embedding in the operational forms in future years?Answer: The item bank will include the number of items that are needed as determined by the NDE and the contractor.

Item Bank 2.a.iii Can NDE clarify the meaning of “multiple formats for presenting assessment items?”Answer: Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for reading.

General Question What is the purpose of the practice tests? Are they to mirror the design of the operational test in terms of number of sessions and items, or should they be a shorter version?Answer: Practice tests can be online only so they can be different.

General Question Will the contractor be required to develop items for the practice tests, or will the items be drawn from the existing item bank?Answer: Practice test items should come from the item bank.

General Question Can NDE provide the test specifications or information on the test design for each grade/content area in terms of the number of items, sessions, and administration time?Answer: Not all of these decisions have been made. The Table of Test Specifications will be available in Fall 2008.

General Question For the operational tests, should both Form A and Form B be administered in a given year, or should just one form be administered?Answer: NDE anticipates that both Form A and Form B will be used in a given year.

21

RFP pg


Question

General Question What is the design of the field test(s)? How many forms are to be developed per grade?Answer: Only one form is needed for the field tests.

General Question Is NDE’s preference that the Year 1 reading field test be administered to all students?Answer: The bidder should propose to do a sample of students for the initial (Spring 2009) field test and include the rationale for the choice.

General Question Is NDE’s intent for the field test to be administered in both paper-pencil and online formats?Answer: Yes

2.b.i.a Online Assessments Is it NDE’s expectation that the contractor will be responsible for the development of any audio content associated with the audio capability? If so, what parts of the tests (items, passages, instructions, etc.) require audio capabilities?Answer: Yes. The parts of the tests to have audio will be decided jointly by NDE and the contractor.

2.b.i.a Online Assessments Does NDE expect audio capability to be available for ALL of the test forms developed or for a subset of the forms (e.g., all operational forms? One operational form per grade/content area? One field test form per grade/content area?)Answer: To be decided jointly by NDE and the contractor.

3.b.i Student Identification and Tracking Can NDE elaborate on what is provided in the “program” information for each student?Answer: The file record information on the NSSRS is at: http://www.nde.state.ne.us/nssrs/. Program information might include if a student is in a Title I program, if that information is requested by the Contractor.

3.b.i Student Identification and Tracking Is class roster information included in the data files provided by NDE or must the online and paper/pencil assessments capture this information at the time of testing?Answer: The NeSA must capture information about classroom/teachers. The NSSRS does not contain any link between the student and the classroom or teacher. It is anticipated that this information is needed in order for the NeSA to generate the reports for districts.

3.b.i Student Identification and Tracking Will each school be required to test all students in the same way (paper-pencil OR online?)Answer: Yes, the decision (online or paper/pencil) is at the school level.

3.b.i Student Identification and Tracking Is the web-based system mentioned in 3.c.ii an independent system simply to verify test ordering information for the NeSA or is it an SIS reporting tool?Answer: The NeSA system will need to be able to do both – verify test ordering information and also contain student information – demographic information from NDE and student results from the tests.

3.b.i Student Identification and Tracking The term “composite scores” (7.a.ii) can mean different things. Please clarify what NDE has in mind when referring to composite scores.Answer: Composite score means total score.

3.b.i Student Identification and Tracking Has the NDE considered the use of a vertical scale for the NeSA?Answer: Yes

Reporting of Results Please clarify if the reports listed in Section (8.a.i) are for the operational test

22

http://www.nde.state.ne.us/nssrs/

RFP pg


Question

administrations.Answer: Yes.

Reporting of Results What is the anticipated timeline for the delivery of operational reports?Answer: Electronic reports should be available to schools within two weeks after the test window, if possible.

Pg 41, 8.iv & 8.v. Reporting of Results Both of these sections refer to Section H1(a) of the RFP. This section designation does not appear to match the RFP section numbering. What RFP section is being referred to? Answer: This reference in both of these sections should consider all of the reports in H1. Reporting of Results.

Pg 5 Evaluation Can NDE provide the points or weightings that will be assigned to each category in the Evaluation criteria?Answer: NDE is not required to do so.

Section I, Page I Section I, Page 1, indicates the initial contract period will be “for a period of one (1) year and nine (9) months, effective on or about October 1, 2008 through on or about June 30, 2010”. Section B, 10, C, Page 44, states that the “initial budget should only cover October 1 – June 30 – 8 months” and Form A (Page 45) indicates we are to submit a separate form for “each fiscal year starting with a 9 month ‘year’ 2008-2009.” Section I. A, Page 1, indicates contractor start date is October 15, 2008. Please confirm that bidders are to prepare costs for a full 9 months of activities (October 1, 2008 – June 30, 2009) as one Form A Budget Summary plus costs for July 1, 2009 – June 30, 2009 as a second Form A Budget Summary covering the initial contract period.Answer: First-year costs are for a 9-month year that runs from October 1, 2008 to June 30, 2009.The five years are as follows:

1. October 1, 2008-June 30, 20092. July 1, 2009-June 30, 20103. July 1, 2010-June 30, 20114. July 1, 2011-June 30, 20125. July 1, 2012-June 30, 2013

Pg 13 Section III G Section III.G. Page 13 refers to the potential for multiple contract awards resulting from this RFP. Is this boilerplate legal text for all NDE RFPs, i.e., not something NDE wishes for this particular contract?

Or is it something that NDE is seriously considering for this contract? If the latter, could NDE indicate how such awards might be broken down (e.g. specific components)?

Answer: This is boilerplate language for all State RFPs. NDE intends to have a single contract resulting from this RFP.

Section 4 Technical Approach Page 27 - 29 includes a table, Implementation LB 1157 Work Plan, followed by the statement that this work plan identifies the major activities and timelines that will be the basis of the initial and subsequent contracts. Please clarify the selected contractor’s role in the following:

a. Standards Revision i. Is contractor to provide support for meetings

related to standards revision? ii. If so, will contractor be responsible for costs

other than those related to contractor staff travel and participation?

23

RFP pg


Question

iii. It appears that the revised standards will not be finalized until after the field testing for each grade/content. Is this assumption correct?

b. AYP Student Performance 2008-09 STARS Reading/STARS Mathematics and 2009-2010 STARS Mathematics

i. Is contractor responsible for developing a scoring and reporting system for the current STARS assessments?

ii. What other tasks and activities are expected related to the current STARS program?

1. data collection?2. data analysis?3. technical report?

c. NRT Recommendationi. Will the NeSA contractor have any

responsibilities related to the NRT administration?*

d. Reportingi. Will the NeSA contractor be responsible for

reporting at individual student level for STARS in spring 2009 for reading and math and in 2010 for math?

ii. Will the NeSA contractor be responsible for reporting NRT scores and subscores beginning in spring 2010?*

*Page 29: Local assessment, statewide writing assessment, and norm-referenced testing are not part of this proposal request.

Answer: None of the above is included in this RFP.

Pg 30 Section 4 Section 4. Page 30 states that the “contractor’s response must discuss how the development of items and test forms will address this issue [ELL inclusion].

Scope of Work Page 31 refers to ownership of items developed for this project, including test items not used on operational test forms.

Scope of Work Page 33 indicates that items and tasks will be provided by NDE. What role is the NeSA contractor expected to play in the development of items for the NeSA assessments?

Answer: The development of items for the NeSA is the responsibility of NDE.

Pg 32 A.1.c Scope of Work Scope of Work A.1.c . Page 32 states that bidders are to include weekly project management meetings in NE during the initial year of the contract and that this requirement may be reduced to bimonthly meetings after the initial year. Are bidders to assume the initial “year of the contract” is from October 1, 2008 through June 30, 2009 (first nine months)? Or does the initial year of the contract include July 1, 2009 – June 30, 2010?

Answer: The initial “year” is less than 12 months only due to the need to align contracts to the State fiscal year. Weekly meetings were specified for the initial year.

Pg 33 A.2.a.s Scope of Work Scope of Work A.2.a.s Page 33 states that there be equating of multiple

24

RFP pg


Question

forms including embedded items. Does the NDE have a preference for developing a pre-equated calibrated item bank so that test forms can be fully pre-equated?

Or does the NDE prefer that the scores be finalized by means of post-equating after the operational testing?

Answer: No preference.

Pg 33 A3 Scope of Work Scope of Work A3 Page 33 refers to annual meetings of the National Advisory Committee and the state technical committee and state advisory committees.

a. Please indicate the total number of State Technical Committee meetings to be budgeted for during the October 2008 – June 30, 2009 period.

b. Please indicate the total number of State Advisory Committee meetings to be budgeted for during the October 2008 – June 30, 2009 period.

c. Please clarify whether these committees are the same as the State Technical Advisory Committee and the Statewide Assessment Advisory Group referenced on page 29.

Answer: The contractor is not required to be present at the meetings of these groups by may attend and may include the cost of attending in the bid. The contractor will assist NDE in meeting advisory group recommendations. Therefore, attendance at the meetings will be helpful.There are three advisory groups: State Technical (1 meeting a year); State Advisory Committee (2 times a year) and the Statewide Assessment Advisory Group (at least 2 times a year). All meetings are held in Lincoln or Omaha.

Pg 34 B2 Scope of Work Scope of Work B2 Page 34 indicates the system must provide a means for collecting the results of the locally scored alternate assessments. Page 37 indicates the online testing system is to allow for teacher or local recording of the results of alternate assessments.

a. Will the NeSA contractor be responsible for test construction, administration, and scoring and reporting for the alternate assessments?

b. Are there two forms per year for the alternate assessments as with NeSA?

c. Are there new forms each year for the alternate assessments?

Answer: NeSA contractor is responsible only for collecting the results of the alternate assessments.

Pg 33 B1 Scope of Work Scope of Work B1Page 33 refers to “items and tasks”; however, there is also indication that the NeSA will consist solely of multiple choice items (constructed response as an option). Please clarify what is meant by the reference to tasks.

Answer: In this proposal, tasks are test items.

Pg 33 B1 Scope of Work Please provide the following information regarding the delivery of items and tasks to the contractor:

a. Numbers of items by grade and content to be providedb. Types of items by grade and content (i.e., multiple choice,

gridded response, constructed response)c. Number of items by “concept” (the subscore reporting

25

RFP pg


Question

category (see page 39) by grade and subjectd. Format for delivery of items for inclusion in the

item banke. Status of items (already approved by bias and content

committees and by NDE as appropriate for field testing? Any previously used items?)

f. Types of art or graphics that accompany the items. Format of art or graphics received (e.g. Adobe Illustrator)

Answer: (a) and (b) have not yet been determined. (c) will be included in the Table of Test Specifications available in Fall 2008; (d) Delivery of items will be by electronic file (WORD or EXCEL); (e) Items will be appropriate for field testing; (f) to be determined – items still under development.

Pg 33 B Scope of Work Scope of Work B Page 33 also indicates that the NeSA system and item bank will include “both regular and modified assessments”.

Is this a reference to modified assessments for the 2% population? If so, will the NeSA contractor be responsible for test form construction, administration, scoring and reporting for the modified assessment?

a. Are there two forms per year for the modified assessments as with NeSA?

b. Are there new forms each year for the modified assessments?

Answer: Modified assessments are for the “2% population”. Yes, the contractor will be responsible for test form construction, administering, etc. No, there will not be a need for two forms per year for the modified assessments.

Pg 33 Scope of Work Does the reference to multiple formats for presenting items mean for online format and for paper/pencil format?

Answer: Yes

Pg 34 B.1 Scope of Work What percentage of operational items will be released each year?

Answer: The number of items released in the initial year will be minimal but subject to growth over time.

Pg 35 B4b Scope of Work Are Form A and Form B spiraled for administration? Or is one form intended as either (1) a makeup or (2) a breach form? If not either these two reasons, what is the intended use? Furthermore, would both Form A and Form B be given online as well as in paper and pencil formats? For the sake of linking the two forms (A & B), will NDE allow common items on the two forms in a given year?

Answer: NDE anticipates that Form A and Form B would both be used in each administration of the tests – both online and paper/pencil. The two forms would need to be equated and could contain some of same items (common items).

Pg 34 B.3.a. Scope of Work Scope of Work requires a one-day style guide meeting.

a. Are bidders to assume this is a one-time only activity and to include costs in the October 2008 – June 2009 budget?

b. Is this meeting with NDE only, or will this include a committee of Nebraska educators? Is the contractor responsible for logistics, travel/reimbursement for any

26

RFP pg


Question

participants other than contractor staff? If so, how many and at what rates?

Answer: Bidders can assume this is an activity that would occur prior to the initial test in each subject area. This will include NDE staff. If Nebraska educators are included, NDE will cover their costs.

Pg 38 D3 Scope of Work Scope of Work indicates there are 10 half-day workshops each year. Is each workshop held in a different location? Is the contractor responsible for logistics, travel/reimbursement for any participants other than contractor staff? If so, how many and at what rates?

Answer: Administrator and Reporting Training:NDE anticipates that meetings will be held in various locations across the State. NDE will assist contractor in securing meeting locations. Contractor is responsible for any costs for the meeting rooms and any refreshments provided. For these meetings, participants will be responsible for their own travel costs, substitutes, etc. The contractor will be responsible for pre-registration, materials, etc. Contractor should plan on an average of 75 participants at each meeting.

Pg 43 H2a Scope of Work Indicates there are 10 one-half day workshops following release of test results. Is each workshop held in a different location? Is the contractor responsible for logistics, travel/reimbursement for any participants other than contractor staff? If so, how many and at what rates?

Answer: Administrator and Reporting Training:NDE anticipates that meetings will be held in various locations across the State. NDE will assist contractor in securing meeting locations. Contractor is responsible for any costs for the meeting rooms and any refreshments provided. For these meetings, participants will be responsible for their own travel costs, substitutes, etc. The contractor will be responsible for pre-registration, materials, etc. Contractor should plan on an average of 75 participants at each meeting.

Pg 43 I Scope of Work Will this contract include any standard setting activities related to the alternate assessment? Does it include any standard setting activities related to the modified assessment? If so, which content areas and during which fiscal year(s)?Answer: No, standard setting for the alternate assessment is not in the RFP.

Pg 36 C4 Scope of Work “The contractor will produce Braille versions of test booklets and related test materials (one test form per grade level). The contractor should budget for the production of 10 sets of Braille materials per grade.”

Should the contractor budget for production of both contracted and uncontracted Braille in any grades?

Answer: Sorry, do not understand the terms “contracted and uncontracted”.

Pg 36 C4d Scope of Work “The contractor’s response should discuss issues involved in the various approaches to accommodating English language learners (e.g., translated, side-by-side) and recommend an approach.”

Does the state wish contractors to cost the recommended approach to accommodating English language learners?

Can the NDE provide the languages desired and the number of translated tests to be produced by language?

27

RFP pg


Question

Are bidders to provide separate costs for each approach?

Answer: The bidder should cost the recommended approach to accommodating English Language Learners. Spanish is the preferred language. The more information provided the better.

Pg 34 B1d Scope of Work “The contractor will develop a unique practice test for each subject and grade level. Only one set of practice tests will be developed during the course of the contract – following the field and prior to the initial operational test.”

Should the practice tests be provided to the NDE in web-postable, and/or printable versions, or should the practice tests also be made available in versions that allow online practice testing, or in printed versions? If the latter, please provide estimated counts for students taking the online practice tests or printed copies to be produced.

Answer: Practice tests are online only.

Pg 34 B1f Scope of Work “Nebraska would consider proposals that include contractor supplied test items in an agreed-upon timeline between the contractor and NDE. If including this option, the costs must be identified.”

Should contractor-supplied items be understood as a supplement to or replacement of state-supplied items?

Will the costs of items provided under this option be weighed in the award selection?

Answer: Supplement to the state-supplied items. Optional activities are considered outside the evaluation of the cost proposals.

Pg 35 B.4.a Scope of Work “The contractor will support meetings of the NDE management team and the contractor to select items to be included on the pilot test and operational test forms.”

Please define the word “support” in the meeting context. Does this include such things as meals, site, lodging for all participants, or is it limited to materials and contractor attendance and participation?

Answer: Materials and contractor attendance and participation.

Pg 42 H.1.d Scope of Work “The contractor will provide NDE with copies of each set of web based reports described in section H1(a) above for archiving.”

Should these be provided on a CD or DVD, or will ftp-type transmission be acceptable?

Answer: Any of these formats would be acceptable.

Pg 43 H3 Scope of Work “At the request of NDE, the contractor will retrieve and deliver to the appropriate Department images of student answer documents, actual student test materials, printouts of results, and/or other reports in response to concerns about the accuracy of reported results. All requests must be made through the NDE project manager. The contractor will bear the costs to perform this task… a) The contractor’s response must include a cost figure and timetable for retrieving and delivering these reports upon request of the NDE. The cost for this service will be charged upon request and

28

RFP pg


Question

should not be included in the budget.”

What is the anticipated time limitation for requesting/rerunning reports, retrieving images of student multiple choice responses, and original student answer documents? Please clarify or reconcile “contractor will bear the costs” and “cost will be charged upon request.”

Answer:

Pg 36 C5 Scope of Work Should the manuals include both paper/pencil and online administration information? Or does NDE want separate manuals for each administration mode?

Answer: Contractor’s preference.

Pg 23 On page 23, in the Nebraska Identity and Access Management Standard for State Government Agencies - Standard 1.0, it states that all state and government web applications that require authentication and authorization of users will utilize the enterprise directory, know as Nebraska Directory Services. Is this geared more towards internal Nebraska groups, or does this apply to contractors as well? Does a contractor need to create an authentication/authorization system that works with Nebraska infrastructure, or would a contractor be allowed to use its own Identity and Access Management standards and processes?

Answer: Contractor is not expected to create an authentication/authorization system that works with Nebraska infrastructure. Contractor is expected to provide its own Identity and Access Management system and processes.

Pg 34 B2, A Our standard online tests can contain audio and be authored in whatever font is appropriate for the intended audience. However, accommodations capabilities that include zoom and text to speech are an additional cost. In order to appropriately cost, would you provide an estimate of the number of students that would require accommodations capabilities?

Answer:

Pg 43 Section H2a “The contractor will prepare materials for and support the administration of reporting workshops within one month of the release of results. Department and contractor staff will conduct the workshops. At each workshop, the contractor will be responsible for on-site registration, distribution of materials, and related tasks. The contractor should budget for 10 half-day workshops.”

Please clarify the contractor’s responsibilities regarding and test reporting or reporting workshop attendance that may be required as the result of the first year administration, which is for the Reading pilot test.

Answer: This would be dependent upon the bidder’

Pg 33, Section B1a “The contractor will accept, from NDE, items and tasks for each subject to be assessed for the creation of the item bank.”

On what dates will items be available from NDE for adding to the bank, and how many items will be available at each date?

Answer: The initial items will be available in Winter 2008-09.

29

RFP pg


Question

Please confirm that “Modified assessments” in section 2 is referringto assessments of modified academic achievement standards.Answer: Yes. NDE will provide the modified assessments.

What is the approximate number of items in the existing item pool bycontent area and grade? What is the anticipated alignment of theexisting items to the newly revised standards?Answer: Nebraska has no existing item bank. Items for the item bank are under development and are using the revised standards.

What is the expected number of items per form and anticipated itemrefresh and rate?Answer: The number of items released in the initial year will be minimal but subject to growth over time.

What is the desired number of items to be released following eachadministration?Answer: The number of items released in the initial year will be minimal but subject to growth over time.

Does the Department anticipate that the item pool provided by theDepartment will be sufficient to populate the desired number of testforms over the life of the contract?Answer: Yes, item development by NDE will be an on-going process throughout the term of this contract and beyond.

Will the items from the practice tests be come from the existing itempool?Answer: Yes

Please provide clarification on what is meant by “developing multipleformats for presenting assessment items” in section 2 a iii.Answer: Multiple forms means two sets of tests. Example: Form A and B. Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for reading.

General What is the testing window?Answer: The proposed testing window is Spring but no dates have been establish.

General Will there be make-up or retest administrations?

Answer: No.

GeneralWill the tests be timed? If so, how long should each content area assessment be?

Answer: To be determined. NDE anticipates about 60 items per test.

ELL Assessment Does the NDE require translated test forms? If so, in what language(s)?

Answer: The proposal asks for options for serving English Language Learners. The latest count of all English Language Learner students is:Grades 3-8: 7,890 ELL studentsGrade 11: 614 ELL studentsThe closest estimate we have is that 10% of the ELL students are Spanish speaking.

30

ELL Assessment Does the item bank already contain translated items?

Answer: Nebraska has no existing item bank. Items for the item bank are under development.

ELL Assessment Who is responsible for obtaining translations: NDE or contractor?Answer: NDE would be responsible for obtaining translations for test items. The contractor would be responsible for translations needed for administration or forms.

ELL AssessmentDoes the NDE require special English-language versions of the general assessment for ELL students? What modifications are intended?Answer: The proposal asks the bidder to provide some options.

ELL Assessment How do districts assess ELL students currently?Answer: Districts currently use classroom based assessments, some of which are in native languages.

27

Section IV, A, sec 3, i Summary of Bidder’s Proposed Personnel/Management Approach

Must references be provided for every personnel member named, or may we provide references only for those personnel who are client-facing?

Answer: Only required for personnel named who will be directly involved.

32Section A1, Management Team (c):

Can the vendor propose that weekly project management meetings be held via WebEx, video-conferencing, or some other electronic means?Answer: Yes

33Section B, Assessment System Development

The RFP mentions modified assessments here and only here. Is this referring to modified assessments for the 2% population? If so, will the contractor be responsible for developing modified assessments? Can the NDE provide test specifications? How many students will take the modified assessment? Will the contractor need to provide costs for standard-setting studies? Answer: Modified assessments are for the “2% population” and will be developed by the NDE. Contractor will need to include costs for standard setting studies for the modified assessments.

33 Section B1. Item Bank, How many items, per content area, are in the bank?Answer: Nebraska has no existing item bank. Items for the item bank are under development.

33 Section B1. Item Bank, Are the items aligned to the old or revised standards?Answer: Items under development are using the revised standards.

33 Section B1. Item Bank, Are there only MC items in the bank?Answer: Nebraska has no existing item bank. Items for the item bank are under development.

33 Section B1. Item Bank, Are/will all reading items be linked to a passage?Answer: Most will be passage dependent.

33 Section B1. Item Bank, Will the vocabulary standard be assessed?Answer: This will be included in the Table of Test Specifications.

33 Section B1. Item Bank,Will the vendor be responsible for obtaining permissions for passages, graphics, and other stimuli? If so, for what formats (paper/pencil. Web, etc.)?Answer: No, NDE will be responsible for obtaining permissions and copyrights.

31

33 Section B1. Item Bank,Have items been through content, alignment, and/or bias and sensitivity reviews? If not will the contractor be responsible for conducting review sessions?Answer: Yes

33 Section B1. Item Bank, In what format will the vendor receive the items?Answer: File – either MS WORD or EXCEL (Office 2003 or 2007 as preferred)

34 Section B1, Item Bank, “The contractor will develop multiple formats for presenting assessment items.” What does the NDE mean by “multiple formats?”Answer: Multiple formats might include selected response, passages, comprehension charts, graphics, etc. for reading.

34 B1.e. Item BankDoes the NDE want access to the items only for viewing, or does NDE desire item-editing capability?Answer: Both would be preferable.

35Section B4 (a), Content of Test Forms

“The contractor will develop at least two forms of tests (i.e., Form A and B) per year per subject.” Is one of the two forms to be a security or retest form?Answer: NDE anticipates that both Form A and Form B will be used with each test administration. The forms should be equated and can contain common items.

36 Section C4, Paper/Pencil Tests:Does the NDE prefer to have all content areas in one test booklet or does the NDE prefer separate booklets for each content area?Answer: No preference

36 Section C4, Paper/Pencil Tests: Does the NDE require consumable test booklets? If so, at all grades?Answer: No, NDE would accept scannable answer sheets.

37Section D1 (3), Online Administration

Does NDE require that on-line tests contain items that are identical to the paper-and-pencil tests, or will NDE entertain different/innovative on-line items?Do any districts currently administer online assessments? If so, approximately how many?Answer: Online and paper/pencil do not need to be identical but equated.

Approximately one-third of Nebraska’s districts use an online assessment for the classroom based STARS. However, these are not the largest districts in the State.

42Section H1 (h), Reporting of Results

How many items will be released each year?

Answer: The number of items released in the initial year will be minimal but subject to growth over time.

42Section H1 (h), Reporting of Results

What does NDE anticipate the reporting date to be for the first operational assessment in reading?Answer: Electronic reports should be available to classroom teachers and administrators two weeks after the test window, if possible.

32

33

GradeYears

1991-1992 1996-1997 2001-2002 2005-2006 2006-2007PK 2,534 3,750 4,541 6,167 6,933

K 22,160 21,847 20,197 21,861 22,080

1 22,501 22,213 20,120 20,946 21,452

2 22,220 21,431 20,078 20,412 20,880

3 22,151 21,149 20,665 20,235 20,449

4 22,442 21,421 21,148 20,273 20,338

5 22,653 22,103 21,529 20,524 20,402

6 22,252 22,527 22,223 20,681 20,618

7 21,363 22,997 21,725 21,521 20,993

8 20,569 22,836 21,708 21,967 21,604

9 21,223 23,955 23,744 24,837 24,411

10 19,410 23,651 22,629 23,548 23,233

11 19,022 21,358 21,941 21,381 22,376

12 17,957 19,644 21,487 21,195 21,366

Total 278,457 290,882 283,735 285,548 287,135

Enrollment chart – Nebraska Students from the State of the Schools Report

In 2006-07, Nebraska had 1, 068 school buildings in 254 districts.

Nebraska Assessment System RFP

Business

Transcript of Nebraska Assessment System RFP