Connected Vehicle Environment (CVE) Test Plan · herein and any trade name that may appear in the...

Smart Columbus

Connected Vehicle Environment (CVE) Test Plan

for the Smart Columbus

Demonstration Program

FINAL REPORT | April 17, 2020

Produced by City of Columbus

Notice

This document is disseminated under the sponsorship of the Department of Transportation in the interest of information exchange. The United States Government assumes no liability for its contents or use thereof.

The U.S. Government is not endorsing any manufacturers, products, or services cited herein and any trade name that may appear in the work has been included only because it is essential to the contents of the work.

Acknowledgement of Support

This material is based upon work supported by the U.S. Department of Transportation under Agreement No. DTFH6116H00013.

Disclaimer

Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the Author(s) and do not necessarily reflect the view of the U.S. Department of Transportation.

Connected Vehicle Environment (CVE) Test Plan – Final Report | Smart Columbus Program | i

Abstract

The purpose of this test plan is to establish a common framework for testing elements of the Smart

Columbus Connected Vehicle Environment system, including the system’s units, components, and subsystems. This plan will facilitate processes to be used by project stakeholders including the City of

Columbus, the System Integrator, hardware component suppliers, application suppliers, and other parties.

The plan categorizes all elements that make up the system; outlines the testing strategy; defines the test

tasks; defines interactions with other system elements; and provides governance for executing all testing activities and related equipment, including tools for logging, tracking, monitoring, and reporting test

outcomes.

The primary goals of the test plan are to provide guidance for:

1. Evaluating how each element of the system conforms with system requirements

2. Assessing whether each element satisfies its intended use and user needs as described in the Smart Columbus Concept of Operations

These determinations will be made using a blend of inspection, demonstration, testing, analysis, and

documentation review of various products, systems, and data. The expected outcome is to provide a basis for final acceptance of the system and move to the next project phase.

Connected Vehicle Environment (CVE) Test Plan – Final Report | Smart Columbus Program | iii

Table of Contents

Chapter 1. Project Background ............................................................................................ 1

Chapter 2. Risks and Contingencies .................................................................................... 5

Chapter 3. Test Items ........................................................................................................... 7

3.1. Test Components ................................................................................................ 7

3.2. Functionalities To Be Tested................................................................................ 8

3.2.1. Roadside Equipment ......................................................................................9

3.2.2. Light-Duty Vehicle Onboard Unit ..................................................................... 10

3.2.3. Heavy-Duty Vehicle Onboard Unit ................................................................... 12

3.2.4. Emergency Vehicle Onboard Unit.................................................................... 14

3.2.5. Transit Vehicle Onboard Unit .......................................................................... 15

3.2.6. Traffic Connected Vehicle Management System ................................................. 18

3.2.7. Transit Connected Vehicle Management System ................................................ 19

3.3. Functionalities Not To Be Tested ....................................................................... 19

3.3.1. Heavy-Duty Vehicle Onboard Unit ................................................................... 19

3.3.2. Data Distribution to Third Parties ..................................................................... 20

Chapter 4. Approach .......................................................................................................... 21

4.1. Testing Approach .............................................................................................. 21

4.2. Test Readiness Reviews .................................................................................... 21

4.3. Test Process ..................................................................................................... 21

4.3.1. First Article Testing ....................................................................................... 22

4.3.2. Channel 180 Congestion Testing ..................................................................... 23

4.3.3. Burn-In...................................................................................................... 23

4.3.4. Post-Installation Testing................................................................................. 24

4.3.5. Final System Testing .................................................................................... 24

4.3.6. Regression Testing ...................................................................................... 24

4.3.7. Operational Period ....................................................................................... 25

4.4. Testing Schedule............................................................................................... 25

4.5. Roles ................................................................................................................ 26

4.6. Test Tools.......................................................................................................... 28

4.6.1. Connected Vehicle Test Tool........................................................................... 28

4.6.2. Wireshark .................................................................................................. 28

4.6.3. Onboard Unit.............................................................................................. 29

4.6.4. Test Result Log ........................................................................................... 29

4.6.5. Defect Tracker ............................................................................................ 29

4.6.6. Change Request Log ................................................................................... 29

Table of Contents

iv | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Final Report

4.7. Environmental Needs ........................................................................................ 30

4.7.1. Bench Facility ............................................................................................. 30

4.7.2. Demonstration Area Facility ........................................................................... 31

4.7.3. Test Preparation .......................................................................................... 32

4.8. Measures and Metrics ....................................................................................... 32

4.9. Test Criteria ....................................................................................................... 33

4.9.1. Item Pass/Fail............................................................................................. 33

4.9.2. Testing Suspension and Resumption ............................................................... 34

Chapter 5. Test Deliverables .............................................................................................. 35

Chapter 6. Test Cases ........................................................................................................ 37

6.1. Onboard Unit..................................................................................................... 37

6.2. Onboard Unit Application .................................................................................. 44

6.3. Roadside Unit ................................................................................................... 50

6.4. Message Handler............................................................................................... 55

6.5. Traffic Signal Controller..................................................................................... 60

6.6. Traffic CV Management System ......................................................................... 62

6.7. Transit CV Management System ........................................................................ 69

Chapter 7. Test Scenarios .................................................................................................. 71

Chapter 8. Test Procedures................................................................................................ 79

8.1. Onboard Unit..................................................................................................... 79

8.2. Onboard Unit Application ................................................................................ 103

8.3. Roadside Unit ................................................................................................. 124

8.4. Message Handler............................................................................................. 136

8.5. Traffic Signal Controller................................................................................... 147

8.6. Traffic CV Management System ....................................................................... 153

8.7. Transit CV Management System ...................................................................... 168

Appendix A. Test Result Summary...................................................................................... 171

Appendix B. Terminology and Conventions......................................................................... 175

Appendix C.Acronyms and Abbreviations .......................................................................... 177

Appendix D.Glossary ......................................................................................................... 181

Table of Contents

Connected Vehicle Environment Test Plan –Final Report | Smart Columbus Program | v

List of Tables

Table 1: Connected Vehicle Environment Project Scope................................................................... 2

Table 2: Risks and Contingencies for Initial CVE Testing .................................................................. 5

Table 3: Device/Subsystem Test Matrix.......................................................................................... 7

Table 4: Metropolitan Area Test Phases and Time Periods ...............................................................25

Table 5: Alum Creek Area Test Phases and Time Periods ................................................................25

Table 6: Test Roles and Responsibilities .......................................................................................26

Table 7: Onboard Unit Test Cases ...............................................................................................37

Table 8: Onboard Unit Application Test Cases ...............................................................................44

Table 9: Roadside Unit Test Cases ..............................................................................................50

Table 10: Message Handler Test Cases ........................................................................................55

Table 11: Traffic Signal Controller Test Cases ................................................................................60

Table 12: Traffic CV Management System Test Cases ....................................................................62

Table 13: Transit CV Management System Test Cases ...................................................................69

Table 14: Testing Scenarios with Requirements Traceability.............................................................71

Table 15: Onboard Unit Test Procedures.......................................................................................79

Table 16: Onboard Unit Application Test Procedures..................................................................... 103

Table 17: Roadside Unit Test Procedures.................................................................................... 124

Table 18: Message Handler Test Procedures............................................................................... 136

Table 19: Traffic Signal Controller Test Procedures ....................................................................... 147

Table 20: Traffic CV Management System Test Procedures ........................................................... 153

Table 21: Transit CV Management System Test Procedures .......................................................... 168

Table 22: Test Result Log ......................................................................................................... 171

Table 23: Defect Tracker .......................................................................................................... 172

Table 24: Change Request Log ................................................................................................. 172

Table 25: Test Summary Report ................................................................................................ 172

Table 26: Defect Matrix ............................................................................................................ 173

Table 27: Test Case Status ....................................................................................................... 173

Table 28: Numbering Convention Details .................................................................................... 175

Table 29: Acronym List............................................................................................................. 177

Table 30: Glossary .................................................................................................................. 181

List of Figures

Figure 1: BSMs.........................................................................................................................93

Figure 2: Transit OBU ................................................................................................................94

Figure 3: Transit RLVW ..............................................................................................................95

Figure 4: Transit RSSZ ..............................................................................................................96

Figure 5: Transit Signal Priority ...................................................................................................97

Figure 6: Transit V2I ..................................................................................................................98

Figure 7: Transit TrCVMS ...........................................................................................................99

Figure 8: EV V2I ..................................................................................................................... 100

Figure 9: HDV V2I ................................................................................................................... 101

Table of Contents

vi | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Final Report

Figure 10: Transit V2I + OTA .................................................................................................... 102

Figure 11: LDV V2V................................................................................................................. 115

Figure 12: LDV RLVW ............................................................................................................. 116

Figure 13: LDV RSSZ .............................................................................................................. 117

Figure 14: LDV App Priority ...................................................................................................... 118

Figure 15: LDV V2I.................................................................................................................. 119

Figure 16: LDV V2I + OTA ........................................................................................................ 120

Figure 17: LDV RLVW+SRM..................................................................................................... 121

Figure 18: LDV RLVW+SRM+OTA............................................................................................. 122

Figure 19: LDV RLVW+SRM+2OTA ........................................................................................... 123

Figure 20: RSU Message Forward............................................................................................. 134

Figure 21: RSU Broadcast........................................................................................................ 135

Figure 22: Message Handler..................................................................................................... 146

Figure 23: Traffic Signal Controller ............................................................................................. 152

Figure 24: TCVMS .................................................................................................................. 167

Figure 25: TrCVMS ................................................................................................................. 169

Figure 26: Numbering Convention Definitions .............................................................................. 175

Connected Vehicle Environment (CVE) Test Plan – Final Report | Smart Columbus Program | 1

Project Background

In 2016, the U.S. Department of Transportation (USDOT) awarded $40 million to the City of Columbus, Ohio, as the winner of the Smart City Challenge. With this funding, Columbus intends to address the most

pressing community-centric transportation problems by integrating an ecosystem of advanced and innovative technologies, applications, and services to bridge the sociotechnical gap and meet the needs of

residents of all ages and abilities. In conjunction with the Smart City Challenge, Columbus was also awarded a $10 million grant from Paul G. Allen Philanthropies to accelerate the transition to an electrified,

low-emissions transportation system.

With the awards, the City established a strategic Smart Columbus program with the following vision and mission:

• Smart Columbus vision – Empower residents to live their best lives through responsive, innovative,

and safe mobility solutions

• Smart Columbus mission – Demonstrate how Intelligent Transportation Systems (ITSs) and equitable access to transportation can have positive impacts on everyday challenges faced by cities

To enable these new capabilities, the Smart Columbus program is organized into three focus areas addressing unique user needs: enabling technologies, emerging technologies, and enhanced human

services. The Connected Vehicle Environment (CVE) project primarily addresses needs in the enabling technologies focus area. The CVE project is one of the eight projects in the Smart Columbus program and is

a significant enabler of other technologies delivered through the other seven projects. The CVE project will integrate smart traveler applications and connected vehicles (CVs) into its transportation network by

focusing on deploying CV infrastructure and CV applications as follows:

• CV inf rastructure – The project will focus on building out the physical and logical CV infrastructure, which will consist of CV hardware and software (e.g., roadside units (RSUs), onboard units (OBUs),

f ront and backhaul communications, equipment interfaces). The CVE will generate the needed transportation-related data the applications use.

• CV applications and data – The project scope also consists of deploying CV-specific applications that

will leverage the data generated by the infrastructure to deliver real-time safety awareness and mobility services. Data will be collected, related, stored, and made available for use in other Smart

Columbus project applications.

The CVE is expected to enhance safety awareness and mobility for vehicle operators and improve pedestrian safety in school zones by deploying CV infrastructure on the roadside and CV equipment in

vehicles. The CVE will also provide sources of high-quality data for traffic management and safety purposes.

The foundation for the CVE is the Columbus Traffic Signal System (CTSS), which is a high-speed network backbone. When complete, the CTSS will interconnect 1,250 traffic signals in the Columbus region and

provide uniform signal coordination capability throughout the system. CTSS Phase D, which will connect all CVE corridors except for Alum Creek Drive, is expected to be completed by Q2 2020. An expansion of the

CTSS to connect Alum Creek Drive will be included in the CVE project.

CV infrastructure deployment will occur along four major corridors/areas. The deployment of in-vehicle devices will target populations that are located near, or that frequently use, the corridors where infrastructure

is deployed. Table 1 lists the improvements associated with the CVE.

Chapter 1. Project Background

2 | Smart Columbus Program | Connected Vehicle Environment (CVE) Test Plan – Final Report

Table 1: Connected Vehicle Environment Project Scope

Infrastructure Applications and Data

100+ RSUs 1,500 – 1,800 OBUs CV Applications Data Capture

The project will install RSUs and necessary communications equipment at ~90 signalized intersections in the project areas.

The project will install onboard units (OBUs) on participating private, f leet, emergency, transit, and f reight vehicles.

The project will deploy vehicle-to-vehicle (V2V) safety, vehicle-to-inf rastructure (V2I) safety, and V2I mobility applications.

The project will capture, relate, store, and respond to data generated by the inf rastructure and used by the applications for traffic management.

Source: City of Columbus

The intent of the CVE project is to improve safety awareness and mobility of travelers by deploying CV technology as part of a larger initiative within the City to improve the overall transportation system. CV

technology will also be deployed to support improvement in freight operations, another of the City’s goals.

CV is just one component, but if it proves to be effective, other projects can also benefit from the positive outcomes. The goal of the CVE project is not to develop applications to a high level of maturity, but to

leverage what has already been developed. Therefore, it is important to understand that the ability of the CVE to address the user needs captured in the CVE Concept of Operations (ConOps) depends on the

availability of hardware and software solutions that have been previously deployed (and subsequently improved upon). To this end, throughout the CVE systems engineering process, several applications have

been considered. Each application was scrutinized in detail to ensure that only applications that were ready for deployment are included in the deployment of the CVE. Performance requirements detailed in the

System Requirements document (see V2V Safety, V2I Safety, and V2I Mobility functional groups) outline expectations for each application to be deployed. The implementation of software is expected to

demonstrate the efficacy of the deployed infrastructure.

The applications and technology to be deployed as part of the CVE project are the same as (or very similar to) applications and technology employed in other connected vehicle projects. As similar applications are

developed and employed as part of the CV pilot projects, their maturity will continue to increase. It is expected that prospective vendors will be constantly improving applications, based on experience in testing

and implementation. Therefore, design and implementation of the CVE will draw on improvements made to applications through these development efforts.

The Architecture Reference for Cooperative and Intelligent Transportation (ARC-IT)1 and its predecessor,

the CV Reference Implementation Architecture (CVRIA),2 describe CV applications that have been researched in the context of the national ITS architecture. Furthermore, the ITS CodeHub (previously Open

Source Application Development Portal) contains software for applications that have been developed.3 When possible, applications on ARC-IT, CVRIA, and ITS CodeHub will be used as-is or will have minimal

modifications made to address user needs documented in the ConOps.

Given that the primary scope of the CVE project is to realize the benefits of deploying CV technology into an operational environment, only applications that have demonstrated sufficient levels of development and

testing are being considered for implementation. While (as mentioned above) prospective vendors will be

1 https://local.iteris.com/arc-it/ 2 https://local.iteris.com/cvria/

3 ITS CodeHub: https://its.dot.gov/code/

https://local.iteris.com/arc-it/

https://local.iteris.com/cvria/

https://its.dot.gov/code/

Chapter 1. Project Background


improving applications, the CVE project will be designed so that added functionality concepts (those that

require further development and that are not implemented as part of this project) can be integrated with the CVE once development and testing have matured to a point at which applications are deployment-ready.

Additionally, because of the networked nature of devices in the CVE, several policies and constraints related to information technology (IT) and data security are expected to be developed as part of the deployment.

The backhaul network that supports roadside equipment in the CVE will require establishment of a new network (on existing dark fiber). The City of Columbus Department of Technology (responsible for managing

Columbus’ fiber network) has been engaged to establish necessary network security policies and design.


Risks and Contingencies

2.1. RISKS AND CONTINGENCIES

Some systems, functionality, or devices may not be available during initial testing activities. Table 2 lists a

few risks and contingencies for initial CVE testing.

Table 2: Risks and Contingencies for Initial CVE Testing

Risk Contingency

Internet protocol version 6 (IPv6) is not available during initial testing.

OBU certificate top-off and over-the-air (OTA) firmware updates cannot be tested until the IPv6 system is operational.

The Smart Columbus Operating System (Operating System) may not be available/accessible during initial testing.

A laptop computer connected to and simulating the Operating System will be used. Data intended to be sent to the Operating System will be sent to and stored on this laptop, demonstrating the process of sending data to the Operating System.

Message Handler hardware may not be available during initial testing.

A laptop computer connected to the Signal Controller and RSU will be set up to perform the message handler functions.

Some OBU applications may not be available during initial testing.

OBU applications will be tested in the controlled environment as they are available, prior to being released to OBUs installed in vehicles. Applications will be installed in OBUs installed in vehicles through an OTA f irmware update.

OBUs and RSUs from one or

more manufacturers may not pass bench testing.

OBUs will not be installed in vehicles until they have passed bench

testing. RSUs that do not pass bench testing will not be installed for First Article testing (see Section 4.3.1) until they pass. RSUs from another manufacturer may be installed in their place.


2.2. ASSUMPTIONS

The City of Columbus has the following assumptions related to testing the CVE:

1. DSRC devices are OmniAir Certified or will be OmniAir certified before system “Go-Live”. The City does not plan to test IEEE 802.11p, 1609.2, 1609.3, or 1609.4 standards compliance, as devices will have been tested for detailed standards compliance during OmniAir certification testing.

2. DSRC Device Manufacturers will be on-site to support all tests involving their hardware

3. DSRC Device vendors will provide all ancillary hardware, cables, wiring, etc. required to operate their device

4. DSRC Test Tools have been verified to properly receive and decode SAE J2735 messages. The City does not plan to verify tools.

5. RSUs f rom all vendors have the same external interfaces for connecting to Signal Controllers, Message Handler, SCMS, Backhaul, etc.

Chapter 2. Risks and Contingencies


6. Additional test iterations will be required if a system, functionality, or device is not available during initial testing. These additional testing activities will be coordinated and scheduled based on the availability of the stakeholders.

All Tests described in this document must pass in order for the CVE to pass Acceptance Testing.


Test Items

3.1. TEST COMPONENTS

The Smart Columbus CVE is composed of the following Dedicated Short Range Communications (DSRC) device types:

• RSU

• Light-duty vehicle (LDV) OBU with human machine interface (HMI)

• Heavy-duty vehicle (HDV) OBU

• Transit vehicle OBU

• Emergency Vehicle (EV) OBU

Light-duty vehicles are the only vehicle type with an HMI provided by the Smart Columbus project and

therefore the only vehicle type in which operator alerts/warning will be tested.

Table 3 lists the CVE devices and subsystems to be tested, as well as the devices and subsystems against

which each will be tested.

Table 3: Device/Subsystem Test Matrix

Device/Subsystem Under Test Devices or Subsystems Tested Against

Kapsch RSU Message Handler

Siemens OBU

SCMS

CVE network

Danlaw RSU Message Handler

Siemens OBU

SCMS

CVE network

Siemens RSU Message Handler

Siemens OBU

SCMS

CVE network

Message Handler Kapsch RSU

Danlaw RSU

Siemens RSU

Econolite signal controller

Siemens signal controller

Traf fic connected vehicle management system (TCVMS)

CVE network

Position correction system

Chapter 3. Test Items


Device/Subsystem Under Test Devices or Subsystems Tested Against

Siemens OBU Kapsch RSU

Danlaw RSU

Siemens RSU

Transit connected vehicle management system (TrCVMS)

SCMS

Siemens OBU firmware server

CVE network

Forward Collision Warning (FCW) LDV application

Electronic Emergency Brake Light (EEBL) LDV application

Lane Change Warning (LCW) LDV application

Intersection Movement Assist (IMA) LDV application

Red Light Violation Warning (RLVW) LDV application

Reduce Speed School Zone (RSSZ) warning LDV application

Signal Priority for transit and HDVs

Signal Pre-emption for EVs

TCVMS Message Handler

Operating System

CVE network

City CVE IT network RSUs

Message Handler

SCMS

Siemens OBU firmware server

Position Correction System Message Handler

Transit Connected Vehicle

Management System (TrCVMS) Siemens (transit) OBU

Operating System

CVE network


3.2. FUNCTIONALITIES TO BE TESTED

This section contains a high-level description of the functionalities of Smart Columbus CVE elements that

will be tested as part of this test plan. Because of the potential re-allocation of the 5.9 GHz band proposed by the Federal Communications Commission (FCC) in its Notice of Proposed Rulemaking (NPRM) released

on December 12, 2019, and the fact that the FCC will only grant RSU licenses for the existing DSRC 10 MHz Channel 180, Smart Columbus RSUs will only broadcast on Channel 180. Since DSRC OBUs do not

require an FCC license to broadcast, vehicle-to-vehicle communications will remain on the Safety Channel, 172.



3.2.1. Roadside Equipment

The following functionality of the Smart Columbus CVE roadside equipment will be tested:

• The RSU’s ability to broadcast wireless access in vehicular environments (WAVE) service advertisements (WSAs) once per second on Channel 180. WSAs will advertise signal phasing and

timing (SPaT), MAP, Radio Technical Commission for Maritime Services (RTCM), and internet protocol (IP) services and contain the WAVE routing advertisements (WRA).

A successful test of this functionality indicates:

▪ The RSU can generate and broadcast WSAs as defined in Institute of Electrical and

Electronics Engineers (IEEE) 1609.3

• The RSU’s ability to broadcast J2735 SPaT messages 10 times per second on Channel 180


▪ The Signal Controller sends SPaT data to the Message Handler

▪ The Message Handler encodes SPaT data into J2735 SPaT messages

▪ The Message Handler sends J2735 SPaT messages to the RSU as “immediate forward”

messages

▪ The RSU broadcasts SPaT messages on Channel 180

• The RSU’s ability to broadcast J2735 MAP messages once per second on Channel 180


▪ TCVMS operators can generate MAP messages

▪ Traffic Connected Vehicle Management System (TCMS) sends MAP messages to the

Message Handler

▪ Message Handler stores MAP messages

▪ The CVE network supports connection between the TCVMS and the Message Handler

▪ The Message Handler sends J2735 MAP messages to the RSU as immediate forward

messages

▪ RSU broadcasts MAP messages on Channel 180

• The RSU’s ability to broadcast RTCM messages once every 2 seconds on Channel 180


▪ The CVE network supports connection between the Position Correction System and the

Message Handler

▪ The Position Correction System sends RTCM data to the Message Handler

▪ The Message Handler encodes RTCM data into J2735 RTCM messages

▪ The Message Handler sends J2735 RTCM messages to the RSU as immediate forward messages

▪ RSU broadcasts RTCM messages on Channel 180

• The RSU’s ability to broadcast a signal status message (SSM) once per second, while a request is

active, on Channel 180




▪ The Signal Controller sends signal priority status data to the Message Handler

▪ The Message Handler encodes signal priority status data into J2735 SSMs

▪ The Message Handler sends J2735 SSMs to the RSU as immediate forward messages

▪ RSU broadcasts SSMs on Channel 180

• The RSU’s ability to request and receive IEEE 1609.2 certificates from the Ohio Department of Transportation (ODOT) SCMS


▪ The CVE network supports a connection between the RSU and the ODOT SCMS

▪ The RSU determines it needs additional certificates

▪ The RSU sends a certificate request to the ODOT SCMS over IPv4

▪ The RSU receives and processes new certificates from the SCMS

• The RSU’s ability to forward basic safety messages (BSMs) and signal request messages (SRMs) to

the Message Handler


▪ The RSU is configured properly to forward messages to the Message Handler

• The RSU’s ability to broadcast RSSZ traveler information messages (TIMs) once per second on Channel 180


▪ The CVE network supports connection between the school zone system and the Message

Handler

▪ The school zone system sends school zone sign data to the Message Handler

▪ The Message Handler determines when the school zone sign is flashing/active

▪ The Message Handler updates the Start Time and Duration in the stored J2735 RSSZ TIM when the school zone sign is flashing/active

▪ The Message Handler sends J2735 RSSZ TIMs to the RSU as immediate forward messages.

▪ The RSU broadcasts RSSZ TIMs on Channel 180

3.2.2. Light-Duty Vehicle Onboard Unit

The following functionality of the Smart Columbus CVE LDV OBUs will be tested:

• The LDV OBU’s ability to broadcast Part I BSMs 10 times per second on Channel 172


▪ The OBU can generate and broadcast BSMs as defined in Society of Automotive Engineers (SAE) J2945/1

• The LDV OBU’s ability to receive and process RTCM messages on Channel 180


▪ The CVE network supports a connection between the Position Correction System and the

Message Handler





▪ The Message Handler sends J2735 RTCM messages to the RSU

▪ The RSU broadcasts J2735 RTCM messages on Channel 180

▪ LDV OBU updates position based on RTCM

• The LDV OBU’s ability to provide EEBL, FCW, LCW, and IMA warnings to drivers


▪ Remote OBUs are broadcasting BSMs

▪ The host LDV OBU receives and processes BSMs from remote vehicles

▪ The host LDV OBU’s EEBL, FCW, LCW, and IMA OBU applications are functioning as intended

▪ The host LDV OBU’s HMI is functioning as intended

• The LDV OBU’s ability to provide RLVW warnings to drivers




▪ The Message Handler sends J2735 SPaT messages and MAP messages to the RSU

▪ The RSU broadcasts SPaT and MAP messages on Channel 180

▪ The LDV OBU RLVW application is functioning as intended

▪ The LDV OBU’s HMI is functioning as intended

• The LDV OBU’s ability to provide RSSZ warnings to drivers


▪ The CVE network supports connection between the school zone system and the Message

Handler



▪ The Message Handler encodes a J2735 RSSZ message when the school zone sign is flashing/active

▪ The Message Handler sends J2735 RSSZ messages to the RSU while the school zone sign is

flashing/active

▪ The RSU broadcasts RSSZ messages on Channel 180

▪ The OBU RSSZ application is functioning as intended

▪ The LDV OBU’s HMI is functioning as intended

• The LDV OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an RSU


▪ The RSU is broadcasting WSAs containing WRAs for IPv6 services on Channel 180



▪ The LDV OBU receives and processes WSAs containing WRAs

▪ The LDV OBU determines it needs additional certificates

▪ The LDV OBU sends a certificate request to the ODOT SCMS over IPv6 through the RSU

▪ The RSU acts as an IPv6 gateway

▪ The CVE network supports IPv6 connectivity to the ODOT SCMS

▪ The LDV OBU receives and processes new certificates from the SCMS

• The LDV OBU’s ability to request and receive firmware updates over the air (OTA) from the OBU manufacturer through an RSU over IPv6 on Channel 180



▪ The LDV OBU receives and processes WSAs containing WRAs


▪ The CVE network supports IPv6 connectivity to the OBU manufacturer

▪ The LDV OBU determines the latest firmware version available from the OBU manufacturer

▪ The LDV OBU determines it needs a firmware update

▪ The LDV OBU downloads the latest firmware from the OBU manufacturer

▪ The LDV OBU updates its firmware

3.2.3. Heavy-Duty Vehicle Onboard Unit

The following functionality of the Smart Columbus CVE HDV OBUs will be tested:

• The HDV OBU’s ability to broadcast Part I BSMs 10 times per second on Channel 172


▪ The OBU can generate and broadcast BSMs as defined in SAE J2945/1

• The HDV OBU’s ability to receive and process RTCM messages on Channel 180


▪ The CVE network supports a connection between the Position Correction System and the Message Handler





▪ The HDV OBU updates position based on RTCM messages

• The HDV OBU’s ability to generate and broadcast J2735 SRMs on Channel 180


▪ The Signal Controller is sending SPaT data to the Message Handler

▪ The Message Handler is encoding SPaT data into J2735 SPaT messages



▪ The Message Handler is sending J2735 SPaT messages and MAP messages to the RSU

▪ The RSU is broadcasting WSAs, advertising signal priority/preemption, SPaT, and MAP

messages on Channel 180

▪ The HDV OBU signal priority request application detects priority is available from a WSA

▪ The HDV OBU signal priority request application detects when the vehicle is in an approach where priority is available

▪ The HDV OBU generates and broadcasts SRMs requesting priority on Channel 180

• The HDV OBU’s ability to receive and log J2735 SSMs from Channel 180


▪ The HDV OBU is broadcasting J2735 SRMs on Channel 180

▪ The RSU is forwarding SRMs to the Message Handler

▪ The Message Handler is sending signal priority requests to the Signal Controller

▪ The Signal Controller is processing signal priority requests

▪ The Signal Controller is sending signal priority status data to the Message Handler

▪ The Message Handler is encoding signal priority status data into J2735 SSMs

▪ The Message Handler is sending J2735 SSMs to the RSU

▪ The RSU is broadcasting SSMs on Channel 180

▪ The HDV signal priority request application is functioning as intended

• The HDV OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an RSU



▪ The HDV OBU receives and processes WSAs containing WRAs

▪ The HDV OBU determines it needs additional certificates

▪ The HDV OBU sends a certificate request to the ODOT SCMS over IPv6 through the RSU



▪ The HDV OBU receives and processes new certificates from the SCMS

• The HDV OBU’s ability to request and receive firmware updates OTA from the OBU manufacturer through an RSU over IPv6 on Channel 180



▪ The HDV OBU receives and processes WSAs containing WRAs



▪ The HDV OBU determines the latest firmware version available from the OBU manufacturer



▪ The HDV OBU determines it needs a firmware update

▪ The HDV OBU downloads the latest firmware from the OBU manufacturer

▪ The HDV OBU updates its firmware

3.2.4. Emergency Vehicle Onboard Unit

The following functionality of the Smart Columbus CVE EV OBU will be tested:

• The EV OBU’s ability to broadcast Part I BSMs10 times per second on Channel 172



• The EV OBU’s ability to receive and process RTCM messages on Channel 180


▪ The CVE network supports a connection between the Position Correction System and the

Message Handler





▪ The EV OBU updates position based on RTCM messages

• The EV OBU’s ability to generate and broadcast J2735 SRMs on Channel 180





▪ The RSU is broadcasting WSAs, advertising signal priority/preemption, SPaT, and MAP messages on Channel 180

▪ The EV OBU signal preemption request application detects preemption is available from a

WSA

▪ The EV OBU signal preemption request application detects when the vehicle is in an approach

where preemption is available

▪ The EV OBU generates and broadcasts SRMs requesting preemption on Channel 180

• The EV OBU’s ability to receive and log an SSM


▪ The EV OBU is broadcasting J2735 SRMs on Channel 180


▪ The Message Handler is sending signal preemption requests to the Signal Controller

▪ The Signal Controller is processing signal preemption requests



▪ The Signal Controller is sending signal preemption status data to the Message Handler

▪ The Message Handler is encoding signal preemption status data into J2735 SSMs



▪ The EV OBU signal preemption request application is functioning as intended

• The EV OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an RSU



▪ The EV OBU receives and processes WSAs containing WRAs

▪ The EV OBU determines it needs additional certificates

▪ The EV OBU sends a certificate request to the ODOT SCMS over IPv6 through the RSU



▪ The EV OBU receives and processes new certificates from the SCMS

• The EV OBU’s ability to request and receive firmware updates OTA from the OBU manufacturer through an RSU over IPv6 on Channel 180



▪ The EV OBU receives and processes WSAs containing WRAs



▪ The EV OBU determines the latest firmware version available from the OBU manufacturer

▪ The EV OBU determines it needs a firmware update

▪ The EV OBU downloads the latest firmware from the OBU manufacturer

▪ The EV OBU updates its firmware

3.2.5. Transit Vehicle Onboard Unit

The following functionality of the Smart Columbus CVE Transit vehicle OBU will be tested:

• The transit OBU’s ability to broadcast Part I BSMs 10 times per second on Channel 172



• The transit OBU’s ability to receive and process RTCMs on Channel 180


▪ The CVE network supports a connection between the Position Correction System and the Message Handler







▪ The transit OBU updates position based on RTCM messages

• The transit OBU’s ability to generate and broadcast J2735 SRMs on Channel 180





▪ The RSU is broadcasting WSAs, advertising signal priority/preemption, SPaT, and MAP messages on Channel 180

▪ The transit vehicle OBU signal priority request application detects priority is available from a

WSA

▪ The transit vehicle OBU signal priority request application detects when the vehicle is in an

approach where priority is available

▪ The transit vehicle OBU generates and broadcasts SRMs requesting priority on Channel 180

• The transit OBU’s ability to log EEBL, FCW, LCW, and IMA events


▪ The transit OBU’s EEBL, FCW, LCW, and IMA applications are functioning as intended

▪ The transit OBU message logging process is functioning as intended

• The transit OBU’s ability to log RLVW events




▪ The Message Handler sends J2735 SPaT messages and MAP messages to the RSU

▪ The RSU broadcasts SPaT and MAP messages on Channel 180

▪ The transit OBU’s RLVW application is functioning as intended


• The transit OBU’s ability to log RSSZ events


▪ The CVE Network supports connection between the school zone system and the Message

Handler



▪ The Message Handler encodes a J2735 RSSZ message when the school zone sign is flashing/active



▪ The Message Handler sends J2735 RSSZ messages to the RSU while the school zone sign is

flashing/active

▪ The RSU broadcasts RSSZ messages on Channel 180

▪ The transit OBU’s RSSZ application is functioning as intended


• The transit OBU’s ability to receive and log an SSM


▪ The transit OBU is broadcasting J2735 SRMs on Channel 180


▪ The Message Handler is sending signal priority requests to the Signal Controller

▪ The Signal Controller is processing signal priority requests

▪ The Signal Controller is sending signal priority status data to the Message Handler

▪ The Message Handler is encoding signal priority status data into J2735 SSMs



▪ The transit OBU signal priority request application is functioning as intended

• The transit OBU’s ability to send log files to the TrCVMS


▪ The transit OBU logs events

▪ The transit OBU is connected to the transit vehicle’s internal communication network

▪ The transit OBU determines when the transit vehicle is in communication range of the transit

vehicle garage network

▪ The TrCVMS system is connected to the transit vehicle garage communication network

▪ The transit OBU offloads log files to the TrCVMS

• Transit OBU’s ability to request and receive IEEE 1609.2 certificates from the ODOT SCMS through an RSU



▪ The transit OBU receives and processes WSAs containing WRAs

▪ The transit OBU determines it needs additional certificates

▪ The transit OBU sends a certificate request to the ODOT SCMS over IPv6 through the RSU



▪ The Transit OBU receives and processes new certificates from the SCMS

• The transit vehicle OBU’s ability to request and receive firmware updates OTA from the OBU

manufacturer through an RSU over IPv6 on Channel 180





▪ The transit vehicle OBU receives and processes WSAs containing WRAs



▪ The transit vehicle OBU determines the latest firmware version available from the OBU

manufacturer

▪ The transit vehicle OBU determines it needs a firmware update

▪ The transit vehicle OBU downloads the latest firmware from the OBU manufacturer

▪ The transit vehicle OBU updates its firmware

3.2.6. Traffic Connected Vehicle Management System

The following functionality of the Smart Columbus CVE TCVMS will be tested:

• The traffic management staff’s ability to send MAP messages to the Message Handler at each

applicable roadside location using the TCVMS


▪ The MAP generation tool user interface is functioning as intended

▪ The MAP generation process is functioning as intended

▪ The CVE supports connectivity between the TCVMS and the Message Handler at each applicable roadside location

• The traffic management staff’s ability to send RSSZ messages to the Message Handler at each

applicable roadside location using the TCVMS


▪ The RSSZ generation tool user interface is functioning as intended

▪ The RSSZ generation process is functioning as intended


• The TCVMS’ ability to log SPaT, RTCM, SSM, SRM, and BSM data received from Message

Handlers at each applicable roadside location



▪ RSUs forward BSMs and SSMs to the Message Handler

▪ Signal controllers send SPaT data and signal priority/preemption data to Message Handlers

▪ Message Handlers generate SPaT and SSM messages

▪ Message Handlers send SPaT, RTCM, SSM, SRM, and BSM messages to the TCVMS

▪ The TCVMS logging process is functioning as intended



• The TCVMS’ ability to send SPaT, MAP, RSSZ, RTCM, SSM, SRM, and BSM data to the Operating System


▪ The CVE supports connectivity between the TCVMS and the Operating System

▪ The TCVMS logging process is functioning as intended

▪ The TCVMS distribution process is functioning as intended

• The TCVMS’ ability to manage RSUs


▪ The CVE supports connectivity between the TCVMS and the Message Handler at each

applicable roadside location

▪ The TCVMS sends RSU command and control messages to Message Handlers at each applicable roadside location

▪ Message Handlers send command and control messages to RSUs

▪ RSUs send command and control responses to Message Handlers

▪ Message Handlers send RSU command and control responses to the TCVMS

3.2.7. Transit Connected Vehicle Management System

The following functionality of the Smart Columbus CVE TrCVMS will be tested:

• The TrCVMS’ ability to receive and archive log files from transit vehicle OBUs


▪ The TrCVMS system is connected to the transit vehicle garage communication network

▪ Transit OBUs log events

▪ Transit OBUs are connected to the transit vehicle internal communication network

▪ Transit OBUs determine when transit vehicles are in communication range of the transit vehicle garage network

▪ Transit vehicles successfully send log files to the TrCVMS

▪ The TrCVMS decodes and stores data contained in transit vehicle OBU log files

3.3. FUNCTIONALITIES NOT TO BE TESTED

This section contains a high-level description of the functionalities of the Smart Columbus CVE elements

that will not be tested as part of this test plan.

3.3.1. Heavy-Duty Vehicle Onboard Unit

Based on the lack of clear definition and guidance from SAE and the cost and schedule associated with

developing and testing a reasonable solution, BSM Part II will not be tested for HDVs.



3.3.2. Data Distribution to Third Parties

Testing of the Operating System will be described in the Operating System test plan. The CVE elements providing CV data to the Operating System is described in this CVE test plan; however, the distribution of

CV data to third parties by the Operating System is outside of the scope of the plan.


Approach

This section describes how the City of Columbus will test the CVE.

4.1. TESTING APPROACH

The following four methods will be used to verify the operation and functionality of individual components

and subsystems, as well as the overall Smart Columbus CVE:

• Inspection – A nondestructive examination of the device or system using one or more of the senses (visual, auditory, somatosensory). This may include simple physical handling and measurements.

• Demonstration – A trial run of the device or system as it is intended to operate or function to verify that results are as expected.

• Testing – Verification of the device or system using a controlled and predefined series of inputs,

data, or stimuli to ensure the system will produce a predefined output as specified by the requirements.

• Analysis – Verification of the device or system using models, calculations, and testing equipment.

Analysis allows someone to make predictive statements about the typical performance of the device, application, or system based on the confirmed test results of a sample set or by combining the

outcome of individual tests to conclude something new about the device, application, or system. Analysis is often used to predict the breaking point or failure of a device, application, or system by

using nondestructive tests to extrapolate the failure point.

Review of documentation including, but not limited to, Test Reports from independent labs/facilities will also be utilized to confirm a device meets certain requirements. For Example, OmniAir certification and Test

Reports will be utilized to confirm DSRC devices meeting 802.11 to IEEE 1609 standards.

4.2. TEST READINESS REVIEWS

The City of Columbus will meet with the OBU and RSU vendors independently every other week. During

each meeting the vendor will report on the status of their hardware and applications, as appropriate, and the City will report on the status of the infrastructure (backhaul, Signal Controllers, etc.). These meetings will

serve as Test Readiness Reviews (TRR) to determine if the hardware, applications, and infrastructure are ready to proceed into formal testing. Testing will not begin unless the vendor and the City agree that

applicable components are available and ready for testing.

4.3. TEST PROCESS

Each CVE device needs to be configured and tested prior to deployment to ensure proper operation within the system. Configurations need to be tested prior to deployment, because once a device is deployed,

access to it will be limited; a bucket truck will be required if an RSU cannot be accessed remotely, and OBUs will not be accessible after they are installed in a vehicle.

Device functionality and component integration need to be verified before the system is deployed as well as

af ter equipment is installed at roadside locations and in vehicles. Testing will be completed in the following phases:

Chapter 4. Approach


• Local Device Assembly Test (LDAT) – LDAT testing exercises the functionality and operation of each standalone device. Devices will be connected to appropriate backhaul for connectivity to the

TCVMS, SCMS, and other cloud services as applicable. An example of a CVE LDAT is verifying the ability of an RSU to broadcast WSAs, be managed through the TCVMS, and request and receive

certificates from the SCMS.

• Subsystem Test (SST) – SST serves as integration testing, exercising the functionality and operation of each subsystem. Devices will be connected to form representative subsystems and be

connected to appropriate backhaul for connectivity to the TCVMS, SCMS, and other cloud services as applicable. An example of CVE SST is verifying the ability of an RSU to broadcast SPaT, MAP,

TIMs, SSM, and WSAs, as well as be managed through the TCVMS and request and receive certificates from the SCMS.

• Final System Test (FST): FST is the final stage of functional and operational testing and serves as

the basis for system acceptance. FST testing exercises the functionality and operation of the entire CVE on a corridor-by-corridor basis. Subsystems are connected to appropriate backhaul for

connectivity to the TCVMS, SCMS, and other cloud services as applicable. The FST verifies the ability of an OBU to receive SPaT, MAP, TIMs, SSM, and WSAs, as applicable, along each of the

CVE corridors. RSUs are managed through the TCVMS and RSUs and OBUs request and receive certificates from the SCMS, as applicable.

• Burn-in – Burn-in consists of CVE infrastructure equipment running in “normal” operation mode for a

predetermined amount of time. For the CVE, a 30-day burn-in period will begin after the FST. Burn-in demonstrates equipment and system reliability and enables the operation and maintenance

processes to be refined prior to installing the full complement of roadside equipment.

• Operational Period– The operational period begins after system acceptance and continues until the end of the project. During the operational period, the CVE is under operations and maintenance, with vendors on-call to support as needed and CV data is collected, archived and processed as needed.

Initial configuration parameters will be recorded in the Device Configuration document for each CVE device type.

The following subsections describe the test stages for the Smart Columbus CVE.

4.3.1. First Article Testing

To ensure device functionality and identify configuration parameters before installation, devices will be tested in controlled environments. These tests will be conducted on “First Article” devices, which are the first

devices supplied by vendors that meet the Smart Columbus CVE requirements. Testers should refer to the CVE System Requirements (SyRS) and Interface Control Document (ICD) for expected device operation.4

First Article testing may be completed in one or more phases, depending on the availability of hardware, services, or OBU applications.

4.3.1.1. BENCH TESTING

Bench Testing will involve setting up relevant devices in an indoor controlled lab-type environment in which devices can be configured and tested to aid in the development of configuration and installation

documentation before installation at the roadside or in vehicles. During bench testing, RSUs will be connected to signal controllers, the SCMS, and the TCVMS to ensure all configuration parameters are set

4 These and other Smart Columbus documents are available at https://smart.columbus.gov/projects/connected-

vehicle-environment.

Chapter 4. Approach


appropriately to enable operation. OBUs will also be configured to broadcast BSMs and test connectivity to

the SCMS through an RSU. The Traffic Management Center (TMC) will be the location for bench testing. An initial round of LDAT and SST will be completed during bench testing. After the SST is completed, an initial

10-day burn-in period will begin.

4.3.1.2. DEMONSTRATION AREA TESTING

Demonstration Area Testing will involve setting up relevant devices in an outdoor controlled environment in

which vehicles can be driven at appropriate speeds and perform appropriate maneuvers to test applications. SST will be completed during demonstration area testing, as this will be closer to a field deployment setup

and configuration than bench testing. Test intersections will be created by generating MAP messages with lanes and approaches defined within the area, along with a Traffic Signal Controller generating SPaT data.

Similarly, a test TIM will be generated with lanes and approaches defined within the area to represent a school zone. These SPaT, MAP, and TIM messages will support SPaT, MAP, RSSZ, SRM, and SSM testing.

4.3.1.3. FIELD TESTING

After demonstration area testing is completed, First Article devices will be installed at up to six locations along Morse Road and in up to five test vehicles for field testing. SST will be completed after First Article

device installation. Once SST is completed, FST will begin on the First Article installations.

4.3.2. Channel 180 Congestion Testing

Under normal conditions, a single RSU broadcast messages on 3 separate DSRC channels; 172 (SPaT and

MAP), 178 (WSA and TIM), and another Service Channel (SSM and IPv6 Services). Giving the FCC’s spectrum re-allocation NPRM, RSUs would only be able to broadcast on Channel 180.

Additional testing is required to verify Smart Columbus OBU applications and over-the-air updates operate

as intended under the channel congestion caused by RSUs broadcasting only on Channel 180.

RSUs will be tested to verify they can broadcast SPaT, MAP, TIM, SSM, and WSAs on Channel 180 are the

defined rates.

OBU applications and firmware updates will be tested to verify they operate as intended with SPaT, MAP, TIM, SRM, SSM, and WSAs broadcasting on Channel 180.

4.3.3. Burn-In

After the successful completion of First Article Channel 180 Congestion Testing, the First Article devices will be subjected to a 30-day burn-in period to ensure they can operate for an extended period. During burn-in,

RSUs will operate 24/7, broadcasting applicable messages (e.g., SPaT, MAP, TIM, WSA, etc.) and be managed by the TCVMS. Vehicles will be selected for OBU installation based on driver travel habits,

including driving through the corridor daily or weekly. Vehicles with OBUs will also be driven through the corridor based on a predetermined schedule as needed, to collect data and operate OBU applications.

Major and minor failures that occur during the burn-in period will be recorded in maintenance logs with the

date, time, and location of the failure, along with the corrective action(s) taken. These logs will be available for inspection by Smart Columbus staff during the burn-in period and provided to the CVE system owner

and test manager, hereafter referred to as the Management Team, at the conclusion of the burn-in period.

A minor failure is defined as a failure that affects the operation of a device but not the operation of the overall system. During resolution of a minor failure, the 30-day burn-in will be suspended. After the failure is

resolved, the burn-in period will resume. Frequent occurrences of minor failures, however, may indicate a system flaw. If a system flaw is identified, the 30-day burn-in will be reset, after the flaw has been corrected.

Chapter 4. Approach


A major failure is defined as a failure that affects the entire system, requires more than 48 hours to resolve,

or results in less than 95% of CVE roadside and onboard equipment under test being operational at any given time. All tested CVE equipment will be monitored during burn-in testing to ensure this operational

condition is met. When a major failure occurs, the Management Team will be notified, and a corrective action plan developed within 48 hours of detection or identification. After the major failure has been

resolved, a failure report containing the nature of the failure, the features or functionality affected by the failure, and the corrective actions taken will be provided to the Management Team. The 30-day burn-in

period will start over at day 1 after the resolution of a major failure.

4.3.4. Post-Installation Testing

After the 30-day burn-in period, installation of the remaining infrastructure locations and vehicles will begin

en masse.

Post-Installation testing will consist of verifying the functionality and operation of all devices immediately following installation in the field, at infrastructure locations, and in vehicles.

For infrastructure locations, the RSU, Signal Controller, Message Handler, and so forth will be tested to

ensure proper integration, operation, and connectivity to the local (intersection) network as well as connectivity to the TCVMS and SCMS.

For vehicle installations, the OBU will be tested to ensure the device is broadcasting BSMs at the

appropriate rate (e.g., 10 times per second), can receive messages from an RSU, and can connect to the SCMS through an RSU.

SST is completed during Post-Installation testing.

Major and minor failures that occur during post-installation testing will be recorded in test result logs (TRLs)

with the date, time, and location of the failure, along with the corrective action(s) taken. These logs will be provided to the Smart Columbus Management Team at the end of each day during post-installation testing.

4.3.5. Final System Testing

FST will take place for each corridor in the deployment area. The intent of corridor FST is to ensure

applications are operational throughout the entire length of the corridor and that vehicles driving the corridor receive the appropriate notification or warning. Corridor FST will begin after post-installation SST is

completed for each infrastructure location along the corridor.

Major and minor failures that occur during FST will be recorded in TRLs with the date, time, and location of the failure, along with the corrective action(s) taken. These logs will be provided to the Smart Columbus

Management Team at the end of each day during FST.

4.3.6. Regression Testing

If major or minor failures occur during FST, the effected portions of the system will undergo Regression

Testing to ensure the failure resolution(s) do not impact other system functionality or operations.

Once final system testing is completed successfully and the system has been accepted, final configuration parameters will be recorded in the device configuration document for each CVE device type and the CVE

operational period will begin.

Chapter 4. Approach


4.3.7. Operational Period

During the Operational Period, CVE device health and status parameters will be monitored to ensure infrastructure devices are operating properly such that the overall system is broadcasting messages and

collecting data as intended. Device faults will be investigated and resolved as they occur to keep system availability high to maximize data dissemination and collection. More details related to the operation and

maintenance of the CVE infrastructure can be found in the CVE Operations and Maintenance Plan.

4.4. TESTING SCHEDULE

Table 4 indicates the expected timeframe during which each testing phase will occur for the Columbus

metropolitan area.

Table 4: Metropolitan Area Test Phases and Time Periods

Test Phase Time Period

First Article Testing

December 1, 2019 - April 3, 2020

Bench Testing (LDAT and SST)

Demonstration Area Testing (SST)

Field Testing (SST and FST)

Burn-In

Remaining Deployment Test

April 6 - June 5 RSU (SST)

OBU (SST)

Post-Installation Testing

June 8 - June 26 RSU (SST)

OBU (SST)

Final System Testing (FST) July 6 - July 24 (Holiday)

Regression July 27 - August 7

Go Live (begin Operation Period) August 10, 2020


Fiber must be installed and tested as part of the City CVE IT network along Alum Creek. This will delay the testing of the Alum Creek area slightly. Table 5 indicates the expected timeframe during which each testing

phase will occur for the Alum Creek area.

Table 5: Alum Creek Area Test Phases and Time Periods

Test Phase Time Period

Remaining Deployment Test

April 6 - July 3 RSU (SST)

OBU (SST)

Post-Installation Testing July 13 - July 24 (Holiday)

RSU (SST)

Chapter 4. Approach


OBU (SST)

Final System Testing (FST) July 27 - August 7

Regression August 10 - August 14

Go Live (begin Operation Period) August 17, 2020


4.5. ROLES

Table 6 contains the anticipated test roles and their responsibility for testing CVE devices before and after installation.

Table 6: Test Roles and Responsibilities

Title Description Responsibility

CVE project manager City of Columbus representative

overseeing the completion of the CVE project

Ryan Bollo

(Smart City Program Office)

Test facility owner Represents the physical location where components will be deployed to verify compliance prior to installation

City of Columbus TMC staff

OBU installation facility owner

Represents the physical location where OBUs are installed in vehicles

Columbus Police (Precincts 1, 3, 4, 5, 6, 7, 17,18)

Columbus Fire (Stations 7, 13, 16, 18, 19, 24)

Central Ohio Transit Authority (COTA) (McKinley, Fields)

Department of Public Safety (25th Ave, Morse, Groves)

Franklin County Engineer's Office (East Maintenance)

Clinton Township Fire (61)

Mifflin Township Fire (132)

2 Brothers Automotive: LDV OBU installers

Procat Towing: LDV OBU installers

SoundChamber Car Audio: LDV OBU installers

RSU installation location owner

Represents the roadside locations in which RSU and supporting equipment are installed

City of Columbus

ODOT

Franklin County

City of Obetz

Chapter 4. Approach



Test manager Supervises and controls all tests, reviews and approves test procedures, has the authority to direct all test activities, and is responsible for communicating test status to all stakeholders

Notifies the City and other key stakeholders of the test schedule at least one week in advance of the scheduled start

May not necessarily be present during all test activities but will communicate regularly with the test conductor for status updates

Signs off on all completed tests based on functionality demonstration

WSP

Test conductor Responsible for running the daily test

activities and demonstrating test results to test manager

Kapsch

Danlaw

Siemens

Traf fic data consumer Authorized individual (or service) that consumes CV data from OBUs and RSUs

City of Columbus TCVMS staff

Transit data consumer Authorized individual (or service) that

consumes CV data from transit vehicles COTA TrCVMS staff

Data producer A system or systems that produce data to be consumed

Kapsch

Danlaw

Siemens

Traf fic management staff

The traffic management staff use the TCVMS to collect data from the CVE

City of Columbus

Transit management staff

The transit management staff use the TrCVMS to collect data from Transit OBUs

COTA

Stakeholders General testing role with domain knowledge in transportation

Jeff Kupko (Safety Manager)

Data recorder Responsible for recording the outputs and overall results of each test

Present for RSU bench testing and rides in a vehicle for OBU application testing

Provides all observations to test manager at the end of the test day

WSP

Chapter 4. Approach



Vehicle drivers Operate vehicles during each applicable test/demonstration. Follow the explicit instructions in the test script and stay in constant communication with the test conductor to receive further instructions and inform the test conductor of any incidents or exceptions they experience during a test run. Have a skill level appropriate for testing and demonstrating V2V applications

Siemens

Test observers Witness test runs at the City's discretion and may be present during bench testing or ride in vehicles during OBU application testing

At the City’s discretion


Some roles can be combined; one person can assume up to two roles (e.g., the test manager can also be

the data recorder).

4.6. TEST TOOLS

A key aspect of system testing is the use of both hardware and software tools that enable development,

tracking, and traceability throughout the process. Operational scenarios from the system ConOps as well as requirements and design elements specified in the CVE System Design Document (SDD) form the basis of

test scenarios described in Chapter 7. These test scenarios make up the acceptance criteria for the operational readiness of each RSU location and OBU-equipped vehicle.

This section describes the tools that will be used to assess individual components and subsystems, as well

as the overall CVE system.

4.6.1. Connected Vehicle Test Tool

A Connected Vehicle Test Tool (CVTT) (a specialized Kapsch OBU) will be used to visualize SPaT, MAP, and TIM messages and to verify WSA, BSM, and RTCM message content. The CVTT stores received

DSRC messages in packet capture (pcap) files for both real-time and post-processing analysis. DSRC pcap files contain a packet for each received message. Packets include the IEEE 802.11 message header, along

with IEEE 1609.2 (security) header and tail, 1609.3 header, and the SAE J2735 message payload. Analysis of pcap files are used to verify the proper operation of RSUs and OBUs. Kapsch will self-certify the CVTT

prior to delivery.

4.6.2. Wireshark

Wireshark will be used to decode pcap files collected by the CVTT both in real-time and for post-processing.

Using Wireshark, the Data Recorder will verify a) an OBU or RSU is broadcasting the correct message(s) and b) SPaT, MAP, TIM, SRM, SSM, and WSA messages include the correct content; the data contained

within the message.

Chapter 4. Approach


4.6.3. Onboard Unit

The OBU will be used to demonstrate vehicle-based applications and to verify proper operation of the applications, as well as to capture CV messages for troubleshooting.

4.6.4. Test Result Log

The CVE project team will rely on tools based on Microsoft Excel to manage the test cases, scenarios, and test results. A TRL will be used to document the test case ID, test objective, number of test runs, and the test

results for each test. The test manager will be required to enter the run dates, name, result, and comments, where applicable. Appendix A.1 includes a sample TRL.

4.6.5. Defect Tracker

The defect tracker will be used to record anomalies detected during the execution of a test case or scenario.

An observation is considered a defect when the result of an activity does not match the expected outcome outlined in the test procedure.

Test conductors will be encouraged to provide as much information as possible when recording defects, so

vendors and integrators can repeat the issue, identify its root cause, and resolve it. Information to provide includes inputs, conditions, steps at which failure occurred, the appropriate test identifier(s), expected

results, actual results, and defect frequency (e.g., every time, intermittent).

The severity of each defect will be assessed and assigned to a category as follows:

• Critical – The most serious classification, meaning the feature or product is unusable. Defects of this severity should be brought to the immediate attention of the Management Team for further inspection,

coordination, and decision-making.

• High – This classification indicates that the feature or product varies significantly from functional or technical specifications. Defects of this severity should be brought to the attention of the Management Team for further inspection, coordination, and decision-making.

• Medium – This classification indicates that the feature or product is operational according to

functional or technical specifications but is not capable of meeting performance expectations.

• Low – The least-serious classification, this indicates the observation is cosmetic.

The test manager will monitor the defect log for corrective action. The test conductor is responsible for understanding and reproducing (where possible) the defect, summarizing a response and the activities

taken to resolve the issue, and capturing metadata associated with the resolution (e.g., assigned name, date, status, description). If a conflict arises between a design element that ties to a requirement and the

deployed amenity, the test manager will coordinate with the appropriate vendor and the system owner to determine whether a change to the design and/or requirement is appropriate. The City of Columbus project

manager (who is also the system owner) will be responsible for reviewing and approving all requests to make changes that impact the system design and requirements. See Section 4.6.6 for more information

about the change request log.

The defect tracker will also be leveraged to measure the feasibility and readiness of the subcomponents to be promoted to production. See Appendix A.2 for a sample defect tracker.

4.6.6. Change Request Log

The ability to track system design changes or changes to requirements associated with a feature is a fundamental strategy for configuration management and an important aspect of managing projects and

Chapter 4. Approach


maintaining traceability across the Smart Columbus program. A change request log provides the test

conductor with a mechanism to capture and justify requests for change, which often derive from a defect or an enhancement request. The City of Columbus project manager is responsible for assessing the impact of

the change as it relates to the project objectives, schedule, cost, and so forth, and for providing the final approval of all change requests. The Change Request Log in Appendix A.3 provides a form for recording

all the change requests logged throughout the testing process, along with justifications and authorization status.

4.7. ENVIRONMENTAL NEEDS

This section describes the general test environments required to conduct all First Article tests. During testing, all items within the test environment including hardware, software, and configuration elements under

test will be managed by the test team.

4.7.1. Bench Facility

Bench tests involve conducting tests in a controlled environment to assess the initial functionality of the

Smart Columbus CVE OBUs, RSUs, and supporting equipment. Subsections below list general facility and equipment requirements.

4.7.1.1. GENERAL FACILITIES REQUIREMENTS

The facilities required to conduct the bench tests include:

• Workbench space to support four independent test teams

• Surge protection power strips

• A conference room large enough to accommodate a minimum of 12 people; should contain a whiteboard or similar equipment

• Internet access for up to 12 people

• Access to the workbench and conference room space from 8 a.m. to 4:30 p.m., Monday through Friday

• Access to windows to provide access to open sky (for Global Positioning System (GPS))

4.7.1.2. GENERAL EQUIPMENT REQUIREMENTS

The equipment required to conduct the bench tests includes:

• Six RSUs, two each from Kapsch, Danlaw, and Siemens (device suppliers will provide antennas, power, and other cabling and components required to operate the RSUs)

• Up to four OBUs configurable as LDV, HDV, EV, and transit, as needed (device suppliers will provide antennas, power, and other cabling and components required to operate the OBUs)

• Four laptops (one per device under test; provided by suppliers)

• Hand tools (e.g., pliers, screw drivers, socket sets)

• Power or battery tools (e.g., drill, screwdriver, flashlight)

• 110V AC-to-12V DC power supplies

• Assortment of CAT5 Ethernet patch cables (for connecting laptops to devices being tested)

Chapter 4. Approach


• Spectrum Analyzer capable of supporting frequencies up to and including 6 GHz

• DSRC protocol analyzer capable of decoding and logging WSA, BSM, SPaT, MAP, RTCM, SRM, SSM, and TIM (see Sections 4.6.1 and 4.6.2).

• Other relevant software tools used during device certification testing.

4.7.2. Demonstration Area Facility

The demonstration area will be used to conduct additional tests that cannot be conducted on the bench, such as the V2V applications. OBUs will be installed in vehicles, and roadside equipment will be installed at

locations along the drive route.

4.7.2.1. GENERAL FACILITIES REQUIREMENTS

The demonstration area should include:

• Access to power for up to four RSUs, four signal controllers, and four other devices

• At least two vehicles (provided by OBU suppliers and Smart Columbus as required)

• Space to set up four independent RSU test stations

• Space to drive vehicles to test V2V applications

• Space to accommodate a virtual intersection for SPaT, MAP, and SRM testing

• Space to accommodate a virtual school zone for RSS testing

• Parking lot space accommodating up to two LDVs simultaneously

• A conference room large enough to accommodate a minimum of 12 people

• Internet access for up to 12 people

• Access to the facility from 8 a.m. to 4:30 p.m., Monday through Friday

4.7.2.2. GENERAL EQUIPMENT REQUIREMENTS

Equipment required to conduct the field tests includes:

• Six RSUs, two each from Kapsch, Danlaw, and Siemens (device suppliers will provide antennas,

power, and other cabling and components required to operate the RSUs)

• Up to four OBUs configurable as LDV, HDV, EV, and transit, as needed (device suppliers will provide antennas, power, and other cabling and components required to operate the OBUs)

• Four laptops (one per device under test – provided by the suppliers)

• Assortment of CAT5 Ethernet patch cables (for connecting laptops to devices being tested)

• Spectrum analyzer capable of supporting frequencies up to and including 6 GHz

• DSRC protocol analyzer capable of decoding and logging WSA, BSM, SPaT, MAP, RTCM, SRM, SSM, and TIM

• Other relevant tools used during device certification testing

Chapter 4. Approach


4.7.2.3. VEHICLES

The OBU vendor(s) will provide vehicles and vehicle drivers to conduct V2V application testing and demonstration. Drivers must be trained in vehicle maneuvers required to test applications in a safe and

reasonable manor that protects against property damage and personal injury. The City of Columbus is not responsible if any property damage or personal injury occurs. At a minimum, the following are required to

demonstrate V2V applications:

• Two LDVs (LDV, transit, EV, and HDV OBUs will be tested in these vehicles)

• One OBU installed in each vehicle

• One driver for each vehicle

• One data recorder for each vehicle (provided by the City)

It is assumed the Smart Columbus Program Management Office will provide the facilities required to

conduct both the First Article and demonstration area tests. The test team will provide the core test team members, coordinate with various stakeholder personnel to attend relevant tests, conduct all tests, and

provide test results to Smart Columbus for review.

4.7.3. Test Preparation

Before tests are conducted, the test environment will be inspected to ensure all hardware and software

required to execute each test are readily available.

For First Article testing, a site survey will be conducted of the applicable test facility(ies) to verify tools and space are available, as well as to verify connectivity to the SCMS, the TCVMS, and the RTCM source prior

to setting up devices. A spectrum analysis may also be conducted to verify the test facility is free of interfering radio frequency transmissions in the 5.9 GHz band and neighboring bands, as interference can

adversely affect the operations of DSRC devices.

For demonstration area testing, a site survey will be conducted to verify space is available, as well as to verify connectivity to the SCMS and the TCVMS prior to setting up the RSUs and signal controllers. A

spectrum analysis may be conducted to verify the demonstration area is free of interfering radio frequency transmissions in the 5.9 GHz band and neighboring bands, as interference can adversely affect the

operations of the DSRC devices.

4.8. MEASURES AND METRICS

The City of Columbus will use Microsoft Excel to capture anomalies, incongruences, errors, or any other output inconsistent with the expected test case result. These tools will capture the following testing metrics:

• Total number of:

Test cases

Testers

• Number of test runs per case

• Number and percentage of:

Test cases passed

Test cases failed

Test cases deferred

Defects found (relative to total cases)

Chapter 4. Approach


High severity defects

Defects accepted

Defects rejected

Defects deferred

A high-level test summary report will be developed to track the overall number of test cases executed, as

well as the number that have passed and failed. A high-level defect matrix will be developed to track the number of defects opened, closed, canceled, resolved, and deferred.

The City of Columbus will analyze these data points to determine the feasibility and operational readiness of

the CVE to receive final acceptance test approval. See Appendix A.4 for a sample test summary report and defect matrix.

4.9. TEST CRITERIA

4.9.1. Item Pass/Fail

Each test case consists of several unique properties, all of which should be considered holistically during

the testing evaluation process. Properties include, but are not limited to, test identifier (ID), test objective, expected outcome, number of test runs that must be completed, and status. Test cases will be categorized

as follows:

• Planned – The test case has been defined, the responsible party (role) has been identified, testers have been assigned, and the case is ready for testing.

• In progress – The test case is underway but has not been completed.

• Pass – A pass value indicates tests have completed the defined number of runs by various testers

without error and the expected result has been achieved and logged in the TRL (see Appendix A.1 for a sample TRL). It is expected that each time this test is performed, independent of who is testing,

the same successful results will be achieved. There may be instances when a tester identifies a defect during the procedure, yet the test case still achieves the stated outcome. The case can still

pass, but the testers must log the defect and bring it to the attention of the test manager. This can happen when bugs are minor, not critical to essential functionality of the feature being tested, such as

an image being out of alignment or a misspelling.

• Fail – A test case is marked as failed when part or all the expected outcome is not met. When a defect is detected or observed it will be assigned a defect ID, for traceability, and a description of the results will be logged for comparison after treatment. All failed test cases must include one or more

defects to capture the details surrounding the failure and to track its status.

• Deferred – A test case is marked as deferred when the case cannot be performed at the time of testing or when there is a change in requirements. Most often this will occur when a software product

is being released in increments and the functionality is not ready when it is time to test the current release. This also applies to any features the system may include for which testing will be performed

outside of the scope of this test plan. If a test is deferred, the test manager should provide a brief reason for the delay. The test manager is responsible for tracking deferred cases and evaluating the

most appropriate time and/or response for addressing the case. For a sample test case form, see Appendix A.5.

Chapter 4. Approach


4.9.2. Testing Suspension and Resumption

If a critical defect is detected that is significant enough that, if not addressed, one or more iterations of the same tests would need to be performed again, testing should be suspended until the defect is resolved, to

conserve project budget and testers’ time. The test manager should be notified immediately and will work with vendors and other appropriate stakeholders to correct the issue as quickly as possible. Testing will

resume once the test manager has confirmed the issue has been resolved. Some examples of when testing would be suspended are:

• One or more defects found associated with the RSU’s or OBU’s ability to broadcast messages

• One or more defects found associated with the OBU’s ability to provide appropriate notification or warning to the driver.

• A communication network failure occurs, rendering RSUs isolated from the TCVMS, SCMS, or other

cloud services required for normal operation


Test Deliverables

During testing, several artifacts will be used to support and enhance the testing process. The following artifacts make up part of this testing plan:

• Test cases

• Test scenarios

• Testing matrix

• Defects matrix with corrective actions

• Change request log

• Error logs, bug reports, and/or screen captures (where feasible)

• Acceptance (see Appendix A)

• CVE test report webinar


Test Cases

The test cases designed for this test plan will focus on testing the system requirements, interfaces, data, and system design for the CVE system. The scenarios will expand on these essential functions to test the system holistically. The number of testers shown in the tables below were derived by considering the type of testers that would be knowledgeable in the test objective and how important the

objective is to the overall CVE project. Metrics are based on the CVE SyRS.

6.1. ONBOARD UNIT

Table 7: Onboard Unit Test Cases

Test Case ID Test Type Function Verification Method Test Objective

CVE Test Locations Tester Role Metric Pass Criteria

CVE-OBUTC001-v01

First Article

Post-installation

BSM broadcast Demonstration Verify OBUs broadcast BSMs 10 times per second as def ined in SAE J2945/1.

Bench/vehicle installation site

Test conductor

Data producer

Data recorder

OBUs broadcast BSMs at a rate of 10/sec

Data recorder observes OBUs broadcasting 10 BSMs per second

CVE-OBUTC002-v01

First Article Transit vehicle OBU – FCW event capture trigger

Demonstration Verify a host transit vehicle OBU logs an FCW-based transit vehicle interaction event only when an FCW threat exists

Demonstration area Test conductor

Data producer

Data recorder

Transit vehicle OBU logs an FCW event, under appropriate vehicle maneuver

Data recorder observes FCW event in log file, af ter appropriate vehicle maneuver

CVE-OBUTC003-v01

First Article Transit vehicle OBU – EEBL event capture trigger

Demonstration Verify a host transit vehicle OBU logs an EEBL-based interaction event only when EEBL threat exists


Data producer

Data recorder

Transit vehicle OBU logs an EEBL event, under appropriate vehicle maneuver

Data recorder observes EEBL event in log file, af ter appropriate vehicle maneuver

Chapter 6. Test Cases




CVE-

OBUTC004-v01 First Article Transit vehicle

OBU – IMA event capture trigger

Demonstration Verify a host transit

vehicle OBU logs an IMA-based transit vehicle interaction event only when an IMA threat exists


Data producer

Data recorder

Transit vehicle OBU

logs an IMA event, under appropriate vehicle maneuver

Data recorder

observes IMA event in log file, after appropriate vehicle maneuver

CVE-OBUTC005-v01

First Article Transit vehicle OBU – LCW event capture trigger

Demonstration Verify a host transit vehicle OBU logs an LCW-based transit vehicle interaction event only when an LCW threat exists


Data producer

Data recorder

Transit vehicle OBU logs an LCW event, under appropriate vehicle maneuver

Data recorder observes LCW event in log file, af ter appropriate vehicle maneuver

CVE-OBUTC006-v01

First Article Transit vehicle OBU – RLVW event capture trigger

Demonstration Verify a host transit vehicle OBU logs an RLVW-based transit vehicle interaction event only when an RLVW threat exists


Data producer

Data recorder

Transit vehicle OBU logs an RLVW event, if the vehicle is approaching the intersection in such a way that the vehicle will run a red light

Data recorder observes RLVW event in log file, af ter appropriate vehicle maneuver





CVE-


OBU – RSSZ event capture trigger

Demonstration Verify a host transit

vehicle OBU logs an RSSZ-based transit vehicle interaction event only upon receiving a RSSZ message f rom an RSU and the host vehicle is in, or approaching, the school zone traveling above the school zone speed limit

School zone

demonstration area Test conductor

Data producer

Data recorder

Transit vehicle OBU

logs an RSSZ event, if the vehicle is traveling above the RSSZ speed limit in the RSSZ

Data recorder

observes RSSZ event in log file, af ter the vehicle travers the RSSZ traveling above the RSSZ speed limit

CVE-OBUTC008-v01

First Article Transit vehicle OBU – TSP event capture trigger

Demonstration Verify a host transit vehicle OBU logs a TSP-based transit vehicle interaction event when broadcasting SRMs


Data producer

Data recorder

Transit vehicle OBU logs an TSP event, when the OBUs requests signal priority

Data recorder observes TSP event in log file, after the OBU requests signal priority

CVE-OBUTC009-v01

First Article Transit vehicle OBU – temporary message logging

Demonstration Verify a transit vehicle OBU logs messages (BSM, SPaT, MAP, SRM, SSM, RTCM, TIM) received over a 5-minute period


Data producer

Data recorder

Transit vehicle OBU log files only contain received messages for a given period (e.g., 5 minutes)

Data recorder observes OBU log f ile only contains messages for a predetermined amount of time (e.g., the last 5 minutes of testing)





CVE-


OBU – associating logged messages with a transit vehicle interaction event

Demonstration Verify a transit

vehicle OBU event logs consists of messages received 3 seconds before the event and 3 seconds after the warning generating the event was issued


Data producer

Data recorder

Transit vehicle OBU

logs received message for 3 seconds before the event and 3 seconds after the event.

Data recorder

observes OBU log f ile contains messages for 3 seconds before an event and 3 seconds after an event

CVE-OBUTC011-v01

First Article Transit vehicle OBU – upload transit vehicle interaction event data

Demonstration Verify a transit vehicle OBU uploads transit vehicle interaction event data to the TrCVMS when arriving at the COTA garage, and deletes local transit vehicle interaction events after uploading

COTA McKinley Ave Garage

COTA Fields Ave Garage

Test conductor

Data producer

Data recorder

Transit data consumer

Transit vehicle OBU uploads OBU log f iles to TrCVMS and deletes uploaded log files from internal storage. TrCVMS archives OBU log files

Data recorder observes TrCVMS archive contains OBU log files with current date and time stamp (of test) and OBU does not contain log files

CVE-OBU0TC12-v01

First Article EV SRM broadcast Demonstration Verify the EV OBU broadcasts SRMs only when lights and sirens are active, and it is approaching an equipped intersection

Priority/preemption demonstration area

Test conductor

Data producer

Data recorder

EV OBUs broadcasts SRMs only when the vehicles lights and sirens are on

Data recorder observes SRMs are only broadcast when the EV’s lights and sirens are on

CVE-OBUTC013-v01

First Article EV SSM Receipt Demonstration Verify the EV OBU receives and logs SSMs


Test conductor

Data producer

Data recorder

EV OBU logs received SSMs

Data recorder observes OBU log f ile contains SSMs af ter an SRM is issued





CVE-


SRM broadcast Demonstration Verify the transit

vehicle OBU broadcasts SRMs only when approaching an equipped intersection

Priority/preemption

demonstration area Test conductor

Data producer

Data recorder

Transit vehicle OBU

only broadcasts SRMs when in range of an RSU broadcasting WSAs containing Provider Service Identifier (PSID) 0x20-40-96

Data recorder

observes OBU log f ile contains SRMs only when WSAs containing PSID 0x02-40-96 are received

CVE-OBUTC015-v01

First Article Transit vehicle SSM receipt

Demonstration Verify the transit vehicle OBU receives and logs SSMs


Test conductor

Data producer

Data recorder

Transit vehicle OBU logs received SSMs

Data recorder observes OBU log f ile contains SSMs af ter an SRM is broadcast

CVE-OBUTC016-v01

First Article HDV SRM broadcast

Demonstration Verify the HDV OBU broadcasts SRMs only when approaching an equipped intersection


Test conductor

Data producer

Data recorder

HDV OBU only broadcasts SRMs when in range of an RSU broadcasting WSAs containing PSID 0x20-40-96

Data recorder observes OBU log f ile contains SRMs only when WSAs containing PSID 0x02-40-96 are received

CVE-OBUTC017-v01

First Article HDV SSM Receipt Demonstration Verify the HDV OBU receives and logs SSMs


Test conductor

Data producer

Data recorder

HDV OBU logs received SSMs

Data recorder observes OBU log f ile contains SSMs af ter an SRM is broadcast

CVE-OBUTC018-v01

First Article LDV OBU certif icate top off

Demonstration Verify LDV OBU can request, download, and apply new certif icates through an RSU

Bench Test conductor

Data producer

Data recorder

LDV OBU stores new pseudonym certif icates in appropriate directory

Data recorder observes LDV OBU contains pseudonym certif icates for 3 full years





CVE-OBUTC019-v01

First Article LDV OBU f irmware, over-the-air, update

Demonstration Verify LDV OBU can request, download, and apply firmware updates through an RSU


Data producer

Data recorder

LDV OBU FW version number ref lect newest version

Data recorder observes LDV FW version number before and after update to confirm changes

CVE-OBUTC020-v01

First Article Transit OBU certif icate top off

Demonstration Verify Transit OBU can request, download, and apply new certif icates through an RSU


Data producer

Data recorder

Transit OBU stores new pseudonym certif icates in appropriate directory

Data recorder observes Transit OBU contains pseudonym certif icates for 3 full years.

CVE-OBUTC021-v01

First Article Transit OBU f irmware, over-the-air, update

Demonstration Verify Transit OBU can request, download, and apply firmware updates through an RSU


Data producer

Data recorder

Transit OBU FW version number ref lect newest version

Data recorder observes Transit FW version number before and af ter update to conf irm changes

CVE-OBUTC022-v01

First Article HDV OBU certif icate top off

Demonstration Verify HDV OBU can request, download, and apply new certif icates through an RSU


Data producer

Data recorder

HDV OBU stores new pseudonym certif icates in appropriate directory

Data recorder observes HDV OBU contains pseudonym certif icates for 3 full years.

CVE-OBUTC023-v01

First Article HDV OBU f irmware, over-the-air, update

Demonstration Verify HDV OBU can request, download, and apply firmware updates through an RSU


Data producer

Data recorder

HDV OBU FW version number ref lect newest version

Data recorder observes HDV FW version number before and after update to confirm changes

CVE-OBUTC024-v01

First Article Emergency Vehicle OBU certif icate top off

Demonstration Verify Emergency Vehicle OBU can request, download, and apply new certif icates through an RSU


Data producer

Data recorder

Emergency Vehicle OBU stores new pseudonym certif icates in appropriate directory

Data recorder observes Emergency Vehicle OBU contains pseudonym certif icates for 3 full years.





CVE-OBUTC025-v01

First Article Emergency Vehicle OBU f irmware, over-the-air, update

Demonstration Verify emergency vehicle OBU can request, download, and apply firmware updates through an RSU


Data producer

Data recorder

Emergency Vehicle OBU FW version number ref lect newest version

Data recorder observes emergency vehicle FW version number before and af ter update to conf irm changes

CVE-OBUTC026-v01

Channel 180 Congestion

1-Transit OBU (SRM) 1-RSU (SPaT, MAP, TIM, SSM, WSA)

Demonstration Verify the transit vehicle OBU receives and logs SSMs when the OBU is broadcasting SRMs and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


Field

Test conductor

Data producer

Data recorder


Data recorder observes OBU log file contains SSMs after an SRM is broadcast

CVE-OBUTC027-v01

Channel 180 Congestion

1-Transit OBU (SRM, OTA) 1-RSU (SPaT, MAP, TIM, SSM, WSA)

Demonstration Verify the transit vehicle OBU receives and logs SSMs when the OBU is broadcasting SRMs, a firmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


Test conductor

Data producer

Data recorder


Data recorder observes OBU log file contains SSMs after an SRM is broadcast




6.2. ONBOARD UNIT APPLICATION

Table 8: Onboard Unit Application Test Cases



CVE-App001-v01 First Article LDV OBU – emergency electronic brake light warning trigger

Demonstration Verify a host LDV OBU issues an EEBL warning when it receives a BSM from a remote OBU (under conditions in which an EEBL warning should be issued)

Demonstration area

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues an EEBL warning only under appropriate vehicle maneuvers

Data recorder observes EEBL warning from the LDV HMI

CVE-App002-v01 First Article LDV OBU – forward collision warning trigger

Demonstration Verify a host LDV OBU issues an FCW warning when it receives a BSM from a remote OBU (under conditions in which an FCW warning should be issued)

Demonstration area

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues an FCW warning only under appropriate vehicle maneuvers

Data recorder observes FCW warning from the LDV HMI

CVE-App003-v01 First Article LDV OBU – lane change warning trigger

Demonstration Verify a host LDV OBU issues an LCW warning when it receives a BSM from a remote OBU (under conditions in which an LCW warning should be issued)

Demonstration area

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues an LCW warning only under appropriate vehicle maneuvers

Data recorder observes LCW warning from the LDV HMI





CVE-App004-v01 First Article LDV OBU –

intersection movement assist warning trigger

Demonstration Verify a host LDV

OBU issues an IMA warning when it receives a BSM from a remote OBU (under conditions in which an IMA warning should be issued)

Demonstration

area • Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues

an IMA warning only under appropriate vehicle maneuvers

Data recorder

observes IMA warning from the LDV HMI

CVE-App005-v01 • First Article

• FST

LDV OBU – red light violation warning trigger

Demonstration Verify a host LDV OBU issues an RLVW warning when it receives SPaT and MAP from an RSU (under conditions in which an RLVW warning should be issued)

Demonstration area

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues an RLVW warning only if the vehicle is approaching the intersection in such a way that the vehicle will run a red light

Data recorder observes RLVW warning from the LDV HMI

CVE-App006-v01 • First Article

• FST

LDV OBU –

reduced speed school zone warning trigger


OBU issues an RSSZ warning when it receives a TIM f rom an RSU (under conditions in which an RSSZ warning should be issued)

School zone

demonstration area

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues

an RSSZ warning only if the vehicle is traveling above the RSSZ speed limit in the RSSZ zone

Data recorder

observes vehicles speed and RSSZ warning from the LDV HMI





CVE-App008-v01 First Article LDV OBU –

warning arbitration Demonstration Verify a high priority

warning takes precedence over a low priority warning when both warning types are present. (see CVE System Design Document (SDD) for message priority rankings)

Demonstration

area • Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues

the high priority warning when both a high and low priority event exists

Data recorder

observes high priority warning f rom the LDV HMI

CVE-App009-v01 Channel 180 Congestion

1-LDV OBU (RLVW)

1-RSU (SPaT, MAP, TIM, WSA)

Demonstration Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when SPaT, MAP, TIM, and WSAs are broadcast on Channel 180 from the same RSU

• Demonstration area

• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder




1-LDV OBU (RLVW and OTA)

1-RSU (SPaT, MAP, TIM, WSA)

Demonstration Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a f irmware update is in process and SPaT, MAP, TIM, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder







CVE-App011-v01 Channel 180

Congestion

1-LDV OBU

(RLVW)

1-Transit vehicle OBU (SRM)

1-RSU (SPaT, MAP, TIM, SSM, WSA)


OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues

an RLVW warning only if the vehicle is approaching the intersection in such a way that the vehicle will run a red light

Data recorder

observes RLVW warning from the LDV HMI


1-LDV OBU (RLVW)

1-Transit vehicle OBU (SRM and OTA)


Demonstration Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, a firmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder








Congestion

1-LDV OBU

(RLVW, OTA)




OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 2 OBUs have a f irmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues


Data recorder



2-LDV OBU (RLVW, OTA)



Demonstration Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 3 OBUs have a f irmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder








Congestion

3-LDV OBU

(RLVW, OTA)




OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 4 OBUs have a f irmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder

LDV OBU issues


Data recorder






Demonstration Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 5 OBUs have a f irmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


• Field

• Test conductor

• Data producer

• Vehicle drivers

• Data recorder






6.3. ROADSIDE UNIT

Table 9: Roadside Unit Test Cases



CVE-RSU001-v01 First Article

Post-installation

VDTO – BSM Forwarding

Demonstration Verify RSU forwards BSMs, received from passing vehicles, to Message Handler.


• Field

• All RSU installation locations

• Test conductor

• Data producer

• Data recorder

RSU forwards BSMs to the Message Handler

Data recorder observes the Message Handler receiving BSMs f rom the RSU

CVE-RSU002-v01 • First Article

• Post-installation

Signal priority – SRM forwarding

Demonstration Verify RSU forwards SRMs, received from approaching vehicles, to the Message Handler


• Field


• Test conductor

• Data producer

• Data recorder

RSU forwards SRMs to the local Message Handler

Data recorder observes SRMs being sent to the Message Handler; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time



SPaT broadcast Demonstration Verify RSUs broadcast SPaT messages received f rom the Message Handler over DSRC on Channel 180 at 10 Hz


• Field


• Test conductor

• Data producer

• Data recorder

RSU broadcast SPaT messages

Data recorder observes the RSU broadcasting SPaT messages; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool







SPaT broadcast Demonstration Verify SPaT

broadcast by RSUs matches the signal head status for all lanes


• Field


• Test conductor

• Data producer

• Data recorder

SPaT status

matches signal head status

Data recorder

observes the SPaT status presented on the CVTT matches the status of the signal head for all lanes



MAP broadcast Demonstration Verify RSUs broadcast MAP messages received f rom the Message Handler over DSRC on Channel 180 at 1 Hz


• Field

• All RSU

installation locations

• Test conductor

• Data producer

• Data recorder

RSU broadcast MAP messages

Data recorder observes the RSU broadcasting MAP messages; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool



MAP Demonstration Verify MAP file matches intersection geometry


• Field


• Test conductor

• Data producer

• Data recorder

Decoded MAP message broadcast by the RSU matches the applicable intersection geometry

Data recorder observes decoded MAP message overlaid on Google Maps, or similar map tool



RTCM broadcast Demonstration Verify RSUs broadcast RTCM messages, received from the Message Handler, over DSRC on Channel 180 every 2 seconds.


• Field


• Test conductor

• Data producer

• Data recorder

RSU broadcast RTCM messages

Data recorder observes the RSU broadcasting RTCM messages; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool







Signal priority –

SSM broadcast Demonstration Verify RSUs

broadcasts SSM, received from the Message Handler, over DSRC on Channel 180 in response to receiving an SRM


• Field

• RSUs supporting priority/preemption

• Test conductor

• Data producer

• Data recorder

RSU broadcast

SSM messages

Data recorder

observes the RSU broadcasting SSM messages; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool



School zone – TIM broadcast

Demonstration Verify RSUs broadcasts TIMs, received from the Message Handler, on Channel 180 at 1 Hz


• Field

• RSUs supporting

RSSZ

• Test conductor

• Data producer

• Data recorder

RSU broadcast RSSZ based TIM messages

Data recorder observes the RSU broadcasting TIMs; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool



System Monitoring Demonstration Verify RSUs report Channel Busy Ratio to TCVMS, when above a predetermined threshold


• Field


• Test conductor

• Data producer

• Data recorder

RSU sends channel busy ratio (CBR) parameters to the TCVMS only when the CBR is above a specified threshold (e.g., 80%)

Data recorder observes that only CBR values above a given threshold (e.g., 80%) are logged in the TCVMS

CVE-RSU012-v01 Channel 180 Congestion

Broadcast all messages on Channel 180 at their appropriate rates

Demonstration Verify RSUs can broadcast SPaT, MAP, TIM, SSM, and WSAs on Channel 180

Field • Test conductor

• Data producer

• Data recorder

RSUs broadcast SPaT, MAP, TIM, SSM, and WSAs at the appropriate rates on Channel 180

Data recorder observes RSUs broadcasting SPaT, MAP, TIM, SSM, and WSAs at the appropriate rates on Channel 180






Post-installation

WSA broadcast Demonstration Verify RSU

broadcast WSA with correct parameters (including WRA) on Channel 180 at 1 Hz



• Test conductor

• Data producer

• Data recorder

WSAs broadcast by

an RSU contains the correct parameters

Data recorder

observes WSAs broadcast by the RSU contain the correct parameters; either through inspection of the RSU log file (post-processing) or through a live over-the-air data capture tool


Post-installation

IPv6 services Demonstration Verify RSUs support OBU IPv6 connectivity to cloud-based services (SCMS, OBU f irmware update services, etc.)



• Test conductor

• Data producer

• Data recorder

OBUs are able to connect to the SCMS through the RSU

Data recorder observes OBUs are able to connect to the SCMS through the RSU; either through inspection of the OBU log file (post-processing) or through a live over-the-air data capture tool

CVE-RSU014-v01 First Article RSU communication range for RLVW

Test Verify the RSU communication range of Channel 180 (23 dBm max transmit power)

First Article RSU installation locations

• Test conductor

• Data producer

• Data recorder

RLVW is provided to driver in time for the driver to react safely

Data recorder observes location at which OBUs begin to receive SPaT and MAP messages from RSU; through inspection of the OBU log file (post-processing) or through a live over-the-air data capture tool





CVE-RSU015-v01 First Article RSU

communication range for signal priority

Test Verify the RSU

communication range of Channel 180 (23 dBm max transmit power

First Article RSU


• Test conductor

• Data producer

• Data recorder

Signal timing is

adjusted to enable the requesting vehicle to traverse the intersection under Green

Data recorder

observes location at which OBUs begin receiving WSAs, advertising SRMs, f rom RSU and the OBU begins broadcasting SRMs; through inspection of the OBU log file (post-processing) or through a live over-the-air data capture tool

CVE-RSU016-v01 First Article RSU communication range for signal preemption

Test Verify the RSU communication range of Channel 180 (23 dBm max transmit power

First Article RSU installation locations

• Test conductor

• Data producer

• Data recorder

Signal timing is adjusted to enable the requesting vehicle to traverse the intersection under Green

Data recorder observes location at which OBUs begin receiving WSAs, advertising SRMs, f rom RSU and the OBU begins broadcasting SRMs; through inspection of the OBU log file (post-processing) or through a live over-the-air data capture tool





CVE-RSU017-v01 First Article RSU

communication range for RSSZ

Test Verify the RSU

communication range of Channel 180 (23 dBm max transmit power)

First Article RSU


• Test conductor

• Data producer

• Data recorder

RSSZ warning is

provided to driver in time for the driver to react safely

Data recorder

observes location at which OBUs begin receiving TIMs from RSU; through inspection of the OBU log file (post-processing) or through a live over-the-air data capture tool


6.4. MESSAGE HANDLER

Table 10: Message Handler Test Cases

Test Case ID Test Type Function Verification Method Test Objective CVE Test Locations Tester Role Metric Pass Criteria

CVE-MH001-v01 • First Article


SPaT message generation

Demonstration Verify the Message Handler generates and forwards SAE J2735-2016 SPaT messages, based on SPaT data received from the local signal controller, to the local RSU


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler generates J2735 SPaT messages

Data recorder observes Message Handler sending J2735 SPaT messages to the local RSU; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time






Signal priority –

Authorization Demonstration Verify the Message

Handler properly determines if OBU requesting priority/preemption is authorized to receive priority/preemption


• Field

• RSU Locations supporting priority/preemption

• Test conductor

• Data producer

• Data recorder

Message Handler

verifies (via 1609.2 certificates) signal priority/preemption requests from RSU

Data recorder

observes the Message Handler verifying incoming SRMs; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time



Signal priority – Place call to Signal Controller

Demonstration Verify the Message Handler places a priority call to the local Signal Controller based on an authorized SRM.


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler places a priority call based on received SRM

Data recorder observes the result of the Message Handler sending a priority call to the Signal Controller; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time






Signal preemption –

Place call to Signal Controller

Demonstration Verify the Message

Handler places a preemption call to the local Signal Controller based on an authorized SRM.


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler

places a preemption call based on received SRM

Data recorder

observes the Message Handler sending a preemption call to the Signal Controller either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time



Signal priority – SSM generation

Demonstration Verify the Message Handler generates and forwards SAE J2735-2016 SSMs, based on status data received from the local Signal Controller, to the local RSU


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler sends SSM to RSU

Data recorder observes the Message Handler sending an SSM to the RSU; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time






RTCM message

generation Demonstration Verify the Message

Handler generates and forwards SAE J2735-2016 RTCM messages, based on data received f rom the Position Correction System


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler

sends RTCM to RSU

Data recorder

observes the Message Handler sending a RTCMs to the RSU; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time

CVE-MHTC009-v01

• First Article


BSM forwarding Demonstration Verify the Message Handler forwards BSMs received from the RSU to the TCVMS


• Field


• Test conductor

• Data producer

• Data consumer

• Data recorder

Message Handler forwards BSMs to the TCVMS

Data recorder observes BSMs f rom applicable RSU in the TCVMS

CVE-MHTC010-v01

• First Article


SPaT message forwarding

Demonstration Verify the Message Handler forwards SPaT messages to the TCVMS



• Test conductor

• Data producer

• Data consumer

• Data recorder

Message Handler forwards SPaT messages to the TCVMS

Data recorder observes SPaT messages from applicable RSU in the TCVMS

CVE-MHTC011-v01

• First Article


SRM Forwarding Demonstration Verify the Message Handler forwards SRMs received from the RSU to the TCVMS


• Field


• Test conductor

• Data producer

• Data consumer

• Data recorder

Message Handler forwards SRMs to the TCVMS

Data recorder observes SRMs f rom applicable RSU in the TCVMS

CVE-MHTC012-v01

• First Article


SSM Forwarding Demonstration Verify the Message Handler forwards SSMs generated f rom signal controller data to the TCVMS


• Field


• Test conductor

• Data producer

• Data consumer

• Data recorder

Message Handler forwards SSMs to the TCVMS

Data recorder observes SSMs f rom applicable Message Handler in the TCVMS




CVE-MHTC013-

v01 • First Article


RTCM Forwarding Demonstration Verify the Message

Handler forwards RTCM messages generated from Position Correction System data to the TCVMS


• Field


• Test conductor

• Data producer

• Data consumer

• Data recorder

Message Handler

forwards RTCM messages to the TCVMS

Data recorder

observes RTCM messages from applicable Message Handler in the TCVMS

CVE-MHTC014-v01

• First Article


Store MAP message

Demonstration Verify the Message Handler stores MAP messages received f rom the TCVMS


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler stores MAP messages received f rom the TCVMS

Data recorder observes MAP messages stored in the appropriate location (directory) in the Message Handler

CVE-MHTC015-v01

• First Article


Send MAP messages to RSUs

Demonstration Verify the Message Handler sends SAE J2735-2016 MAP messages to the local RSU


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler sends MAP messages to the local RSU

Data recorder observes RSUs broadcasting MAP messages received f rom the Message Handler

CVE-MHTC016-v01

• First Article


Store TIM message Demonstration Verify the Message Handler stores TIM messages received f rom the TCVMS


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler stores TIM messages received f rom the TCVMS

Data recorder observes TIM messages stored in the appropriate location (directory) in the Message Handler

CVE-MHTC017-v01

• First Article


Send TIM message to RSUs

Demonstration Verify the Message Handler sends SAE J2735-2016 TIM messages to the local RSU


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler sends TIM messages to the local RSU

Data recorder observes RSUs broadcasting TIM messages received f rom the Message Handler




CVE-MHTC018-



Update TIM Start

Time and Duration Demonstration Verify the Message

Handler updates the Start Time and Duration of the stored TIM


• Field


• Test conductor

• Data producer

• Data recorder

Message Handler

updates TIM Start Time and Duration to the correct values

Data recorder

observes RSUs broadcasting TIM messages with updated Start Time and Duration values


6.5. TRAFFIC SIGNAL CONTROLLER

Table 11: Traffic Signal Controller Test Cases



CVE-TSCTC001-v01

• First Article


SPaT data Demonstration Verify Traffic Signal Controller sends SPaT data to the Message Handler


• Field


• Test conductor

• Data producer

• Data recorder

Signal controller sends SPaT data to Message Handler

Data recorder observes the Signal Controller sending data to the Message Handler; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time

CVE-TSCTC002-v01

• First Article


Signal priority call Demonstration Verify Traffic Signal Controller processes a signal priority call f rom the Message Handler


• Field

• All RSU installation

locations

• Test conductor

• Data producer

• Data recorder

Signal controller modifies signal timing based on a priority call from the Message Handler

Data recorder observes the Signal Controller changing the signal timing through the Signal Controller user interface





CVE-TSCTC003-



Signal preemption

call Demonstration Verify Traffic Signal

Controller processes a signal preemption call from the Message Handler


• Field


• Test conductor

• Data producer

• Data recorder

Signal Controller

modifies signal timing based on a preemption call from the Message Handler

Data recorder

observes the Signal Controller changing the signal timing through the Signal Controller user interface

CVE-TSCTC004-v01

• First Article


Signal priority status Demonstration Verify Traffic Signal Controller sends priority status to the Message Handler


• Field

• All RSU installation

locations

• Test conductor

• Data producer

• Data recorder

Signal Controller sends signal priority status data to Message Handler

Data recorder observes the Signal Controller sending signal priority status data to the Message Handler; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time

CVE-TSCTC005-v01

• First Article


Signal preemption status

Demonstration Verify Traffic Signal Controller sends preemption status to the Message Handler


• Field


• Test conductor

• Data producer

• Data recorder

Signal Controller sends signal preemption status data to Message Handler

Data recorder observes the Signal Controller sending signal preemption status data to the Message Handler; either through inspection of the Message Handler log file (post-processing) or through a Message Handler engineering interface in real-time




6.6. TRAFFIC CV MANAGEMENT SYSTEM

Table 12: Traffic CV Management System Test Cases



CVE-TCVMS001-v01

First Article MAP input Demonstration Verify traffic management staff can input a MAP message payload to the TCVMS and specify which RSU it should be broadcast from

Traf fic Management Center

Test conductor

Data producer

Traf fic data consumer

Data recorder

Traf fic management staff sends a MAP f ile to an RSU through the TCVMS user interface

Data recorder observes traffic management staff uploading a MAP f ile to an RSU through the TCVMS user interface and inspecting the RSU MAP file storage directory

CVE-TCVMS002-v01

First Article TIM input Demonstration Verify traffic management staff can input a TIM payload to the TCVMS and specify which RSU it should be broadcast from


Test conductor

Data producer


Data recorder

Traf fic management staff sends a TIM f ile to an RSU through the TCVMS user interface

Data recorder observes traffic management staff uploading a TIM file to an RSU through the TCVMS user interface and inspecting the RSU TIM f ile storage directory

CVE-TCVMS003-v01

First Article Receive/archive MAP

Demonstration Verify the TCVMS archives the MAP message locally and forwards it to the Operating System


Test conductor

Data producer


Data recorder

TCVMS archives MAP file

Data recorder observes archived MAP files through inspection of the TCVMS archive data storage folder





CVE-TCVMS004-

v01 First Article Receive/archive

TIM Demonstration Verify the TCVMS

archives the TIM locally and forwards it to the Operating System

Traf fic Management

Center Test conductor

Data producer


Data recorder

TCVMS archives

TIM f ile

Data recorder

observes archived TIM f iles through inspection of the TCVMS archive data storage folder

CVE-TCVMS005-v01

First Article Receive/archive BSM

Demonstration Verify the TCVMS archives BSMs locally and forwards it to the Operating System


Test conductor

Data producer


Data recorder

TCVMS archives BSMs and send BSMs to the Operating System

Data recorder observes archived BSMs through inspection of the TCVMS archive data storage folder and either through inspection of the TCVMS log file (post-processing) or through a TCMS engineering interface in real-time

CVE-TCVMS006-v01

First Article Receive/archive SPaT

Demonstration Verify the TCVMS archives the SPaT messages locally and forwards it to the Operating System


Test conductor

Data producer


Data recorder

TCVMS archives SPaT messages and sends SPaT message to the Operating System

Data recorder observes archived SPaT messages through inspection of the TCVMS archive data storage folder and either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time





CVE-TCVMS007-


SRM Demonstration Verify the TCVMS

archives SRMs locally and forwards it to the Operating System

Traf fic Management


Data producer


Data recorder

TCVMS archives

SRM messages and sends SRM message to the Operating System

Data recorder

observes archived SRM messages through inspection of the TCVMS archive data storage folder and either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time

CVE-TCVMS008-v01

First Article Receive/archive SSM

Demonstration Verify the TCVMS archives SSMs locally and forwards it to the Operating System


Test conductor

Data producer


Data recorder

TCVMS archives SSM messages and sends SSM message to the Operating System

Data recorder observes archived SSM messages through inspection of the TCVMS archive data storage folder and either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time





CVE-TCVMS009-


RTCM Demonstration Verify the TCVMS

archives RTCMs locally and forwards it to the Operating System

Traf fic Management


Data producer


Data recorder

TCVMS archives

RTCM messages and sends RTCM message to the Operating System

Data recorder

observes archived RTCM messages through inspection of the TCVMS archive data storage folder and either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time

CVE-TCVMS010-v01

Operational Performance metrics configuration

Demonstration Verify traffic management staff can specify a performance metric to the TCVMS (and specify how it is calculated)

NOTE: This is an Operating System test case


Test conductor

Data producer


Data recorder

Traf fic management staff enters a performance metric in the TCVMS user interface

Data recorder observes traffic management staff entering performance metrics into the TCVMS user interface





CVE-TCVMS011-

v01 First Article Performance

metrics reporting Demonstration Verify generated

performance metrics are uploaded to the Operating System.


Traf fic Management


Data producer


Data recorder

TCVMS sends

performance metric to the Operating System

Data recorder

observes the TCVMS sending performance metrics to the Operating System; either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time

CVE-TCVMS012-

v01 First Article Personally

identifiable information (PII) removal

Demonstration Verify PII is

removed prior to sending data to Operating System


Traf fic Management


Data producer


Data recorder

Data sent to

Operating System does NOT contain PII as defined in the Data Privacy Plan

Data recorder

observes PII has been removed from data prior to send to the Operating System through inspection of a TCVMS log file of data sent to the Operating System

CVE-TCVMS013-v01

First Article RTCM configuration Demonstration Verify traffic management staff can specify a source for each RSU to receive RTCM data.


Test conductor

Data producer


Data recorder

Traf fic management staff specify a RTCM source for each RSU

Data recorder observes traffic management staff specify RTCM source per RSU through the TCVMS user interface





CVE-TCVMS014-

v01 First Article Unauthorized

network activity detection

Demonstration Verify TCVMS can

detect unauthorized network activity.

NOTE: This is a Traf fic Management Center test case

Traf fic Management


Data producer


Data recorder

TCVMS detects

unauthorized network activity

Data recorder

observes TCVMS detecting unauthorized network activity; either through inspection of the TCVMS log file (post-processing) or through a TCVMS engineering interface in real-time

CVE-TCVMS015-v01

First Article Traf fic signal cabinet tamper detection

Demonstration Verify TCVMS can detect traffic signal cabinet tampering and issue the proper notifications to Traffic CV management staff.



Test conductor

Data producer


Data recorder

TCVMS detects cabinet tamper event and provides appropriate notification to traffic management staff through email

Data recorder observes notification of cabinet tamper event

CVE-TCVMS016-v01

First Article RSU remote configuration

Demonstration Verify traffic management staff can modify configurable setting of an RSU using the TCVMS.


Test conductor

Data producer


Data recorder

Traf fic management staff modifies RSU configuration parameters through the TCVMS user interface

Data recorder observes traffic management staff updating RSU configuration parameters through the TCVMS user interface and inspecting the configuration of the RSU(s) directly





CVE-TCVMS017-

v01 First Article Dashboard Demonstration Verify Traffic

management staff have access to the following RSU data via the TCVMS: uptime status, connectivity, RSU graphical display, etc.

Traf fic Management


Data producer


Data recorder

TCVMS user

interface displays RSU data

Data recorder

observes RSU data through the TCVMS user interface and through inspection of the RSU(s) directly

CVE-TCVMS018-v01

First Article Unauthorized network activity notification

Demonstration Verify alert is issued for unauthorized network activity using dashboard icon and an email is sent to traffic CV management staff



Test conductor

Data producer


Data recorder

TCVMS detects unauthorized network activity and provides appropriate notification to traffic management staff through a dashboard icon and email

Data recorder observes notification of unauthorized network activity

CVE-TCVMS019-v01

First Article RSU status notification

Demonstration Verify alert is issued when RSU is not operating as intended (degraded/failure conditions, or limited network connectivity) using dashboard icon and an email is to traffic CV management staff



Test conductor

Data producer


Data recorder

TCVMS sends a notification to traffic management staff of unauthorized network activity through a dashboard icon and email to TCVMS staff

Data recorder observes traffic management staff notification





CVE-TCVMS020-

v01 First Article Log notification and

emails sent to staff Demonstration Verify TCVMS logs

all alerts/emails to traffic CV management staff


Traf fic Management


Data producer


Data recorder

TCVMS logs alerts

and emails sent to TCVMS staff

Data recorder

observes TCVMS log files containing alerts and email notifications

CVE-TCVMS021-v01

First Article Archived data query Demonstration Verify traffic management staff can query archived data on the TCVMS



Test conductor

Data producer


Data recorder

Traf fic management staff enters data query parameters into the TCVMS user interface

Data recorder observes traffic management staff entering data query parameters into the TCVMS user interface


6.7. TRANSIT CV MANAGEMENT SYSTEM

Table 13: Transit CV Management System Test Cases



CVE-TrCVMS001-v01 First Article Performance metrics configuration

Demonstration Verify transit management staff can specify a performance metric to the TrCVMS (and specify how it is calculated)



Test conductor

Data producer


Data recorder

Transit management staff enters a performance metric in the TrCVMS user interface

Data recorder observes transit management staff entering performance metrics into the TrCVMS user interface





CVE-TrCVMS002-v01 First Article Performance

metrics reporting Demonstration Verify generated

performance metrics are uploaded to the Smart Columbus Operating System

COTA McKinley

Ave Garage


Test conductor

Data producer


Data recorder

TrCVMS sends

performance metric to the Operating System

Data recorder

observes the TrCVMS sending performance metrics to the Operating System; either through inspection of the TrCVMS log file (post-processing) or through a TrCVMS engineering interface in real-time

CVE-TrCVMS003-v01 First Article Archived data query

Demonstration Verify Transit Management staff can query archived data on the TrCVMS



Test conductor

Data producer


Data recorder

Transit Management staff enters data query parameters into the TrCVMS user interface

Data recorder observes Transit Management staff entering data query parameters into the TrCVMS user interface

CVE-TrCVMS004-v01 First Article Receive/archive transit vehicle interaction event

Demonstration Verify the TrCVMS archives the transit vehicle interaction events and forwards them to the Operating System



Test conductor

Data producer


Data recorder

TrCVMS archives data from transit vehicle logs

Data recorder observes archived transit vehicle log f iles through inspection of the TrCVMS system archive data storage folder



Test Scenarios

Test scenarios are critical to the acceptance of the CVE, as the scenarios allow the system to be tested holistically, end-to-end, from all active participant viewpoints. Scenarios are made up of a series of test procedures used to simulate the system in a real-world operational environment. This approach validates the system’s ability to meet the concepts established through the project ConOps and

forms the basis of acceptance testing for the given production release.

The scenarios outlined below reflect the capabilities the system must be able to perform to receive acceptance. The CVE capability and performance will be deemed acceptable provided all testing elements within a scenario successfully pass the test from each participant’s viewpoint.

Table 14 provides a comprehensive list of scenarios that will be tested with traceability to the requirement(s) published in the SyRS, ICD, and ConOps.

Table 14: Testing Scenarios with Requirements Traceability

Function Acceptance Test Scenario Scenario Description

Test Case ID Reference Requirement ID Reference

Emergency electronic brake light

CVE-ATS0100-v01 LDV driver receives appropriate EEBL alerts

CVE-OBU001-v01

CVE-OBU003-v01 CVE-App001-v01

CVE-App008-v01 CVE-RSU007-v01

CVE-RSU012-v01

CVE-MH008-v01

CVE-DR3005-V01

CVE-DR3006-V01 CVE-PR1105-V01

CVE-PR3003-V01

CVE-PR3007-V01

CVE-PR3008-V01 CVE-PR3009-V01

CVE-DR1374-V01

CVE-DR1375-V01

CVE-FN1319-V01 CVE-FN1321-V01

CVE-FN3111-V01

CVE-FN3112-V01


Forward collision warning

CVE-ATS0101-v01 LDV driver receives appropriate FCW alerts

CVE-OBU001-v01


CVE-App008-v01

CVE-RSU007-v01 CVE-RSU012-v01

CVE-MH008-v01

CVE-DR3005-V01


CVE-PR3003-V01

CVE-PR3007-V01


CVE-DR1374-V01

CVE-DR1375-V01


CVE-FN3111-V01

CVE-FN3112-V01


Intersection movement assist

CVE-ATS0102-v01 LDV driver receives appropriate IMA alerts

CVE-OBU001-v01

CVE-OBU004-v01

CVE-App004-v01 CVE-App008-v01

CVE-DR3005-V01

CVE-DR3006-V01


CVE-PR3007-V01

CVE-PR3008-V01

CVE-PR3009-V01 CVE-DR1374-V01

CVE-DR1375-V01

CVE-FN1319-V01


CVE-FN3112-V01

CVE-FN3227-V01

CVE-FN3228-V01

Chapter 7. Test Scenarios




CVE-RSU007-v01

CVE-RSU012-v01

CVE-MH008-v01

Lane change warning/blind spot warning

CVE-ATS0103-v01 LDV driver receives appropriate LCW alerts

CVE-OBU001-v01


CVE-App008-v01


CVE-MH008-v01

CVE-DR3005-V01


CVE-PR3003-V01

CVE-PR3007-V01


CVE-DR1374-V01

CVE-DR1375-V01


CVE-FN3111-V01

CVE-FN3112-V01


Traf fic signal priority/preemption

CVE-ATS0104-v01 Transit vehicles receive appropriate traffic signal priority

CVE-OBU014-v01

CVE-RSU002-v01


CVE-RSU005-v01



CVE-RSU012-v01

CVE-MH002-v01 CVE-MH003-v01

CVE-MH004-v01


CVE-TSCTC001-v01 CVE-TSCTC002-v01

CVE-TSCTC003-v01

CVE-TSCTC004-v01

CVE-DR1144-V01

CVE-DR1145-V01

CVE-DR1146-V01 CVE-DR1147-V01

CVE-DR1148-V01



CVE-DR1153-V01


CVE-DR1156-V01



CVE-DR1161-V01


CVE-DR1164-V01


CVE-DR1167-V01

CVE-DR1378-V01

CVE-DR1379-V01


CVE-DR1382-V01



CVE-DR1387-V01


CVE-DR1390-V01



CVE-DR1395-V01


CVE-DR1398-V01


CVE-PR1401-V01

CVE-DR1422-V01

CVE-DR1423-V01


CVE-DR1426-V01



CVE-DR1431-V01


CVE-DR1434-V01


CVE-PR2999-V01 CVE-AR3121-V01

CVE-AR3122-V01


CVE-FN1309-V01


CVE-FN1313-V01

CVE-FN3238-V01

CVE-FN3240-V01

CVE-IF1340-V01 CVE-IF1342-V01

CVE-IF1345-V01



CVE-IF2985-V01


CVE-PR1369-V01


CVE-PR3237-V01 CVE-SR3123-V01

CVE-SR3125-V01

CVE-SR3127-V01 CVE-SR3130-V01

CVE-FN1439-V01


CVE-FN1448-V01





CVE-DR1168-V01

CVE-DR1169-V01


CVE-DR1172-V01



CVE-DR1177-V01


CVE-DR1181-V01



CVE-DR1375-V01

CVE-DR1402-V01

CVE-DR1404-V01


CVE-DR1407-V01



CVE-DR1412-V01


CVE-DR1415-V01



CVE-DR1420-V01

CVE-FN1314-V01

CVE-FN1319-V01


CVE-FN1327-V01



CVE-FN2980-V01


CVE-FN3000-V01



CVE-FN3228-V01

CVE-FN3236-V01

CVE-FN3002-V01

CVE-DR1477-V01

CVE-DR1478-V01 CVE-FN1498-V01

CVE-FN1499-V01



CVE-FN1510-V01


CVE-FN1513-V01



CVE-PR1529-V01

CVE-PR1530-V01


CVE-ATS0105-v01 EVs receive appropriate traffic signal preemption

CVE-OBU012-v01


CVE-App008-v01


CVE-RSU004-v01



CVE-RSU011-v01

CVE-RSU012-v01 CVE-MH002-v01

CVE-MH003-v01

CVE-DR1144-V01


CVE-DR1147-V01


CVE-DR1150-V01



CVE-DR1155-V01


CVE-DR1158-V01

CVE-DR1378-V01


CVE-DR1381-V01


CVE-DR1384-V01



CVE-DR1389-V01


CVE-DR1392-V01

CVE-DR1422-V01


CVE-DR1425-V01


CVE-DR1428-V01



CVE-DR1433-V01


CVE-DR1436-V01

CVE-FN3236-V01


CVE-IF1340-V01


CVE-IF1346-V01



CVE-PR1365-V01


CVE-PR3213-V01





CVE-MH005-v01

CVE-MH006-v01

CVE-MH008-v01 CVE-TSCTC001-v01

CVE-TSCTC002-v01


CVE-DR1159-V01

CVE-DR1160-V01


CVE-DR1163-V01



CVE-DR1168-V01


CVE-DR1171-V01



CVE-DR1176-V01


CVE-DR1179-V01



CVE-DR1374-V01

CVE-DR1375-V01

CVE-DR1393-V01

CVE-DR1394-V01


CVE-DR1397-V01



CVE-DR1402-V01


CVE-DR1406-V01



CVE-DR1411-V01


CVE-DR1414-V01



CVE-PR2995-V01

CVE-DR1420-V01

CVE-PR2999-V01

CVE-AR3121-V01

CVE-AR3122-V01 CVE-FN1113-V01

CVE-FN1308-V01



CVE-FN1314-V01


CVE-FN1325-V01



CVE-FN2977-V01


CVE-FN3000-V01



CVE-FN3227-V01

CVE-FN3228-V01

CVE-PR3226-V01

CVE-PR3237-V01


CVE-SR3127-V01

CVE-SR3130-V01 CVE-FN1439-V01


CVE-FN1448-V01


CVE-FN1500-V01

CVE-FN3107-V01 CVE-PR1531-V01


CVE-FN1503-V01


CVE-FN1513-V01



CVE-FN1519-V01


CVE-ATS0106-v01 HDVs receive appropriate traffic signal priority

CVE-OBU016-v01


CVE-RSU002-v01


CVE-RSU005-v01

CVE-PR3007-V01


CVE-DR1144-V01


CVE-DR1147-V01

CVE-DR1175-V01


CVE-DR1178-V01


CVE-DR1182-V01

CVE-PR1401-V01

CVE-AR3121-V01 CVE-AR3122-V01

CVE-FN1113-V01


CVE-FN1311-V01

CVE-IF1361-V01

CVE-IF1363-V01 CVE-PR1365-V01

CVE-PR1366-V01


CVE-PR3226-V01





CVE-RSU006-v01

CVE-RSU007-v01


CVE-RSU012-v01



CVE-MH008-v01


CVE-TSCTC003-v01

CVE-TSCTC004-v01

CVE-DR1148-V01

CVE-DR1149-V01


CVE-DR1152-V01



CVE-DR1157-V01


CVE-DR1160-V01



CVE-DR1165-V01


CVE-DR1168-V01



CVE-DR1173-V01

CVE-DR1174-V01

CVE-PR1183-V01

CVE-PR2993-V01


CVE-DR1378-V01



CVE-DR1383-V01


CVE-DR1386-V01



CVE-DR1391-V01


CVE-DR1394-V01



CVE-PR1399-V01

CVE-PR1400-V01

CVE-FN1312-V01

CVE-FN1313-V01


CVE-FN1325-V01



CVE-FN2979-V01


CVE-FN2988-V01



CVE-FN3228-V01


CVE-FN3240-V01



CVE-IF1347-V01

CVE-IF1359-V01

CVE-PR3237-V01

CVE-SR3123-V01


CVE-SR3130-V01



CVE-FN3002-V01

CVE-FN1502-V01 CVE-PR1527-V01

CVE-PR1528-V01



CVE-FN1505-V01


CVE-FN1511-V01



CVE-FN1519-V01

Vehicle data for traffic operations

CVE-ATS0107-v01 TCVMS and Operating System receive vehicle data from OBUs and RSUs

CVE-OBU001-v01


CVE-TCVMS001-v01

CVE-TCVMS002-v01 CVE-TCVMS003-v01

CVE-TCVMS004-v01

CVE-DR3005-V01


CVE-PR3003-V01


CVE-PR3009-V01

CVE-FN1335-V01


CVE-FN2989-V01


CVE-IF1361-V01

CVE-SR3128-V01


CVE-DR1276-V01


CVE-FN1439-V01

CVE-FN1456-V01


CVE-FN2911-V01


CVE-FN3032-V01





CVE-TCVMS005-v01

CVE-TCVMS006-v01


CVE-TCVMS009-v01



CVE-TCVMS014-v01


CVE-TCVMS017-v01



CVE-AR3121-V01

CVE-AR3122-V01


CVE-FN1309-V01



CVE-FN1318-V01


CVE-FN1327-V01


CVE-IF1362-V01

CVE-IF1363-V01


CVE-PR1369-V01


CVE-PR3244-V01 CVE-PY2912-V01

CVE-PY3120-V01


CVE-SR3124-V01

CVE-SR3126-V01

CVE-FN1440-V01

CVE-FN1441-V01


CVE-FN1444-V01



CVE-FN1449-V01


CVE-FN1453-V01

CVE-FN1454-V01

CVE-FN3041-V01

CVE-FN3110-V01


CVE-PR3033-V01

CVE-PY3034-V01 CVE-DR1562-V01


CVE-FN1587-V01


CVE-FN1590-V01

CVE-FN1591-V01

Transit vehicle interaction event recording

CVE-ATS0108-v01 Transit vehicles record appropriate interaction events

CVE-OBU002-v01

CVE-OBU003-v01 CVE-OBU004-v01


CVE-OBU007-v01


CVE-OBU010-v01

CVE-OBU015-v01 CVE-RSU007-v01


CVE-DR1144-V01



CVE-DR1149-V01


CVE-DR1152-V01



CVE-DR1157-V01


CVE-DR1160-V01

CVE-DR1168-V01



CVE-DR1173-V01


CVE-DR1176-V01



CVE-DR1182-V01


CVE-DR1374-V01

CVE-DR1412-V01



CVE-DR1417-V01


CVE-DR1420-V01



CVE-DR1426-V01


CVE-DR1429-V01

CVE-PR2999-V01

CVE-AR3121-V01 CVE-AR3122-V01


CVE-FN1309-V01


CVE-FN1313-V01



CVE-FN2980-V01


CVE-FN3010-V01





CVE-DR1161-V01

CVE-DR1162-V01


CVE-DR1165-V01


CVE-DR1375-V01

CVE-DR1406-V01


CVE-DR1409-V01


CVE-DR1430-V01

CVE-DR1431-V01


CVE-DR1434-V01


CVE-FN3111-V01

CVE-FN3112-V01


CVE-IF2978-V01

CVE-IF2985-V01

Red light violation

warning CVE-ATS0109-v01 LDV driver receives

appropriate RLVW alerts

CVE-OBU006-v01

CVE-App005-v01

CVE-App008-v01



CVE-MH001-v01

CVE-MH008-v01 CVE-TSCTC001-v01

CVE-DR1144-V01

CVE-DR1145-V01

CVE-DR1146-V01



CVE-DR1151-V01


CVE-DR1154-V01



CVE-DR1159-V01


CVE-DR1162-V01



CVE-DR1167-V01


CVE-DR1170-V01

CVE-DR1171-V01

CVE-DR1172-V01

CVE-DR1173-V01



CVE-DR1178-V01


CVE-DR1182-V01



CVE-DR1378-V01


CVE-DR1381-V01



CVE-DR1386-V01


CVE-DR1389-V01

CVE-DR1390-V01

CVE-DR1391-V01

CVE-DR1392-V01



CVE-DR1397-V01


CVE-PR1400-V01

CVE-PR1401-V01 CVE-AR3121-V01


CVE-FN1308-V01


CVE-FN1312-V01



CVE-FN1327-V01


CVE-FN3111-V01

CVE-FN3112-V01

CVE-FN3227-V01

CVE-FN3228-V01


CVE-FN3240-V01 CVE-IF1342-V01

CVE-IF1345-V01


CVE-IF1356-V01



CVE-PR1369-V01


CVE-PR3237-V01


CVE-SR3130-V01 CVE-FN1439-V01

CVE-FN1441-V01


CVE-FN3002-V01





Reduced speed

school zone CVE-ATS0110-v01 LDV driver receives

appropriate RSSZ alerts

CVE-OBU007-v01

CVE-App006-v01

CVE-App008-v01



CVE-RSU012-v01

CVE-MH008-v01

CVE-DR1144-V01

CVE-DR1145-V01

CVE-DR1146-V01



CVE-DR1151-V01


CVE-DR1154-V01



CVE-DR1159-V01


CVE-DR1162-V01



CVE-DR1167-V01

CVE-DR1168-V01

CVE-DR1169-V01



CVE-DR1174-V01


CVE-DR1177-V01



CVE-PR1183-V01


CVE-DR1375-V01



CVE-DR3090-V01

CVE-DR3091-V01

CVE-DR3092-V01

CVE-DR3093-V01 CVE-AR3121-V01


CVE-FN1308-V01


CVE-FN1311-V01



CVE-FN1325-V01


CVE-FN1335-V01



CVE-FN3228-V01

CVE-FN3236-V01

CVE-FN3238-V01

CVE-FN3240-V01 CVE-IF1341-V01


CVE-IF1360-V01

CVE-IF2978-V01 CVE-PR2994-V01

CVE-SR3125-V01



CVE-FN1440-V01


CVE-FN1443-V01



Vehicle data for transit operations

CVE-ATS0111-v01 TrCVMS receive vehicle data from transit vehicle OBUs

CVE-OBU009-v01


CVE-TrCVMS001-v01 CVE-TrCVMS003-v01

CVE-TrCVMS004-v01

CVE-TrCVMS006-v01

CVE-AR3121-V01

CVE-AR3122-V01

CVE-FN1113-V01

CVE-FN1308-V01

CVE-FN1309-V01

CVE-FN1311-V01

CVE-FN1312-V01

CVE-FN1313-V01



Test Procedures

The test procedures designed for this test plan provide high-level process steps for testing components, subsystems, and the overall system against the system requirements, interfaces, and system design for the CVE. Entry Criteria describe the capabilities of various CVE components that must be present before the Test Procedure can begin. Exit Criteria describes the expected outcomes of the

test procedure. Data Output describes the data file or log that records that demonstrates the Exit Criteria was met.

8.1. ONBOARD UNIT

Table 15: Onboard Unit Test Procedures

Test Case ID Objective Entrance Criteria Procedure Exit Criteria Data Output

CVE-OBU001-v01 Verify OBUs broadcast BSMs as

def ined in SAE J2945/1 (10 BSMs/sec)

NOTE: This Test Procedure applies to LDV, HDV, EV, and Transit Vehicle OBUs

See Figure 1 for system setup

No DSRC devices are

broadcasting in Test Area

OBU is configured for "normal" operation, including broadcasting BSMs on Channel 172

CVTT is configured to log pcap files on Channel 172

1. Power on the OBU under test

2. Power on the connected vehicle test tool (CVTT)

3. Capture 5 minutes of BSMs with the CVTT

4. Review pcap file to confirm OBU broadcast 10 BSMs/sec

OBU broadcasts 10 BSMs/sec pcap file containing BSMs

Chapter 8. Test Procedures



CVE-OBU002-v02 Verify a transit vehicle FCW interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


OBU1 (LDV) is configured for "normal" operation, including broadcasting BSMs

OBU2 (transit) is configured for "normal" operation, including broadcasting BSMs and logging FCW events

Values (in seconds) are configured in the transit OBU for retaining logged messages before an event (x) and after an event (y) is detected

A location is set up as a typical roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) is stopped in the roadway

2. Vehicle with OBU2 (V2) (transit) approaches V1 from behind

3. Based on BSMs from OBU1, OBU2 determines a threat exists

4. Since V2 (transit) does not contain an HMI, OBU2 only records the FCW event

5. Review V2 (transit) OBU2 log f iles to confirm FCW was recorded

OBU2 records an FCW event along with all other CV messages received for x seconds before the FCW event and y seconds after the FCW event

OBU2 log file containing all CV messages received for x seconds before the FCW event and y seconds after the FCW event

CVE-OBU003-v02 Verify a transit vehicle EEBL interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


OBU1 (LDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172

OBU2 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and logging EEBL events


A location is set up as a typical roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) (transit) on the roadway

2. V1 performs a hard-braking maneuver such that an EEBL event flag is set in its BSMs


4. Since V2 (transit) does not contain and HMI, OBU2 only records the EEBL event

5. Review V2 (transit) OBU2 log f iles to confirm EEBL was recorded

OBU2 records an EEBL event along with all other CV messages received for x seconds before the EEBL event and y seconds after the EEBL event

OBU2 log file containing all CV messages received for x seconds before the EEBL event and y seconds after the EEBL event




CVE-OBU004-v02 Verify a transit vehicle IMA interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


OBU1(LDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172

OBU2 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and logging IMA events


A location is set up as a typical two lane (opposite direction of travel) roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) approaches vehicle with OBU2 (V2) (transit) in roadway segment in a lane to the left of V2

2. V2 begins to turn left in front of V1


4. Since V2 (transit) does not contain and HMI, OBU2 only records the IMA event

5. Review V2 (transit) OBU2 log f iles to confirm IMA was recorded

OBU2 records an IMA event along with all other CV messages received for x seconds before the IMA event and y seconds after the IMA event

OBU2 log file containing all CV messages received for x seconds before the IMA event and y seconds after the IMA event




CVE-OBU005-v02 Verify a transit vehicle LCW interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


OBU1 (LDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172

OBU2 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and logging LCW events


A location is set up as a typical two lane (same direction of travel) roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) drives on the left side of the vehicle with OBU2 (V2) (transit)

2. V2 attempts to change lanes to the left


4. Since V2 (transit) does not contain and HMI, OBU2 only records the LCW event

5. Review V2 (transit) OBU2 log f iles to confirm LCW was recorded

OBU2 records an LCW event along with all other CV messages received for x seconds before the LCW event and y seconds after the LCW event

OBU2 log file containing all CV messages received for x seconds before the LCW event and y seconds after the LCW event

CVE-OBU006-v02 Verify a transit vehicle RLVW interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


RSU is configured for "normal" operation, including broadcasting SPaT and MAP on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and logging RLVW events


A location is set up as a simple intersection in the demonstration area

1. Vehicle with OBU1 (V1) (transit) approaches the intersection at a speed in which the light will be Red when V1 arrives at the intersection

2. V1 maintains constant speed and heading

3. Based on SPaT and MAP from the RSU, OBU1 determines a threat exists

4. Since V1 (transit) does not contain and HMI, OBU1 only records the RLVW event

5. Review V2 (transit) OBU1 log f iles to confirm RLVW was recorded

OBU1 records an RLVW event along with all other CV messages received for x seconds before the RLVW event and y seconds after the RLVW event

OBU1 log file containing all CV messages received for x seconds before the RLVW event and y seconds after the RLVW event




CVE-OBU007-v02 Verify a transit vehicle RSSZ interaction event log consists of all received messages for a predetermined period before and af ter the warning generating the event was issued.


RSU is configured for "normal" operations, including broadcasting a TIM representing a RSSZ on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and logging RSSZ events


A location is set up as a school zone (reduced speed roadway segment) in the demonstration area

1. Vehicle with OBU1 (V1) (transit) approaches the school zone segment above the school zone speed limit

2. Based on the RSSZ from the RSU, OBU1 determines a threat exists

3. Since V1 (transit) does not contain and HMI, OBU1 only records the RSSZ event

4. Review V2 (transit) OBU1 log f iles to confirm RSSZ was recorded

OBU1 records an RSSZ event along with all other CV messages received for x seconds before the RSSZ event and y seconds after the RSSZ event

OBU1 log file containing all CV messages received for x seconds before the RSSZ event and y seconds after the RSSZ event




CVE-OBU008-v02 Verify a transit vehicle TSP interaction event log consists of an SRM and all received messages for a predetermined period before and after the warning generating the event was issued.


RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSM, and WSAs advertising SRM on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172, SRMs on Channel 180, and logging TSP events



1. Vehicle with OBU1 (V1) approaches the intersection at a speed in which the light will be Red when vehicle arrives at the intersection

2. Based on the SPaT, MAP, and WSA from the RSU, OBU1 broadcasts an SRM, requesting signal priority

3. OBU1 records the SRM event

OBU1 records an TSP event along with all other CV messages received for x seconds before the TSP event and y seconds after the TSP event

OBU1 log file containing all CV messages received for x seconds before the TSP event and y seconds after the TSP event

CVE-OBU009-v01 Verify a Transit vehicle OBU logs messages (SPaT, MAP, SRM, SSM, TIM) that it receives for a pre-determined period


RSU is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, SSM, and WSAs advertising SRM on Channel 180

OBU1 (transit) is configured for "normal" operation, including logging received messages

Value (x) (in seconds) is configured in the transit OBU for retaining logged messages


1. Vehicle with OBU1 (V1) approaches the intersection

2. OBU1 records SPaT, MAP, TIM, and WSA

OBU1 records all received messages

OBU1 log file containing all CV messages received for x seconds




CVE-OBU011-v01 Verify a transit vehicle OBU uploads transit vehicle interaction event data to the TrCVMS when arriving to the COTA garage, and deletes local transit vehicle interaction events af ter uploading


RSU is configured for "normal" operations

RSU is connected to the TrCVMS

RSU is broadcasting WSAs advertising Log Offload service on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and offloading log files

TrCVMS is configured for normal operations, including receiving log files from Transit vehicles

1. OBU1 approaches RSU and begins offloading its log files to the TrCVMS

2. OBU1 deletes its log files, once off loaded

3. Verify log files are archived in the TrCVMS

4. Verify OBU1 deleted its log f iles

TrCVMS contains OBU1 log f iles

OBU1 deleted log files

Screen shot, including date and time stamp, of TrCVMS f ile directory depicting OBU1 log files

Screen shot, including date and time stamp, of empty OBU1 log file directory

CVE-OBU012-v01 Verify the EV OBU broadcasts SRMs only when lights and sirens are active, and it is approaching an equipped intersection


RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs on Channel 180 and advertising SRM

OBU1 (EV) is installed in an EV, configured for "normal" operations, including broadcasting BSMs on Channel 172 and SRMs on Channel 180 when the lights and sirens are on


CVTT is configured to capture DSRC packets

1. EV approaches the demonstration area intersection with its lights and sirens off (not requesting preemption)

2. EV approaches the demonstration area intersection with its lights and sirens on at a speed in which OBU1 broadcasts SRMs (to request preemption)

OBU1 only broadcasts SRMs when approaching the intersection advertising SRM when the EV lights and sirens are on

CVTT pcap file containing WSAs and SRMs

CVTT pcap file depicting the EV only broadcast SRMs when SRM is advertised and the lights and sirens are on




CVE-OBU013-v01 Verify the EV OBU receives an SSM


RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs and advertising SRM on Channel 180

RSU is connected to a Message Handler sending J2735 SSMs to the RSU

Message Handler is connected to a signal controller sending signal status data to the Message Handler

OBU1 (EV) is installed in an EV and configured for "normal" operations, including broadcasting BSMs on Channel 172, broadcasting SRMs on Channel 180, and logging SSMs

CVTT is configured to capture DSRC packets on Channels 172 and 180

1. EV approaches the demonstration area intersection with its lights and sirens on at a speed in which OBU1 broadcasts SRMs (to request preemption)

2. RSU forwards SRMs to Message Handler

3. Message Handler sends a preemption call to the Signal Controller

4. Signal controller sends signal status data to the Message Handler

5. Message Handler sends SSMs to the RSU

6. RSU broadcast SSMs

7. OBU1 receives and logs SSMs

OBU1 logs received SSMs OBU1 log file containing SSMs

CVTT pcap file containing WSAs, SRMs, and SSMs




CVE-OBU014-v02 Verify the transit vehicle OBU broadcasts SRMs only when approaching an equipped intersection


RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs advertising SRM on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and SRMs on Channel 180



1. OBU1 drives around the demonstration area, avoiding the intersection (not requesting priority)

2. OBU1 approaches the demonstration area intersection at a speed in which OBU1 broadcasts SRMs (to request priority)

OBU1 only broadcasts SRMs when approaching the intersection advertising SRM


CVTT pcap file depicting OBU1 only broadcast SRMs when SRM is advertised




CVE-OBU015-v01 Verify the transit vehicle OBU receives an SSM


RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM on Channel 180



OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172, broadcasting SRMs on Channel 180, and logging SSMs


1. A location is set up as a simple intersection in the demonstration area

2. RSU is broadcasting SPaT and MAP for an intersection in the demonstration area and WSAs advertising SRM on Channel 180

3. CVTT is configured to capture packets on Channel 180



6. Message Handler sends a priority call to the Signal Controller










CVE-OBU016-v01 Verify the HDV OBU broadcasts SRMs only when approaching an equipped intersection


RSU is configured for "normal" operations, including broadcasting SPaT and MAP and WSAs advertising SRM on Channel 180

OBU1 (HDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and SRMs on Channel 180





4. OBU1 drives around the demonstration area, avoiding the intersection (not requesting priority)


OBU1 only broadcasts SRMs when approaching the intersection advertising SRM


CVTT pcap file depicting OBU1 only broadcast SRMs when SRM is advertised




CVE-OBU017-v01 Verify the HDV OBU receives an SSM


RSU is configured for "normal" operations, including broadcasting SPaT, MAP, SSMs, and WSAs advertising SRM on Channel 180



OBU1 (HDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172, broadcasting SRMs on Channel 180, and logging SSMs







6. Message Handler sends a priority call to the Signal Controller










CVE-OBU018-v01 Verify a host transit vehicle OBU broadcasts SRMs, receives SSMs, and logs the TSP event with congestion on Channel 180


An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, SSM, and WSAs advertising SRM on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and SRMs on channel 180

CVTT is configured to capture DSRC packets on Channel 180

An RSU, Message Handler, and Signal Controller in the project limits are configured for Signal Priority



3. OBU1 Logs the TSP event

• OBU1 broadcasts SRMs in response to WSAs

• RSU broadcasts SSMs in response to SRM

• OBU1 logs the TSP event

• OBU1 log file containing the TSP event

• CVTT pcap containing SPaT, MAP, TIM, WSA, SSM, and SSM




CVE-OBU019-v01 Verify a host transit vehicle OBU broadcasts SRMs, receives SSMs, logs the TSP event, and downloads a firmware update with congestion on Channel 180


An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, SSM, and WSAs advertising SRM on Channel 180 and serving as an IPv6 Gateway

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and broadcasting SRMs and performing over-the-air f irmware updates on channel 180


An RSU, Message Handler, and Signal Controller in the project limits are configured for Signal Priority

1. Vehicle with OBU1 (V1) drives into communication range of the RSU

2. Vehicle with OBU1 (V1) begins downloading a f irmware update




• OBU1 downloads a firmware update

• OBU1 broadcasts SRMs in response to WSAs

• RSU broadcasts SSMs in response to SRM

• OBU1 records an TSP event

• OBU1 log file containing the TSP event and FW download

• CVTT pcap containing SPaT, MAP, TIM, SSM, SRM, and OTA messages




Wireshark Laptop

CVTT OBU

LDV

BSM

Transit Vehicle

EV

HDV

BSM

BSM

BSM


Figure 1: BSMs



Remote Vehicle

Wireshark Laptop

BSM

CVTT OBU

Transit Vehicle

OBU Log File


Figure 2: Transit OBU



MessageHandler

Traffic Signal Controller

RSU

SPaT\Map

SPaT\Map

SPaT DataWireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File


Figure 3: Transit RLVW



MessageHandler

RSU

TIM

TIM

Wireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File

School Zone SystemStart Time &

Duration


Figure 4: Transit RSSZ



MessageHandler


RSU

SPaT\Map\SSM

SPaT\Map\SSM

SPaT Data\Status Data

Wireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File

SRM

SRM

PriorityReq


Figure 5: Transit Signal Priority



MessageHandler


RSU

SPaT\MapTIM\SSM

SPaT\Map\TIM\SSM


Wireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File

SRM

SRM

PriorityReq


Duration


Figure 6: Transit V2I



RSU

Wireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File

OBU Log FIles

TrCVMS

OBU Log Files


Figure 7: Transit TrCVMS



MessageHandler


RSU

SPaT\Map\SSM

SPaT\Map\SSM


Wireshark Laptop

CVTT OBU

SRM

SRM

PreemptReq

EV


Figure 8: EV V2I



MessageHandler


RSU

SPaT\Map\SSM

SPaT\Map\SSM


Wireshark Laptop

CVTT OBU

SRM

SRM

PreemptReq

HDV


Figure 9: HDV V2I



MessageHandler


RSU

SPaT\MapTIM\SSM

SPaT\Map\TIM\SSM\FW Update


Wireshark Laptop

CVTT OBU

Transit Vehicle

OBU Log File

SRM\FW Request

SRM

PriorityReq

Transit OBUFirmware Server

FW Update


Duration


Figure 10: Transit V2I + OTA



8.2. ONBOARD UNIT APPLICATION

Table 16: Onboard Unit Application Test Procedures


CVE-App001-v01 Verify a host LDV OBU issues an EEBL warning when it receives a BSM from a remote OBU (under conditions in which an EEBL warning should be issued)


OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172

OBU2 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing EEBL alerts to the driver

A location is set up as a normal roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) on the road segment

2. V1 performs a hard-braking maneuver such that an EEBL event flag is set in OBU1’s BSM.

3. Based on BSMs from OBU1, OBU2 determines a threat exists, and issues an EEBL alert to the driver

4. Data recorder logs an EEBL alert in the test log

OBU2 issues an EEBL alert Data recorder test logs indicating EEBL alert was issued

CVE-App002-v01 Verify a host LDV OBU issues an FCW warning when it receives a BSM from a remote OBU (under conditions in which an FCW warning should be issued)



OBU2 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing FCW alerts to the driver

A location is set up as a normal roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) is stopped on the roadway segment

2. Vehicle with OBU2 (V2) approaches V1 from behind

3. Base on BSMs from OBU1, OBU2 determines a threat exists and issues an FCW alert

4. Data recorder logs an FCW alert in the test log

OBU2 issues an FCW alert Data recorder test logs indicating FCW alert was issued




CVE-App003-v01 Verify a host LDV OBU issues an LCW warning when it receives a BSM from a remote OBU (under conditions in which an LCW warning should be issued)



OBU2 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing LCW alerts to the driver

A location is set up as a typical two lane (same direction of travel) roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) drives next to the vehicle with OBU2 (V2), on the left side

2. V2 attempts to change lanes to the left

3. Based on BSMs from OBU1, OBU2 determines a threat exists and issues an LCW alert

4. Data recorder logs an LCW alert in the test log

OBU2 issues an LCW alert Data recorder test logs indicating LCW alert was issued

CVE-App004-v01 Verify a host LDV OBU issues an IMA warning when it receives a BSM from a remote OBU (under conditions in which an IMA warning should be issued)



OBU2 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing IMA alerts to the driver

A location is set up as a normal two lane (opposite direction of travel) roadway segment in the demonstration area

1. Vehicle with OBU1 (V1) approaches vehicle with OBU2 (V2) in the lane to the lef t of V2

2. V2 begins to turn left in front of V1

3. Based on BSMs from OBU1, OBU2 issues an IMA alert

4. Data recorder logs an IMA alert in the test log

OBU2 issues an IMA alert Data recorder test logs indicating IMA alert was issued




CVE-App005-v01 Verify a host LDV OBU issues an RLVW warning when it receives SPaT and MAP from an RSU (under conditions in which an RLVW warning should be issued)



OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing RLVW alerts to the driver


1. Vehicle with OBU1 (V1) approaches the intersection at a speed in which the light will be Red when V1 arrives at the intersection


3. Based on SPaT and MAP from the RSU, OBU1 determines a threat exists and issues a RLVW alert

4. Data recorder logs an RLVW alert in the test log

OBU1 issues an RLVW alert Data recorder test logs indicating RLVW alert was issued

CVE-App006-v01 Verify a host LDV OBU issues an RSSZ warning when it receives a TIM f rom an RSU (under conditions in which an RSSZ warning should be issued)


RSU is configured for "normal" operations, including broadcasting a TIM representing an RSSZ on Channel 180

OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing RSSZ alerts to the driver


1. Vehicle with OBU1 (V1) approaches the school zone segment above the school zone speed limit

2. Based on the TIM from the RSU, OBU1 determines a threat exists and issues and RSSZ alert

3. Data recorder logs an RSSZ alert in the test log

OBU1 issues an RSSZ alert Data recorder test logs indicating RSSZ alert was issued




CVE-App008-v01 Verify a high-priority warning takes precedence over a low priority warning when both warning types are present.


OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and displaying RSSZ alerts to the driver

OBU2 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and displaying EEBL and RSSZ alerts to the driver

RSU is configured for "normal" operations, including broadcasting a TIM representing an RSSZ on Channel 180


1. Vehicle with OBU1 (V1) leads vehicle with OBU2 (V2) in the school zone driving above the school zone speed limit

2. Both V1 and V2 drivers receive RSSZ alerts

3. While in the school zone, V1 performs a hard-braking maneuver such that an EEBL event is contained its BSM.

4. Verify the HMI in V2 only shows an EEBL warning

V2 only shows EEBL alert with and EEBL and RSSZ alerts present

Data recorder test logs indicating an EEBL alert was issued

CVE-App009-v01 Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when SPaT, MAP, TIM, and WSAs are broadcast on Channel 180 from the same RSU

1-LDV OBU (RLVW)

1 RSU (SPaT, MAP, TIM, WSA)


OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and displaying RLVW alerts to the driver

An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, and WSAs on Channel 180

CVTT configured to capture messages on Channel 180




4. Data recorder observes a RLVW alert on OBU1’s HMI and logs the alert in the test log

RLVW alert appears on OBU1 HMI

Data recorder test logs indicating RLVW alert was issued

CVTT pcap containing SPaT, MAP, TIM, and WSA




CVE-App010-v01 Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a f irmware update is in process and SPaT, MAP, TIM, and WSAs are broadcast on Channel 180 f rom the same RSU


1 RSU (SPaT, MAP, TIM, WSA)


OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172, downloading f irmware updates on Channel 180, and displaying RLVW alerts to the driver

An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, WSAs, and supporting IPv6 services on Channel 180










OBU1 log file containing the FW downloaded

CVTT pcap containing SPaT, MAP, TIM, WSA, and OTA messages




CVE-App0011v01 Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU

1-LDV OBU (RLVW)

1-Transit OBU (SRM)

1 RSU (SPaT, MAP, TIM, SSM, WSA)


LDV OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and displaying RLVW alerts to the driver

Transit OBU2 is configured for “normal” operation, including broadcasting BSMs on Channel 172 and SRMs on Channel 180

An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, SSM, and WSAs on Channel 180


Message Handler and Signal Controller in the project limits are configured for Signal Priority








OBU2 request Signal Priority

OBU2 logs the TSP event


OBU2 log file containing the TSP Event

CVTT pcap containing SPaT, MAP, TIM, WSA, SSM, and SRM




CVE-App012-v01 Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, a firmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU

1-LDV OBU (RLVW)

1-Transit OBU (SRM and OTA)



LDV OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172 and displaying RLVW alerts to the driver

Transit OBU2 is configured for “normal” operation, including broadcasting BSMs on Channel 172, downloading f irmware updates, and broadcasting SRMs on Channel 180

An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, TIM, SSM, and WSAs and supporting IPv6 on Channel 180















OBU2 log file containing the TSP Event

CVTT pcap containing SPaT, MAP, TIM, WSA, SSM, SRM, and OTA messages




CVE-App013-v01 Verify a host LDV OBU issues an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 2 OBUs have a firmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU


1-Transit OBU (SRM, OTA)



LDV OBU1 is configured for "normal" operation, including broadcasting BSMs on Channel 172, downloading f irmware updates on Channel 180, and displaying RLVW alerts to the driver



















OBU1 log file containing the FW download

OBU2 log file containing the TSP Event and FW download





CVE-App014-v01 Verify 2 host LDV OBUs issue an RLVW warning (under conditions in which an RLVW warning should be issued) when a Transit vehicle OBU is broadcasting SRMs, 3 OBUs have a firmware update is in process, and SPaT, MAP, TIM, SSMs, and WSAs are broadcast on Channel 180 from the same RSU





LDV OBU1 and OBU2 are configured for "normal" operation, including broadcasting BSMs on Channel 172, downloading f irmware updates on Channel 180, and displaying RLVW alerts to the driver





1. Vehicle with OBU1 (V1) and OBU2 (V2) drives into communication range of the RSU

2. OBU1 and OBU2 begin downloading a firmware update

3. V1 and V2 approach the intersection at a speed in which the light will be Red when they arrive

4. V1 and V2 maintain constant speed and heading


6. OBU3 begins downloading a f irmware update

7. V3 approaches the intersection at a speed in which the light will be Red when vehicle arrives at the intersection

8. Based on SPaT and MAP from the RSU, OBU1 and OBU2 determine a threat exists and issues a RLVW alert


10. Data recorder1 observes a RLVW alert on OBU1 HMI and logs the alert in the test log

RLVW alert appears on OBU1 and OBU2 HMI



Data recorder test logs indicating OBU1 issued and RLVW















LDV OBU1, OBU2, and OBU3 are configured for "normal" operation, including broadcasting BSMs on Channel 172, downloading f irmware updates on Channel 180, and displaying RLVW alerts to the driver





1. Vehicle with OBU1 (V1), OBU2 (V2), and OBU3 (V3) drive into communication range of the RSU

2. OBU1, OBU2, and OBU3 begin downloading a firmware update

3. V1, V2, and V3 approach the intersection at a speed in which the light will be Red when they arrive

4. V1, V2, and V3 maintain constant speed and heading



7. V4 approaches the intersection at a speed in which the light will be Red when vehicle arrives at the intersection

8. Based on SPaT and MAP from the RSU, OBU1, OBU2, and OBU3 determine a threat exists and issues a RLVW alert


RLVW alert appears on OBU1, OBU2, and OBU3 HMI






















LDV OBU1, OBU2, OBU3, and OBU4 are configured for "normal" operation, including broadcasting BSMs on Channel 172, downloading f irmware updates on Channel 180, and displaying RLVW alerts to the driver




1. Vehicle with OBU1 (V1), OBU2 (V2), OBU3 (V3), and OBU4 (V4) drive into communication range of the RSU

2. OBU1, OBU2, OBU3, and OBU4 begin downloading a f irmware update

3. V1, V2, V3, and V4 approach the intersection at a speed in which the light will be Red when they arrive

4. V1, V2, V3, and V4 maintain constant speed and heading



7. V5 approaches the intersection at a speed in which the light will be Red

RLVW alert appears on OBU1, OBU2, OBU3, and OBU4 HMI
















when vehicle arrives at the intersection

8. Based on SPaT and MAP from the RSU, OBU1, OBU2, OBU3, and OBU4 determine a threat exists and issues a RLVW alert










Remote Vehicle

Wireshark Laptop

CVTT OBU

LDV

LDV HMI

BSM


Figure 11: LDV V2V



MessageHandler


RSU

SPaT\Map

SPaT\Map

SPaT DataWireshark Laptop

CVTT OBU

LDV

LDV HMI


Figure 12: LDV RLVW



MessageHandler

RSU

TIM

TIM

Wireshark Laptop

CVTT OBU


Duration

LDV

LDV HMI


Figure 13: LDV RSSZ



MessageHandler

RSU

TIM

TIM

Wireshark Laptop

CVTT OBU


Duration

LDV

LDV HMI

Remote Vehicle


Figure 14: LDV App Priority



MessageHandler


RSU

SPaT\Map\TIM

SPaT\Map\TIM\WSA

SPaT Data

Wireshark Laptop

CVTT OBU


Duration

LDV

LDV HMI


Figure 15: LDV V2I



MessageHandler


RSU

SPaT\Map\TIM

SPaT\Map\TIM\WSA\FW Update

SPaT Data

Wireshark Laptop

CVTT OBU

LDV OBUFirmware Server

FW Update

LDV

LDV HMI


Duration

FW Request


Figure 16: LDV V2I + OTA



MessageHandler


RSU

SPaT\Map\TIM\WSA\SSM

Wireshark Laptop

CVTT OBU

LDV

LDV HMI


Duration

Transit Vehicle

OBU Log File

SRM

SPaT\MapTIM\SSM


SRM

PriorityReq


Figure 17: LDV RLVW+SRM



MessageHandler


RSU

SPaT\Map\TIM\WSA\SSM\FW Update

Wireshark Laptop

CVTT OBU

LDV

LDV HMI


Duration

Transit Vehicle

OBU Log File

SPaT\MapTIM\SSM


SRM

PriorityReq

SRM\FW request


FW Update


Figure 18: LDV RLVW+SRM+OTA



MessageHandler


RSU

SPaT\Map\TIM\WSA\SSM\FW Update

Wireshark Laptop

CVTT OBU

LDV

LDV HMI


Duration

Transit Vehicle

OBU Log File

SPaT\MapTIM\SSM


SRM

PriorityReq


FW Update

SRM\FW request

FW request


FW Update


Figure 19: LDV RLVW+SRM+2OTA



8.3. ROADSIDE UNIT

Table 17: Roadside Unit Test Procedures


CVE-RSU001-v02 Verify RSU forwards BSMs, received from passing vehicles, to Message Handler.


RSU is configured for "normal" operation, including forwarding BSMs to the Message Handler

RSU is connected to a network capable of accessing the Message Handler

OBU is configured for "normal" operation, including broadcasting BSMs on Channel 172

Message Handler is configured for "normal" operations, including accepting BSMs from the local RSU

Message Handler is connected to a network capable of accessing the RSU

1. OBU, broadcasting BSMs, is brought into communication range of the RSU Under Test

2. The Message Handler is monitored to verify it is receiving BSMs

Message Handler successfully receives BSMs from the RSU/OBU

OBU log file containing transmitted BSMs

Message Handler log file containing BSMs

CVE-RSU002-v01 Verify RSU forwards SRMs,

received from approaching vehicles, to the Message Handler


RSU is configured for

"normal" operation, including forwarding SRMs to the Message Handler

RSU is connected to a Message Handler

OBU is configured to broadcast SRMs continually (or on-demand) on Channel 180

1. OBU, broadcasting SRMs, is

brought into communication range of the RSU Under Test

2. The Message Handler is monitored to verify it is receiving SRMs

Message Handler successfully

receives SRMs

OBU log file containing

SRMs

Message Handler log file containing SRMs




CVE-RSU003-v01 Verify RSUs broadcast SPaT messages received from the Message Handler over DSRC on Channel 180 at 10 Hz


RSU is configured for "normal" operation, including broadcasting SPaT on Channel 180 using the Immediate Forward functionality

RSU is connected to a Message Handler sending J2735 SPaT messages


1. Begin broadcasting SPaT f rom the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting SPaT based on the messages provided by the Message Handler

CVTT successfully captures SPaT messages from the RSU

CVTT pcap file containing SPaT

CVE-RSU004-v01 Verify SPaT broadcast by RSUs match the signal head status



RSU is connected to a Message Handler sending J2735 SPaT and MAP Messages


1. Begin broadcasting SPaT f rom the Message Handler

2. Begin broadcasting MAP from the Message Handler

3. RSU broadcasts SPaT and MAP

4. Using the CVTT SPaT/MAP visualization functionality, monitor the CVTT to verify SPaT status matches the signal head status for all lanes within the intersection

SPaT status matches the signal head status

CVTT log file containing SPaT and MAP messages




CVE-RSU006-v02 Verify RSUs broadcast MAP messages received from the Message Handler over DSRC on Channel 180 at 10 Hz


RSU is configured for "normal" operation, including broadcasting MAP messages on Channel 180 using the Immediate Forward functionality

RSU is connected to a Message Handler sending J2735 MAP Messages

Message Handler contains a MAP message for its installation intersection


1. Begin broadcasting the MAP message from the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting the MAP message provided by the Message Handler

CVTT successfully captures MAP Messages from the RSU

CVTT pcap file containing the MAP messages

CVE-RSU007-v02 Verify MAP message matches intersection geometry


RSU is configured for "normal" operation, including broadcasting MAP messages on Channel 180 using the Immediate Forward functionality

RSU is connected to a Message Handler sending J2735 MAP Messages

Message Handler contains a MAP message for its installation intersection

Message Handler is configured for “normal” operations including sending MAP messages to the RSU as Immediate Forward messages


1. Begin broadcasting the MAP message from the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting the MAP message provided by the Message Handler

3. Using the CVTT MAP display functionality verify the MAP message matches/aligns with the RSU installation intersection geometry

The MAP message displayed by the CVTT matches the lanes and approaches for RSUs installation intersection

Screen shot of the MAP displayed by the CVTT




CVE-RSU008-v01 Verify RSUs broadcast RTCM messages received from the Message Handler over DSRC on Channel 180 once every 2 seconds


RSU is configured for "normal" operation, including broadcasting RTCMs on Channel 180 using the Immediate Forward functionality

RSU is connected to a Message Handler sending J2735 RTCM messages

Message Handler is configured for “normal” operations including generating and sending RTCM messages to the RSU as Immediate Forward messages


1. Begin broadcasting RTCMs f rom the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting RTCMs based on the messages provided by the Message Handler

CVTT successfully captures RTCM Messages from the RSU

CVTT pcap file containing RTCM




CVE-RSU009-v01 Verify RSUs broadcast SSMs received from the Message Handler over DSRC on Channel 180 at 1 Hz


RSU is configured for "normal" operation, including broadcasting SSMs on Channel 180 using the Immediate Forward functionality

RSU is connected to a Message Handler sending J2735 SSM messages

Message Handler is configured for “normal” operations including generating and sending SSMs to the RSU as Immediate Forward messages


1. Begin broadcasting SSMs f rom the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting SSM based on the messages provided by the Message Handler

CVTT successfully captures SSMs from the RSU

CVTT pcap file containing SSMs

CVE-RSU010-v01 Verify RSUs broadcast TIMs on Channel 180 at 1 Hz.


RSU is configured for "normal" operation, including broadcasting TIMs on Channel 180 using the Immediate Forward functionality

RSU is connected to the Message Handler

Message Handler is configured for "normal" operations, including sending TIMs to the RSU as Immediate Forward messages


1. Begin broadcasting the TIM message from the Message Handler

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting the TIM message provided by the Message Handler based on the message stored on the device

3. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting TIMs based on the messages provided by the Message Handler

RSU broadcasts TIM (for RSSZ) CVTT pcap containing TIMs (for RSSZ)




CVE-RSU011-v01 Verify RSUs report Channel Busy Ratio to TCVMS, when above a predetermined threshold


RSU (1) is configured for "normal" operation, including broadcasting WSA, SPaT, MAP, and TIM on Channel 180

RSU (2) is configured for "normal" operation, including detecting and reporting Channel Busy Ratio

TCVMS is configured for "normal" operation, including receiving and logging Channel Busy Ratio


1. RSU (2) is configured for a Channel Busy Raito threshold of 5% (which should trigger a Channel Busy Report with minimal DSRC activity)

2. Using the CVTT message capture functionality capture WSA, MAP, and TIMs from RSU (1)

3. Begin broadcasting WSA, MAP, and TIM from RSU (1)

4. Monitor RSU (2) to verify it detects a Channel Busy Ratio above 5% and sends a Channel Busy Report to the TCVMS

RSU (2) sends Channel Busy Report to TCVMS

RSU log file containing the Channel Busy Report indicating Channel Busy Raito is above 5%, sent to the TCVMS

TCVMS log file containing Channel Busy Report indicating Channel Busy Raito is above 5%

CVTT pcap file containing WSA, MAP, and TIM from RSU (1)

CVE-RSU012-v01 Verify RSU broadcast WSA with

correct parameters (including WRA)


RSU is configured for "normal"

operation, including broadcasting WSAs containing WRAs on Channel 180

1. Begin broadcasting WSAs

f rom the RSU

2. Using the CVTT message capture functionality, monitor the RSU to verify it is broadcasting WSAs containing the correct parameters, including the WRA

CVTT successfully captures

WSA containing the correct parameters, including the WRA

CVTT pcap file containing WSAs

CVE-RSU013-v01 Verify IPv6 connectivity to SCMS


RSU is configured for "normal" operation, including requesting new certificates f rom the SCMS

RSU is connected to a network capable of accessing the SCMS

1. Remove certificates from the RSU

2. From the RSU, initiate a certificate request to the SCMS

3. Monitor the certificate directory on the RSU to verify the RSU has received and stored new certificates

RSU successfully requests, receives, and stores certificates f rom the SCMS

Screen shot of the RSU directory containing the new certificates




CVE-RSU014-v01 Document RSU Communication Range for RLVW on Channel 180


An RSU in the project limits is configured for "normal" operation, including broadcasting SPaT and MAP on Channel 180

OBU1 (LDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing RLVW alerts to the driver

CVTT is configured to log the Latitude, Longitude, and Receive Signal Strength of where each SPaT and Map message are received on Channel 180




4. Data recorder logs an RLVW alert in the test log

5. (post-process) Data recorder plots the RSU Latitude and Longitude and the Latitude, Longitude, and Receive Signal Strength of each received SPaT and Map message.

6. Data recorder compares the Latitude and Longitude of the RSU to the Latitude and Longitude of the first received SPaT and Map message,

7. Data recorder records the distance between the RSU and the first received SPaT and Map messages as the RSU RLVW Range in the test log

OBU1 issues an RLVW alert

CVTT log file contains the latitude, longitude, and Receive Signal Strength of where each SPaT and Map message are received

Data recorder test logs indicating an RLVW alert was issued

Data recorder test logs indicating an RSU RLVW Range

CVTT log file containing the latitude, longitude, and Receive Signal Strength of where each SPaT and Map message is received




CVE-RSU015-v01 Document RSU Communication Range for Signal Priority on Channel 180


An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, SSM, and WSAs advertising SRM on Channel 180

OBU1 (transit) is configured for "normal" operation, including broadcasting BSMs on Channel 172, SRMs on Channel 180, and logging TSP events


CVTT is configured to log the Latitude, Longitude, and Receive Signal Strength of where each SPaT, Map, and WSA message are received on Channel 180




4. (post-process) Data recorder plots the RSU Latitude and Longitude and the Latitude, Longitude, and Receive Signal Strength of each received SPaT, Map, and WSA message.

5. Data recorder compares the Latitude and Longitude of the RSU to the Latitude and Longitude of the first received SPaT, Map, and WSA message

6. Data recorder records the distance between the RSU and the first received SPaT, Map, and WSA messages as the RSU Signal Priority Request Range in the test log

OBU1 records an TSP event


Data recorder test logs indicating an RSU Signal Priority Request Range

CVTT log file containing the latitude, longitude, and Receive Signal Strength of where each SPaT, Map, WSA message is received




CVE-RSU016-v01 Document RSU Communication Range for Signal Preemption on Channel 180


An RSU in the project limits is configured for "normal" operations, including broadcasting SPaT, MAP, SSM, and WSAs advertising SRM on Channel 180

OBU1 (EV) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and SRMs on Channel 180

Message Handler and Signal Controller in the project limits are configured for Signal Preemption

CVTT is configured to log the Latitude, Longitude, and Receive Signal Strength of where each SPaT, Map, and WSA message are received on channel 180


2. Based on the SPaT, MAP, and WSA from the RSU, OBU1 broadcasts an SRM, requesting signal preemption

3. (post-process) Data recorder plots the RSU Latitude and Longitude and the Latitude, Longitude, and Receive Signal Strength of each received SPaT, Map, and WSA message.

4. Data recorder compares the Latitude and Longitude of the RSU to the Latitude and Longitude of the first received SPaT, Map, and WSA message

5. Data recorder records the distance between the RSU and the first received SPaT, Map, and WSA messages as the RSU Signal Preemption Request Range in the test log


Data recorder test logs indicating an RSU Signal Preemption Request Range

CVTT log file containing the latitude, longitude, and Receive Signal Strength of where each SPaT, Map, and WSA message is received




CVE-RSU017-v01 Document RSU Communication Range for RSSZ on Channel 180


An RSU in the project limits is configured for "normal" operation, including broadcasting TIM, representing an RSSZ, on Channel 180

OBU1 (LDV) is configured for "normal" operation, including broadcasting BSMs on Channel 172 and providing RSSZ alerts to the driver

CVTT is configured to log the Latitude, Longitude, and Receive Signal Strength of where each TIM is received on Channel 180

1. Vehicle with OBU1 (V1) approaches the school zone segment above the school zone speed limit

2. Based on the TIM from the RSU, OBU1 determines a threat exists and issues and RSSZ alert

3. Data recorder logs an RSSZ alert in the test log

4. (post-process) Data recorder plots the RSU Latitude and Longitude and the Latitude, Longitude, and Receive Signal Strength of each received TIM.

5. Data recorder compares the Latitude and Longitude of the RSU to the Latitude and Longitude of the first received TIM.

6. Data recorder records the distance between the RSU and the first received TIM as the RSU RSSZ Range in the test log

OBU1 issues an RSSZ alert


Data recorder test logs indicating an RSSZ alert was issued

Data recorder test logs indicating an RSU RSSZ Range

CVTT log file containing the latitude, longitude, and Receive Signal Strength of where each SPaT and Map message are received




MessageHandler

RSU

Wireshark Laptop

CVTT OBU

LDV

Transit Vehicle

OBU Log File

SRM/BSM

SRM/BSM

BSM


Figure 20: RSU Message Forward



MessageHandler


RSU

SPaT\Map\TIM\WSA\SSM\RTCM\FW Update

Wireshark Laptop

CVTT OBU

LDV

School Zone System

Start Time &Duration

Transit Vehicle

SPaT\MapTIM\SSM\RTCM


SRM

PriorityReq


FW Update

SRM

FW request

RTCMRTCM


Figure 21: RSU Broadcast



8.4. MESSAGE HANDLER

Table 18: Message Handler Test Procedures


CVE-MH001-v01 Verify the Message Handler generates and forwards SAE J2735-2016 SPaT messages, based on SPaT data received f rom the local Signal Controller, to the local RSU


Message Handler is configured for "normal" operation, including generating J2735 SPaT messages and sending them to an RSU

Message Handler is connected to a network capable of accessing the Signal Controller

Signal controller is configured for "normal" operation, including sending SPaT data to the Message Handler

Signal controller is connected to a network capable of accessing the Message Handler

1. Connect a laptop (L1) running Wireshark between the Signal Controller and the Message Handler such that data can f low between the Signal Controller and the Message Handler and Wireshark can monitor the data packets.

2. Log received packets on L1

3. Power on the Message Handler

4. Power on the Signal Controller

5. Connect a second laptop (L2) running Wireshark to the RSU port on the Message Handler (L2 acts as the RSU)

6. Configure Wireshark on L2 to capture packets from the Message Handler


8. Monitor Wireshark on L1 to be sure data is flowing between the Signal Controller and the Message Handler

9. Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 SPaT messages

Wireshark captures J2735 SPaT messages from the Message Handler

pcap file from L2 containing SPaT messages sent by the Message Handler

pcap file from L1 containing data sent to the Message Handler by the Signal Controller




CVE-MH002-v01 Verify the Message Handler properly determines if OBU requesting priority/preemption is authorized to receive priority/preemption


Message Handler is configured for "normal" operation, including distinguishing between authorized priority/preemption requests and unauthorized priority/preemption requests based on the 1609.2 certificate


RSU is configured for "normal" operation, including forwarding J2735 SRMs to the Message Handler


OBU1 is configured to broadcast SRMs signed with a signal priority certificate, requesting signal priority (authorized)

OBU2 is configured to broadcast SRMs signed with a signal priority certificate, requesting signal preemption (unauthorized)

1. Power on RSU and Message Handler

2. Power on OBU1 and OBU2

3. OBU1 broadcasts SRMs signed with a signal priority certificate, requesting signal priority (authorized)

4. OBU2 broadcasts SRMs signed with a signal priority certificate, requesting signal preemption (unauthorized)

5. RSU forwards SRMs from both OBU1 and OBU2 to the Message Handler

6. Message Handler logs authorized signal priority requests and unauthorized signal preemption requests

Message Handler authorizes OBU1 to request priority

Message Handler denies OBU2 from requesting priority

Message Handler log file identifying authorized and unauthorized priority requests




CVE-MH004-v01 Verify the Message Handler places a priority call to the local Signal Controller based on an authorized SRM.


Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a priority call to a Signal Controller




OBU1 is configured to broadcast authorized J2735 SRMs requesting priority


2. Connect a laptop (L1) running Wireshark to the Signal Controller port of the Message Handler (L1 acts as the signal controller)



5. Power on OBU1

6. OBU1 broadcasts SRMs signed with a signal priority certificate

7. In Wireshark on L1, inspect packets coming from the Message Handler to verify received packets contain priority calls from OBU1

8. Message Handler logs signal priority requests from OBU1 and signal priority calls sent to the Signal Controller (L1)

Message Handler sends priority calls from OBU1 to the Signal Controller

Message Handler log file containing SRMs received f rom the RSU and priority calls sent to the Signal Controller

Wireshark pcap files containing signal priority calls f rom OBU1




CVE-MH005-v01 Verify the Message Handler places a preemption call to the local Signal Controller based on an authorized SRM


Message Handler is configured for "normal" operation, including receiving SRMs from and RSU and placing a preemption call to a signal controller




OBU1 is configured to broadcast authorized J2735 SRMs requesting preemption


2. Connect a laptop (L1) running Wireshark to the Signal Controller port of the Message Handler (L1 acts as the signal controller)



5. Power on OBU1

6. OBU1 broadcasts SRMs signed with a signal preemption certificate

7. In Wireshark on L1, inspect packets coming from the Message Handler to verify received packets contain preemption calls from OBU1

8. Message Handler logs signal preemption requests from OBU1 and signal preemption calls sent to the Signal Controller (L1)

Message Handler sends preemption calls from OBU1 to the Signal Controller (L1)

Message Handler log file containing SRMs received f rom the RSU and preemption calls sent to the Signal Controller (L1)

Wireshark pcap files containing signal preemption calls from OBU1




CVE-MH006-v01 Verify the Message Handler generates and forwards SAE J2735-2016 SSMs, based on status data received from the local Signal Controller, to the local RSU


Message Handler is configured for "normal" operation, including generating J2735 SSM messages and sending them to an RSU

Message Handler is connected to a network capable of accessing the Signal Controller

Signal controller is configured for "normal" operation, including sending priority/preemption request status data to the Message Handler

Signal controller is connected to a network capable of accessing the Message Handler








8. Monitor Wireshark on L1 to be sure data is flowing between the Signal Controller and the Message Handler

9. Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 SSM messages

Wireshark captures J2735 SPaT messages from the Message Handler

pcap file from L2 containing SPaT messages sent by the Message Handler

pcap file from L1 containing data sent to the Message Handler by the Signal Controller




CVE-MH007-v01 Verify the Message Handler reports a cabinet tamper events to TCVMS


Message Handler is configured for "normal" operation, including detecting cabinet tamper events and sending events to the TCVMS

Message Handler is connected to a network capable of accessing the TCVMS

TCVMS is configured for "normal" operation, including receiving cabinet tamper events

TCVMS is connected to a network capable of accessing the Message Handler

1. Connect a toggle switch (S1) to the cabinet tamper detection input/output (IO) port of the Message Handler

2. Set S1 to "inactive" (no event detected)

3. Connect a laptop (L1) running Wireshark to the TCVMS port on the Message Handler (L1 acts as the TCVMS)



6. In Wireshark on L1, inspect packets coming from the Message Handler in Wireshark to verify the Message Handler is NOT sending event messages

7. Set S1 to "active" (event detected)

8. In Wireshark on L1, inspect packets coming from the Message Handler to verify the Message Handler is sending event messages

9. Message Handler logs events based on S1 set to "active"

Wireshark captures event messages from the Message Handler when an event is detected

Message Handler log file containing event messages sent to the TCVMS

Wireshark pcap files containing event messages sent by the Message Handler to the TCVMS




CVE-MH008-v01 Verify the Message Handler generates and forwards SAE J2735-2016 RTCM messages, based on data received from the Position Correction System to the local RSU


Message Handler is configured for "normal" operation, including generating J2735 RTCM messages and sending them to an RSU

Message Handler is connected to a network capable of accessing a RTCM source

1. Connect a laptop (L1) running Wireshark between the RTCM source and the Message Handler such that data can flow between the RTCM source and the Message Handler and Wireshark can monitor the data packets.







8. Monitor Wireshark on L1 to be sure data is flowing between the RTCM source and the Message Handler

9. Inspect packets coming from the Message Handler on L2 to verify they conform to J2735 RTCM messages

Wireshark captures J2735 RTCM messages from the Message Handler

pcap file from L2 containing RTCM messages sent by the Message Handler

pcap file from L1 containing data sent to the Message Handler by the RTCM source




CVE-MHTP009-v01 Verify Message Handler stores MAP files received from the TCVMS


Message Handler is configured for "normal" operation, including accepting MAP files from the TCVMS


TCVMS is configured for "normal" operations, including sending MAP files to Message Handlers


A MAP file is loading on the TCVMS for the Message Handler under test

1. From TCVMS, send the MAP f ile to the Message Handler

2. Inspect Message Handler to verify the MAP file is loaded in the appropriate location (directory)

MAP file is successfully loaded on the Message Handler from the TCVMS

MAP file from TCVMS

Screen shot of Message Handler directory containing the MAP file

CVE-MHTP010-v01 Verify Message Handler sends MAP messages to the local RSU as Immediate Forward messages


Message Handler is configured for "normal" operation, including sending MAP messages to a local RSU at 1 Hz as Immediate Forward messages

1. Connect a laptop (L1) running Wireshark to the RSU port on the Message Handler (L1 acts as the RSU)


3. Begin sending MAP messages from the Message Handler to L1 (acting as an RSU)


5. Inspect packets coming from the Message Handler on L1 to verify the Message Handler is sending J2735 MAP messages

Message Handler sends the MAP message to L1 at 1 Hz as Immediate Forward messages

Pcap from Wireshark capture on L1 containing MAP messages sent from the Message Handler




CVE-MHTP011-v01 Verify Message Handler sends TIM messages to the local RSU as Immediate Forward messages


Message Handler is configured for "normal" operation, including sending TIM messages to a local RSU at 1 Hz as Immediate Forward messages



3. Begin sending TIMs from the Message Handler to L1 (acting as an RSU)


5. Inspect packets coming from the Message Handler on L1 to verify the Message Handler is sending J2735 TIM messages

Message Handler sends the TIM message to L1 at 1 Hz as Immediate Forward messages

Pcap from Wireshark capture on L1 containing TIM messages sent from the Message Handler




CVE-MHTP012-v01 Verify Message Handler updates the Start Time and Duration of a stored TIM when the school zone is active


Current time is within 5 minutes of a School Zone Speed Limit being active

Message Handler is configured for "normal" operation, including updating the Start Time and Duration of a stored TIM and sending TIMs to the RSU as Immediate Forward messages

Message Handler is connected to a network capable of accessing the School Zone System

School Zone System is connected to a network capable of accessing the Message Handler



3. Begin sending TIMs from the Message Handler to L1

4. Log received packets on L1 both before and after the School Zone Speed Limit becomes active

5. Inspect packets coming from the Message Handler on L1 to verify the TIM contains the Start Time and Duration of the active School Zone Speed Limit

Message Handler updates the Start Time and Duration of the stored TIM

Message Handler sends the TIM to L1 at 1 Hz as Immediate Forward messages

Pcap from Wireshark capture on L1 containing TIM from the Message Handler with the Start Time and Duration of the most recent School Zone Speed Limit




Traffic Signal Controller Wireshark Laptop(s)

RSU

SRM (authorized)

Transit Vehicle

SRM (unauthorized)

EV

MessageHandler

On

Door OpenSwitch

Position Correction System

TCVMS

School Zone System

Start Time &Duration


Figure 22: Message Handler



8.5. TRAFFIC SIGNAL CONTROLLER

Table 19: Traffic Signal Controller Test Procedures


CVE-TSC001-v01 Verify Traffic Signal Controller sends SPaT data to the Message Handler


Signal controller is configured for "normal" operation, including sending SPaT data to the Message Handler

1. Connect a laptop (L1) running Wireshark between the Signal Controller to the Message Handler port (L1 acts as Message Handler)



4. Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is send SPaT data

Wireshark captures J2735 SPaT data from the Signal Controller

pcap file from L1 containing SPaT data sent from the Signal Controller to the Message Handler




CVE-TSC002-v01 Verify Traffic Signal Controller processes a signal priority call f rom the Message Handler



Message Handler is connected to the RSU


Signal controller is configured for "normal" operation including processing priority calls from the Message Handler

Signal controller is connected to the Message Handler


1. Power on RSU, Message Handler, OBU1, and Signal Controller


3. RSU forwards SRM to the Message Handler

4. Message Handler sends a signal priority call to the Signal Controller

5. Signal controller changes timing based on the priority call

6. Data recorder log the time change in the test log

Signal controller change timing based on the priority call

Test log indicating Signal Controller changes timing appropriately




CVE-TSC003-v01 Verify Traffic Signal Controller processes a signal preemption call from the Message Handler





Signal controller is configured for "normal" operation including processing priority calls from the Message Handler




2. OBU1 broadcasts SRMs signed with a signal preemption certificate


4. Message Handler sends a signal preemption call to the Signal Controller

5. Signal controller changes timing based on the preemption call

6. Data recorder logs the time change in the test log

Signal controller change timing based on the preemption call

Test log indicating Signal Controller changes timing appropriately




CVETSC004-v01 Verify Traffic Signal Controller sends priority status to the Message Handler





Signal controller is configured for "normal” operation including processing priority calls from the Message Handler








6. Message Handler sends a signal priority call to the Signal Controller

7. Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is sending signal priority status data to the Message Handler

Signal controller send signal priority status to the Message Handler

pcap file from L1 containing signal status data sent from the Signal Controller to the Message Handler




CVE-TSC005-v01 Verify Traffic Signal Controller sends preemption status to the Message Handler


Message Handler is configured for "normal" operation, including receiving SRMs from an RSU and placing a preemption call to a Signal Controller



Signal controller is configured for "normal" operation including processing preemption calls from the Message Handler






4. OBU1 broadcasts SRMs signed with a certificate authorizing it request preemption


6. Message Handler sends a signal preemption call to the Signal Controller

7. Inspect packets coming from the Signal Controller on L1 to verify the Signal Controller is sending signal preemption status data to the Message Handler

Signal controller send signal priority status to the Message Handler

pcap file from L1 containing signal status data sent from the Signal Controller to the Message Handler




Wireshark Laptop

SRM

Transit VehicleMessageHandler

Traffic Signal ControllerRSU


Figure 23: Traffic Signal Controller



8.6. TRAFFIC CV MANAGEMENT SYSTEM

Table 20: Traffic CV Management System Test Procedures


CVE-TCVMS001-v01 Verify traffic CV management staff can input a MAP message payload to the TCVMS and specify which RSU it should be broadcast from


TCVMS is configured for "normal" operations, including sending MAP files to RSUs

TCVMS is connected to a network capable of accessing the RSU

RSU is configured for "normal" operation

RSU is connected to a network capable of accessing the TCVMS

A MAP file is stored on the TCVMS and ready to be sent to an RSU

1. Login to the TCVMS and configure MAP message for specific RSU

2. Execute TCVMS process for sending MAP messages to designated RSUs

3. Confirm MAP message payload matches the message sent from the TCVMS

TCVMS successfully loads MAP message to RSU

Screen shot, including date and time stamp, of RSU directory before sending MAP message

Screen shot, including date and time stamp, of RSU directory after sending MAP message, showing the payload of the MAP message

MAP message payload from TCVMS

CVE-TCVMS002-v01 Verify traffic CV management staff can input a TIM payload to the TCVMS and specify which RSU it should be broadcast from


TCVMS is configured for "normal" operations, including sending TIM messages to RSUs


RSU is configured for "normal" operation


A TIM message is stored on the TCVMS and ready to be sent to an RSU

1. Login to the TCVMS and configure TIM for specific RSU

2. Execute TCVMS process for sending TIM messages to designated RSUs

3. Confirm TIM message payload matches the message sent from the TCVMS

TCVMS successfully loads TIM to RSU

Screen shot, including date and time stamp, of RSU directory before sending TIM message

Screen shot, including date and time stamp, of RSU directory after sending TIM message, showing the payload of the TIM message

TIM message payload from TCVMS




CVE-TCVMS003-v01 Verify the TCVMS archives the MAP message locally and forwards it to the Operating System


TCVMS is configured for "normal" operations, including sending MAP messages to the Operating System

TCVMS is connected to a network capable of accessing the Operating System

Operating System is configured for "normal" operation, including receiving MAP messages from the TCVMS

Operating System is connected to a network capable of accessing the TCVMS

A MAP message is stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm MAP messages for RSU are stored locally in the designated folder

2. Execute TCVMS process for sending MAP messages to the Operating System

3. Confirm MAP message is archived in the Operating System

MAP messages are stored locally on the TCVMS

MAP messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing MAP message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending MAP message.

Screen shot, including date and time stamp, of Operating System directory after sending MAP message, showing MAP message payload.




CVE-TCVMS004-v01 Verify the TCVMS archives the TIM locally and forwards it to the Smart Columbus Operating System (Operating System)


TCVMS is configured for "normal" operations, including sending TIM messages to the Operating System


Operating System is configured for "normal" operation, including receiving RSU messages from the TCVMS


An TIM message is stored on the TCVMS and is ready to be sent to the Operating System

1. Login to the TCVMS and generate a TIM for a specific RSU

2. Monitor TCVMS to confirm the TIM is sent to the Operating System

3. Confirm the TIM just generated is archived in the Operating System

TIM messages are stored locally on the TCVMS

TIM messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing TIM message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending TIM message.

Screen shot, including date and time stamp, of Operating System directory after sending TIM message, showing TIM message payload.




CVE-TCVMS005-v01 Verify the TCVMS archives BSMs locally and forwards them to the Operating System


TCVMS is configured for "normal" operations, including sending BSMs to the Operating System


Operating System is configured for "normal" operation, including receiving BSMs from the TCVMS


BSMs are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm BSMs are stored locally in the designated folder

2. Execute TCVMS process for sending BSMs to the Operating System

3. Confirm BSMs are archived in the Operating System

BSMs are stored locally on TCVMS

BSMs are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing BSM payloads.

Screen shot, including date and time stamp, of Operating System directory before sending BSMs.

Screen shot, including date and time stamp, of Operating System directory after sending BSMs, showing BSM payloads.




CVE-TCVMS006-v01 Verify the TCVMS archives the SPaT message locally and forwards it to the Operating

See Figure 24 for system setup System

TCVMS is configured for "normal" operations, including sending SPaT messages to the Operating System


Operating System is configured for "normal" operation, including receiving SPaT messages from the TCVMS


SPaT messages are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm SPaT messages are stored locally in the designated folder

2. Execute TCVMS process for sending SPaT messages to the Operating System

3. Confirm SPaT messages are archived in the Operating System

SPaT messages are stored locally on TCVMS

SPaT messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing SPaT message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending SPaT messages.

Screen shot, including date and time stamp, of Operating System directory after sending SPaT messages, showing SPaT message payloads.




CVE-TCVMS007-v01 Verify the TCVMS archives the SRM locally and forwards it to the Operating System


TCVMS is configured for "normal" operations, including sending SRM messages to the Operating System


Operating System is configured for "normal" operation, including receiving SRM messages from the TCVMS


SRM messages are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm SRM messages are stored locally in the designated folder

2. Execute TCVMS process for sending SRM messages to the Operating System

3. Confirm SRM messages are archived in the Operating System

SRM messages are stored locally on TCVMS

SRM messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing SRM message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending SRM messages.

Screen shot, including date and time stamp, of Operating System directory after sending SRM messages, showing SRM message payloads.




CVE-TCVMS008-v01 Verify the TCVMS archives the SSM locally and forwards it to the Operating System


TCVMS is configured for "normal" operations, including sending SSM messages to the Operating System


Operating System is configured for "normal" operation, including receiving SSM messages from the TCVMS


SSM messages are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm SSM messages are stored locally in the designated folder

2. Execute TCVMS process for sending SSM messages to the Operating System

3. Confirm SSM messages are archived in the Operating System

SSM messages are stored locally on TCVMS

SSM messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing SSM message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending SSM messages.

Screen shot, including date and time stamp, of Operating System directory after sending SSM messages, showing SSM message payloads.




CVE-TCVMS009-v01 Verify the TCVMS archives the RTCM locally and forwards it to the Operating System


TCVMS is configured for "normal" operations, including sending RTCM messages to the Operating System


Operating System is configured for "normal" operation, including receiving RTCM messages from the TCVMS


RTCM messages are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm RTCM messages are stored locally in the designated folder

2. Execute TCVMS process for sending RTCM messages to the Operating System

3. Confirm RTCM messages are archived in the Operating System

RTCM messages are stored locally on TCVMS

RTCM messages are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing RTCM message payloads.

Screen shot, including date and time stamp, of Operating System directory before sending RTCM messages.

Screen shot, including date and time stamp, of Operating System directory after sending RTCM messages, showing RTCM message payloads.

CVE-TCVMS010-v01 Verify traffic CV management staff can specify a performance metric to the TCVMS (and specify how it is calculated)

NOTE: This is a Smart Columbus Operating System test case


TCVMS is configured for "normal" operations, including generating performance metrics

Data related to performance metrics are stored on the TCVMS

1. Login to the TCVMS

2. Configure performance metric(s) to be generated in TCVMS

3. Execute TCVMC command to generate performance metric(s)

4. Confirm TCVMS returns request performance metric(s)

TCVMS returns requested performance metric(s)

Screen shot, including date and time stamp, of configuration of TCVMS performance metric(s) to be generated

Screen shot, including date and time stamp, of performance metric(s) returned by the TCVMS




CVE-TCVMS011-v01 Verify generated performance metrics are uploaded to the Operating System.



TCVMS is configured for "normal" operations, including sending performance metrics to the Operating System. NOTE: This is a Smart Columbus Operating System test case


Operating System is configured for "normal" operation, including receiving performance metrics from the TCVMS


Performance metrics are stored on the TCVMS and ready to be sent to the Operating System

1. Login to the TCVMS and confirm performance metrics are stored locally in the designated folder

2. Execute TCVMS process for sending performance metrics to the Operating System

3. Confirm TCVMS performance metrics are archived in the Operating System

Performance metrics are stored locally on TCVMS

TCVMS performance metrics are stored on the Operating System

Screen shot, including date and time stamp, of TCVMS directory showing performance metric files.

Screen shot, including date and time stamp, of Operating System directory before sending performance metrics.

Screen shot, including date and time stamp, of Operating System directory after sending performance metrics, showing TCVMS performance metric files.




CVE-TCVMS012-v01 Verify PII is removed prior to sending data to Operating System



RSU is configured for "normal" operation, including forwarding BSMs to the TCVMS


OBU is configured for "normal" operation, including broadcasting BSMs

TCVMS is configured to receive BSMs from RSUs, log "raw" (with PII) BSMs in log1, remove PII from received BSMs, log BSMs with PII removed (log2),"normal" operation, and send BSM log f iles to the OS



Operating System is configured for "normal" operation, including receiving data from the TCVMS


1. OBU, broadcasting BSMs, is brought into communication range of the RSU

2. RSU forwards BSMs to TCVMS

3. TCVMS receives BSMs

4. TCVMS logs "raw" (with PII) BSMs (log1) (for testing)

5. TCVMS removes PII and logs (log2) BSMs in "normal" operation

6. Confirm BSMs logged in log2 do not contain PII

TCVMS removes PII from received BSMs prior to logging

BSMs without PII are stored on the Operating System

TCVMS log1, containing BSMs with PII

TCVMS log2, containing BSMs without PII

Operating System log files containing BSM without PII (f rom TCVMS log2)




CVE-TCVMS013-v01 Verify traffic CV management staff can specify a source for each RSU to receive RTCM data


TCVMS is configured for "normal" operations, including sending configuration parameters to RSUs


RSU is configured for "normal" operation, including receiving configuration parameters from the TCVMS


RTCM source configuration parameters are configured on the TCVMS and ready to be sent to RSUs

1. Login to the TCVMS and confirm RTCM source parameters

2. Execute TCVMS process for sending configuration parameters to RSUs

3. Confirm RSU is configured with the correct RTCM source parameters

RSUs are receiving RTCM messages from the specified NTRIP caster

Screen shot, including date and time stamp, of TCVMS showing RSU RTCM configuration parameters.

Screen shot, including date and time stamp, of RSU configuration before sending the RTCM source parameters.

Screen shot, including date and time stamp, of RSU configuration after sending configuration parameters, showing RTCM configuration parameters.

CVE-TCVMS014-v01 Verify TCVMS can detect unauthorized network activity.

NOTE: This is a Traffic Management Center test case


TCVMS is configured to send an email to operations staff if a user enters the wrong password more than three times

1. Attempt to log into the TCVMS four times using a wrong password

2. Af ter the fourth attempt, confirm an email was sent to operations staff

TCVMS detects excessive failed user log in attempts

TCVMS sends an email to operations staff indicating excessive failed user log in attempts

Email to operations staff from the TCVMS system indicating excessive failed user log in attempts




CVE-TCVMS015-v01 Verify TCVMS can detect traffic signal cabinet tampering and issue the proper notifications to Traf fic CV management staff.



Message Handler is configured for "normal" operation, including detecting cabinet tamper events and sending events to the TCVMS


TCVMS is configured for "normal" operation, including receiving cabinet tamper events and sending an email to operations staff


1. Connect a toggle switch (S1) to the cabinet tamper detection input/output (IO) port of the Message Handler

2. Set S1 to "inactive" (no event detected)


4. Confirm TCVMS is not detecting a cabinet tamper event

5. Set S1 to "active" (event detected)

6. Confirm TCVMS detected a cabinet tamper event

7. Upon a cabinet tamper event, confirm an email was sent operations staff

TCVMS detects a cabinet tamper event

TCVMS sends an email to operations staff indicating a cabinet tamper event

Email to operations staff from the TCVMS system indicating a cabinet tamper event




CVE-TCVMS016-v01 Verify traffic CV management staff can modify configurable setting of an RSU using the TCVMS.


TCVMS is configured for "normal" operations, including sending configuration parameters to RSUs


RSU is configured for "normal" operation, including receiving configuration parameters from the TCVMS


RSU configuration parameters are configured on the TCVMS and ready to send to RSUs

1. Login to the TCVMS and confirm RSU configuration parameters

2. Execute TCVMS process for sending configuration parameters to RSUs

3. Confirm RSU is configured with the parameters provided by the TCVMS

RSUs are configured with parameters provided by TCVMS

Screen shot, including date and time stamp, of TCVMS showing RSU configuration parameters.

Screen shot, including date and time stamp, of RSU configuration before sending the parameters.

Screen shot, including date and time stamp, of RSU configuration after sending configuration parameters, showing RSU configuration parameters.

CVE-TCVMS017-v01 Verify Traffic CV management staff have access to the following RSU data via the TCVMS: uptime status, connectivity, RSU graphical display, etc.


TCVMS is configured for "normal" operations, including pulling status elements from RSUs


RSU is configured for "normal" operation, including receiving requests for status elements from the TCVMS


1. Login to the TCVMS

2. Execute TCVMS process for retrieving status elements f rom RSUs

3. Confirm TCVMS receives status elements from RSUs

TCVMS shows status elements f rom RSUs

Screen shot, including date and time stamp, of RSU showing status elements.

Screen shot, including date and time stamp, of RSU status in TCVMS before requesting update.

Screen shot, including date and time stamp, of RSU status in TCVMS after requesting an update, parameters, showing RSU status.




CVE-TCVMS018-v01 Verify alert is issued for unauthorized network activity using dashboard icon and an email is sent to traffic CV management staff



TCVMS is configured for "normal" operations, including displaying/changing dashboard icons and sending email alerts when unauthorized network activity is detected

TCVMS is connected to a network capable of send emails to the appropriate staff members

1. Login to the TCVMS and monitor the dashboard and administrator email

2. From a separate console/computer, attempt to log into the TCVMS with the wrong credentials 3 times

3. Monitor TCVMS dashboard for icon indicating a failed log in attempt

4. Monitor Administrator email for a failed log in attempt notification

TCVMS dashboard indicates a failed login attempt

Administrator receives failed log in email notification

Screen shot, including date and time stamp, of dashboard alert

Copy of email notification with date and time stamp

CVE-TCVMS019-v01 Verify alert is issued when RSU is not operating as intended (degraded/failure conditions, or limited network connectivity) using dashboard icon and an email is to traffic CV management staff


TCVMS is configured for "normal" operations, including displaying/changing dashboard icons and sending email alerts when an RSU is malfunctioning

TCVMS is connected to a network capable of accessing RSUs


1. Login to the TCVMS and monitor the dashboard and administrator email

2. From a separate console/computer, reboot an RSU

3. Monitor TCVMS dashboard for icon indicating a loss of connection to an RSU

4. Monitor Administrator email for a Loss of RSU Connectivity notification

TCVMS dashboard indicates a lost RSU connection

Administrator receives Loss of RSU connection email notification

Screen shot, including date and time stamp, of dashboard alert

Copy of email notification with date and time stamp

CVE-TCVMS020-v01 Verify TCVMS logs all alerts/emails to traffic CV management staff



TCVMS is configured for "normal" operations, including archiving alerts and email notifications

1. Login to the TCVMS and confirm Alerts and email notifications are stored locally in the designated folder

Alerts and email notifications are stored locally on TCVMS

Screen shot, including date and time stamp, of TCVMS directory showing stored alerts and email notifications




CVE-TCVMS021-v01 Verify traffic CV management staff can query archived data on the TCVMS



TCVMS is configured for "normal" operations, including archiving data

1. Login to the TCVMS and run a query for data, for a given date and time range

2. Confirm data matching the query criteria is returned

Returned data matches the query criteria

Screen shot, including date and time stamp, of query run against the TCVMS

Screen shot, including data and time stamp, of the data returned from the query


MessageHandler

TCVMS

RSU

SCOS


Remote Vehicle

BSM

On

Door OpenSwitch

Admin Email


Figure 24: TCVMS



8.7. TRANSIT CV MANAGEMENT SYSTEM

Table 21: Transit CV Management System Test Procedures


CVE-TrCVMS001-v01 Verify transit CV management staff can specify a performance metric to the TrCVMS


TrCVMS is configured for "normal" operations, including generating performance metrics

Data related to performance metrics are stored on the TrCVMS

1. Login to the TrCVMS

2. Configure performance metric(s) to be generated in TrCVMS

3. Execute TrCVMS command to generate performance metric(s)

4. Confirm TrCVMS returns request performance metric(s)

TrCVMS returns requested performance metric(s)

Screen shot, including date and time stamp, of configuration of TrCVMS performance metric(s) to be generated

Screen shot, including date and time stamp, of performance metric(s) returned by the TrCVMS

CVE-TrCVMS002-v01 Verify generated performance metrics are uploaded to the Operating System


TrCVMS is configured for "normal" operations, including sending performance metrics to the Operating System

TrCVMS can connect to the Operating System

Operating System is configured for "normal" operation, including receiving performance metrics from the TrCVMS

Performance metrics are stored on the TrCVMS and ready to be sent to the Operating System

1. Login to the TrCVMS and confirm performance metrics are stored locally in the designated folder

2. Execute TrCVMS process for sending performance metrics to the Operating System

3. Confirm TrCVMS performance metrics are archived in the Operating System

Performance metrics are stored locally on TrCVMS

TrCVMS performance metrics are stored on the Operating System

Screen shot, including date and time stamp, of TrCVMS directory showing performance metric files.

Screen shot, including date and time stamp, of Operating System directory before sending performance metrics.

Screen shot, including date and time stamp, of Operating System directory after sending performance metrics, showing TrCVMS performance metric files.




CVE-TrCVMS003-v01 Verify transit CV management staff can query archived data on the TrCVMS


TrCVMS is configured for "normal" operations, including archiving data

1. Login to the TrCVMS and run a query for data, for a given date and time range

2. Conf irm data matching the query criteria is returned

Returned data matches the query criteria

Screen shot, including date and time stamp, of query run against the TrCVMS

Screen shot, including data and time stamp, of the data returned from the query

CVE-TrCVMS004-v01 Verify the TrCVMS archives the transit vehicle interaction events and forwards them to the Operating System


TrCVMS is configured for "normal" operations, including sending transit vehicle event logs to the Operating System

TrCVMS can connect to the Operating System

Operating System is configured for "normal" operation, including receiving transit vehicle event logs from the TrCVMS

Transit vehicle event logs are stored on the TrCVMS and ready to be sent to the Operating System

1. Login to the TrCVMS and conf irm transit vehicle event logs are stored locally in the designated folder

2. Execute TrCVMS process for sending transit vehicle event logs to the Operating System

3. Conf irm transit vehicle event logs are archived in the Operating System

Transit vehicle event logs are stored locally on TrCVMS

Transit vehicle event logs are stored on the Operating System

Screen shot, including date and time stamp, of TrCVMS directory showing transit vehicle event logs.

Screen shot, including date and time stamp, of Operating System directory before sending transit vehicle event logs.

Screen shot, including date and time stamp, of Operating System directory after sending transit vehicle event logs, showing transit vehicle event logs.


Transit Vehicle

Log Off load

TrCVMS SCOS


Figure 25: TrCVMS


Test Result Summary

This appendix contains examples of some of the Microsoft Excel tools used to capture and document test activities.

A.1 TEST RESULTS

Table 22 shows the TRL used to record results of each test.

Table 22: Test Result Log

Date Name Tester Role Test ID

Test Objective

Run Number

Test Result (P/F) Comment


A.2 DEFECT TRACKER

The defect tracker will be used during testing to capture, track, monitor, and address anomalies observed during testing. For each entry, the development team will work to understand and reproduce the defect (where possible), identify the root cause, summarize a response, and log the activities taken to resolve the issue. As outlined in Section 4.6.5, the defect tracker helps prioritize defects based on severity level (critical to low) and maintains traceability to the test ID and status. The status field provides a simplified view of the states a defect passes through as it moves toward resolution and closure. A defect can have the following status values:

• Opened – The defect has been logged and reported for correction.

• Re-Opened – The defect was once closed and then re-opened for modification.

• Closed – The defect was received, reviewed, and determined to be not a defect (i.e., it was a duplicate entry or a request for enhancement). In these cases, no corrective action is taken, and the

development team provides an explanation while closing out the defect ticket.

• Canceled – The functionality was canceled and therefore the defect is canceled by default.

• Resolved – The defect has been reviewed and verified, and a resolution was implemented to solve the problem.

• Returned – The defect was returned to the tester for additional information.

• Deferred – The defect was designated for correction for a later date.

If a conflict arises between a design element that tied to a requirement and a software product, the development manager will coordinate with the test manager to determine whether a change to the system

Appendix A. Test Result Summary


design and/or requirement is appropriate. The City of Columbus project manager will carefully review all

requests to make a change that impacts the system design and requirements. All change requests will be captured in the change log.

Table 23 shows a sample defect tracker for the CVE.

Table 23: Defect Tracker

Date

Defect

No.

Defect

Description Severity

Defect

Status

Test

ID

Date

Found

Assigned

To

Resolution

Description Comment


A.3 CHANGE REQUEST LOG

Table 24 is a sample change request log for the CVE project.

Table 24: Change Request Log

CR ID Description Justification Defect ID Requirement Status


A.4 TEST SUMMARY

Table 25 shows a sample test summary report.

Table 25: Test Summary Report

Date Test Cases Planned

Test Cases Executed

Test Cases Passed

Test Cases Failed

Test Cases Deferred


Appendix A. Test Result Summary


Table 26 shows a sample defect matrix.

Table 26: Defect Matrix

Defect

Release Open Closed Canceled Resolved Deferred


A.5 TEST CASE STATUS

Table 27 is a sample test case status log for the CVE.

Table 27: Test Case Status

Test ID Test Objective

No. of

Test Runs Expected Outcome Status



Terminology and Conventions

B.1 NUMBERING CONVENTION

Each testing element is assigned a unique ID for traceability and configuration management. Test cases and scenarios for all projects in the Smart Columbus program will follow the same convention, each element

representing an identifiable attribute of the traced metric. The numbering convention is as shown in Figure 26.

Project Abbreviation

01

Test Type Code

Test Number V Static Character

Version Number

02 03

(a) (b) (c) (d) (e)


Figure 26: Numbering Convention Definitions

Where:

03

01

02

Identifies the Project

Identifies the device under test and the test number

Identifies the version of the Test

The definitions of the components in Figure 26 are listed in Table 28

Table 28: Numbering Convention Details

Octet Description Data Type, Casing

Number of Characters or Digits

a. Project abbreviation The designated Smart Columbus project acronym (i.e., CVE)

String, upper case

Variable

Appendix B. Terminology and Conventions


Octet Description Data Type, Casing

Number of Characters or Digits

b. Device type code • OBU: On-Board Unit

• App: OBU Application

• RSU: Roadside Unit

• MH: Message Handler

• TSC: Traffic Signal Controller

• TCVMS: Traffic Connected Vehicle Management

System

• TrCVMS: Transit Connected Vehicle Management System

String, upper case

2

c. Test number Integer incrementing by 1, indicating the number of Test Cases or Test Procedures

Integer 3

d. “v” static character Static letter “v” represents the version for the particular test objective and procedure

Character 1

e. Version number Integer incrementing by 1, indicating the number of revisions made to the test element being traced

Integer 2


An example of a test case for the integration of CVE would be CVE-OBU001-v01.

1. “CVE” is the project abbreviation.

2. “OBU001” is the Test Case #1 for the On-Board Units

3. “v01” is version #1 of Test Case OBU001


Acronyms and Abbreviations

Table 29 contains project-specific acronyms used throughout this document.

Table 29: Acronym List

Abbreviation/Acronym Definition

ARC-IT Architecture Reference for Cooperative and Intelligent Transportation

BSM basic safety message

CBR channel busy ratio

ConOps [CVE] Concept of Operations

COTA Central Ohio Transit Authority

CTSS Columbus Traffic Signal System

CVE Connected Vehicle Environment

CVRIA CV Reference Implementation Architecture

CV connected vehicle

CVTT connected vehicle test tool

DSRC dedicated short-range communications

EEBL electronic emergency brake light

EV emergency vehicle

FCC Federal Communications Commission

FCW forward collision warning

FST f inal system test

GPS Global Positioning System

HDV heavy-duty vehicle

HMI human machine interface

ICD Interface Control Document

ID identifier

IEEE Institute of Electrical and Electronics Engineers

IMA intersection movement assist

IO input/output

IP internet protocol

IPv6 internet protocol version 6

IT information technology

Appendix C. Acronyms and Abbreviations



ITS Intelligent Transportation System

L1 laptop 1

L2 laptop 2

LCW lane change warning

LDAT local device assembly test

LDV light-duty vehicle

MAP mobile application part

OBUs onboard units

ODOT Ohio Department of Transportation

Operating System [Smart Columbus] Operating System

OTA over the air (or over-the-air for adjectival use)

pcap packet capture file

PII personally identifiable information

RLVW red light violation warning

RSSZ reduce speed school zone warning

RSUs roadside units

RTCM Radio Technical Commission for Maritime Services

S1 toggle switch

SAE Society of Automotive Engineers

SCMS security credential management system

SDD [CVE] System Design Document

SPaT signal phasing and timing

SRM signal request message

SSM signal status message

SST subsystem test

SyRS [CVE] System Requirements

TBD to be decided

TCVMS Traf fic Connected Vehicle Management System

TIM Traveler Information Message

TMC Traf fic Management Center

TrCVMS Transit Connected Vehicle Management System

TRL Test Result Log

USDOT U.S. Department of Transportation

V2I Vehicle-to-Infrastructure

Appendix C. Acronyms and Abbreviations



V2V vehicle-to-vehicle

WAVE wireless access in vehicular environments

WRA wireless access in vehicular environments routing advertisement

WSA wireless access in vehicular environments service advertisement



Glossary

Table 30 contains definitions of project-specific terms used throughout this document.

Table 30: Glossary

Term Definition

CV test tool (CVTT) The CVTT is a specialized OBU designed to capture DSRC messages for decoding, visualizing DSRC messages (e.g., MAP, TIM), and running CV applications for system acceptance

position correction system A position correction system provides global navigation satellite system (GNSS) data consisting of carrier phase and code range measurements in support of three-dimensional positioning. Correction data are used to improve the precision of GPS data. Enhanced post-processed coordinates approach a few centimeters relative to the National Spatial Reference System, both horizontally and vertically.

Smart Columbus Operating System The Operating System collects and archives CV data for distribution to third parties.

Traf fic CV Management Center (TCVMS)

The TCVMS is the operation and maintenance center of the CVE. RSUs are monitored from, and data are archived to, the TCVMS.

Transit CV Management Center (TrCVMS)

The TrCVMS is the CV data center for COTA. Transit OBUs upload the log files to the TrCVMS and TrCVMS, and staff can review and process transit vehicle log files.


Connected Vehicle Environment (CVE) Test Plan · herein and any trade name that may appear in the...

Documents

Transcript of Connected Vehicle Environment (CVE) Test Plan · herein and any trade name that may appear in the...