ONE ECONOMY/BROADBAND OPPORTUNITY COALITIONEVALUATION PLAN

JUNE 27, 2011

Joint Center for Political and Economic Studies

Evaluation Team

Joint Center for Political and Economic Studies (lead)

National Urban League Policy Institute (NULPI)

Polling – Princeton Survey Research Associates International (PSRAI)

Goals

Goal One: Assess and determine how well the One Economy/BBOC program achieves the following performance metrics: 150,000 new households subscribe to

broadband service between April 2, 2010–March 31, 2013 as a result of the BTOP program.

Goals

Goal Two: Compare how well the One Economy/BBOC program extends broadband use among individuals and households in five select test cities relative to five control cities without program services. A list of questions includes:

How can technology meet individuals’ demand for information on a 24/7 basis in a way that aligns with their location, literacy and language? (measure: effectiveness of public purpose media)

What marketing strategies and messages are more useful to communities that have been slow to adopt broadband? (measure: effectiveness of national awareness campaign)

How does access in the home facilitate higher rates of broadband usage? (measure: effectiveness of affordable broadband access in affordable housing)

To what extent does digital literacy training, especially when conducted by local residents, facilitate broadband adoption among hard to reach communities? (measure: effectiveness of Digital Connectors program and mobile vans)

In what way do the cumulative program impacts or discrete elements of the BTOP program contribute to a “lift” in sustainable broadband adoption? (measure: impacts of individual versus cumulative program elements)

Goals

Goal Three: Assess and document the shift in the culture of use, and relevant “success factors” in sustainable broadband adoption among hard to reach communities, especially low-income, minority, language-challenged households, as a result of the digital ecosystem created by BTOP program elements (i.e., Digital Connectors, Access Services, Public Purpose Media, and Mobile Training).

Specific questions to be asked:

What is the social dividend of technology as shown in improvements in education, health, and employment?

How does easy access to broadband in the home facilitate greater opportunities, and supports for low-income households?

How effective are digital literacy programs that employ local people to promote new technology, and its benefits to low-income residents?

What is the length of subscription for low-income households once connected to broadband services?

Evaluation Design and Process Comparative research design Process:

Identification of Test and Control Cities, Data Collection Tools, and Completion of the Evaluation Plan

Deployment of Evaluation Resources and Implementation

Creation of the Interim and Final Reports

Program Logic Model

Criteria for Test Cities

Receive all three BTOP Services Digital Connectors Access Services Computer Centers

Has one BBOC affiliate in the city Have a poverty rate that is at least 10% Have a diverse population that highlights one of

the major racial or ethnic groups of the BBOC Coalition (African American, Asian American or Hispanic American)

Represent a distinct geographic region

Criteria for Control Cities

Receive one or no BTOP services Have a poverty rate that is at least 10% Have a diverse population that highlights one

of the major racial or ethnic groups of the BBOC Coalition (African American, Asian American or Hispanic American)

Represent a distinct geographic region Have minimal BTOP investments (under $5

million) from other national programs as indicated through NTIA public records on the BTOP program

Test Cities Control Cities Chicago, IL Los Angeles, CA Washington, DC San Diego, CA Raleigh, NC

Columbus, OH Austin, TX Baltimore, MD Omaha, NE Charleston, SC

Demographic and Economic Characteristics

Source: US Census

Summary of BTOP Services by Test and Control Cities

Coverage Areas of Major Carriers by Test and Control Cities

Performance Indicators

The number of broadband subscribers, new versus existing by demographics

The number of trainings completed and the level of consumer satisfaction

The number of households connected The number of community computing centers engaged The number of training hours completed by Digital

Connectors – orientation and consumer digital literacy training

The number of web users of available online content The number of connections made by consumers to relevant

services (e.g., educational, employment, health care, etc) after broadband subscription (data will be gathered through pre- and post-assessment surveys)

Secondary Indicators

The number of program participants enrolling and completing workforce development programs post-program engagement

The number of program participants enrolling and completing educational programs post-program engagement

The number of program participants retaining viable employment

The number of program participants advancing their expertise in technology courses post-program engagement

Evaluation Challenges

Identifying a stable population to evaluate over the course of the grant will be one challenge faced in the program evaluation.

Ensuring consistent data collection at all levels of the program to ensure a robust inventory of participant responses.

Anticipating the potential lag times between participants’ program enrollment, and actual service connections.

Disentangling in both test and control cities the affects of existing market conditions, and other digital access resources (e.g., libraries, community technology centers, schools, etc.).

Capturing the racial, ethnic and language diversity of constituents within the Test and Control cities, especially when gathering information for the program evaluation.

Evaluation Team

For more information, contact:Nicol Turner-Lee, Ph.D. –

nturnerlee@jointcenter.orgAlicia Durfee – adurfee@jointcenter.org