The CUTGroup

Daniel X. O’Neil

Sonja Marziano

Social Enterprise Class

Northwestern University

February 25, 20151

@danxoneil @smartchicago #CUTGroup

Agenda-cize

• My personal journey

• Smart Chicago

• Origins

• Components

• Tools

• Methods

• Examples

2

@smartchicago #CUTGroup

My personal journey

• Web worker since June 15, 1998

3

@smartchicago #CUTGroup

First

• I made my first civic app in 1999

• Text-based WAP systems were just

coming into general use on phones

• I tried to think of the most compelling use

case for this technology. In other words,

what can’t wait for you to get back to your

desk?

• The answer: Killer on the Loose

April 21, 2015 4@danxoneil

Most compelling wireless utility,

1999

April 21, 2015 5@danxoneil

I’ve worked with some developers

April 21, 2015 6@danxoneil

I’ve worked with some developers

April 21, 2015 7@danxoneil

April 21, 2015 8@danxoneil

April 21, 2015 9@danxoneil

April 21, 2015 10@danxoneil

Our principles

11

@smartchicago #CUTGroup

The Strategy

12@smartchicago

Access, Skills, and Data

• Access to the Internet for all– Broadband at home

– Public computer centers and Community technology centers

– Next generation stuff

• Skills once you’re on the Internet– All on ramps

• Data so that there’s something worth looking at once you’re on and know how to work it– Apps

– Infrastructure

13

@smartchicago #CUTGroup

The Founders

14@smartchicago

Three Chicago Institutions

• The John D. and Catherine T. MacArthur Foundation– A center for ideation and conception

– 2007 report, “The City that NETWorks”

– The primary source of funding

• Chicago Community Trust– Provides housing for the organization and is fiscal agent

– At the center of funding and knowledge for the region

– Critical financial, human resources, and other support

• The City of Chicago– The most important policy lever in the region

– Departmental relationships

– Getting things done

15

@smartchicago #CUTGroup

CUTGroup

16

@smartchicago #CUTGroup

• The Civic User Testing Group is a set of

regular Chicago residents who get paid to test

civic apps

– http://www.cutgroup.org/

• The CUTGroup Book: Civic User Testing

Group as a New Model for UX Testing, Digital

Skills Development, and Community

Engagement in Civic Tech

– http://www.cutgroupbook.org/

Origins

17

@smartchicago #CUTGroup

• Stemmed from Illinois Open Tech Challenge

(IOTC)

– Very successful in recruiting tech talent

– Not so successful in recruiting residents

– My experiences

• We basically came to the thought: “If it doesn’t

work for you, it doesn’t work.”

Three Main Components

18

@smartchicago #CUTGroup

• UX Testing

• Digital Skills

• Community

Engagement

Components: UX Testing

19

@smartchicago #CUTGroup

• Legitimate UX testing -- the developers who work with us get specific, actionable feedback from relevantly situated users

• Both quantitative and qualitative info

• We’ve found glaring bugs, discovered unique insights, and helped people plan new product releases.

• We stray from UX test design principles, however, in a very key way: by requiring the developer to participate in the test.

Components: Digital Skills

20

@smartchicago #CUTGroup

• Each test consists of a mix of 10-20 people of widely varying digital skills, all learning and teaching each other, in the same room, out in plain public view

• Developers are learning how to conduct UX tests and to make their sites better

Components: Community Engagement

21

@smartchicago #CUTGroup

• Biggest surprise to us

• Conduct tests in Connect Chicago locations– More than 250 places in Chicago where you can

use a computer for free

– http://weconnectchicago.org/

• We were surprised to see how much people like the CUTGroup. – Same last two questions on every CUTGroup test

form:• “Did you like this CUTGroup test?”

• “Anything else to add?”

Components: Community Engagement

22

@smartchicago #CUTGroup

• Diverse group of testers from different

backgrounds and neighborhoods

Tools

23

@smartchicago #CUTGroup

Tools

24

@smartchicago #CUTGroup

• Mailchimp: http://mailchimp.com/

– E-mail campaigns

• Wufoo forms: http://www.wufoo.com/

– Collect information

– Recruitment e-mails

– Tests

• Excel

• Patterns: software that helps collect data from

different systems; assists with segmenting

More Tools

25

@smartchicago #CUTGroup

• Devices

– Laptops & iPads

– Encourage testers to use their own devices

– We can segment based on tester devices

• Recording equipment:

– http://www.smartchicagocollaborative.org/hardware-

and-software-for-cutgroup-and-civic-hacker-events/

• Gift cards:

– http://www.smartchicagocollaborative.org/gift-cards-for-

cutgroup/

– $5 gift card for sign up, $20 gift for a test

• Candy!

Methods: Pre-Test

26

@smartchicago #CUTGroup

• Developer meeting– Target testers, Type of test, Questions to ask

– What do we want to learn?

• Recruitment– 1st e-mail campaign

– The more we ask of testers, the smarter we get about testing!

• Segmenting– 2nd e-mail campaign asking for time availability

• Scouting for test locations– We want to be all over the city

Methods: During Test

27

@smartchicago #CUTGroup

• Proctoring– Different test types:

direct proctoring, focus groups, remote, self-driven/in-person

– Depends on developer meeting and what we want to learn from the test

• We use Wufoo forms to ask test questions with or without a proctor

Methods: Post-Test

28

@smartchicago #CUTGroup

• Analysis

– In-depth analysis of different questions

• Publishing results for every test

• Follow Up

– Github Issues; example from CUTGroup #9:

Tests

29

@smartchicago #CUTGroup

We have conducted 12 tests to date:

• #1 – FreedomPop Router Test

• #2 – Go2School User Application Test

• #3 – Chicago Health Atlas

• #4 – EatSafe.co

• #5 – ChicagoWorksforYou.com

• #6 – OpenStreetMap Editor

• #7 – EveryBlock iPhone App

• #8 – Waitbot App

• #9 – Foodborne Chicago

• #10 – Build it! Bronzeville

• #11 – Expunge.io

• #12 – Roll with Me

Examples: EatSafe.co

• EatSafe.co Tester #8

– https://www.youtube.com/watch?v=rGjMhz3TIkI

30

@smartchicago #CUTGroup

Examples: OpenStreetMap

• OpenStreetMap Tester #5

– http://youtu.be/BWNZxVhV7Kk

31

@smartchicago #CUTGroup

Foodborne Chicago• An app that searches Twitter

for tweets related to food

poisoning and helps report

these incidents to the

Chicago Department of

Public Health

• https://www.foodbornechicag

o.org/

32

@smartchicago @foodbornechi #CUTGroup

Foodborne Questions

Here is a list of questions we wanted to answer through this

test

• How do users feel about responding to a random tweet?

• Does the form require too much information?

• Would users feel better or worse if there was more

Chicago Department of Public Health or 311 visibility?

• Why might residents on the South side of Chicago not

respond as often as residents from other Chicago

neighborhoods?

33

@smartchicago @foodbornechi #CUTGroup

Foodborne Results

• How do users feel about responding to a

random tweet?

- Twitter is used as a private communications

network for the majority of our testers.

- Twitter handle & logo helps

- Tweets should have three elements:

compassion, question, and an official element.

• Does the form require too much information?

- Surprised that form wasn’t too long per testers

34

@smartchicago @foodbornechi #CUTGroup

Foodborne Results

• Would users feel better or worse if there was more

Chicago Department of Public Health or 311 visibility?

- Wanted stronger connection to City of Chicago

• Why do residents on the South side of Chicago not

respond as often as residents from other Chicago

neighborhoods?

- Twitter users see Twitter as a way to connect with

people they know, or are within in their circles

whether that be their friends, their work, or their

community.

35

@smartchicago @foodbornechi #CUTGroup

Improvements

• Change the Tweet

• URL Update

• Be more Official

• More about Process

• Being part of more Networks

36

@smartchicago @foodbornechi #CUTGroup

The CUTGroup Book

http://www.cutgroupbook.org

37

Be a Tester!

• Are you a Chicago Resident?

• Interested in testing?

• Sign Up Here http://www.cutgroup.org/

38

@smartchicago #CUTGroup

Sign up to test yer stuff!

• Are you a developer?

• Want to do this?

• Sign Up Here

http://www.cutgroup.org/developers.html

39

@smartchicago #CUTGroup

So ya thanks

• @SmartChicago

• @DanXONeil

• @SmartChgoKyla

• @SSMarziano

• www.SmartChicagoCollaborative.org

40