GREATER BOSTON EVALUATION NETWORK

2014 FIRST ANNUAL SURVEY RESULTS

Designed by Amanda Sutter and Erica Clarke

WHO WE ARE

By the numbers

• 89 Email Addresses on Evaluators’ Roundtable Email Distribution List

• 57 People on new LinkedIn page

• 45 Survey Respondents

The majority of members are white females. The group can benefit from outreach to diversify membership both by gender and race.

Female82%

Did not answer

3%

Male15%

3%

5%

5%

85%

Multi-racial or mult-ethnic

Black or African American

Asian

White

The group is younger, but...Members have a variety of experience levels, with most are early in their careers.

AGE EXPERIENCE

30s37%

Did not answer5%

40s25%

50s3%

20s30%

3%

10%

13%

21%

21%

33%

More than 20 years

less than 2 years

16-20 years

11-15 years

6-10 years

2-5 years

…the members are highly educated.Most members have at least a Master’s degree.

Master's degree

67%

Doctorate degree

23%

Bachelor's degree

10%

EVALUATION ROLES

75% Report their primary employment is in evaluation

Most members identify as evaluator for their professional identity.

“Policy analyst” “Data analyst”

3%

10%

13%

13%

63%

College or universityfaculty or instructor

Other

Program manager orcoordinator

Researcher

Evaluator (anycapacity)

Members engage in diverse evaluation activities, but collecting and analyzing data are universal activities.Studying, teaching, and planning evaluations are the least common roles.

15%

23%

28%

43%

45%

60%

60%

85%

93%

93%

100%

100%

Teaching evaluation

Planning/contracting for evaluations

Student in evaluation

Writing about evaluation

Training others in evaluation

Evaluation or data capacity building

Technical assistance

Using data for continuous improvement

Writing reports and data visualization

Conducting evaluations

Collecting data

Data analysis

3%

3%

3%

3%

6%

8%

11%

64%

Other

Foundation

Self-employed

Company in business or industry

Local or state government

College or university

Research, evaluation or consulting firm

Non-profit organization

75% Work only in the US

GBEN members overwhelmingly work in non-profit organizations.No members are students, retirees, or work for the federal government.

“Employee of a non-profit research organization housed at a university”

Members vary drastically in the amount of time they spend on evaluation related work. Most spend either all of there time OR less than 25% of their time on evaluation.

0%

14%

14%

14%

19%

39%

No Time

76%-99%

51%-75%

26%-50%

1%-25%

All Time

WHAT WE EVALUATE

GBEN works in a wide variety of fields… Non-profits, partnerships, education, and health-related fields are where we work most.

15%

15%

15%

15%

18%

20%

20%

23%

23%

35%

38%

43%

43%

45%

48%

58%

Special needs populationsSocial work

Public policy and public administrationBusiness and industry

Evaluation methodsHuman services

FoundationsSTEM

GovernmentWorkforce and economic development

Child care and early childhood educationK-12 education

Health and public healthCommunity partnerships and coalitions

Youth developmentNon-profits

… And supports evaluation across the spectrum in the Greater Boston area. No current members work in law and criminal justice or LGBT issues.

Other responses listed: community development public educationfamily supportchild abuse preventionnewborn home visitingincarceration & re-entryliteracy

3%

3%

3%

3%

3%

3%

5%

5%

8%

8%

8%

8%

10%

10%

13%

13%

Organizational behavior

Information systems

Indigenous peoples

Human resources

Environmental programs

Disaster and emergency

Evaluation theory

Arts and culture

Medicine

Media

International and cross cultural

Human development

Educational technologies

Adult education

Other

Higher education

Only 28% conduct evaluation as a consultant some of the time

Few members serve as external consultants, but for those that do, they work primarily with non-profit organizations.

8%

8%

8%

25%

42%

42%

50%

58%

92%

Other

International organizations

Other for-profit companies

Non-profit evaluation and consulting firms

Federal government

Colleges or universities

Foundations

State or local government

Other non-profit organizations

GBEN PREFERENCES

Respondents connect to a variety of local networking groups. People benefit from the opportunity to meet others in the field, learn, and share ideas.

Reported Benefits:Networking and socializingLearning from othersTools, strategies, new techniquesOpen discussionMaking connectionsVenting and talking through issuesSharing tips and resources

79%

39%

11%

Evaluators'Roundtable

CWU OutcomesWorkgroup

Tech Networks ofBoston

Some other affiliations:American Educational Research AssociationAmerican Public Health AssociationAmerican Psychological AssociationAssociation for Community Health ImprovementNational Association of Social WorkersSociety for Community Research and ActionSociety for Research in Child DevelopmentSociety for the Research of Educational Effectiveness

58% Report that are not

members of AEA

Respondents have various national professional affiliations. GBEN has an opportunity to serve as a resource to connect those who are not members of national associations to the larger professional community.

GBEN members prioritize networking the most.Members are least interested in connecting to other affiliates across the country.

3%

26%

51%

56%

59%

62%

64%

74%

74%

82%

Do no plan to participate

Collaboraton with other affiliates across country

Get better sense of breadth of evaluation field

Opportunities to present work and trouble-shoot challenges

Professional development

Local social activies

Resources, information and job postings on the web

Intellectual stimulation

Informal sharing of skills, tools, practices

Local professional networking activities

Respondents want to hear from peers how evaluation is done at other organizations.

• I like to see what solutions that other evaluators/organizations have successfully deployed, and be able to ask, "how would they do things differently." Then use that discussion to inform our own evaluation techniques.

• I'm interested in learning how others share information about program participants, progress and outcomes.

Informal sharing of skills, tools, practices:

Frequently mentioned PD wishes:• Data visualization (5)• General evaluation skills (4)• Quantitative/statistics (3)• Evaluation fundamentals (2)• Presentation skills/reporting (2)

Respondents also mentioned wanting:• Information about local conferences and workshops• Networking opportunities• Lists of best practices• Tools in evaluation• And more!

Professional Development:

Members want variety in the time and frequency of GBEN activities.GBEN should plan short events at various times to meet different needs and schedules.

FREQUENCYTIME OF DAY

85% Prefer 1-5 hours per month for activities

8%

20%

35%

38%Quarterly

Every other month

Monthly

Twice/yr50%

50%

53%

60%Noon or Lunch

Morning

Evening

Afternoon

5%

16%

18%

21%

39%

50%

I am not interested

Help plan one or more social events

Ready documents to become an affiliate

Attend monthly planning meetings/calls

I do not have time to help

Attend one or more social events

87% Stated that we could contact them for help

Finally for affiliate planning, most members are interested in participating, but namely in attending social events.Around 20% are willing to help with planning meetings or tasks.

Connect with us!

• Greater Boston Evaluation Network LinkedIn Page: https://www.linkedin.com/groups/Greater-Boston-Evaluation-Network-8177131/about

• To join the Evaluators’ Roundtable, contact Erica Clarke: esclarke@partners.org

• For questions on the survey, contact Amanda Sutter: amandallsutter@gmail.com