FORUM

Do’s and Dont’s for Offering Effective Recommendations

MICHAEL HENDRICKS and MEREDITH PAPAGIANNIS

Recommendations are one of the most critical products of any evaluation. Audiences

typically spend more time studying our recommendations than any other product,

including our findings or our conclusions. And, because recommendations are so

important, we evaluators are beginning to pay them more attention, although sur-

prisingly little attention and surprisingly slowly. Six years ago we lamented that “Unfortunately, we as a profession have paid little

attention to the topic of recommendations” (Hendricks, 1984). Two years ago Michael Patton reminded us that little has changed:

Recommendations have long troubled me because they have seemed to me the weakest part

of evaluation. We have made enormous progress in ways of studying programs, method-

ological diversity, and a variety of data-collection techniques and designs. The payoff from

those advances comes in the recommendations we make. But we have made very little

progress in how to construct useful recommendations. (Patton, 1988)

To speed progress in this area, the following “Do’s and Dont’s” are offered for

developing, presenting, and following up on evaluation recommendations. These

suggestions are just that: a “menu” of possible actions, not a prescription to be followed in any one situation. Evaluators work under many different circumstances, so some of our suggestions are more appropriate for certain situations than for others. The

wise reader should carefully peruse the menu and select only those suggestions which fit a particular situation.

Michael Hendricks 0 MH Associates, Washington, DC, and Meredith Papsgiannis 0 Children, Youth and Family Services,

Florida Department of Health and Rehabilitative Services.

Evaluation Practice, Vol. 1 I, No. 2, 1990, pp. 121-125 Copyright 0 1990 by JAI Press, Inc.

ISSN: 0191-8036 All tights of reproduction in any form reserved.

121

122 Evaluation Practice, 11(2), 1990

STEP ONE: DEVELOP EFFECTIVE RECOMMENDATIONS

DO consider all issues in your evaluation to be “fair game” for recommendations:

0 In addition to looking for helpful recommendations in the issue areas established in advance, be alert to possibilities for action in other, unforeseen areas. These unexpected recommendations can some- times be the most insightful and valuable of them all.

DON’T wait until the end of your evaluation to begin thinking about recommen- dations:

0 Devote time to recommendations throughout your evaluation. For example, begin recording possible recommendations (perhaps on color-coded index cards) as soon as you start collecting data and forming impressions. Keep recording these possibilities as the evalu- ation progresses.

DO draw possible recommendations from a wide variety of sources:

a “Cast a wide net” around a variety of sources; e. g . , earlier studies of the same program, particularly successful sites, similar programs, etc. Many of these “imported” recommendations might be sur- prisingly relevant to your current situation.

0 Involve program staff at all levels. Their differing perspectives often produce varying and especially good ideas, and we should almost always ask their opinions.

0 Get decisionmakers to speculate in advance about what recommenda- tions should be offered. For example, Roberts-Gray, et al. (1987) encourages decisionmakers to specify what recommendations should be offered given certain patterns of findings. She then simply (a) produces the findings and (b) shows the decisionmakers what they themselves had decided to recommend.

DO work closely with agency personnel throughout the process:

l It can be threatening when an outsider offers recommendations, and we need an explicit strategy to minimize any perceived threat. One basic rule is not to surprise those involved - “it’s not a birthday party.” Work openly and honestly with all stakeholders every step of the way.

0 As you begin to focus on likely recommendations, work actively to build acceptance for them. Include both persons who must approve your recommendations and persons who must implement them.

Do’s and Dont’s for Offering Effective Recommendations 123

DO consider the contexts into which your recommendations must fit:

0 Recommendations must make sense within the program’s larger sociological, political, and organizational context. Remember, eval- uation is a political activity, and that almost every recommendation has political implications.

STEP TWO: PRESENT EFFECTIVE RECOMMENDATIONS

DO generally offer only realistic recommendations:

0 As a general rule, limit your recommendations to ones which have a reasonable chance of being accepted and which can be implemented with reasonable efforts. This might sometimes mean recommending the “best result possible” instead of the “best possible result.”

0 However, remember that it is our ethical responsibility as objective analysts to make whatever recommendations we feel are appropriate, even if it seems like a “lost cause” at the time. Times change, and causes can be “found” more quickly than we realize.

DO decide how specific you want your recommendations to be

0 Consider the possibility of providing “options” from which deci- sionmakers can choose (cf. Wholey, 1986) instead of, or along with, specific recommended actions.

0 Also, recognize that some situations call for very general recommen- dations (e.g., “Problem A should be corrected”), while other situations call for specific recommendations (e.g., “To correct problem A, unit X unit should take action Y by date 2”).

DO think twice before recommending fundamental changes:

0 Generally avoid recommending fundamental changes (e.g., restruc- turing a program’s basic objectives or overall approach), because basic changes threaten the organization’s status quo, can be quite complex, and are often politically risky. If possible, reach your objectives via incremental improvements within the existing frame- work.

124 Evaluation Practice, 11(2), 1990

DO show the future implications of your recommendations:

0 Show, in as much detail as possible, the specific benefits expected from each recommendation and the types and amounts of resources necessary to implement each one.

l Consider planning an implementation strategy in advance, as an integral part of each recommendation. In addition to helping “sell” your recommendations, such a plan can also be a starting point once your recommendations are accepted.

DO make your recommendations easy for decisionmakers to understand:

Don’t offer a long, undifferentiated list of recommendations. Catego- rize your recommendations in meaningful ways, such as short-term vs. long-term, inexpensive vs. expensive, major vs. minor, or admin- istrative vs. regulatory vs. legislative. Test the clarity of your recommendations by using “cold readers” to ensure that each recommendation conveys what you want to convey. Whenever possible, accept all suggested editorial revisions, so long as each revision preserves the meaning of your recommendation. Draw a clear physical boundary between your recommendations and your findings or conclusions, perhaps by indenting, bolding, and/or underlining your recommendations. However, tie each recommenda- tion conceptually to your findings, perhaps by using graphic displays connecting findings and recommendations or by presenting each recommendation immediately after its relevant findings (see below). Decide carefully where to place your recommendations within the text of your written report. We prefer to integrate our recommenda- tions directly into the text (right after the relevant findings, as mentioned above), but other evaluators place theirs at the front or back of the report or even in multiple locations. Don’t expect decisionmakers to change their style of receiving infor- mation just for your evaluation. Learn how they normally receive information and adapt to that. In most situations, this means dis- cussing your recommendations in a personal meeting if at all possi- ble.

STEP THREE: EFFECTIVELY FOLLOW-UP ON RECOMMENDATIONS

DO stay involved after your recommendations have been accepted:

l If appropriate, remind key persons of those recommendations which have been accepted and which still require action. While this is generally more difficult for external than for internal evaluators, it can be done by both if handled diplomatically.

Do’s and Dont’s for Offering Effective Recommendations 125

0 If asked, help to implement the accepted recommendations by (a) developing an implementation plan, (b) actually implementing some recommendations yourself (such as those involving analyses), and/or (c) monitoring the progress of the recommended actions.

If an important recommendation is not accepted, DO look for other opportunities to recommend it again

0 Continue to monitor the important issues, and especially look for upcoming decision points at which your key recommendations could be offered again. Events occur, new decisionmakers arrive, more information becomes available, and all these changes can create more receptive environments for our recommendations.

REFERENCES

Hendricks, M. (1984). Finis. Evaluation News, 5(4), 94-96. Hendricks, M. and Handley, E.A. (in press). Improving the recommendations from evaluation studies.

Evaluation and Program Planning, forthcoming. Patton, M.Q. (1988). The future and evaluation. Evaluation Practice, 9(4), 90-93. Roberts-Gray, C., Buller, A., and Sparkman, A. (1987). Linking data with action: Procedures for

developing recommendations. Evaluation Review, I I, 678-684. Wholey, J.S. (1986). Options, not recommendations. Paper presented at the American Evaluation

Association annual conference, Kansas City, KS.