Should evaluators judge whether services are appropriate?

Evaluafion andProgram Planning, Vol. 8, pp. 37-4.1985 Printed in the USA. All rights reserved.

0149-7189/85 $3.00 + .OO Copyright 0 1985 Pergamon Press Ltd

SHOULD EVALUATORS JUDGE WHETHER SERVICES ARE APPROPRIATE?

MICHAEL HENDRICKS

MH Associates, Washington, DC

ABSTRACT

If an Aid for Dependent Children (AFDC) mother receives a cash reward for helping police apprehend a criminal, she automatically loses part of her children’s benefits. Is this appropriate? More central to this article, does an evaluator of the AFDC program have the right to judge whether this is appropriate? Although decision makers’ abilities to rectiyy such situations are incre~ingly constrained, they clearIy need to know about such seemingly inappropriate aspects of services. But should evaluators be a source for this information? Yes-even though there are undeniable risks involved. This article d&cusses these issues and offers 1% suggestions for gathering and presenting information on appropriateness.

Mothers receiving Aid for Dependent Children (AFDC) automatically lose part of their benefits if they receive extra income from an outside source, even if the extra income comes as a reward for helping the local police apprehend a criminal.

Senior citizens are automatically eligible for free lunches at tax-supported facilities around the country, even if the senior citizen is a millionajre and arrives in a chauffeur- driven limousine.

Until recently, a child under 18 whose parents had died was automatically given survivor’s benefits by the federal government, even if the child had deliberately murdered his or her parents.

indigent elderly persons are automaticalIy eligible to be supported in nursing homes by the Medicaid program, even if they have children with substantial incomes.

Obviously, these examples differ in some important ways, because they were chosen partly to reflect a variety of political views. However, they share the common aspect that, although the services being provided (AFDC payments for needy mothers and children, health care for the elderly) are quite admirable and needed, there are certain features of the services, or the ways they are delivered, that might seem clearly inappropriate to some observers (such as penalizing

helpful citizens or subsidizing lunches for millionaires). This article asks whether evaluators should judge the appropriateness of the services we evaluate, much as we now judge the resources, activities, impacts, and costs of those services. In other words, do we as evaluators have the obligation-or even the right-to ask whether services are being “appropri- ately” applied?

There are difficulties in raising this question. First, what concepts are included in a definition of appropriate? The dictionary defines appropriate as “suitable or fitting for a particular purpose, person, or occasion” (~arnhart & Stein, 1970). Applying this de~nition to social programs, I define appropriate services as “services that suitably accomplish both the letter and the spirit of their original intentions.” Under this definition, free lunches for elderly indigents would be appropriate, whereas free lunches for elderly millionaires would not.

Concerning this definition, it is important to realize at the outset that I am not asking whether evaluators should judge the appropriateness of an entire program or set of programs, such as subsidized lunches or welfare payments. Evaluators have the right and the responsibility to influence these decisions as private

Special thanks to Mary Anne Bunda, Jim Cliatt, Dan Geller, Lis Handley, Karen Kirkhart, Ted Koontz, Mike Mangano, Alan Meyer, Bryan MItchelI, Jack Molnar, Bill Moran, Jon Moreli, Rob Orwin, Richard Schmidt, Nick Smith, Joan Sieber, Joe Wholey, Mark Yessian, and Norm Zimiich for hefpful comments and discussions.

Requests for reprints should be sent to Dr. Michael Hendricks, MH Associates, Suite 101, 2140 20th Street, NW, Washington, DC 20009.

31

38 MICHAEL HENDRICKS

citizens, but they have no special authority to affect the outcome via their role as evaluators (unless, of course, they are called upon to provide information about the impacts of previous or similar rules). In- stead, what this article addresses is the proper role evaluators should play when faced with questionable effects or implications of certain policies or procedures, or both.

A second difficulty in raising this question is agree- ing on ways to judge appropriateness, because discussions about appropriateness are ultimately discussions about values. The examples at the beginning of this article were problematic because two or more values conflicted in one situation. Weighting the different values and assessing the overall appropriateness is a personal, subjective process; each of us balances conflicting vaiues in an individual, idiosyncratic way. To avoid the morass of moral relativism, I am judging any service to be potentially inappropriate if a reasonable observer might raise reasonable concerns. This perspective is intended to force us, as evaluators, to consider all objections to services and to deny us the convenient definitions of commonly accepted or orig- inally intended that obscure legitimate minority views.

A final difficulty of this issue is the emotion that is sometimes generated by subjective judgments about appropriateness. In discussions with colleagues, I have

realized my own emotions about this topic and the fact that others have equally strong emotions that can differ from mine. It is important to recognize these personal emotions and to divorce them as much as possible from the discussion of an evaluator’s proper role. But, as humans, we may not fully succeed. The emotion generated by the Catholic bishops’ discussions of American nuclear defense strategy is an example of the difficulty of separating emotion from discussions about conflicting values (J. Cliatt, personal communi- cation, 1983).

To reduce these difficulties, this article asks three specific questions. First, how can decision makers (whom I define as the commissioners and recipients of evaluation, whether they are normally called program managers, policymakers, or administrators) use information about the appropriateness of services? Sec- ond, should evaluators be the ones to provide this sort of information to decision makers? Decision makers have many sources, and perhaps evaluators are not the best source for this information. Third, how could evaluators go about gathering, analyzing, and presenting information about appropriateness if asked to do so? Although I am pleased to raise issues, I claim no definitive answers, and I certainly hope to see further work along these lines.

HOW MIGHT DECISION MAKERS USE INFORMATION ABOUT THE APPROPRIATENESS OF SERVICES?

How might decision makers use information about the times the changes are as simple as revising guidances appropriateness of services if it were available to sent to local project managers. Not surprisingly, most them? Although decision makers might conceivably decision makers would prefer to change programs ad- use this information in a variety of negative ways (to ministratively, rather than statutorily. bargain for higher budgets, to embarrass rivals), evalu- There are constraints, however, even on adminis- ators would presumably be providing the information trative changes, and these constraints go beyond pro- in the hope of improving services. In other words, cedural difficulties. The bureaucratic environment of- evaluators would be hoping to convince the decision fers few rewards for innovative management, yet there maker to change the inappropriate service. can be severe punishment for violating accepted pro-

But is the evaluator correct in assuming that the cedures (Fottler, 1981). Also, there is intense public decision maker has the capability to make the needed scrutiny of changes in programs, especially those that changes? At the federal level, programs can be changed are controversial or that involve powerful advocacy either statutorily (by changing the law) or adminis- groups (Whorton & Worthley, 1981). Finally, decision tratively (by changing the regulations or implementing makers often have little autonomy to define or re- guidances). Statutory changes are always possible, and define objectives of the program. Given these pres- the survivor’s benefits mentioned earlier no longer go sures on decision makers, information about the ap- to children who deliberately kill their parents, because propriateness of services could add an uncomfortable the Reagan administration sought and achieved just anxiety to their decisions (Foster, 1981). such a change. However, Congress must approve these Of more concern, however, is a recent trend toward changes, so the process can be cumbersome and the reducing the discretionary powers of decision makers tendency might be to avoid this route if possible. to accomplish the spirit of the law. Effective manage-

Administrative changes, on the other hand, do not ment of public programs requires not only a scrupu- require congressional approval and can be imple- lous consistency to avoid arbitrary processes, but also mented by the administering agency. Often this re- a consistency of ends even more than of means. “What quires public notice of proposed changes, but some- is suggested is not the complete elimination of laws,

Should Evaluators Judge Service Appropriateness? 39

but a more reasoned appreciation of public needs and appropriate responses” (Foster, 1981, p. 33).

Unfortunately, several recent Supreme Court decisions could undermine the willingness of decision makers to consider both the letter (means) and the spirit (ends) of the law. Each of these opinions in- volves the proper balance between discretionary powers and personal liability for making decisions. In each case, the Court has narrowed the discretionary powers of decision makers and increased their personal liability.

Since the late 17OOs, the relative balance of discretion versus liability for decision makers has fluctu- ated, but it has generally tilted toward discretionary powers for public officials. In 1971, however, the Court (in Bivens v. Six Unknown Named Agents of the Federal Bureau of Narcotics) limited discretionary immunity to only those cases that do not violate constitutional rights and in which the decision maker acts within his or her authority. The need for federal officials to go to court to defend their actions was a revolutionary change from past practices (Rosen- bloom, 1980).

However, federal officials still had almost complete immunity for actions taken within their authority. In 1979, in Butz v. Economou, the Court further limited that immunity to only those instances in which the decision maker acts in good faith and with a reasonable belief that his or her actions will not violate a citizen’s constitutional rights. In other words, the shield of discretionary authority no longer protected a decision maker from a personal liability suit. Since Butz v. Economou, the courts have widened the possibility for personal liability (Miller 1980-81).

“[T]he liability policy for state and local officials provides a trend similar to the success of doctrines applied to federal officials” (Miller, 1980-81, p. 30). In fact, state and local officials face greater liability than do federal officials, because the court decided in 1980 (in Owen v. City of Independence, Main v. Thiboutat, and Maher v. Gagne) that state and local officials face liability regardless of their intentions or their knowledge. In other words, if a citizen can show a violation of his or her constitutional rights, government decision makers responsible for the violation are personally liable, even though the violation was unintentional and unknown (Groszyk & Madden, 1981). Officials at all levels now face a liability they once thought remote.

What will be the effects of these rulings? Opinions differ, but most people agree they will focus more attention on the exact letter of the laws, both the laws implementing the program and case laws circumscrib- ing its application. Because court rulings on personal liability will focus on specific legal issues, officials at all levels will pay increasingly closer attention to the specifics of these laws.

It seems likely that this closer attention will increase

the review and coordination necessary for decision making, which in turn will lead to longer times required for decisions. This is because the courts have held that, even with formal delegations of authority, the top official is legally culpable for officials under his or her direction. Henceforth, lower level officials may be prohibited from using discretionary powers to rectify inappropriate situations.

This increased review and coordination-and the time required - could create administrative inertia, wasted resources, lowered morale, and departure of more independent-minded staff. “Right or wrong” might be replaced by “technically legal or not,” and the end (justice, fairness, morality, appropriateness) replaced by the means (law), as we lose sight of the ultimate aims of programs and services.

Discretion is a necessary part of public officials’ jobs, and, no doubt, the responsibilities of most offices require it. Damage suits attempt to limit irresponsible use of discretion but, in so doing, can easily limit the motivation of public servants to use discretion at all. (Miller, 1980-81, P. 30)

[Glovernment bureaucracy may turn apprehensive and become more bloated, less effective, less timely, and more concerned with the impact of any action upon itself than on the recipients of the assistance it provides. (Groszyk & Madden, 1981, p. 271)

Others, however, view these rulings as a positive force to insure equitable treatment and to avoid whim- sical denial of rights.

A government of laws is frequently contrasted with a government of men. But a government of laws actually in- tensifies, rather than narrows, the responsibility of those men-and women-who are administrative officers; it demands, not that they be deprived of decision-making in matters of large importance, but that in reaching their decisions they weigh as objectively as they can all of the relevant factors and all of the competing interests which their decisions may affect. . . . It demands, in short, fairness in both substance and procedure. (Willcox. 1%8, p. 105)

To the decision maker confronted by a future environment of virtually unbounded uncertainty, the circum- scribing nature of the legal process can only have positive connotations; by routinizing the possible states of nature, it is bound to produce greater accountability to the elec- torate, thereby protecting the people and even the decision maker himself. . . . (Foster, 1981, p. 31)

The likely result lies somewhere in between. De- cisions might take longer and might involve more reviews at more levels. This will probably frustrate some officials, but it might result in more equitable decisions overall. The key question, however, is whether these procedures will diminish responsiveness to individual circumstances. Efficiency in general (routinization) can often lead to inefficiency in specific

40 MICHAEL

situations. Cops on the beat and caseworkers face this dilemma constantly.

For evaluators, this diminishing discretion implies less willingness by decision makers to act on evaluation findings. Information about appropriateness might not be used to rectify the situation immediately. The legal constraints, as well as the organizational, public relations, and political constraints mentioned earlier, could dampen decision makers’ ability to respond. But does this mean they shouldn’t receive such information?

On the contrary, there are at least three reasons why decision makers should still receive information about appropriateness of services. First, while statutory and administrative changes are not simple, both types of changes can be made and, in fact, are made regularly. As mentioned earlier, an administration that opposes payments to children who deliberately kill their parents can change the program to prohibit such payments. But these changes will not be sought unless examples of inappropriateness are brought to decision makers.

Second, decision makers cannot do their jobs prop- erly without such information. Studies of managerial work (e.g., Mintzberg, 1973) show that decision makers are, first and foremost, brokers of information. Their function is to obtain different types of information from different sources and to apply that ex- ternal information to their internal responsibilities within the organization (McGowan & Loveless, 1981). To broker information effectively means to adopt a broad perspective on which information might be useful and to discard information only after carefully considering it. Information about appropriateness must be included in that broad perspective if the decision maker intends to learn all relevant information. In addition, some of this information might be useful in protecting decision makers from possible liability.

Third, information about appropriateness can im- prove public administration and decrease the public’s skepticism. Knowledgeable persons have written recently about this skepticism and the need to overhaul

HENDRICKS

the way programs are administered (Argyris, 1980; Drucker, 1980; Staats, 1980-81). Each worries about short-term perspectives, single-issue analyses, and in- ability to learn from experience. Argyris wrote most comprehensively:

There are two major categories of learning that occur in organizations. The first is single-loop learning or thermostat learning; the second is double-loop learning. Single- loop learning occurs whenever a match is produced or whenever a mismatch is corrected without having to question or change organizational assumptions and policies. For example, a new procedure for welfare payments or for correcting errors in the way a service is presently per- formed, are examples of single-loop learning.

Double-loop learning occurs whenever a match is produced or a mismatch is detected and corrected and changes are made in the basic assumptions and policies of the unit. A thermostat that detects and corrects the tem- perature in a room without questioning its program is single-loop learning. If the thermostat were to ask why it was set at 65 degrees, or why it should be measuring heat, that would be double-loop learning. (1980, pp. 206-207)

Argyris claimed that single-loop learning dominates everyday actions in an organization and can blind a decision maker to the larger issues of importance. In- formation about appropriateness stimulates double- loop learning, because it forces the decision maker to question or change organizational assumptions and policies when two or more values conflict. This questioning can alter the “boundaries of the feasible” by redefining services and their impacts. If appropriate changes can then be made in the services, decision makers will be viewed as more legitimate, capable, and attuned to public needs.

For these three reasons, decision makers need regular information about appropriateness of services, in spite of the increasingly formidable constraints on immediate use of the information. Whether or not evaluators should provide this information is, however, an entirely separate question, which is addressed in the next section.

SHOULD EVALUATORS BE THE ONES TO JUDGE APPROPRIATENESS?

Although decision makers need a steady diet of information about appropriateness of services, should evaluators set the table? There are many other sources of information - public opinion, the media, superiors in the organization’s hierarchy, program experts, advocacy groups- and perhaps evaluators should defer judgments to these groups.

Perhaps judging appropriateness would violate the evaltiator’s proper role. Perhaps the evaluator should be the quintessential staff person, dealing only with means and leaving the discussions of ends and appropriateness to others. From this viewpoint, an evaluator should only follow orders and not act unilaterally to expand the scope of issues to examine.

In my opinion, however, judging appropriateness is a proper role for an evaluator. A thorough evaluation is not complete without an examination of whether the services are embracing the values embodied in their objectives.

Is it possible to be aware of the nature of a problem which led to creation of a program-with the goal (aim, mission, purpose objective, etc.) of reduction or elimination of the problem-and not evaluate appropriateness? I believe not. . . . Thus, it seems that anyone who knows the purpose of a program or knows something which allows im- putation of a purpose, cannot avoid evaluating its appropriateness. (Den&on, 1980, p. 92)


And, more directly:

Is the social scientist only a hired hand responsible for providing utilitarian answers to questions raised by the funding agency? Or, are there not other questions that the professional is obliged to ask - necessary questions, whether one seeks objectivity, radical critique, or simply an honest day’s pay for an honest day’s work? (Tuchfeld, 1981, p. 169)

From this perspective, a complete evaluation is a broad evaluation that examines all aspects of the services. In the absence of direct instructions from the decision maker, all types of information are open to examination.

But perhaps appropriateness would encroach upon the legitimate role of others, perhaps even on the decision maker. Our form of government assigns authority to leaders, elected and appointed, with a larger perspective on the services than do evaluators, who view the program from only one angle (Hare, 1972). Given that decision makers and evaluators play different roles (Caro, 1980; Duncan, 1980), perhaps the decision maker has the right to determine what will be examined: “Only the manager can know what questions are relevant to the decision he must make. Frequently, even he cannot be sure, but he is in a much better posi- tion to judge the relevance of a question than is the evaluator” (Havens, 1981, p. 483).

But I think that evaluators, rather than encroaching on the role of others, would more likely be filling a vacuum by providing information not currently available from any source. Argyris’s (1980) concern about the dominance of single-loop learning suggests that someone- anyone-is needed to break the routine of “thermostat” learning by examining the question of the basic values and assumptions of the services. Evalua- tors could easily argue that they are the best persons to do so.

Which leads to a final concern. Perhaps judging appropriateness would be risky for an evaluator, for several reasons. Most obvious is the threatened or ac- tual punishment if the decision maker did not welcome the judgment. This is especially true if the decision maker had explicitly ruled out any such judgments or if the judgment came as a surprise. For this reason, it might be easier for outside evaluators to judge appropriateness than for inside evaluators (Bruce-Briggs, 1978; Maynard-Moody, 1982; Siegel & Doty, 1979).

Another risk is that of appearing morally superior to the decision maker. Information about appropriate-

ness often reduces to information about the relative importance of two or more values that are conflicting in a given situation (e.g., insured health care for the elderly versus family responsibility of children). This is a subjective judgment, but evaluators who report their judgments in a “scientific” report might unwittingly imply that their judgment is superior to the judgment of others.

This leads to the greatest risk-that of losing credibility as an objective, impartial analyst. Although it may be possible to report inappropriate services without implying that changes are necessary, it would certainly be extremely difficult. Evaluators would correctly be seen as agents of change, and not just change for greater effectiveness or efficiency but change that redirects part of a program’s impact. If so, evaluators “risk losing the traditional protections of our scientific impartiality” (Haan, 1982, p. 1104). For some evaluators, this risk of losing credibility might not be worth the possible gains.

But credibility is exactly the reason why evaluators should be willing to judge appropriateness. All other groups who might report their judgments have clear vested interests and will represent those interests as fully as possible. Evaluators, on the other hand, have been patiently building reputations as persons dis- interested in turf struggles and interested only in com- prehensive and accurate analyses of the world as it exists. For this reason, evaluators can have great credibility as objective observers, and their judgments sometimes carry great weight.

In almost all instances, an evaluator is not limited to only those assignments from the decision maker but is responsible for reporting any other findings that seem relevant and important. This duty is incumbent on an evaluator not only for the credibility of the evaluation at hand, but also for the credibility’of the evaluation profession. Obviously there will be mitigating factors in deciding what findings are relevant and important enough: How many persons are affected? How great is the potential harm? Can the harm be corrected?

An evaluator’s actions must be conditional on the individual situation, because it is an “unreasonable wish to have simple rules for our conduct in a complex world” (Hare, 1972, p. 68). If individual evaluators wish to maintain both their credibility and their effectiveness, they might have to walk the fine line between unthinking conformity to the status quo and disloyal rebellion against organizational objectives. The next section discusses some possible guidelines for walking that line.

HOW CAN EVALUATORS JUDGE APPROPRIATENESS?

The introduction to this article admitted that simply nal definition of appropriate services: services that defining appropriateness is a difficult task. Judging suitably accomplish both the letter and the spirit of appropriateness seems even more difficult. In order to their original intentions. This definition requires judg- offer some practical suggestions, I revert to the origi- ing whether the intent of a service is being manifested

42 MICHAEL

in all circumstances or whether, under some conditions and in some locations, services are inappropriate for the original intent. To accomplish this objective, an evaluator might consider the following 18 suggestions:

1.

2.

3.

4.

5.

6.

Whenever possible, negotiate up front with the decision maker to examine appropriateness as an integral part of the evaluation. This is a strategy recommended by the ERS Standards (Evaluation Research Society, 1981), which makes eminently good sense, because evaluations are most likely to be utilized if they are user-focused rather than evaluator-focused (Connolly & Porter, 1980). It also reduces the risks and decreases the possibility of awkward situations developing later.

involve as many different perspectives in the evaluation as possible. This includes particularly the design and analysis stages but might also be ex- tended to data collection with appropriate safe- guards. Different perspectives, as represented by critics, rejected clients, legislators, program staff, frontline workers, and successful participants, will insure that minority opinions are explicitly de- signed into the study and are examined.

Similarly, recruit and use interdisciplinary team members to conduct the evaluation. Common disciplines such as sociology, psychology, policy analysis, and anthropology are helpful, but less common sources such as theologians, police officers, and average citizens often provide insights that might otherwise have been overlooked.

Design an evaluation with a broad mandate, rather than with a premature narrowing to specific questions. Although there will certainly be times when a narrowly focused evaluation is necessary, it is always desirable to examine additional aspects that become apparent during the study’s planning.

Insist that the decision maker review and approve the design before beginning data collection. One of the key leverage points an evaluator possesses is the ability to define the issues of interest and the nature of the questions to be asked (Tuchfeld, 1981). If the design indicates that appropriateness is a legitimate area for inquiry, the risks are dimin- ished and the decision maker is prepared for later information on the topic.

Focus the data collection on the client’s level by in- cluding site visits to local delivery sites. Examining data from afar is not likely to highlight inappropriate services, whereas actually visiting sites, observing conditions and activities, examining documents in files, and talking with clients, staff, and officials results in a more complete picture of possible problem areas.

HENDRICKS

7.

8.

9.

10.

11.

12.

While on-site, contact as wide a variety of respondents as possible. Just as many perspectives can help broaden the range of issues to design into the evaluation, many respondents can also provide a broad range of viewpoints on services. Most evaluations only scratch the surface of possible types of respondents who possess useful and in- sightful information about the services.

Pay particular attention to unintended side effects and intervening variables that affect how services manifest themselves at the local level. Almost by definition, inappropriate services can be unexpected side effects of appropriate services (e.g., subsidizing millionaires’ lunches along with indigents’ lunches), so Striven’s (1972) ideas of goal- free evaluation are especially relevant. Intervening variables are likewise important, because they may determine the effects at any given site (e.g., staff disapproving of certain services might sabotage them for certain client groups).

To examine side effects and intervening variables, use unstructured data-collection methods, at least for part of the data collection. Although it is possible to discover inappropriateness through struc- tured instruments (e.g., income data from lunch recipients), it is unlikely that an evaluator will be foresighted enough to ask all appropriate questions beforehand. Open-ended questions enable respondents to raise issues of interest to them and thus are more likely to produce unexpected information.

As information about appropriateness begins to appear, take every opportunity to inform the decision maker about the emerging trend. Use interim reports, progress briefings, and personal conver- sations to alert the decision maker. As we noted earlier, information about appropriateness stimulates double-loop learning, which requires a re- examination of basic assumptions. This process is difficult and time consuming, so it is better to start as early as possible.

Keep this information confidential as long as ethically possible. Although it may be necessary to divulge it at some point, decision makers appre- ciate the opportunity to ponder issues in relative privacy, without having their options constrained by premature pressures from disclosure.

Arrange a personal briefing on the findings. De- cision makers work verbally more than from written documents, so they are accustomed to dealing with difficult issues in that manner (Lau, New- man, & Broedling, 1980). Furthermore, questions will inevitably arise, and the evaluator must insure that the findings are being interpreted correctly (Hendricks, 1982).


13.

14.

15.

Begin the briefing with a basic history and ex- planation of the service’s values and assumptions. This must be done tactfully to avoid patronizing the audience, but this basic program framework can show inconsistencies discussed later.

Be as specific as possible when discussing inappropriateness. Examples, anecdotes, case studies, and direct quotes can illustrate situations and can bring the information to life. One example of an unacceptable condition, provided it reflects a serious and widespread problem, can be enor- mously helpful in producing changes in a program.

Be scrupulously fair in presenting the findings, taking great pains to include information that might disagree with your interpretations but that makes a more balanced briefing for the decision maker. Credibility is a fragile trait that must be developed patiently but that can be destroyed in- stantly. If the decision maker once suspects that the evaluator is biasing the presentation, all further information (but especially that regarding appropriateness) will be suspect. Public opinion polls are one example of the kinds of outside information that might be provided (Rycroft, 1978).

16. Consider providing different sets of recommendations for different assumptions. One set might recommend actions if the decision maker agrees that some services are inappropriate, whereas another set might recommend actions otherwise. Besides allowing for all possible circumstances during the briefing, preparing these different sets of recommendations shows the decision maker that you are sincerely impartial in your analysis.

17

18.

Suggest interim steps that the decision maker might take immediately. One of these is to study the situation further, because questioning basic assumptions is a process that can reasonably take more than one look. Another recommendation might be to pilot test other methods of delivering services that offer promise of avoiding the inappropriateness.

As far as possible, show the decision maker how to implement each recommendation. Much as specific examples help illustrate the inappropriateness of some services, so can specific guidance on recommendations help determine exactly which steps should be taken.

CONCLUSIONS

This article has asked whether evaluators should judge the appropriateness of services. Defining and judging appropriateness, allowing decision makers the discretionary powers to use such information, creating an evaluator role that allows such judgments to be made, and finding appropriate methods to make the judgments are all extremely problematic. Yet it is legitimate and necessary for evaluators to perform these tasks, and, in doing so, they should emulate the architect.

Architects have certain obligations to their sponsors, mostly to create functional buildings economically. However, architects also claim the right-and the duty -to think not only of what is obligatory but also of what is possible. They define their role as creating works that are both functional and beautiful. Evalu- ators should also claim the right to explore the potential of new dimensions for their work.

REFERENCES

ARGYRIS, C. (1980). Making the undiscussable and its undiscuss- DUNCAN, W. J. (1980). Knowledge transfer in administrative ability discussable. Public Administration Review, 40, 205-213. science. Public Administration Review, 40, 341-349.

BARNHART, C. L., & STEIN, J. (Eds.). (1970). The American college dictionary. New York: Random House.

EVALUATION RESEARCH SOCIETY. (1981). Standards for program evaluation. Potomac, MD: Author.

BRUCE-BRIGGS, B. (1978). The politics of policy analysis. Policy Review, 5, 41-57.

FOSTER, Cl. D. (1981). Law, morality, and the public servant. Public Administration Review, 41. 29-34.

CARO, F. G. (1980). Leverage and evaluation effectiveness. Evalu- ation and Program Planning, 3, 83-89.

FOTTLER, M. D. (1981). Is management really generic? Academy of Management Review, 6, 1-12.

CONNOLLY, T., & PORTER, A. L. (1980). A user-focused model for the utilization of evaluation. Evaluation and Program Planning, 3, 131-140.

GROSZYK, W. S., & MADDEN. T. J. (1981). Managing without immunity: The challenge for state and local government officials in the 1980s. Public Administration Review, 41. 268-278.

DENISTON, 0. L. (1980). Whether evaluation- whether utilization. Evaluation and Program Planning, 3. 91-94.

HAAN, N. (1982). Can research on morality be “scientific”? American Psychologist, 37, 1096-I 104.

DRUCKER, P. F. (1980). The deadly sins in public administration. Public Adminrstration Review, 40, 103-106.

HARE, R. M. (1972). Applications of moralphilosophy. Berkeley, CA: University of California Press.

44 MICHAEL HENDRICKS

HAVENS, H. S. (1981). Program evaluation and program management. Publtc Administratron Review, 41, 480-485.

HENDRICKS, M. (1982). Oral policy briefings. In N. L. Smtth (Ed.), Communicatron strategies in evaluatton. Beverly Hills, CA: Sage.

LAU, A. W., NEWMAN, A. R., & BROEDLING, L. A. (1980). The nature of managerial work in the public sector. Public Ad- mmistration Review, 40, 513-520.

MAYNARD-MOODY, S. (1982). Reconsidering charity: Some possible negative effects of evaluation research. Administration and Society, 13, 379-403.

MCGOWAN, R. P., & LOVELESS, S. (1981). Strategies for information management: The administrator’s perspective. Public Adminrstration Revrew, 41, 331-339.

MILLER, G. J. (1980-81). Administrative malpractice policy before and after Butz v. Economou. The Bureaucrat, 9. 25-32.

MINTZBERG, H. (1973). The nature of managerial work. New York: Harper & Row.

ROSENBLOOM, D. H. (1980). Public administrators’ official immunity and the Supreme Court: Developments during the 1970s.

Public Administration Review, 40, 166-173.

RYCROFT, R. W. (1978). Selecting policy evaluation criteria: To- ward a rediscovery of public administration. Midwest Review of Public Admuustration, 12, 87-98.

SCRIVEN, M. (1972). Pros and cons about goal-free evaluation. Evaluatton Comment, 3, l-4.

SIEGEL, K., & DOTY, P. (1979). “Advocacy research” versus “management review”: A comparative analysis. Policy Analysis, 5, 37-65.

STAATS, E. B. (1980-81). Overhauling the federal government: A reform agenda. Policy Studies Journal, 9, 1206-1215.

TUCHFELD, B. S. (1981). Accountability to whom? Evaluation and Program Planning, 4, 169-172.

WHORTON, J. W., & WORTHLEY, J. A. (1981). A perspective on the challenge of public management: Environmental paradox and organizational culture. Academy of Management Review, 6, 357-361.

WILLCOX, A. W. (1968). Public services under a government of laws. The HEWForum Papers, 101-111.

Should evaluators judge whether services are appropriate?

Documents

Transcript of Should evaluators judge whether services are appropriate?