Consultee satisfaction with a psychiatric consultation-liaison nursing service

7

Click here to load reader

Transcript of Consultee satisfaction with a psychiatric consultation-liaison nursing service

Page 1: Consultee satisfaction with a psychiatric consultation-liaison nursing service

Consultee Satisfaction With a Psychiatric Consultation-Liaison

Nursing Service Linda Newton and Keith G. Wilson

A consumer satisfaction survey was conducted with nurses who made consec- utive consultation requests to a general hospital psychiatric consultation-liaison nursing service. In each of 69 responses, global satisfaction ratings indicated that the consultee either was satisfied or very satisfied with the assistance pro- vided. Particular strengths were noted in relation to the ease with which the consultants could be notified, the speed of response, and the clarity with which their formulations were expressed. The respondents were least satisfied with the extent of written documentation and the amount of follow-up. In a multiple regression analysis, only the total number of visits emerged as a significant predictor of satisfaction scores. However, comparisons of replies of most sat- isfied and least satisfied also indicated that high satisfaction ratings were more likely to be given to consultations requesting assistance with patients’ famiiies rather than to consultations requesting help in the management of patients with major psychiatric disorders. 0 1990 by W.B. Saunders Company.

A LTHOUGH THE provision of general hospi- tal consultation services represents a rela-

tively new specialization within the field of psy- chiatric nursing, the role of the nurse consultant has become well-established (Robinson, 1982). As often has been noted, the nurse consultant is in the ideal position to integrate concerns regarding a pa- tient’s psychiatric and psychosocial care with the constraints imposed by daily hospital routines (Barton and Kelso, 1971; Goldberg, Tull, Sulli- van, Wallace & Wool, 1984). In settings in which nursing consultation services have been intro- duced, psychiatric consultants have found a corre- sponding decline in consultations pertaining to pa-

From the Department of Nursing-Psychiatry, Health Sciences Centre, Winnipeg, Manitoba; and the Depart- ment of PsychiaOy, University of Manitoba.

Address reprint requests to Linda Newton, R.N., B.A., B.S.N., Department of Nursing-Psychiatry, Health Sci- ences Centre, 820 Sherbrook St, Winnipeg, Manitoba, Canada R3A lM4.

0 1990 by W.B. Saunders Company. 0883-9417/90/0404-0006$3.00/0

tient-staff conflicts, and assessments for transfers to psychiatry (Lipowski & Wolston, 1981). These findings have been attributed to the impact of the nurse consultant on the heightening of staff sensi- tivity to patients’ psychological needs, facilitating appropriate referrals, and providing skilled psy- chotherapeutic interventions to patients on medical and surgical wards (Lipowski, 1981; Lipowski & Wolston, 1981).

Despite the growth of psychiatric consultation- liaison nursing programs, little formal research has been conducted to identify the range of services provided by nurse consultants. In the only avail- able comparative study of nurse-versus-physician consultations, Stickney and Hall (198 1) found that nurses were more likely than physicians to be con- sulted for behavior management problems, pallia- tive care for terminal patients, and the provision of support to patients’ families. Given the growing importance of the psychiatric nurse’s contribution to these aspects of the psychological care of the medically ill, it is appropriate that more attention be directed toward the evaluation of the role and

264 Archives of Psychiatric Nursing, Vol. IV, No. 4 (August), 1990: pp. 264-270

Page 2: Consultee satisfaction with a psychiatric consultation-liaison nursing service

CONSULTEE SATISFACTION 265

impact of the nurse consultant. With the current emphasis on quality assurance in hospital settings, it becomes especially relevant to monitor the level of care that is provided by such emerging pro- grams.

One approach that is gaining acceptance in this context involves the surveying of individuals who have been the consumers of services about their perceptions of the treatment that was provided. In many mental health settings, this type of consumer satisfaction survey has become an important source of feedback for program evaluation (Lebow, 1983; Lorefice, Borus, & Keefe, 1982).

In the case of hospital consultation programs, there are, in fact, two sets of interested consumers: the patients and the staff members who initiate the consultations. Both perspectives have been the fo- cus of research investigation with physician-based consultation services, although there has been no comparable work with nursing consultation-liaison programs. Interestingly, while the patients who re- ceive consultations with psychiatrists generally view them quite favorably (Moses & Barzilay, 1967; Schwab, 1967; Schwab, Clemmons, Valder, & Raulerson, 1966), their physician colleagues ap- pear to be more critical. Sasser and Kinzie ( 1978,1979), for example, surveyed nonpsychiat- tic residents about their attitudes towards the value of psychiatric consultations, and found that only 36% of those residents who had used psychiatric services reported them to be “usually” or “always” helpful.

In the present study, a similar consumer satis- faction survey was undertaken with the profes- sional colleagues of psychiatric consultation- liaison nurses. The target group of survey respondents were those individuals who made con- secutive consultation requests to the consultation- liaison service. It was considered that this group of consumers would have a relatively high degree of sophistication with regard to the benefits of con- sultations, and would be able to provide accurate and critical feedback regarding several dimensions of the consultants’ performance. Hence, at one level, the satisfaction survey was implemented as a quality-assurance activity designed to assist in the identification of specific factors associated with good performance ratings from the consultees. At a more general level, the survey was intended to provide some initial empirical information regard- ing consultee perceptions of nurse consultants and

the value of their contribution to the psychosocial care of medically ill patients.

METHOD

The study was conducted at the Health Sciences Centre in Winnipeg, Manitoba, Canada. The Health Sciences Centre is an 1,100 bed hospital complex, which is the major teaching and tertiary- care facility in a Canadian province of approxi- mately 1.1 million residents. Patients throughout the center who are in need of assessment for psy- chiatric disorders or coping and adjustment prob- lems are seen either by a general hospital psychi- atry consultation team, or by a consultation-liaison nurse. In practice, the psychiatry and nursing ser- vices work collaboratively, with the nurses dealing most frequently with those patients and family members who are experiencing acute psychosocial crises or adjustment difficulties.

The psychiatric consultation-liaison nursing ser- vice is comprised of one full-time and one part- time staff member. It operates according to a nurse-to-nurse consultee/consultant model. That is, explicit written requests are made for psychiat- ric nursing assistance in handling the specific psy- chosocial concerns of a patient or, in many cases, a patient’s family members. In most instances, the goal of the service is to assist ward nurses in de- veloping a plan for nursing care, which is then implemented by the ward staff. When necessary, the nurse consultant maintains an active involve- ment in monitoring the implementation and effec- tiveness of this plan. Occasionally, patients requir- ing particularly intensive or specialized bedside psychotherapeutic interventions are followed over the long term.

The satisfaction survey was undertaken for all consultation requests answered over the 3-month period from January 1, 1988, to March 31, 1988. At the point of termination with a given patient (or at the end of March, 1988, for patients who were continuing), the psychiatric nurse consultant placed a copy of a Quality Assurance Feedback Form on the patient’s chart. The form was com- prised of two parts. One part was filled out by the nurse consultant, and included various demo- graphic and diagnostic information about the pa- tient, as well as information about the nature of the service provided (e.g., who was served, the elapsed time to response, the number of visits, what recommendations were made). The second

Page 3: Consultee satisfaction with a psychiatric consultation-liaison nursing service

266 NEWTON AND WILSON

part of the form was completed anonymously by the nurse who had been most involved with the consultant on the case. This section asked the re- spondent to rate, on 4-point scales, various fea- tures characterizing that particular consultation (e.g., ease of notifying the consultant, speed of reply, adequacy of written and verbal communica- tion with the staff, clarity and practicality of rec- ommendations, usefulness of suggestions in man- agement of the case, overall satisfaction with the nurse consultant’s contribution).

In order to encourage a high response rate, the feedback form was accompanied by instructions for the respondent to return it when completed to the Department of Psychiatry Service Evaluation Team, and a preaddressed envelope was enclosed.

RESULTS

Patient Characteristics

The psychiatric nurse consultants were involved with a total of 75 patients over the target time frame. Completed feedback forms were received for 69 of these patients, for an excellent overall response rate of 92%. Of the 69 patients for whom feedback was available, 30 were women and 39 were men. They ranged in age from infancy (in these cases the consultants were asked to respond to families in crisis) to 90 years (M = 50.23 yr). Although the patients themselves represented a wide range of medical problems, the most com- mon medical diagnoses were cancer (27.5%), kid- ney disease (18.8%)) cardiovascular disorders (14.5%), and trauma (11.6%). The consultation requests came from a variety of wards and ser- vices, including general medical units (37.7%), surgical wards (24.6%), and specialty units or clin- ics (24.6%).

At the time of responding to a consultation re- quest, the psychiatric nurse consultant coded the specific referral question as belonging to one of 19 possible categories. In fact, most could be clas- sifed as either nonspecific requests for evaluation (29%), requests for intervention with the patient’s family (24.6%), or requests for advice regarding nursing management ( 18%).

The consultants also made psychiatric diagnoses with each patient according to the criteria of the Diagnostic and statistical manual of mental disor- ders, revised ([DSM III-R] American Psychiatric Association, 1987). The diagnostic classifications

Table 1. DSM Ill-R Diagnostic Classifications

Diagnosis

Phase of life problem

Family circumstances

Organic mental disorder

Affective disorder

Adjustment disorder

Uncomplicated bereavement

Anxiety disorder

Sexual dysfunction

No psychiatric diagnosis

(staff conference only)

N %

31 44.9

14 20.3

7 10.1

5 7.3

4 5.8

3 4.4

2 2.9

1 1.4

2 2.9

are presented in Table 1. The largest category (44.9%) included patients who received the diag- nosis, Phase of life problem, which was used to classify individuals who were experiencing diffi- culties in coping with the stresses of their medical disorders (but who did not meet full criteria for diagnoses of specific psychiatric syndromes).

The second largest category (20.3%), Family circumstances, comprised patients who were in- volved in family conflicts, as well as patients whose family members were in need of psychoso- cial support. Excluding diagnoses of uncompli- cated bereavement, a total of 27.5% (19 of 69) of the consultation requests were for patients with pri- mary psychiatric disorders.

Consideration of DSM III-R Axis IV (Severity of psychosocial stressors) and Axis V (Global as- sessment of functioning in the past year) ratings confirmed that, for the most part, the patients (or their family members, when applicable) were usu- ally considered by the consultants to have been well-adjusted psychologically. Typically, their current difficulties could be classified as predict- able reactions (M Axis V = 74.68) to circum- stances of quite severe stress (it4 Axis IV = 4.62) associated with their medical problems.

Consultation Characteristics

As a matter of practice, the psychiatric nurse consultants attempt to respond as quickly as pos- sible to each consultation request, although staff- ing and workload considerations place limits on their response times. In the present series, 45 of 69 consultations (65.2%) received an initial acknowl- edgement within a day. However, an additional 17.4% of the consultations went for 4 days or more before receiving a response (usually extending over weekends, when no service is offered).

In 37 of the 69 cases (53.6%), the service pro-

Page 4: Consultee satisfaction with a psychiatric consultation-liaison nursing service

CONSULTEE SATISFACTION 267

vided by the nurse consultant was given directly to the patient. In an additional 12 ( 17.4%) other fam- ily members were seen in addition to the patient, while the family was seen alone in 14 (20.3%) cases. In the remaining eight requests, the consult- ant worked with the nursing staff rather then di- rectly with the patient or family. The amount of contact on a given case varied widely, ranging from one to 65 visits. In most instances, however, the involvement was short-term. Fifty-six consul- tations (81.2%) were completed within one to three visits.

At the point of termination with a case, the nurse consultant documented the recommendations that had been made according to 17 different catego- ries. These categories covered a variety of behav- ioral, diagnostic, and medication-oriented sugges- tions. As per the results shown in Table 2, the consultants relied primarily on seven general cat- egories of recommendation.

The most frequent recommendation was for fol- low-up from the consultant nurse in the monitoring of the proposed treatment plan, (36 of 69 = 52.2%). In general, this was offered whenever the management of the patient involved at least one follow-up visit by the consultant. Specific sugges- tions for modifying patient-staff interactions were also common (22 of 69 = 31.9%), as were rec- ommendations for further consultations by other specialists (18 of 69 = 26.1%).

Satisfaction with Consultations

The specific questions from the feedback form are shown in Table 3, along with the percentages of respondents who selected each response cate- gory for each question. The format of each ques- tion was designed so that the selection of ratings 3

Table 2. Consultant Recommendations

Recommendation

Have consultant follow-up

Recommendations concerning staff

interactions

Bring in other consultants

Arrange outpatient follow-up

Review/revise medications

Other specific recommendations

N %

36 52.2

22 31.9

18 26.1

7 10.1

3 4.3

3 4.3 No further contact beyond initial

meeting with consultee nurse 5 7.2

NOTE. More than one recommendation could be made in

any individual case.

Table 3. Responses to Quality Assurance Feedback Form

Rating

Item 1 2 3 4

Ease of notification 0 0 23.9 76.1

Promptness of response 0 0 33.3 66.7

Adequacy of written

documentation 0 3.1 65.6 31.3

Adequacy of verbal

communication 1.5 4.6 38.5 55.4

Clarity of recommendations 0 1.5 34.8 63.6

Practicality of recommendations 0 1.6 40.6 57.8

Recommendations were followed 0 8.3 66.7 25.0

Satisfaction with

recommendations 0 0 49.2 50.8

Suggestions were useful 0 1.6 42.9 55.6

Sufficient follow-up 0 11.9 62.7 25.4

Overall satisfaction 0 0 34.3 65.7

Likelihood of making another

referral 0 1.5 25.0 73.5

NOTE. For each item, the table entries refer to the percent-

age of consultee nurses who responded with ratings at each of

the different levels. In general, ratings of 1 and 2 correspond to

different levels of relative dissatisfaction with the consultant’s

performance, while ratings of 3 and 4 pertain to different lev-

els of satisfaction.

and 4 would allow the respondent to express two levels of satisfaction with the service, while re- sponse choices of 1 or 2 indicated relative dissat- isfaction. Occasionally, respondents did not an- swer individual items (for a total of 6.74% missing data). Usually, a missing data point was accompa- nied by a note indicating that the question was not applicable in the context of the particular case, or that the respondent was uncertain as to the appro- priate choice. The percentages shown in Table 3 are based on those cases for whom a valid response alternative was available.

Overall, as is apparent from Table 3, the con- sultees expressed high levels of satisfaction with the nursing consultation service. Only 10 of 69 replies (14.5%) had any individual items that were scored lower than 3 on the 4-point scale. The ques- tions addressing general levels of satisfaction (i.e., item 8, “How satisfied are you with the recom- mendations that were made?“; item 11, “Overall, how satisfied are you with the consultant’s input on this case?“) were particularly favorable. No responses lower than 3 (“satisfied”) were submit- ted for either of these items.

There was some indication that the 12-item scale could be considered meaningfully as a general sat- isfaction index with good internal consistency (al-

Page 5: Consultee satisfaction with a psychiatric consultation-liaison nursing service

266 NEWTON AND WILSON

pha = .84). Variability across items within the scale was examined by comparing the scores ob- tained for each individual item from the mean score derived from responses to the remaining items. In this series of 12 t-tests, five items emerged as having relatively high scores, while three had relatively low scores. Since some items had missing data, the degrees of freedom in these analyses varied somewhat.

Compared with the other items on the scale, the consultees found it very easy to notify the consult- ants, t(60) = 4.26, P < .OOl, and reported that the consultations were answered promptly, t(60) = 2.20, P < .05. The consultees also felt that the consultants were able to express recommendations clearly, t(61) = 2.56, P < .05. Finally, the con- sultees expressed high levels of overall satisfac- tion, t(61) = 3.90, P < .OOl, and reported that they would be more likely to make another con- sultation request in the future, t(61) = 5.75, P < .OOl.

Lower-than-average scores were found for items reflecting the adequacy of the consultants’ written documentation, t(60) = -4.54, P < .OOl, the extent to which there was sufficient follow-up, t(56) = -5.99, P < .OOl, and whether the con- sultants’ recommendations were actually followed, t(59) = -6.69, P < .OOl. In absolute terms, however, the average ratings for these items were still indicative of satisfactory performance by the nurse consultants (i.e., mean scores greater than 3.0).

Correlates of Consultee Satisfaction

A multiple regression analysis was conducted in order to determine whether the consultees’ satis- faction scores could be predicted by the demo- graphic or diagnostic characteristics of the pa- tients, or by aspects of the consultants’ responses. A stepwise hierarchical multiple regression proce- dure was used. Included as predictors were the patient’s age, sex, the presence/absence of a major Axis I psychiatric disorder, Axis IV and V diag- nostic ratings, presence/absence of family involve- ment in the consultation, elapsed time to consul- tation acknowledgement, and the number of visits. The criterion variable was the total satisfaction score obtained by summing responses to all 12 items (complete data were available for only 53 cases). The only variable to enter the equation as a significant predictor was the number of visits that

the nurse consultant had made on the case, F( 1,5 1) = 4.21, P < .05, Multiple R = .28, indicating that higher satisfaction ratings were given to con- sultations involving frequent visits by the consult- ant.

However, an alternative analysis was also in- structive. With this method, two groups of consul- tation replies were formed on the basis of their having received relatively high or low total satis- faction scores. The mean total was 42.32 (SD = 3.69). Scores below 40 (N = 14) were classified as “least satisfied, ’ ’ while scores above 44 (N = 15) were identified as “most satisfied. ” These scores corresponded with the upper and lower quartiles of the distribution of satisfaction scores. The two groups were then compared on a range of characteristics related to the consultation. Given the relatively small sample sizes for these analy- ses, Fisher’s Exact Test was used to make all sta- tistical comparisons.

The higher-rated consultations were less likely to pertain to patients with primary Axis I psychi- atric disorders (13.3% v 50%, for higher- versus-lower rated consultations, respectively, P = .04). Rather, these consultations more often involved requests for assistance in working with the patients’ families (33.3% v O%, P = .03). Finally, confirming the results of the regression analysis, it was evident that the more highly rated consults were also associated with more extended involvement by the consultant. All of the lower- rated consultations were conducted within 1 to 3 visits, while 40% of the highly-rated group in- volved four visits or more (P = .Ol).

DISCUSSION

Based on the present series of consecutive con- sultations, it is evident that the psychiatric nurse consultants are called upon by a variety of clinical services within the hospital. As with the series re- ported by Stickney and Hall (1981), however, pa- tients with cancer were the single largest group of medical patients to receive referrals. This suggests that part of the unique role that is emerging for nurse consultants involves work with terminal pa- tients and theirfamilies in dealing with issues per- taining to death and dying (Stickney & Hall, 1981). More generally across services, most of the consultation requests involved patients or family members who were having difficulties in coping

Page 6: Consultee satisfaction with a psychiatric consultation-liaison nursing service

CONSULTEE SATISFACTION 269

with the stresses associated with serious medical illnesses.

In general, the psychiatric nurse consultants pro- vide a prompt, short-term service in which an in- tervention plan is devised collaboratively with the ward staff. It was apparent from the results of the satisfaction survey that, for the most part, the nurses who made the initial consultation requests maintain a favorable view of this arrangement. In each of the 69 survey responses, items dealing most directly with global satisfaction reflected rat- ings of “satisfied” or “very satisfied.” Also of interest is the finding that in response to the ques- tion, “Were the consultant’s suggestions useful in the management of this patient?“, all but one con- sultee responded that the recommendations were “useful” or “very useful. ” This result stands in marked contrast to that of Sasser and Kinzie (1978- 79) in their survey of physicians’ attitudes towards psychiatric consultations. The reasons for the dis- crepancy may lie in differences between the two groups of consultees (i.e., physicians may be more critical of one another than are nurses) or in dif- ferences in the types of patients seen (i.e., psychi- atrists may see more difficult cases). Another ma- jor difference between the studies, however, is that whereas the present survey obtained evaluative feedback on a consultation-by-consultation basis, Sasser and Kinzie (1978-79) asked respondents to reply on the basis of overall past experience. This source of data is more heavily weighted towards global impressions, which may be biased by pre- vailing prejudices. For quality assurance purposes, it is clearly preferable to link evaluative ratings as closely as possible to actual case-by-case perfor- mance .

On the other hand, consumer satisfaction sur- veys are not without their own sources of potential bias. Studies conducted with patient groups have often found similarly favorable global satisfaction ratings for various mental health services (Lebow, 1983; Levois, Nguyen, & Attkisson, 1981; El Guebaly, Toews, Leckie, & Harper, 1983). Inter- pretively, it can be difficult to know on the basis of relatively high absolute ratings whether the respon- dent is genuinely pleased with the service pro- vided, or merely reluctant to provide negative feedback, even if justified (Lebow, 1983; Levois et al., 1981; El Guebaly et al., 1983). In some respects, the fact that the consumers of the consul- tation service are professional colleagues, with a

good understanding of the goals and limitations of consultations, provides a basis for accepting the present results at face value.

In addition, there was some consistent variabil- ity across items. This indicates that the consultees were not indiscriminately favorable in their rat- ings. Rather, it appears that they recognized par- ticular areas of relative strength and weakness in an overall context of generally satisfactory perfor- mance. Relative strengths emerged with regard to the ease with which the nurse consultants could be notified, the speed of response, and the clarity with which they were able to formulate their recommen- dations. These recommendations were not always followed completely by the ward staff. However, this may not necessarily point to an inadequacy in the nature of the recommendations themselves. Billowitz and Friedson (1978-79) have noted that in the case of psychiatrists’ recommendations, sug- gestions aimed at achieving direct results with little effort by the ward staff are followed to the greatest extent. Suggestions requiring more active involve- ment may be appropriate therapeutically, but more difficult to implement by busy ward personnel. Hence, these suggestions are less likely to be fol- lowed.

Survey items that pertained to written documen- tation and the amount of follow-up also received relatively low scores, suggesting that the consultee nurses might have appreciated more help in these areas than was actually provided. This appears to be particularly true of the amount of follow-up, which emerged in the regression analysis as the single most significant predictor of overall consul- tee satisfaction. In addition, however, the compar- isons of “most satisfied” and “least satisfied” responses also indicated that higher satisfaction ratings were more likely to pertain to family con- sultations, rather than to consultations involving patients suffering from major psychiatric disor- ders. Thus, the picture that emerges of the more highly rated consultation is one in which the nurse consultant is called in to help with a family in crisis, and stays on for an extended involvement as a primary therapist. By contrast, the typical low- rated consultation is one involving a patient with a major psychiatric difficulty, with whom the con- sultant makes behavioral and referral suggestions, but maintains less direct long-term involvement. In these cases, the consultee nurses may well have been hoping for more hands-on case management

Page 7: Consultee satisfaction with a psychiatric consultation-liaison nursing service

270 NEWTON AND WILSON

than could reasonably be provided by the nurse consultants, or perhaps the facilitation of transfers to psychiatry wards. This finding underscores the need for ongoing liaison activities between the consultation service and the ward staff, with con- tinuing education as to the scope of the nurse con- sultant’s role in the context of the overall psychi- atric consultation team.

CONCLUSION

In summary, the present survey was able to doc- ument that medical-surgical nurses are very appre- ciative of the assistance they receive from the psy- chiatric consultation-liaison nursing program. The responses also provided the consultants with some objective feedback regarding the manner in which their performance is viewed by the consumers of their services. Nevertheless, because the study was necessarily limited to one hospital, replication in other settings is required before any broad gener- alizations can be made about the field of consul- tation nursing as a whole. In addition, it would clearly be worthwhile to include those other ser- vice consumers-the patients and their families- in future research of this type. In conclusion, how- ever, it should be emphasized that the implemen- tation of the survey required relatively little time and effort on the part of either the consultants or the consultees. Hence, it provided a simple and cost-effective means of program evaluation, yield- ing data of practical value in the quality assurance effort. Based on these data, we are able to provide a relatively compelling demonstration of the value of the consultation-liaison program as perceived by the nurses who rely on it.

REFERENCES

American Psychiatric Association. (1987). Diagnostic and sta- tistical munuul, (ed. 3, rev.). Washington, DC: Author.

Barton, D., & Kelso, M.T. (1971). The nurse as a psychiatric

consultation team member. Psychiatry in Medicine, 2, 108-115.

Billowitz, A., & Friedson, W. (1978-79). Are psychiatric con- sultants’ recommendations followed? International Journal of Psychiatry in Medicine, 9, 179-189.

El Guebaly, N., Toews, J., Leckie, A., & Harper, D. (1983). On evaluating patient satisfaction: Methodological is- sues. Canadian Journal of Psychiatry, 28, 24-29.

Goldberg, R.J., Tull, R., Sullivan, N., Wallace, S., & Wooly, M. (1984). Defining discipline roles in consultation psychiatry: The multidisciplinary team approach to psy- chosocial oncology. General Hospital Psychiatry, 6, 17-23.

Lebow, J.L. (1983). Research assessing consumer satisfaction with mental health treatment: A review of findings. Evaluation and Program Planning, 6, 21 l-236.

Levois, M., Nguyen, T.D., & Attikisson, C.C. (1981). Arti- fact in client satisfaction assessment: Experience in mental health settings. Evuluution and Program Plun- ning, 4. 139-150.

Lipowski, Z.J. (1981). Liaison psychiatry, liaison nursing, and behavioral medicine. Comprehensive Psychiatry, 22, 554-561.

Lipowski, Z.J., & Wolston, E.J. (1981). Liaison psychiatry: Referral patterns and their stability over time. American Journal of Psychiatry, 138, 1608-1611.

Lorefice, L.S., Borus, J.F., & Keefe, C. (1982). Consumer evaluation of a community mental health service. I: Care delivery patterns. American Journal of Psychiatry, 139. 1331-1334.

Moses, R., & Barzilay, S. (1967). The influence of psychiatric consultation on the course of illness of the general hos- pital patient. Comprehensive Psychiatry, 8. 16-26.

Robinson, L. (1982). Psychiatric liaison nursing 1962-1982: A review and update of the literature. General Hospital Psychiutry, 4, 139- 145.

Sasser, M., & Kinzie, J.D. (1978-79). Evaluation of medical- psychiatric consultation. Inrernufionul Journal of Psy- chiatry in Medicine, 9, 123-134.

Schwab, J.J., Clemmons, R.S., Valder, M.J., & Raulerson, J.D. (1966). Medical inpatients’ reactions to psychiatric consultations. Journul of Nervous and Mental Diseuse, 142, 215-222.

Schwab, J.J. (1967). Evaluating psychiatric consultation work. Psychosomurics, 8, 16-26.

Stickney, S.K., & Hall, R.C.W. (1981). The role of the nurse on a consultation-liaison team. Psychosomufics. 22, 224-235.