The Introduction of Computer Assisted Telephone Support in a Community Mental Health Setting
Eve Agee, Michael Blank, Jeanne C. Fox, Barbara J. Burkett, * John Pezzoli; Southeastern Rural Mental Health Research Center, University of Virginia
Abstract
This study examined the usefulness of a computer-based telecommunications
technology, SHARENET, designed and implemented to enhance delivery of mental
health services by offering (1) a more efficient forum for interaction between
providers a consumers and (2) an additional social support network for consumers.
This article presents the qualitative results of this study. These results
are discussed in association with problems that arose from the designers'
lack of experience with seriously mentally ill individuals and emphasize
the importance of adapting technology to fit the special needs of particular
populations. Additionally, discussion of system implementation and usage
demonstrates the importance of recognizing the reluctance of service providers
to embrace new methodologies and technology that they believe to be in conflict
with their own treatment philosophies.
The role of the mental health care provider is constantly being redefined and expanded in response to changes in health care market forces. New technologies are rapidly being designed to facilitate delivery and quality of care. Mental health providers have expressed concerns that these technologies may disrupt care or replace face to face contact. In an era of cost cutting and managed care, such fears may lessen provider acceptance of new technologies. To date, only one study has examined the impact that the introduction of new technology may have on service providers (Saetnan, 1991).
In October 1995 the Southeastern Rural Mental Health Research Center, an interdisciplinary rural mental health research institute at the University of Virginia conducted a study of a computer-based telecommunications technology, called ShareNet. A state mental health service agency served as the study site (name and location of the agency and the technology have been changed to protect consumer and staff confidentiality).
The purpose of the study was to determine if ShareNet could enhance the delivery of services by offering a more efficient forum for interaction between providers and consumers, as well as providing an additional network of social support for consumers. This article presents only the qualitative results of this study.
Background
The research on computer-assisted care systems accessed via touch tone telephones,
although limited, suggests that this technology can effectively improve
communication between patients with physical ailments and their health care
providers (Alemi & Higley, 1995; Alemi, Stephens, & Butts, 1992).
Estroff (1981) reported that mental health consumers in the U.S. frequently
have difficulties developing vital social networks. Therefore, the Southeastern
Rural Mental Health Research Center decided to test the efficacy of a computer
based telephone service intended to offer (1) a more efficient forum for
interaction between providers and consumers and (2) an additional social
support network for consumers, in a mental health care setting.
Hypotheses
The research team hypothesized that the ShareNet system would: (1) facilitate
consumers in building auxiliary support networks, (2) decrease consumer
usage of crisis lines, and (3) increase consumer contact with case managers.
Method and Materials
Participants
All clients of the state mental health agency who met the state Medicaid criteria for severe and persistent mental illness (N=425) were randomized to either an experimental or control group in August 1995. Following the randomization process, the research staff and case managers contacted potential study participants and extended an invitation to become involved in the study. The research staff visited consumers in psychosocial rehabilitation programs, and adult care homes, and telephoned consumers living in private homes. Consumers who chose to participate were paid $10 upon completion of two training sessions.
Forty-three percent (n=89) of those randomized to receive ShareNet training (the experimental group) chose to participate and underwent training during a four-month period. The research staff continued to provide follow-up training for consumers at the state mental health agency or in participants' homes as needed.
The ShareNet System
ShareNet was designed and technically supported by a private corporation for the entire 16 month research period. This corporation had previously designed similar computer-assisted care systems for consumers with physical health problems. The ShareNet system was physically located at a state mental health department, which served approximately 500 consumers with serious and persistent mental illness. In addition to the 16 case managers who actually worked with the ShareNet system, the mental health staff included three psychiatrists, three supervisors, two pharmacists, and two secretaries.
Consumers accessed the system menu via their touch tone telephone. Consumers dialed the ShareNet number then entered their personal access code followed by their password. The ShareNet menu consisted of 7 options, referred to as (1) Care Mail, (2) TalkNet, (3) Inspirational Messages, (4) Calendar of Events, (5) Info-Line, (6) Dispatch, and (7) Reminder System. Alternatively, ShareNet called consumers to alert them to updated information, and to provide reminders. Calls to consumers were placed during times pre-specified by the consumers as being convenient. Once the phone was answered, ShareNet would identify itself then ask the person answering to press 1 or 2 to indicate whether he or she was the intended respondent. If the user selected 1, he or she would then enter his or her individual password. If ShareNet received the correct password, it would then relay pre-recorded information.
Using the Care Mail feature, consumers and/or case managers could initiate and transmit confidential messages. To send a message, the user entered the telephone number of the person they wished to contact and the selected message would be sent to the designated account during the time specified by the receiving party. After receiving messages, users could respond to the sender, save or delete the message.
TalkNet was an 'electronic support network' for consumers, and consisted of recorded, non simultaneous discussion groups initiated by consumers. Users could listen to recorded messages related to ongoing discussions, add their own messages related to existing topics, or begin new topics. Upon initiating this feature, users were informed that others could hear any messages they recorded. When users began new discussion topics, they recorded topic headings, which were added to the list of active discussions. Discussion topics included relationships, sexually transmitted diseases, and dreams, as well as the ShareNet system itself.
The Inspirational Messages option provided spiritual and motivational messages, and the Calendar of Events provided information for consumers regarding hours of operation and activities at Local Outreach. Consumers and staff at two of Local Outreach's psychosocial rehabilitation centers regularly updated the Inspirational Message line. Staff and ShareNet participants in the clerical unit of the state mental health service regularly updated the Calendar of Events.
The Info-Line option provided a venue for consumers to ask questions on a variety of subjects while allowing others the opportunity to hear the both questions and the providers' responses. After entering Info-Line, consumers chose to either listen or ask a question. Consumers who wanted to listen to questions could select the subject headings for the questions and responses they wanted to hear. Consumers with questions to ask chose particular mental health experts for whom they then recorded their questions. ShareNet contacted the specified expert, who could then record a response at his/her convenience and, if appropriate, post the questions and responses in the Info-Line menu.
The Dispatch option was an automated assessment system, which administered and recorded responses to standardized instruments, including health status and quality of life questionnaires. Consumers were asked a series of yes/no questions that were to be answered by pressing 1 or 2 on touch tone phones. The computer stored the responses and downloaded the data into quantitative software programs for subsequent analysis. The questionnaires used included (1) the Quality of Life Interview (Lehman, 1988), (2) the Brief Symptom Inventory (Derogatis & Melisaratos, 1983), (3) the Medical Outcomes Survey (Ware & Sherbourne, 1992), and (4) the Consumer Satisfaction Questionnaire (Nguyen, Attkinsson, & Stegner, 1983).
Using the Reminder System, providers could program ShareNet up to one
year in advance to contact a consumer on the day before an appointment and
to request confirmation of the consumer's intention to attend. If the consumer
indicated that he could not attend the appointment, ShareNet contacted the
provider to reschedule another appointment and notified the consumer of
the new appointment date and time. This feature could also be used to provide
medication reminders.
The research staff met with administrators of the state mental health agency
in July 1995 to introduce and discuss the ShareNet system. Case managers
were trained on ShareNet during August 1995 and the program implemented
in October 1995. During discussions, case managers decided they would all
moderate TalkNet discussion groups on a regular basis to ensure that any
potentially harmful statements made did not remain on the public forum.
Evaluation Procedures
Interviews were conducted with the sixteen case managers employed at the state mental health agency. The initial interviews were conducted in August 1995, before ShareNet was implemented, to examine case managers' expectations of the system. Follow-up interviews were conducted one year later to assess changes in case managers' attitudes over time. Two of the case managers who participated in the initial interviews were not present for the follow-up interviews because of staff changes. Consequently, fourteen case managers participated in both initial and follow-up interviews.
All interviews were semi-structured, using previously selected, open-ended questions designed to elicit case managers' perceptions of ShareNet. All interviews were conducted by the primary author. The interviews were taped and transcribed. The transcripts were coded and analyzed by thematic analysis (Miles & Huberman, 1994, pp 338) to identify the major themes in each transcript. Each theme was then cross-coded through use of second-level explanatory codes that facilitated analysis among and between themes. Analyses were then examined and verified by testing out competing explanations. The conclusions were presented to the case managers and their responses were noted.
In addition, the investigators observed consumer interaction with the state mental health agency case managers from August 1995 through January 1997. Field notes were recorded immediately after each observation session. These notes were analyzed and included with the interviews.
Results
Case Manager Usage
Case managers used the ShareNet system for a total of 61.42 hours during the 16-month research period. The average usage time was 16 minutes per month; and each case manager logged in an average of six times each month. Care Mail was the most popular ShareNet option with 92.9% (n=13) of the case managers utilizing this service. Alternatively, the Reminder option was least utilized with only 4 case managers trying this option.
Consumer Usage
ShareNet was used by consumers for a total of 1,205 hours. However, three
consumers were responsible for 85% of the total hours. When these three
consumers were included in the calculations, the average connection time
per consumer was 52 minutes per month. However, when these three consumers
are excluded, the average monthly connection time drops to 8 minutes. Among
all consumers, Care Mail was the most popular ShareNet feature. However,
the three consumers who utilized ShareNet most frequently accessed TalkNet
more than any other option.
Case Managers' Evaluation of ShareNet: Initial Interview
During the initial interviews, case managers expressed varied expectations of ShareNet's effectiveness. Four of the case managers stated that they thought ShareNet would benefit the consumers of the state mental health agency's services. These four case managers said that, in addition to the potential benefits of increased social support, learning the skills needed to navigate within ShareNet could help consumers in other areas of their lives.
However, the majority of the case managers were uncertain about the feasibility of implementing ShareNet technology with this consumer population. Several case managers considered ShareNet too difficult for many of the consumers.
Reactions to the individual features of ShareNet also varied. Most case
managers expected Care Mail to be a time-saving device, allowing consumers
to leave exact messages while decreasing the number of calls receptionists
would receive. Five case managers were enthusiastic about the Reminder System
and expressed expectations that this option would "help people make
appointments" and limit the amount of time lost because of missed appointments.
One case manager said,
Several of the case managers considered the appointment and medication Reminder
System a good idea, but were uncertain about its efficacy with this consumer
population. These case managers expressed the opinion that consumers who
make regular appointments consistently show up for those appointments. Among
the comments were
and
One case manager who worked primarily with consumers who lived in adult
care homes and attended psychosocial rehabilitation centers stated,
Three case managers with similar caseloads also stated that they would not
have much need for an appointment reminder but acknowledged the potential
benefit for other consumers.
Many of these same case managers expressed similar attitudes about the potential
for medication reminders. Even though many thought that this was a good
idea in theory, several commented that they were uncertain how it would
work in practice. One case manager said,
Four other case managers mentioned that medication reminders might negatively
affect those consumers with a paranoid component
to their illness.
Case Mangers' Perspectives One Year Later
The follow-up interviews and participant observations with the case managers
suggested that their patterns of low utilization were partially shaped by
their initial expectations of the ShareNet services. Care Mail was the one
aspect of ShareNet in which case managers had expressed initial interest.
The usage patterns show that those case managers who indicated the greatest
initial interest later used Care Mail significantly more than other ShareNet
features.
At the time of the follow-up interviews, many case managers viewed ShareNet
as an impersonal, non-interactive means of communication with consumers.
They were concerned about their relationships with participants who had
become frustrated with the ShareNet system. In addition, some case managers
stated that ShareNet discouraged consumers from engaging in a dialogue about
an issue or concern. Case managers said that they considered getting consumers
to interact and talk about their concerns to be part of their job.
During interviews, case managers characterized the majority of the consumers as "withdrawn," "isolated," and "retiring," and suggested that what they considered to be the "non-interactive character" of ShareNet encouraged or at least allowed the kind of withdrawn behavior that they were working with consumers to change.
Discussion
Perhaps the most valuable result of this study has been the opportunity to identify potential problem areas related to the design and implementation of new technology in mental health settings, highlighting the importance of tailoring system operation and function to meet the needs of this specialized population.
Identified Problems
It is very likely that the adoption of competing technology at the same agency, the introduction of voice mail and personal computers for case managers, recruiting problems related to the final experimental design, and frustration related to technical problems, adversely affected consumer and case manager attitudes toward ShareNet.
Unfortunately, several ShareNet features did not operate as originally
described and the system designers were slow in responding to the specialized
needs of consumers and case managers. Consequently, technical difficulties
were common throughout the research period.
A change in experimental design due to the relocation of the state mental
health agency office in January of 1996 affected both recruitment of consumers
and user utilization. The research team initially planned on randomizing
consumers at the case manager level. However, agency administrators anticipated
changes in case manager case loads during the research period and decided
in July 1995 that randomization at the case manager level would cause difficulties.
Therefore, a design that randomized participants to experimental and control
groups at the consumer level was implemented. As a result, only 10% to 20%
of the clients in each case manager's case load had been trained on the
use of ShareNet. Case manager usage and the perceived usefulness of the
technology may have been influenced by the low percentage of ShareNet participants
per case manager. Many of the group features ShareNet offered, such as a
function which allowed the case manager to send the same message to all
consumers on a list, appear to have been less than ideally useful for the
case managers because of the relatively small percentage of their total
case load with ShareNet training. All case managers commented that if they
could have contacted their entire caseload population through ShareNet,
they would have found it more effective and time saving.
Another factor that influenced usage of ShareNet was the introduction of a voice mail system at the new location. Before this, consumers had to dial a central telephone number and talk with the receptionist to reach their case managers. If their case managers could not speak with them, consumers had to leave messages with the receptionist or call back later. Although ShareNet, allowed consumers to leave confidential messages, the new voice mail system also had this capability and involved dialing only one number. Many consumers perceived voice mail as easier to use and preferred it to ShareNet.
After moving to the new location, the state mental health agency provided
each case manager with a personal computer. The time investment required
to develop effective computer skills may have decreased the time that case
managers spent utilizing ShareNet.
Case managers' use of the ShareNet system was additionally shaped by consumer
difficulties and their resultant expressions of frustration. The original
log-in procedure required consumers to enter their seven-digit telephone
number as an access code to be followed by their password. Consumers who
lived in group homes where they shared the same phone had to enter an extension
number between the access code and the password. Participants frequently
had difficulty remembering numbers and entering them into the system within
the allotted time.
Case managers worried that encouraging consumers to use a technology perceived
as too difficult could jeopardize treatment relationships. The log-in procedure
was simplified to a three-digit access code followed by a four-digit password
in May 1996. However, by that time, many consumers were already frustrated
with ShareNet.
The Reminder and Dispatch systems had major programming and usage problems. Case managers were originally told they could program the Reminder System to call consumers at specified times. However, once ShareNet was implemented, it was learned that the Reminder System could not be programmed to deliver a reminder on a specific date or time. In the event that ShareNet reminders were delivered during hours when users had chosen not to receive ShareNet calls, the message would not be delivered.
The Dispatch component of ShareNet could not be implemented until May 1996. Unlike other delays in the system, the absence of the Dispatch was not noticed by case managers or consumers because the Dispatch was used only by the research staff.
The Dispatch component was programmed to verbally deliver one questionnaire per week. Many participants, previously frustrated by other aspects of ShareNet, simply hung up on the system when it called. However, ShareNet would continue to call the consumer until they responded to the Dispatch. Consumers who did not respond would receive calls from ShareNet every day until the consumer answered the Dispatch questions. Some consumers were called more than once a day, depending on how they set their calling parameters. Participants, family members and group home staff became very frustrated by these calls. Many participants complained to their case managers, and 23 consumers chose to be voluntarily suspended from the system. In response to consumer complaints, several case managers decided to cease any contact they had with consumers on ShareNet.
The randomization of the consumers to either the ShareNet intervention or control group was problematic. Many consumers in the control group wanted to join ShareNet or have the opportunity to make the $10 for participating in the training sessions. This contention concerning opportunity for payment was particularly apparent among consumers who attended group functions or lived in group settings. Other researchers have noted that monetary inducement to participation can create demand problems when some individuals within a small network are excluded from the opportunity of financial gain. This situation proved particularly problematic for case managers, as it conflicted with the goal of affirming consumers' importance by including them in agency services and activities. Case managers reported that inclusion of consumers is an essential part of treatment because consumers have been excluded from so many areas of society.
The potential for investigator bias may have existed. The investigators trained case managers to use ShareNet, met with them on a regular basis to discuss their problems and concerns with the system, and encouraged them to increase their utilization of the system. As a result, the investigators' role increased beyond that of mere observers. However, as Estroff (1981) suggests, such circumstances also provide an opportunity to be accepted as part of the staff. If such was the case, closer relationships with case managers may have been facilitated which, in turn, may have increased participant observation opportunities.
Conclusions
Unfulfilled expectations of ShareNet benefits, perceived constraints imposed by the experimental design, conflicts with treatment ideology, and the introduction of competing technologies all adversely affected case manager attitudes and utilization of ShareNet. Despite initial expectations that ShareNet could save time, after one year of implementation, case managers did not view this technology as either enhancing service delivery or saving time. Rather, they viewed it as conflicting with their treatment goals.
Borgman (1984) suggests that as technological devices are increasingly employed to disburden people from social and bodily engagement, these devices simultaneously restrict people from participating in certain types of actions and relationships. Borgman contends that the inevitable result of using a new technology is a change in the person's experience of the world that entails negative as well as positive implications. For the case managers, problems with implementation and conflict between technological demands and treatment ideology led to the perception that ShareNet lacked the capacity for responsive interaction. This perception proved to be a significant factor in the case managers' rejection of the technology. However, the adoption of other competing technologies such as a voice mail system and personal computers confirms that case managers would embrace new technologies in service delivery which they considered to be useful and accessible.
Many challenges should be expected when conducting research and implementing new technologies in a health services setting. This exploratory project faced many difficulties because the designers of this technology lacked experience working with a public mental health service agency. The special needs of the study population were not addressed prior to initiation of the study. To successfully implement a similar computer-assisted telecommunications system, case managers need to participate in the design of the technology. Successful implementation of technology requires eliciting case manager involvement, compatibility with treatment ideologies and knowledge of the population to be served.
Alemi, F., & Higley, P. (1995). Reaction to talking computers in assessing
health risk. Medical Care, 33(1),
Alemi, F., Stephens, R. C., & Butts, J. (1992). Case management: A telecommunication practice model. Progress and Issues in Case Management. U.S. Department of Health and Human Resources, 261-273.
Borgman, A. (1984). Technology and the Character of Contemporary Life. University of Chicago Press, Chicago:IL.
Derogatis, L., & Melisaratos, N. (1983). The brief symptom inventory. Psychological Medicine, 13, 595-605.
Estroff, S. (1981). Psychiatric deinstitutionalization: A sociocultural analysis. Journal of Social Issues, 37(3), 116-132.
Lehman, A. (1988). The quality of life interview for the chronically mentally ill. Quality of Life, 11, 51-62.
Miles, M., & Huberman, A. (1994). Qualitative Data Analysis: An Expanded Sourcebook. (2ed.). Sage Publications, Inc. Thousand Oaks:CA.
Nguyen, T., Attkinsson, C., & Stegner, B. L. (1983). Assessment of patient satisfaction. Evaluation and Program Planning, 6, 299-314.
Saetnan, A. R. (1991). Rigid politics and technological flexibility: The anatomy of a failed hospital intervention. Science, Technology and Human Values, 16(4), 419-447.
Ware, J., & Sherbourne, C. (1992). The MOS 36 item short form health survey. Medical Care, 30(6), 299-314.
© 1997, Journal of Rural Community Psychology