Process Outcome Measures > Treatment Outcome | Treatment Process | Training Outcome


Treatment Process

 

Process measures are designed to help evaluators understand how a program or intervention works. For motivational interviewing programs, process measures are designed to evaluate therapist fidelity (the degree to which motivational interviewing is being administered as intended), interpersonal constructs that are thought to be important drivers of the effects of motivational interviewing, or client reactions that may moderate motivational interviewing. Because these measures tap into the core processes by which MI may exert its effects, many of them are also often used as measures of motivational interviewing training outcome.

 

Special Issues in Treatment Process Research

 

- Gathering Practice Samples

 

- Motivational Interviewing Treatment Integrity Scale 3.1.1 (MITI)


- Behavior Change Counseling Index (BECCI)


- Motivational Interviewing Skills Code (MISC 2.1)


- Motivational Interviewing Sequential Code for Observing Process Exchanges (SCOPE)


- Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (MIA:STEP)


- Working Alliance Inventory


- Motivation (link to Treatment Outcome page)

 

 


 

Special Issues: Gathering Practice Samples

 

A necessary component of many treatment process instruments is that they require coding of practice samples. There are several strategies for collecting practice samples, each with their own unique benefits and drawbacks. In this section, we will highlight a few of the option available.

 

Live Observation of Clients: This involves sitting in on clinician interactions and coding in real time. The main benefit of this approach is the likely high external validity or generalizeability of coding, as there is not opportunity for the clinician to select clients or interactions with specific characteristics. An additional benefit of this approach is that if any feedback will be given regarding the practice sample, it can be done immediately following the interaction, when it is more likely to have an effect. In some cases, live observation can be less logistically challenging, than other approaches as it does not require special scheduling for clinicians or clients or the use of recording devices. In settings where client flow is slow or variable, there are high no show rates, or there are often arising emergencies, live observation can sometimes be inefficient for the observer, who may have to wait for long periods until an appropriate candidate for observation arrives. Additionally, live observation has the potential to be uncomfortable for clients and clinicians and could alter normal behavior in important ways.

 

Video or Audio Taping of Clients:

Another method for collecting practice data involves asking clinicians to provide audio- or videotaped work samples. Samples should consist of an interview with an actual client being treated by the provider. In order to be accurately rated with most instruments, the sample should be at least 20-minutes in duration and should focus on a clear target behavior (e.g. alcohol misuse, smoking, medication adherence). Digital audio recorders are becomingly an increasingly cost-effective, convenient, and popular means of gathering practice samples as they allow for the easy recording, playback, storage, and transmission of audio files.

 

Recording real client interactions raises important privacy concerns. The purpose of recording as for supervisory or program evaluation purposes should be explained to the client and signed consent for audio or video-taping should be obtained. Tapes should be stored in a secure location and destroyed after use. Unless non-verbal or other video images will be coded, it may be advisable to use an audio-tape format which is less invasive and is less identifiable should a tape be lost or inadvertently viewed.  A client’s decision to consent to record should be voluntary and it should be made clear that refusal to tape will not affect their care in any way. Despite these concerns, many agencies report high rates of agreement to be taped by participants.

 

Another important consideration of this approach is the documented difficulty that can arise from provider compliance with practice sample submission requirements. These difficulties have been documented even in tightly controlled randomized trials (Miller et al., 2004), which often have more resources and time to encourage and track submission than are available in a typical evaluation study. In this study no more than 76% of providers (with decreasing submission rates over time) mailed in practice samples despite repeated prompts and self-selection to participate in the training study. Improved rates of compliance may be possible in workplaces in which submission of tapes is linked to supervisor feedback or performance evaluation. 

Standardized Patients: If real-client interactions are not feasible for an evaluation, standardized patients may provide another option. Standardized patients (SPs) are actors, or other professionals, who are trained to engage providers in different types of clinical encounters for the purposes of training and evaluation. SPs are becoming increasingly used to train and evaluate providers in the use of communication skills, such as motivational interviewing.  SPs can be used as educational tools or for program evaluation. These interactions can be audio or video recorded to allow for the objective rating of interactions by coders. Some studies have also trained SPs to provide evaluation and feedback themselves.  SPs can be planned or unplanned and announced or unannounced.  Depending on the nature of the project and the sensitivity of results to variability in SP performance, it may sometimes be feasible and appropriate to train research staff or lay people, versus professionally trained actors, to perform as SPs. In fact, research suggests that in some circumstances peer role plays may be just as effective as simulated patients. One benefit of SPs is that they can allow large batches of practice samples to be gathered at specific times, such as training events or staff meetings which will enhance compliance. For a full discussion standardized patients see: http://hpp.sagepub.com/content/13/2/169.full

 

Several tips for creating effective roles for evaluation include give the standard patient plenty to of material regarding their motivation state, including reasons to change the target behavior, reasons not to change the target behavior, and a readiness estimate.  For optimal demonstration of MI skills, it can be helpful to develop patient roles that involve some ambivalence and moderate levels of readiness (e.g. 7 on a scale from 1 to 10). In addition, in pre-post or multiple group designs, remember to create at least two patient roles and counterbalance them across participants.

 

Relevant Citations:

 

Baer, J. S., Wells, E. A., Rosengren, D. B., Hartzler, B., Beadnell, B., & Dunn, C. (2009). Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing training for community counselors. Journal of Substance Abuse Treatment, 37, 191-202.

 

Lane, C., Hood, K., & Rollnick, S. (2008). Teaching motivational interviewing: Using role play is as effective as using simulated patients. Medical Education, 42, 637-644.

 

Weaver, M. & Erby, L. (2011). Standardized patients: A promising tool for health education and health promotion. Health Promotion and Practice, 13: 169-174.

 

Miller, W.R., Yahne, C.E., A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050-1062.

 

Special Issues: Determining Inter-rater Agreement

 

 

Arrow

 

 

(back to top)

 


 

Motivational Interviewing Treatment Integrity Scale (MITI 3.1)

 

The Motivational Interviewing Treatment Integrity Scale (MITI; Moyers et al. 2005; Moyers et al., 2010) is an instrument designed to measure the degree to which a practitioner is interacting with a client in way that is consistent with motivational interviewing. The MITI is the most commonly used measure of MI fidelity within research and clinical contexts.

 

To use the MITI, trained raters observe or listen to a 20 minute segment of an interaction. The MITI yields two types of scores: behavior counts and global scores. During the interaction, the rater counts specific MI behaviors, such as questions and reflections. These are later tallied and specific summary scores, such as reflection to question ratio, are calculated. Following the interaction, the rater gives his or her overall judgment of the consistency of the interaction with specific MI dimensions, such as empathy, on a five point scale. For both behavior counts and global measures, the MITI includes score thresholds that are suggested for beginning proficiency and competency in MI. To reliably use of the MITI, it is recommended that raters participate in a MITI training course and demonstrate inter-rater agreement with an established rater.

 

The MITI has been through several iterations but the most current version, 3.1.1, is available here.

 

The developers of the MITI also provide several coded transcripts which can be used to practice with coding: http://casaa.unm.edu/code/miti.html

 

Relevant Citations:

 

Moyers, T.B., Martin, T., Manuel, J. K., Miller, W.R., & Ernst. D. (2010). Motivational Interviewing Treatment Integrity 3.1.1 : Revised Global Scales.

 

Moyers, T. B., Martin, T., Manuel, J. K., Hendrickson, S. M. L., & Miller, W. R.  (2005).  Assessing competence in the use of motivational interviewing.  Journal of Substance Abuse Treatment, 28(1), 19-26.

 

Arrow

 

 

(back to top)

 


 

Behavior Change Counseling Index (BECCI)

 

The Behavior Change Counseling Index (BECCI; Lane et al., 2002; Lane et al., 2005) is brief measure designed to measure skillfulness at conducting motivational interviewing based consultations in healthcare settings. The BECCI includes 11 Likert-scale items, which yield a total score. The instrument has an accompanying detailed coding manual describing the development of instrument and explanation of each item. The primary purpose of the BECCI is to assess changes in practitioner behavior before and after training, or to assess the degree to which interventionists are administering MI-informed behavior change counseling as intended. The instrument has acceptable levels of validity, reliability and responsiveness (Lane et al., 2005). The coding manual and instrument can be found here.

 

Relevant Citations:

 

Lane, C., Huws-Thomas, M., Hood, K., Rollnick, S., Edwards, K., & Robling, M. (2005). Measuring adaptations of motivational interviewing: The development and validation of the behavior change counseling index (BECCI). Patient Education and Counseling, 56, 166-173.

 

Arrow

 

 

(back to top)

 


 

Motivational Interviewing Skills Code (MISC 2.1)

 

The MISC is a detailed coding system for documenting processes of motivational interviewing. The MISC can be used to measure fidelity, provide feedback to clinicians, or evaluate the effectiveness of training, but, because of its complexity, has been largely replaced by the MITI in research and evaluation endeavors. The MISC is still used as a valuable tool for conducting in depth process evaluations seeking to identify the mechanisms of action of motivational interviewing. The MISC is completed in 3 passes, requiring coders to record global impressions, as well as track 18 counselor behaviors, and 7 client behaviors. The instrument is available here.

 

Relevant Articles:

 

Moyers, T.B., Miller, W.R., & Hendrickson, S.M. (2005). How does motivational interviewing work? Therapist interpersonal skill predicts client involvement within motivational interviewing sessions. Journal of Consulting and Clinical Psychology, 73: 590-598.

 

Bertholet, N., Faouzi, M., Gmel, G., Gaume, J., & Daeppen, J.B. (2010). Change talk sequence during brief motivational intervention, towards or away from drinking. Addiction, 105: 2106-2112.


Gaume, J., Gmel, G., Faouzi, M., Daeppen, J.B (2009). Counselor skill influences outcomes of brief motivational interventions. Journal of Substance Abuse Treatment, 37, 151-159.


Guame, J., Gmel, G., Faouzi, M., Daeppen, J.B. (2008). Counsellor behaviors and patient language during brief motivational interventions: a sequential analysis of speech. Addiction, 103: 1793-1800.

 

Arrow

 

 

(back to top)

 


 

Motivational Interviewing Sequential Code for Observing Process Exchanges (SCOPE)

 

The SCOPE is a tool designed to analyze processes of motivational interviewing that may occur in sequential exchanges between therapist and client. Administration of the SCOPE requires that taped therapy interactions be recorded and transcribed. During a first pass, trained raters must parse the transcript into utterances. During a second pass therapist and client utterances are coded as belonging to behavioral categories that represent important theoretical constructs in MI (30 categories for therapist, 6 for clients). Use of the instrument requires extensive training and is likely only appropriate for those with a strong research interest in psychotherapy processes. The SCOPE has been applied to clinical research on therapeutic interactions to investigate the impact of in-session therapist behaviors on in-session client language and the moderating impact of client language on outcome (Moyers et al., 2007).  The coding manual can be accessed at: http://casaa.unm.edu/download/scope.pdf


Relevant Articles:


Moyers, T.B. , Martin, T., Christopher, P.J., Houck, J.M., Tonigan, J.S. (2007). Client language as a mediator of motivational interviewing efficacy: Where is the evidence? Alcoholism: Clinical and Experimental Research, 31: 40s-47s.

Arrow

 

 

(back to top)

 


 

Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency (MIA: STEP)

 

The MIA: STEP (Martino et al., 2006) was developed as part of a part of an initiative designed to bring interventions with good research evidence into the hands of clinicians. The package is a collection of tools for mentoring clinicians in the use of MI skills while conducting assessments. Included is a manual for rating clinician adherence and competence that was based on an adaptation of a broader clinical competence measure called the Yale Adherence and Competence Scale. The rating system was designed to be applied to 20 minute MI interaction segments that could precede and follow a substance abuse assessment, but, in principle, could be applied to any sample of MI practice.

 

The rating scale consists of 10 MI-consistent behaviors and 6 MI-inconsistent behaviors that are evaluated in terms of frequency/extensiveness and competence/skill level. The manual provides detailed descriptions of all rating items, as well as consolidated clinician skill summaries that can be used for self-assessment. The manual also includes guidance on strategies and skills for providing feedback to clinicians based on their ratings. To use the rating system, the developers recommend that supervisors have completed a two day introductory MI workshop. The MIA:STEP Manual can be accessed here.

 

Relevant Citations:

 

Martino, S., Ball, S.A., Gallon, S.L., Hall, D., Garcia, M., Ceperich, S., Farentinos, C., Hamilton, J., and Hausotter, W. (2006) Motivational Interviewing Assessment: Supervisory Tools for Enhancing Proficiency. Salem, OR: Northwest Frontier Addiction Technology Transfer Center, Oregon Health and Science University.

 

Arrow

 

 

(back to top)

 


 

Working Alliance Inventory (WAI)

 

The WAI (Horvath, 1986) is designed to measure working alliance between clients and therapists. The WAI has been found to be a strong predictor of therapeutic outcome across treatment modalities and is thought to tap into some general processes of treatment effect. The instrument has three subscales: task (agreement about what needs to be done to reach the client’s goals), goals (understanding of what the client hopes to gain), and bond (trust and confidence that the tasks will lead the client to his or her goals).

Versions of the instrument can be found here. (author permission is required for use).

 

Relevant Articles:

 

Horvath, A. O., Greenberg, L. (1986). The development of the Working Alliance Inventory: A research handbook. In L. Greenberg and W. Pinsoff (Eds.) Psychotherapeutic Processes: A Research Handbook, New York: Guilford Press.

 

Horvath, A. O., & Greenberg, L. S.(Eds.). (1994). The working alliance: Theory Research and Practice. New York: John Wiley & Sons.

 

Horvath, A. O.(1995). The therapeutic relationship: from transference to alliance. In Session: Psychotherapy in Practice, 1(1), 7-18.

 

Arrow

 

 

(back to top)