Veterans Health Administration Mental Health Program Evaluation

215 Pages·2011·5.5 MB·English

Veterans Health
Administration Mental
Health Program Evaluation
Capstone Report
Katherine E. Watkins, Harold Alan Pincus
Sponsored by the U.S. Department of Veterans Affairs
HEALTH
The research described in this report was sponsored by the U.S. Department of Veterans
Affairs and was conducted by Altarum Institute and RAND Health, a division of the
RAND Corporation.
Altarum Institute integrates independent research and client-centered consulting to deliver
comprehensive, systems-based solutions that improve health and health care. A nonprofit,
Altarum serves clients in the public and private sector.
The RAND Corporation is a nonprofit institution that helps improve policy and
decisionmaking through research and analysis. RAND’s publications do not necessarily
reflect the opinions of its research clients and sponsors.
R® is a registered trademark.
© Copyright 2011 RAND Corporation
Permission is given to duplicate this document for personal use only, as long as it
is unaltered and complete. Copies may not be duplicated for commercial purposes.
Unauthorized posting of RAND documents to a non-RAND website is prohibited. RAND
documents are protected under copyright law. For information on reprint and linking
permissions, please visit the RAND permissions page (http://www.rand.org/publications/
permissions.html).
Published 2011 by the RAND Corporation

1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138

1200 South Hayes Street, Arlington, VA 22202-5050

4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213-2665

RAND URL: http://www.rand.org/

To order RAND documents or to obtain additional information, contact

Distribution Services: Telephone: [phone];

Fax: [phone]; Email:
Program Evaluation of VHA Mental Health Services
Capstone Report

Authors:
Katherine E. Watkins
Brad Smith
Susan M. Paddock
Thomas E. Mannle, Jr.
Abigail Woodroffe
Jacob Solomon
Melony Sorbero
Carrie Farmer
Kimberly Hepner
David M. Adamson
Lanna Forrest
Catherine Call
Harold Alan Pincus
Program Evaluation of VHA Mental Health Services: Capstone Report i
PREFACE

U.S. policymakers and medical professionals increasingly recognize that quality mental health care is a
key contributor to better, healthier lives for veterans with mental illness, and that setting measurable
goals, promoting evidence-based processes of care, and monitoring performance plays a vital role in
improving healthcare delivery and, ultimately, patient outcomes. In 2004, responding to the U.S.
government’s New Freedom Commission on Mental Health (2002), the U.S Department of Veterans
Affairs (VA) finalized a 5-year Mental Health Strategic Plan (MHSP), which emphasizes mental health as
an important part of veterans’ overall health. Major goals contained in the MHSP include raising
awareness of the importance of mental health, eliminating disparities in services, increasing early mental
health screening, assessments, and referrals to care, and providing high-quality, evidence-based care.
In 2006, to assess progress toward MHSP goals, the VA commissioned RAND and the Altarum Institute
to conduct an independent evaluation of the quality of the VA’s mental health care system. The request
was pursuant to the Government Performance and Results Act, which requires executive branch agencies
to evaluate their programs, and was in keeping with the VA’s commitment to ensuring to providing the
best care to U.S. veterans.
The RAND/Altarum team conducted the study between August 2006 and November 2010. The
evaluation represents the most comprehensive evaluation of a mental health care system ever undertaken.
The evaluation focused on the quality of care delivered to veterans with one or more of five mental health
or substance abuse diagnoses: (1) schizophrenia; (2) bipolar disorder; (3) posttraumatic stress disorder
(PTSD); (4) major depressive disorder; (5) substance use disorder (SUD).
The evaluation’s results should be of interest to policymakers in the areas of national defense and
veterans’ affairs, to mental health professionals, and to veterans and other audiences interested in
veterans’ health issues.
This research was conducted within the RAND/University of Pittsburgh Health Institute under
subcontract to the Altarum Institute. The research was sponsored by the United States Department of
Veterans Affairs.
A profile of RAND Health, abstracts of its publications, and ordering information can be found at
www.rand.org/health.

Program Evaluation of VHA Mental Health Services: Capstone Report ii
ACKNOWLEDGEMENTS
The authors would like to thank the following who contributed to the production of this report.
Our Steering Committee: Antonette Zeiss, Daniel Kivlahan, Robert Gresen, Kathy Henderson, Stanlie
Daniels, Lynnette Nilan, Gregory Hinrichsen, Ira Katz, and Anita Lord
Barbara Stephens, the Contracting Officer Technical Representative
Our RAND Corporation quality reviewers Katherine Hoggatt, Greer Sullivan, and Paul Shekelle
Katherine Gechter, Marcela Horvitz-Lennon, Robyn Gerdes, Leah Caldarone, and Teryn Maddox for
their work on the contract
James Torr and Lynn Polite for the work on editing, proofreading, and formatting the report
Program Evaluation of VHA Mental Health Services: Capstone Report iii
ABBREVIATIONS
AOD alcohol or other drug dependence
BMI body mass index
BP bipolar I disorder
CAPS Clinician-Administered PTSD Scales
CBOC community-based outpatient clinic
CBT cognitive behavioral therapy
CERT Center for Education and Research on Therapeutics, Rutgers University
CM contingency management, therapy
COD co-occurring [mental health and substance use] disorders
CPT Current Procedural Terminology
CWT compensated work therapy
DHHS Department of Health and Human Services
DISCH NDE Discharge National Data Extract
DoD Department of Defense
DSM Diagnostic and Statistical Manual of Mental Disorders
DSS Decision Support System, VHA
E&M evaluation and management
EBP evidence-based practice
ECHO Evidence of Care and Health Outcomes survey
ESS effective sample size
FY fiscal year
GAF Global Assessment of Functioning
GLMM generalized linear mixed model
HEDIS Healthcare Effectiveness Data and Information Set
HIV human immunodeficiency virus
ICD-9-CM International Classification of Diseases, 9th Revision, Clinical Modification
IOM Institute of Medicine
IT information technology
MCS Mental Component Summary, VR-12
MDD major depressive disorder
MH mental health
MHEI Mental Health Enhancement Initiatives
MHICM mental health intensive case management
MHSP Mental Health Strategic Plan
NCQA National Committee for Quality Assurance
NDC National Drug Code
NPCD National Patient Care Database
NQF National Quality Forum
NSDUH National Survey on Drug Use and Health
NTE new treatment episode
OEF/OIF Operation Enduring Freedom/Operation Iraqi Freedom
OPC outpatient care file
OUT NDE Outpatient National Data Extract
Program Evaluation of VHA Mental Health Services: Capstone Report iv
ABBREVIATIONS, CONTINUED
PABAK prevalence-adjusted bias-adjusted kappa statistic
PCS Physical Component Summary, VR-12
PFSA parent facility service area
PTF patient treatment file
PTSD posttraumatic stress disorder
RCT randomized controlled trial
RPT relapse prevention therapy
SAMHSA Substance Abuse and Mental Health Services Administration
SCHIZ schizophrenia
SD Standard deviation
SHEP Survey of Healthcare Experiences of Patients
SSRI selective serotonin reuptake inhibitor
SUD substance use disorder
USPHSTF United States Public Health System Task Force
VA Department of Veterans Affairs
VAMC Veterans Administration medical center
VHA Veterans Health Administration
VISN Veterans Integrated Service Network
VR-12 Veterans RAND 12 Item Health Survey
VR-36 Veterans RAND 36 Item Health Survey

Program Evaluation of VHA Mental Health Services: Capstone Report v
Table of Contents
Preface ............................................................................................................................................. ii
Acknowledgements ......................................................................................................................... iii
Abbreviations .................................................................................................................................. iv
Abbreviations, continued ................................................................................................................. v
Chapter 1. Introduction .................................................................................................................... 1
Background: Quality Improvement in Mental Health Care ................................................ 1
Assessing the VA’s Mental Health and Substance Use Care ............................................. 2
Goals of the Program Evaluation and Research Questions ................................................. 3
Overall Conceptual Framework for the Evaluation ............................................................ 4
Organization of the Report .................................................................................................. 7
References ........................................................................................................................... 7
Chapter 2. Methods .......................................................................................................................... 9
The Study Population .......................................................................................................... 9
Cohort Selection............................................................................................................. 9
Study Population for Veteran Survey and Medical Record Review ............................ 11
Evidence-Based Practices ................................................................................................. 12
Assessing VHA Performance at the VISN Level ............................................................. 13
Specific Steps in Assessing the Structure, Process, and Outcomes of VA Mental
Health Care ............................................................................................................. 15
Structure ....................................................................................................................... 15
Process ......................................................................................................................... 16
Outcomes ..................................................................................................................... 19
Where to Find More Information About the Evaluation Methodology ............................ 21
References ......................................................................................................................... 22
Chapter 3. Veterans in the Study Cohort: A Detailed Profile ........................................................ 24
A Demographic and Clinical Overview of the Study Cohort ........................................... 24
Residence ..................................................................................................................... 27
Clinical Complexity and Functioning .......................................................................... 29
Utilization and Costs ......................................................................................................... 30
Utilization and Costs by Utilization Category ............................................................. 34
Utilization and Costs by Cohort ................................................................................... 35
Summary ........................................................................................................................... 37
Program Evaluation of VHA Mental Health Services: Capstone Report vi
References ......................................................................................................................... 37
Chapter 4. The Structure and Capacity of VA Mental Health Care ............................................... 38
Policy Context of Facility Surveys ................................................................................... 38
Facility Survey Overview ................................................................................................. 39
Reported Availability and Accessibility of Mental Health Services ............................... 39
Acute Psychiatric Inpatient Beds ................................................................................. 40
Mental Health Crisis Management .............................................................................. 40
Psychotherapy and Pharmacotherapy .......................................................................... 41
PTSD and SUD Treatment Services ............................................................................ 42
Services for Women Veterans ..................................................................................... 43
Mental Health Consumer Councils and Mental Health Executive Committees .......... 44
Drive Times from Veterans’ Residences to Mental Health Services ........................... 45
Evidence-Based Practices ................................................................................................. 48
Availability of Evidence-Based Practices .................................................................... 48
Locations at Which EBPs Are Offered ........................................................................ 54
Extent of Alignment Between the Potential Target Population and the
Availability of EBPs ............................................................................................... 55
Program Characteristics of EBPs ................................................................................. 58
VA Mental Health Workforce Characteristics .................................................................. 60
Recruitment Difficulties ............................................................................................... 60
Staff Retention ............................................................................................................. 61
Psychiatrist Board Certification ................................................................................... 62
Limitations ........................................................................................................................ 62
Summary ........................................................................................................................... 63
References ......................................................................................................................... 64
Chapter 5. Documented Processes of Care .................................................................................... 65
Findings for Assessment and Diagnosis ........................................................................... 65
Initial Assessments ....................................................................................................... 68
Follow-Up Assessments ............................................................................................... 69
New Treatment Episode Performance Indicators ............................................................. 69
Evidence-Based Practices ................................................................................................. 72
Substance-Use Disorders .................................................................................................. 76
Chronic-Disease Management .......................................................................................... 80
Trends in VHA Performance Over Time .......................................................................... 80
Follow-Up After Psychiatric Hospitalization ................................................................... 83
Program Evaluation of VHA Mental Health Services: Capstone Report vii
Data Limitations ............................................................................................................... 86
Summary ........................................................................................................................... 86
References ......................................................................................................................... 87
Chapter 6. Variations in Care ......................................................................................................... 90
Data and Indicators ........................................................................................................... 91
Variation by Gender .......................................................................................................... 95
Variations in Characteristics ........................................................................................ 96
Variations in Performance Indicators .......................................................................... 97
Variation by Age ............................................................................................................. 100
Variations in Characteristics ...................................................................................... 101
Variations in Performance Indicators ........................................................................ 102
Variation by OEF/OIF Status.......................................................................................... 107
Variations in Characteristics and Utilization ............................................................. 107
Variations in Performance Indicators ........................................................................ 108
Variation by Rural/Urban Status ..................................................................................... 111
Variations in Veteran Characteristics ........................................................................ 112
Variations in Performance Indicators ........................................................................ 113
Variation by Diagnostic Cohort ...................................................................................... 116
Variations in the Study Cohort .................................................................................. 116
Variations in Performance Indicators ........................................................................ 117
Variation by VISN .......................................................................................................... 119
Variations in the Study Cohort .................................................................................. 119
Variations in Performance Indicators ........................................................................ 120
Limitations ...................................................................................................................... 125
Summary ......................................................................................................................... 125
References ....................................................................................................................... 126
Chapter 7. Veterans’ Perceptions and Outcomes of Care ............................................................ 129
Timeliness ....................................................................................................................... 129
Patient-Centeredness ....................................................................................................... 130
Reasons Cited for Using VHA Services ......................................................................... 131
Patient Satisfaction.......................................................................................................... 132
Veterans’ Perceptions of Their Own Improvement and the Helpfulness of Care ........... 134
Context: The ECHO Field Test Report ........................................................................... 135
Limitations ...................................................................................................................... 137
Summary ......................................................................................................................... 137
Program Evaluation of VHA Mental Health Services: Capstone Report vii i