Vocational Rehabilitation: Better Measures and Monitoring Could
Improve the Performance of the VR Program (23-SEP-05,
GAO-05-865).
The Department of Education (Education) provides more than $2.5
billion annually to the states for a federal-state vocational
rehabilitation (VR) program to help individuals with disabilities
become employed. This program is among a large number of federal
programs intended to assist people with disabilities. In 2003 GAO
placed federal disability programs on its list of high-risk
programs because many of these programs have not kept up with
scientific advances and economic and social changes. GAO prepared
this report under the Comptroller General's authority as part of
an effort to assist policy makers in determining how federal
disability programs could more effectively meet the needs of
individuals with disabilities and addressed it to each committee
of jurisdiction. In this report, GAO assesses the (1) extent to
which state VR agencies assist individuals in achieving
employment, and (2) performance measures and monitoring practices
Education uses to manage this decentralized program and achieve
legislative goals.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-865
ACCNO: A37939
TITLE: Vocational Rehabilitation: Better Measures and Monitoring
Could Improve the Performance of the VR Program
DATE: 09/23/2005
SUBJECT: Aid for the disabled
Employment assistance programs
Employment of the disabled
Federal aid to states
Federal/state relations
Monitoring
Performance appraisal
Performance measures
Program evaluation
Program management
State-administered programs
Strategic planning
Vocational rehabilitation
Vocational Rehabilitation Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-865
United States Government Accountability Office
GAO
Report to Congressional Committees
September 2005
VOCATIONAL REHABILITATION
Better Measures and Monitoring Could Improve the Performance of the VR Program
GAO-05-865
[IMG]
September 2005
VOCATIONAL REHABILITATION
Better Measures and Monitoring Could Improve the Performance of the VR Program
What GAO Found
Of the more than 650,000 individuals exiting the state VR programs in
fiscal year 2003, one-third (217,557) obtained a new job or maintained
their existing job for at least 90 days after receiving services.
Education's data showed that the remaining two-thirds exited the VR
program without employment most often because the individual refused
services or failed to cooperate with the VR counselor (46 percent of the
time) or could not be located or contacted (24 percent). The VR program
purchased more than $1.3 billion in services for all individuals who
exited the program in fiscal year 2003, two-thirds of which were used to
provide services to individuals exiting with employment. Employment,
earnings, and the amount of purchased services received while in the VR
program varied significantly by individuals' disability type and other
characteristics. In addition, state VR agencies varied substantially in
the employment rates they achieved, the characteristics of individuals
they served, their frequency of providing certain services, and their
service expenditures.
Individuals Exiting the VR Program, Fiscal Year 2003
.
Without employment
With employment
0 1020304050607080
Percent
During the application phase After limited services After services under
an employment plan
Source: GAO analysis of Education data.
Education's performance measures are not comprehensive, and its monitoring
of state VR agencies has not resulted in timely feedback. Education does
not comprehensively measure the performance of certain key populations,
such as students transitioning from school to work, and tracks only the
individuals who exit the program, not those still receiving services. In
addition, Education's performance measures do not take into consideration
all the variation among the state VR agencies or allow for comparisons
with other workforce programs. Education's monitoring reports, which are
its primary means of providing feedback to state VR agencies, are
frequently late and based on data that are more than 2 years old.
Consequently, state VR agencies do not receive the timely feedback needed
to improve the efficiency and effectiveness of their programs. In managing
the performance of the VR program, Education also does not censure poorly
performing state VR agencies, reward strong performance, or take full
advantage of opportunities to disseminate best practices. Education
recently decided to eliminate its regional offices, which conducted most
of the monitoring of state VR agencies, making the details of the future
monitoring process unclear. United States Government Accountability Office
Contents
Letter
Results in BriefBackgroundOne-Third of Individuals Exited the VR Program
Nationwide with
Employment, although Employment Rates Varied Significantly among State VR
Agencies
Education's VR Performance Measures Are Not Comprehensive, and Its
Monitoring of State VR Agencies' Performance Has Not Resulted in Timely
Feedback
ConclusionsRecommendations for Executive ActionAgency Comments and Our
Evaluation
1
3 4
11
30 37 39 40
Appendix I Scope and Methodology
Appendix II State VR Agency Exit and Employment Rates, Fiscal Year 2003
Appendix III State VR Agency Population Proportions by Primary Impairment,
Fiscal Year 2003
Appendix IV Social Security Beneficiaries' Employment Rates by State VR
Agency, Fiscal Year 2003
Appendix V State VR Agency Percentages of Total Agency Service Budget
Spent on Each Service Category, Fiscal Year 2003 54
Appendix VI State VR Agency Total Administrative Costs as a Percentage of
Total Expenditures, Fiscal Year 2003 60
Appendix VII State VR Agency Average Total Expenditures in Fiscal Year
2003 Per Person Exiting with Employment in Fiscal Year 2003
Appendix VIII Comments from the Department of Education
Appendix IX GAO Contacts and Staff Acknowledgments
Tables
Table 1: Performance Indicators and Performance Targets for State VR
Agencies
Table 2: Time in Program, Number of Services, and Cost of Services by Type
of Exit from the VR Program, Fiscal Year 2003
Table 3: Percentage of Individuals in the VR Program Receiving Services by
Each of the 22 Service Categories Tracked by Education, Fiscal Year 2003
Table 4: Transitioning Students Exiting the VR Program with Employment
after Services under an Employment Plan, Fiscal Year 2003
Table 5: Per Capita Service Costs by Each Type of State VR Agency, Fiscal
Year 2003
Table 6: Blind or Visually Impaired Individuals Exiting with Employment
from Blind State VR Agencies versus Combined State VR Agencies, Fiscal
Year 2003
9
15
16
23 29
30
Figures
Figure 1: Individuals Exiting the VR Program, Fiscal Years 1997
2003 6 Figure 2: Individuals Exiting the VR Program, Fiscal Year 2003 12
Figure 3: Reasons for Individuals Exiting the VR Program without
Employment, Fiscal Year 2003 13 Figure 4: Primary Impairments of
Individuals Exiting the VR Program, Fiscal Year 2003 18
Figure 5: Rates of Exit from the VR Program by Primary
Impairment Group, Fiscal Year 2003 19 Figure 6: Median Hourly Wage by
Impairment Group among
Individuals Exiting the VR Program with Employment
who did Not Require Ongoing Support Services, Fiscal
Year 2003 20 Figure 7: Employment Status of Individuals Exiting the VR
Program in Fiscal Year 2003 Who Previously Participated
in the VR Program 21
Abbreviations
GPRA Government Performance and Results ActIEP individualized education
programOMB Office of Management and BudgetPART Program Assessment Rating
ToolRSA Rehabilitation Services Administration VR vocational
rehabilitationWIA Workforce Investment Act
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
September 23, 2005
The Honorable Michael B. Enzi
Chairman
The Honorable Edward M. Kennedy
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions United States Senate
The Honorable John A. Boehner Chairman The Honorable George Miller Ranking
Minority MemberCommittee on Education and the Workforce House of
Representatives
The Department of Education (Education) provides more than $2.5 billion
annually to the states for a federal-state vocational rehabilitation
(VR)program to help individuals with disabilities become employed,
consistentwith the Rehabilitation Act of 1973. In fiscal year 2003, these
state programs provided services to more than 1 million individuals
withdisabilities. Although Education provides more than three-quarters of
the program's funding, states have significant latitude in how they
administertheir VR programs.
In the past several years, key legislative changes have been enacted
relating to the VR program. For example, the Rehabilitation Act was
reauthorized as part of the Workforce Investment Act (WIA) in 1998, whenVR
became 1 of 17 mandated partners in state workforce investmentsystems. In
addition, the Ticket to Work and Work Incentives Improvement Act of 1999
expanded the types of rehabilitation servicesavailable to Social Security
disability beneficiaries by providing for privatized services in a market
previously dominated by the public VR program.
More recently, in 2003, we placed the modernization of federal disability
programs on our list of high-risk programs because many of these
programs have yet to incorporate scientific advances and economic and
social changes that have redefined the relationship between impairments
and the ability to work.1 In addition, these programs have faced long
standing challenges in ensuring the timeliness and consistency of
decisions related to benefits and services for people with disabilities.
We have prepared this report under the Comptroller General's authority as
part of a continued effort to help policy makers better understand the
extent of support provided by federal programs to people with disabilities
and to assist them in determining how these programs could more
effectively meet the needs of individuals with disabilities in the 21st
century. As it may prove helpful in the deliberation of committees with
jurisdiction over VR issues, we have addressed this report to each of
these committees. In this report, we assess (1) the extent to which state
VR agencies assist individuals in achieving employment and (2) the
performance measures and monitoring practices Education uses to manage
this decentralized program and achieve legislative goals.
To perform our review, we analyzed data from two datasets maintained by
Education, one a record of the cases for all individuals who exited the VR
program after their cases were closed during fiscal year 2003 (the most
recent year for which data were available for us to use in time for
production of this report) and the other a record of the expenditures for
each state VR agency in fiscal year 2003. We determined that the data we
used were sufficiently reliable for our purposes by performing electronic
testing for obvious errors in accuracy and completeness, reviewing
available documentation, and interviewing Education and state VR agency
officials knowledgeable about the data. We also reviewed relevant laws,
regulations, Education's policy documents relating to the VR program, and
the required state plans for each state VR agency's program. Further, we
interviewed key program officials at the national and regional levels,
selected state VR agency officials, and advocates for people with
disabilities. Finally, we conducted site visits to state VR agencies in
California, Maryland, Minnesota, New Mexico, Tennessee, and Virginia. We
selected these sites to achieve a mix of state VR agency structures,
operations, and performance as well as to achieve geographic diversity. We
conducted our review from August 2004 through September 2005 in accordance
with generally accepted government auditing standards. For a more complete
explanation of our methodology, see appendix I.
1GAO, High-Risk Series: An Update, GAO-03-119 (Washington, DC: January
2003).
Results in Brief
Of the more than 650,000 individuals exiting the state VR programs in
fiscal year 2003, one-third (217,557) obtained a new job or maintained
their existing job for at least 90 days after receiving customized
services. Education's data showed that the remaining two-thirds exited the
VR program without employment most often because the individual refused
services or failed to cooperate with the VR counselor (46 percent of the
time) or could not be located or contacted (24 percent). The VR program
purchased more than $1.3 billion in services for all individuals who
exited the program in fiscal year 2003, two-thirds of which was used to
provide services to individuals exiting with employment. Employment,
earnings, and the amount of purchased services received while in the VR
program varied significantly by individuals' type of disability and other
characteristics. In addition, state VR agencies varied substantially in
the employment rates they achieved, the characteristics of individuals
they served, their frequency of providing certain services, and their
service expenditures. For example, state VR agency employment rates ranged
from 20 to 74 percent in fiscal year 2003.
Education's performance measures are not comprehensive, and its monitoring
of state VR agencies has not resulted in timely feedback. Education does
not comprehensively measure the performance of certain key populations,
such as students transitioning from school to work, and tracks only the
individuals who exit the program, not those still receiving services. In
addition, the performance targets Education sets for state VR agencies do
not take into consideration all the demographic or economic variations
among states. Further, Education's performance measures do not allow for
comparison of the VR program with other workforce programs. Education's
monitoring reports, which are its primary means of providing feedback to
state VR agencies, are issued over 2 years after performance data have
been collected. Consequently, state VR agencies do not receive the timely
feedback needed to improve the efficiency and effectiveness of their
programs. In managing the performance of the VR program, Education also
does not censure poor performers, reward strong performers, or take full
advantage of opportunities to disseminate best practices to the state VR
agencies. As part of its larger reorganization, Education recently decided
to eliminate its regional offices, which conducted most of the monitoring
of state VR agencies, making the details of the future monitoring process
unclear.
We are making several recommendations to Education so that it can improve
its oversight of state VR agencies and help them most effectively target
their resources to achieve better employment rates. In this regard, we
recommend that Education reevaluate its performance measures to
Background
ensure better alignment with program goals and develop a better monitoring
process that includes timelier and more effective feedback to state VR
agencies. In commenting on the report, Education indicated that it is in
full agreement that better measures and monitoring could improve the
performance of the VR program. In addition, Education highlighted
initiatives either planned or under way to improve the management of the
VR program.
Title I of the Rehabilitation Act of 1973 authorizes a federal-state VR
program to provide services to persons with disabilities so that they may
prepare for and engage in gainful employment.2 Education provided an
estimated $2.6 billion in fiscal year 2005 in grants to the states and
territories based on a formula that considers the state's population and
per capita income. Grants to individual states ranged from about $9
million to nearly $250 million. Four of the five territories received less
than $3 million each. The act generally requires states to match federal
funds at a ratio of 78.7 percent federal to 21.3 percent state dollars.
Each state and territory designates a single VR agency to administer the
VR program, except where state law authorizes a separate agency to
administer VR services for individuals who are blind. Education provides a
single Title I grant to each state. States authorizing a separate blind VR
agency decide how the grant will be apportioned between the general VR
agency and the blind VR agency. Education tracks the performance of 80
state VR agencies-24 states have separate blind and general agencies; and
26 states, the District of Columbia, and five territories each have a
single combined agency. The 80 state VR agencies are housed in various
departments of state government, such as state departments of labor or
education, or they may be free-standing agencies or commissions.3 State VR
agencies also vary in their operations and locations. For example, some
agencies provide services through several offices located throughout the
state, while others provide services through one central location.
Education collects information about all individuals who exit each state
VR agency's program during a particular fiscal year, as reported by the
2This legislation was most recently reauthorized as part of the Workforce
Investment Act of 1998.
3In this report, the term state VR agencies refers to agencies in the 50
states, the District of Columbia, and the territories of American Samoa,
Guam, Northern Marianas Islands, Puerto Rico, and the Virgin Islands.
80 state VR agencies. The record for each individual exiting the program
includes information such as whether or not each individual became
employed, the weekly earnings and hours worked for individuals if they
exited the VR program with employment, the types and costs of services
they received, and demographic factors, such as impairment type, gender,
age, race and ethnicity, public benefits, and income from work at the time
of application. Education also collects summary information on agency
expenditures in a number of categories from each state VR agency.
Education tracks individuals in terms of seven types of case closures,
which can be collapsed into four categories, for individuals who
o exited without employment, during the application phase (including
individuals who were found ineligible; individuals who could not be
determined to be eligible or ineligible for various reasons such as they
could not be located or contacted, they failed to cooperate or they
refused services; and individuals who were found eligible, but were placed
on a waiting list);
o exited without employment, with limited services (including
individuals who were found eligible, but who left the program before an
employment plan could be developed, or agreed to an employment plan, but
left before receiving services under that plan);4
o exited without employment, after receiving services under an
employment plan; and
o exited with at least 90 days of employment, after receiving services
under an employment plan.
Education considers several types of work activities to meet its
definition of employment. First, Education counts as employment the paid
work activity of an individual, who may or may not require ongoing support
services, in an integrated work setting, that is, a setting typically
found in the community where individuals both with and without
disabilities
4State VR agencies are required to develop a written individualized plan
for employment for each eligible individual that includes the specific
employment goal, the rehabilitation services needed to achieve that goal,
the entities that will provide the services, and the methods available for
procuring the services. The plan must be agreed to and signed by the
eligible individual or by the individual's representative and approved by
the VR counselor.
interact.5 Second, Education counts self-employment as employment, whether
the business is managed by the eligible individual or the state VR agency.
Finally, Education considers certain types of unpaid work activity to be
employment, such as homemakers whose work activity is keeping house for
themselves or others in their households and unpaid workers in a family
business or family farm.
While the total number of individuals exiting the VR program has increased
slightly over the past several years, the number of individuals exiting
with employment has remained relatively stable. (See fig. 1.)
Figure 1: Individuals Exiting the VR Program, Fiscal Years 1997-2003
700,000
600,000
500,000
400,000
300,000
200,000
100,000
0 1997 1998 1999 2000 2001 2002 2003
Fiscal year
Total exits
Exits with employment Source: GAO analysis of Education data.
State VR agencies that determine they will not be able to serve all
eligible individuals who apply for services are required to state the
order in which they will select individuals for services. Agencies using
an order of selection process must develop criteria for ensuring that
individuals with
5In contrast, Education does not count as employment the work activity of
individuals who perform their work for a public or nonprofit organization
in a segregated or sheltered setting, that is, a setting in which the
eligible individuals primarily interact with other individuals with
disabilities. Education's term for this is "extended employment."
the most significant disabilities will be selected first for services.
Thirty
nine of the 80 state VR agencies were using an order of selection process
in fiscal year 2003. Beginning fiscal year 2004, 42 of the agencies are
using an order process.
The Rehabilitation Act requires state VR agencies to enter into
cooperative agreements with other entities that are part of the state's
workforce investment system. This workforce investment system includes a
One-Stop system, which is required to provide a number of
employment-related services to job seekers and employers at a single
location. The act also requires state VR agencies to coordinate with
public education officials to facilitate the transition of students with
disabilities from school to work. Students with disabilities receive
special education and related services from their school under an
individualized education program (IEP). Beyond these required
interactions, state VR agencies may also enter into third-party
cooperative agreements with other state or local agencies to coordinate
the services provided to their common program participants.
Education must abide by several statutes and executive branch directives
to measure and monitor the performance of the VR program. The 1998
amendments to the Rehabilitation Act required that Education establish and
publish evaluation standards and performance indicators for the VR
program. The standards and indicators were supposed to include outcome and
related measures of program performance that facilitate the accomplishment
of the purpose and policy of the act. The act also gave Education the
authority to reduce or suspend payments to state VR agencies that have
performance falling below a certain level and fail to enter into a program
improvement plan or substantially comply with the terms and conditions of
such a plan. The act also directed Education to conduct annual review and
periodic on-site monitoring of state VR agencies to determine, in part,
whether they were complying with the standards and indicators. Education
performs this monitoring function through the 10 regional offices of its
Rehabilitation Services Administration (RSA).
In response to the 1998 amendments, Education established new performance
measures in June 2000 that consisted of two standards for evaluating the
performance of the state VR agencies, one relating to the agencies'
performance in assisting individuals in obtaining, maintaining, or
regaining high-quality employment and the other relating to the agencies'
performance in ensuring that individuals from minority backgrounds have
equal access to services. In addition, Education published performance
indicators that establish what constitutes minimum compliance with these
evaluation standards and required performance targets for each indicator.
Six performance indicators were published for the employment standard, and
one was published for the minority service standard. State VR agencies
must meet or exceed performance targets in four of the six categories for
the first standard and meet or exceed the performance target for the
second standard in order to have passing performance. Table 1 provides
details on these standards and indicators.
Table 1: Performance Indicators and Performance Targets for State VR Agencies
Performance target
General or combined state VR agencies
Blind state VR agenciesa
Performance Indicator
1. Employment standard
Change in employment-the number of individuals exiting the Equal or exceed
previous Equal or exceed VR program with employment in the current
performance year performance year previous compared with the number
exiting with employment in the prior performance year performance year
Employment rate-percentage of individuals receiving services 55.8% 68.9%
under an employment plan who exit the VR program with
employment
Competitive employment rate-percentage of individuals exiting 72.6% 35.4%
the VR program with employment who were competitively
employedb
Significant disability rate-percentage of individuals exiting the 62.4%
89.0% VR program with competitive employment who have significant
c
disabilities
Wage Ratio-ratio of the average hourly earnings of individuals 0.52 0.59
exiting the VR program with competitive employment to the average hourly
earnings for all employed individuals in the state
Increase in self-support-the difference between the percentage 53.0 30.4
of individuals exiting the VR program with competitive
employment who report their own income as the largest single
source of economic support at the time they exit the VR program
and the percentage who report their own income as the largest
single source of economic support at the time they apply for VR
services
2. Equal access to services standard
The service rate for all individuals with disabilities from minority 0.80
0.80 backgrounds as a ratio to the service rate for all non-minority
d
individuals with disabilities
Source: GAO analysis of Education data.
aBlind state VR agencies must report each year the aggregated data for the
2 previous years for performance indicators 1.1 through 1.6.
bEducation defines competitive employment as work that is performed on a
full-time or part-time basis in an integrated setting for which the
individual is compensated at or above the minimum wage but not less than
the customary wage and level of benefits paid by the employer for the same
or similar work performed by individuals without disabilities. Education
also counts in this category individuals whose earnings from
self-employment are equivalent to at least the minimum wage.
cThe Rehabilitation Act defines a significant disability as one that
seriously limits one or more functional capacities and can be expected to
require multiple VR services over an extended period of time.
dEducation defines service rate as the result obtained by dividing the
number of individuals who exit the VR program after receiving one or more
services under an employment plan by the total number of individuals who
exit the VR program.
The Government Performance and Results Act of 1993 (GPRA) also requires
federal executive branch agencies such as the Department of Education to
set goals, measure their performance, and report on their accomplishments.
Agencies are required to develop annual performance plans that use
performance measurement to reinforce the connection between the long-term
strategic goals outlined in their strategic plans and the day-to-day
activities of their managers and staff. Among its performance goals for
fiscal year 2005, Education is assessing its performance in assisting
state VR agencies to achieve required performance targets on one
performance target-1.2.
In 2002, the Office of Management and Budget (OMB) directed that the
performance of a range of federal job training and employment programs be
measured consistently to allow for the comparison of results across these
programs. These common measures would be consistent with the common goals
of these programs, that is, to improve participants' employment and
earnings and focus on measures of outcomes and efficiency. OMB identified
the VR program as one of the federal programs that would be targeted for
using the common measures.
Using its Program Assessment Rating Tool (PART), OMB assessed the
effectiveness of the VR program in 2003 as part of its effort to hold
federal agencies accountable for accomplishing results. The PART
evaluation looks at four areas of assessment-program purpose and design,
strategic planning, management, and results and accountability. Programs
are rated in one of five categories: effective, moderately effective,
adequate, ineffective or results not demonstrated. OMB will use the rating
and relating findings to make decisions about budget and policy. OMB rated
the effectiveness of the VR program as adequate and made recommendations
to Education for improving program management and performance measures. As
part of its assessment, OMB reviewed Education's performance indicators
1.2, 1.3, and 1.5.
One-Third of Individuals Exited the VR Program Nationwide with Employment,
although Employment Rates Varied Significantly among State VR Agencies
More than 217,000 individuals with disabilities exited the state VR
programs with employment in fiscal year 2003 after receiving customized
services. This group represents one-third of the 650,543 individuals who
left the program nationwide in fiscal year 2003 after submitting an
application for services. The most common reasons that the remaining
two-thirds of the individuals left the program without a job were that the
individual refused services, failed to cooperate, or could not be located
or contacted. State VR agencies collectively purchased more than $1.3
billion in services for all individuals who exited the program in fiscal
year 2003, two-thirds of which was used to provide services to individuals
exiting with employment. Employment, earnings, and the amount of purchased
services received while in the VR program varied significantly by
individuals' type of disability and other characteristics. In addition,
the state VR agencies varied substantially in the employment rates they
achieved, the characteristics of individuals they served, their frequency
of providing certain services, and their service expenditures.
One-Third of Individuals Exited the VR Program Nationwide with Employment
in Fiscal Year 2003