Defense Management: Actions Are Needed to Improve the Management
and Oversight of the National Guard Youth Challenge Program
(29-NOV-05, GAO-06-140).
The fiscal year 1993 National Defense Authorization Act
established the National Guard Youth Challenge Program as a pilot
program to evaluate the effectiveness of providing military based
training to improve the life skills of high school dropouts. The
Assistant Secretary of Defense for Reserve Affairs, under the
authority of the Under Secretary of Defense for Personnel and
Readiness, is responsible for overall policy for the program. The
National Guard Bureau (NGB) provides direct management and
oversight. In 1998, Congress permanently authorized the program
and began decreasing the federal cost share until it reached its
current level of 60 percent in 2001. Conference Report 108-767
directed GAO to review the program. Specifically, GAO reviewed
(1) historical trends of the program; (2) the extent of analyses
performed to determine program costs and the need to adjust the
federal and state cost share; and (3) NGB oversight of the
program. GAO is also providing information on Reserve Affairs'
and states' efforts to obtain funding from alternative sources.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-140
ACCNO: A42177
TITLE: Defense Management: Actions Are Needed to Improve the
Management and Oversight of the National Guard Youth Challenge
Program
DATE: 11/29/2005
SUBJECT: Cost analysis
Education or training costs
Federal aid to states
Federal/state relations
National Guard
Performance measures
Program evaluation
Program management
School dropouts
Students
Program costs
Pilot programs
National Guard Youth Challenge Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-140
Defense Management Defense Management Defense Management ement Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Defense Management Defense Management Defense Management Defense
Management Defense Management Defense Management Defense Management
Contents
Letter 1
Results In Brief 3
Background 6
Challenge Program Expenditures and Participation Have Increased and NGB
Has Reported Positive Performance Outcomes 9
Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program
Costs or the Need to Adjust the Federal and State Cost Share 18
The National Guard Bureau Lacks a Complete Framework for Oversight of the
Challenge Program, Making It Difficult to Measure Program Effectiveness 21
Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding
Support for the Challenge Program, While Some Participating States Have
Obtained Alternative Funding Support as a Result of Their Efforts 25
Conclusions 29
Recommendations for Executive Action 29
Agency Comments and Our Evaluation 30
Appendix I Scope and Methodology 33
Appendix II National Guard Youth Challenge Program Sites 36
Appendix III Comments from the Department of Defense 37
Appendix IV GAO Contact and Staff Acknowledgment 40
Tables
Table 1: Hypothetical Examples Showing How Changing the Cost Basis or
Cost-Share Ratio Would Affect Yearly Challenge Program Funding Amounts 19
Table 2: States Selected for Site Visits or Interviews 34
Figures
Figure 1: Annual Expenditures for the National Guard Youth Challenge
Program 11
Figure 2: Number of National Guard Youth Challenge Program Sites by
Reporting Year 12
Figure 3: Youth Participation in National Guard Youth Challenge Program 13
Figure 4: Challenge Program Sites by 2004 Graduation Results 14
Figure 5: Number of GEDs Earned by Challenge Students 16
Figure 6: Post-Residential Placement Data, 2003-2005 17
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Abbreviations
DMARS Data Management and Reporting System DOD Department of Defense DOJ
Department of Justice DOL Department of Labor ED Department of Education
GED general educational development NGB National Guard Bureau
United States Government Accountability Office
Washington, DC 20548
November 29, 2005 November 29, 2005
The Honorable John Warner Chairman The Honorable Carl Levin Ranking
Minority Member Committee on Armed Services United States Senate The
Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority
Member Committee on Armed Services United States Senate
The Honorable Duncan L. Hunter Chairman The Honorable Ike Skelton Ranking
Minority Member Committee on Armed Services House of Representatives The
Honorable Duncan L. Hunter Chairman The Honorable Ike Skelton Ranking
Minority Member Committee on Armed Services House of Representatives
After Operation Desert Storm and the end of the Cold War, Congress began
to reevaluate the focus of U.S. military activities and proposed using
some military assets and training to help address critical domestic
challenges such as drugs, poverty, and unemployment. In particular, some
policymakers saw an opportunity to use the military's capabilities to
address domestic needs by providing role models for youth and training and
educational opportunities for the disadvantaged. Section 1091 of the
National Defense Authorization Act for Fiscal Year 1993 established the
National Guard Youth Challenge Program (Challenge Program) as a pilot
program to evaluate the feasibility and effectiveness of military based
training to improve the life skills and employability of high school
dropouts. After the program was permanently authorized in 1998, Congress
began limiting the share of state operating costs covered by the federal
government through the Department of Defense (DOD), which initially paid
for 100 percent of these costs. DOD now pays 60 percent of state operating
costs, and states provide 40 percent. In addition, the authorizing
legislation for the Challenge Program allows the Secretary of Defense to
use nondefense funding sources in support of the program. After Operation
Desert Storm and the end of the Cold War, Congress began to reevaluate the
focus of U.S. military activities and proposed using some military assets
and training to help address critical domestic challenges such as drugs,
poverty, and unemployment. In particular, some policymakers saw an
opportunity to use the military's capabilities to address domestic needs
by providing role models for youth and training and educational
opportunities for the disadvantaged. Section 1091 of the National Defense
Authorization Act for Fiscal Year 1993 established the National Guard
Youth Challenge Program (Challenge Program) as a pilot program to evaluate
the feasibility and effectiveness of military based training to improve
the life skills and employability of high school dropouts. After the
program was permanently authorized in 1998, Congress began limiting the
share of state operating costs covered by the federal government through
the Department of Defense (DOD), which initially paid for 100 percent of
these costs. DOD now pays 60 percent of state operating costs, and states
provide 40 percent. In addition, the authorizing legislation for the
Challenge Program allows the Secretary of Defense to use nondefense
funding sources in support of the program.
The Assistant Secretary of Defense for Reserve Affairs, under the
authority of the Under Secretary of Defense for Personnel and Readiness,
is responsible for management oversight of the Challenge Program,
including policy and funding issues, while the National Guard Bureau (NGB)
is responsible for the administration of the program, including daily
management and oversight. For example, NGB maintains frequent contacts
with state programs and conducts program evaluations through a The
Assistant Secretary of Defense for Reserve Affairs, under the authority of
the Under Secretary of Defense for Personnel and Readiness, is responsible
for management oversight of the Challenge Program, including policy and
funding issues, while the National Guard Bureau (NGB) is responsible for
the administration of the program, including daily management and
oversight. For example, NGB maintains frequent contacts with state
programs and conducts program evaluations through a contractor, while
Reserve Affairs prepares the program budget based on input from NGB and
ensures that states are able to provide their required share. To
participate in the program, states must sign a cooperative agreement with
NGB that lays out roles and responsibilities among the states and NGB, and
also provides guidance on the content of the program. Since 1993, NGB has
used a cost of $14,000 per student as a basis for providing funds to state
programs.
Conference Report 108-767 directed the Comptroller General to conduct a
study of the Challenge Program.1 Specifically, we examined: (1) historical
trends of the Challenge Program, including program expenditures,
participation, and performance; (2) the extent to which Reserve Affairs
and NGB have determined actual program costs and the consequent need to
adjust the federal and state cost share; and (3) the extent to which NGB
has provided oversight of the program. We also determined the extent to
which Reserve Affairs and participating states have made an effort to
obtain alternative funding support for the Challenge Program.
In performing our work, we interviewed officials from the Office of the
Assistant Secretary of Defense for Reserve Affairs; NGB; the Departments
of Education, Labor, and Justice; the contractor that monitors and
evaluates state programs; the National Guard Youth Foundation; and
selected states. We also reviewed the program's annual reports and other
documents provided by Reserve Affairs and NGB as well as the data
management system used by states to collect Challenge Program student
information and track individual and program activities. We concluded that
the data were sufficiently reliable for our purposes. We analyzed program
evaluations, resource management reviews, and audits, in addition to
reviewing existing policies and procedures for managing the program. We
conducted a survey of all 29 established programs to collect information
on their budgets and expenditures, federal and state funding levels, each
state's ability to fund their individual program, and the availability of
funding from other sources. We also visited or contacted eight Challenge
Program sites in seven states and asked them about any alternative sources
of funding for their programs. A detailed description of our scope and
methodology is presented in appendix I. We conducted our work from January
2005 to October 2005 in accordance with generally accepted government
auditing standards.
1Conference Report 108-767 accompanied the Ronald W. Reagan National
Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375
(2004).
Results In Brief
According to NGB, program expenditures and state participation in the
Challenge Program have increased since the program began, and the program
has achieved positive performance outcomes. Between fiscal years 1998 and
2004, total expenditures for the Challenge Program, including funds spent
to cover the federal and state cost shares and federal management
expenses, have increased from about $63 million to about $107 million. For
fiscal year 2004, for example, NGB spent approximately $61.6 million for
the federal share of the program and $5.8 million for NGB management
costs, while states contributed approximately $40.5 million. Ten states
participated in the first year of the pilot program in 1993; and today
there are 29 Challenge Program sites in 24 states and Puerto Rico.
Additionally, 9 states remain on a waiting list to start new programs, and
several states have expressed an interest in expanding their existing
programs. Some common performance outcomes that NGB reports include the
number of graduates earning a general educational development credential,
changes in program students' scores on standardized math and reading
tests, and the number of community service hours performed by Challenge
Program students. For example, the Challenge Program reported that 70
percent of graduates in 2004 earned a general education development
credential. Another performance outcome that NGB reports is the number of
placements of program graduates in continuing education, the military, or
the labor force at the time they complete their 12-month post-residential
phase. For example, in 2004, 3,698 graduates were placed in one of these
categories by the end of the post-residential phase of the program.
Although Reserve Affairs and NGB have expressed concern about the current
program funding level and have suggested increasing both the cost basis
used to determine funding needs and the federal cost share, we found that
neither Reserve Affairs nor NGB has performed analyses to support the need
for such changes. Good budget practices, included in the Office of
Management and Budget's Federal Financial Accounting Standards, state that
agencies should determine actual costs of their activities on a regular
basis and that reliable cost information is crucial for effective
management of government operations. Since 1993, NGB has used a cost of
$14,000 per student as the basis for determining the amount of funds
needed to cover program operating costs. However, due to cost variations
between states, program officials that we surveyed reported that they
actually spent between $9,300 and $31,031 per graduate. According to NGB,
all 29 programs are providing the services required by the cooperative
agreements, and several states have added program enhancements such as
field trips or vocational classes. However, some of these states reported
that they reduced nonrequired services to stay within their budgets. For
example, they implemented staff pay and hiring freezes, eliminated the
student stipend, and eliminated some enrichment activities such as field
trips and vocational classes. In addition, Reserve Affairs, NGB, and
participating states have suggested that the cost-share ratio be changed
from its current 60 percent federal share to a 75 percent federal share
because they believe that the current required 40 percent state share is
sometimes difficult for states to meet. However, neither Reserve Affairs
nor NGB has compiled or analyzed data on actual program costs, states'
financial situations, or the impact of adjusting the federal and state
cost share. Reserve Affairs has asked the National Guard Bureau to
determine a new funding formula for the program based on individual state
needs, but, at the time of our review, NGB had not yet done so and Reserve
Affairs has not given NGB a deadline for completion. Without better cost
and financial information, DOD cannot justify future funding requests or a
change in the cost share ratio.
Although NGB has several mechanisms in place for overseeing the Challenge
Program, it lacks a complete oversight framework, making it difficult to
measure the effectiveness of the program and to adequately address audit
and review findings. NGB conducts several oversight activities, including
informally communicating with state program directors and having an
outside evaluator conduct yearly operational evaluations and biennial
resource management reviews of the Challenge Program, and these reports
have prompted some changes to the program. However, until recently, NGB
did not have a formal mechanism for tracking the findings of these
reports. In response to our review, NGB recently implemented a mechanism
for tracking these findings. As part of its oversight activities, NGB does
require each state program to report certain performance outcome measures,
such as academic test scores. However, it does not require states to
establish performance goals in these areas. A complete oversight framework
includes performance goals and measures against which to objectively
measure performance as well as a mechanism for tracking findings of audits
or reviews and responding to those findings. Without clear and agreed upon
performance goals, there is no objective yardstick against which to fully
measure program performance and effectiveness and thereby assess DOD's
return on investment. In addition, Standards for Internal Control in the
Federal Government states that agencies need to ensure that the findings
of audits and reviews are promptly resolved. Under the cooperative
agreements governing the Challenge Program, United States property and
fiscal officers in each state are required to conduct full audits of the
individual Challenge Programs at least once every three years. However,
according to property and fiscal officers that we spoke with, all of these
audits have not been conducted as required, due to the low priority placed
on these audits and lack of staff for the property and fiscal officer. If
these audits are not conducted, it may be difficult to ensure that federal
interests are adequately protected. Copies of these audits are not
currently being provided to program managers at NGB because, according to
the Chief of Property and Fiscal Affairs at NGB, there is no specific
requirement to do so. If the National Guard does not see the audits, it
cannot ensure that the findings of these audits are promptly resolved.
The authorizing legislation for the Challenge Program allows the Secretary
of Defense to use nondefense funding sources in support of the program,
and allows states to supplement program funds from other resources.
However, Reserve Affairs has not adopted a formal strategy for pursuing
nondefense funding, while some states have been successful in securing
alternative funding support for their programs. Reserve Affairs has
primarily adopted informal strategies to contact agencies outside DOD to
inform them about the Challenge Program and seek opportunities for
partnerships, but has not been successful in securing funds from these
agencies. We found that because Reserve Affairs has not made a formal
business case to request funds from these nondefense agencies, these
agencies are unable to determine whether or not they are specifically able
to fund the Challenge Program. Officials from some federal agencies told
us that that their agencies have general authority to provide funds to
other programs if those programs are consistent with their agency's
interests. However, officials at the Department of Justice, the Department
of Labor, and the Department of Education stated that Reserve Affairs
needed to present more specific information to them before they could
determine whether funds could be provided to the Challenge Program.
According to those officials, at a minimum, any such request should
contain the amount of funding sought and a sufficiently detailed
description of the program to allow the agency receiving the request to
determine whether it would be an appropriate use of funds. Until Reserve
Affairs makes a more formal request for funding, other agencies will be
unable to determine if they can provide funds for the Challenge Program.
At the state level, some states we contacted had made efforts to obtain
alternative funding support to enhance or maintain their program's
operations. For example, some state programs organized into charter
schools to obtain funding from local school districts, applied for
National School Lunch Program funding, obtained grants from nonprofit
organizations, and received additional funding from the state government
beyond the required match. Some states reported obstacles in securing
supplemental funding, such as the prohibition against using Challenge
funds to hire grant writers.
We are making recommendations designed to improve DOD's management and
oversight of the Challenge Program and to strengthen efforts in obtaining
alternative funding support for the program. In commenting on a draft of
this report, DOD concurred with our recommendations. DOD's comments and
our evaluation of them appear later in this report.
Background
The National Guard Youth Challenge Program is a 17-month program that
serves at-risk youth at 29 sites in 24 states and Puerto Rico. The purpose
of the program is to improve the education, life skills, and employment
potential of students by providing military-based training, supervised
work experience, and knowledge in eight core program components.2 Students
must be 16 to 18 years old, drug-free, unemployed, high school dropouts,
and not in trouble with the law. NGB reports that more than 59,000
students have been graduated from the Challenge Program since it began as
a pilot in 1993. The program was authorized by 32 U.S.C. S:509 on a
permanent basis in fiscal year 1998, at which time states were to begin
paying a share of operating costs.
Challenge Program Structure
Each Challenge Program site operates two residential classes per year, one
of which begins in January and the other around July.3 A typical
graduation goal is 100 students per class, or 200 per year, although
several programs graduate more students. In 2004, for example, Illinois
graduated almost 800 students in 2004, and Louisiana's three sites
combined graduated more than 950 students. The residential phase of the
program runs 22 weeks and includes a 2-week Pre-Challenge phase. During
Pre-Challenge, applicants are assessed for their ability and motivation to
complete the remaining 20 weeks of the residential program. Those who
successfully complete Pre-Challenge are then formally enrolled in the
Challenge Program in numbers that equal each program's graduation target
plus normal program attrition rate. In the residential phase, students
receive military-based training and supervised work experience.
Additionally, each state develops a curriculum that incorporates the eight
core components and the tasks, conditions, and standards that students
must complete to demonstrate progress in those components.4 Each student
must receive a score of at least 80 percent on each core component to be
graduated from the program.
2 The eight core program components are: leadership/followership, job
skills, responsible citizenship, health and hygiene, service to the
community, academic excellence, life-coping skills, and physical fitness.
3 NGB reports on site activities by NGB class numbers. The first class
under the pilot program, NGB-1, began in July 1993, and NGB-25 started in
July 2005.
During the 12-month post-residential phase, individuals who have
successfully completed the residential phase are involved in placement
activities, which include employment, education, volunteer activities, or
any combination of the three or military service. The graduates work with
adult mentors who were matched with them during the residential phase.
These mentors provide guidance and support to the graduates and are
required to contact the youths twice each month at a minimum. Program
staff use the written, post-residential action plan that each student
prepares and updates during the residential phase to monitor placement
activities. Mentors also use this plan during their interactions with
graduates. The Challenge Program reports youth placement activities at the
end of the 12-month follow-up period. To further assess the long-term
impact of the program, NGB has contracted with AOC Solutions to conduct a
retrospective longitudinal study of program graduates as well as students
who did not complete the program.
Program Funding
Prior to 1998, the federal government, through DOD, completely funded the
Challenge Program. In fiscal year 1998, Congress began requiring states to
provide a minimum of 25 percent of their programs' operating costs. The
state cost share increased 5 percent each year until fiscal year 2001,
when it reached the current funding requirement of 40 percent.5 Although
some states had provided more funds than required in the past, program
funding each year is now determined by the 40 percent share, which is
based on $14,000 for each youth targeted for graduation. In addition to
the federal and state funds used to operate the program sites, DOD also
provides funds for NGB management expenses such as program evaluations,
contractor-provided training, and travel for training and workshops. These
NGB program management costs are not subject to the federal/state 60/40
cost share requirement.
4A task is a clearly defined and measurable activity accomplished by an
individual. Condition describes the training situation, environment, or
field conditions under which the cadet must perform the task. The standard
establishes criteria for how well a task must be performed.
532 U.S.C. S:509(d)(4).
Each state submits a budget to NGB that is based on that state's target
for number of graduates. Since the program's inception, the funding
provided by NGB has been based on a cost per student of $14,000.6 For
example, if a state has a target of 100 students per class (200 per year)
to graduate, the estimated program costs would be $2.8 million. The
federal contribution, or 60 percent of the total, would be $1.68 million,
while the state contribution would be $1.12 million.
To receive federal funding, a state must certify that it has sufficient
funds to provide its 40 percent share. State funds can be composed of
cash, noncash supplies, services, or a combination of these sources.
States are allowed to provide additional funding (over and above the 40
percent share) to the program from sources such as individual and
corporate donations, additional moneys from the state general fund or
other state revenue sources, or other federal funding. Some Challenge
Program sites, for example, operate as alternative schools and are
reimbursed by their state education agencies for portions of their program
costs.
Management, Oversight, and Support
Reserve Affairs, under the authority of the Under Secretary of Defense for
Personnel and Readiness, is responsible for preparing the annual budget
and reviewing state budgets and funding certifications. Reserve Affairs is
to monitor program compliance with DOD policy, issue supplemental policy
guidance, and submit the Challenge Program annual report to Congress. NGB
provides day-to-day administration and oversight of the Challenge Program,
issuing regulations, and submitting budgets and annual report drafts to
Reserve Affairs. NGB has contracted with AOC Solutions to assist with the
oversight of the Challenge Program. AOC Solutions performs the annual
operational evaluations and the biennial resource management reviews. This
contractor also pulls together the program information for the annual
report and maintains and oversees the Data Management and Reporting System
(DMARS), which is used to collect student data and report on individual
and program activities. NGB has also contracted with Dare Mighty Things to
provide training and technical assistance to Challenge Program staff.7
Finally, NGB has United States property and fiscal officers in each state
who are responsible for receiving and accounting for all federal Challenge
Program funds and property under control of the National Guard in that
state. The property and fiscal officers are also responsible for ensuring
that federal funds are properly obligated and expended.
6Funding allocations to the states have historically been based on target
enrollments; that is, the maximum number of students per class that are
funded annually in a program's budget. Beginning with fiscal year 2004,
funding allocations are now based on target graduation, which is the
maximum number of students identified in a program's budget for
participation in and graduation from the program. According to NGB
officials, although allocations were originally based on a site's target
enrollment, this figure was treated as a graduation goal. These numbers
differ by state.
NGB enters into cooperative agreements with governors of states approved
to participate in the Challenge Program. The cooperative agreements
describe the responsibilities of the states and NGB as well as the
funding, costs, and regulations for operating National Guard Youth
Programs.8 The cooperative agreements also define the eight core
components and provide guidance on how to run the residential and
post-residential phases and other aspects of the Challenge Program. Each
Challenge Program state is also required to submit state plans and budget
estimates for their state. These state plans must include details on the
state's procedures and be consistent with overall program guidance
provided by DOD. For example, state plans include information on
application and selection procedures, staffing and staff training, and a
detailed budget.
Challenge Program Expenditures and Participation Have Increased and NGB Has
Reported Positive Performance Outcomes
According to NGB, Challenge Program expenditures and state participation
have increased since the program began, and the program has achieved
positive program performance outcomes over time. Since the program's
inception, total expenditures have increased from about $63 million to
about $107 million per year. The number of states participating in the
Challenge Program has also increased, and several states have expressed
interest in adding a program or expanding existing ones. Challenge sites
must account for their activities throughout the year, and NGB has
reported positive performance outcomes over time.
7Contracts with AOC Solutions and Dare Mighty Things were competitively
awarded for a 1-year base period with four 1-year options.
8The Master Youth Programs Cooperative Agreement covers two youth
programs: the National Guard Youth Challenge Program and the National
Guard STARBASE Program. The STARBASE Program was established to improve
the knowledge and skills of at-risk youth in math, science and technology.
Program Expenditures Have Increased Over Time
NGB reports that overall federal expenditures for the Challenge Program
have increased over time, but states have also increased their
expenditures since the program was permanently authorized in fiscal year
1998. Between fiscal years 1998 and 2004, total expenditures for the
Challenge Program, including funds spent to cover the federal and state
cost shares and federal management expenses, have increased from about $63
million to about $107 million. For fiscal year 2004, for example, NGB
expenses included $61.6 million for the federal cost share and $5.8
million for NGB management costs, while states contributed approximately
$40.5 million.9 In addition, in 2000 and 2001, the Challenge Program
received $5 million and $7,483,500 respectively from the Department of
Justice. Reserve Affairs stated that the primary use of these funds has
been to start new Challenge Program sites. Since 2001, four programs were
established using these funds and three programs' operations were
maintained in 2002. In total, approximately $5.97 million remain unspent
in a nonexpiring account. Officials at Reserve Affairs and NGB told us
that these funds remain unspent because no new Challenge Programs have
started. According to these officials, new programs have not been
established because state governments have not committed the required 40
percent match. (See fig. 1 for total program expenditures from fiscal year
1998 to 2004, broken down by federal and state cost share and NGB
management expenses.)
9At the time of our review, final expenditure reports for fiscal year 2005
were unavailable.
Figure 1: Annual Expenditures for the National Guard Youth Challenge
Program
Note: Federal cost shares for fiscal years 2001 and 2002 include $6.5
million from Department of Justice. State cost share amounts are the
required state matches only and do not include any additional funding or
in-kind support the states might have provided.
The Number of States Participating in the Challenge Program Has Increased
When the pilot program began in 1993, there were 10 Challenge Program
sites in 10 states. The program has now grown to 29 sites in 24 states and
Puerto Rico.10 In fiscal year 2005, Wyoming received funds to start up a
program site. According to NGB, Wyoming will begin its first class in
January 2006. In addition to those states currently operating Challenge
Program sites, there are also nine states that have expressed interest in
establishing new programs.11 For example, according to NGB officials,
representatives from Washington and Indiana National Guard units have
visited some existing program sites and are in the process of developing
state programs. Other states are interested in expanding their programs to
serve more youth at existing sites or to open new locations. On the other
hand, for various reasons including difficulty meeting the state match
requirement, lack of state support, and substandard facilities, four
states have discontinued their Challenge Programs. Connecticut, a pilot
program state, dropped its program in 1994 after two classes. Colorado
discontinued its program after 1999, Missouri after 2002, and New York
after 2003. Figure 2 describes the number of Challenge Program sites for
each year since the program began. Appendix II identifies the individual
states with Challenge Program sites.
10Louisiana has three Challenge Program sites, and Georgia and South
Carolina each operate two sites.
Figure 2: Number of National Guard Youth Challenge Program Sites by
Reporting Year
Note: Reporting year is the time frame covered in annual reports. It
includes the July class from the previous year and the January class from
the current year.
11According to NGB, as of August 2005, the District of Columbia and eight
states had expressed interest in starting a Challenge Program: Alabama,
Colorado, Connecticut, Delaware, Indiana, Nevada, Pennsylvania, and
Washington.
Student participation in and graduation from the Challenge Program have
also increased over time. States are required to track the number of youth
who have applied to the Challenge Program, enrolled in the third week of
the program (after the 2-week Pre-Challenge phase), and were graduated
from the residential phase. According to NGB, the target graduation rate
for 2004 was 6,961; the actual number of enrollees was 8,920; and 7,003
students were graduated, or 79 percent of those enrolled, from the
Challenge Program. Figure 3 shows the target numbers, the actual number of
students who were enrolled in the residential phase at week 3, and the
number that graduated from the program from 2000 through 2004.
Figure 3: Youth Participation in National Guard Youth Challenge Program
Note: Reporting year is the time frame covered in annual reports. It
includes the July class from the previous year and the January class from
the current year.
Although all Challenge Programs graduate two classes per year, the number
of graduates per class varies. In addition, some states have multiple
programs. For example, Louisiana has three Challenge Programs and, in
2004, graduated a total of 952 students. Figure 4 identifies those states
currently participating in the Challenge Program by the number of
graduates they reported for 2004.
Figure 4: Challenge Program Sites by 2004 Graduation Results
Note: The 2004 reporting year covers NGB classes 21 and 22, which began in
July 2003 and January 2004.
NGB Has Reported Positive Performance Outcomes
NGB has reported positive performance outcomes in academic performance,
community service activities, and post-residential placements. Program
performance information is tracked by each Challenge site and submitted to
NGB. Each year, the Challenge Program reports on outcomes for the two
classes completing the 22-week residential phase during that reporting
year and for the two preceding classes as they complete their 1-year
post-residential follow-up phase. The Challenge Program sites use the same
automated system, DMARS, to collect information on students and report on
their progress and activities. The information collected in DMARS is
reviewed by the contractor through weekly and monthly reports and during
random checks of source documents during operational evaluation site
visits.
Some residential phase outcomes of the Challenge Program, such as the
number of graduates earning a general educational development (GED)
credential or high school degree and changes in scores on standardized
math and reading tests, are tied to the core component of Academic
Excellence.12 For example, NGB reported that 70 percent of graduates in
2004 earned a GED. Figure 5 illustrates the outcomes of GED attainment for
the past 5 reporting years. Students also take the Tests of Adult Basic
Education, a series of tests that identify individual education levels in
various academic subject areas. Each state program tests its students
early in the residential phase and then toward the end of the 22-week
period, and it reports the changes in test scores. In 2004, for example,
NGB reported that graduating students improved 1.7 grade levels in reading
and 1.8 grade levels in math during the residential phase. Another core
component, Service to the Community, requires each student to perform a
minimum of 40 hours of service to the community or conservation project
activities. The number of community service hours performed by each
student is tracked, and the total number of hours for each site is another
outcome that the Challenge Program reports annually. For example, in 2004,
NGB reported that Challenge Program students performed more than 590,000
hours of community service, such as maintaining historical cemeteries and
parks and supporting organizations such as Special Olympics and Habitat
for Humanity.
12In some states, younger youths are not eligible to take the GED due to
age restrictions, and some youths are not academically prepared to take
the tests. Some programs offer alternative degrees, such as adult
education diplomas; some programs are accredited high schools and can
offer high school diplomas or credits; and many youths return to their
home high schools to earn their diplomas.
Figure 5: Number of GEDs Earned by Challenge Students
Each month of the post-residential phase, each Challenge Program graduate,
or that individual's mentor, reports on the graduate's post-residential
activities. Following the 12-month post-residential phase, each Challenge
Program site reports graduate placements in continuing education, the
military, or the labor force. These placements are verified with schools,
the military, and employers and are documented. Program representatives
are not always able to contact all graduates for placement information and
therefore placement data reflect only the students contacted, not all
graduates. Education placements include returning to high school or going
to a post-secondary or vocational-training institution, which students may
be attending full- or part-time. Some Challenge Program graduates also
enter the military, into either the active or the reserve forces.
Post-residential employment placements can be full- or part-time, and they
include those graduates who are self-employed. Graduates can have
placements in more than one of these categories. For example, an
individual might be attending school and working part-time. Challenge
Program sites continue to update their placement records after the
12-month follow-up period when they come in contact with former students.
During the longitudinal study, for example, the contractor has been able
to update placement data based on information received from state program
officials and graduates. Figure 6 shows post-residential placement trends
for the past 3 reporting years. The total numbers of graduates placed in
these 3 years are 2,407 in 2003; 3,698 in 2004; and 4,086 in 2005.
Figure 6: Post-Residential Placement Data, 2003-2005
Notes: Program representatives are not always able to contact all
graduates for placement information and therefore placement data reflect
only the students contacted, not all graduates.
aAccording to DOD, data for 2005 are preliminary, may change, and have not
yet been officially published in an annual report.
bMiscellaneous placements include caregivers, volunteers, homemakers,
incarcerated, or individuals who are unable to hold an approved placement
due to disability and/or hospitalization. Graduates must care for an
individual or work as a volunteer for a minimum of 30 hours per week to be
considered placed as a caregiver or volunteer.
Reserve Affairs and NGB Have Not Analyzed Data to Determine Actual Program Costs
or the Need to Adjust the Federal and State Cost Share
Although Reserve Affairs and NGB have expressed concern about the current
program funding level and have suggested increasing both the cost basis
used to determine funding needs and the federal cost share, we found that
neither Reserve Affairs nor NGB has performed analyses to support the need
for such changes. Good budget practices, included in the Office of
Management and Budget's Federal Financial Accounting Standards,13 state
that agencies should determine actual costs of their activities on a
regular basis and that reliable cost information is crucial for effective
management of government operations. Without better cost and financial
information, DOD cannot justify future funding requests or a change in the
cost-share ratio.
Other than calculating how inflation has affected program costs, NGB has
not analyzed data on actual program costs. Since 1993, NGB has used a cost
of $14,000 per student as the basis for determining the amount of funds
needed to cover program operating costs. In 2003, NGB calculated that if
that amount were adjusted for inflation, it would be $18,000. The results
of our survey of all Challenge Programs showed that in 2004, states
actually spent between $9,300 and $31,031 per graduate with an average of
$15,898 per graduate.14 In addition, our survey showed that, on average,
states estimated that the program should be funded at approximately
$16,900 per target graduate to cover all of the services in the
cooperative agreements, although the estimates ranged from $14,000 to
$31,800.15 Most Challenge Program officials also told us that increasing
the cost per student funding level for the program without increasing the
federal cost share would negatively impact their programs because their
states would be unlikely to come up with the additional state match money.
Because costs vary between states due to regional differences in salary
levels, staff benefits, and facility costs, Reserve Affairs has asked NGB
to determine a new funding formula for the program based on individual
state needs. At the time of our review, NGB had not yet done this and
Reserve Affairs has not given NGB a deadline for completion.
13Office of Management and Budget, Statement of Federal Financial
Accounting Standards, Number 4, Managerial Cost Accounting Concepts and
Standards for the Federal Government, (Washington, D.C.: July 31, 1995).
14We calculated this number with information provided to us by individual
states on our survey. According to our calculations, in 2004, the Fort
Gordon, Georgia Program spent $9,300 per graduated student and Alaska
spent $31,031 per graduated student. In order to cover the high personnel
costs in Alaska and to provide program enhancements, the state contributes
more than its required 40 percent share.
15 Five programs said that the current funding level of $14,000 per
graduate was sufficient: Ft. Gordon, Georgia; Camp Beauregard, Louisiana;
Montana; New Mexico; and Puerto Rico.
In addition to expressing a desire to change the amount of funding per
student, Reserve Affairs, NGB, and participating states have suggested
that the cost-share ratio be changed from its current 60 percent federal
share to a 75 percent federal share because they believe that the current
40 percent state share is sometimes difficult for states to meet; however,
neither Reserve Affairs nor NGB has analyzed states' financial situations
or the impact of adjusting the federal and state cost share. Challenge
Program officials told us that increasing the federal cost share of the
program would be beneficial because it would enable states to expand their
existing programs; give states more flexibility in funding their programs;
and allow programs to restore to students some services that had been
eliminated due to budgetary constraints. Although we did not analyze how
changing the cost basis or the cost-share ratio would affect specific
states, we prepared hypothetical examples for illustrative purposes. Table
1 shows how changing the cost basis and the cost-share ratio would affect
federal and state required funding levels.
Table 1: Hypothetical Examples Showing How Changing the Cost Basis or
Cost-Share Ratio Would Affect Yearly Challenge Program Funding Amounts
Per individual program Total Challenge Program
Typical
number Total
of number of
students State cost Federal cost students Total required Total required
per share per share per in state cost federal Total program
program program program programa share costshare costb
Current 200 $1,120,000 $1,680,000 7,000 $39,200,000 $58,800,000 $98,000,000
funding
situation
of
$14,000
per
student
and 60
percent
federally
funded
Change in 200 $1,352,000 $2,028,000 7,000 $47,320,000 $70,980,000 $118,300,000
cost
basis to
$16,900c
per
student
and 60
percent
federally
funded
Change in 200 $700,000 $2,100,000 7,000 $24,500,000 $73,500,000 $98,000,000
funding
ratio to
75
percent
federally
funded
with cost
basis of
$14,000
per
student
Change in 200 $845,000 $2,535,000 7,000 $29,575,000 $88,725,000 $118,300,000
cost
basis to
$16,900c
per
student
and 75
percent
federally
funded
Source: GAO analysis.
aApproximate number of students currently in program.
bThis amount does not include NGB overhead costs.
cThis amount is based on the average amount that states estimated in our
survey would cover all of the services outlined in the cooperative
agreements.
In our survey, states reported varying views on whether they were
experiencing difficulty in meeting their share of program costs. We did
not verify the basis for their responses. Our survey showed that some
states are able to provide funds above the required match; some states
provide only the required match; and some states are unable to provide a
match based on $14,000 per student and therefore fund the program at a
lower level and receive less money from NGB. For example, California is
able to provide additional support beyond the required match through
additional money provided from the state general fund and funding from the
program's local school district. In 2004, California spent approximately
$20,200 per graduate. Oregon, on the other hand, funds its program
primarily through state education money. Due to recent state budget
difficulties, Oregon cannot fund the program at $14,000 per student. In
2004, Oregon spent approximately $12,600 per graduate. According to the
National Guard Bureau, all 29 programs are providing the services required
by the cooperative agreements, and several states have added program
enhancements such as field trips or vocational classes. However, some
states reported that they reduced nonrequired services to stay within
their budgets. For example, they implemented pay and hiring freezes;
eliminated the student stipend; and eliminated program enrichment
activities, such as field trips and vocational classes. Some states told
us that additional funding, provided by a change in the cost-share ratio
or an increase in the per student funding amount, would allow them to
restore some of these services.
The National Guard Bureau Lacks a Complete Framework for Oversight of the
Challenge Program, Making It Difficult to Measure Program Effectiveness
Although NGB has several mechanisms in place for overseeing the Challenge
Program, it lacks a complete oversight framework, making it difficult to
measure the effectiveness of the program. A complete oversight framework,
as suggested by the Government Performance and Results Act of 1993 (the
Results Act)16 and Standards for Internal Control in the Federal
Government,17 includes performance goals and measures against which to
objectively measure performance as well as a mechanism for tracking
findings of audits or reviews and responding to those findings. Currently,
NGB does not require participating states to establish performance goals
for individual programs and therefore does not have a firm basis for
evaluating program outcomes and DOD's return on investment. In accordance
with the cooperative agreements, U.S. property and fiscal officers in each
state are required to conduct full audits of state Challenge Programs at
least every 3 years. However, these audits have not been conducted as
required; and, when audits are conducted, copies of the results are not
provided to NGB for review. Without regular audits and access to results,
NGB cannot be assured that programs are using federal funds appropriately
and that audit findings are addressed.
The National Guard Bureau Conducts Several Oversight Activities and Recently
Implemented a Mechanism for Tracking Audit Findings
NGB conducts several oversight activities for the Challenge Program. In
accordance with the DOD Instruction18 and cooperative agreements governing
the Challenge Program, NGB is responsible for the overall administration
of the program, including program oversight. NGB uses both informal and
formal mechanisms to oversee the program. On an informal basis, NGB
frequently communicates with state program directors via e-mail and
telephone calls. In addition, according to NGB, if a program director has
a problem or an issue that he or she feels NGB needs to be involved with,
he or she will initiate contact. Formal oversight of the program is
conducted by NGB through AOC Solutions with yearly operational evaluations
and biennial resource management reviews of all 29 Challenge Programs. The
purpose of the operational evaluations is to assess the programs'
compliance with the cooperative agreements and the implementation of the
eight core components. The resource management reviews focus on assessing
programs' financial accountability and reviewing resources including
staffing levels and salaries; food service costs; and physical inventory
of property.
16 Pub. L. No. 103-62 (1993).
17GAO, Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).
18Department of Defense, Instruction 1025.8, National Guard Challenge
Program, March 20, 2002.
Both the operational evaluations and the resource management reviews have
identified areas for improvement and, according to NGB, changes were made
to the program. For example, the operational evaluation of one state
program conducted in fiscal year 2004 reported that program staff was
calling the students inappropriate names. According to NGB, a staff member
was dismissed as a result of this finding. Another operational evaluation
of a different program conducted in fiscal year 2005 found that over 90
percent of the community service hours accumulated by the students were
for kitchen patrol. The cooperative agreements state that work in the
dining facility may not be counted towards community service hours.
According to NGB, this program has completely revamped its community
service program and students now participate in such activities as
visiting with residents of the local veterans' home and caring for a
historic cemetery.
Until recently, NGB did not have a formal mechanism for tracking the
findings of the reports conducted by AOC Solutions. During our review, we
discussed with NGB the importance of keeping track of review findings in
order to adequately respond to these findings, in accordance with
Standards for Internal Control in the Federal Government. In response to
our review, in October 2005, NGB provided new guidance to AOC Solutions
regarding the operational evaluations and resource management reviews,
which required, among other things, AOC Solutions to review findings from
previous evaluation reports to determine whether corrective actions have
been taken where warranted. In addition, officials at NGB told us that
they currently monitor responses to audit and review findings informally
with individual program directors.
The National Guard Bureau Does Not Require Participating States to Establish
Performance Goals for the Challenge Program
Although NGB requires state Challenge Programs to report on certain
outcome measures, such as GED attainment and number of graduates, NGB does
not require states to establish any performance goals in these areas to
measure the effectiveness of the program. The establishment of performance
goals is consistent with the principles of effective management as set
forth in the Results Act and would allow NGB to better evaluate the
overall performance of the program and to assess DOD's return on its
investment. Without clear and agreed upon performance goals, there is no
objective yardstick against which to fully measure program performance and
thereby assess DOD's return on investment.
Although it may not be reasonable to have the same performance goal for
all state programs, it would be appropriate for each state program to
negotiate a performance goal for defined performance areas such as
increases in standardized test scores or physical fitness levels. Similar
state programs, overseen by the Department of Labor, set individual
negotiated levels of performance for specified core performance measures.
These measures are used to provide information for systemwide reporting
and evaluation for program improvement. For example, state Workforce
Investment Act19 programs negotiate performance measures for youth ages 14
to 18 in three areas: attainment of basic skills; attainment of high
school diplomas or their equivalents; and placement in education, the
workforce, or the military. For the area of diploma attainment, the goals
range from 42.8 percent of participants in Louisiana to 68 percent of
participants in New Hampshire.
Currently, Challenge Program states are required to submit state plans
annually. These state plans are required to contain long-term and annual
performance goals and are to be updated annually. However, NGB has not
provided guidance on specific performance areas where states should focus
their goals; therefore, states may not have goals in the same performance
areas, making it difficult for NGB to compare performance across programs.
For example, California's state plan contains a goal to acquire additional
sources of funding through grants and charitable contributions. Oregon's
state plan, on the other hand, contains a goal stating that 80 percent of
graduates from the residential portion of the program will be placed in
education, the military, or employment but does not contain any goal
related to acquiring additional sources of funding. In addition, states
are not currently held accountable to the goals that they do set since the
evaluation process does not measure the states' performance against their
goals. NGB told us that beginning in January 2006, states would be held
accountable to the goals outlined in their state plans.
19The Workforce Investment Act of 1998 was passed with the goal of
consolidating, coordinating, and improving employment, training, literacy,
and vocational rehabilitation programs in the United States, Pub. L. No.
105-220 (1998). Youth programs under the Workforce Investment Act seek to
assist youth with employment and educational goals and the services needed
to achieve those goals.
U.S. Property and Fiscal Officers Have Not Conducted Required Audits and Audit
Results Are Not Provided to NGB
Although the cooperative agreements governing the Challenge Program
currently require U.S. property and fiscal officers to perform full audits
of state Challenge Programs at least every 3 years, and prior to January
2005, the cooperative agreements required full audits every year, these
audits have not been conducted as required. For example, according to NGB,
out of a required 29 audits, only 14 were conducted in 2003, and only 7
audits were conducted in 2004. According to property and fiscal officers
that we spoke with, audits were not conducted due to the lower priority
placed on these audits compared to other audits that needed to be
conducted within the state and a lack of staff to conduct the audits.
Because the property and fiscal officers are responsible for ensuring that
federal dollars are appropriately spent, if these audits are not
conducted, it may be difficult to ensure that federal interests are
adequately protected.
When the property and fiscal officers do conduct audits of the Challenge
Program, they are not currently providing copies of the audit results to
NGB because, according to the Chief of Property and Fiscal Affairs at NGB,
there is no specific requirement to do so. Standards for Internal Control
in the Federal Government states that agencies need to ensure that the
findings of audits and reviews are promptly resolved. If NGB does not
review these audits, it cannot ensure that audit findings are resolved or
identify trends across programs that may require action at a programwide
level. According to officials at NGB, the audits remain internal to the
state and the property and fiscal officer works directly with the state
Challenge Program site to resolve any issues. In addition, if any audit
findings require action from NGB, the state property and fiscal officer
will contact NGB and ask for assistance. NGB needs to be aware of all
audit findings, including those reported by the property and fiscal
officers to effectively manage the Challenge Program.
We reviewed the most recent property and fiscal officer audits for each
Challenge Program participating state and found that they did identify
areas that needed improvement. For example, an audit conducted of one
program in fiscal year 2004 found unspent program dollars totaling
approximately $180,000 that needed to be returned to the NGB. According to
a follow-up report by the property and fiscal officer, this money was
returned to NGB. Another audit conducted during fiscal year 2003 found
that oversight of budget expenditures for one program was not adequate to
ensure accurate, timely, and complete accounting of expenditures and
recommended that key management controls over the program be identified,
documented, published and tested. A follow-up audit of the program found
that this recommendation was implemented.
Reserve Affairs Lacks a Formal Approach for Securing Alternative Funding Support
for the Challenge Program, While Some Participating States Have Obtained
Alternative Funding Support as a Result of Their Efforts
Reserve Affairs has not adopted a formal strategy for pursuing nondefense
funding, while some participating states have obtained alternative funding
support for their programs as a result of their own efforts. Because
Reserve Affairs has not made a formal business case to request funds from
nondefense agencies, these agencies are unable to determine whether or not
they are able to fund the Challenge Program. As a result, Reserve Affairs
is potentially missing out on additional sources of funding that could
enhance the program. Some participating states have obtained alternative
funding support to enhance or maintain services provided to Challenge
Program students.
Nondefense Agencies Are Unable to Determine If They Can Fund the Challenge
Program without a More Formal Request for Funding
Reserve Affairs has not made a formal business case to request funds from
nondefense agencies and therefore these agencies are unable to determine
whether or not they are able to fund the Challenge Program. Although the
authorizing legislation for the Challenge Program allows the Secretary of
Defense to use nondefense funding sources in support of the program,
Reserve Affairs has not adopted a formal strategy for pursuing nondefense
funding. Rather, Reserve Affairs has primarily adopted informal strategies
to contact agencies outside DOD to inform them about the Challenge Program
and seek opportunities for partnerships. For example, Reserve Affairs
officials have sent information via e-mail to officials at federal
agencies with programs targeting at-risk youth (i.e. Department of Justice
(DOJ), Department of Labor (DOL), and Department of Education (ED)) to
inform them about the Challenge Program. These e-mails did not include
specific requests for program funding. In addition, in July 2003, Reserve
Affairs presented senior DOL officials with a proposal for forming an
interagency partnership with the Challenge Program, but did not ask for a
commitment of funding from DOL in a specified amount. Additionally,
officials from these agencies have been invited to events sponsored by the
National Guard Youth Foundation in support of the Challenge Program.
Lastly, Reserve Affairs officials told us that they participate or have
participated on a number of interagency councils and working groups that
represent at-risk youth such as the President's Crime Prevention Council,
the White House Federal Interagency Working Group on Service, the
Corporation for National and Community Service, the National Civilian
Community Corps, the Math Science Initiative, and the Juvenile Justice and
Delinquency Prevention Council.20
Officials from DOJ, DOL, and ED noted that their agencies have general
authority to provide funds to other programs if the transferred funds were
used to support a program consistent with their agency's interests. It is
not unprecedented for an agency to transfer funds to the Challenge
Program. In fiscal years 2000 and 2001, DOD and DOJ signed an interagency
agreement under which DOJ agreed to provide $5 million in 2000 and
$7,483,500 in 2001 to the Challenge Program.21 As DOJ concluded in this
situation, and as officials at DOJ, DOL and ED stated, the transfer of
funds to another agency must be consistent with the purpose and legal
requirements of the program from which funds would be transferred.
Moreover, these officials stated that sufficient funding must be available
in the program from which the funds would be transferred, and making such
a transfer must be in the interest of the agency.
Officials at DOJ, DOL, and ED stated that Reserve Affairs needed to
present more specific information to them before they could make a
determination as to whether funds could be provided to the Challenge
Program. These officials stated that an executive branch agency that
wishes to make a request for funding from another executive branch agency
could make such a request in any number of ways, either formally or
informally. At a minimum, they noted, any such request should contain the
amount of funding sought and a sufficiently detailed description of the
program to allow the agency receiving the request to determine whether it
would be an appropriate use of funds. At the time of our review, none of
the agency officials we met with were aware of any specific request from
Reserve Affairs concerning this matter. Until Reserve Affairs makes a more
formal request for funding, other agencies will be unable to determine if
they are able to provide funds for the Challenge Program.
20As of 1997 and 2000, respectively, the President's Crime Prevention
Council and the White House Federal Interagency Working Group on Service
no longer hold meetings.
21Specifically, the interagency agreement was signed between the
Department of Defense and the Department of Justice, Office of Justice
Programs, Office of Juvenile Justice and Delinquency Prevention. The
authority for this transfer of funds was found in the Juvenile Justice and
Delinquency Prevention Act of 1974, as amended (codified at 42 U.S.C.
S:5665). According to the interagency agreement signed in 2001, the funds
were to be used to fund up to eight new Challenge Programs; reduce
negative behaviors of program youth including delinquency, violence,
substance use, school suspensions and expulsions, and unplanned teen
pregnancies; and to engage up to 200 youth in each of the program sites in
constructive programming.
Some Participating States Have Obtained Alternative Funding Support to Enhance
or Maintain Services Provided to Challenge Program Students
Under cooperative agreements with NGB, states are not required to seek
funding support beyond the required federal and state contributions;
however, some states have made efforts to obtain alternative funding
support for their programs to enhance or maintain the services provided to
Challenge Program students. Under authorizing legislation for the
Challenge Program, states may accept, use, and dispose of gifts or
donations of money, other property, or services for the Challenge Program.
Reserve Affairs and NGB officials said that they encouraged states to seek
out additional funding sources for their programs if they want to enhance
the services provided to program students.
To assist state programs in their efforts to obtain that additional
support, NGB, through its contractor Dare Mighty Things, shares
information among state programs on strategies for successfully organizing
a 501 (c) (3) corporation,22 developing a fundraising policy that focuses
on the long-term vision of the Challenge Program, accessing and sharing a
grant writer with other programs, applying for National School Lunch
Program funds, and educating state legislators to secure funding support.
By providing information and examples of programs that are currently
implementing these strategies, other state Challenge Programs have the
information available to them on how to obtain additional funding support
for their programs. In our review of the program, we found that some
states had identified strategies for soliciting additional funding support
from nonprofit organizations. For example, in the annual plans submitted
to NGB, some states developed specific strategies for obtaining additional
funding support such as setting a goal for the number of grants they would
apply for and establishing a 501 (c) (3) corporation to raise funds on
behalf of the program.
We saw further evidence of states' efforts to obtain additional funding
support through our site visits and in our survey of the 29 Challenge
Program sites. For example, in one state we visited, the Challenge site
was also a charter school, which allowed the program to receive additional
funding from the local school district. Moreover, in four states we
visited, the state legislatures provided additional funding beyond the
required state match of 40 percent to pay for additional staffing costs or
facilities. According to our survey of all Challenge Programs, 28 out of
29 programs identified receiving some type of funding support beyond the
required federal and state contributions to pay for program expenditures
incurred during 1999 through 2004. These programs identified receiving
additional funding from the states' general fund beyond the required state
match as well as support from other state agencies such as the state
Department of Education. Additionally, the programs we surveyed relied on
funding assistance from other federal agencies such as the Department of
Agriculture, which provides funding under the National School Lunch
Program; nonprofit organization grants; the programs' private 501 (c) (3)
corporation; and donations from private individuals.
22According to the Internal Revenue Code, a 501 (c) (3) corporation is an
organization exempt from taxation, and organized and operated exclusively
for religious, charitable, scientific, testing for public safety,
literary, or educational purposes (26 U.S.C. S:501 (c) (3)).
Results from our site visits and national survey also showed that state
Challenge Programs rely on donations of goods and services to support the
program. For example, states we visited relied on donations from private
citizens, corporations, schools, and the states' National Guard units for
such items as computers and software, exercise equipment, books, uniforms,
and shoes. Some programs also received support from the surrounding
community in the form of donated services such as transportation
assistance and medical services from local doctors and nurses. Our survey
further showed that 21 out of 29 programs received some form of donated
goods and services. Program officials we met stated that donations of
goods and services were vital to programs because many times they do not
have funds available to acquire equipment or services for the program.
Donations of goods and services can enhance the basic program beyond the
military-based training and academics required under the cooperative
agreements adding little or no cost to the program.
Despite the success that many states had in obtaining additional funding
support and donations of goods and services for their programs, some
states reported obstacles in securing supplemental funding. For example,
the cooperative agreements governing the Challenge Program do not allow
states to use Challenge Program funds to hire full-time grant writers.
Lastly, some programs we visited expressed concern that if they received
funding support outside the federal and state required contributions, then
their state governments might reduce their allocations.
Conclusions
Although Reserve Affairs, NGB, and participating states have suggested the
current cost basis of $14,000 per student is not sufficient to sustain the
program and that the cost-share ratio should be changed, Reserve Affairs
and NGB have done little analysis to show what the actual costs of the
program are and how changing the cost-share ratio for the program would
impact participating states. Without better cost and financial
information, DOD cannot justify future funding requests or a change in the
cost-share ratio.
Until NGB establishes clear and agreed upon performance goals, there is no
objective yardstick against which to fully measure program performance.
Without these performance goals in place, NGB does not have a firm basis
for evaluating program outcomes and DOD's return on investment.
Although property and fiscal officers are required to conduct full audits
of the Challenge Program at least every 3 years, these audits have not
been conducted due to competing priorities at the state level and a lack
of staff. Without these audits, it may be difficult to ensure that federal
interests are adequately protected. Also, because NGB does not receive
copies of audits conducted by property and fiscal officers, it cannot know
what the findings of these audits were and if changes were made to the
programs based on these findings.
Until Reserve Affairs makes a more formal request for funding, other
agencies will be unable to determine if they can provide funds for the
Challenge Program. Without a more formal process that outlines the amount
of funding needed and a detailed description of the Challenge Program and
its specific funding needs, Reserve Affairs is potentially missing out on
additional sources of funding that could enhance the program.
Recommendations for Executive Action
To improve the management and oversight of the National Guard Youth
Challenge Program, we recommend that the Secretary of Defense, direct the
Under Secretary of Defense for Personnel and Readiness, in consultation
with the Assistant Secretary of Defense for Reserve Affairs and the Chief
of the National Guard Bureau, to take the following three actions:
o Determine the actual costs of the Challenge Program, including
states' ability to fund their share of the program, and use this
information, as appropriate, to support funding requests or a
request to change the cost-share ratio.
o Establish performance goals to measure the effectiveness of the
Challenge Program.
o Direct U.S. property and fiscal officers to conduct audits as
required and require that copies of audit results are provided to
the appropriate office at the National Guard Bureau in order to
ensure that the results of audits are promptly reviewed and
resolved.
To strengthen efforts at obtaining alternative funding in support
of the National Guard Youth Challenge Program, we recommend that
the Secretary of Defense direct the Undersecretary of Defense for
Personnel and Readiness, in consultation with the Assistant
Secretary of Defense for Reserve Affairs, to develop more formal
strategies for requesting alternative funding support for the
Challenge Program. Such strategies may include submitting requests
for funding that include the amount of funding requested and a
sufficiently detailed description of the proposed program to allow
potential providers of funds, such as nondefense agencies, to
determine whether it would be an appropriate use of their funds.
In written comments on a draft of this report, DOD concurred with
our recommendations. In its overall comments, DOD asserts that our
report reaffirms that the program met its congressional mandate to
improve the life skills and employment potential of participants
by providing military-based training and supervised work
experience. Further, DOD's comments state that our report
validates the core program components of assisting participants to
receive a high school diploma or its equivalent; developing
leadership skills; promoting fellowship and community service;
developing life coping and job skills; and improving physical
fitness, health and hygiene. DOD also states that it appreciates
GAO's confirmation that it is properly executing the program.
We do not agree with DOD's characterization of our report. Whereas
our report provides background data on program performance and
mentions that DOD has reported positive outcomes from the
Challenge Program, we do not assess whether or not DOD met its
congressional mandate; we do not validate the program's core
components, nor do we confirm that DOD is properly executing the
program, as DOD's comments suggest. Rather our objectives were to
examine: (1) historical trends of the National Guard Youth
Challenge Program, including program expenditures, participation,
and performance; (2) the extent to which Reserve Affairs and the
NGB have determined actual program costs and the need to adjust
the federal and state cost-share; and (3) the extent to which the
NGB has provided oversight of the program. We also determined the
extent to which Reserve Affairs and participating states have made
an effort to obtain alternative funding support for the Challenge
Program.
In response to our recommendation to determine the actual costs of
the Challenge Program, including the ability of states to fund
their share, DOD concurred, but claims that the matching fund
requirement makes the program vulnerable to state budget cuts. DOD
also contends that until recently, sluggish state revenues made
the cost share requirement more burdensome and inhibited the
ability of both the NGB and the states to focus on determining the
actual costs of the Challenge Program. DOD claims that the budget
shortfalls required the states to reduce and/or eliminate program
services and that, with state revenues slowly closing their budget
gaps, restoring funding for needed program services is being
considered. DOD also notes that it is currently working on a new
funding formula for the program.
Although DOD's comments suggest that adequate funding was not
available for needed services, through our survey and other work,
we found that all 29 programs were able to provide the services
required by the cooperative agreements, and several states
reported they added program enhancements such as field trips or
vocational classes. However, some states reported that they
reduced nonrequired services to stay within their budgets.
Furthermore, the basis for DOD's comment regarding the effect of
the matching fund requirement and the level of state revenues on
the program is unclear. As we reported, neither Reserve Affairs
nor NGB has compiled or analyzed data on actual program costs,
states' financial situations, or the impact of adjusting the
federal and state cost share. We continue to believe that any
future funding request, or a request to change the cost share
requirement, should be based on an analysis of the actual costs of
the program.
DOD's comments are printed in their entirety in appendix III.
We are sending copies of this report to the Secretary of Defense
and interested congressional committees. We will also make copies
available to others upon request. This report will be available at
no charge on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions regarding this report,
please contact me at (202) 512-9619 or [email protected]. Contact
points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff
who made key contributions to this report are listed in appendix
IV.
Sharon L. Pickup Director, Defense Capabilities and Management
To examine historical trends of the National Guard Youth Challenge
Program, including program expenditures, participation, and
performance, we interviewed officials from the Office of the
Assistant Secretary of Defense for Reserve Affairs; the National
Guard Bureau (NGB); and the contractor that monitors and evaluates
state programs, AOC Solutions. We reviewed other documentation
provided by Reserve Affairs and NGB, such as program funding
summaries. We also reviewed the Challenge Program's annual reports
submitted to Congress from 1994 through the present, excluding
1997,1 which provided us background information on the program.
For data on participation and performance outcomes, including
general education development credential attainment, community
service hours, and post-residential placements, we relied on the
Data Management and Reporting System (DMARS) that is used by NGB
to collect participant information and track individual and
program activities. DMARS was implemented for the 2003 reporting
year, and the contractor has performed procedures to clean up the
data as far back as 2000. We conducted data reliability tests on
the program's annual reports and data management system to
conclude that the data are sufficiently reliable for our purposes.
We did not compare the costs or outcomes of this program to other
similar youth programs currently funded by the federal government
since that was not within the scope of this engagement.
To assess the extent to which Reserve Affairs and NGB determined
actual program costs and analyzed states' ability to fund their
share of the program, we interviewed officials at Reserve Affairs
and NGB as well as officials at state programs. We reviewed and
analyzed internal budget documents prepared by state programs and
submitted to NGB annually as well as documents prepared by
individual program directors that discussed the funding situation
in their state. We also reviewed and analyzed good budget
practices as described in federal financial accounting standards.
Using a semi-structured questionnaire, we interviewed eight state
Challenge Program directors and their budget officers in seven
states and discussed sources of funding for their programs, actual
costs of operating their programs, and the types of services that
are provided to program participants given the available funding.
States selected for site visits or telephone interviews are shown
in table 2. There were a number of factors which affected the
judgmental sampling of these state programs. States were chosen
because of their length of time in the program, size, geographic
location, experience of the program's director, and whether or not
there were multiple programs in the state. Additionally, in July
2005, we surveyed all 29 participating Challenge Programs and
asked them about the actual costs of their programs and their
assessment of their states' ability to fund the programs. This
survey had a response rate of 100 percent.
Table 2: States Selected for Site Visits or Interviews
Source: GAO.
To assess the extent to which NGB has provided oversight of the
Challenge Program, we interviewed officials at Reserve Affairs and
NGB as well as program officials at the state level. To determine
the oversight responsibilities of federal and state officials
involved in the program, we reviewed and analyzed pertinent laws,
the Department of Defense's (DOD) Instruction governing the
program, and the NGB Master Cooperative Agreement, which defines
the terms and conditions of the program in each state. We also
interviewed the independent contractor hired by NGB to conduct
yearly and biennial on-site program reviews, AOC Solutions. We
obtained access to its data management system and analyzed the
program evaluations and resource management reviews it completed
on behalf of NGB since 2003. We also reviewed and analyzed
individual state program's annual plans and NGB's strategic plan
to identify the types of goals set by the Challenge program. We
consulted previous GAO work regarding performance measurement and
evaluations, identified best practices for establishing and
measuring performance goals, and reviewed Standards for Internal
Control in the Federal Government and the Government Performance
and Results Act of 1993. We interviewed representatives of the
U.S. Property and Fiscal Officer's Office in six states as well as
the NGB Chief of Property and Fiscal Affairs to determine the time
frame for completing audits and mechanisms for reporting,
tracking, and resolving issues that arise out of the PFO audits.
To determine the extent to which Reserve Affairs and participating
states have obtained alternative funding support for the program,
we reviewed relevant laws, policies, and reports to determine the
relevant authorities for receiving and transferring funds between
federal agencies. We also interviewed federal officials at Reserve
Affairs and the Departments of Labor, Education, and Justice to
determine the extent to which these agencies discussed the
possibility of transferring funds to DOD in support of the
Challenge program. To determine states' efforts at obtaining
alternative funding support for their program's operations, we
conducted a survey of all 29 participating programs to collect
information about their sources of funding for the program, how
funding is distributed across different program operational
functions, the sources and types of donated goods and services,
and descriptions of strategies programs use to obtain alternative
funding support. Using a semi-structured questionnaire, we
interviewed eight state Challenge Program directors and members of
their staff in seven states and discussed sources of funding for
their programs, the strategies these specific programs use for
obtaining alternative funding support and donations, and the
difficulties these states face in seeking out other funding.
We provided a draft of this report to officials at DOD for their
review and incorporated their comments where appropriate. We
conducted our work from January 2005 to October 2005 in accordance
with generally accepted government auditing standards.
Source: GAO analysis of data from AOC Solutions, Inc. for NGB
Classes 21 and 22.
Note: The Wyoming Youth Challenge Program received start-up funds
in 2005, and will begin its first class in January 2006. Nevada
does not have a Challenge Program site but sends approximately 24
youths per year to the Arizona site. Until fiscal year 2005, the
District of Columbia sent 60 youths per year to the Maryland site.
Sharon L. Pickup (202) 512-9619
In addition to the individual named above, Harold Reich, Karyn
Angulo, Leslie Bharadwaja, Susan Ditto, K. Nicole Harms, Wilfred
Holloway, Jessica Kaczmarek, Stanley Kostyla, Julia Matta, Renee
McElveen, and John Van Schaik made key contributions to this
report.
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO
posts newly released reports, testimony, and correspondence on its
Web site. To have GAO e-mail you a list of newly posted products
every afternoon, go to www.gao.gov and select "Subscribe to
Updates."
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
Agency Comments and Our Evaluation
Appendix I: Scope and Methodology Appendix I: Scope and Methodology
1According to NGB, there was no annual report published for 1997 because
program officials were not sure whether or not Congress would permanently
authorize the program.
State Program Name
Arizona Arizona Project Challenge
California California Grizzly Youth Academy
Illinois Illinois Lincoln's Challenge Academy
Maryland Maryland Freestate Challenge Academy
Oregon Oregon National Guard Youth Challenge Program
South Carolina Camp Long
South Carolina Columbia
Wisconsin Wisconsin National Guard Challenge Academy
Appendix II: National Guard Youth Challenge Program Sites Appendix II:
National Guard Youth Challenge Program Sites
Calendar year
first class Number of
State Program name began graduates in 2004
Alaska Alaska Military Youth 1994 192
Academy
Arizona Arizona Project Challenge 1993 182
Arkansas Arkansas National Guard 1993 181
Youth Challenge Academy
California California Grizzly Youth 1998 204
Academy
Florida Florida Youth Challenge 2001 227
Academy
Georgia Georgia - Fort Gordon Youth 2000 276
Challenge Academy
Georgia - Fort Stewart 1993 444
Youth Challenge Academy
Hawaii Hawaii National Guard Youth 1994 207
Challenge Academy
Illinois Illinois Lincoln's 1993 785
Challenge Academy
Kentucky Kentucky Bluegrass 1999 160
Challenge Academy
Louisiana Louisiana Youth Challenge 1993 407
Program - Camp Beauregard
Louisiana Youth Challenge 2002 214
Program - Camp Minden
Louisiana Youth Challenge 1999 331
Program - Gillis W. Long
Maryland Maryland Freestate 1993 224
Challenge Academy
Michigan Michigan Youth Challenge 1999 174
Academy
Mississippi Mississippi National Guard 1994 404
Youth Challenge Program
Montana Montana Youth Challenge 1999 144
Program
New Jersey New Jersey Youth Challenge 1994 178
Program
New Mexico New Mexico Youth Challenge 2001 197
Academy
North Carolina North Carolina Tarheel 1994 200
Challenge Academy
Oklahoma Oklahoma Thunderbird Youth 1993 221
Academy
Oregon Oregon National Guard Youth 1999 221
Challenge Program
Puerto Rico Puerto Rico Youth Challenge 1999 208
Program
South Carolina South Carolina Youth 2002 108
Challenge Academy - Camp
Long
South Carolina Youth 1998 257
Challenge Academy -
Columbia
Texas Texas Seaborne Challenge 1999 188
Corps
Virginia Virginia Commonwealth 1994 140
Challenge Program
West Virginia West Virginia Mountaineer 1993 175
Challenge Academy
Wisconsin Wisconsin National Guard 1998 154
Challenge Academy
Total Graduates 7,003
Appendix III: Comments from the Department of Defense Appendix III:
Comments from the Department of Defense
Appendix IV: A Appendix IV: GAO Contact and Staff Acknowledgment
GAO Contact
Acknowledgments
(350643)
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone
To Report Fraud, Waste, and Abuse in Federal Programs
Congressional Relations
Public Affairs
www.gao.gov/cgi-bin/getrpt?GAO-06-140.
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Sharon Pickup at (202) 512-9619 or
[email protected].
Highlights of GAO-06-140, a report to congressional committees
November 2005
DEFENSE MANAGEMENT
Actions Are Needed to Improve the Management and Oversight of the National
Guard Youth Challenge Program
The fiscal year 1993 National Defense Authorization Act established the
National Guard Youth Challenge Program as a pilot program to evaluate the
effectiveness of providing military based training to improve the life
skills of high school dropouts. The Assistant Secretary of Defense for
Reserve Affairs, under the authority of the Under Secretary of Defense for
Personnel and Readiness, is responsible for overall policy for the
program. The National Guard Bureau (NGB) provides direct management and
oversight. In 1998, Congress permanently authorized the program and began
decreasing the federal cost share until it reached its current level of 60
percent in 2001. Conference Report 108-767 directed GAO to review the
program. Specifically, GAO reviewed (1) historical trends of the program;
(2) the extent of analyses performed to determine program costs and the
need to adjust the federal and state cost share; and (3) NGB oversight of
the program. GAO is also providing information on Reserve Affairs' and
states' efforts to obtain funding from alternative sources.
What GAO Recommends
GAO recommends a number of actions designed to improve the management and
oversight of the National Guard Youth Challenge Program. DOD concurred
with GAO's recommendations.
Between fiscal years 1998 and 2004, total expenditures for the Challenge
Program, including funds spent to cover the federal and state cost shares
and federal management expenses, have increased from about $63 million to
$107 million. During this same period, participation in the program has
grown from 10 sites in 10 states to 29 sites in 24 states and Puerto Rico.
Since the program's inception, NGB has reported positive performance
outcomes in academic performance, community service activities, and
post-residential placements. For example, in 2004, NGB reported graduating
7,003 students, or 79 percent of those enrolled, with 70 percent of those
graduates earning a high school equivalent diploma.
While Reserve Affairs and NGB have expressed concern about the current
program funding level and have suggested increasing both the cost basis
used to determine funding needs and the federal cost share, neither has
performed analyses to support the need for such changes. Federal financial
standards state that reliable cost information is crucial for effective
management of government operations. Since 1993, NGB has used $14,000 per
student as the basis for determining the amount of funds needed to cover
program operating costs, and applied the federal-state cost share to this
amount. To keep pace with inflation, NGB has suggested increasing the per
student cost to $18,000. Reserve Affairs has reported some states are
having difficulty meeting their share and, in 2004, recommended the
federal share be increased from 60 percent to 75 percent. However, neither
Reserve Affairs nor NGB has compiled or analyzed data on actual program
costs, states' financial situations, or the impact of adjusting the
federal and state cost-share. Without better cost and financial
information, the Department of Defense (DOD) cannot justify future funding
requests or a change in the cost-share ratio.
Although NGB uses various oversight mechanisms, it lacks a complete
oversight framework, making it difficult to measure program effectiveness
and to adequately address audit and review findings. Also, some audits
have not been performed as required. The Government Performance and
Results Act suggests a complete oversight framework including goals and
measures against which to objectively evaluate performance. While NGB
requires states to report certain performance outcomes, it does not
require states to establish performance goals in these areas, and
therefore does not have a firm basis for evaluating program outcomes and
DOD's return on investment. Existing agreements require state programs to
be audited at least every three years. However these audits have not been
conducted as required and no provisions exist for submitting audit results
to NGB. Without regular audits and access to results, NGB cannot be
assured that programs are using federal funds appropriately and that audit
findings are addressed.
*** End of document. ***