Defense Infrastructure: NORAD and USNORTHCOM Need to Reevaluate
Vulnerabilities Associated with Moving the NORAD Command Center
from Cheyenne Mountain to Peterson Air Force Base, and to
Acknowledge Acceptance of the Risks (18-SEP-08, GAO-08-1054R).
In July 2006, the former Commander of North American Aerospace
Defense Command (NORAD) and United States Northern Command
(USNORTHCOM) announced plans to relocate certain functions from
Cheyenne Mountain to create an integrated command center in
Building 2 at Peterson Air Force Base (AFB), Colorado. In May
2007, we reported that NORAD and USNORTHCOM had not analyzed the
anticipated operational effects--both positive and negative--of
the relocation, and that the Department of Defense (DOD) could
not discern the full costs or security implications of the move
until ongoing security assessments had been completed and a
protection level designated for the integrated command center. We
suggested that Congress should consider restricting DOD's
authority to fund the relocation until all security analyses were
complete, the full costs for the move were determined, and DOD
provided Congress with an analysis of the operational effects of
the proposed realignments. As a result, in the National Defense
Authorization Act for Fiscal Year 2008 (hereinafter referred to
as the Act), Congress directed the Secretary of Defense to submit
a report by March 1, 2008, assessing the relocation of the NORAD
Command Center and related functions from Cheyenne Mountain to
Peterson AFB. The Act required the report to contain (1) an
analysis comparing the total costs associated with the
relocation, including costs determined as part of ongoing
security-related studies of the relocation, to anticipated
operational benefits from the relocation; (2) a detailed
explanation of the backup functions that will remain located at
Cheyenne Mountain, and how those functions will maintain
operational connectivity with their related commands; (3) the
final plans for the relocation of the NORAD Command Center and
related functions; and (4) the findings and recommendations
resulting from the independent security and vulnerability
assessment of Peterson AFB, including the Secretary of Defense's
plans for mitigating any security and vulnerability risks identif
ied and estimates for associated costs and scheduling. The Act
mandated that we review DOD's report and the final plans for the
relocation, and that we report to Congress within 120 days. On
March 3, 2008, DOD submitted its report to Congress. DOD's report
included a cost-benefit analysis comparing the following three
alternatives: Status quo--retain separate command centers at
Cheyenne Mountain and Peterson AFB. Establish a combined and
integrated command center at Peterson AFB with reach-back
capability to the computer systems at Cheyenne Mountain.
Establish a combined command center at Peterson AFB that
duplicates the systems at Cheyenne Mountain.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-08-1054R
ACCNO: A84287
TITLE: Defense Infrastructure: NORAD and USNORTHCOM Need to
Reevaluate Vulnerabilities Associated with Moving the NORAD
Command Center from Cheyenne Mountain to Peterson Air Force Base,
and to Acknowledge Acceptance of the Risks
DATE: 09/18/2008
SUBJECT: Accountability
Air Force bases
Cost analysis
Critical infrastructure
Defense cost control
Defense operations
Defense procurement
Facility security
Information classification
Military facilities
Relocation allowances
Reporting requirements
Risk assessment
Risk management
Schedule slippages
Security assessments
Security policies
Strategic planning
NORAD Command Center
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-08-1054R
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to [email protected].
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-08-1054R:
United States Government Accountability Office:
Washington, DC 20548:
September 18, 2008:
Congressional Committees:
Subject: Defense Infrastructure: NORAD and USNORTHCOM Need to
Reevaluate Vulnerabilities Associated with Moving the NORAD Command
Center from Cheyenne Mountain to Peterson Air Force Base, and to
Acknowledge Acceptance of the Risks:
In July 2006, the former Commander of North American Aerospace Defense
Command (NORAD) and United States Northern Command (USNORTHCOM)
announced plans to relocate certain functions from Cheyenne Mountain to
create an integrated command center in Building 2 at Peterson Air Force
Base (AFB), Colorado. In May 2007, we reported that NORAD and
USNORTHCOM had not analyzed the anticipated operational effects--both
positive and negative--of the relocation, and that the Department of
Defense (DOD) could not discern the full costs or security implications
of the move until ongoing security assessments had been completed and a
protection level designated for the integrated command center.[Footnote
1] We suggested that Congress should consider restricting DOD's
authority to fund the relocation until all security analyses were
complete, the full costs for the move were determined, and DOD provided
Congress with an analysis of the operational effects of the proposed
realignments.
As a result, in the National Defense Authorization Act for Fiscal Year
2008[Footnote 2] (hereinafter referred to as the Act), Congress
directed the Secretary of Defense to submit a report by March 1, 2008,
assessing the relocation of the NORAD Command Center and related
functions from Cheyenne Mountain to Peterson AFB. The Act required the
report to contain (1) an analysis comparing the total costs associated
with the relocation, including costs determined as part of ongoing
security-related studies of the relocation, to anticipated operational
benefits from the relocation; (2) a detailed explanation of the backup
functions that will remain located at Cheyenne Mountain, and how those
functions will maintain operational connectivity with their related
commands; (3) the final plans for the relocation of the NORAD Command
Center and related functions; and (4) the findings and recommendations
resulting from the independent security and vulnerability assessment of
Peterson AFB, including the Secretary of Defense's plans for mitigating
any security and vulnerability risks identified and estimates for
associated costs and scheduling. The Act mandated that we review DOD's
report and the final plans for the relocation, and that we report to
Congress within 120 days. On March 3, 2008, DOD submitted its report to
Congress.[Footnote 3] DOD's report included a cost-benefit analysis
comparing the following three alternatives:[Footnote 4]
* Status quo--retain separate command centers at Cheyenne Mountain and
Peterson AFB.
* Establish a combined and integrated command center at Peterson AFB
with reach-back capability to the computer systems at Cheyenne
Mountain.
* Establish a combined command center at Peterson AFB that duplicates
the systems at Cheyenne Mountain.
DOD's report to Congress also described the functions remaining at
Cheyenne Mountain, provided a diagram of the final configuration of the
command center at Peterson AFB, summarized the Air Force Space
Command's classified security and vulnerability assessment, known as
the Systems Effectiveness Assessment (SEA), and included the SEA as an
attachment.[Footnote 5] Our report to Congress,[Footnote 6] which was
classified by DOD, was issued on July 1, 2008, and provides additional
details on the security issues surrounding the relocation of the NORAD
Command Center from Cheyenne Mountain to Peterson AFB. This report is
the unclassified version of our classified report.
Because of the nature of the assets being moved, the Air Force must
designate a protection level for the assets being moved from Cheyenne
Mountain to Peterson AFB. The Air Force uses its protection level
system to allocate security resources based on the respective risks
associated with different assets. If resources are not available to
meet the assigned protection level requirements, then the commander
must obtain permanent exceptions or temporary waivers from the security
requirements and develop compensatory measures.[Footnote 7] The Air
Force designated the functions moving into the integrated command
center as Protection Level-1, signifying that the loss, theft,
destruction, misuse, or compromise of these assets would result in
great harm to the strategic capability of the United States.
DOD is proceeding with its plans to relocate the NORAD Command Center
and other functions from Cheyenne Mountain to Peterson AFB and,
according to DOD officials, as of May 29, 2008, operations had already
begun at the combined command center.
In reviewing DOD's report to Congress, our objectives were (1) to
evaluate DOD's assumptions in its cost-benefit analysis of the three
alternatives, and their effect on the recommendation; (2) to determine
the extent to which DOD's report assessed and contained a plan to
mitigate the security risks DOD identified at Peterson AFB; and (3) to
determine the extent to which the final relocation plans take into
account security issues raised in DOD's report.
To conduct our evaluation, we reviewed DOD's report to Congress and the
associated security study completed by Air Force Space Command. To
assess the assumptions DOD used in its cost-benefit analysis related to
the relocation and to determine how they affected the recommendation,
we reviewed the cost-benefit analysis, examining the costs as well as
the benefits, and determined whether DOD had completed a sensitivity
analysis for key sources of uncertainty. However, we did not
independently verify or validate the cost estimate. We examined the
assumptions, such as the discount rate used, and how benefits were
measured. We also performed a sensitivity analysis for benefits to
determine how sensitive the outcomes were to changes in benefit scores.
To determine the extent to which DOD's report assessed and contained a
plan to mitigate the security risks DOD identified at Peterson AFB, we
compared the Air Force Space Command's security study with DOD's report
to Congress, examining how DOD characterized the risks, mitigation
plans, and cost and schedule estimates contained in the security study.
To determine the extent to which the final relocation plans presented
in DOD's report took into account security issues raised in DOD's
report, we compared the report's presentation of plans with the
report's summary of security issues. We also reviewed prior GAO work on
the Cheyenne Mountain relocation. We conducted our work from April to
July 2008 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives. We prepared
this unclassified version of our classified report from August to
September 2008.
Summary:
DOD's report to Congress neither recognized the uncertainty of benefit
scoring of the three options it analyzed for the planned relocation of
certain functions from Cheyenne Mountain to Peterson AFB, Colorado, nor
included a sensitivity analysis for the benefits used in calculating
the cost-benefit ratio for the options. The scoring of the benefit
factors was based on functional managers' subjective estimates of the
factors' relative importance and fulfillment of requirements. However,
DOD's cost-benefit analysis did not recognize the uncertainty of the
benefits. Moreover, although Office of Management and Budget (OMB)
guidance calls for the performance of a sensitivity analysis of key
sources of uncertainty, such as, in this case, the subjective scoring
of benefits, there is no indication that DOD performed such an analysis
regarding either costs or benefits, and we found that a slight change
in the benefit scores could significantly change the outcome as
measured by the cost-benefit ratio. For example, raising the benefit
score for the status quo by just 5 percent--a change that, in our
opinion, falls within the margin of imprecision for a subjective
judgment--would cause the status quo to become the preferred option. In
addition, based on the limited cost information in DOD's report to
Congress, it is unclear how sensitive DOD's cost estimates are to
different assumptions.
DOD's report to Congress did not provide a detailed mitigation strategy
for all of the security and vulnerability risks identified in Air Force
Space Command's classified security assessment and did not include all
plans, costs, or schedule estimates. Also, it is unclear whether
security upgrades meet necessary requirements. DOD's report understates
the security challenges at Peterson AFB. Further, like the SEA, DOD's
report does not address the full spectrum of threats and hazards
associated with Peterson AFB. The classified version of this report
contains information about the specific threats that were excluded from
the scope of the Air Force Space Command's security assessment, and
raises questions about how fully the SEA addresses certain key threats
and all hazards. DOD's report also did not include plans, costs, or
schedule estimates for mitigating all risks identified, since some
recommendations were awaiting a conceptual design from Air Force Space
Command before plans, costs, or schedule estimates could be determined.
Furthermore, although the missions moving from Cheyenne Mountain were
designated as Protection Level-1, at the time DOD issued its report to
Congress, it did not state whether all recommended security upgrades
met Protection Level-1 requirements and, therefore, whether waivers
would be needed to begin operations at Peterson AFB. DOD officials told
us that they had obtained waivers and, as a result, the new combined
command center met the necessary security requirements as of May 28,
2008. DOD subsequently provided us with copies of three waivers (known
as a Request for Deviation from Security Criteria), each of which was
approved on May 20, 2008.
The section of the DOD report regarding final plans for the relocation
does not identify any security issues, including those that were
identified in the SEA. Rather, it includes only a diagram of the final
configuration of the command center at Peterson AFB.
We are recommending that the Commander of NORAD and USNORTHCOM
reevaluate the full spectrum of security vulnerabilities associated
with moving the NORAD Command Center and related functions from
Cheyenne Mountain to Peterson AFB, and that the Commander certify that
he is fully aware of and accepts all of the risks.
In written comments on a draft of this report, DOD disagreed with our
recommendation that the Commander of NORAD and USNORTHCOM reevaluate
the full spectrum of security vulnerabilities associated with moving
the NORAD Command Center and related functions from Cheyenne Mountain
to Peterson AFB. DOD stated that the SEA focused on threats considered
most likely to affect Peterson AFB and Building 2, and that DOD viewed
a threat assessment covering all possible threats encompassing both
Cheyenne Mountain and Building 2 as unfocused. DOD stated that it
considers the risk of certain other key threats--which we identified in
the classified version of this report--to be low and outweighed by the
benefits provided by the combined command center. However, we note that
although the SEA did develop a "threat spectrum" that defined a range
of potential threats to NORAD and USNORTHCOM, at the time of our
review, DOD could not provide any documented evidence of having
performed a risk assessment that analyzed the most likely threats to
Peterson AFB and Building 2, nor any documented basis for its assumed
assessment of low probability for certain key threats that were
excluded from the scope of the SEA. We continue to believe that DOD
should document having performed a risk assessment that analyzed the
most likely threats to Peterson AFB and Building 2, along with the
basis for its assumed assessment of low probability and adequate
warning of certain key threats. DOD neither agreed nor disagreed with
our recommendation that the Commander certify that he is fully aware of
all of the risks associated with moving the NORAD Command Center and
related functions from Cheyenne Mountain to Peterson AFB, and accepts
those risks. DOD stated that through the waiver process and other
mitigation actions, the Commander has formally accepted the outstanding
actions and associated risks related to a Protection Level-1 facility.
DOD recently provided us with copies of three waivers; however, it was
unclear to us that the Commander had explicitly accepted the risks
posed by the full spectrum of threats or hazards. Thus, we continue to
believe that our recommendation has merit and that he should certify
that he accepts those risks.
Subsequent to DOD's letter containing the comments restated above, DOD
provided us with another letter on August 29, 2008, containing its
comments on our final classified report. In its additional comments,
the department stated that NORAD and USNORTHCOM are in the process of
implementing GAO's recommendation that the Commander of NORAD and
USNORTHCOM reevaluate the full spectrum of security vulnerabilities
associated with moving the NORAD Command Center and related functions
from Cheyenne Mountain to Peterson AFB. Specifically, DOD stated that a
new Director of Security has been appointed and is leading a Security
Tiger Team, which has partnered with Sandia National Laboratories to
evaluate all threats and vulnerabilities to the headquarters. Moreover,
DOD stated that in concert with other planned vulnerability
assessments, the Security Tiger Team is recommending actions to
mitigate vulnerabilities and the Commander is incrementally approving
changes to the security posture of the headquarters as a result of this
process. We have not verified or validated the information provided in
these additional comments.
Background:
During a series of major exercises conducted in 2005, the NORAD/
USNORTHCOM Commander directed planning, operations, and command and
control elements from two separate command centers. In the course of
the exercises, the Commander identified impediments to unity of effort
and time-critical decision making, and he attributed these impediments
to the geographic separation of the two command centers. A subsequent
analysis conducted by a NORAD/USNORTHCOM senior official concluded that
having a single command center at Peterson AFB represented the only
option that offered both the physical space required for a consolidated
command center and a strengthened unity of effort between the commands.
A USNORTHCOM study[Footnote 8] outlined a second option to move certain
functions out of Cheyenne Mountain while retaining the core computer
systems there, providing what DOD refers to as "reach-back." NORAD and
USNORTHCOM officials stated that once the functions and their
associated personnel were moved, they intended to use Cheyenne Mountain
as an alternate command center.
DOD's Cost-Benefit Analysis Does Not Recognize the Uncertainty of
Benefits and Lacks a Sensitivity Analysis:
DOD's report to Congress neither recognized the uncertainty of benefit
scoring of the three options it analyzed for the planned relocation of
certain functions from Cheyenne Mountain to Peterson AFB, Colorado, nor
included a sensitivity analysis for the benefits used in calculating
the cost-benefit ratio for the options. DOD's report used subjective
and imprecise measurements of the benefits of the three options it
analyzed for the planned relocation of certain functions from Cheyenne
Mountain to Peterson AFB, Colorado. We recognize that subjectivity can
be involved in estimating costs and benefits, which typically are
uncertain because of imprecision in both underlying data and modeling
assumptions. However, OMB guidance states that "because uncertainty is
common to many analyses, its effects should be analyzed and reported."
[Footnote 9] As required by the Act, DOD's report to Congress included
an analysis comparing the total costs associated with the relocation of
the NORAD Command Center and related functions against the anticipated
operational benefits. DOD calculated the costs and benefits for the
following three alternatives:
* Alternative 1--Status Quo: retaining separate command centers (split
operations) at Cheyenne Mountain and Peterson AFB.
* Alternative 2--Reach-back Capability: establishing a combined and
integrated command center at Peterson AFB (the primary command center),
with reach-back capability to key computer systems at Cheyenne Mountain
(the alternate command center).
* Alternative 3--Duplicate Systems: establishing a combined command
center at Peterson AFB (the primary command center) that duplicates the
capabilities at Cheyenne Mountain (the secondary command center). This
third alternative would result in stand-alone systems at both sites.
In calculating costs, DOD considered nonrecurring investment costs and
recurring costs. For all three alternatives, the nonrecurring
investment costs were sustained in the first year of analysis, and the
recurring costs would be sustained in every year over the 10-year
period of analysis. Costs that were identical for each of the
alternatives were not considered. All of the costs were presented in
2008 constant dollars. The total costs for the three alternatives over
the 10-year period of analysis were calculated in present value
[Footnote 10] terms using a 2.8 percent discount rate.[Footnote 11]
Table 1 shows the total costs for each alternative.
Table 1: Comparison of Total Costs for the Three Relocation
Alternatives (2008 constant dollars):
Present value of total costs:
Alternative 1- Status Quo: $20,011,111;
Alternative 2- Reach-back Capability: $71,762,643;
Alternative 3- Duplicate Systems: $137,038,661.
Source: DOD.
[End of table]
The derived benefits from each alternative could not be measured
monetarily, so DOD considered nine nonmonetary factors. According to
DOD, these benefit factors were analyzed during a meeting of NORAD/
USNORTHCOM functional managers. Each manager ranked the nonmonetary
benefits, and the weight points were assigned on a scale of 1 through
10 to reflect each benefit's relative importance; the more important
the benefit, the greater the number of weight points. Each of the three
alternatives was weighted on a continuous scale from 0 to 100 percent,
with 0 percent signifying that the alternative does not meet all
requirements and 100 percent signifying that the alternative meets all
requirements. These two weight values--weight points and requirements
percentages--were multiplied to derive a benefit score. The benefit
score was divided into the total cost of an alternative to determine
the cost-benefit ratio for each alternative. As table 2 shows,
Alternative 2--Reach-back Capability--has the lowest cost-benefit
ratio, at $1,028,855, indicating that it had the lowest cost per unit
of benefit, that is, the cheapest alternative relative to benefits.
Table 2: Cost-Benefit Analysis of the Three Relocation Alternatives:
Benefit factors: Superior decision making;
Weight points: 10.0;
Alternative 1--Status Quo: Requirements (percent): 15;
Alternative 1--Status Quo: Benefit score: 1.5;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 10.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 10.0.
Benefit factors: Full spectrum integration;
Weight points: 9.0;
Alternative 1--Status Quo: Requirements (percent): 15;
Alternative 1--Status Quo: Benefit score: 1.4;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 9.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 9.0.
Benefit factors: Simultaneous command and control processes;
Weight points: 9.0;
Alternative 1--Status Quo: Requirements (percent): 25;
Alternative 1--Status Quo: Benefit score: 2.3;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 9.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 9.0.
Benefit factors: Dispersed command and control;
Weight points: 8.0;
Alternative 1--Status Quo: Requirements (percent): 40;
Alternative 1--Status Quo: Benefit score: 3.2;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 7.2;
Alternative 3--Duplicate Systems: Requirements (percent): 90;
Alternative 3--Duplicate Systems: Benefit score: 7.2
Benefit factors: Shared understanding;
Weight points: 8.0;
Alternative 1--Status Quo: Requirements (percent): 20;
Alternative 1--Status Quo: Benefit score: 1.6;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 8.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 8.0.
Benefit factors: Responsive and tailorable organization;
Weight points: 8.0;
Alternative 1--Status Quo: Requirements (percent): 20;
Alternative 1--Status Quo: Benefit score: 1.6;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 8.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 8.0.
Benefit factors: Shared quality information;
Weight points: 7.0;
Alternative 1--Status Quo: Requirements (percent): 30;
Alternative 1--Status Quo: Benefit score: 2.1;
Alternative 2--Reach-back Capability: Requirements (percent): 100;
Alternative 2--Reach-back Capability: Benefit score: 7.0;
Alternative 3--Duplicate Systems: Requirements (percent): 100;
Alternative 3--Duplicate Systems: Benefit score: 7.0.
Benefit factors: Robust networking;
Weight points: 7.0;
Alternative 1--Status Quo: Requirements (percent): 40;
Alternative 1--Status Quo: Benefit score: 2.8;
Alternative 2--Reach-back Capability: Requirements (percent): 75;
Alternative 2--Reach-back Capability: Benefit score: 5.3;
Alternative 3--Duplicate Systems: Requirements (percent): 95;
Alternative 3--Duplicate Systems: Benefit score: 6.7.
Benefit factors: Flexible synchronization;
Weight points: 7.0;
Alternative 1--Status Quo: Requirements (percent): 40;
Alternative 1--Status Quo: Benefit score: 2.8;
Alternative 2--Reach-back Capability: Requirements (percent): 90;
Alternative 2--Reach-back Capability: Benefit score: 6.3;
Alternative 3--Duplicate Systems: Requirements (percent): 90;
Alternative 3--Duplicate Systems: Benefit score: 6.3.
Benefit factors: Benefit score;
Alternative 1--Status Quo: Benefit score: 19.20;
Alternative 2--Reach-back Capability: Benefit score: 69.75;
Alternative 3--Duplicate Systems: Benefit score: 71.15.
Benefit factors: Total cost (in 2008 constant dollars);
Alternative 1--Status Quo: $20,011,111;
Alternative 2--Reach-back Capability: $71,762,643;
Alternative 3--Duplicate Systems: $137,038,661.
Benefit factors: Cost-benefit ratio;
Alternative 1--Status Quo: $1,042,245;
Alternative 2--Reach-back Capability: $1,028,855;
Alternative 3--Duplicate Systems: $1,926,053.
Source: DOD.
[End of table]
In examining how DOD scored benefits, we noted several concerns. First,
the benefit score for Alternative 1, Status Quo, is significantly lower
than the benefit score for the other two alternatives. Alternative 1's
benefit is 72 percent lower than that of Alternative 2, Reach-back
Capability, and 73 percent lower than that of Alternative 3, Duplicate
Systems. Second, only a slight change in the benefit scores would
change the cost-benefit score rankings of two of the three
alternatives. For example, if the benefit score for each of the nine
nonmonetary benefit factors for Alternative 1, Status Quo, were
increased by as little as 5 percent--a change that, in our opinion,
falls within the margin of imprecision for a subjective judgment--
Alternative 1 would become the preferred option (rather than
Alternative 2) based on its cost-benefit ratio (see table 3).
Table 3: Cost-Benefit Analysis with Revised Benefit Score for
Alternative 1 (2008 constant dollars):
Benefit score:
Alternative 1--Status Quo: 19.20;
Alternative 1--Status Quo, with a 5 percent higher benefit score:
20.16;
Alternative 2--Reach-back Capability: 69.75;
Alternative 3--Duplicate Systems: 71.15.
Total cost:
Alternative 1--Status Quo: $20,011,111;
Alternative 1--Status Quo, with a 5 percent higher benefit score:
$20,011,111; Alternative 2--Reach-back Capability: $71,762,643;
Alternative 3--Duplicate Systems: $137,038,661.
Cost-benefit ratio:
Alternative 1--Status Quo: $1,042,245;
Alternative 1--Status Quo, with a 5 percent higher benefit score:
$992,615;
Alternative 2--Reach-back Capability: $1,028,855;
Alternative 3--Duplicate Systems: $1,926,053.
Source: GAO analysis of DOD data.
[End of table]
The sensitivity of the benefit scores is important for three reasons.
First, benefits are predicated on the functional managers' subjective
estimates of relative importance and fulfillment of requirements.
Because managerial estimates are not objective measures--like dollars,
time, or distance--there is a degree of imprecision to the measurement.
Second, the preferred alternative--Alternative 2, Reach-back
Capability--was already known to the managers before the benefit
scoring was conducted. The extent to which this affected managers'
scoring, coupled with the lack of anonymity in the scoring, cannot be
determined. Third, the cost-benefit ratio between the preferred
solution--Alternative 2, Reach-back Capability--and Alternative 1,
Status Quo, differed by only 1.3 percent. According to OMB guidance,
[Footnote 12] a sensitivity analysis should have been performed and
reported to determine the cost-benefit ratio values' sensitivity to the
uncertainty of benefit scoring and the results of this analysis. There
is no indication that DOD performed a sensitivity analysis regarding
either the costs or the benefits; it is not mentioned in DOD's report
to Congress. In addition, based on the limited cost information in
DOD's report to Congress, it is unclear how sensitive DOD's cost
estimates are to different assumptions.
DOD's Report to Congress Does Not Provide a Detailed Risk Mitigation
Strategy; Does Not Include All Plans, Costs, or Schedule Estimates; and
Does Not Clearly Indicate Whether Upgrades Meet Necessary Requirements:
DOD's report to Congress does not provide a detailed mitigation
strategy for all of the security and vulnerability risks identified in
the Air Force's SEA. First, DOD's report to Congress understates the
security challenges at Peterson AFB. Second, as the SEA itself
acknowledges, the SEA did not analyze security risks associated with a
specific key capability at Peterson AFB, and thus the DOD report lacks
this information. Third, like the SEA, DOD's report does not address
the full spectrum of threats or hazards associated with Peterson AFB.
Fourth, DOD only summarizes the SEA recommendations in its report
rather than presenting a detailed discussion of the actions needed to
mitigate security vulnerabilities. Moreover, as DOD was still waiting
for a conceptual design to be submitted by Air Force Space Command that
would address certain recommendations, its report did not include all
plans, costs, or schedule estimates for these recommended actions.
Furthermore, although the missions moving from Cheyenne Mountain have
been designated as Protection Level-1, DOD's report does not state
whether all recommended measures will meet the necessary requirements
and, therefore, whether waivers and compensatory measures are needed to
begin operations at Peterson AFB.
DOD's Report Does Not Fully Detail Mitigation Strategies:
First, DOD's report to Congress understates the security challenges at
Peterson AFB. According to the Act, DOD's report to Congress must
include the findings and recommendations of an independent security and
vulnerability assessment of Peterson AFB and the Secretary of Defense's
plans for mitigating any security and vulnerability risks identified as
part of that assessment. DOD's report noted the existence of some
security issues, but not to the extent as is presented in the SEA.
Second, the SEA acknowledges that it did not analyze security risks
associated with a specific key capability that DOD classified.
Consequently, security risks associated with that capability were not
included in DOD's report to Congress--even though the SEA noted that
diverse redundancy with regard to this key capability was needed to
eliminate or mitigate single points of failure.
Third, DOD's report, like the SEA, does not address the full spectrum
of threats or all hazards, such as natural disasters. Our
aforementioned classified report contains information about the
specific threats that were excluded from the scope of the Air Force
Space Command's security assessment. According to the SEA, the
assessment team considered a wide range of threats that it culled from
Air Force policy documents, local Air Force Office of Special
Investigations reports, historical data, and previous studies. The SEA
states that although protecting soft targets from certain key types of
attacks would be very difficult and costly, the assessment team would
have to perform a new assessment to reflect a new threat, should the
threat change. Our classified report raised questions about how fully
the SEA addresses certain key threats and all hazards.
Finally, while DOD's report to Congress appears to address all of the
recommended security upgrades contained in the noncomprehensively
scoped SEA, those upgrades and their mitigation strategies are only
summarized. Moreover, some of the solutions have been submitted as
unfunded requests. DOD officials told us on May 29, 2008, that when
operations began at the new integrated command center shortly before
that date, DOD substituted some alternative measures to mitigate needed
upgrades that had not been performed. However, we cannot verify whether
these measures are adequate due to the limited scope of the SEA and the
parameters of our review.
DOD's Report Does Not Include All Plans, Costs, or Schedule Estimates;
and Does Not Clearly Indicate Whether Upgrades Meet Necessary
Requirements:
DOD's report to Congress lists security upgrades recommended in the SEA
that are awaiting final conceptual design. Consequently, DOD's report
did not include plans, costs, or schedule estimates for these upgrades.
Additional information provided by NORAD and USNORTHCOM indicates that
compensatory measures have been performed while awaiting final design
for these upgrades, and for funding to be secured. DOD recently
provided us with a copy of the December 2007 Sandia National
Laboratories study on which these conceptual design recommendations
were based. However, we have not analyzed the study to determine
whether all recommended security upgrades are being implemented or
whether waivers have been approved and compensatory measures put in
place. Security enhancements have a cumulative effect and, without a
detailed analysis, it is difficult to evaluate whether the planned
upgrades identified in DOD's report will achieve the desired level of
protection.
Further, it is unclear whether assets moving from Cheyenne Mountain to
Building 2 at Peterson AFB will be protected in accordance with Air
Force policy. As mentioned earlier, if NORAD and USNORTHCOM cannot meet
Protection Level-1 requirements for the integrated command center
because of resource or funding constraints, then NORAD and USNORTHCOM
will have to request waivers and develop compensatory measures.
However, neither DOD's report to Congress nor the SEA stated whether
all recommended security upgrades would enable Building 2 to meet
necessary requirements or whether waivers would be needed to begin
operations.
Final Plans for Relocation Do Not Take Security Issues into Account:
As required by the Act, DOD includes in its report a section on its
final plans for relocating the NORAD Command Center and related
functions. However, this section consists solely of a configuration
diagram of the new integrated command center and a time frame for when
it will commence operations. The section does not include any of the
security issues DOD identified in either its report or the SEA.
Conclusions:
DOD is proceeding with its plans to relocate the NORAD Command Center
and other functions from Cheyenne Mountain to Peterson AFB and,
according to DOD officials, operations at the combined command center
had begun by May 29, 2008. However, our review of DOD's report to
Congress showed that it did not recognize the uncertainty of benefit
scoring or include a sensitivity analysis, thus rendering its
comparison of alternatives subject to very different outcomes with only
slight changes to subjectively estimated benefit scores. Furthermore,
DOD's report did not include certain key threats, which we identified
in the classified version of this report, and it understated the
security issues surrounding the relocation, as detailed in the SEA.
Recommendations for Executive Action:
To help mitigate the security and vulnerability risks identified in,
and incorporate certain key threats excluded from, the Air Force Space
Command's security assessment, we recommend that the Secretary of
Defense, through the Joint Chiefs of Staff, direct the Commander of
NORAD and USNORTHCOM to take the following two actions:
* Reevaluate the full spectrum of security vulnerabilities associated
with moving the NORAD Command Center and related functions from
Cheyenne Mountain to Peterson AFB.
* Certify that he is fully aware of all the risks associated with
moving the NORAD Command Center and related functions from Cheyenne
Mountain to Peterson AFB, and accepts those risks.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD disagreed with our
first recommendation and neither agreed nor disagreed with our second
recommendation. DOD's comments are reprinted in their entirety in
enclosure I.
DOD disagreed with our recommendation that the Commander of NORAD and
USNORTHCOM reevaluate the full spectrum of security vulnerabilities
associated with moving the NORAD Command Center and related functions
from Cheyenne Mountain to Peterson AFB. In its comments, DOD stated
that the SEA focused on threats considered most likely to affect
Peterson AFB and Building 2 and stated that the Defense Threat
Reduction Agency will conduct a Balanced Survivability Assessment in
the fall of 2008 to further refine Headquarters NORAD and USNORTHCOM
security needs. DOD stated that it views a threat assessment covering
all possible threats encompassing both Cheyenne Mountain and Building 2
as unfocused, and that it has prioritized resources according to most
likely scenarios. However, although the SEA did develop a "threat
spectrum" that defined a range of potential threats to NORAD and
USNORTHCOM, DOD could not provide any documented evidence of having
performed a risk assessment that analyzed the most likely threats to
Peterson AFB and Building 2 in order to prioritize resources. DOD
states that should there be a credible threat to Peterson AFB, command
center functions could be transferred back to Cheyenne Mountain. We
continue to believe that DOD should document having performed a risk
assessment that analyzed the most likely threats to Peterson AFB and
Building 2, along with the basis for its assumed assessment of low
probability of certain key threats that are identified in the
classified version of this report.
DOD neither agreed nor disagreed with our recommendation that the
Commander of NORAD and USNORTHCOM certify that he is fully aware of all
the risks associated with moving the NORAD Command Center and related
functions from Cheyenne Mountain to Peterson AFB, and that he accepts
those risks. DOD stated in its comments that its report to Congress was
based on a security analysis completed in May 2007. DOD stated that to
date, NORAD and USNORTHCOM have implemented the measures necessary for
Building 2 to meet required security levels, and that those mitigation
items not approved for implementation either were covered in other
approved actions or have been waived pending implementation. DOD states
that through the waiver process, the Commander formally accepted the
outstanding actions and associated risks related to a secure facility.
DOD recently provided us with copies of three waivers; however, it was
unclear to us that the Commander had explicitly accepted the risks
posed by the full spectrum of threats or hazards. Without such added
insight into the risks accepted by the Commander, we continue to
believe that our recommendation has merit and that he should certify
that he accepts those risks.
Subsequent to DOD's letter containing the comments restated above, DOD
provided us with another letter on August 29, 2008, containing its
comments on our final classified report. In its additional comments,
the department stated that NORAD and USNORTHCOM are in the process of
implementing GAO's recommendation that the Commander of NORAD and
USNORTHCOM reevaluate the full spectrum of security vulnerabilities
associated with moving the NORAD Command Center and related functions
from Cheyenne Mountain to Peterson AFB. Specifically, DOD stated that a
new Director of Security has been appointed and is leading a Security
Tiger Team, which has partnered with Sandia National Laboratories to
evaluate all threats and vulnerabilities to the headquarters. Moreover,
DOD stated that in concert with other planned vulnerability
assessments, the Security Tiger Team is recommending actions to
mitigate vulnerabilities and the Commander is incrementally approving
changes to the security posture of the headquarters as a result of this
process. We have not verified or validated the information provided in
these additional comments. DOD's additional comments are reprinted in
their entirety in enclosure II.
We are sending copies of this report to other interested congressional
parties. We are also sending copies to the Secretary of Defense; the
Chairman, Joint Chiefs of Staff; the Secretary of the Air Force; and
the Commanders of NORAD/USNORTHCOM and USSTRATCOM. Copies will be made
available to others upon request. In addition, this report will be
available at no charge on our Web site at [hyperlink,
http://www.gao.gov/].
If you or your staff have any questions about this report, please
contact me at (202) 512-5431 or [email protected]. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made key contributions
to this report are listed in enclosure III.
Signed by:
Davi M. D'Agostino:
Director:
Defense Capabilities and Management:
Enclosures - 3:
List of Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan L. Hunter:
Ranking Member:
Committee on Armed Services:
House of Representatives:
Enclosure I: Comments from the Department of Defense:
Note: Page numbers in the draft report may differ from those in the
final report.
Unclassified:
GAO Report - Dated June 3, 2008:
GAO Code 351199/GA0-08-807C:
"Defense Infrastructure: NORAD and USNORTHCOM Need to Reevaluate the
Full Spectrum of Vulnerabilities Associated with Moving the NORAD
Command Center from Cheyenne Mountain to Peterson Air Force Base, and
to Acknowledge Acceptance of the Risks" (U):
Cleared: For Open Publication:
July 11, 2008:
Office of Security Review, Department of Defense:
Unclassified Department Of Defense Comments To The Recommendations (U):
(U) Recommendation 1: The GAO recommends that the Secretary of Defense,
through the Joint Chiefs of Staff, direct the CDR of North American
Aerospace Defense Command (NORAD) and United States Northern Command
(USNORTHCOM) to reevaluate the full spectrum of security
vulnerabilities associated with moving the NORAD Command Center and
related functions from Cheyenne Mountain to Peterson Air Force Base,
(Page 15/GAO Draft Report)
(U) DOD Response: DOD does not concur. The Security Effectiveness
Analysis (SEA) focused on threats considered most likely to impact
Peterson Air Force Base and Building 2. In addition, the Defense Threat
Reduction Agency will conduct a Balanced Survivability Assessment in
the fall of 2008, to include Networks and Information Integration to
further refine HQ NORAD and USNORTHCOM security needs. 'The concept
behind preparing for the most likely scenarios is to prioritize
resources accordingly. We believe we have done this. The command is
also still utilizing Cheyenne Mountain as an alternate command center
and can resume full capability of all functions.
(U) We acknowledge that there are threats from which Cheyenne Mountain
would provide better protection. However, we consider the probability
of this type of risk to be low and outweighed by the benefits provided
by the combined Command Center.
(U) Recommendation 2: The GAO recommends that the Secretary of Defense,
through the Joint Chiefs of Staff, direct the CDR of North American
Aerospace Defense Command (NORAD) and United States Northern Command
(USNORTHCOM) to certify that he is fully aware of all the risks
associated with moving the NORAD Command Center and related functions
from Cheyenne Mountain to Peterson Air Force Base, and accepts those
risks. (Page 15/GAO Draft Report)
(U) DOD Response: Our report to Congress in March 2008 was based on a
security analysis completed in May 2007. Since the completion of the
security analysis report, the Commander has taken appropriate and
additional steps in response to the analyses completed. The Commander
has formally accepted the outstanding actions and associated risks
related to a PL-1 facility, and understands that risk is an inherit
element in command and acknowledges that risk.
Enclosure (1)
[End of enclosure]
Enclosure II: Additional Comments from the Department of Defense on the
Final Classified Report:
Note: Page numbers in the draft report may differ from those in the
final report.
North American Aerospace Defense Command And United States Northern
Command:
Maj Gen John H. Bordelon, USAF:
Chief of Staff, NORAD and USNORTHCOM:
250 Vandenberg St., Ste 3804:
Peterson AFB CO 80914-3804:
August 29, 2008:
Ms. Davi M. D'Agostino:
Director, Defense Capabilities and Management:
U.S. Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548:
Dear Ms. D'Agostino:
This is the Department of Defense (DoD) response to the GAO final
report, 'Defense Infrastructure: NORAD and USNORTHCOM Need to
Reevaluate the Full Spectrum of Vulnerabilities Associated with Moving
the NORAD Command Center from Cheyenne Mountain to Peterson Air Force
Base, and to Acknowledge Acceptance of the Risks', July 1, 2008 (GAO
Code 351199/GAO-08-807RC). We acknowledge receipt of this report and
also acknowledge that our comments are included in the final report.
General Renuart has reviewed and concurs with the Department's comments
and we have included them as an attachment to this letter. Work
continues on the numerous security initiatives referred to in our
previous response, and the Commander of North American Aerospace
Defense Command and US Northern Command has provided detailed updates
and responses, in person, to Members of the House Armed Services
Committee.
Our point of contact is Commander Joel Paine, USN at
[email protected] or [email protected].
Sincerely,
Signed by:
John H. Bordelon:
Major General, USAF:
Attachment:
Department of Defense Comments to GAO Report (GAO code 351199/GA0-08-
807RC):
GAO Report - Dated JULY 1, 2008:
GAO Code 351199/GAO-08-807RC:
"Defense Infrastructure: NORAD and USNORTHCOM Need to Reevaluate the
Full Spectrum of Vulnerabilities Associated with Moving the NORAD
Command Center from Cheyenne Mountain to Peterson Air Force Base, and
to Acknowledge Acceptance of the Risks" (U):
Department Of Defense Comments To The Recommendations (U):
Recommendation 1: The GAO recommends that the Secretary of Defense,
through the Joint Chiefs of Staff, direct the Commander of North
American Aerospace Defense Command (NORAD) and United States Northern
Command (USNORTHCOM0 to reevaluate the full spectrum of security
vulnerabilities associated with moving the NORAD Command Center and
related functions from Cheyenne Mountain to Peterson Air Force Base.
(Page 15/GAO Draft Report)
DOD Response: NORAD and USNORTHCOM are in the process of accomplishing
this task. A new Director of Security has been appointed (at the
Colonel level) and is leading a Security Tiger Team to evaluate all
threats and vulnerabilities to the headquarters. The Tiger Team has
partnered with Sandia Labs in this process. In addition, we have
enlisted the support of the JSIVA team (8-12 Sep 2008) to evaluate our
methods to address identified vulnerabilities. The DTRA-led Balanced
Survivability Assessment team started an assessment of our command
centers in August 2008, with the physical evaluation scheduled to occur
29 Sep to 10 Oct 2008. In concert with our partners, the Security Tiger
Team is recommending actions to mitigate vulnerabilities and the
Commander is incrementally approving prudent changes to the security
posture of our headquarters as a result of this process.
Recommendation 2: The GAO recommends that the Secretary of Defense,
through the Joint Chiefs of Staff, direct the Commander of North
American Aerospace Defense Command (NORAD) and United States Northern
Command (USNORTHCOM) to certify that he is fully aware of all the risks
associated with moving the NORAD Command Center and related functions
from Cheyenne Mountain to Peterson Air Force Base, and accepts those
risks. (Page 15/GAO Draft Report)
DOD Response: Our report to Congress in March 2008 was based on a
security analysis completed in May 2007. Since the completion of the
security analysis report, the Commander has taken appropriate and
additional steps in response to the analyses completed. The Commander
has formally accepted the outstanding actions and associated risks
related to a PL-1 facility, and understands that risk is an inherent
element in command and acknowledges that risk.
[End of section]
Enclosure III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Davi M. D'Agostino, (202) 512-5431 or [email protected]:
Acknowledgments:
In addition to the contact named above, Mark A. Pross, Assistant
Director; Gregory A. Marchand; Charles W. Perdue; Marc J. Schwartz;
Kimberly C. Seay; and Cheryl A. Weissman made key contributions to this
report.
[End of section]
Footnotes:
[1] GAO, Defense Infrastructure: Full Costs and Security Implications
of Cheyenne Mountain Realignment Have Not Been Determined, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-803R] (Washington, D.C.: May
21, 2007).
[2] Pub. L. No. 110-181, � 361 (2008).
[3] NORAD/USNORTHCOM, Report to Congress on Relocation of North
American Aerospace Defense Command Center (Colorado Springs, Colo.:
January 2008).
[4] A fourth alternative--combine the command center at Cheyenne
Mountain--was deemed by DOD as infeasible.
[5] Air Force Space Command, Systems Effectiveness Assessment for
Headquarters North American Aerospace Defense Command and United States
Northern Command, Peterson Air Force Base (Colorado Springs, Colo.:
Oct. 11, 2007).
[6] GAO, Defense Infrastructure: NORAD and USNORTHCOM Need to
Reevaluate the Full Spectrum of Vulnerabilities Associated with Moving
the NORAD Command Center from Cheyenne Mountain to Peterson Air Force
Base, and to Acknowledge Acceptance of the Risks (Washington, D.C.:
July 1, 2008).
[7] Air Force Instruction 31-101, The Air Force Installation Security
Program, � 6.3.2 (Washington, D.C.: Mar. 1, 2003).
[8] U.S. Northern Command, The NORAD-USNORTHCOM Transformation Analysis
Report (Colorado Springs, Colo.: July 2006).
[9] OMB Circular A-94, Guidelines and Discount Rates for Benefit-Cost
Analysis of Federal Programs (Washington, D.C.: Oct. 29, 1992).
[10] Present value is taking into account the time value of money in
calculating the value of future costs.
[11] The discount rate is the interest rate used in present value
calculations.
[12] OMB Circular A-94.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: [email protected]:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, [email protected]:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, [email protected]:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
*** End of document. ***