Cooperative Threat Reduction: DOD Needs More Reliable Data to
Better Estimate the Cost and Schedule of the Shchuch'ye Facility
(31-MAY-06, GAO-06-692).
Until destroyed, Russia's stockpile of chemical weapons remains a
proliferation threat, vulnerable to theft and diversion. Since
1992, Congress has authorized the Department of Defense (DOD) to
provide more than $1 billion for the Cooperative Threat Reduction
(CTR) program to help the Russian Federation construct a chemical
weapons destruction facility (CWDF) at Shchuch'ye to eliminate
about 14 percent of its stockpile. Over the past several years,
DOD has faced numerous challenges that have increased the
estimated cost of the facility from about $750 million to more
than $1 billion and delayed the facility's operation from 2006
until 2009. DOD has attributed the increase cost and schedule to
a variety of factors. In this report, we (1) assess the
facility's progress, schedule, and cost and (2) review the status
of Russia's efforts to destroy all of its chemical weapons.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-692
ACCNO: A54980
TITLE: Cooperative Threat Reduction: DOD Needs More Reliable
Data to Better Estimate the Cost and Schedule of the Shchuch'ye
Facility
DATE: 05/31/2006
SUBJECT: Chemical weapons
Chemical weapons disposal
Construction costs
Cost analysis
Facility construction
Foreign governments
International cooperation
Schedule slippages
Security threats
Subcontractors
DOD Cooperative Threat Reduction Program
Shchuch'ye (Russia)
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-692
United States Government Accountability Office
Report to the Chairman, Subcommittee on
Federal Financial Management, Government Information, and International
Security, Committee on Homeland Security and Governmental Affairs, U.S. Senate
May 2006
COOPERATIVE THREAT REDUCTION
DOD Needs More Reliable Data to Better Estimate the Cost and Schedule of the
Shchuch'ye Facility
GAO-06-692
COOPERATIVE THREAT REDUCTION
DOD Needs More Reliable Data to Better Estimate the Cost and Schedule of
the Shchuch'ye Facility
What GAO Found
Although DOD has made visible progress over the past 2 years in
constructing the chemical weapons destruction facility at Shchuch'ye, it
continues to face numerous challenges that threaten the project's schedule
and cost. Primarily, key buildings on the site have fallen behind schedule
due to difficulties working with Russian subcontractors. Such delays have
been costing DOD more than $3 million per month since October 2005 and
will continue until the award of a crucial subcontract, possibly in June
2006. Uncertain progress of Russian construction on the site,
unpredictable Russian regulatory requirements, and various technical
issues, such as testing the facility, could cause further schedule delays
and increase costs. Also, DOD lacks a reliable earned value management
(EVM) system to record, predict, and monitor the project's progress. DOD
allocated $6.7 million to the project's contractor in September 2004 to
establish an EVM system and expected to have a validated EVM system in
place by March 2005. DOD cannot use the current EVM system to assess the
final schedule and cost for completing the Shchuch'ye facility because it
contains flawed and unreliable data. In addition, the contractor has not
yet conducted an IBR of the Shchuch'ye project.
Furthermore, it remains uncertain whether the Russian government can
destroy its entire chemical weapons stockpile by the Chemical Weapons
Convention (CWC) extended deadline of 2012. As of March 2006, Russia had
destroyed about 3 percent of its 40,000 metric tons of chemical weapons at
two completed destruction facilities. To eliminate the remainder of its
chemical weapons over the next six years, the Russian government must
construct and operate five additional destruction facilities, including
Shchuch'ye. The Russian government has indicated that it will need
continued international assistance to destroy the remaining stockpile.
United States Government Accountability Office
Contents
Letter 1
Results in Brief 2 Background 3 Construction Has Progressed, but Project
Is behind Schedule and
Faces Substantial Challenges 7 Russia Has Developed a Destruction Plan and
Increased Funding
but May Not Meet Its Destruction Deadlines 22 Conclusion 26
Recommendations for Executive Action 27 Agency Comments and Our Evaluation
27
Appendix I Scope and Methodology
Appendix II Lack of Reliable EVM Data Limits DOD's Ability to Estimate
Schedule and Cost for Constructing the CWDF 31
Appendix III Comments from the Department of Defense
Appendix IV GAO Contact and Staff Acknowledgments
Tables
Table 1: Schedule for Russian CWDFs 23 Table 2: International Assistance
for Russian Destruction, as of
April 2006 25 Table 3: Examples of Accounting System Errors Understating
the
Variance at Completion from January 2006 Report 32
Figures
Figure 1: Russian Chemical Weapons Stockpile 5 Figure 2: Completed
Firehouse at Shchuch'ye, November 2005 8 Figure 3:
Administration/Cafeteria Building under Construction, November 2005 9
Page i GAO-06-692 CTR Assistance to Shchuch'ye
Figure 4: Construction Progress on Building 101 in November 2003
and November 2005 10
Figure 5: U.S. Construction Status of Key Structures at Shchuch'ye,
March 2006 11
Figure 6: Status of Key CWDF Milestones, as of May 2006 13
Abbreviations
CTR Cooperative Threat Reduction
CWC Chemical Weapons Convention
CWDF chemical weapons destruction facility
DCAA Defense Contract Audit Agency
DCMA Defense Contract Management Agency
DOD Department of Defense
DTRA Defense Threat Reduction Agency
EVM Earned Value Management
FAR Federal Acquisition Regulation
IBR integrated baseline review
RFP Requests for Proposal
Rostekhnadzor Federal Service for Ecological,
Technological, and Nuclear Oversight
VAT value added tax
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office Washington, DC 20548
(2)(2) review the status of Russia's efforts to destroy all its
chemicalreview the status of Russia's efforts to destroy all its chemical
Page 1 GAO-06-692 CTR Assistance to Shchu
Results in Brief
government auditing standards. (See app. 1 for details on our scope and
methodology.)
Since our last visit to the site in November 2003, DOD and Parsons have
made progress in constructing the CWDF at Shchuch'ye. For example, several
buildings are at or near completion, including the fire station, housing
complex, and warehouse. However, DOD faces substantial challenges that
could threaten the project's cost and schedule. First, the construction of
key buildings is behind schedule. The construction of the main destruction
building is delayed due to subcontractor bids that were incomplete or
excessively high. The control building is behind schedule because a major
Russian subcontractor went bankrupt. As of February 2006, DOD estimated
that the construction of the entire CWDF was about 40 percent complete,
compared with the more than 52 percent scheduled for completion at that
time. Second, uncertain progress of Russian construction of utilities
(electricity, water, and gas) required to operate the facility could delay
the destruction process. Third, a new Russian regulatory agency has levied
additional and unplanned safety and administrative requirements on the
project. In addition, potential difficulties in implementing the next
critical step-systemization-in which all of the facility's components
(destruction, electrical, water, etc.) are tested to ensure
interoperability and performance-could impact cost and schedule. While DOD
estimates that it will turn over the Shchuch'ye facility to the Russian
government in December 2009, such an estimate appears optimistic given the
construction and other unknown delays DOD may encounter. Furthermore, the
EVM system that Parsons is using to record, predict, and monitor progress
contains flawed and unreliable data. Our analysis revealed serious
discrepancies in the data, such as improper calculations and accounting
errors. For example, we found that from September 2005 through January
2006 Parsons' EVM reports did not capture almost $29 million in actual
costs for the CWDF project. In addition, we found that DOD and Parsons
have not yet conducted an integrated baseline review (IBR) for the
Shchuch'ye project.
To improve DOD's efforts to accurately measure progress on the Shchuch'ye
project and estimate its final completion date and cost, we are
recommending that the Secretary of Defense ensure that Parsons' EVM system
contains valid, reliable data and that it reflects actual cost and
schedule conditions. Until Parsons' system produces reliable EVM data, we
are also recommending that the Secretary of Defense withhold a portion of
Parsons' award fee. Finally, we are recommending that the
Page 2 GAO-06-692 CTR Assistance to Shchuch'ye
Secretary of Defense require Parsons to perform an IBR of the Shchuch'ye
project once the contract for completing Building 101 has been awarded.
DOD concurred with our recommendation regarding the improvement of
Parsons' EVM system data and provided technical comments that we
incorporated where appropriate. The Department of State did not provide
comments.
In addition to the Shchuch'ye project, the Russian government has
ambitious plans to eliminate its chemical weapons stockpile. Since 2002,
Russia has destroyed about 3 percent of its declared 40,000 metric tons of
chemical weapons at two completed destruction facilities. To eliminate the
remaining stockpile and meet the Chemical Weapons Convention (CWC)
extended deadline of 2012, the Russian government will have to destroy
about 38,000 metric tons of chemical weapons. The Russian government's
destruction plan to eliminate all chemical weapons by 2012 may be
unrealistic as it depends on the construction of seven destruction
facilities-two have been built, two are under construction, and three have
not been started. Furthermore, the Russian government's priority is to
destroy nerve agents contained in large munitions, because destroying the
larger-sized munitions first would allow Russia to meet its CWC
destruction deadlines faster. Accordingly, the destruction of smaller
munitions at Shchuch'ye may become less of a priority for the Russian
government. In addition, the Russian government has indicated that it will
need continued international assistance to destroy the remaining
stockpile. Its destruction plan estimates that about $5.6 billion is
needed to eliminate the entire Russian stockpile. Since 2002,
international donors, including the United States, have committed almost
$2 billion for Russian chemical weapons destruction efforts.
Russia possesses the world's largest declared chemical weapons stockpile,
which is stored at seven sites across the country (see fig. 1). When
declared in 1998, the Russian stockpile included 32,500 metric tons of
nerve agents and 7,500 metric tons of blister agents.1 As of March 2006,
Russia had destroyed about 1,158 metric tons of blister agents, about 3
percent of its stockpile. Under the CWC, Russia must destroy all of its
1
Nerve agents affect the transmission of nerve impulses in the nervous
system. Nerve agents are easily dispersed and highly toxic when absorbed
through the skin or via respiration. Blister agents, which can be lethal
if inhaled, generally cause burns on contact with skin. The blister agents
include mustard gas and lewisite.
Page 3 GAO-06-692 CTR Assistance to Shchuch'ye
chemical weapons by the extended deadline of 2012.2 The CWC is a
multilateral arms control treaty that bans the development, production,
stockpiling, transfer, and use of chemical weapons and requires the
destruction of existing chemical weapons stocks. Until destroyed, chemical
weapons remain a proliferation threat.
The CWC requires the destruction of existing chemical weapons stocks and
production facilities by 2007 with a possible extension to 2012.
Page 4 GAO-06-692 CTR Assistance to Shchuch'ye
Figure 1: Russian Chemical Weapons Stockpile
Sources: DOD (data); Nova Development and Map Resources (images).
In 1992, the United States agreed to assist the Russian government in
eliminating its chemical weapons stockpile. The United States has
committed to fund the design, construction, equipment acquisition and
installation, systems integration, training, and start-up of the
Shchuch'ye facility. When completed, the facility will house about 100
buildings and structures, including the destruction buildings where
chemical munitions are destroyed; the administration building where the
destruction process is controlled; and support buildings such as the
boiler house, which provides heat to the entire facility. As originally
planned, the facility's construction was expected to begin in March 2001
and to be completed in 2005. However, a 2-year congressional freeze on
funding postponed the start of construction until March 2003.
DOD's Defense Threat Reduction Agency (DTRA) manages the implementation of
the CTR program. To construct the Shchuch'ye facility, DTRA--through the
U.S. Army Corps of Engineers, the contract manager for the project-has
contracted with Parsons, which in turn subcontracts the design and
construction work to Russian contractors. Contracts are executed, managed,
and reviewed in accordance with DOD and Federal Acquisition Regulations
(FAR). Subcontractors submit bids in response to Requests for Proposal
(RFP) issued by Parsons. Parsons then awards the subcontract on the basis
of safety records, past performance, quality of work, price, and other
factors. After awarding these contracts, Parsons works with the
subcontractors to conduct technical evaluations of the schedule and cost
of the work. CTR assistance will finance the construction of all buildings
and structures on site, except for one. The Russian Federation has agreed
to fund the construction of a second destruction building (Building 101A)
nearly identical to Building 101, the
U.S. funded destruction structure. Russia is also funding the construction
of utilities (gas, electricity, water) needed to operate the facility and
to support the local community.
Since 1992, Congress has passed 27 laws addressing the CTR program.3 The
legislation includes various DOD requirements for CTR funding, conditions
on CTR expenditures, and mandates to report on the implementation of the
CTR program. Some legislative provisions apply to the entire CTR program;
others are directed at the Shchuch'ye project, including a requirement for
a presidential certification that the project is in the U.S. national
security interest.4 The President's certification authority and the waiver
of a prior prohibition on funding chemical weapons
3
For a summary of the legislation, see appendix II of GAO, Cooperative
Threat Reduction: DOD Has Improved Its Management and Internal Controls,
but Challenges Remain, GA0-05-329 (Washington, D.C.: June 30, 2005). Since
publication of that report, Congress has passed two laws containing
measures addressing CTR. Pub. L. 109-103, Div A (The National Defense
Authorization Act for Fiscal Year 2006) and Pub. L. 139-148, Div A (The
Department of Defense Appropriations Act of 2006).
4
Pub. L. 108-375, sec. 1303 (amending Pub. L. 108-136, sec. 1306).
Construction Has Progressed, but Project Is behind Schedule and Faces
Substantial Challenges
destruction in Russia expire on December 31, 2006.5 In addition, Congress
has conditioned funding for the Shchuch'ye facility on the Secretary of
Defense's certification that, among other conditions, Russia has allocated
at least $25 million to eliminating its chemical weapons and has developed
a practical plan for destroying its chemical weapons stockpile.6
Since our last visit to the Shchuch'ye site in 2003, we found that Parsons
and DOD had made progress in constructing the facility. Several support
buildings such as the fire station, worker housing, and warehouse had been
completed; and many of the other structures, including the
administration/cafeteria building, the processing building, and storage
buildings were well under construction. However, key buildings had fallen
behind schedule, affecting the facility's overall cost and schedule.
Uncertain progress of Russian construction at the facility and on its
infrastructure, an unpredictable Russian operating environment, and
assorted technical issues could continue to impact the project's cost and
schedule. Furthermore, the failure of Parsons to develop and implement a
usable EVM system has limited DOD's efforts to oversee project schedule
and cost.
DOD Has Made Progress but Is Experiencing Delays
During our visit to the Shchuch'ye site in November 2005, we observed
substantial construction progress compared with our visit in November
2003. In 2003, the site consisted mainly of concrete foundations for the
destruction buildings, with only the specialist camp7 and warehouse under
construction. By 2005, however, the support structures of many buildings
had been built, and several buildings were at or near completion,
including
5
See Pub. L. 106-65, sec. 1305.
6
See Pub. L. 107-107, sec. 1308. The Secretary of Defense must certify that
there has been
(1) information provided by Russia, that the United States assesses to be
full and accurate, regarding the size of the chemical weapons stockpile of
Russia; (2) a demonstrated annual commitment by Russia to allocate at
least $25 million to chemical weapons elimination; (3) development by
Russia of a practical plan for destroying its stockpile of nerve agents;
(4) enactment of a law by Russia that provides for the elimination of all
nerve agents at a single site; (5) an agreement by Russia to destroy or
convert its chemical weapons production facilities at Volgograd and
Novocheboksarsk; and (6) a demonstrated commitment from the international
community to fund and build infrastructure needed to support and operate
the facility.
7
The specialist camp is the building to house contractors working on site.
the specialist camp, warehouse, gas rescue station,8 and fire station.
(Fig. 2 shows the completed fire station.)
Figure 2: Completed Firehouse at Shchuch'ye, November 2005
Source: GAO.
Also under construction were the boiler house and the
administration/cafeteria building, seen in figure 3.
The gas rescue station will serve as a training center and equipment depot
for dealing with hazardous materials on site.
Page 8 GAO-06-692 CTR Assistance to Shchuch'ye
Figure 3: Administration/Cafeteria Building under Construction, November 2005
Source: GAO.
The concrete outer shells of Building 101 and the administration/control
building had been completed. While Building 101 was still open to the
elements and contained no inner walls, Russian subcontractors were
installing outlets and control panels inside the drywall of the
administration building. (See fig. 4 for a comparison of the construction
work completed on Building 101 in November 2003 and November 2005.) We
also observed piping and wiring being installed above ground for site wide
electrical, heat, and water utilities.
Figure 4: Construction Progress on Building 101 in November 2003 (left) and
November 2005 (right)
Source: GAO.
Despite such progress, the CWDF project has not met scheduled milestones,
primarily because of a delay in awarding the contract for the completion
of the CTR-funded destruction building (Building 101), pictured in figure
4. In January 2005, DOD estimated that the CWDF would cost $1.039 billion
and be transferred to the Russian Federation by July 2009. However, in
March 2006, DOD officials stated that they were unable to estimate when
the entire facility will be completed and at what cost until they award a
contract for the completion of Building 101. As of February 2006, DOD
estimated that the construction of the entire CWDF was about 40 percent
complete, compared with the more than 52 percent scheduled for completion
at that time. As indicated in figure 5, the construction of certain key
structures is behind schedule, including the destruction building
(Building 101), the control building (administration building), the boiler
house, and the water circulation building.
Figure 5: U.S. Construction Status of Key Structures at Shchuch'ye, March
2006
Source: DOD.
a
Building schedule modified by Parsons to reflect longer timelines.
Building 101 is on the "critical path", that is, delays in finishing the
building will prolong construction on other parts of the Shchuch'ye
facility. Although the exterior shell of Building 101 is on schedule, the
award of the construction contract for the remainder of Building 101 is
behind
Page 11 GAO-06-692 CTR Assistance to Shchuch'ye
schedule. Parsons had planned to award the subcontract for the balance of
the building in June 2005, but it may not be awarded until summer 2006.
Since October 2005, Parsons has incurred costs for personnel salaries,
rent, and transportation of more than $3 million per month, which will
continue until the subcontract is awarded. Where possible, Parsons has
reduced or delayed recruitment of personnel planned for management of
Building 101. Construction activity is still ongoing at other buildings
throughout the site.
The delay in awarding the contract for the remainder of Building 101 has
impacted the overall schedule for completing the facility's construction.
As part of its program management, DOD estimates dates for key project
milestones at Shchuch'ye. These include a milestone schedule with
objective (ideal) completion dates, threshold (latest acceptable) dates,
and estimated completion dates for key activities. As of May 2006,
however, DOD does not expect to meet key milestone dates for the CWDF.
According to this schedule (as shown in fig. 6), construction of the
facility will be delayed by about 1 year, testing using simulated nerve
agent will begin some 15 months later than planned, and live agent
demonstration will be delayed by about 8 months. While DOD estimates that
it will turn over the Shchuch'ye facility to the Russian government in
December 2009, such an estimate appears optimistic given the construction
and other unknown delays that DOD may encounter in testing the facility
with simulated and live nerve agent. DOD officials stated that these
milestones may slip even further.
Figure 6: Status of Key CWDF Milestones, as of May 2006
Shchuch'ye CWDF Delayed Due to Difficulties Working with Russian
Subcontractors
Source: DOD.
The delays in constructing key buildings at the CWDF result from problems
Parsons and DOD have had with Russian subcontractors, including the
bankruptcy of one major subcontractor, problems in soliciting adequate
bids, and difficulty maintaining a competitive-bidding process.
First, the 2005 bankruptcy of the Russian construction subcontractor
Magnitostroy delayed construction of key buildings. This company was cited
during the initial source selection process during 2000 to 2001 for its
technical abilities, logistical capability, competitive pricing, and
financial responsibility and was the first construction subcontractor to
work on the Shchuch'ye project. According to DOD and Parsons officials,
Magnitostroy enjoyed the strong support of the Russian government.
However, it was discovered in 2005 that a senior executive embezzled
millions of dollars from the company in 2003. As a result, the company was
unable to afford sufficient labor to complete its work at the site,
according to DOD and Parsons officials. The most serious delay involved
the construction of the administration building-the command building that
will control the
Page 13 GAO-06-692 CTR Assistance to Shchuch'ye
destruction process. Although scheduled to be complete at the time of our
visit in November 2005, construction of the administration building was
only about 36 percent complete. By January 2006, Parsons had assumed
direct responsibility for the construction of the building and had divided
most of the remaining work among Magnitostroy's subcontractors. Similarly
at that time, two other Magnitostroy buildings were behind schedule,
requiring Parsons to extend their completion dates. Given these delays,
Parsons has not provided Magnitostroy with RFPs on any new construction
packages.
Second, DOD and Parsons officials stated that Russian subcontractors had
not provided detailed cost and scheduling information in their bids.
Although Parsons cited incomplete bids as the cause of the delay, DOD
criticized Parsons for a "lack of urgency" in resolving the Building 101
bid issue. Parsons had particular difficulty soliciting adequate bids on
the construction package for the work remaining on Building 101.9 This
construction package will complete the building's physical structure and
install the equipment and processing systems needed to destroy the
chemical munitions. According to DOD and Parsons officials, it is the
largest, most complex construction package of the CWDF project. After
Magnitostroy's bankruptcy, two other contractors, Spetztroy and
Stroytransgaz, bid on the remaining Building 101 construction package.
According to DOD officials, their bids arrived after the June 2005
deadline and did not include adequate cost and schedule data. Despite a
deadline extension, neither subcontractor submitted a complete bid until
the end of December 2005. At that time, only Spetzstroy submitted a
responsive bid. Its bid price, however, was $239 million over DOD's
budget.
Third, the small pool of approved Russian subcontractors has made it
difficult to maintain a competitive-bidding process. According to DOD, the
subcontractors for the CWDF are selected through a series of joint
selection committees. The Russian government develops a list of approved
companies that Parsons and a joint commission comprising DOD and Russian
government officials examine. In the initial round of subcontractor
selections in 2000 to 2001, Magnitostroy was the first CWDF subcontractor
chosen. A second round of selections in 2003 added four more
subcontractors: Promstroy, Spetztroy, Stroyprogress, and Stroytransgaz.
According to DOD officials, before Magnitostroy's 2005
Construction packages are "mini contracts" for completing specific tasks
associated with buildings and infrastructure that are awarded to
subcontractors on a competitive basis.
Page 14 GAO-06-692 CTR Assistance to Shchuch'ye
Cost and Schedule Subject to Uncertain Progress of Russian Construction, an
Unpredictable Russian Operating Environment, and System Testing Issues
Uncertain Russian Progress in Completing Facility and Infrastructure
bankruptcy, Magnitostroy, Stroytransgaz, and Spetztroy were the only
subcontractors that were capable of completing larger construction
efforts. The small number of Russian contractors discouraged effective
competition and limited the number of construction packages that could be
awarded.
In March 2005, DOD requested that the Russian Federation expand the
subcontractor pool to ensure completion of the Shchuch'ye facility on time
and within budget. The Russian government added one small specialty
subcontractor, Vneshstrojimport, but did not restart the selection process
to find a replacement for Magnitostroy. In December 2005, Stroytransgaz
withdrew from competition, and the sole remaining contractor, Spetztroy,
submitted a bid for $310 million to complete Building 101. However, DOD
had budgeted only $71 million for the construction package. To reconcile
the cost difference, DOD paid for an independent cost analysis that
validated the original Parsons estimate of $56 million. Parsons and DOD
also sought the assistance of the Russian government to negotiate with
Spetzstroy to lower its bid. When negotiations failed to produce a
compromise, Parsons canceled the RFP for the balance of Building 101 on
March 2, 2006.
In March 2006, DOD resubmitted a request for more subcontractors and
provided the Russian government with a list of five potential companies,
three of which were added to the pool. In April 2006, Parsons issued a new
RFP for the remainder of Building 101. According to DOD officials, Parsons
has and will continue to conduct weekly meetings with the bidders and make
personnel available for questions and clarifications regarding the RFP.
The cost and schedule of the Shchuch'ye facility are subject to continuous
risks. The Russian Federation's uncertain progress in completing work on
Building 101A and required utilities could delay the final system testing
for the CWDF. The Russian government's failure to complete promised social
infrastructure could generate local opposition to the CWDF. DOD and
Parsons must also operate in an unpredictable Russian environment with
changing legal and technical requirements that could directly affect
schedule and cost.
Russian Federation progress in completing Building 101A, as well as the
industrial and social infrastructure surrounding the CWDF, remains
uncertain. According to DOD officials, the Russian government's method of
construction scheduling contains few itemized tasks, making it difficult
Russia's Operating Environment Is Unpredictable
to accurately gauge construction progress and uncover issues that could
cause delays. Although DOD and Parsons monitor Russian Federation
construction progress through monthly progress reports and project site
visits, the Russian government has not always followed jointly agreed upon
schedules. DOD and Parsons officials remain concerned that systemization
timelines could be affected if both destruction buildings are not
completed at the same time. Furthermore, Russian progress in constructing
utilities for the CWDF and the local community has produced mixed results.
For instance, we observed that the Russian government has installed only
one of three power lines needed to operate the CWDF. According to Parsons
and DOD officials, although the Russian government completed the new water
line to the CWDF and the town of Shchuch'ye in 2004, the more water the
CWDF uses, the less the town has available. This may lead to a competition
for water when the facility begins consuming substantially more water when
testing of the facility's systems begins, and during operation.
Furthermore, when the Russian government constructed a new gas line to the
CWDF and through the town of Shchuch'ye, it did not connect the line to
local homes as promised. A local Shchuch'ye official stated that most
local residents cannot afford to pay for connection to the main gas line
and expressed concerns that the Russian government will not fulfill its
obligations to the local population. To allay public concerns that may
impact the CWDF, DOD uses public outreach offices to conduct opinion polls
and educate the local populace on the CWDF.10
DOD and Parsons must contend with an unpredictable Russian business
environment that can affect cost and schedule through unexpected changes
in Russian legal, technical, and administrative requirements.11 New
regulatory requirements have impacted the CWDF; in one case, stopping work
on a building until it could be redesigned to comply with new Russian
electrical codes. In November 2005, a new Russian regulatory agency--the
Federal Service for Ecological, Technological and Nuclear Oversight
(Rostekhnadzor)--performed a surprise audit at the Shchuch'ye CWDF. The
agency cited Parsons with noncompliance in several areas, including
environmental and industrial safety reviews, permits, licenses, and
certifications. While Parsons and DOD officials were not aware of these
requirements, they agreed to implement corrective
10
The public outreach offices are located in Shchuch'ye, Chelyabinsk, and
Kurgan.
11
According to its agreement with DOD, the Russian government must identify
and obtain all legal permits, licenses, and certifications required to
design, construct, equip, commission, and operate the Shchuch'ye CWDF.
Page 16 GAO-06-692 CTR Assistance to Shchuch'ye
Testing Issues Could Delay Destruction Process
actions. As of March 2006, Parsons had resolved 82 percent of the
Rostekhnadzor audit findings and was working to mitigate the remainder.
DOD continues to negotiate with Rostekhnadzor to meet the requirements of
Russian law and is working with the Russian government to identify
feasible solutions. Additionally, Parsons has contracted with consultants
that specialize in helping companies conform to Russian fire, ecological,
and industrial safety regulations at the local and national levels.
Furthermore, DOD and Parsons must review new technical requirements raised
by Russian government officials. According to DOD officials, some new
requirement requests are justified as they relate to the operation of the
CWDF, while most others are attempts to transfer cost and risk from Russia
to the United States. For example, as a result of code and space
deficiencies, DOD accepted the Russian requirement for an additional
laboratory building on site, construction of which will increase the
project's cost by an additional $12 million. However, DOD officials have
resisted approving Russian requests that they believe are unnecessary or
that fall within Russian responsibilities at the site. DOD refused to
allow the Russian government to incorporate a new machine into the
destruction process, which would have required significant redesign and
testing of the process, and led to schedule delays and increased project
costs.
Russian requirements for long-term visas and value added tax (VAT)
exemptions for equipment have affected cost and schedule. The Russian
government provides most DOD and Parsons personnel with only 6-month
visas, requiring workers to temporarily leave the country while their
visas are reissued. One DOD official estimated that transportation costs
associated with this practice totaled approximately $3 million as of
November 2005. However, DOD officials have noticed improvement in how
quickly the Russian Federation processes visas. In addition, when the
Russian government reorganized in early 2004, the office in charge of
Russian customs was dissolved, leaving no agency able to approve the VAT
exemptions for more than 6 months. During that time, all equipment shipped
from the United States was halted, causing a 3-month slip in the CWDF
construction schedule. In late 2004, the Russian Federation eventually
reestablished a new VAT office, and equipment delivery was resumed. Since
that time, DOD has encountered no VAT-related delays.
Issues associated with the testing of the CWDF's utilities and automated
destruction system (systemization) could further delay the schedule and
increase costs. DOD officials identified systemization of the CWDF as the
next major challenge after resolving the bid issue for Building 101.
Systemization consists of a series of tests to ensure the safety,
function, and interoperability of the CWDF internal systems-i.e., water,
gas, electric, heat, and the chemical munitions destruction process. Such
testing could be delayed if either destruction building (101 or 101A) or
essential utilities are not completed on time. The automated destruction
process is complex, involving the drilling, draining, and decontamination
of various sizes and types of munitions, and the neutralization of the
nerve agent they contain. Ensuring that this system works and interfaces
properly with the rest of the facility will require the testing and
calibrating of roughly 1,000 different processes, according to a DOD
official. DOD officials noted that U.S. experiences with destroying
chemical weapons found that systemization often encounters difficulties
and delays and has the potential to increase costs. Furthermore, DOD and
Parsons must compete the systemization contract between two Russian
subcontractors, Redkino and Giprosintez, selected by the Russian
government. Given previous difficulties working with subcontractors,
Parsons may experience delays in obtaining adequate and reasonably priced
bids.
DOD is attempting to mitigate systemization risk by exploring options to
test the CWDF's systems using Russian rather than U.S. methods. Although
the Shchuch'ye facility is a Russian design, it is currently planned to
undergo testing procedures similar to those DOD uses in the United States.
According to DOD officials, Russian systemization methods are less
involved than U.S. processes, which must adhere to stringent environmental
and operating regulations and can take 16 to 18 months to complete. The
Russian government, however, systemized its CWDF at Kambarka within 6 to 9
months. While DOD officials caution that each CWDF is unique, given the
types of munitions to be destroyed, they have begun exploring whether
Russian methods may allow for streamlining and compression of the
systemization schedule at Shchuch'ye, while still maintaining acceptable
safety levels. Parsons and its subcontractors are also testing the
automated destruction system equipment before it is installed in Building
101.
EVM System Has Not Been Effectively Implemented and Contains Flawed Data
Parsons' EVM System Is Not Yet Useful to DOD Managers
DOD policy and guidance12 require the use of EVM to measure program
performance. EVM uses contractor reported data to provide program managers
and others with timely information on a contractor's ability to perform
work within estimated cost and schedule. It does so by examining variances
reported in contractor performance reports between actual cost and time of
performing work tasks and the budgeted or estimated cost and time. In
September 2004, DOD modified its contract with Parsons, allocating about
$6.7 million and requiring the company to apply EVM to the Shchuch'ye
project. Parsons was expected to have a validated EVM system by March
2005.13 As of April 2006, Parsons had not developed an EVM system that
provided useful and accurate data to CWDF program managers. In addition,
our analysis found that the project's EVM data are unreliable and
inaccurate. According to DOD officials, these problems stem in part from
Parsons' outdated accounting system. EVM guidance states that
surveillance14 of an EVM system should occur over the life of the
contract. DOD has not yet conducted an IBR15 for the Shchuch'ye project
and does not plan to do so until after Parsons awards the subcontract to
complete Building 101, possibly in June 2006.
In December 2005 a Parsons' self-evaluation16 stated that the EVM system
for the CWDF was "fully implemented." In contrast, DOD characterized
Parsons' EVM implementation as a "management failure," citing a lack of
experienced and qualified Parsons staff. DOD withheld approximately
12
Defense Contract Management Agency, Department of Defense Earned Value
Management Implementation Guide, (Washington, D.C.: Apr. 7, 2005). See
also DOD Memorandum: Revision to DOD Earned Value Management Policy,
(Washington, D.C.: Mar. 7, 2005).
13
According to DOD, a validated EVM system was not required at the time the
Parsons contract was awarded. A modified EVM system, implemented in
September 1998, was maintained and used until December 2004.
14
Surveillance is the process of reviewing the health of the EVM system
process. The purpose of surveillance is to focus on using an EVM system
effectively to manage cost, schedule, and technical performance. An
effective surveillance process ensures that the key elements of the
process are maintained over time.
15An IBR verifies the technical content of the baseline. It also ensures
that contractor personnel understand and have been adequately trained to
collect EVM data. The review also verifies the accuracy of the related
budget and schedule to ensure that risks have been properly identified,
and it also assesses whether the contractor meets the program's
objectives.
16
Award fee contracts allow government agencies to adjust the amount of fee
paid to contractors based on the contractor's performance. Parsons is
required to do a self assessment of its performance at the end of each
award fee evaluation period.
Page 19 GAO-06-692 CTR Assistance to Shchuch'ye
Parsons' EVM Data Are Inaccurate and Unreliable
$162,000 of Parsons' award fee due to concerns over the EVM system. In
March 2006, DOD officials stated that at that point in implementation, EVM
was not yet a usable tool in managing the Shchuch'ye project. DOD
officials stated that Parsons needs to demonstrate that it incorporates
EVM into project management rather than simply fulfilling a contractual
requirement. DOD expects Parsons to use EVM to estimate cost and schedule
impacts and their causes and, most importantly, to help eliminate or
mitigate identified risks.
Parsons' EVM staff stated that they underestimated the effort needed to
incorporate EVM data into the system, train staff, and develop EVM
procedures. Parsons officials were also surprised by the number of
manhours required to accomplish these tasks, citing a high level of staff
turnover as contributing to the problem. According to the officials,
working in a remote and isolated area caused many of the non-Russian
employees to leave the program rather than extend beyond their initial
tour of duty.
Based on our review of Parsons' monthly EVM data for September 2005
through January 2006, we found that the data are inaccurate and unreliable
and that Parsons is exercising poor quality control over its EVM data.
Specifically, we discovered numerous instances where data were not added
properly for scheduled work; Parsons' EVM reports, therefore, did not
accurately capture data needed by project management to make informed
decisions about the Shchuch'ye facility. For example, we found that from
September 2005 through January 2006, Parsons' EVM reports contained
addition errors that did not capture almost $29 million in actual costs
for the CWDF project. Such cost omissions and other errors may cause DOD
and Parsons project officials to overestimate the amount of project
funding available.
Moreover, we found several instances where the accounting data were not
allocated to the correct cost accounts, causing large cost over-runs and
under-runs. This problem occurred because the accounting data were placed
in the wrong account or Parsons' accounting system was unable to track
costs at all levels of detail within EVM. A Parsons official stated that
the company was taking measures to identify these inaccuracies and
allocate the accounting data to the proper cost accounts. These problems,
however, have led to numerous accounting errors in the EVM reports. Such
mistakes underestimate the true cost of the CWDF project by ignoring cost
variances that have already occurred. Cost variances compare the earned
value of the completed work with the actual cost of the work performed.
Until Parsons fixes its accounting system, manual
Page 20 GAO-06-692 CTR Assistance to Shchuch'ye
adjustments will have to be made monthly to ensure that costs are properly
aligned with the correct budget. Such continuous adjustments mean that the
system is consistently reflecting an inaccurate status of the project for
Parsons and DOD managers. (For specific examples of our findings regarding
Parsons' EVM data, see app. II.)
EVM guidance states that surveillance of an EVM system should occur over
the life of the contract to guarantee the validity of the performance data
provided to the U.S. government. Initial surveillance activities involve
performing an IBR of a project within 6 months of awarding a contract and
as needed throughout the life of a project. DOD and Parsons have not yet
conducted an IBR for the Shchuch'ye project. Program managers are expected
to use EVM reports that have been validated by an IBR. Without verifying
the baseline, monthly EVM reporting, which tracks project work against a
set budget and schedule, is neither meaningful nor valid. Parsons and DOD
officials explained that while an IBR has been discussed, one will not be
conducted until Parsons awards a contract for completing Building 101. DOD
officials estimate that the award process for this contract may not be
completed until summer 2006, approximately a year later than planned.
According to Parsons, as of January 2006, about $66 million of scheduled
work has not been completed as planned, due to the delay in awarding the
subcontract for the balance of Building 101. DOD officials stated that
while they recognize the importance of conducting surveillance over an EVM
system, they currently are focused on the immediate need of establishing a
usable EVM system on which to perform surveillance.
Furthermore, DOD requires all EVM systems to undergo a compliance audit or
"validation" conducted by the Defense Contract Management Agency (DCMA)
with assistance from the Defense Contract Audit Agency (DCAA). DCAA found
that Parsons' accounting process was inadequate. A DCAA official on the
validation team stated that Parsons is relying on an outdated accounting
system that has difficulty capturing actual costs for the Shchuch'ye
project and placing them into appropriate cost categories. The DCAA
official stated that Parsons management should have discovered such
accounting errors before the EVM report was released to DOD. DCAA
therefore questioned whether Parsons can generate correct accounting data
and recommended that Parsons update its accounting system. As of April
2006, DCMA and DCAA had not yet validated Parsons' EVM system. (For more
information regarding DCMA and DCAA's assessments of Parsons' EVM system
see app. II.)
Page 21 GAO-06-692 CTR Assistance to Shchuch'ye
Russia Has Developed a Destruction Plan and Increased Funding but May Not Meet
Its Destruction Deadlines
Since our report in March 2004,17 the Russian government has approved a
plan to destroy its chemical weapons stockpile and has begun financing
significantly more of its own destruction activities. However, as of April
2006, the Russian government's progress in destroying its chemical weapons
stockpile has been limited, and the Russian government's destruction plan
may be overly ambitious and reliant on international assistance.
Russia Has Developed a Destruction Plan That May Prove Overly Ambitious
We reported in early 2004 that Russia's lack of a credible chemical
weapons destruction (CWD) plan had hindered destruction activities.
However, in October 2005, the Russian government approved a plan for
destroying its entire chemical weapons stockpile by the CWC-established
deadline of 2012.18 The October 2005 plan calls for using seven
destruction facilities to eliminate the entire chemical weapons stockpile.
Destruction of the chemical weapons stockpile at Gorniy was completed in
December 2005. As of March 2006, only the facility at Kambarka is
operational. The plan outlines the construction of the remaining five
sites, including Shchuch'ye, where nerve agent is to be eliminated.
According to the Russian plan, the blister agents stored at Gorniy and
Kambarka were to be destroyed first. In December 2005, the Russian
government completed its destruction efforts at Gorniy and began
destroying chemical weapons at Kambarka. In accordance with the plan,
destruction will next be focused on nerve agents. The storage sites near
Leonidovka, Maradykovskiy, and Pochep house large nerve-agent munitions,
while those near Shchuch'ye and Kizner store smaller munitions. Table 1
depicts the schedule for Russian chemical weapons destruction facilities.
17GAO, Nonproliferation: Delays in Implementing the Chemical Weapons
Convention Raise Concerns About Proliferation, GAO-04-361 (Washington,
D.C.: Mar. 31, 2004).
18
The Government of the Russian Federation Resolution, No. 639 (Oct. 24,
2005). On Amendments to the Federal Target-Oriented Program, "Chemical
Weapons Stockpiles Destruction in the Russian Federation."
Page 22 GAO-06-692 CTR Assistance to Shchuch'ye
Table 1: Schedule for Russian CWDFs
Actual or estimated Amount of
Actual or completion Quantity of declared
date for agent
Russian estimated destroying Type of to be stockpile
destruction date of chemical destroyed
facility site operation weapons weapons (metric (metric
tons) tons)
Gorniy December December 2005 Bulk 0 1,120
2002 blister
Kambarka December 2010 Bulk 6,347 6,360
2005 blister
Maradykovskiy 2006 2012 Large nerve 6,960 6,960
munitions
Shchuch'ye 2008 2012 Small nerve 5,440 5,440
munitions
Leonidovka 2008 2012 Large nerve 6,960 6,960
munitions
Pochep 2008 2012 Large nerve 7,520 7,520
munitions
Kizner 2009 2012 Small nerve 5,640 5,640
munitions
Sources: GAO analysis of Russian government and DOD data.
While the Russian plan indicates that the CWDF at Shchuch'ye will be
operational by 2008, DOD estimates that the facility may not be
operational until 2009. Furthermore, the Russian government's priority is
to destroy nerve agents contained in large munitions, because destroying
the larger-sized munitions first would allow Russia to meet its CWC
destruction deadlines faster. Accordingly, the destruction of smaller
munitions at Shchuch'ye19 may become less of a priority for the Russian
government.
However, the Russian government's destruction plan to eliminate all
chemical weapons by 2012 may be unrealistic. It depends on the
construction of seven facilities, but only two have been built, two are
under construction, and three have not been started. Although the CWDF at
Maradykovskiy may be operational in mid-2006, the Shchuch'ye facility is
still under construction, and only minimal work has begun at the three
remaining sites of Kizner, Leonidovka, and Pochep. According to its CWC
destruction schedule, Russia must eliminate 20 percent of its chemical
weapons stockpile by April 2007. As of March 2006, it had eliminated about
3 percent at Gorniy and Kambarka. Between April 2007 and April
The nerve agents to be destroyed at the Shchuch'ye CWDF are stored at the
Planovy arsenal located about 10 miles away.
Page 23 GAO-06-692 CTR Assistance to Shchuch'ye
Russia Has Significantly Increased Funding, but Destruction Efforts Need
International Assistance
2012, Russia must eliminate the remainder of its chemical weapons
stockpile (about 80 percent) at five destruction facilities that have yet
to be completed. It will be extremely difficult for the Russian government
to complete and operate the last three facilities by its proposed schedule
and meet its CWC commitment to destroy all stockpiles at these sites by
the extended deadline of April 2012.
Similarly, as of April 2006, DOD announced that the United States will not
be able to meet the CWC extended destruction deadline of 2012. DOD
estimates indicate that about 66 percent of the U.S. declared chemical
weapons stockpile will be destroyed by April 2012. As of March 2006, the
United States had destroyed about 36 percent of its declared stockpile. In
the United States, DOD had five operating chemical weapons destruction
facilities as of March 2006, and two additional facilities were being
designed.20
According to the Russian destruction plan, the estimated cost for
eliminating the entire Russian chemical weapons stockpile is more than 160
billion rubles-about $5.6 billion.
Over the past 6 years, Russia has substantially increased its annual
funding for its chemical weapons destruction efforts. In 2000, the Russian
government spent about $16 million for chemical weapons destruction. By
2005, it had spent almost $400 million. For 2006, the Russian government
plans to spend more than $640 million. For chemical weapons elimination at
Shchuch'ye, the Russian government has budgeted about $144 million since
fiscal year 2000. Russian funding at the site supports construction of one
of the two destruction buildings (Building 101A), as well as the
industrial and social infrastructure (utilities, roads, schools, etc.)
needed to support the facility's operations.
The Russian government will need continued international assistance to
complete destruction of its chemical weapons stockpile. The United States,
Canada, Germany, Italy, United Kingdom, and other donors have committed
almost $2 billion in assistance, with the United States
20
The U.S. facilities operational as of March 2006 include Umatilla, Ore.;
Newport, Ind.; Deseret, Utah; Pine Bluff, Ark.; and Anniston, Ala. The
facilities at Blue Grass, Ky.; and Pueblo, Colo., remain in the design
phase. In February 2006, the facility at Edgewood, Md., began closing
procedures. As of November 2003, all chemical weapons at Johnston Atoll
were destroyed and the destruction facility dismantled.
Page 24 GAO-06-692 CTR Assistance to Shchuch'ye
committing the largest amount, about $1.039 billion. The Russian
government estimates it will need about $5.6 billion to eliminate its
entire stockpile. All U.S. assistance for destroying Russian chemical
weapons is being provided to the CWDF at Shchuch'ye.21 As of March 2006,
other international donors, such as Canada and the United Kingdom, are
also providing significant assistance to Shchuch'ye to help fund the
Russian destruction building (Building 101A) and the infrastructure needed
to support the facility's operation.22 Although Italy is providing some
funding for Shchuch'ye infrastructure, most of its contributions are
committed to the construction of the CWDF at Pochep. Russia has been
relying on German assistance to destroy its stockpile of blister agents at
the Gorniy and Kambarka destruction facilities. Table 2 describes the
commitments and types of assistance provided by international donors.
Table 2: International Assistance for Russian Destruction, as of April
2006
Committed funding for
Russian destruction Areas to receive
International (U.S. dollars)a Types of projects being
donors international assistance funded
Belgium $100,000 Shchuch'ye To be determined
Canada 89,150,537 Shchuch'ye Industrial infrastructure,
railway, and equipment for
Building 101A
Czech Republic 232,458 Shchuch'ye Industrial infrastructure
(electrical substations)
Denmark 117, 970 Various locations Public outreach efforts
European Union 14,156,452 Gorniy, Equipment at Gorniy and
Kambarka, and Kambarka, and industrial
Shchuch'ye infrastructure at Shchuch'ye
(electrical substation)
Finland 871,771 Gorniy and other Equipment at Gorniy and
locations public outreach efforts
France 7,077,976 Shchuch'ye Environmental surveys and
other projects to be
determined
Germany 233,573,198 Gorniy and Kambarka Equipment for the
construction and operation of
both facilities
Ireland 94,376 Shchuch'ye To be determined
Italy 439,660,257 Shchuch'ye and Pochep Infrastructure (gas pipeline)
at Schuch'ye and the
construction of the Pochep
CWDF
Netherlands 9,028,325 Shchuch'ye Equipment for Building 101A
21
In addition to funds for destroying Russian chemical weapons, other CTR
assistance is being provided to help eliminate former Russian chemical
weapons production facilities at Volgograd and Novocheboksarsk.
22
The United States, the United Kingdom, and Canada, in conjunction with the
Russian Federation, formed a working group in November 2003 to coordinate
all international assistance to the Shchuch'ye site.
Page 25 GAO-06-692 CTR Assistance to Shchuch'ye
Committed funding for
Russian destruction Areas to receive
International donors (U.S. dollars)a Types of projects being
international assistance funded
New Zealand 1,158,433 Shchuch'ye Industrial infrastructure
(electrical substation)
Industrial Infrastructure
Norway 3,250,969 Shchuch'ye (electrical substation)
and
other projects to be
determined
Nuclear Threat 1,000,000 Shchuch'ye Infrastructure (railway
Initiative bridge)
Sweden 952,988 Shchuch'ye To be determined
Switzerland 6,984,707 Kambarka, Sanitary and hygiene
Shchuch'ye, monitoring system at
and other locations Shchuch'ye, equipment at
Kambarka, and public
outreach efforts
United Kingdom 141,196,728 Shchuch'ye and Industrial infrastructure,
other equipment for Building
locations 101A, and public outreach
efforts
United States 1,039,200,000 Shchuch'ye Construction of the
destruction facility
Total $1,987,807,145
Conclusion
Sources: GAO analysis of data from DOD, State, and international donors.
a
Donor commitments converted from foreign currencies to U.S. dollars using
the annual 2005 exchange rate.
To facilitate additional international contributions, the Russian
government has provided potential donors a list of CWDF projects requiring
assistance. Primarily, assistance is needed for the construction of the
destruction facilities at Kizner, Leonikovka, and Pochep, as well as
related infrastructure support. The Russian government is also seeking
international funding to support operations at the Kambarka and
Maradykovskiy facilities.
Until destroyed, Russia's stockpile of chemical weapons-especially nerve
agents contained in small munitions, such as those stored at Shchuch'ye-
remain a proliferation threat, vulnerable to diversion and theft. Since
1992, the United States has been providing CTR assistance for the CWDF at
Shchuch'ye to help reduce the threats posed by these weapons. Originally
designed as a pilot facility to "jump start" Russian chemical weapons
destruction efforts, Shchuch'ye may no longer be a priority for the
Russian government. Delays in implementing the Shchuch'ye project over the
past 14 years led the Russian government to begin destruction efforts at
other sites. Disagreements between the United States and Russia over the
types of munitions to destroy and how to destroy them, negotiations to
resolve outstanding issues, restrictions on U.S. funding, and difficulties
with Russian subcontractors, among other factors, have delayed the
Shchuch'ye facility's completion and increased its costs. Although
progress has been made on the physical construction of the facility over
the past 3 years, DOD continues to encounter numerous challenges that
affect the completion of the Shchuch'ye CWDF. Furthermore, DOD currently
cannot reliably estimate when the Shchuch'ye facility will be completed
and at what cost. Parsons' EVM system, implemented to help manage the
schedule and cost of the Shchuch'ye project, contains unreliable and
inaccurate data; thus, DOD cannot use it as a management tool. Even with
significant international assistance at Shchuch'ye and other destruction
facility sites, the Russian government will likely fail to destroy its
entire chemical weapons stockpile by the CWC extended deadline of 2012.
Unreliable EVM data limit DOD's efforts to accurately measure progress
Recommendations for
on the Shchuch'ye project and estimate its final completion date and cost.
Executive Action As such, we recommend that the Secretary of Defense
direct DTRA, in conjunction with the U.S. Army Corps of Engineers, to take
the following three actions:
o ensure that Parsons' EVM system contains valid, reliable data and that
the system reflects actual cost and schedule conditions;
o withhold a portion of Parsons' award fee until the EVM system produces
reliable data; and
o require Parsons to perform an IBR after awarding the contract for
completing Building 101.
Agency Comments and Our Evaluation
DOD provided comments on a draft of this report, which are reproduced in
appendix III. DOD concurred with our recommendation that DTRA in
conjunction with the U.S. Army Corps of Engineers ensure that Parsons' EVM
system contains valid, reliable data and reflects actual cost and schedule
conditions, and require that Parsons perform an IBR after awarding the
contract for completing Building 101. DOD partially concurred with our
recommendation that a portion of Parsons' award fee be withheld until the
EVM system produces reliable data. DOD stated that it had withheld a
portion of Parson's award fee in a previous period. DOD further noted that
an award fee must be based on the merits of the contractor's performance
and until the performance period is completed, it cannot prejudge Parsons'
performance and predetermine the withholding of award fees based on our
recommendation. DOD also provided technical comments, which we have
incorporated where appropriate. The Department of State was provided a
draft of this report but did not provide comments.
We are providing copies of this report to the Secretaries of Defense and
State and interested congressional committees. We will also make copies
available to others upon request. In addition, this report will be
available on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please
contact me at (202) 512-8979 or [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on the
last page of this report. GAO staff who made major contributions to this
report are listed in appendix IV.
Sincerely yours,
Joseph A. Christoff Director, International Affairs and Trade
Appendix I: Scope and Methodology
To assess the progress of the Shchuch'ye facility, we collected and
analyzed Department of Defense (DOD) and Parsons Global Services, Inc.
(Parsons) contractor documents and met with relevant officials.
Specifically, we met with officials from the Cooperative Threat Reduction
(CTR) Policy Office, the office of the Assistant to the Secretary of
Defense for Nuclear and Chemical and Biological Defense Programs, the
Defense Threat Reduction Agency (DTRA), and the U.S. Army Corps of
Engineers. Within DTRA, we obtained information from the Director of the
Cooperative Threat Reduction Directorate, as well as the program and
project managers, for chemical weapons elimination. We also met with
officials from the Threat Reduction Support Center in Springfield,
Virginia. In addition, we met with officials from the DTRA office and the
Chemical Weapons Destruction Support Office in Moscow.
We traveled to the Russian Federation to observe construction of the
CTRfunded chemical weapons destruction facility at Shchuch'ye. At
Shchuch'ye and Chelyabinsk, we met with personnel from Parsons and the
U.S. Army Corps of Engineers. In Moscow, we met with Russian government
officials at the Federal Agency for Industry, the Ministry of Foreign
Affairs, the Duma, and the Accounts Chamber of the Russian Federation. We
also analyzed the reliability of the earned value management (EVM) data
for the Shchuch'ye project. Specifically, we examined Parsons' EVM reports
for a 5-month period from, September 2005 to January 2006, to assess the
Shchuch'ye destruction facility's cost and schedule. We checked the EVM
data to see if there were any mathematical errors or inconsistencies that
would lead to the data being unreliable. We interviewed officials from the
Defense Contract Management Agency (DCMA), the Defense Contract Audit
Agency (DCAA), and Parsons officials to better understand the anomalies in
Parsons' EVM data and determine what outside surveillance was being done
to ensure the validity of the EVM data. We also used a data collection
instrument to obtain detailed information from DOD on the Shchuch'ye
project, including the contract, program management activities,
independent cost estimates, risk analysis, and award fees.
To obtain information on Russian elimination efforts and international
donor assistance for Russian chemical weapons destruction, we met with
U.S., Russian, and international donor officials and obtained copies of
pertinent documents, including the Russian chemical weapons destruction
plan. We obtained information from officials in the Bureau of European and
Eurasian Affairs and the Bureau of International Security and
Nonproliferation at the Department of State. At DOD, we met with officials
and acquired documents from the Office of the Secretary of Defense for
Page 29 GAO-06-692 CTR Assistance to Shchuch'ye Appendix I: Scope and
Methodology
Cooperative Threat Reduction Policy. In Moscow, we obtained information
from Russian government officials at the Accounts Chamber, the Federal
Agency for Industry, the Ministry of Foreign Affairs, and the Duma. At
Shchuch'ye, we spoke with a local government official involved with public
outreach efforts. We obtained data from the U.S., Russian, British,
Canadian, and German governments as well as the G-8 Global Partnership on
the assistance committed and provided for Russian chemical weapons
destruction efforts. To assess the reliability of these data, we
corroborated other nations' data wherever possible, comparing and
cross-checking documents and information. We interviewed officials from
the United States, Canada, Germany, the United Kingdom, and the Russian
Federation. We determined that data on funding and assistance provided for
Russian chemical weapons destruction were sufficiently reliable for the
purposes of this report. We also determined that data on the status of
Russian and U.S. chemical weapons elimination were sufficiently reliable
for the purposes of this report.
The information on Russian law in this report does not reflect our
independent legal analysis but is based on interviews and secondary
sources. We performed our work from June 2005 through May 2006 in
accordance with generally accepted government auditing standards.
Appendix II: Lack of Reliable EVM Data Limits DOD's Ability to Estimate Schedule
and Cost for Constructing the CWDF
GAO's Analysis Reveals Parsons' EVM Data are Unreliable and Require
Surveillance
Measuring and reporting progress against cost and schedule commitments is
vital to effective program management. To measure program performance, DOD
requires the use of EVM, a concept that has been used by DOD since the
1960s for measuring program performance. Through EVM, program offices can
determine a contractor's ability to perform work within cost and schedule
estimates by examining variances between the actual and estimated costs
and time to perform work tasks. EVM offers many benefits when done
properly and serves as a means to measure performance and identify
deviations from planned activities, allowing program managers to mitigate
risks. Based on our analysis of Parsons' EVM data, and the findings of
DCMA and DCAA, the data are inaccurate and unreliable. Without reliable
schedule and cost estimates, DTRA has limited means to accurately assess
when the Shchuch'ye facility will be completed and at what cost.
In reviewing Parsons' monthly EVM data for September 2005 through January
2006, we discovered numerous instances of data not adding properly for
scheduled work. Further, Parsons' EVM reports are not capturing all of the
data needed by project management to make informed decisions about the
Shchuch'ye facility. Such errors may cause DOD and Parsons project
officials to overestimate the amount of funding available to cover future
risks, such as the systemization of the Shchuch'ye facility. Moreover, we
found several instances where the accounting data were not allocated to
the correct cost accounts causing large cost over-runs and under-runs. In
these cases, the accounting data were placed in the wrong account, or
Parsons' accounting system was unable to track costs at the level of
detail EVM requires. Parsons officials stated that measures are being
taken to identify these inaccuracies and allocate the accounting data to
the proper cost accounts. These problems, however, have led to numerous
accounting errors in Parsons' EVM reports.
Furthermore, in reviewing Parsons' EVM reporting data, we found several
errors that a Parsons' official attributes to the company's accounting
system. For instance, current EVM period data are not accurate due to
historical data corruption, numerous mistakes in accounting accruals, and
manual budget adjustments. Such mistakes underestimate the true cost of
the CWDF project by ignoring cost variances that have already occurred.
For example, the Moscow project management task was budgeted at a cost of
$100,000. According to the January 2006 EVM report, the work has been
completed but the actual cost was $2.6 million, resulting in an overrun of
approximately $2.5 million. The EVM report, however, fails to capture the
expected $2.5 million overrun. Such data are misleading and
Page 31 GAO-06-692 CTR Assistance to Shchuch'ye
Appendix II: Lack of Reliable EVM Data Limits DOD's Ability to Estimate Schedule
and Cost for Constructing the CWDF
skew the project's overall performance. As indicated in table 3, this is
just one example of accounting system errors. In the case of the Moscow
project management task, Parsons officials explained that this error
occurred because the budget for this account was misaligned and,
therefore, caused a false cost variance. Parsons officials stated they
would be issuing an internal change order to correct this mistake.
Table 3: Examples of Accounting System Errors Understating the Variance at
Completion from January 2006 Report
Dollars in millions
Work Work Actual scheduled performed costs
GAO analysis of variance at Budget at Estimate at completion completion
completion understated by
Design task (27); task management $2.1 $2.1 $5.5 $2.1 $2.1 $3.4
Design task (27); project management, Moscow $0.1 $0.1 $2.6 $0.1 $0.1 $2.5
Design task (27); construction packages $9.4 $9.0 $20.2 $9.9 $9.9 $10.3
Source: GAO analysis of Parsons data.
Until Parsons' management updates the company's accounting system, these
types of manual adjustments will have to be made through monthly change
orders to ensure that costs are properly aligned with the correct budget.
Such continuous adjustments do not allow the EVM system to provide timely
and accurate information to Parsons and DOD managers.
In addition, DOD guidance and best practices require program managers to
conduct an integrated baseline review (IBR) as needed to ensure that the
baseline for tracking cost, technical information, and schedule status
reflects (1) all tasks in the statement of work, (2) adequate resources in
terms of staff and materials to complete the tasks, and (3) integration of
the tasks into a well-defined schedule. Program managers are required to
use EVM reports that have been validated by an IBR. Without verifying the
baseline, monthly EVM reporting-which tracks project work against a set
budget and schedule-is insufficient and invalid.
Parsons and DOD officials explained that while an IBR has been discussed,
one will not be conducted until the contract for completing Building 101
has been awarded. DOD officials estimate that the contractaward process
may not be completed until June 2006, resulting in a 1 year
Page 32 GAO-06-692 CTR Assistance to Shchuch'ye Appendix II: Lack of
Reliable EVM Data Limits DOD's Ability to Estimate Schedule and Cost for
Constructing the CWDF
DCMA and DCAA Have Concerns with Parsons' EVM Implementation
delay. Such a delay not only prevents Parsons from holding an IBR, but it
also jeopardizes DOD's ability to accurately estimate the cost and
schedule to complete the CWDF program. Until the costs have been
negotiated for building the remainder of Building 101, it is unclear
whether the CWDF at Shchuch'ye will be completed on time and within
budget. DTRA officials explained that if the costs for this effort exceed
the original estimate, they will have to cover the shortfall using
management reserve funds. Using management reserve funds for construction
leaves less contingency funding available to complete and test the
Shchuch'ye facility.
Until December 2004, DTRA was using EVM data from a simplified Parsons EVM
process. In September 2004, DTRA directed Parsons to implement a complete
EVM system that was capable of being validated by DCMA. Although Parsons'
EVM validation was originally scheduled for March 2005, Parsons was unable
to meet this deadline and requested a series of extensions. In September
2005, DCMA officials visited the Shchuch'ye site for a program assistance
visit and then returned in mid-November 2005 to conduct the formal
validation review, 8 months later than planned.
DOD requires all EVM systems to go through a compliance audit or
"validation" conducted by DCMA, with assistance from DCAA. The evaluation
team looks for proof that the system meets the 32 criteria1 for a good EVM
system, as well as 2 to 3 months of reliable EVM data. While the DCMA
official who led the validation team saw much improvement in Parsons' EVM
system from September to November 2005, he stipulated that an EVM
compliance audit only tests whether the contractor has a good, capable EVM
system and knows how to use it. A compliance audit does not identify
whether the system is used properly, the data are reliable, or the
products of the system are read and acted upon by management. The DCMA
official stated that continual surveillance of Parsons' EVM system would
be necessary to ensure these actions were occurring. According to the
official, DCMA does not expect to perform surveillance for the Shchuch'ye
project.
1
The American National Standards Institute/Electronic Industries Alliance
guidance identifies 32 criteria that reliable EVM systems should meet. The
criteria are organized into five categories: organization, planning and
budgeting; accounting; analysis; and revisions and data maintenance.
Page 33 GAO-06-692 CTR Assistance to Shchuch'ye Appendix II: Lack of
Reliable EVM Data Limits DOD's Ability to Estimate Schedule and Cost for
Constructing the CWDF
DCAA also participated in Parsons' EVM validation and produced a
corrective action report stating that its EVM accounting process was
inadequate. Specifically, Parsons did not provide adequate documentation
that direct costs of almost $300,000 were based on accurate and reliable
accounting data. The source of the accounting data used by Parsons may be
unreliable, causing actual costs for September 2005 to be significantly
understated. For September 2005, Parsons subtracted almost $1 million
without providing sufficient data that the adjustment was reasonable and
allowable. A DCAA official stated that these findings are the result of
Parsons' reliance on an outdated accounting system that has difficulty
capturing actual costs for the Shchuch'ye project into a proper cost
ledger. The official noted that the software Parsons uses to query the
accounting system and pull data into the EVM reports also caused errors.
DCAA was also concerned with Parsons' ability to apply effective EVM data
quality control. According to DCAA officials, Parsons' management should
have discovered such accounting errors before the EVM report was released
to DOD. DCAA therefore questioned whether Parsons can generate correct
accounting data and recommended that Parsons update its accounting system.
Appendix III: Comments from the Department of Defense
Appendix III: Comments from the Department of Defense
Appendix III: Comments from the Department of Defense
Appendix IV: GAO Contact and Staff Acknowledgments
Joseph Christoff (202) 512-8979 or [email protected]
GAO Contact
In addition to the individual named above, Muriel Forster (Assistant
Director), Jerome Brown, Lynn Cothern, Jennifer Echard, David Hancock,
Beth Hoffman Leon, and Karen Richey contributed to this report. Joanna
Chan, Martin DeAlteriis, Mark Dowling, Jennifer Mills, and Jena Sinkfield
also provided assistance.
(320363)
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202) 512-4800
Public Affairs
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON
RECYCLED PAPER
*** End of document. ***