Space Projects: Astrophysics Facility Program Contains Cost and Technical Risks (Letter Report, 01/28/94, GAO/NSIAD-94-80). The Advanced X-ray Astrophysics Facility (AXAF) is to be the third in the National Aeronautics and Space Administration's (NASA) series of four "great observatories," which are expected to provide new data and insights into the age, evolution, and composition of the universe. By redesigning and dividing the AXAF into two satellites, NASA has been able to make the program more affordable. The restructuring cut program costs by 54 percent without compromising the AXAF's scientific mission. Affordability for the AXAF-I--the satellite devoted to high-resolution x-ray imaging--may still be a concern, however, because funding reserves for it may fall short. Also, NASA may not have fully accounted for the increased technical risks inherent in the new design. As now designed, AXAF-I cannot be repaired in orbit, which means that hardware failures could scuttle a mission. Further minimizing the risk of hardware failures demands the highest quality parts and rigorous testing, which would boost costs significantly. NASA expects the restructure program to achieve most of the original science objectives set for AXAF. The objectives should be achieved because, owing to a higher orbit for AXAF-1 and greater individual use of instruments, the satellites would operate more efficiently than the originally planned single satellite. --------------------------- Indexing Terms ----------------------------- REPORTNUM: NSIAD-94-80 TITLE: Space Projects: Astrophysics Facility Program Contains Cost and Technical Risks DATE: 01/28/94 SUBJECT: Aerospace research Testing Cost control Satellites Space exploration Product performance evaluation Research and development costs Aerospace engineering Mission budgeting Scientific research IDENTIFIER: NASA Advanced X-Ray Astrophysics Facility Program NASA Hubble Space Telescope Program NASA Compton Gamma Ray Observatory NASA Space Infrared Telescope Facility NASA AXAF Charge Coupled Device Imaging Spectrometer NASA High Resolution Camera NASA High Energy Transmission Grating Spectrometer NASA Low Energy Transmission Grating Spectrometer Astro-E Mission (Japan) ************************************************************************** * This file contains an ASCII representation of the text of a GAO * * report. Delineations within the text indicating chapter titles, * * headings, and bullets are preserved. Major divisions and subdivisions * * of the text, such as Chapters, Sections, and Appendixes, are * * identified by double and single lines. The numbers on the right end * * of these lines indicate the position of each of the subsections in the * * document outline. These numbers do NOT correspond with the page * * numbers of the printed product. * * * * No attempt has been made to display graphic images, although figure * * captions are reproduced. Tables are included, but may not resemble * * those in the printed version. * * * * A printed copy of this report may be obtained from the GAO Document * * Distribution Facility by calling (202) 512-6000, by faxing your * * request to (301) 258-4066, or by writing to P.O. Box 6015, * * Gaithersburg, MD 20884-6015. We are unable to accept electronic orders * * for printed documents at this time. * ************************************************************************** Cover ================================================================ COVER Report to Congressional Requesters January 1994 SPACE PROJECTS - ASTROPHYSICS FACILITY PROGRAM CONTAINS COST AND TECHNICAL RISKS GAO/NSIAD-94-80 Space Projects Abbreviations =============================================================== ABBREV AXAF - Advanced X-ray Astrophysics Facility NASA - National Aeronautics and Space Administration Letter =============================================================== LETTER B-255776 January 28, 1994 The Honorable Barbara A. Mikulski Chair, Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations United States Senate The Honorable John D. Rockefeller, IV Chairman, Subcommittee on Science, Technology, and Space Committee on Commerce, Science, and Transportation United States Senate The Honorable Louis Stokes Chairman, Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations House of Representatives The Honorable Ralph M. Hall Chairman, Subcommittee on Space Committee on Science, Space, and Technology House of Representatives The National Aeronautics and Space Administration (NASA) is developing the Advanced X-ray Astrophysics Facility (AXAF) to be one of the mainstays of this nation's space science program during the next decade. It is to be used to investigate important questions such as the age and origin of the universe. We reviewed the status of the 1992 redesigned AXAF program to (1) assess the reasonableness of NASA's estimate of program costs and (2) determine the extent to which the redesign will provide scientific returns comparable to the original program. We recognize that a recent congressional direction may affect a portion of this program, but the cost and technical impacts of that decision are uncertain at this time. However, we believe the results of our work in other segments of the program are still relevant. This report was prepared pursuant to our statutory authority, not at the committees' request. It is intended to provide a better understanding of the implications of NASA's decision to restructure the program. BACKGROUND ------------------------------------------------------------ Letter :1 AXAF is to be the third in NASA's series of four "great observatories." It is to complement the Hubble Space Telescope, which views objects in the visible and ultraviolet bandwidth of the electromagnetic spectrum,\1 the Compton Gamma Ray Observatory, and the yet-to-be-developed Space Infrared Telescope Facility. These observatories are intended to provide new data and insights for studies of the age, evolution, and composition of the universe and its objects. The Congress authorized NASA to begin developing AXAF in fiscal year 1989. Our 1992 report\2 on the program stated that (1) the program's estimated cost had increased by about 23 percent since it was first approved, (2) the funding reserves had been used to offset additional increases, and (3) the possibility existed that costs would increase further since NASA might decide to launch AXAF on an expendable launch vehicle rather than on the shuttle. We also reported that, while test results on the AXAF's outer mirrors were encouraging, a number of significant challenges remained for NASA to successfully launch and operate the observatory. In 1992, NASA concluded that AXAF, as then designed, was not affordable. Budget cutbacks and the prospect of continued cost growth caused NASA to restructure the program. To reduce cost, NASA divided the program into two separate satellites or missions. One satellite, AXAF-I (imaging), will emphasize high-resolution x-ray imaging. Its instruments will include the AXAF Charge Coupled Device Imaging Spectrometer, the High Resolution Camera, the High Energy Transmission Grating\3 Spectrometer, and the Low Energy Transmission Grating Spectrometer. AXAF-I is currently expected to be launched on the shuttle in September 1998. The other satellite, AXAF-S, was to concentrate on high-energy, high-resolution spectroscopy using the X-ray Spectrometer. NASA planned to develop AXAF-S in-house at Marshall Space Flight Center by using civil service personnel and limited contractor support. However, the Conference Report accompanying NASA's Fiscal Year 1994 Appropriations Act indicates a $19-million reduction in the spacecraft portion of AXAF-S development funding and directs NASA to include the spectrometer on the future Japanese Astro-E mission. At the completion of our current review, NASA did not have a plan to implement this change. -------------------- \1 The electromagnetic spectrum is the entire array of energy wavelengths as a continuum, from gamma rays to radio waves. The four observatories cover the spectrum from gamma rays to infrared. \2 Space Projects: Status and Remaining Challenges of the Advanced X-ray Astrophysics Facility (GAO/NSIAD-92-77, Feb. 28, 1992). \3 Transmission gratings are devices that are used to bend x-rays in a predictable way, thereby enabling scientists to determine the wavelength of the energy. RESULTS IN BRIEF ------------------------------------------------------------ Letter :2 By redesigning and dividing AXAF into two satellites, NASA was able to make the program more affordable. The restructuring reduced estimated program development and operating costs by 54 percent without sacrificing much of the program's science content. Revising the AXAF-S program in accordance with the recent congressional action will further reduce program costs by an undetermined amount. Affordability for the AXAF-I portion of the program, however, may still be a concern because funding reserves for this portion may be inadequate. Also, NASA may not have fully accounted for the increased technical risks inherent in the new design. As currently designed, AXAF-I cannot be repaired in orbit, which means that hardware failures could seriously degrade or destroy the mission. Further reducing the risk of hardware failures would require the highest quality parts and rigorous testing, which would increase costs significantly. NASA expects the restructured program to achieve most of the original science objectives established for AXAF. The objectives should be achieved because, owing to a higher orbit for AXAF-I and greater individual use of instruments, the satellites would operate more efficiently than the originally planned single satellite. PROGRAM COST IS UNCERTAIN ------------------------------------------------------------ Letter :3 In February 1992, before the program was restructured, NASA estimated that AXAF would cost about $5.6 billion to develop and operate. In September 1993, NASA estimated the total cost to be about $2.6 billion--a 54-percent reduction. (See table 1 for a comparison of the estimates before and after NASA restructured the program.) Officials told us that the fiscal year 1994 appropriations conferees directed NASA to terminate the spacecraft part of AXAF-S, and that action will further reduce the program's cost, but the officials have not yet determined the amount of the reduction. Consequently, the amounts shown in the table do not include changes that may result from the fiscal year 1994 congressional action. For example, one change would eliminate the planned Delta II launch for AXAF-S (about $58 million). Table 1 February 1992 and September 1993 Cost Estimates (Dollars in millions) Feb. 1992 Sept. 1993 Cost element estimate\a estimate\a Difference ------------------ ------------ ------------ ------------ Advanced $54 $54 0 technology development Development 2,000 1,773 $(227) Mission operations 3,325 529 (2,796) and data analysis Launch support 145 226 81 Tracking and data 55 2 (53) support Construction of 18 18 0 facilities ============================================================ Total $5,597 $2,602 $(2,995) ------------------------------------------------------------ \a Neither estimate includes costs for civil service personnel. Most of the $2,995 million decrease (about $2.8 billion) is attributed to a change in operating concepts. Under the original design, NASA planned to operate AXAF for 15 years and to maintain and service it in orbit using the shuttle. Under the new design, the life expectancy of the satellites was reduced to about 5 years and NASA does not plan to service the satellites. Design changes such as reducing the number of mirrors and science instruments account for most of the remaining reduction. FUNDING RESERVES MAY BE INADEQUATE ------------------------------------------------------------ Letter :4 While the 1992 redesign decision and recent congressional action will significantly reduce estimated costs, affordability may still be a concern in AXAF development. Our analysis of funding focuses on AXAF-I because, given the recent congressional action, NASA has no current estimate of the AXAF-S's cost. NASA's cost estimate to develop AXAF-I includes reserves to cover contingencies and programmatic changes that may occur during the development program. The level of reserves that NASA includes in program cost estimates varies from program to program and depends on the amount of uncertainty and risk of the particular program. However, in past reviews, NASA cost estimating officials have stated that in complex programs, prior to the critical design review, funding reserves should equal about 30 percent of development costs.\4 As of October 1993, NASA's estimate for AXAF-I included $197.3 million in reserves--about 21 percent of the remaining estimated development cost. Some of the AXAF-I reserves have already been earmarked to fund certain activities. For example, NASA forecasts a high probability (60 to 90 percent) that contractor overhead and general and administrative rates will increase and estimates that the increase could consume over 10 percent of the total available reserves. In addition, most of the reserves are allocated toward the end of the redesigned observatory's 7-year development effort. About 70 percent of AXAF-I's remaining reserves are earmarked for use in the last 3 years of the development effort. Consequently, the program must be managed using minimal amounts of reserve funding in the early years of development. During NASA's fiscal year 1993 oversight hearings, the NASA Administrator testified that the greatest demand for reserve funds normally occurs early in a program's development. If adequate reserves are not available when needed, schedule slips and higher costs will likely result. -------------------- \4 See Space Transportation: NASA Has No Firm Need for Increasingly Costly Orbital Maneuvering Vehicle (GAO/NSIAD-90-192, July 31, 1990); Space Projects: Status and Remaining Challenges of the Advanced X-ray Astrophysics Facility (GAO/NSIAD-92-77, Feb. 28, 1992); and NASA Program Costs: Space Missions Require Substantially More Funding Than Initially Estimated (GAO/NSIAD-93-97, Dec. 31, 1992). TECHNICAL RISK REMAINS A QUESTION ------------------------------------------------------------ Letter :5 Under NASA's redesigned concept, neither of the satellites would be serviceable in orbit, meaning technical problems that cannot be corrected through ground communications could degrade or destroy the mission. NASA may not have fully accounted for this risk.\5 The risks associated with the inability to service AXAF-I are clearly illustrated by NASA's experience with the first two observatory missions. The Hubble Space Telescope required at least one servicing mission to correct problems discovered after it was deployed, and the Compton Gamma Ray Observatory experienced problems both during and after deployment. It required an unscheduled spacewalk by two shuttle astronauts to properly deploy an antenna, and it has since experienced failures in two data recorders and degradation in onboard batteries. NASA categorizes its payloads in terms of the risk it is willing to assume. All payloads are assigned to one of four categories, ranging from A to D, based on assessments of the consequences of mission failure. Factors such as cost, complexity, mission priority, and degree of acceptable risk are weighed in deciding upon a design approach. Payloads assigned the highest classification are considered to have the highest priority and cost. Despite its high priority and cost, NASA classified the original AXAF as a class B payload. In making this determination, NASA noted that AXAF would operate in a low-earth orbit where it could be serviced by the shuttle. The classification permits the use of less costly components and a lower level of testing than is required for a class A payload. NASA has not changed the payload classification for the imaging mission even though it will no longer be accessible to the shuttle for in-orbit repair. Upgrading AXAF-I's classification could result in significant programmatic and cost impacts. For example, only the highest quality parts would be used in the system's design, a complete set of flight spare replacement units could be required, and more stringent qualification testing would have to be performed. The Chief of the Payloads Assurance Office estimated that these changes could increase AXAF-I's development cost by one third to one half. The changes could also delay completion of the development program. -------------------- \5 As a consequence of the recent congressional action, our analysis relates only to AXAF-I risk. REDESIGNED OBSERVATORIES ARE TO PROVIDE SIGNIFICANT SCIENCE ------------------------------------------------------------ Letter :6 In September 1992, the National Research Council was asked to evaluate the scientific content and return of the restructured program. In an April 28, 1993, report, the task force assigned to the review reported that the restructured program ". . . maintained essentially all of the outstanding scientific capabilities of the baseline mission." The task force concluded that the program was capable of being the ". . . centerpiece of international efforts in x-ray astronomy for the foreseeable future." AXAF-I is expected to achieve a greater angular resolution than any prior x-ray mission. This improvement should make it possible to resolve and distinguish between energy sources, such as neutron stars and quasars, at greater distances and in closer proximity to each other than ever before. The council has remained steadfast in its support for the program. In 1982 and 1991, the council's astronomy and astrophysics committees recommended new ground- and space-based programs for development. The committees considered large and expensive development efforts, as well as smaller, more focused experiments. Both committees strongly recommended AXAF. NASA plans to place AXAF-I into a high-earth elliptical orbit at a minimum altitude ranging from 10,000 to 100,000 kilometers. The original AXAF design provided for a low-earth orbit of about 600 kilometers. In a low-earth orbit, the earth would periodically obstruct the observatory's view of a target, which, in turn, would require NASA to repoint the antennas and reacquire the target. According to NASA, the higher orbit would eliminate this problem for most targets. (See fig. 1 for depiction of orbital viewing efficiencies.) Figure 1: AXAF Orbital Viewing Efficiency (See figure in printed edition.) In addition, the new design would permit more efficient use of the science instruments. This approach separated the imaging mission from the primary spectroscopy mission, and eliminated one spectrometer. This would have meant that all of the remaining instruments could be used for longer periods of time, and simultaneous imaging and spectroscopic observations could be made. As a result, more observations per unit of viewing time would have been possible. NASA officials told us the Fiscal Year 1994 Appropriations Conference Committee's direction to fly the X-ray Spectrometer on Astro-E (the Japanese mission) could affect the instrument's performance as well as the potential for simultaneous viewing of cosmic events. According to AXAF-S program management officials, instrument weight, life expectancy, spacecraft orbit, launch date, proprietary use of science data, and number of instruments to be flown on the satellite could affect the scientific return from the X-ray Spectrometer. According to the officials, if Astro-E is used to launch the spectrometer, each of these matters will be subject to negotiation with Japan. Therefore, NASA has not determined the impact of this redirection on the program's science capabilities. RECOMMENDATION ------------------------------------------------------------ Letter :7 We recommend that the NASA Administrator review the AXAF-I development program to determine whether (1) funding reserves are realistic in light of the program's uncertainty and risk and (2) the AXAF-I payload classification is prudent given the complexity, costs, and lack of repair capability inherent in the new design. AGENCY COMMENTS ------------------------------------------------------------ Letter :8 NASA found our report to be accurate and clear and agreed with our recommendations. NASA's detailed comments are reprinted in appendix I. SCOPE AND METHODOLOGY ------------------------------------------------------------ Letter :9 We analyzed budget submission documentation, cost estimates, briefing reports, legislative language, project planning documents, and technical studies at Marshall Space Flight Center and NASA headquarters. We then discussed cost and performance issues with program management, engineering, quality assurance, and procurement officials. To assess the reasonableness of the cost estimates, we compared the detailed cost breakouts of the original and restructured programs, identified significant differences, verified the accuracy of NASA's calculations when possible, and discussed NASA's position with program and project management officials. We also assessed the adequacy of funding reserves by (1) comparing the reserve totals as allocated by year to a generally accepted standard for similar development programs, (2) analyzing known programmatic requirements, (3) identifying potential additional costs unaccounted for in the overall estimate, and (4) obtaining NASA's position relating to funding adequacy. To determine the scientific capabilities of the two redesigned satellites, we compared the major performance characteristics of the original design with those projected for the new design. We then identified the types and amount of scientific investigations possible under the new design. We also analyzed internal studies and policies regarding technical and programmatic risks giving particular attention to the lack of serviceability in the new design. We performed our review from December 1992 through November 1993 in accordance with generally accepted government auditing standards. We are also sending copies of this report to the Administrator, NASA; other appropriate congressional committees; the Director, Office of Management and Budget; and other interested parties. Please contact me at (202) 512-8412 if you or your staff have any questions concerning this report. The major contributors to this report are listed in appendix II. Donna M. Heivilin, Director Defense Management and NASA Issues (See figure in printed edition.)Appendix I COMMENTS FROM THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION ============================================================== Letter MAJOR CONTRIBUTORS TO THIS REPORT ========================================================== Appendix II NATIONAL SECURITY AND INTERNATIONAL AFFAIRS DIVISION, WASHINGTON, D.C. David R. Warren, Associate Director ATLANTA REGIONAL OFFICE Lee Edwards, Acting Assistant Director John Gilchrist, Evaluator in Charge Wendy Smythe, Site Senior