Homeland Security: Federal Protective Service Could Better
Measure the Performance of Its Control Centers (29-SEP-06,
GAO-06-1076).
The Department of Homeland Security's Federal Protective Service
(FPS) through its control centers (MegaCenters) helps provide for
the security and protection of federally owned and leased
facilities. This report (1) identifies the services MegaCenters
provide, (2) determines how FPS assesses MegaCenter performance
and whether FPS links MegaCenter performance measures to FPS-wide
measures, and (3) examines how MegaCenters and selected
organizations compare in the services they provide. To address
these issues, GAO reviewed FPS's performance measures and past
MegaCenter assessments, assessed the MegaCenters' performance
measures, and interviewed officials and collected relevant
information at FPS, the four MegaCenters, and nine selected
security organizations.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-1076
ACCNO: A61609
TITLE: Homeland Security: Federal Protective Service Could
Better Measure the Performance of Its Control Centers
DATE: 09/29/2006
SUBJECT: Facility security
Federal facilities
Federal property management
Homeland security
Monitoring
Performance measures
Physical security
Police
FPS MegaCenters
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-1076
* Report to the Chairman, Committee on Government Reform, House of
Representatives
* September 2006
* HOMELAND SECURITY
* Federal Protective Service Could Better Measure the Performance
of Its Control Centers
* Contents
* Results in Brief
* MegaCenters Provide Alarm and Radio Monitoring and Dispatch from
Four Locations
* MegaCenters' and FPS's Performance Measures Are Not Linked
* Nine Selected Organizations Provide Some of the MegaCenters'
Services in a Generally Comparable Manner
* Conclusions
* Recommendations for Executive Action
* Agency Comments
* Briefing for the Committee on Government Reform, U.S. House of
Representatives, August 14, 2006
* Comments from the Department of Homeland Security
* GAO Contact and Staff Acknowledgments
* PDF6-Ordering Information.pdf
* Order by Mail or Phone
United States Government Accountability Office
Report to the Chairman, Committee on Government Reform, House of
Representatives
September 2006
HOMELAND SECURITY
Federal Protective Service Could Better Measure the Performance of Its Control
Centers
a
GAO-06-1076
HOMELAND SECURITY
Federal Protective Service Could Better Measure the Performance of Its
Control Centers
What GAO Found
FPS MegaCenters provide three primary security services-alarm monitoring,
radio monitoring, and dispatching of FPS police officers and contract
guards. These and other services are provided around the clock from four
locations-Battle Creek, Michigan; Denver, Colorado; Philadelphia,
Pennsylvania; and Suitland, Maryland. With a fiscal year 2006 budget of
$23.5 million, the MegaCenters monitor alarms at over 8,300 federal
facilities, covering almost 381 million square feet, and have available
for dispatch over 7,800 FPS police officers and contract guards.
The MegaCenter Emergency Response Process
Source: GAO analysis of FPS data.
FPS MegaCenter managers assess MegaCenter operations through a variety of
means, including reviewing data about volume and timeliness of operations,
listening to and evaluating a sample of calls between operators and FPS
police officers and contract guards, and receiving informal feedback about
customer satisfaction. FPS managers have also developed performance
measures for assessing MegaCenter operations. However, these measures are
of limited use because they are not always clearly stated or measurable
and do not address governmentwide priorities of efficiency, cost of
service, and outcome-which are among the attributes that GAO has
identified for successful performance measures. In addition, the
MegaCenters do not measure a key activity-the time from alarm to officer
dispatch-that would link MegaCenter performance to an FPS-wide performance
measure of response time. Without a corresponding measure, FPS is limited
in its ability to evaluate the MegaCenters' contribution to the FPS-wide
measure of response time.
Nine selected security organizations-including federal and local police
and private entities-offer some of the MegaCenters' services as well as
provide and assess these services in a manner that is generally similar to
the MegaCenters. Like the MegaCenters, many of the selected organizations
have centralized their operations. They also use regular call reviews and
volume and time measures to assess the quality of the services they
provide. A major difference between the MegaCenters and some selected
organizations is the use of a computer-aided dispatch system, which
enables these organizations to automate many functions.
United States Government Accountability Office
Contents
Letter 1
Results in Brief 3
MegaCenters Provide Alarm and Radio Monitoring
and Dispatch
from Four Locations 5
MegaCenters' and FPS's Performance Measures 6
Are Not Linked
Nine Selected Organizations Provide Some of
the MegaCenters'
Services in a Generally Comparable Manner 9
Conclusions 10
Recommendations for Executive Action 11
Agency Comments 12
Appendixes
Appendix I: Briefing for the Committee on Government
Reform, U.S.
House of Representatives, August 14, 2006 14
Appendix II: Comments from the Department of Homeland 61
Security
Appendix III: GAO Contact and Staff Acknowledgments 63
Abbreviations
CAD computer-aided dispatch
DHS Department of Homeland Security
FPS Federal Protective Service
GSA General Services Administration
ICE U.S. Immigration and Customs Enforcement
IT information technology
UL Underwriters Laboratories
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
September 29, 2006
The Honorable Tom Davis Chairman Committee on Government Reform House of
Representatives
Dear Mr. Chairman:
Since the 1995 bombing of the Alfred P. Murrah Federal Building in
Oklahoma City and the September 11, 2001, attacks on the World Trade
Center and the Pentagon, terrorism has threatened the nation's security,
including the physical security of federal facilities. The Homeland
Security Act of 2002 created the Department of Homeland Security (DHS), a
new federal department with the mission of preventing terrorist attacks
within the United States, which includes safeguarding federal facilities.1
DHS, through its Federal Protective Service (FPS), provides law
enforcement and security services to federal agencies that occupy
facilities under the jurisdiction of the General Services Administration
(GSA) and DHS, protecting millions of federal employees, contractors, and
citizens.2 As part of its approach to facility protection, FPS provides
support for its law enforcement and security services through four control
centers known as MegaCenters that are located in Battle Creek, Michigan;
Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.
Because of the important role MegaCenters play in ensuring the safety of
federal facilities and their occupants, our objectives were to (1)
identify the services the MegaCenters provide and how they provide them,
(2) determine how FPS assesses and measures the performance of MegaCenter
operations and how FPS links MegaCenter performance measures to FPS-wide
performance measures, and (3) examine how the MegaCenters compare to
selected security organizations in the services they provide and in the
methods they use to provide them.
To determine the services offered by the MegaCenters and how the
MegaCenters provide these services, we interviewed managers at the four
1Federal facilities include government-owned and -leased space.
2Under agreement, FPS authority can be extended to provide its law
enforcement and security services to any property with a significant
federal interest.
MegaCenters and the MegaCenter branch chief, toured three
MegaCenters-Denver, Philadelphia, and Suitland, collected information on
services and workload data from MegaCenter management, and conducted
document reviews. To determine how FPS assesses MegaCenter performance and
how FPS links MegaCenter performance measures to FPS-wide performance
measures, we reviewed documentation on FPS's performance measures and past
MegaCenter assessments and interviewed MegaCenter management and FPS
headquarters officials. We also assessed the MegaCenters' 11 performance
measures to determine whether they were consistent with selected
attributes of successful performance measures we have identfied, that is,
that they were linked to agency mission and goals; clearly stated;
contained measurable targets; sufficiently covered the program's core
activities; and addressed governmentwide priorities of quality,
timeliness, efficiency, cost of service, and outcome.3
To determine how the MegaCenters compare to selected security
organizations we (1) identified criteria for selecting comparable
organizations and selected four public and five private organizations; (2)
interviewed officials at the nine selected organizations; (3) toured four
of the organizations' control centers-both public and private-to observe
their security operations, procedures, and technology; (4) collected
detailed service and workload information from the organizations; (5) used
the information gathered to compare the MegaCenters and the selected
organizations with respect to services offered, organizational structure,
quality assessment practices, and technology utilization; and (6)
interviewed officials from security industry standard-setting and
accreditation associations (associations) because these associations were
identified as having information on security industry organizations,
operations, quality assessment practices, and technology utilization
practices.4 We used two approaches for selecting the private and public
organizations. We selected private organizations from industry lists of
the top 20 largest security service providers and system integrators in
terms of 2005 revenue that provided services such as alarm monitoring and
access
3We did not assess whether the performance measures were objective,
reliable, overlapping, or balanced.
4We interviewed officials from the following associations: Association of
Public Safety Communications Officials, the Commission on Accreditation
for Law Enforcement Agencies, the National Burglar and Fire Alarm
Association, the National Fire Protection Association, Priority Dispatch,
the Security Industry Association, and Underwriters Laboratories.
control.5 With the help of one of the associations we interviewed, we were
able to make contact with the five security organizations that were
selected as our comparison group.6 For public organization comparisons, we
selected three federal organizations: U.S. Postal Inspection Service, U.S.
Customs and Border Protection, and U.S. Park Police. We selected these
federal organizations because each had a law enforcement branch;
centralized control center(s) that offered, at a minimum, one of the
MegaCenters' primary services; and nationwide operations, characteristics
most similar to those of the MegaCenters. We also selected the Denver
Police Department to serve as a nonfederal public organization that
provided dispatch and radio monitoring services through a central control
center. Because we judgmentally selected the organizations, the
information we collected from them cannot be generalized. We conducted our
work from October 2005 through September 2006 in accordance with generally
accepted government auditing standards.
This report summarizes the information we provided to your office during
our August 14, 2006, briefing and, in addition, contains recommendations
to the Secretary of Homeland Security to improve the MegaCenters'
performance measures. The briefing slides are included in appendix I.
Results in Brief
The primary services that FPS MegaCenters provide are remote monitoring of
building alarm systems as well as radio monitoring and dispatch of FPS
police officers and contract guards. These and other services are provided
around the clock from four locations across the country: Battle Creek,
Denver, Philadelphia, and Suitland. Each MegaCenter has a sister center
with redundant capabilities that can serve as an emergency backup, and
each is operated by full-time federal employees and private contractors.
In fiscal year 2006, the MegaCenters had a budget of $23.5 million-
accounting for about 5 percent of FPS's total budget-to protect and
5We looked at two lists: one was in terms of gross revenue, the other was
in terms of revenue.
6In specific situations, two of the private security organizations
dispatched either contract guards or their own guards. However, because
these were situational circumstances, we did not include them as being
similar to the MegaCenters' dispatching service. For example, one
organization only uses contract guards when local jurisdictions will not
dispatch their own police officers without actual verification of the
nature of the alarm. In these cases, the organization will have a guard
check whether there is a burglary in progress, and if so, the organization
will contact the police.
monitor over 8,300 federal facilities and dispatch over 7,800 FPS police
officers and contract guards. To provide these services, the MegaCenters
rely on a variety of information technology (IT) systems, communications
systems, and other equipment.
FPS MegaCenter managers assess MegaCenter activities through a variety of
means, including reviewing reports on the timeliness and volume of
operations, listening to and evaluating a sample of calls between
operators and FPS police officers and contract guards, and receiving
informal feedback about customer satisfaction. Also, FPS managers have
developed performance measures for assessing MegaCenter operations. These
performance measures reflect some of the attributes of successful
performance measures we have identified, but also contain some weaknesses
because they are not always clearly stated or measurable and do not
address the governmentwide priorities of efficiency, cost of service, and
outcome. In addition, the MegaCenters do not have a performance measure
that corresponds to the FPS-wide performance measure that is applicable to
the MegaCenters' operations-the patrol and response time measure, which
tracks the elapsed time from the receipt of an alarm to a police officer's
arrival on the scene.7 While the FPS-wide patrol and response time measure
covers the MegaCenters' activities and reflects their performance, the
MegaCenters do not have their own measure that covers only the activities
for which they are responsible-from the receipt of the alarm to the
officer's dispatch. Without clearly stated and measurable performance
measures, including a measure that corresponds to FPS's agencywide patrol
and response time measure, FPS cannot compare the MegaCenters' performance
over time, assess their contribution to agencywide measures, and identify
opportunities for their improvement. We are recommending that FPS (1)
establish MegaCenter performance measures that meet the attributes of
successful performance measures we have identified; (2) develop a
performance measure for the MegaCenters that corresponds to the FPS-wide
patrol and response time measure and covers the MegaCenters' operations,
from alarm to dispatch; and (3) routinely assess the extent to which the
MegaCenters meet established performance measures. In commenting on a
draft of this report, DHS generally agreed with our findings and
recommendations.
7In addition to patrol and response time, FPS has three other agencywide
performance measures (1) timely deployment of countermeasures, (2)
functionality of countermeasures, and (3) facility security index (an
average success rate for the other measures).
Page 4 GAO-06-1076 Federal Protective Service
The nine selected security organizations offer some of the same security
services as the MegaCenters, and the services the organizations offer are
delivered and assessed in a manner generally similar to that of the
MegaCenters. For example, like the MegaCenters, many organizations have
centralized their control center operations, have backup capability,
allocate workload among centers based on geographic location, and use
regular call reviews as well as volume and time measures to assess the
quality of the services they provide. A few organizations offer services
that the MegaCenters do not offer. One major difference between the
MegaCenters and the organizations is that three organizations use a
computer-aided dispatch (CAD) system. Selected organizations and
associations referred to CAD systems as being beneficial for dispatching
services by allowing for faster operator response, automatic recording of
all operator actions enabling easier performance analysis, and automatic
operator access to standard operating procedures and response
prioritization. MegaCenters have identified a need and developed a plan
for a CAD system, but FPS has not allocated funding for such a system.
MegaCenters Provide Alarm and Radio Monitoring and Dispatch from Four
Locations
FPS MegaCenters provide federal agencies with three primary security
services-alarm monitoring, radio monitoring, and dispatch-through four
locations using a variety of IT systems. MegaCenters monitor intrusion,
panic, fire/smoke, and other alarms. They also monitor FPS police
officers' and contract guards' radio communication to ensure their safety
and to provide information, such as criminal background or license plate
histories, to officers upon request. In addition, they exercise command
and control authority by dispatching FPS police officers or contract
guards. MegaCenters also provide a variety of other services. For example,
they notify federal agencies regarding national emergencies and facility
problems and remotely diagnose problems with federal agency alarms. They
also receive and transcribe FPS police officer incident reports.
Individual MegaCenters may also provide unique services not provided by
other MegaCenters, such as facility-specific access control and remote
programming of alarms via the Internet. One MegaCenter also provides an
after-hours telephone answering service for the Drug Enforcement
Administration and for GSA building maintenance emergencies.
The MegaCenters are located in Battle Creek, Denver, Philadelphia, and
Suitland. Each MegaCenter has a sister center with redundant capability as
backup in case of a failure at that MegaCenter. Suitland is paired with
Battle Creek, and Philadelphia is paired with Denver. A force of 1,014 FPS
police officers and 6,842 contract guards is available for the MegaCenters
to dispatch in response to alarms and other emergencies. In fiscal year
2006, the MegaCenters were supported by a budget of $23.5 million, which
accounts for about 5 percent of FPS's total budget.8 The MegaCenters are
operated by 23 full-time federal employees-some of whom manage the
centers-and about 220 private contractors to provide around the clock
security services for over 8,300 federal facilities.
The MegaCenters rely on a variety of IT systems, communications systems,
and other equipment to provide their security services. The IT systems
enable MegaCenter staff to, among other activities, monitor alarms and
radio communications of FPS police officers and contract guards. For
communications systems, MegaCenters have regional and national toll-free
numbers for tenants and the public to contact the MegaCenters during
emergencies. Other equipment includes dictation machines, which enable FPS
police officers to dictate reports about incidents that occur at
facilities.
MegaCenters' and FPS's Performance Measures Are Not Linked
MegaCenters use various means to assess operations, but their performance
measures have weaknesses and are not linked to FPS-wide performance
measures. MegaCenter managers assess MegaCenter operations through a
variety of means, including reviewing data about volume and timeliness of
operations, listening to and evaluating a sample of calls between
operators and FPS police officers and contract guards, and receiving
informal feedback about customer satisfaction. FPS managers also have
developed 11 performance measures for assessing MegaCenter operations:
o distribute emergency notification reports (also known as SPOT reports)
within 30 minutes of notification;
* review problem alarm reports daily;
* o obtain regular feedback about customer satisfaction from field
operations;
o continuously review all SPOT reports and other outgoing information to
ensure 100 percent accuracy;
8As of August 2006, MegaCenter officials reported a revised MegaCenter
budget of approximately $19 million for fiscal year 2006.
Page 6 GAO-06-1076 Federal Protective Service
o transcribe dictated offense and incident reports into the database
management system within 8 hours of receipt of the report;
o submit reviewed contractor billing reports and time sheets within 7
business days after the last day of the month;
o prepare and review contractor reports for quality assurance plan;
o maintain completely accurate (nonduplicative) case control numbers;
o meet Underwriters Laboratories (UL) guidelines and requirements
continuously;9
o test failover of alarm, radio, and telephone systems weekly;10 and
o monitor calls and review recorded call content for adherence to
standard procedures at least monthly.
The Government Performance and Results Act of 1993 requires federal
agencies to, among other things, measure agency performance in achieving
outcome-oriented goals. Measuring performance allows organizations to
track the progress they are making toward their goals and gives managers
critical information on which to base decisions for improving their
progress. We have previously reported on some of the most important
attributes of successful performance measures. These attributes indicate
that performance measures should (1) be linked to an agency's mission and
goals; (2) be clearly stated; (3) have quantifiable targets or other
measurable values; (4) be reasonably free of significant bias or
manipulation that would distort the accurate assessment of performance;
(5) provide areliable way to assess progress; (6) sufficiently cover the
program's core activities; (7) have limited overlap with other measures;
(8) have balance or not emphasize one or two priorities at the expense of
9UL certifies control centers that provide all elements of service
required by UL's standards, including appropriate operator response to
fire alarm signals and proper equipment inspection, testing, and
maintenance.
10"Failover" is the capability to switch over automatically to a redundant
or standby system in the event of a system failure. Each MegaCenter tests
its ability to run its sister-center's operations in case that center has
system failure.
Page 7 GAO-06-1076 Federal Protective Service
others; and (9) address governmentwide priorities of quality, timeliness,
efficiency, cost of service, and outcome.11
We assessed the 11 FPS MegaCenter performance measures against selected
attributes: linkage to mission and goals, clarity, and measurable targets.
Ten of the 11 MegaCenter performance measures were aligned with FPS's
mission to protect federal properties and personnel and with the
MegaCenter program's mission to provide high-quality and standardized
alarm monitoring, radio monitoring, and dispatch. We found no link between
timely review of contractor time sheets and billing statements and FPS's
mission, however, primarily because this measure seems to be related to
administrative activities. In addition, while 6 of the 11 performance
measures have measurable targets-a key component for measuring
performance, none of the MegaCenter performance measures met the clarity
attribute because FPS could not provide information about how managers
calculate the measures-a key component in the clarity attribute. For
example, the performance measure that the centers test the failover
ability of alarm, radio, and telephone systems weekly is measurable
because it has a quantifiable target but does not meet the clarity
attribute because FPS could not describe its methodology for calculating
it.
We also assessed whether, collectively, the MegaCenters' 11 performance
measures sufficiently cover their core program activities (i.e., alarm
monitoring, radio monitoring, and dispatch) and address governmentwide
priorities of quality, timeliness, efficiency, cost of service, and
outcome. Most of the MegaCenter performance measures relate to the three
core activities. For example, regular feedback on customer service and
monthly review of operator calls cover aspects of the dispatch and
radio-monitoring functions. Other performance measures, like distributing
emergency notification reports in 30 minutes, help fulfill other critical
support functions. However, two performance measures-reviewing contractor
quality assurance plans and timely review of contractor time sheets and
billing statements-relate to administrative activities that are not
strictly related to MegaCenter core activities. Additionally, the
MegaCenter performance measures do not collectively address all of the
governmentwide priorities. The MegaCenter performance measures primarily
address the governmentwide priorities of quality and timeliness.
11See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing
Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002),
pp.2-3, 46-53.
Page 8 GAO-06-1076 Federal Protective Service
For example, the MegaCenter measures pertaining to transcribing reports
within 8 hours and reviewing recorded calls to see if the operator
followed standard operating procedures address aspects of service
timeliness and quality, respectively. None of the measures relate to the
governmentwide priorities of efficiency, cost of service, and outcome.
Finally, FPS does not link MegaCenter performance measures to FPS-wide
performance measures, specifically the patrol and response time measure.
FPS established FPS-wide performance measures to assess its efforts to
reduce or mitigate building security risks. The performance measures that
FPS established were (1) timely deployment of countermeasures, (2)
functionality of countermeasures, (3) patrol and response time, and (4)
facility security index. The one measure that relates to the MegaCenters-
patrol and response time-assesses FPS's ability to respond to calls for
service and measures the average elapsed time from when a law enforcement
request is received (e.g., alarm, telephonic request from a building
tenant, FPS police officer-initiated call) to the time an officer arrives
at the scene. FPS's goal is to reduce response times by 10 percent in
fiscal year 2006. The MegaCenters are responsible for part of the patrol
and response activity that is being measured because the MegaCenters
receive alarms and emergency calls and dispatch FPS police officers or
contract guards to the scene. However, although data pertaining to this
activity exist in the MegaCenters' records management system, they do not
measure the timeliness of this activity, and FPS has not developed a
performance measure that would identify the MegaCenters' contribution
toward meeting FPS's measure.
Nine Selected Organizations Provide Some of the MegaCenters' Services in a
Generally Comparable Manner
The nine selected security organizations generally do not provide all
three of the MegaCenters' primary services. However, the services these
organizations offer are provided similarly by the MegaCenters with the
exception of a CAD system, which three organizations use and the
MegaCenters do not. The MegaCenters provide three primary services (i.e.,
alarm monitoring, radio monitoring, and dispatch), and the selected
organizations provide all or some of these three main services. For
example, the Park Police provide all three services, while the private
organizations focus on providing alarm monitoring and offer some services
the MegaCenters do not. Like the MegaCenters, all of the private
organizations reviewed have centralized operations: the number of their
national control centers ranges from two to five. Work allocation (i.e.,
how incoming alarms and calls are assigned) among centers varies by
organization but overall is similar to the MegaCenter structure. For
example, most of the organizations assign calls and alarms to a specific
center based on the geographic location of the call or signal. However,
the Postal Inspection Service and one private organization are unique
because they are able to allocate workload to centers based on demand and
operator availability. The organizations use a variety of methods to
measure the quality of their services, many similar to methods used by the
MegaCenters. For example, like the MegaCenters, most review a sample of
operator calls on a regular basis. Two entities have established
measurable performance goals for their centers. While there are
similarities in the services offered, number of centers, work allocation,
and service quality appraisals between the organizations reviewed and the
MegaCenters, three organizations use a CAD system, which the MegaCenters
do not. A CAD system is a tool used by the Denver Police Department for
dispatching and officer tracking and by the Postal Inspection Service for
officer tracking. The Park Police also uses a CAD system with limited
capabilities at its San Francisco center and plans to purchase and upgrade
the system for all three of its centers. Selected organizations and
associations referred to CAD systems as being beneficial for dispatching
services by allowing for faster operator response, automatic operator
access to standard operating procedures and response prioritization, and
automatic recording of operator actions enabling easier performance
analysis. Since 2003, FPS and DHS both have assessed MegaCenter technology
and have identified needs for technology upgrades, including the
installation of a CAD system for the MegaCenters. Our guide on IT
investment decision making-based on best practices in the public and
private sector-stresses that part of achieving maximum benefits from an IT
project requires that decisions be made on a regular basis about the
status of the project.12 To make these decisions, senior managers need
assessments of the project's impact on mission performance and future
prospects for the project. While the MegaCenters have assessed their
technology on many occasions and have determined that some refreshment is
needed, FPS has not yet allocated the funding for such upgrades.
Conclusions
FPS MegaCenters play a key role in protecting federal facilities, those
who
enter these facilities, and the FPS police officers and contract guards
whose calls the MegaCenters respond to and monitor. How well the
MegaCenters are fulfilling their role and carrying out their
responsibilities
12GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
D.C.: February 1997).
Page 10 GAO-06-1076 Federal Protective Service
is uncertain because they do not generate much of the information that
would be useful for assessing their performance.
To their credit, the MegaCenters have established performance measures for
a number of their activities and operations, and these measures are
aligned with the MegaCenters' mission. However, the measures have
weaknesses, both individually and collectively, compared with the selected
attributes of successful performance measures that we have identified.
Many of the individual measures are neither quantifiable nor clearly
stated, and collectively the measures do not address the governmentwide
priorities of efficiency, cost of service, and outcome. As a result, FPS
cannot compare performance across the MegaCenters or over time, and
without such information, FPS is limited in its ability to identify
shortfalls and target improvements.
Although FPS has established an FPS-wide performance measure for response
time-from the alarm to the FPS police officer's arrival on the scene-that
incorporates the MegaCenters' operations, the MegaCenters have not
established a comparable measure for their operations alone. Without such
a measure, FPS cannot evaluate the MegaCenters' contribution-from the
alarm to the FPS police officer's dispatch-to the FPS-wide measure for
response time and identify opportunities for improvement.
Recommendations for Executive Action
We recommend that the Secretary of Homeland Security direct the Director
of the Federal Protective Service to take the following three actions:
o establish MegaCenter performance measures that meet the attributes of
successful performance measures we have identified;
o develop a performance measure for the MegaCenters that is directly
linked to the FPS-wide response time measure and covers the scope of
the MegaCenters' operations, from alarm to dispatch; and
o routinely assess the extent to which the MegaCenters meet established
performance measures.
Agency Comments
We provided a draft of this report to DHS, the Department of the Interior,
and the U.S. Postal Service for their review and comment. DHS provided
comments in a letter dated September 6, 2006, which are summarized below
and reprinted in appendix II. DHS also provided technical comments, which
we incorporated into the report where appropriate. The Postal Service
informed us that it had no comments on this report. The Department of the
Interior did not provide comments on this report.
DHS generally agreed with the report's findings and recommendations. DHS
stated that FPS and the U.S. Immigration and Customs Enforcement (ICE)
have undertaken a comprehensive review of the MegaCenters to identify,
among other things, ways in which performance can be better measured. DHS
noted that through this broad approach, FPS personnel will be able to
generate and track the kind of information necessary to assess the
MegaCenters' performance. This one-time review may help FPS identify
information needed to assess the MegaCenters' performance and, therefore,
develop appropriate performance measures. In order to reliably assess
performance over time, FPS should not only establish appropriate
performance measures, but also routinely assess performance using these
measures. We therefore clarified our recommendation to include the routine
use of established performance measures to assess the MegaCenters'
performance. With regard to the report's discussion of CAD system
capabilities, DHS said that ICE's Chief Information Officer is currently
assessing the MegaCenters' technology requirements and recognizes that
previous studies have identified the need for technology upgrades. DHS
indicated that the current assessment will have a meaningful impact on
FPS's technology capabilities.
As agreed with your office, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from
the report date. At that time, we will send copies of this report to other
interested congressional committees and the Secretary of Homeland
Security, and DHS's Assistant Secretary for Immigration and Customs
Enforcement. We will also make copies available to others upon request. In
addition, the report will be available at no charge on the GAO Web site at
http://www.gao.gov.
If you have any questions regarding this report, please contact me at
(202) 512-2834 or [email protected]. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this report. GAO staff who made major contributions to this report are
listed in appendix III.
Sincerely yours,
Mathew J. Scire Acting
Director, Physical Infrastructure Issues
Appendix I
Briefing for the Committee on Government Reform, U.S. House of Representatives,
August 14, 2006
Review of the Federal Protective Service's MegaCenters
Briefing for the Committee on Government Reform
U.S. House of RepresentativesAugust 14, 2006
o Introduction and Objectives
o Scope and Methodology
o Results in Brief
o Background
o Results of GAO Work
o Since the 1995 bombing of the Alfred P. Murrah Federal Building in
Oklahoma City and the September 11, 2001, attacks on the World Trade
Center and Pentagon, terrorism has threatened the nation's security,
including the physical security of federal facilities.
o The Homeland Security Act of 2002 created the Department of Homeland
Security (DHS), a new federal department with the mission of
preventing terrorist attacks within the United States, which includes
safeguarding federal facilities.1
o DHS, through its Federal Protective Service (FPS), provides law
enforcement and security services to federal agencies that occupy
almost 9,000 facilities under the jurisdiction of the General Services
Administration (GSA) and DHS, protecting millions of federal
employees, contractors, and citizens. Under agreement, FPS authority
can be extended to provide its law enforcement and security services
to any property with a significant federal interest.
o As part of its approach to facility protection, FPS provides support
for its law enforcement and security services through four control
centers (known as MegaCenters) located in Battle Creek, Michigan;
Denver, Colorado; Philadelphia, Pennsylvania; and Suitland, Maryland.
1Facilities include government owned and lease space.
* Because of the important role MegaCenters play in assuring the
safety of federal facilities and their occupants, our objectives
were to:
* Identify the services the MegaCenters provide and how they
provide them.
* Determine how FPS assesses and measures the performance of
MegaCenter operations and how FPS links MegaCenter performance
measures to FPS-wide performance measures.
* Examine how the MegaCenters compare to selected security
organizations in the services they provide and in the methods
they use to provide them.
* Document review: Reviewed the Memorandum of Agreement between GSA
and FPS and other documentation related to MegaCenter services as
well as documentation related to (1) FPS's request for a computer
aided dispatch (CAD) system for the MegaCenters; (2) past FPS
assessments of MegaCenter operations; (3) FPS's performance
measures; and
* (4) FPS'sbudget for the MegaCenters.
o Interviews: Interviewed FPS officials, including MegaCenter branch
chief and managers, and staff from the Program Review Office,
Financial Management Division, and other offices; Immigration and
Customs Enforcement's (ICE) Budget Enforcement Office; and officials
from selected public and private organizations; officials from
security industry standard setting and accreditation associations
(associations).
* Selected organizations:
o U.S. Customs and Border Protection
o U.S. Park Police
o U.S. Postal Inspection Service
o Denver Police Department
o 5 private security companies
o We conducted our review in accordance with generally accepted
government auditing standards.
o Remote monitoring of building alarm systems, radio monitoring, and
dispatching of FPS police officers and contract guards are the primary
services FPS MegaCenters provide. These and other services are
provided around the clock from four locations across the country. Each
MegaCenter has a sister center with redundant capabilities that can
serve as an emergency backup and each is operated by full-time federal
employees and private contractors. In addition, the MegaCenters have a
fiscal year 2006 budget of $23.5 million and use a variety of
information technology (IT) systems and other equipment to provide
their services.
* FPS MegaCenter managers assess MegaCenter operations through a
variety of means, including reviewing information on the
timeliness and volume of operations, listening to and evaluating
a sample of calls between operators and FPS police officers and
contract guards, and receiving informal feedback about customer
satisfaction. FPS managers have also developed performance
measures for assessing MegaCenter operations. Although these
MegaCenter measures reflect some attributes of successful
performance measures, they also contain some weaknesses because
they are not always clearly stated or measurable, and do not
address governmentwide priorities of efficiency, cost of service,
and outcome. In addition, they do not directly measure key
operations that would link to FPS-wide performance measures,
which are (1) the timely deployment of countermeasures, (2)
functionality of countermeasures, (3) patrol and response time,
and
* (4) facility security index.
o The nine selected organizations offer some of the MegaCenters' primary
services, and they deliver and assess the services they offer in a
generally similar manner to the MegaCenters. For example, like the
MegaCenters, many of these organizations have centralized their
control center operations, have backup capability, allocate workload
among control centers based on geographic location, and use regular
call reviews as well as volume and time measures to assess the quality
of the services they provide. A few organizations offer services the
MegaCenters do not offer. One difference between the MegaCenters and
the selected organizations is that three of these organizations use a
CAD system, which the MegaCenters do not have. The MegaCenters have
assessed their technology and have identified the need for a CAD;
however FPS has not allocated funds for such a purchase.
* FPS operations are solely funded through security fees and
reimbursements collected from federal agencies for FPS security
services.
* These security fees consist of basic and building-specific
security charges.
o The basic security charges cover the security services
that FPS provides to all federal tenants in
FPS-protected buildings, which include such services
as patrol, monitoring of building perimeter alarms and
dispatching of law enforcement response (MegaCenter
operations), criminal investigations, and security
surveys.
o The building-specific security charges are for FPS
security measures that are designed for a particular
building and are based on the FPS Building Security
Assessment and its designated security level. Such
measures include contract guards, X-ray machines,
magnetometers, cameras, and intrusion detection
alarms. Also, the tenant agencies may request
additional security services such as more guards,
access control systems, and perimeter barriers.
o The above two charges are billed monthly to the tenant
agencies. The basic security charge is the same for
all tenants regardless of the type of space occupied
and is a square footage rate. The building-specific
security charge reflects FPS cost recovery for
security measures specific to a particular building
and the billing is handled differently for single-and
multi-tenant buildings. Single tenant buildings-the
tenant agency is billed for the total cost of the
security measures. Multi-tenant buildings-the tenant
agencies are billed based on their pro rata share of
the square feet occupied within the respective
building.
o FPS uses a reimbursable program to charge individual agencies
for additional security services and equipment that they
request above the level determined for their building.
* FPS bills the tenant agencies for FPS security fees they have
incurred.
o The agencies pay the fees into an FPS account in the
Department of the Treasury, which is administered by FPS.
Congress exercises control over the account through the annual
appropriations process that sets an annual limit-called
obligation authority-on how much of the account FPS can expend
for various activities.
o FPS uses the security fees to finance its various activities
within the limits that Congress sets. The Department of
Homeland Security Appropriations Act for fiscal year 2006
authorized $487 million in obligation authority for FPS
expenses and operations. Through FPS's security fees, funds
are to be collected and credited to FPS's account as an
offsetting collection from tenant agencies.
o Under the FPS reimbursable program, agencies request additional
security services and equipment using a funded Security Work
Authorization. Once the services are provided and the costs are
expensed, FPS bills the agency for the costs, and the funds are
transferred to the FPS account to offset the expenses FPS incurred.
o The DHS Inspector General reported in 2006 that when FPS was part of
GSA it budgeted and paid for FPS's annual administrative support costs
such as financial management, human capital, and IT using funds beyond
those generated by security fees. GSA estimated these FY 2003 support
services to cost about $28 million. According to the report, beginning
in FY 2004, neither DHS's annual budget request nor DHS's
appropriations set aside funding for FPS's support services. In FY
2004, as a component of DHS, FPS paid almost $24 million for support
services using funds from security fees only; a year earlier these
services had been funded by GSA using funds not derived from fees.
o Before GSA established the MegaCenters, FPS used regional and
satellite control centers to monitor alarm systems, dispatch FPS
police officers and contract guards, and perform criminal background
checks. In total, there were 22 regional control centers and 12
satellite control centers, which were located throughout FPS's 11
regions. Most regions had more than 1 control center.
o In 1991, GSA conducted an internal review of the control centers. The
review found that because of significant budgetary and personnel
constraints over more than a decade, the control centers no longer
performed well enough to ensure safe, effective, and efficient FPS
actions to preserve life and property. GSA contracted with Sandia
National Laboratories- the lead laboratory for U.S. Department of
Energy security systems- to conduct an in-depth study of the control
centers' operation and make recommendations.
o In 1993, Sandia issued its study entitled GSA Control Center Upgrade
Program. The Sandia study identified serious shortfalls and problems
that would require a more radical upgrade of the control centers at a
much higher cost than originally believed. After validating the
study's findings, GSA determined that a multimillion dollar upgrade of
all control centers would be prohibitively expensive. The study noted
that the control centers could be consolidated to almost any level to
achieve economies of scale. However, the study recommended against a
single national-level control center because a second center would be
needed to continue operations under catastrophe or failover
conditions.
o GSA concluded that the control center problems that the study
identified were material weaknesses and reported them to Congress. FPS
conducted an operational and technical review of the Sandia study's
findings, which provided a critical assessment of the control centers,
a high-level concept of operations for the centers, and functional
specifications for upgrading the centers. GSA decided to upgrade 11
control centers-one in each region-and address the weaknesses that the
study had identified.
o Within GSA, concerns were raised about the cost of upgrading 11
control centers, how many control centers were really needed, and
whether the centers' operations could be outsourced. GSA established a
project team to investigate these concerns. The team contacted several
public and private sector organizations that operate control centers.
The team found that the organizations were consolidating their control
centers but were unable to assume the operations of FPS control
centers. A decision was made to consolidate additional centers and the
multi-regional control center or "MegaCenter" concept was developed.
GSA endorsed the MegaCenter concept. GSA assembled a core project team
and hired contractors to design, plan, and supervise the construction
of the centers.
o In 1994, GSA issued a bid for MegaCenter technical and performance
specifications and awarded the contract.
o In 1996, FPS reaffirmed that the MegaCenter concept was the best
approach for addressing the control center weaknesses. GSA selected
the MegaCenters sites: Denver, CO; Battle Creek, MI; New York, NY and
Suitland, MD.2
o In 1996, construction began on the Denver MegaCenter and design was
initiated on the Battle Creek MegaCenter.
o In 1997, the Denver MegaCenter was opened, followed by Battle Creek in
1999, Suitland in 2000, and Philadelphia in 2001.
2Site selection for New York, NY was never finalized and eventually was
switched to a site in Philadelphia, PA.
o The MegaCenters' mission is to provide the highest quality, nationally
standardized dispatch, alarm monitoring, and federal law enforcement
emergency response services. Based on this mission statement, we chose
to focus on alarm monitoring, radio monitoring, and dispatch as the
primary services the FPS MegaCenters provide.
* Primary services:
o Alarm monitoring: monitoring intrusion, panic, fire/smoke,
elevator, and/or environmental alarms.
o Radio monitoring: monitoring FPS police officers' and
contract guards' radio communication for safety and
providing information upon request.
o Dispatch: exercising command and control authority by
dispatching FPS police officers and/or contract guards.
Objective 1: MegaCenters provide other services
* Other services:
o Notifying federal agencies regarding national emergencies and
facility problems (also known as SPOT reports)
o Checking criminal background histories (including inquiries to
the National Crime Information Center database, which the
Federal Bureau of Investigation (FBI) maintains) for FPS
police officers responding to a call or an alarm or FPS
regions and other DHS agencies requesting this assistance
o Receiving and transcribing FPS police officer reports
o Providing a toll-free help desk line for agency support,
including remote diagnosis and service of alarms
o Updating quarterly building and emergency contact information
from customer agencies to ensure accurate notifications and
alerts
* Unique services (provided by individual MegaCenter):
o Monitoring and controlling access to buildings
o Using the Internet to program, monitor, and test alarms
o Providing after-hours telephone answering service for Drug
Enforcement Administration and for GSA maintenance
Table 1: Services Provided to MegaCenter Clients
Services GSA, DHS, and FPS regions,
federal agency police officers,
clients and contract
guards
Monitor remote building alarms
X X
Dispatch FPS police officers, contract
guards,
and other response services X X
Monitor FPS police officers' and contract
guards' radio communications for their X
safety
Keep track of FPS police officer and
contract
guard locations X
Provide warrant and other information from
the National Crime Information Center X X
Help maintain lists of buildings and
contacts
for emergency notifications X X
* MegaCenters are located in four cities and are responsible for
providing various services in their respective FPS Regions:
o Philadelphia, PA (FPS Regions 1, 2, 3)
o Battle Creek, MI (FPS Regions 4, 5, 6)
o Denver, CO (FPS Regions 7, 8, 9, 10)
o Suitland, MD (FPS Region 11-National Capital Region)
* Each MegaCenter is paired with a sister center:
o Suitland is paired with Battle Creek
o Philadelphia is paired with Denver
o Sister center pairings provide for redundant capability in case of a
catastrophic failure at any MegaCenter.
o MegaCenters rely upon contractor staff to carry out dispatch and
technical support services.
o Each MegaCenter also has FPS officials located on site to oversee the
center's overall operations.
o MegaCenters operate around the clock.
Table 2: FY 2006 Actual Federal and Contractor Staffing Levels at Each
MegaCenter
MegaCenter Federal staff Primary Contractor Contractor staff
Battle Creek 9 Wackenhut Services Inc.a 74
Denver 4 Gonzales Consulting Servicesa 56
Philadelphia 6 Gonzales Consulting Services 49
Suitland 4 Gonzales Consulting Services 38
Total 23 217
aThe Battle Creek and Denver MegaCenters have other contracts for
personnel to provide services, such as technical support, however, these
contracts are small in terms of the number of personnel provided- three in
Denver and five in Battle Creek.
Objective 1: Historical budget and MegaCenter allocations
* Recent FPS MegaCenter budgets:
o Fiscal year 2004 $20.0 million
o Fiscal year 2005 $21.5 million
o Fiscal year 2006 $23.5 million, which accounts for less than 5
percent of FPS's total budget.4
* Estimated fiscal year 2006 budget for each MegaCenter:
o Suitland $6.0 million
o Philadelphia $5.0 million
o Battle Creek $6.5 million
o Denver $6.0 million
o MegaCenter cost information for FY 2004 and prior years is not
available since FPS transitioned from GSA's accounting system. FY 2005
and FY 2006 cost information is also not available because the ICE
accounting system was not modified to capture costs solely for the
MegaCenters. According to FPS, it is working with ICE to establish the
capability in the accounting system to capture MegaCenter costs in the
future.
4As of August 2006, MegaCenter officials reported a revised MegaCenter
budget of approximately $19 million for fiscal year 2006.
Objective 1: MegaCenters rely on IT systems to deliver services
o MegaCenters depend on a variety of IT systems and equipment to deliver
their primary services.
Table 3: Information Technology Systems That Support MegaCenters' Primary
Services
Information technology systems Alarm monitoring Radio monitoring Dispatch
Security Information System and
software (alarm receivers and
signal receivers) X X X
Facility and building client
enterprise information systems X X X
MegaCenter-owned telephone
exchange system and regional X X X
and national toll-free
emergency numbers
Remote programming software X
Radio systems X X X
Patrol and dispatch operations
logs X X X
Voice audio recorders X X X
Software that allows
MegaCenters to access federal X X
and state law enforcement
databases
Source: GAO analysis of FPS data
o MegaCenters also depend on additional information technology systems,
such as failover equipment and servers that facilitate sharing files
with sister centers.
o Reports on operator activities-FPS MegaCenter supervisors-staff who
are responsible for overseeing operators-review information about the
timeliness and volume of operators' activities. For example, they
review reports that describe how long it took operators to send out
emergency notifications and transcribe dictated reports and the number
of problem alarms, among other things.
o Assessments of operator communications-Supervisors and designated
quality assurance staff listen to live conversations between operators
and FPS police officers and contract guards as well as regularly
listen to a sample of taped conversations to identify whether
operators are following standard operating procedures. According to
MegaCenter managers, staff sample taped calls on a monthly basis.
o Comprehensive reviews of MegaCenter operations-In 2003 and 2004, the
MegaCenter managers completed a manager review of each MegaCenter.
These reviews were replaced by more comprehensive program reviews in
which FPS regional staff with subject area expertise were to review
each MegaCenter and report on concerns and best practices related to
MegaCenter management, administration, technology, and equipment. In
2005, the Philadelphia MegaCenter was the first and only MegaCenter to
undergo a program review. Program reviews were to be conducted
annually at each MegaCenter, however, the MegaCenter managers
suspended these reviews due to budgetary constraints.
o Feedback on customer satisfaction-MegaCenter managers and supervisors
use informal means for gathering information about the level of
customer satisfaction. For example, they receive information from
conversations with FPS region program staff and FPS police officers
who, according to MegaCenter managers, do not hesitate to inform them
of performance concerns. Managers also obtain feedback from federal
building tenants and agencies during routine activities, such as when
they update their emergency contact database, and from regional staff
by attending regional staff meetings.
o Underwriters Laboratories (UL) control center certification-Although
not an assessment by the MegaCenter staff themselves, the UL
certification process involves feedback from UL inspectors about
whether the MegaCenter meets technical and performance standards. UL
inspectors complete initial inspections to certify the MegaCenters and
conduct regular inspections once a center is certified. The Denver
MegaCenter was UL certified in 2003 and has since had a subsequent
inspection. The Philadelphia MegaCenter's initial UL inspection is
scheduled to be completed in August 2006. Initial UL inspections at
the remaining two MegaCenters have not been scheduled.
o Performance measurement-FPS has established performance measures for
the MegaCenters.
* FPS has identified 11 performance measures for MegaCenter
operations.
o Distribute emergency notification reports (SPOT reports)
within 30 minutes of notification.
o Review problem alarm reports daily.
o Obtain regular feedback about customer satisfaction from
field operations.
o Continuously review all SPOT reports and other outgoing
information to ensure 100 percent accuracy.
o Transcribe dictated offense and incident reports into
database management system within 8 hours of receipt of the
report.
o Submit reviewed contractor billing reports and time sheets
within 7 business days after the last day of the month.
o Prepare and review contractor reports for quality assurance
plan.
o Maintain completely accurate (non-duplicative) case control
numbers.
o Meet UL guidelines and requirements continuously.5
o Test failover of alarm, radio, and telephone systems
weekly.6
o Monitor calls and review recorded call content for adherence
to standard procedures at least monthly.
5UL certifies centers that provide all elements of service required by
UL's standards, including appropriate operator response to fire alarm
signals and proper equipment inspection, testing, and maintenance.
6Failover is the capability to switch over automatically to a redundant or
standby system in the event of a system failure. Each MegaCenter tests its
ability to run 26 its sister-center's operation in case that center has
system failure.
o The Government Performance and Results Act of 1993 requires federal
agencies to develop strategic plans, link them with outcome-oriented
goals, and measure agency performance in achieving these goals.
Measuring performance allows organizations to track the progress they
are making toward their goals and gives managers critical information
on which to base decisions for improving their progress.
* Organizations need to have performance measures that (1)
demonstrate results, (2) are limited to the vital few,
* (3) cover multiple program priorities, and (4) provide useful
information for decision making in order to track how their
programs and activities can contribute to attaining the
organization's goals and mission.
* We have previously reported on some of the most important
attributes of successful performance measures. These attributes
indicate that performance measures should:
* Be linked to an agency's mission and goals;
* Be clearly stated: a measure's name and definition are
consistent with the methodology used to calculate it;
* Have quantifiable targets or other measurable values;
* Be reasonably free of significant bias or manipulation that
would distort the accurate assessment of performance;
* Provide a reliable way to assess progress;
* Sufficiently cover the program's core activities;
* Have limited overlap with other measures;
* Have balance or not emphasize one or two priorities at the
expense of others; and
* Address governmentwide priorities of quality, timeliness,
efficiency, cost of service, and outcome.
o We assessed the 11 FPS MegaCenter performance measures against
selected attributes of successful performance measures: linkage to
performance goals and mission, clarity, and measurable targets. We
also assessed whether collectively the 11 performance measures
sufficiently cover the MegaCenters' core program activities and
address governmentwide priorities of quality, timeliness, efficiency,
cost of service, and outcome.
o Ten of the eleven MegaCenter performance measures were aligned with
FPS's mission to protect and secure federally owned and leased
properties and personnel and with the MegaCenter program's mission to
provide high quality and standardized alarm monitoring, radio
monitoring, and dispatch. For example, distributing emergency
notification reports (SPOT reports) to federal agencies within 30
minutes helps protect federal personnel and properties by alerting
federal managers of suspicious activities near their locations. We
could find no link between timely review of contractor time sheets and
billing statements and FPS's mission primarily because this measure
seems to be related to administrative activities.
o MegaCenter performance measures lacked clarity and measurable targets.
Although some of the measures seemed clearly stated, FPS could not
provide information about how managers calculate any of the measures-a
key component in the clarity attribute. Six of eleven performance
measures have measurable targets-a key component for measuring
performance. For example, the MegaCenter measure for testing the
failover ability of alarm, radio, and telephone systems on a weekly
basis is measurable because it has a quantifiable target but does not
meet the clarity attribute because FPS could not describe the
methodology for calculating this measure.
o Seven of the eleven MegaCenter performance measures relate to the
MegaCenters' core activities. For example, daily review of problem
alarm reports and weekly failover of alarm systems relate to the
MegaCenters' alarm monitoring operation. Regular feedback on customer
service and monthly review of operator calls cover aspects of the
radio monitoring and dispatch functions. Other performance measures,
like distributing emergency notification reports in 30 minutes, help
fulfill other critical support functions. However, two of the
performance measures-the timely review of contractor time sheets and
preparing and reviewing contractor quality assurance plans-seem to
relate to administrative activities and are not strictly related to
MegaCenter core activities.
o The MegaCenter performance measures primarily address the
governmentwide priorities of quality and timeliness. For example,
transcribing reports within 8 hours and reviewing recorded calls to
see if the operator followed standard procedures address aspects of
service timeliness and quality, respectively. None of the measures
relate to the other governmentwide priorities of efficiency, cost of
service, and outcome.
o FPS has identified four performance measures to assess its efforts to
reduce or mitigate building security risks.
o One of these measures-patrol and response time-assesses FPS's ability
to respond to calls for service within certain time limit goals. It
measures the average elapsed time from when a MegaCener receives a law
enforcement request (e.g., an alarm, telephone request from a building
tenant, FPS officer initiated call) to the time an FPS officer arrives
at the scene. In fiscal year 2005, FPS reported a national average
response time of 47 minutes. Its goal is to reduce response time by 10
percent in fiscal year 2006.
Table 4: FPS-wide Performance Measures for Facility Protection
Performance measure Purpose
Timely deployment of countermeasures To compare actual deployment dates
with planned deployment dates
Countermeasure functionality (e.g., To gauge whether those security
surveillance cameras, X-ray machines) countermeasures for which FPS is
responsible are working as intended,
once deployed
Patrol and response time To assess FPS's ability to respond
to calls for service within certain
time limit goals
Facility security index To calculate FPS's average success
rate for the above three performance
measures
Source: GAO, Homeland Security: Guidance and Standards Are Needed for
Measuring the Effectiveness of Agencies' Facility Protection Efforts,
GAO-06-612 (Washington, D.C.: May 2006).
o FPS does not link the MegaCenter performance measures to the FPSwide
performance measures, specifically, the FPS-wide patrol and response
time measure.
o The MegaCenters are responsible for part of the patrol and response
activity that is being measured because MegaCenter operators are in
control of the alarm and call response and dispatch operations.
However, although some time-related data is recorded in their records
management system, the MegaCenters do not measure the timeliness of
this activity and FPS has not developed a performance measure that
would identify the MegaCenters' contribution toward meeting the
FPSwide measure.
o The Park Police was the only selected organization that provides all
three primary services offered by the MegaCenters. The remaining
selected organizations provide one or two of these MegaCenter primary
services.
Table 5: MegaCenters' Primary Services Offered by Selected Organizations
Organization Alarm monitoring Radio monitoring Dispatch
Federal Protective Services X X X
U.S. Customs and Border X
Protection
U.S. Park Police X X X
U.S. Postal Inspection Service X X
Denver Police Department X X
Private security organizations X
* Selected organizations also offer other services the MegaCenters
offer.
o Many organizations record telephone and radio communications
in case these communications need to be reviewed; officials
from the associations stated that reviewing calls to ensure
procedures are followed correctly is an important control
center practice.
o All public organizations provide information to officers based
on inquires to the FBI's National Crime Information Center
among other databases.7
o All private organizations provide access control services that
allow them to track and restrict facility access.
o One private organization monitors system conditions separately
to distinguish between an intrusion, door propped open,
tamper, or long access, which assists in determining the
appropriate action to take.
o A few organizations offer services the MegaCenters do not.
o Three private organizations provide audio and/or video monitoring
services which allow operators to remotely listen and/or view live audio
and/or video transmission from the secured site. Officials from two
organizations reported that this type of surveillance technology minimizes
the number of on-site guards required for their clients. In addition, this
surveillance technology provides operators with additional intelligence
information to help them decide what further actions to take.
7Like the MegaCenters, Customs and Border Protection, Postal Inspection
Service, and Park Police also retrieve information for officers from the
National Law Enforcement Telecommunication System and the Denver Police
Department retrieves information for officers from the Colorado 33 Crime
Information Center database.
o Workloads and staffing vary by public organization. While Postal
Inspection Service monitors more facilities than the MegaCenters, it does
not dispatch officers. Park Police, an organization that does dispatch
officers, has far fewer officers available to dispatch than the
MegaCenters and, because the Park Police has a different purpose, it
monitors far fewer facilities than the MegaCenters.
Table 6: Workload Statistics of Selected Public Organizations' Centers
Public Center Purpose Number Facilities Dispatchable Center
organization of monitored personnel staffinga
centers
Monitor alarm
Federal systems of federal 4 8,328 7,856 240
facilities and
dispatch
Protective FPS police officers
and contract guards
to these
Service facilities, if
necessary. Monitor
the radios of
officers
and guards to
ensure their safety
and to provide
information to
officers.
Monitor radios
U.S. Customs nationwide for all 1 Not Not 71
customs applicable
investigations
and Border at ports to ensure applicable
officer safety and
coordinate
Protection response, if
necessary.
U.S. Park Monitor alarms and
Police dispatch Park 3 875b 717 36
Police officers and
law enforcement
rangers to protect
national monuments
and parks. Monitor
officer radios to
ensure
officer safety and
provide information
to officers.
Monitor postal Not
U.S. Postal facilities' alarm 2 11,448 applicable 67
systems and radio
Inspection communication of
Postal Inspection
police and federal
Service agents to ensure
officer safety and
to provide
information to
officers.
Denver Provide emergency
Police services to the 1 Not 1,452 129c
city and county of
Department Denver. applicable
aThis includes both full-time federal employees and contract staff, where
applicable.
bThis is the approximate number of alarms monitored by Park Police's two
centers in the District of Columbia and San Francisco. The New York
control center does not monitor alarms. cOne hundred and twenty-nine is an
approximation. Denver Police Department reported 131 personnel, 4 are part
time.
o The selected organizations have organizational structures similar to
the MegaCenters.
* Like the MegaCenters, all five private organizations monitor alarms
through centralized control centers.
o Each of the five private organizations have between 2 to 5
national central control centers.
o One private organization official stated that centralizing and
consolidating monitoring services allows for easier staffing,
better customer service, and higher quality technology.
o Like the MegaCenters, all public and private organizations have backup
ability, with the exception of Customs and Border Protection.8
o Officials from the organizations and associations we interviewed cited
the importance of center redundancy in the event of catastrophic failure
at any one control center.
8Park Police's control center in the District of Columbia has backup
capabilities for all three primary services. The centers in San Francisco
and New York are able to backup their radio and call taking functions,
however, the San Francisco center is not able to backup its alarm
monitoring function.
* Similar to the MegaCenters, most of selected organizations assign
calls and alarms to a specific control center based on the
geographic location of the call or alarm signal.
* o Some organizations reported that this type of work allocation is
beneficial because it reduces stress for the operator and provides
better customer service. For example, one private organization
reported that its operators become familiar with both the people
who call the control centers and their locations which allows them
to provide better directions to responders.
* Postal Inspection Service and one private organization are able to
allocate workload to their control centers based on demand and
operator availability. For example, when an alarm signal or
telephone call comes into Postal Inspection Service, its software
decides which center should receive the signal or call based on
center workload.
* Both organizations reported efficiencies from this type of
work allocation.
o Postal Inspection Service reported a reduction in
required center staff.
o The private organization reported the ability to close
some of its control centers during non-peak hours,
reducing costs.
o Both organizations attribute the ability to allocate work in
this manner to more advanced technology.9
9Postal Inspection Service can only allocate work in this manner for its
alarm monitoring and call taking service. Radio signals must be assigned
to a specific center because the technology at the time Postal Inspection
Service built its control centers was not yet developed to allow radio 36
signals to be assigned to first available center.
o The selected organizations use a variety of methods to ensure centers
are meeting their goals and providing quality services, many similar to
the MegaCenters.
Table 7: Selected Organizations' Methods to Ensure Quality Service
Organization Regular Operator pay Volume Regular and formal
review of a incentives and/or time process for
sample of based on measures soliciting customer
operators performance feedback
calls
Federal X X
Protective
Service
U.S. Customs and X X
Border
Protectiona
U.S. Park Police
U.S. Postal X X
Inspection
Service
Denver Police X X
Department
Private security Xb X Xb Xb
organizations
aCustoms and Border Protection officials reported that they are in the
process of developing a formal customer survey tool. bIndicates that at
least one of the selected private security organizations used these
methods.
* Regular review of calls
* o Most officials from the associations we interviewed agreed that
competent control centers have a quality assurance process that
includes regular review of a random sample of calls to ensure that
the operators asked the correct questions and provided the correct
information when dispatching and to check the timelines and timing
of responses. FPS MegaCenter supervisors monitor each of their
operators calls on a monthly basis to ensure correct procedures are
followed.
* Pay incentives
* o One private organization not only conducts call reviews but also
ties pay incentives to its quality assessment program by linking
employee call review evaluations to spot awards and bonuses. None
of the federal agencies reviewed, including the MegaCenters, used
these types of pay incentives.
* Volume and time measurements
o Examples of measurements are the amount of time it takes to
answer a call, time taken to act on a call, and number of
complaints from responding officers. Although the MegaCenters
have volume and time measurements, they do not report the time
it takes from receipt of alarm signal to officer dispatch.
Some of the measures the MegaCenters use include the time
elapsed between an officer calling in an incident and its
entry into the records management system and the time it takes
to send emergency notification reports to higher management.
They also report volume statistics to their regions, such as
the number of cases they have opened in a month.
o Officials from Customs and Border Protection and the
Commission on Accreditation for Law Enforcement Agencies
stated that many volume and time measurements cannot be used
in law enforcement because timing can depend on circumstances.
For example, the time it takes a Customs and Border Protection
operator to clear a radio call would not be an appropriate
measure because some of their functions are more tactical in
nature- for example providing radio support in an undercover
investigation- and cannot accurately be measured with time and
volume statistics only.
* Customer satisfaction
o Two private organizations regularly and formally survey their
customers to find out if operators were pleasant, delivered
timely service, and seemed to be well trained and informed.
o Other organizations, like MegaCenters, use a more informal
process to gauge customer satisfaction. To formalize its
customer feedback, the Suitland MegaCenter developed a survey
to gauge officer satisfaction, however, it is waiting for
approval to use it.
* o Customs and Border Protection has hired a private consultant to
help them develop a customer satisfaction survey and believe its
use, in conjunction with volume statistics, will provide a more
meaningful measure of their service quality.
* Performance measures
o One private organization and the Postal Inspection Service not
only perform assessment activities but also have established
measurable performance goals for their control centers.
Examples of the private organization's goals are: resolution
of 90 percent of high priority events within 60 seconds and
resolution of 85 percent of inbound and technical assistance
specialist calls within 18 seconds.
o Postal Inspection Service has begun to establish volume and
time measurement goals, such as a 25 percent reduction in the
number of complaints in which standard operating procedures
were not followed and a 10 percent reduction in the time it
takes to answer radio calls. They intend to revise their goals
in fiscal year 2007 once they have actual experience against
which to set a benchmark.
o The MegaCenters have created performance measures, however,
they lack some attributes of successful performance measures,
such as measurable targets.
o The MegaCenters perform dispatch and incident management functions
manually while Denver Police Department uses a CAD system to perform
these functions and the Postal Inspection Service uses a CAD system to
track officer locations and perform incidentmanagement functions. The
Park Police's San Francisco center also uses a CAD system with limited
capabilities, and Park Police has a plan in place to purchase and
upgrade a system for its three centers.
o MegaCenter operators keep track of officers on duty and their
locations by hand or by using an Excel spreadsheet, and when they
dispatch an officer to a federal building they must enter the address
location into the database. In comparison, when a PostalInspection
Service operator tracks an officer to a building, the CAD system
automatically populates the address field and the system shows all
contact information for that building. Without a CAD system, the
operators spend more time retrieving information from different
sources and entering data, such as client contact information.
o CAD systems automate dispatch and incident management functions and
allow for more efficient handling of incidents. Typical CAD system
functions include management of call routing and prioritization,
dispatching, and response procedures. For example, a CAD system can
decide which control center a call should go to based on workload,
prioritize the call for the operator, and automatically display to the
operator the actions to take.CAD systems record times of incidents,
locations, and corresponding actions by dispatchers and officers which
allows for analysis to be conducted to determine, forexample, response
times, workload, the types of incidents requiring response,
andresource allocation needs.
o Selected organizations and associations stated that CAD systems are
beneficial for dispatching services by allowing for faster operator
response, automatic recording of operator actions enabling easier
performance analysis, and automatic operator access to standard
operating procedures and response prioritization. For example, when a
signal comes into a Postal Inspection Service control center, the CAD
system automatically retrieves and displays the response actions for
the operator to follow.
o FPS has repeatedly recognized the need for a CAD system. In 1993, the
Sandia study of FPS's former control centers recommended that the
centers have a CAD system for the most effective and time efficient
dispatch operations while using a minimum center personnel structure.
A 2003 DHS MegaCenter review also stated that the MegaCenters needed a
CAD system. In January 2006, FPS issued a request for information for
a CAD system. No funding has been allocated in fiscal year 2006 for
the MegaCenters to purchase a CAD system.
o Our guide on IT investment decision-making- based on best practices in
the public and private sectors- stresses that part of achieving
maximum benefits from an IT project requires that decisions be made on
a regular basis about the status of the project. To make these
decisions, senior managers need assessments of the project's impact on
mission performance and future prospects for the project.
o Senior managers should regularly question whether (1) the current
system meets organizational needs, (2) the system should be modified
to better meet these needs, (3) a new system is needed to best meet
these needs, or (4) the needs could best be met by outsourcing the
work. Included in the regular review should be costs for operation and
maintenance of the project, such as hardware upgrades, system software
changes, and ongoing user training. Successful IT management requires
that a plan be developed for the continued support and operation of
every IT project.10
* While the MegaCenters have assessed their technology on several
occasions and have determined that some refreshment is needed, no
funding has been allocated by FPS for this use.
o When the MegaCenters were established, FPS intended
scheduled replacement of their technical systems after 5
years, but this timeline has passed. In addition, a 2003 DHS
MegaCenter review found that there were various states of
technologies in systems across centers and proposed planning
for life cycle equipment replacement. However, according to
FPS officials, no technology replacement program has been
established, and equipment is replaced on a per MegaCenter,
as-needed basis.
o A 2005 program review of the Philadelphia MegaCenter found
that no strategic plan had been established to guide and
lead the MegaCenters into the future in terms of technology
and equipment. The review suggested that a national team be
assembled to focus on the MegaCenters' communications and
technology needs. Loss of IT positions at the MegaCenters
has prevented the MegaCenter branch chief from creating this
national team. However, FPS's IT program manager conducts
weekly teleconferences to discuss IT issues in the
MegaCenters, which MegaCenter IT staff attend. In addition,
the MegaCenter branch chief has developed a radio coverage
plan- his main technology priority for the MegaCenters-
which outlines plans to acquire technology systems that will
ensure the MegaCenters can receive radio signals for all
areas in the regions.11 There has been no funding to
implement this plan.
10GAO, Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, GAO/AIMD-10.1.13 (Washington,
D.C.: February 1997).
11Currently there are some areas where FPS police officers' and contract
guards' radios cannot transmit signals to the MegaCenters preventing the
MegaCenter operators from being able to monitor the radios.
o As part of the budget justification process, FPS has submitted the
required analysis and requested funding for a technology upgrade
project for each budget year from 2003 through 2006.12 FPS's proposed
project is to maintain, consolidate, standardize, and enhance current
and future FPS systems and integrate it with DHS systems. Included in
the request for each year is an Operations Reporting and Information
Network (ORION) system to be used at the MegaCenters that would
provide enhanced incident capture and tracking, officer tracking, and
officer safety features and includes a CAD system.
o Although FPS has developed this investment plan, no funds have been
allocated for ORION or any other MegaCenter technology improvements.
An FPS official stated that because of limited funding, the
MegaCenters are not investing in technology, and the only money being
spent on MegaCenter technology is for maintenance so they can maintain
current operations. Another official reported that under GSA, FPS was
given funds for technology investment but since moving to DHS, FPS has
not received these extra funds and must take money from its operating
budget to fund technology purchases.
12Each year agencies submit to the Office of Management and Budget a
Capital Asset Plan and Business Case to justify each request for a major
IT investment. 47
Appendix II
Comments from the Department of Homeland Security
Appendix III
GAO Contact and Staff Acknowledgments
Mathew J. Scire (202) 512-2834 or [email protected].
GAO Contact
Other key contributors to this report were Gerald P. Barnes, Assistant
Staff Acknowledgements
Director; Deirdre Brown; Bess Eisenstadt; Colin Fallon; Brandon Haller;
Richard Hung; Alex Lawrence; Gail Marnik; and Josh Ormand.
*** End of document. ***