Digital Service Programs: Assessing Results and Coordinating with Chief Information Officers Can Improve Delivery of Federal Projects
Highlights
What GAO Found
The General Service Administration's (GSA) 18F and Office of Management and Budget's (OMB) U.S. Digital Service (USDS) have provided a variety of services to agencies supporting their information technology (IT) efforts. Specifically, 18F staff helped 18 agencies with 32 projects and generally provided development and consulting services, including software development solutions and acquisition consulting. In addition, USDS provided assistance on 13 projects across 11 agencies and generally provided consulting services, including quality assurance, problem identification and recommendations, and software engineering. Further, according to GAO's survey, managers were generally satisfied with the services they received from 18F and USDS on these projects (see table).
Results of GAO Survey on Satisfaction with Digital Services Projects
|
Program |
Very satisfied |
Moderately satisfied |
Neither satisfied nor dissatisfied |
Moderately dissatisfied |
No response |
|
18F |
16 |
7 |
0 |
3 |
5 |
|
U.S. Digital Service |
6 |
3 |
0 |
0 |
4 |
Source: GAO survey of agency project managers that engaged with 18F and U.S. Digital Service. | GAO-16-602
Both 18F and USDS have partially implemented practices to identify and help agencies address problems with IT projects. Specifically, 18F has developed several outcome-oriented goals and related performance measures, as well as procedures for prioritizing projects; however, not all of its goals are outcome-oriented and it has not yet fully measured program performance. Similarly, USDS has developed goals, but they are not all outcome-oriented and it has established performance measures for only one of its goals. USDS has also measured progress for just one goal. Until 18F and USDS fully implement these practices, it will be difficult to hold the programs accountable for results.
Agencies are beginning to establish digital service teams. Of the 25 agencies included in the President's proposed funding for agency digital service teams, OMB has established charters with 6 agencies for their digital service teams. In addition, according to the Deputy USDS Administrator, USDS expects to establish charters with an additional 2 agencies by the end of the fiscal year—the Department of Education and the Small Business Administration. For the remaining 16 agencies, as of April 2016, 8 agencies reported that they plan to establish digital service teams but have yet to establish charters with USDS. The other 9 agencies reported that they do not plan to establish digital service teams by September 2016 and most noted that it was because they did not receive requested funding to do so. Further, of the 4 agencies GAO selected to review, only 1 has defined the relationship between its digital service team and the agency Chief Information Officer (CIO). This is due, in part, to the fact that USDS policy does not describe the expected relationship between CIOs and these teams. Until OMB updates its policy and ensures that the responsibilities between the CIOs and digital services teams are clearly defined, it is unclear whether CIOs will be able to fulfill their statutory responsibilities with respect to IT management of the projects undertaken by the digital service teams.
Why GAO Did This Study
In an effort to improve IT across the federal government, in March 2014 GSA established 18F, which provides IT services (e.g., develop websites) to agencies. In addition, in August 2014 the Administration established USDS, which aims to improve public-facing federal IT services. The President's Budget for fiscal year 2016 also proposed funding for agencies to establish their own digital service teams.
GAO was asked to review 18F and USDS. GAO's objectives were to (1) describe 18F and USDS efforts to address problems with IT projects and agencies' views of services provided, (2) assess these programs' efforts against practices for performance measurement and project prioritization, and (3) assess agency plans to establish their own digital service teams. To do so, GAO reviewed 32 18F projects and 13 USDS projects that were underway or completed as of August 2015 and surveyed agencies about these projects; reviewed 18F and USDS in key performance measurement and project prioritization practices; reviewed 25 agencies' efforts to establish digital service teams; and reviewed documentation from four agencies, which were chosen based on their progress made in establishing digital service teams.
Recommendations
GAO is making two recommendations to GSA and two recommendations to OMB to improve goals and performance measurement. GAO is also recommending that OMB update policy regarding CIOs and digital services teams. GSA and OMB concurred with the recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| General Services Administration | To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to ensure that goals and associated performance measures are outcome-oriented and that performance measures have targets, including (1) performance measures and targets tied to fully recovering program costs; and (2) goals, performance measures, and targets for how the program will achieve its mission after September 2016. |
The General Services Administration (GSA) agreed with, and took steps to implement, this recommendation. Specifically, GSA developed six outcome-oriented goals for 18F as well as associated performance measures and targets. In addition, one of the program's goals is to fully recover program costs. By ensuring that all of its goals and performance measures are outcome-oriented and that its performance measures have targets, 18F has a more clear definition of what it wants to accomplish. Additionally, by developing performance measures and targets tied to achieving full cost recovery, GSA is better positioned to begin recovering costs for the program.
|
| General Services Administration | To effectively measure 18F's performance, the Administrator of GSA should direct the Commissioner for the Technology Transformation Service to assess actual results for each performance measure. |
The General Services Administration (GSA) agreed with, and has taken steps to implement, this recommendation. Specifically, in November 2020, the Acting Director of 18F submitted a memo on 18F's fiscal year 2020 performance measures to the Acting Director of GSA's Technology Transformation Services (TTS). The memo included quarterly results for 18F's performance measures, which had previously been provided to TTS leadership, and annual results. The memo also stated that 18F and TTS leadership had reviewed and assessed 18F goals, targets, and associated performance measures internally and informally on a weekly basis. By assessing actual results for each performance measure, GSA will be better positioned to assess the status of 18F's activities and determine the areas that need improvement.
|
| Office of Management and Budget | To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to ensure that all goals and associated performance measures are outcome-oriented and that performance measures have targets. |
Although the Office of Management and Budget (OMB) generally agreed with this recommendation, it has not taken steps to implement the recommendation. In its December 2016 report to Congress, USDS identified three goals: (1) rethink how the federal government builds and buys digital services; (2) expand the use of common platforms, services, and tools; and (3) bring top technical talent into public service. In addition, USDS established performance measures with targets for its third goal and for each of the program's major projects. However, USDS had not established performance measures for the first two goals. Further, the program's third goal was not outcome-oriented. In February 2021, USDS stated that, while its report included the above-mentioned three goals, those goals were not intended to evaluate its ability to achieve its mission. Rather, USDS said that the goals are more accurately described as a furtherance of USDS's mission statement, which is to deliver better government services to the American people through technology and design. USDS said that it has moved toward evaluating and reporting to Congress on the success of individual projects by identifying the challenge, solution, and impact of USDS's work. Although measuring performance on projects can provide USDS with valuable information, without outcome-oriented goals and performance measures and targets for the overall program, it will be difficult to hold the program accountable for results. In December 2022 and May 2024, OMB stated that it does not intend to take any further actions to address this recommendation. However, we maintain that it is important for OMB to fully address this recommendation. As such, we will continue to evaluate OMB's progress in implementing it.
|
| Office of Management and Budget | To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to assess actual results for each performance measure. |
Although the Office of Management and Budget (OMB) generally agreed with this recommendation, it has not taken steps to implement the recommendation. In its December 2016 report to Congress, USDS identified three goals: (1) rethink how the federal government builds and buys digital services; (2) expand the use of common platforms, services, and tools; and (3) bring top technical talent into public service. In addition, USDS established performance measures with targets for its third goal and for each of the program's major projects. However, USDS had not established performance measures for the first two goals. Further, the program's third goal was not outcome-oriented. In February 2021, USDS stated that, while its report included the above-mentioned three goals, those goals were not intended to evaluate its ability to achieve its mission. Rather, USDS said that the goals are more accurately described as a furtherance of USDS's mission statement, which is to deliver better government services to the American people through technology and design. USDS said that it has moved toward evaluating and reporting to Congress on the success of individual projects by identifying the challenge, solution, and impact of USDS's work. Although measuring performance on projects can provide USDS with valuable information, without outcome-oriented goals and performance measures and targets for the overall program, it will be difficult to hold the program accountable for results. In December 2022 and May 2024, OMB stated that it does not intend to take any further actions to address this recommendation. However, we maintain that it is important for OMB to fully address this recommendation. As such, we will continue to evaluate OMB's progress in implementing it.
|
| Office of Management and Budget | To effectively measure performance, prioritize USDS's resources, and ensure that CIOs play an integral role in agency digital service teams, the Director of the Office of Management and Budget should direct the Federal Chief Information Officer to update USDS policy to clearly define the responsibilities and authorities governing the relationships between CIOs and the digital service teams and require existing agency digital service teams to address this policy. In doing so, the Federal Chief Information Officer should ensure that this policy is aligned with relevant federal law and OMB guidance on CIO responsibilities and authorities. |
The Office of Management and Budget (OMB) agreed with, and has taken steps to implement, this recommendation. Specifically, all three of the agencies that had digital service teams as of February 2021--the Departments of Homeland Security (DHS), Health and Human Services (HHS), and Veterans Affairs (VA)--have defined the relationship between the digital service team and the agency CIO. With respect to DHS, we reported in 2016 that DHS had defined the relationship between the executive for managing the digital service team and the agency CIO in the digital service team charter. Regarding HHS, in January 2017, USDS and HHS updated the digital service team charter to provide that the HHS digital service lead is to have unfettered access to the HHS CIO and that the digital service lead is to review and make recommendations on all technology-related budget decisions and procurement-related matters on HHS digital service projects to the CIO. With regards to VA, in February 2021, USDS said that in early 2020, the VA digital service team moved into VA's Office of the Chief Technology Officer, which is in the Office of the CIO. By better defining the relationship between agency CIOs and digital service teams, agency CIOs will be better positioned to carry out their responsibilities for IT management of the projects undertaken by the digital service teams.
|