VA IT Management: Organization Is Largely Centralized; Additional Actions Could Improve Human Capital Practices and Systems Development Processes
GAO-16-403
Published: Aug 17, 2016. Publicly Released: Sep 16, 2016.
Skip to Highlights
Highlights
What GAO Found
The Department of Veterans Affairs (VA) performs key information technology (IT)-related functions, such as leadership, strategic planning, systems development and acquisition, and systems operations and maintenance, largely through its centralized Office of Information & Technology (OI&T), led by the Chief Information Officer (CIO). VA's two IT governance boards are intended to play a role in other key functions, such as investment management. Nevertheless, the department faced challenges in effectively managing IT, including (1) preventing IT activities from occurring outside the control of OI&T, (2) maintaining collaboration between OI&T and business units, and (3) delivering efficient and cost-effective IT capabilities. In response to these and other challenges, the CIO initiated an effort in January 2016 to transform OI&T into a more veteran-focused organization that emphasized transparency, accountability, innovation, and teamwork. The transformation strategy calls for OI&T to stabilize and streamline core processes and platforms, mitigate weaknesses from information security and GAO assessments, and improve outcomes by institutionalizing a new set of IT management capabilities. The CIO intends to complete the transformation by the first quarter of 2017.
Key to an agency's success in effectively managing its IT systems is sustaining a workforce with the necessary knowledge, skills, and abilities to execute a range of management functions that support its mission and goals. VA took steps to implement effective IT human capital practices by documenting an IT human capital strategic plan and initiating an update based on changed priorities, analyzing workforce data, identifying skill gaps for the current year, and implementing an IT training program. However, OI&T had not consistently implemented all of these practices. Specifically, the office had not (1) tracked and reviewed historical and projected leadership retirements and (2) identified skills and competencies needed beyond the current year. Without annually tracking and reviewing data related to leadership retirements or identifying skills needed in future years, OI&T faces a risk of being unprepared to effectively respond to vacancies in key positions and not having the capabilities to deliver IT support that can contribute to improved services for veterans.
Key to successful development and acquisition of IT services is establishing documented processes that reflect best practices. Although there were gaps in some areas, VA's processes generally included best practices for project validation, project planning, requirements management, risk management, project monitoring and control, and process and product quality assurance. In addition, processes for developing and maintaining a project schedule had not fully addressed the majority of the associated best practices. Ensuring that these processes address all key practices will assist the department in effectively managing its IT system development and acquisitions.
Why GAO Did This Study
VA relies extensively on IT to deliver services to millions of our nation's veterans. VA reported spending approximately $3.9 billion in 2015 and received appropriations of approximately $4.1 billion in 2016 to improve and maintain its IT resources. Even as the department has engaged in various attempts to improve its IT management capabilities, GAO has issued numerous reports that highlighted challenges in its efforts.
This study was to determine (1) how VA is organized to manage and perform key IT-related functions and the extent to which it has centralized the management of IT resources, (2) the extent to which VA has implemented effective IT human capital management, and (3) the extent to which VA has established key processes to effectively manage major system development and acquisition efforts. To conduct its study, GAO reviewed VA policies, procedures, and other documentation and compared the department's processes to best practices for human capital management and IT systems development and acquisition. GAO also interviewed VA officials.
Recommendations
GAO is recommending that VA take two actions to assist the department in sustaining a workforce with the necessary knowledge, skills, and abilities to execute its mission and goals, as well as six actions to assist the department in developing comprehensive processes that reflect systems development best practices. VA generally agreed with GAO's conclusions and concurred with GAO's eight recommendations.
Recommendations for Executive Action
| Agency Affected | Recommendation | Status |
|---|---|---|
| Department of Veterans Affairs | To assist VA in sustaining an IT workforce with the necessary knowledge, skills, and abilities to execute its mission and goals, the Secretary of Veterans Affairs should direct the Chief Information Officer to track and review OI&T historical workforce data and projections related to leadership retirements. |
VA concurred with our recommendation and reported that OI&T's Human Capital Management Office had completed a succession planning project that encompassed all senior leadership and included data review and risk assessment for each position. VA also stated that OI&T tracks the gains and losses associated with its leadership positions and provided this information for fiscal year 2016. According to VA, OI&T continues to track, on a recurring basis, gains, losses, and retirement eligibility. VA also stated that data analysis can be conducted to determine the dates when OI&T employees will reach retirement eligibility and provided the number of its employees who hold leadership positions and are eligible for retirement. Additionally, according to VA, OI&T has constituted a formal workforce planning office with responsibilities that include conducting staffing model assessments, and succession planning. VA also provided information on OI&T's current staffing levels and projected staffing level requests. In June 2019, according to VA, the Assistant Secretary for Information and Technology and Chief Information Officer was briefed on OI&T's overall workforce planning effort. This briefing included a workforce analysis that identified staffing levels and gaps in the staffing levels. The briefing also identified key drivers impacting the OI&T staffing levels such as, IT modernization and Electronic Health Record Modernization (EHRM) system implementation. By tracking and reviewing OI&T historical workforce data and projections related leadership retirements, VA will be better prepared to effectively respond to vacancies in key leadership positions.
|
| Department of Veterans Affairs | To assist VA in sustaining an IT workforce with the necessary knowledge, skills, and abilities to execute its mission and goals, the Secretary of Veterans Affairs should direct the Chief Information Officer to identify IT skills needed beyond the current fiscal year to assist in identifying future skills gaps. |
VA concurred with our recommendation and reported that the Information Technology Workforce Development (ITWD) office will produce reports that identify skill gaps and will contain long-term recommendations that show the types of IT skills each organization needs to increase and which proficiency level targets need the most emphasis. In July 2017, VA stated that ITWD reviewed, and updated where needed, the fiscal year 2017 competencies within each OI&T competency model role in order to align the models to the OI&T Transformation initiative. According to the department, the resulting updates support learning solutions that sustain and accelerate OI&T's transformation. Additionally, VA stated that 85 percent of OI&T staff completed a validated competency self-assessment and provided the OI&T fiscal year 2017 Training Gap Analysis Report which shows the strengths and gaps of OI&T by organization, trends between fiscal years 2016 and 2017, findings, next steps, and recommended actions for the next fiscal year. The department also stated that ITWD held meetings to review skill gaps and learning solution reports. VA provided these reports and they present the top gaps and strengths, key findings, and next steps to address the skill gaps. The department also provided a report on the Enterprise Project Management Office (EPMO) Workforce Transformation that identifies IT workforce trends and training, tools, resources, and processes in place that support skill and career development. According to VA, this was an in-depth research and analysis effort to support identification of IT skills needed beyond the current fiscal year and through extensive interviews, ITWD identified specific skills, competencies, and roles that are expected to increase in demand over the next decade. The department also stated that it continues to collect and analyze competency assessment data in the first quarter of each fiscal year and the data is used in requirements gathering meetings with OI&T leaders. According to VA, current organizational skill gaps and future skill needs are discussed in-depth during these meetings. Additionally, next steps for the development of learning engagements in the current fiscal year and recommendations for potential learning interventions beyond the current fiscal year are identified in the meetings. The department stated that OI&T will continue this process going forward. By taking these actions, OI&T has increased assurance that its staff has the capabilities to deliver IT support that contribute to improved services for veterans.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to project planning, to include (1) estimating the level of effort that will need to be expended for work products and tasks, and (2) making adjustments to the project plan to reconcile differences between estimated and available resources. |
VA concurred with our recommendation and stated that OI&T was documenting changes to processes related to project planning as it transitioned to the Veteran-Focused Integration Process (VIP). The department stated that as of October 1, 2016, all projects had transitioned to VIP. In the department's April 2018 VIP Guide, the lifecycle project planning phase calls for project and program managers to maintain an audit file that supports how costs and budget information were derived for individual builds and total lifecycle product costs. Among other things, the capital costs can include government labor costs, contracted labor costs for planning, development, acquisition, system integration; and direct project management and overhead support. Additionally, the VIP Guide requires that all OI&T projects follow the OI&T Agile project management process and use the IBM Rational Tools Suite to manage sprints, among other things. According to VA officials, short Agile sprints in its VIP process will allow the project team to adjust the project plan frequently to reconcile differences between estimated and available resources, with the approval and oversight by the Portfolio Director. The department also stated that the VIP process incorporates the use of daily Scrum, and weekly Scrum of Scrum meetings, which can, and are, used to adjust the project plan as frequently as needed to also reconcile differences between estimated and available resources. By incorporating these recommended project planning practices in its process documentation, OI&T's project managers are able to consistently estimate the effort needed across projects and adjust project plans based on resource availability that could yield to timely completion of projects.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to requirements management, to include identifying changes to be made to plans and work products as a result of requirements baseline changes. |
VA concurred with our recommendation and reported that OI&T is revising its documentation related to requirements management as part of the transition to the Veteran-Focused Integration Process (VIP). According to VA, as of October 1, 2016, all projects had been transitioned to VIP. The department's VIP Guide requires project teams to use the IBM Rational Tool Suite to manage project requirements; configurations and changes; and source code and baselines. According to the department, requirements are tracked using the this tool suite, which provides a snapshot of the original baseline and an audit trail that captures the history of requirement changes. By documenting this practice, OI&T's staff will be better positioned to implement changes and maintain consistent processes across IT projects.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to risk management, to include (1) determining costs and benefits of implementing the risk mitigation plan for each risk and (2) collecting performance measures on risk handling activities. |
VA concurred with our recommendation and according to the department's VIP Guide, project teams are required to use the IBM Rational Tools Suite to manage risks. According to VA officials, by recording project risks, as well as effective responses that have been implemented to mitigate them, teams can track and report steps taken to mitigate risks and an audit trail will show the history of changes made to each risk. The department's Risk Management Framework states that the prioritization approach should consider the cost and expected return on investment of risk mitigation strategies. Additionally, according to VA, this framework allows OI&T to manage risks in an effective and systematic way and help reach its goals and objectives. VA also stated that OI&T developed a Program Management Review strategy that includes reporting of schedule performance and cost performance. Further, through its VIP process, VA has established a hierarchical construct for organizing and managing the delivery of VA's IT products. Through this construct impediments are to be escalated to the next level up if the lower level cannot resolve the impediment. If the impediment is not resolved once it has gone through each level [project, program, and portfolio], it is then escalated to the EPMO and OI&T executives who are to make decisions that can remove or mitigate the impediment. By documenting these processes, OI&T is better positioned to fully ensure that the costs and benefits of executing risk mitigation plans will be consistently determined and that performance measures for risk mitigation efforts are collected.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to project monitoring and control, to include the 10 best practices that were missing from the guidance. |
VA concurred with our recommendation and stated that implementation of the Veteran-Focused Integration Process (VIP) and Agile processes would address the best practices related to project monitoring and control that were missing from its guidance. (1) VA stated that project costs are monitored throughout the life-cycle of a project within the VIP Dashboard and expended effort is reflected in the Burn Down chart. According to the VIP Guide, the VIP Dashboard will include a project's planned, projected, and actual costs and a Burn Down chart is used to monitor work progress. (2) Regarding monitoring resources provided and used, VA stated that Primavera is the system used to track VA employee resources such as hours planned and actual used. The VIP Guide states that this system provides a single project time tracking process for staff to track hours necessary to complete products and work efforts. (3) VA stated that its Talent Management System is used to track training opportunities, which allows leadership to monitor skills and knowledge of project staff. The department also stated that employees must complete an electronic Individual Development Plan (IDP) each year which is reviewed by their supervisor. According to VA this process allows the supervisor the chance to identify knowledge and skill gaps and to address the gaps with appropriate training. According to the VIP Guide, the product team should be cross-functional and include all skills needed. (4) The department reported that data management activities, issues, and impacts will be managed using VIP, Agile, and the IBM Rational Tools Suite. VA stated that periodically reviewing data management activities are performed by the project manager and team, in conjunction with a monthly Metrics Report. (5) VA also stated that identifying and documenting significant data management issues and their impacts will be accomplished through monthly data management reports. According to its VIP Guide, OI&T expects that all products follow the Agile product management process and use the Rational Tools Suite to manage scheduled product sprints and backlog, product requirements, risks and issues, and product planning and engineering documentation, among others. (6) Regarding documenting results of data management activity reviews, VA stated that monthly reports are used to reach out to projects falling outside normal parameters. (7) VA stated that Agile methodologies will require stakeholders to be involved in the daily scrum meetings, user acceptance testing, and acceptance of deliverables, which will address stakeholders being involved regularly. (8) VA also stated that Agile methodologies will also address documenting the results of stakeholder involvement. According to the VIP Guide, the Agile development methodologies require development teams to meet often with stakeholders to ensure transparency and foster a collaborative work environment. (9) VA stated that critical decision events are using Rational based data assessments to report on level of satisfaction of project controls and process compliance requirements. According to the VIP Guide, the Product Owner will have a key role in the decision-making process during the development of the product and will be able to regularly express concerns and/or approvals to best meet user satisfaction. According to the department, collected measures of a project's performance are analyzed, discussed, and archived for lesson learned at Sprint Reviews. (10) VA stated that critical decision events are being held at the portfolio level, and action items from these events are being tracked. VA provided meeting minutes from critical decision events that were held in October and December 2016 that identified action items and the status of those items. By documenting these processes, OI&T is better positioned to ensure that its projects will achieve expected results.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to process and product quality assurance, to include (1) documenting a description of the quality assurance reporting chain and defining how objectivity will be ensured, and (2) periodically reviewing open noncompliance issues and trends with management that is designated to receive and act on them. |
VA concurred with our recommendation and reported that the implementation of the Veteran-Focused Integration Process (VIP), Agile processes, and the Rational Toolset within OI&T would address process and product quality assurance. According to VA, as a part of its VIP process, the product owner is engaged from intake through project completion, which will ensure that the quality of the product is maintained throughout the life cycle. Additionally, VA stated that a release agent is assigned to each project. According to the department's VIP Guide, the primary role of the release agent is to monitor, audit, and report on the completeness, integrity, and traceability of the project data associated with a release. The guide further states that the agent's role is to enhance the likelihood of a successful release by reviewing and ensuring that information provided by the project team is being managed in the repositories within the Rational Tools Suite. According to the VIP Guide, the release agent publishes a release readiness report on the project that is distributed to the VIP triad (portfolio director, product owner, and receiving organization representative) and other stakeholders prior to any production release. This report is to include the most up-to-date picture of the quality and completeness of the release, so that the VIP triad can make an informed decision regarding approval or disapproval of the release. According to VA, the release readiness report informs, but does not direct, the decision-makers. Furthermore, the department stated that the process of periodically reviewing open non-compliance issues and trends with management that is designated to receive and act on them will be accomplished through CIOStat meetings held with OI&T senior leadership. VA also stated that the Rational Quality Manager tool is used to automate routine testing activities to identify non-compliance issues and trends. The department's guide also states that with the VIP process, compliance organizations are monitoring progress and the implementation of the requirements during the entire development cycle. By documenting these best practices related to quality assurance in its processes, OI&T has more assurance that staff will appropriately report quality assurance issues and periodically review open issues and trends so that the relevant manager can act on them.
|
| Department of Veterans Affairs | To assist VA in establishing comprehensive and documented processes that reflect system development and acquisition best practices, the Secretary of Veterans Affairs should direct the Chief Information Officer to revise OI&T's documented processes related to project scheduling, to include the 9 best practices that were missing from the guidance and revise the documented processes where the guidance was contrary to best practices. |
VA concurred with our recommendation and reported that the implementation of VIP and Agile processes within OI&T would address the nine best practices related to project scheduling that are missing from its guidance. According to the department's VIP Guide, all VA OI&T projects must implement the Agile Methodology in which requirements are structured in the form of epics, sub-epics, and user stories. Through the VIP process, project teams are also required to make timely updates to the VIP Dashboard regarding schedule and that the Rational Tools Suite will be used to manage and administer source control and baselines; manage risks and issues; and manage scheduled product sprints and backlogs. Project teams are required to fully populate the tool with all required project information. (1) In regard to capturing all activities, VA stated that business and compliance requirements will be captured during the planning phase and maintained in the IBM Rational Tools Suite to manage scheduled project builds and backlog which will allow the project to more accurately maintain the schedule baseline, capture all schedule changes, and provides an audit trail of all the changes. Further, VA stated that project build planning sessions capture and prioritize all backlog items with high level activities captured in the VIP Dashboard. Further, the departments VIP Guide states that during Sprint Planning, the project team commits to a set of user stories, short descriptions of requirements, to be completed during the Sprint. (2) According to VA, during these planning sessions, user stories are elaborated to tasks (i.e., activities) and that each activity receives a duration estimate and is assigned to a project team member/resource. (3) Regarding updating the schedule with actual progress and logic, VA stated that the use of short development builds within Agile increases the probability of successful adherence to the schedule and provides the flexibility to make schedule changes using the backlog to prioritize requirements. (4) In regard to maintaining a baseline schedule, the department stated that there are high level commitments at the Critical Decision 1 meeting for each project and each build gets elaborated and committed to at a more granular level. Additionally according to the VIP Guide, during Sprint Planning a commitment is made to what will be completed during the Sprint and the Rational Tools Suite will be used to manage project planning documentation and baselines. (5) According to VA, Agile provides the flexibility to sequence when requirements will be developed. VA also stated that part of the Agile process being used by OI&T removes rigid, mandatory constraints as long as project teams follow compliance epics. (6) The department also stated that by implementing the Agile methodology, the concept of a critical path is counterproductive because if a task is critical today, the project team can reprioritize and address the needs of the project immediately. According to VA, Agile supports both sustainment and development projects, by allowing changes to the project backlog to address high priority functionality. (7) In regard to ensuring reasonable float, VA stated that Agile allows flexibility to shift from one build to another based on priorities. (8) The department also stated that the IBM Rational Tools Suite connects requirements, change orders, test cases, and test results in order to have full traceability in a closed loop system, which addresses the practice of verifying that the schedule is traceable horizontally and vertically. (9) Lastly, regarding conducting a schedule risk analysis, according to VA, risks are managed in the Rational Tools Suite and impediments are raised and escalated during daily scrums and scrum of scrum calls. By documenting these practices, OI&T has more assurance that schedules created for its projects will be reliable.
|
Full Report
GAO Contacts
Public Inquiries
Topics
Best practicesHuman capital ITHuman capital managementIT investment managementInformation managementInformation resources managementInformation technologyLabor forceSystems acquisitionSystems development life cycleSystems managementIT managementInformation systemsIT human capital