FINANCIAL MANAGEMENT SYSTEMS

VA Should Improve Its Requirements Development, Cost Estimate, and Schedule

Report to Congressional Committees

February 2025

GAO-25-107256

United States Government Accountability Office

  

View GAO‑25‑107256. For more information, contact Paula M. Rascona, (202) 512-9816 or rasconap@gao.gov; Brian P. Bothwell, (202) 512-6888 or bothwellb@gao.gov; or Vijay A. D’Souza, (202) 512-7650 or dsouzav@gao.gov.

Highlights of GAO‑25‑107256, a report to congressional committees

February 2025

FINANCIAL Management SYSTEMS

VA Should Improve Its Requirements Development, Cost Estimate, and Schedule

Why GAO Did This Study

VA’s core financial system is more than 30 years old. Two prior attempts to replace the system, beginning in 1998, failed after years of development and hundreds of millions of dollars in costs. The current program has spent $1.9 billion since 2016 and completed six incremental deployments of a new system. The current target date for program completion is 2031.

GAO was asked to review the progress of the program. This report examines the extent to which (1) the program’s cost estimate and schedule followed best practices; (2) requirements development and management efforts followed Agile best practices; and (3) the independent program reviews that VA conducted met key elements of effective reviews and addressed identified issues.

GAO evaluated program and independent review documentation, compared it with relevant best practices, and interviewed cognizant VA officials.

What GAO Recommends

GAO is making four recommendations to VA, including that the program fully implement Agile best practices for requirements development and management and that it incorporate key elements of effective independent reviews in VA policy. VA concurred with the recommendations and described actions the department will take to address them.

What GAO Found

In 2016, the Department of Veterans Affairs (VA) established the Financial Management Business Transformation program (the program) to replace its aging financial and acquisition systems. In 2021, GAO made two recommendations to VA to help ensure that the program’s cost estimate and schedule were consistent with GAO-identified best practices. They have not yet been implemented. VA continues to not fully or substantially meet best practices for developing and managing the cost estimate and schedule. As a result, they are unreliable. Without a reliable cost estimate and schedule, VA management risks not making fully informed and sound decisions.

Additionally, the program substantially or fully met five Agile best practices for requirements development and management and partially met the remaining three (see table). Regarding the three partially met practices, (1) if the program does not ensure complete and feasible requirements, it could be working on requirements that are not high priority; (2) without traceability, the program cannot establish that the work is contributing to its goals and providing value; and (3) by not balancing customer needs, the program could be developing functionality that is not immediately necessary.   

Best practice	GAO assessment
Refine requirements	●
Test and validate the system as it is being developed	●
Ensure work is contributing to the completion of requirements	◕
Elicit and prioritize requirements	◕
Manage and further refine requirements	◕
Ensure requirements are complete, feasible, and verifiable	◑
Maintain traceability in requirements decomposition	◑
Balance customer and user needs and constraints	◑

Legend: ●=Fully met; ◕=Substantially met; ◑=Partially met; ◔=Minimally met; ○=Not met

Source: GAO analysis of Department of Veterans Affairs (VA) Financial Management Business Transformation program documentation.  |  GAO-25-107256

GAO also found that VA generally incorporated the five key elements of effective independent verification and validation (independent review) for the program. Specifically, the program incorporated nine of the 10 key sub-elements and partially implemented one sub-element on determining which programs are subject to independent review. Although the program generally incorporated the elements of an effective independent review, VA does not have a department-wide IT acquisition policy that requires independent review or incorporates the key elements. As a result, VA risks not consistently implementing independent reviews for other VA IT programs.

Regarding addressing identified issues, as of November 2024 the independent review team reported that the program resolved 93 percent of the findings and recommendations that the team identified from 2021 to 2024. This is a significant improvement compared to the 27 percent of recommendations reported implemented as of April 2020.

 

 

 

 

 

Abbreviations
CWS	Consolidated Wave Stack
FMBT	Financial Management Business Transformation program
iFAMS	Integrated Financial and Acquisition Management System
IV&V	independent verification and validation
O&M	operations and maintenance
OIT	Office of Information and Technology
RTM	requirements traceability matrix
SQAS	Systems Quality Assurance Service
VA	Department of Veterans Affairs
VBA	Veterans Benefits Administration
VHA	Veterans Health Administration
WBS	work breakdown structure

February 24, 2025

The Honorable Jerry Moran
Chairman
The Honorable Richard Blumenthal
Ranking Member
Committee on Veterans’ Affairs
United States Senate

The Honorable Mike Bost
Chairman
The Honorable Mark Takano
Ranking Member
Committee on Veterans’ Affairs
House of Representatives

The Department of Veterans Affairs (VA) is responsible for administering benefit programs for veterans, their families, and their survivors. These programs include those for pensions, education, disability compensation, home loans, life insurance, vocational rehabilitation, survivor support, medical care, and burial benefits.

VA’s core financial system is more than 30 years old and, according to VA, is extremely difficult to maintain, results in inefficient operations, requires complex manual work-arounds, and does not provide real-time integration between financial and acquisition information across the department. VA’s financial statement auditors have long reported a material weakness related to VA’s financial management systems.[1] Two previous attempts to replace its legacy system, beginning in 1998, failed after years of development and hundreds of millions of dollars in cost.[2] According to agency officials, VA has spent another $1.9 billion from the inception of its current program in fiscal year 2016 through fiscal year 2024.

In 2016, VA established the Financial Management Business Transformation program (FMBT). FMBT is intended to replace VA’s aging financial and acquisition systems with one integrated system to meet the department’s financial management goals and comply with legislation and directives.[3] VA’s November 2024 life cycle cost estimate for FMBT is $8.6 billion. This is an increase of approximately $942 million over the 2023 estimate of $7.6 billion.[4]

You asked us to review FMBT’s progress. In July 2024, we issued a report that addressed key project management practices on collaborating with stakeholders on changes to the program’s cost estimate and schedule, assessing user satisfaction and concerns, and managing program risks.[5] This report examines the extent to which (1) FMBT’s cost and schedule estimates followed best practices, (2) requirements development and management efforts for the program followed Agile best practices, and (3) the design and implementation of VA’s independent verification and validation (IV&V) efforts met key elements of effective IV&V and FMBT addressed identified issues.

To address our first objective, we reviewed FMBT’s October 2023 life cycle cost estimate and program integrated master schedule, dated April 2024, and evaluated supporting documentation.[6] We evaluated those characteristics of a reliable cost estimate and schedule that were less than substantially or fully met from our previous report.[7] We also compared the results of the cost and schedule analyses to results from our past work to determine whether the department’s cost estimate and schedule have improved in meeting GAO best practices. We determined the overall assessment rating for each cost and schedule characteristic by assigning each best practice assessment a numerical score from one to five and calculating the average of the scores to arrive at an overall assessment rating.

To address objective two, we reviewed key FMBT documentation, evaluated this documentation against the GAO Agile Assessment Guide, and interviewed department officials and key stakeholders.[8]

Regarding objective three, we assessed VA policies and procedures against selected leading industry IV&V practices.[9] In addition, we evaluated FMBT IV&V documentation and interviewed VA and FMBT officials. To assess whether VA has resolved issues identified through IV&V, we reviewed documentation on the status of IV&V defects, findings, and recommendations. Appendix I provides additional details on our objectives, scope, and methodology.

We conducted this performance audit from January 2024 to February 2025 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Background

Deployment of the Integrated Financial and Acquisition Management System

FMBT is currently migrating VA’s financial management systems to a software-as-a-service solution configured for the department and hosted in the VA cloud, referred to as Integrated Financial and Acquisition Management System (iFAMS).[10] VA awarded a systems integration contract in April 2018 to support FMBT through incremental deployments of iFAMS, referred to as waves. Each wave delivers federal financial management capabilities to support the functions and activities that VA administrations and staff offices conduct to carry out its mission.

As of October 2024, FMBT has completed six incremental deployments of iFAMS.[11] FMBT is currently working on its next planned deployments, which are due to be implemented at Veterans Benefits Administration (VBA) Loan Guaranty in May 2025 and Veterans Health Administration (VHA) Station 134 in June 2025.[12] FMBT also plans to begin work on the VBA Insurance wave, which is estimated to be implemented in May 2027.

As of September 2024, FMBT estimated program completion in 2031. However, VA program leadership has not yet determined final implementation dates for multiple deployments at VHA that will affect this timeline. FMBT is also anticipating budget cuts that may delay planned work. Specifically, the program budget was cut by over $100 million in VA’s budget request for fiscal year 2025, which according to program officials will delay the start of the VBA Insurance wave.

Prior GAO Recommendations on the FMBT Cost Estimate and Schedule

We previously reviewed the quality of FMBT’s 2019 cost estimate and reported our results in March 2021.[13] We identified 18 best practices associated with a reliable cost estimate, which are summarized into four characteristics: (1) comprehensive, (2) well-documented, (3) accurate, and (4) credible. According to GAO’s Cost Estimating and Assessment Guide, a cost estimate is considered reliable if the assessment ratings for each of the four characteristics are substantially or fully met.[14] If any of the characteristics are scored as not met, minimally met, or partially met, the cost estimate cannot be considered reliable.

In March 2021, we reported that the FMBT 2019 cost estimate was unreliable since it did not fully or substantially meet all characteristics associated with a reliable estimate.[15] Specifically, the cost estimate substantially met one of the four characteristics of a reliable estimate (well-documented), partially met two characteristics (comprehensive and accurate), and minimally met one characteristic (credible). We recommended that the FMBT Deputy Assistant Secretary take steps to help ensure that the program develops a reliable cost estimate using best practices described in GAO’s Cost Estimating and Assessment Guide. Specifically, VA needed to address those cost characteristics that we identified as partially or minimally met. VA concurred with this recommendation and has taken some actions to address it, but it is not yet fully implemented.

We previously reviewed the quality of FMBT’s 2020 schedule and reported our results in March 2021.[16] We identified 10 best practices associated with a reliable schedule, which are summarized into four characteristics: (1) comprehensive, (2) well-constructed, (3) credible, and (4) controlled. According to GAO’s Schedule Assessment Guide, a schedule is considered reliable if the assessment ratings for each of the four characteristics are substantially or fully met.[17] If any of the characteristics are scored as not met, minimally met, or partially met, the schedule cannot be considered reliable.

In March 2021, we reported that the 2020 integrated master schedule was unreliable as it did not fully or substantially meet all characteristics associated with a reliable schedule.[18] Specifically, the schedule substantially met one of the four characteristics of a reliable schedule (controlled) and partially met the other three characteristics. We recommended that the FMBT Deputy Assistant Secretary take steps to help ensure that the program develops a reliable schedule using best practices described in GAO’s Schedule Assessment Guide by addressing those schedule characteristics that were partially or minimally met. VA concurred with this recommendation and has taken some actions to address it.

Use of Agile Development Principles

In March 2023, VA released the most recent version of the FMBT Scaled Agile Framework.[19] Agile is an umbrella term for a variety of practices in software development, including requirements development and management. Generally, Agile emphasizes early and continuous software delivery, customer feedback cycles, regular delivery frequency, the use of collaborative teams, and measuring progress in terms of working software.[20] A Scaled Agile Framework provides guidance for roles and iterative events at different levels in an organization, tailored to each unique context. A glossary of key terms associated with Agile development can be found in appendix II.

Use of IV&V in IT System Development and Acquisition

IV&V (also referred to as independent review in this report) is a process where organizations can reduce the risks inherent in IT system development and acquisition efforts by having a knowledgeable party independent of the developer determine that the system or product meets the users’ needs and fulfills its intended purpose. The IV&V process starts by determining the program’s risks early in the life cycle and then identifying those that could be mitigated or lessened by performing additional reviews and quality assessments.

We have long recognized the use of IV&V as a leading practice for federal agencies in acquiring a product or IT system that is complex, large scale, or high risk.[21] Organizations can fully realize benefits of independent reviews by adopting certain key elements of IV&V. Based on industry standards and leading practices from across the federal government as well as past GAO reports, we identified five key elements and 10 sub-elements for effective IV&V for large and complex IT system development and acquisition programs.[22] These five key elements are (1) establish decision criteria and process, (2) establish independence, (3) define program scope, (4) define program resources, and (5) establish management and oversight. See appendix I for more information on how we identified the five key elements of effective IV&V and the related sub-elements.

The IV&V efforts for FMBT are planned, managed, and executed by VA’s Office of Information and Technology (OIT), Compliance, Risk, and Remediation, referred to as the IV&V team. The team is composed of government staff and contracted resources.[23] According to VA officials, recent FMBT IV&V efforts relate to implementation of the Consolidated Wave Stack, the only fully deployed wave to undergo testing since the start of the current IV&V contract. The team’s efforts are guided by overarching planning documents encompassing all FMBT waves. The team is currently conducting IV&V activities for FMBT at VBA and VHA.

The IV&V team’s activities have been reduced from those outlined in the original plan. The FMBT IV&V Execution Plan, which the team developed in response to funding cuts reported by the team, outlines a reduced scope of IV&V activities and lists specific activities cut. For example, the team is no longer performing quality assurance assessments, common assessments, system and software assessments, hardware and infrastructure assessments, and data migration activities.[24] We previously reported that, according to VA, FMBT mitigated a 2019 funding shortfall by reducing funding for the IV&V contractor and other contract support and pausing the VHA implementation waves.[25] FMBT also mitigated a 2020 funding shortfall by eliminating funding for IV&V contract support, among other measures.[26] Additionally, FMBT leadership rejected, due to insufficient funding, the team’s November 2023 recommendation to conduct additional IV&V tasks on the program. The team also stated that without FMBT providing additional funding for IV&V, the program would face significant reductions in test coverage.

VA Has Addressed Selected Best Practices, but the FMBT Cost Estimate and Schedule Remain Unreliable

VA Has Taken Steps to Improve FMBT Cost Estimate Accuracy, but It Remains Unreliable

We determined that the FMBT’s October 2023 cost estimate was not reliable because it did not fully or substantially meet all four characteristics associated with a reliable cost estimate. Specifically, we found that the estimate improved in the accurate characteristic to substantially met, but it remains partially met in the comprehensive characteristic and minimally met in the credible characteristic. The three characteristics of a reliable cost estimate we evaluated, their associated best practices, and the results of our assessment are summarized in table 1. See appendix III for more detail.

Overall GAO characteristic assessment	Best practice	2021 GAO best practice assessment	2024 GAO best practice assessment
Comprehensive Original 2021 assessment: Partially met Updated 2024 assessment: Partially met	Includes all life cycle costs	◑	◕
	Is based on a technical baseline description that completely defines the program, reflects the current schedule, and is technically reasonable	◑	◑
	Is based on a work breakdown structure (WBS) that is product-oriented, traceable to the statement of work, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted	◑	◑
	Documents all cost-influencing ground rules and assumptions	◑	◑
Accurate Original 2021 assessment: Partially met Updated 2024 assessment: Substantially met	Is based on a model developed by estimating each WBS element using the best methodology from the data collected	◑	◑
	Is adjusted properly for inflation	◔	◕
	Contains few, if any, minor mistakes	◑	◕
	Is regularly updated to ensure it reflects program changes and actual costs	◕	◕
	Documents, explains, and reviews variances between planned and actual costs	◑	◑
	Is based on a historical record of cost estimating and actual experiences from other comparable programs	◑	◑
Credible Original 2021 assessment: Minimally met Updated 2024 assessment: Minimally met	Includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs	◔	◔
	Includes a risk and uncertainty analysis that quantifies the imperfectly understood risks and identifies the effects of changing key cost driver assumptions and factors	◔	◔
	Employs cross-checks—or alternate methodologies—on major cost elements to validate results	○	○
	Is compared to an independent cost estimate that is conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results	○	◑

Legend:

FMBT = Financial Management Business Transformation program

VA = Department of Veterans Affairs

● = Met: VA provided evidence that satisfies the entire criterion

◕ = Substantially met: VA provided evidence that satisfies a large portion of the criterion

◑ = Partially met: VA provided evidence that satisfies about half of the criterion

◔ = Minimally met: VA provided evidence that satisfies a small portion of the criterion

○ = Not met: VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

Notes: To determine the overall assessment for each characteristic, we assigned each best practice assessment a score based on a five-point scale: not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. We calculated the average of the individual best practice assessment scores to determine the overall assessment rating for each characteristic as follows: not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0.

Italicized text denotes best practices that were not reevaluated as part of this review. See app. III for more details. This table does not represent all characteristics and best practices associated with a reliable cost estimate, as we did not reevaluate the well-documented characteristic and its best practices since we rated them as fully or substantially met in our 2021 report (GAO, Veterans Affairs: Ongoing Financial Management System Modernization Program Would Benefit from Improved Cost and Schedule Estimating, GAO‑21‑227 (Washington, D.C.: Mar. 24, 2021)). See app. III for more details on these best practices.

Remaining characteristics and their associated best practices that are not fully or substantially met continue to affect the reliability of the program’s cost estimate. For example, VA has continued to not employ cross-checks of major cost elements.[27] Unless an estimate employs cross-checks, the estimate will have less credibility because stakeholders will have no assurance that alternative estimating methodologies produce similar results. Additionally, while the cost estimate documentation and model contain ground rules and assumptions, this best practice is only partially met because (in part) it does not identify risks associated with assumptions or constraints.

Further, while the program obtained an independent cost estimate, it was not reconciled with the program office cost estimate. The cost estimate would be a more useful management tool if VA addressed these remaining weaknesses. We continue to believe that the recommendation from our prior report is valid and should be fully implemented. This would help ensure that VA management has the information necessary for fully informed and sound decision-making and to minimize the risk of cost overruns.

VA Has Taken Steps to Improve the FMBT Schedule, but It Remains Unreliable

We determined that the April 2024 schedule was not reliable because it did not fully or substantially meet all four characteristics associated with a reliable schedule. Specifically, we found that the schedule improved in the well-constructed characteristic to substantially met but remains partially met in the comprehensive and credible characteristics. The three characteristics of a reliable schedule that we evaluated, their associated best practices, and the results of our assessment are summarized in table 2. See appendix III for more detail.

Overall GAO characteristic assessment	Best practice	2021 GAO best practice assessment	2024 GAO best practice assessment
Comprehensive Original assessment: Partially met Updated assessment: Partially met	Capturing all activities	◑	◕
	Assigning resources to all activities	◔	◔
	Establishing the durations of all activities	◕	◕
Well-constructed Original assessment: Partially met Updated assessment: Substantially met	Sequencing all activities	◑	◕
	Confirming that the critical path is valid	◑	◑
	Ensuring reasonable total float	◑	◕
Credible Original assessment: Partially met Updated assessment: Partially met	Verifying that the schedule can be traced horizontally and verticallya	◑	◑
	Conducting a schedule risk analysis	◔	◑

Legend:

FMBT = Financial Management Business Transformation program

VA = Department of Veterans Affairs

● = Met: VA provided evidence that satisfies the entire criterion

◕ = Substantially met: VA provided evidence that satisfies a large portion of the criterion

◑ = Partially met: VA provided evidence that satisfies about half of the criterion

◔ = Minimally met: VA provided evidence that satisfies a small portion of the criterion

○ = Not met: VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

Notes: To determine the overall assessment for each characteristic, we assigned each best practice assessment a score based on a five-point scale: not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. We calculated the average of the individual best practice assessment scores to determine the overall assessment rating for each characteristic as follows: not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0.

Italicized text denotes best practices that were not reevaluated as part of this review. See app. III for more details. This table does not represent all characteristics and best practices associated with a reliable schedule, as we did not reevaluate the controlled characteristic and its best practices since they were rated as fully or substantially met in our prior report (GAO, Veterans Affairs: Ongoing Financial Management System Modernization Program Would Benefit from Improved Cost and Schedule Estimating, GAO‑21‑227 (Washington, D.C.: Mar. 24, 2021)). See app. III for more details on these best practices.

^aHorizontal traceability ensures that the schedule is planned in a logical sequence, accounts for interdependence of activities, and provides a way to evaluate current status. Vertical traceability ensures that data are consistent between different levels of the schedule.

FMBT’s April 2024 schedule continued to have issues with vertical traceability, including inconsistent dates, between lower-level schedules and upper-level schedule milestones. Additionally, the program performed a schedule risk analysis that was limited in scope and not for the entire effort, affecting the reliability of the program’s schedule. Finally, while the program performs some resource tracking as part of its Agile processes, it had not assigned resources to activities within the integrated master schedule, limiting insight into current or projected resource allocation and increasing the risk of the program falling behind schedule. To address the remaining weaknesses, we continue to believe that the recommendation from our prior report is valid and should be fully implemented. This would help ensure that VA management has the information necessary for fully informed and sound decision-making and to minimize the risk of additional schedule delays.

FMBT Has Generally Followed Best Practices for Requirements Development and Management, but Improvements Are Needed

FMBT has substantially or fully met five of the eight Agile best practices for requirements development and management.[28] We determined that the best practices on ensuring the completeness, feasibility, and verifiability of requirements; maintaining requirements’ traceability; and balancing customer needs and constraints were partially met.[29] For Agile requirements development and management to be considered successful, all assessment ratings for each best practice must be substantially or fully met. Table 3 shows our assessment of the eight Agile best practices for requirements development and management (in order of “fully met” to “partially met” practices). We discuss the three best practices that are partially met in more detail in the sections below the table; for more information on all eight best practices, see appendix IV.

Best practice	GAO assessment
Refine requirements	●
Test and validate the system as it is being developed	●
Ensure work is contributing to the completion of requirements	◕
Elicit and prioritize requirements	◕
Manage and further refine requirements	◕
Ensure requirements are complete, feasible, and verifiable	◑
Maintain traceability in requirements decomposition	◑
Balance customer and user needs and constraints	◑

Legend:

FMBT = Financial Management Business Transformation program

VA = Department of Veterans Affairs

● = Fully met: VA provided evidence that satisfies the entire criterion.

◕ = Substantially met: VA provided evidence that satisfies a large portion of the criterion.

◑ = Partially met: VA provided evidence that satisfies about half of the criterion.

◔ = Minimally met: VA provided evidence that satisfies a small portion of the criterion.

○ = Not met: VA provided no evidence that satisfies any of the criterion.

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

FMBT Does Not Consistently Ensure Requirements Are Complete, Feasible, and Verifiable

FMBT has documentation and policies that reflect best practices; however, the program is inconsistently applying its requirements development policies during program implementation. According to GAO’s Agile Assessment Guide, acceptance criteria, a definition of done, and a definition of ready are to be established for user stories[30] describing requirements prior to their development.[31] Additionally, in the FMBT Scaled Agile Framework policy document, standard definitions of done and ready are included, and the policy requires that a program apply these definitions in its user stories, at a minimum, during requirements development.[32] However, our analysis found that FMBT lists of user stories and requirements backlogs do not consistently include acceptance criteria and definitions of done and ready.

Program officials stated that the required standard definitions are consistently used regardless of whether they are included in program documentation. However, the program did not provide supporting documentation for this practice. Without clear acceptance criteria and definitions of done and ready, FMBT may not be working on the highest-priority requirements. Furthermore, well-defined acceptance criteria can help teams estimate a user story’s complexity.[33] If users and customers are not involved in the review and acceptance process for software functionality, there is an increased risk of software not meeting its intended purpose.

FMBT Lacks a Road Map for Requirements Traceability

FMBT has not developed an Agile road map[34] or comparable document to facilitate the traceability of requirements across the program as it is implemented.[35] Specifically, our analysis found that the program does not have an Agile road map and that the traceability of requirements across user story lists, backlogs, and requirements traceability matrices is irregular. While maintaining requirements traceability matrices is in line with Agile best practices, the lack of an Agile road map prevents the program from sharing, across different levels of the organization, what work is planned in the current and upcoming releases.

GAO’s Agile Assessment Guide calls for requirements to be traceable from the source requirements (e.g., epics, features)[36] to lower-level requirements (e.g., user stories) and vice versa, and the program is to use Agile artifacts, such as a road map, to verify requirements traceability.[37] A properly developed Agile road map associates features with releases, allowing a program to trace user stories back to high-level requirements.

FMBT officials stated that budgetary constraints contributed to the program’s lack of an Agile road map. The program has an implementation timeline, which it refers to as a road map. This document only depicts the schedule of implementation wave deployments throughout the program’s life cycle. The timeline does not include epics and features that would show traceability between requirements. An example of an Agile road map is shown in figure 1.

Specifically, the program’s implementation timeline does not indicate when releases or other Agile planning events occur and provide any insight into when features or epics for the different implementation waves are expected to be developed. Ensuring alignment between the user stories delivered in an iteration and the goals of the program and organization via an agreed-upon artifact (such as a road map that tracks feature prioritization) is one way to exhibit the delivery of the features with the highest value. Without tools, like a road map, to facilitate frequent information dissemination, decision-makers may not have access to performance information and may not be able to act in a timely manner or make improvements or corrective actions. For example, without the ability to trace a user story back to high-level requirements, a program cannot justify whether it is meeting the commitments made to various oversight bodies or establish that its work is contributing to the program goals.

FMBT Requirements Development Process Does Not Adequately Balance Customer Needs

FMBT’s requirements development process does not prioritize requirements based on the value their development would provide to a customer, or “value of work.”[38] Specifically, our analysis found that while FMBT began assigning value of work with its most recent implementation wave, there is no evidence to suggest that the program uses the value of work to prioritize requirements.

According to GAO’s Agile Assessment Guide, one method of measuring the value of work is to consider how frequently a feature will be used in order to develop features that are of immediate value.[39] The guide further states that having a consistent process in place to measure the value of work ensures that requirements are developed based on relative value. For example, without clearly prioritizing work, the developers could work on features that are not “must haves” to the customer, resulting in the delivery of features that may not be used, which could contribute to schedule and cost overruns.

According to FMBT officials, the program is currently working toward a consistent process for measuring the value of work so that features are developed based on relative value. In the meantime, because the program is not yet using an Agile road map, it is unable to track features to ensure that it is prioritizing the highest-value requirements. If FMBT’s requirements development process does not account for the relative value of work, it may develop functionality that is not immediately necessary to meet customer needs. Additionally, if the program does not complete the highest-value requirements first, it risks leaving customers without necessary functionality.

VA IT Acquisition Policy Does Not Include Key Elements of Effective IV&V, but FMBT Has Mostly Addressed IV&V Findings

VA performed independent reviews on FMBT and generally incorporated the key elements of effective IV&V in these efforts.[40] Specifically, we determined that the IV&V team’s use of independent reviews for FMBT met nine key sub-elements and partially met one. However, VA’s IT acquisition policy did not require independent reviews or incorporate any of the 10 key sub-elements of effective IV&V from leading industry practices, as identified by GAO. Additionally, VA substantially addressed most IV&V-identified issues for FMBT.

VA’s Use of IV&V for FMBT Generally Incorporated Key Elements of Effective IV&V, but IT Acquisition Policy Did Not

Along with professional technical organizations, we have long reported on the benefits and use of effective IV&V as a leading practice for high-risk IT programs. Based on relevant leading industry practices referenced in a prior GAO report, there are five key elements of an effective IV&V with 10 sub-elements.[41] The five key elements are (1) establish decision criteria and process, (2) establish independence, (3) define program scope, (4) define program resources, and (5) establish management and oversight.

The team performed IV&V reviews on FMBT and generally incorporated the key elements in these efforts. Specifically, we determined that the team’s use of IV&V for FMBT met nine key sub-elements and partially met one (see table 4). We discuss the one key sub-element that is partially met in more detail below the table; more information on all 10 key sub-elements is included in appendix V.

Key elements and sub-elements of effective IV&V	GAO assessment
(1) Establish decision criteria and process. A risk-based, decision-making process is defined to determine whether or the extent to which programs are to be subject to IV&V, to include:
1.     establishing risk-based criteria for determining which programs, or aspects of programs, to subject to IV&V	◑
2.     establishing a process for using IV&V to improve the management of the IT acquisition/development program	●
(2) Establish independence. The degree of technical, managerial, and financial independence required of the personnel or agents performing IV&V is defined, including:
3.     technical, managerial, and financial independence requirements for the IV&V agent	●
4.     a mechanism for reporting the results of IV&V to program oversight officials, as well as program management	●
(3) Define program scope. The scope of IV&V activities is defined, including:
5.     a definition of the program activities subject to IV&V	●
6.     validation and verification compliance criteria for each program activity subject to IV&V	●
(4) Define program resources. The resources needed for IV&V are specified, including:
7.     the facilities, personnel, funding, tools, and techniques and methods	●
(5) Establish management and oversight. The management and oversight to be performed are specified, including:
8.     the process for responding to issues raised by the IV&V effort	●
9.     the roles and responsibilities of all parties involved in the program	●
10.   how the effectiveness of the IV&V effort will be evaluated	●

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

VA = Department of Veterans Affairs

● = Met: VA provided evidence that satisfies the entire criterion

◑ = Partially met: VA provided evidence that satisfies about half of the criterion

○ = Not met: VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT IV&V documentation.  |  GAO‑25‑107256

VA partially met the key sub-element related to establishing a risk-based process to determine which programs or aspects of the programs to subject to IV&V. This is because VA’s overall decision to perform independent reviews on FMBT was not based on a formal, risk-based process with documented criteria. However, the IV&V team did establish a risk-based process to determine which aspects of the program to subject to IV&V. Specifically, the team used a guide to identify areas of risk to scope its activities within each wave. This guide supported the team’s risk assessment, test preparation, and test execution processes for each FMBT wave.

We reviewed VA’s department-wide IT acquisition policies, procedures, and guidance. We found that these documents do not require independent reviews or incorporate any of the five key elements or 10 sub-elements of effective IV&V. VA’s policy did not include information regarding the IV&V process, procedures, or guidelines, and officials from various VA offices did not provide us with evidence of any related IT acquisition policies, procedures, or other guidance.

In response to our inquiry about why VA did not have an IT acquisition policy related to IV&V, VA acquisition officials stated that the department has not identified a specific need for such policy. OIT IV&V team officials stated that the department based its decision to use IV&V for FMBT primarily on the program’s designation as a major modernization effort. According to the team, the engagement and execution of IV&V has varied for VA programs and projects. To address this, team officials stated that they are developing a framework to determine which VA programs to subject to IV&V, aiming for implementation in 2026. According to team officials, they plan for this framework to use a standard set of criteria for making this determination, although it is not clear if it will incorporate any of the other key elements or sub-elements.

While VA has incorporated most elements of effective IV&V for FMBT, the absence of formal department-wide policies and procedures could hinder these efforts. Having a documented policy that contains the five elements can further ensure that the key elements will be addressed. Without department-wide IT acquisition policies, procedures, and guidance that require and incorporate elements of effective IV&V, VA risks not consistently implementing independent reviews and not meeting its goals for other VA IT programs. This includes the risk of delivering a system with significant defects.

VA Has Substantially Addressed Most Issues Identified Through IV&V Efforts

FMBT has responded to the IV&V team’s recommendations and has taken steps to improve system quality and performance. For example, the team identified that an iFAMS interface was missing specific requirement and acceptance criteria and recommended thoroughly outlining and documenting these criteria. FMBT agreed with this recommendation and took corrective action, and the team verified the successful resolution. Additionally, the team identified 26 critical and high-severity defects, which the program successfully addressed, as determined by the team.[42] As a result, FMBT has substantially addressed most of the defects, as well as findings and recommendations identified through its IV&V efforts.

The IV&V team reports both (1) defects and (2) findings and recommendations to FMBT management and tracks resolution internally. Defects are issues logged against explicitly stated FMBT requirements that were not met. The status of defects is tracked separately from findings and recommendations, which identify, document, and communicate potential areas of improvement. The team reports on defects and findings and recommendations through IV&V weekly status reports, bi-weekly leadership reports, monthly progress reports, and summary reports submitted at the conclusion of implementation wave testing events. See appendix V for more detailed information on the team’s defects, findings and recommendations processes, and the results of the program’s IV&V efforts.

According to the IV&V team’s defect documentation, as of November 2024 the team reported that FMBT resolved 89 percent of the defects that the IV&V team identified from 2020 to 2024 (see fig. 2). For instance, FMBT resolved a separation of duty issue identified by the team related to travel voucher creation.

Further, according to the IV&V team’s findings and recommendations documentation, as of November 2024 FMBT resolved 93 percent of the findings and recommendations that the IV&V team identified from 2021 to 2024 (see fig. 3). This is a significant improvement compared to the 27 percent of IV&V recommendations that the team previously reported had been implemented as of April 3, 2020.[43]

Conclusions

The program continues to not fully or substantially meet related characteristics and associated best practices, resulting in an unreliable cost estimate and schedule. Following cost estimation and scheduling best practices helps ensure that VA management has the information necessary for fully informed and sound decision-making and minimize the risk of cost overruns and schedule delays. Therefore, we reiterate the need for VA to implement our two prior recommendations related to ensuring the FMBT’s cost estimate and schedule are consistent with GAO-identified best practices.

FMBT’s requirements development and management activities substantially or fully met five Agile best practices and partially met the remaining three, which focused on ensuring the completeness, feasibility, and verifiability of requirements; maintaining requirements traceability; and balancing customer needs and constraints. If FMBT does not ensure complete and feasible requirements with clear definitions of done and ready, the program could be working on requirements that are not high priority. Without a road map to trace requirements across the program’s implementation life cycle, FMBT cannot establish that the work is contributing to its goals and providing value. By not balancing customer needs, the program could be developing functionality that is not immediately necessary.

VA’s use of independent reviews for FMBT generally incorporated the five key elements of effective IV&V; however, the department did not fully implement the sub-element on determining which programs to subject to independent reviews. VA does not have department-wide IT acquisition policy, procedures, or other guidance that require independent review or incorporate the key elements of effective IV&V. Without department-wide policy, procedures, and guidance that incorporate these key elements, VA increases the risk of not consistently implementing independent reviews and not meeting its goals for other VA IT programs.

Recommendations for Executive Action

We are making the following four recommendations to VA:

The Secretary of Veterans Affairs should ensure that the FMBT Deputy Assistant Secretary consistently documents and assesses the completeness of all requirements against acceptance criteria and definitions of done and ready, as required by the FMBT Scaled Agile Framework. (Recommendation 1)

The Secretary of Veterans Affairs should ensure that the FMBT Deputy Assistant Secretary facilitates requirements traceability by developing and implementing an Agile road map. This road map should align with best practices in Agile development to monitor the value of work completed and whether it meets stakeholder needs. (Recommendation 2)

The Secretary of Veterans Affairs should ensure that the FMBT Deputy Assistant Secretary balances customer needs and constraints by consistently assigning value of work during requirements development and prioritizing work based on relative value. (Recommendation 3)

The Secretary of Veterans Affairs should ensure that the Deputy Chief Information Officer of OIT incorporates and fully implements key elements of effective IV&V into OIT’s planned IV&V framework and acquisition policy, procedures, and guidance, including a formal, risk-based process for determining which VA IT programs to subject to independent reviews. (Recommendation 4)

Agency Comments

We provided a draft of this report to VA for review and comment. In its written comments, reproduced in appendix VI, VA concurred with our four recommendations and described actions it has taken and will take to address the issues we identified with FMBT. Those actions, if implemented as described, should address our recommendations.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at https://www.gao.gov.

If you or your staff have any questions about this report, please contact Paula M. Rascona at (202) 512-9816 or rasconap@gao.gov, Brian Bothwell at (202) 512-6888 or bothwellb@gao.gov, or Vijay A. D’Souza at (202) 512-7650 or dsouzav@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VII.

Paula M. Rascona
Director
Financial Management and Assurance

Brian P. Bothwell
Director
Science, Technology Assessment, and Analytics

Vijay A. D’Souza
Director
Information Technology and Cybersecurity

This report examines the extent to which (1) the Department of Veterans Affairs (VA) Financial Management Business Transformation program’s (FMBT) cost estimate and schedule followed best practices; (2) the program’s requirements development and management efforts followed Agile best practices; and (3) the design and implementation of VA’s independent verification and validation (IV&V) efforts for the program met key elements of effective IV&V and FMBT addressed identified issues.

To examine VA’s cost estimating practices, we reviewed documentation supporting FMBT’s October 2023 life cycle cost estimate—the most recent estimate available. (The four characteristics and 18 best practices for a reliable cost estimate are presented in table 6 in app. III.) The risk assessment component of internal control was significant to our review of the estimate, along with the underlying principles that management identify, analyze, and respond to (1) risks related to achieving the defined objectives and (2) significant changes that could affect the internal control system. To assess the reliability of the October 2023 cost estimate, we evaluated documentation supporting the estimate, such as the cost estimating models, the program’s October 2023 cost estimate report, and briefings provided to VA management. We assessed the cost estimate—including the methodologies, assumptions, and results—against best practices for developing a comprehensive, accurate, and credible cost estimate identified in GAO’s Cost Estimating and Assessment Guide.[44] We evaluated those characteristics of a reliable cost estimate that were less than substantially or fully met from our previous report. We did not reassess the well-documented characteristic or the best practices of a reliable cost estimate that were found substantially met or met in recommendation follow-up from our prior report.[45]

To examine VA’s scheduling practices, we reviewed documentation on FMBT’s April 2024 integrated master schedule—the most recent schedule available. (The four characteristics and 10 best practices for a reliable schedule are presented in table 7 in app. III.) To assess the reliability of the 2024 FMBT schedule, we evaluated documentation supporting the schedule, such as the integrated project schedules and program baseline. We assessed the schedule documentation against best practices for developing a comprehensive, well-constructed, and credible schedule, as identified in GAO’s Schedule Assessment Guide.[46] We evaluated those characteristics of a reliable schedule that were less than substantially or fully met from our previous report. We did not reassess the controlled characteristic of a reliable schedule as it was found substantially met in our prior report.

For both the program cost estimate and schedule, we rated best practices under each characteristic as follows:

·         Met: VA provided complete evidence that satisfies the entire criterion.

·         Substantially met: VA provided evidence that satisfies a large portion of the criterion.

·         Partially met: VA provided evidence that satisfies about one-half of the criterion.

·         Minimally met: VA provided evidence that satisfies a small portion of the criterion.

·         Not met: VA provided no evidence that satisfies any of the criterion.

We assigned each best practice a score based on a five-point scale: not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. Then, we calculated the average of each characteristic’s best practice score to determine the overall assessment rating for each characteristic as follows: not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0. We compared the results of the cost and schedule analyses to the results from our past work to determine whether the department’s cost estimate and schedule have improved in meeting GAO best practices.

Finally, we provided VA with draft versions of our detailed analyses of FMBT’s cost estimate and schedule so that department officials could verify the information on which we based our findings.

To determine the extent to which FMBT’s efforts for requirements development and management followed GAO best practices, we evaluated the program’s implementation of Agile principles against chapter 5 of GAO’s Agile Assessment Guide, which describes eight best practices for requirements development and management in Agile.[47] We reviewed FMBT documents, including program requirements backlogs, user story lists, requirements traceability matrices, and the FMBT Scaled Agile Framework document to conduct our initial analysis.[48] Then, we met with program officials and leadership to obtain their perspectives on requirements management in an Agile environment and any additional information relevant to our scoring. We then provided the draft analysis to FMBT officials for any additional comment or clarification. We incorporated new information as appropriate.

To assess the design and implementation of VA’s IV&V efforts, we first reviewed the five key elements for effective IV&V identified in a prior GAO report and the 10 sub-elements.[49] To ensure the continued relevance and appropriateness of the key elements of effective IV&V as evaluation criteria, we reviewed the latest versions of the relevant IV&V leading industry practices referenced in the prior report. Specifically, we reviewed the (1) Institute of Electrical and Electronics Engineers, IEEE Standard for System, Software, and Hardware Verification and Validation;[50] (2) International Organization for Standardization, International Electrotechnical Commission, and Institute of Electrical and Electronics Engineers, Systems and Software Engineering—System Life Cycle Processes;[51] (3) International Organization for Standardization, International Electrotechnical Commission, and Institute of Electrical and Electronics Engineers, Systems and Software Engineering—Software Life Cycle Processes;[52] and (4) the Software Engineering Institute, CMMI® for Development.[53] Based on this review, we further clarified the five key elements and 10 sub-elements.

Next, we interviewed VA officials and obtained and reviewed VA IT acquisition policies, procedures, and guidance and documentation related to FMBT’s IV&V efforts. We assessed this documentation against the five key elements and 10 sub-elements of effective IV&V.

To determine the extent to which VA addressed FMBT issues identified through its IV&V efforts, we reviewed documentation on the status of the IV&V team’s identified defects, as well as findings and recommendations as of November 2024. Specifically, we summarized this information through our review of the FMBT IV&V defects, as well as findings and recommendations tracking spreadsheets as of November 2024.

The control activities component of internal control was significant to our evaluation of FMBT’s IV&V efforts, along with the underlying principles that management design control activities to achieve objectives and respond to risks and implement control activities through policies. We assessed the reliability of the IV&V team’s identified defects. We also assessed findings and recommendations data through our review of related documentation, interviews with VA officials, and performance of manual data testing to determine the extent to which the related data fields and key elements were complete and whether there were any incomplete or missing data, or obvious errors. We determined that the team’s identified defects, and findings and recommendations documentation were reliable for the purposes of our engagement.

We conducted this performance audit from January 2024 to February 2025 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Table 5 defines key terms associated with Agile development that are used in this report.

Term	Definition
Acceptance criteria	Criteria by which a work item (usually a user story) is judged to be successful or not. Acceptance criteria are developed to identify when the user story has been completed and meets the preset standards for quality and production readiness.
Agile	An umbrella term for a variety of best practices in software development. Agile software development supports the practice of shorter software delivery. Specifically, Agile calls for the delivery of software requirements in small and manageable predetermined increments based on an “inspect and adapt” approach where the requirements change frequently and software is released in increments. More a philosophy than a methodology, Agile emphasizes early and continuous software delivery, fast feedback cycles, rhythmic delivery cadence, the use of collaborative teams, and measuring progress in terms of working software. There are many specific methodologies that fall under this category, including Scrum, eXtreme Programming, and Kanban.
Backlog	A list of features, user stories, and tasks to be addressed by the team or program, ordered from highest to lowest priority. Newly discovered requirements or defects are added to the backlog.
Backlog refinement	The process for keeping the backlog updated by adding detail and revisiting the order and estimates assigned to work that teams agree to be necessary. This allows details to emerge as knowledge increases through feedback and learning cycles.
Cadence	The rhythm and predictability that a team enjoys by delivering in consistent time boxes.
Capacity	The quantity of resources available to perform useful work.
Customer	Synonymous with business sponsor because the customer requires the product or service. The customer may or may not be a user. The customer is an integral part of the development and has specific responsibilities depending on the Agile methods used. The customer wants continuous improvement of products and services.
Definition of done	A predefined set of criteria that must be met before a work item is considered complete. The definition of done serves as a checklist, identifying all activities/artifacts besides working code that must be completed for a feature to be ready for deployment or release, including testing, documentation, training material development, and certifications.
Definition of ready	A predefined set of criteria specifying the level of detail needed before teams can begin development on a user story. Since detailed requirements evolve throughout the lifespan of the program, a definition of ready helps to ensure that participants work on only the most current and highly ranked requirements and that those requirements always reflect any updates to plans, activities, and work products.
Epic	A large user story that can span one or more releases and is progressively refined into features and then into smaller user stories that are at the appropriate level for daily work tasks and captured in the backlog. Epics are used as placeholders to keep track of and prioritize larger ideas. In the Financial Management Business Transformation program (FMBT), an epic generally takes more than one program increment but less than 1 year to complete.
Feature	A specific amount of work that can be developed within one or two reporting periods. It can be further segmented into user stories. The functionality is described with enough detail that it can remain stable throughout its development and integration into working software. In FMBT, a feature takes more than one sprint but less than one program increment to complete.
Iteration	A predefined, time-boxed and recurring period of time in which working software is created. Instead of relying on extensive planning and design, an iteration relies on rework informed by customer feedback. In FMBT, the recommended iteration duration is 4 weeks.
Product	A tangible item produced to create specific value to satisfy a want or requirement.
Product owner	The person who is accountable for ensuring business value is delivered by creating customer-centric items (typically user stories), ordering them, and maintaining them in the backlog. The product owner defines acceptance criteria for user stories. In Scrum, the product owner is the sole person/entity responsible for managing the backlog. The product owner’s duties typically include clearly expressing the backlog items, prioritizing the backlog items to reflect goals and missions, keeping the backlog visible to all, optimizing the value of development work, ensuring that the developers fully understand the backlog items, and deciding when a feature is “done.” A product owner should be available to the team within a reasonable time for both decision-making and empowerment.
Program increment	A time box in which working software is created across multiple iterations/sprints. In FMBT, a program increment lasts for four iterations/sprints (16 weeks).
Regression testing	A type of software testing that verifies that software that was previously developed and tested still performs correctly after it was changed or interfaced with other software. These changes may include software enhancements, patches, configuration changes, and so forth. During regression testing, new software bugs or regressions may be discovered.
Release/implementation wave	A planning segment of requirements (typically captured as features or user stories in the backlog) that implements needed capabilities. FMBT describes segments of its implementation process as “waves.”
Requirement	A condition or capability needed by a customer to solve a problem or achieve an objective.
Requirements traceability matrix	A tool for demonstrating that low-level requirements are traceable to high-level requirements. A requirements traceability matrix ensures that all functional requirements defined are tested and are traceable to features, user stories, and process flows.
Road map	A high-level plan that outlines a set of releases and the associated features. The road map is intended to be continuously revised as the plan evolves.
Scaled Agile framework	A governance model used to align and collaborate product delivery for modest-to-large numbers of Agile software development teams. The framework provides guidance for roles, inputs, and processes for teams, programs, large solutions, and portfolios. It is also intended to provide a scalable and flexible governance framework that defines roles, artifacts, and processes for Agile software development across all levels of an organization.
Sprint	See iteration. In FMBT, the recommended sprint duration is 4 weeks.
Stakeholder	Anyone who has an interest in the program, specifically parties who may be affected by a decision made by or about the program, or who could influence the implementation of the program’s decisions. Stakeholder engagement is a key part of corporate social responsibility and for achieving the program’s vision. A group or individual with a relationship to a program change, a program need, or the solution can be considered a stakeholder.
Time box	A previously agreed-upon period of time during which a person or a team works steadily toward completing a product.
User story	The smallest level of detail in an Agile program. A requirement definition written in everyday or business language, a user story is a communication tool written by or for customers to guide developers. It can also be written by developers to express nonfunctional requirements, such as security, performance, or quality. Full system requirements consist of a body of user stories. User stories are used in all levels of Agile planning and execution. An individual user story captures the “who,” “what,” and “why” of a requirement in a simple, concise way, and can be limited in detail by what can be handwritten on a small paper notecard (also called “story”). In FMBT, a user story is a work item that takes less than a 4-week sprint to complete.

Source: GAO; Department of Veterans Affairs FMBT documentation.  │  GAO‑25‑107256

In March 2020, we issued GAO’s Cost Estimating and Assessment Guide, where we identified 18 best practices associated with a reliable cost estimate, which are summarized into four characteristics: (1) comprehensive, (2) well-documented, (3) accurate, and (4) credible.[54] According to the guide, a cost estimate is considered reliable if the assessment ratings for each of the four characteristics are substantially or fully met. If any of the characteristics are scored as not met, minimally met, or partially met, the cost estimate cannot be considered reliable.

In our March 2021 report, we found that the 2019 Financial Management Business Transformation program (FMBT) cost estimate was unreliable since it did not fully or substantially meet all characteristics associated with a reliable estimate.[55] In this review, we reevaluated those characteristics of a reliable schedule that were less than substantially or fully met: (1) comprehensive, (2) accurate, and (3) credible. We found that the 2023 FMBT cost estimate was unreliable since it did not fully or substantially meet all characteristics of a reliable estimate. See detailed summaries of our findings for each characteristic of a reliable cost estimate and its associated best practices in table 6.

Overall GAO characteristic assessment	Best practice	GAO best practice assessment	Summary
Comprehensive Original assessment: Partially met Updated assessment: Partially met	The cost estimate includes all life cycle costs.	Original assessment: partially met 2024 assessment: substantially met	The FMBT life cycle cost estimate extends far enough into the future to include sufficient cost information for the last implementation wave as well as operations and support for the program as a whole. By including all costs within the life cycle cost estimate, such as interface costs with future systems, this best practice can improve to be fully met.
	The cost estimate is based on a technical baseline description that completely defines the program, reflects the current schedule, and is technically reasonable.	Original assessment: partially met 2024 assessment: partially met	Appropriate experts developed a technical baseline, which was updated as requirements became better defined. To fully meet this best practice, the technical baseline documentation should contain approving authority signatures, consistency between documents, and a risk discussion or identification of risk. For example, there is a discrepancy in the number of interfaces between the cost estimate and technical baseline.
	The cost estimate is based on a work breakdown structure (WBS) that is product-oriented, traceable to the statement of work, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted.a	Original assessment: partially met 2024 assessment: partially met	The life cycle cost estimate has a WBS in terms of a cost element structure that contains some common elements and three levels of indenture that break larger products down into progressive levels of detail. By improving the cost element structure to use wave names instead of generic terms, match the schedule WBS, properly assign child element costs to a single parent element, include all common elements, provide insight into high-cost elements, include a dictionary that details child elements, and is updated as the estimate is refined, this best practice can improve to be fully met.
	The cost estimate documents all cost-influencing ground rules and assumptions.	Original assessment: partially met 2024 assessment: partially met	The life cycle cost estimate contains ground rules and assumptions. By identifying risks associated with assumptions, constraints, or dependencies, using cost influencing assumptions as inputs to the sensitivity and uncertainty analyses, and ensuring that the estimate aligns with all assumptions, this best practice can improve to be fully met.
Well-documented Original assessment: Substantially met Updated assessment: N/A	The cost estimate shows the source data used, the reliability of the data, and the cost estimating methodology used to derive each element’s cost.	Original assessment: substantially met 2024 assessment: N/A	The well-documented characteristic was originally scored as substantially met overall. Thus, the 2023 cost estimate was not reevaluated for these best practices as part of this review.
	The cost estimate describes how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it.	Original assessment: substantially met 2024 assessment: N/A
	The cost estimate discusses the technical baseline description and the data in the technical baseline are consistent with the cost estimate.	Original assessment: partially met 2024 assessment: N/A
	The cost estimate provides evidence that management reviewed and accepted it.	Original assessment: partially met 2024 assessment: N/A
Accurate Original assessment: Partially met Updated assessment: Substantially met	The cost estimate is based on a model developed by estimating each WBS element using the best methodology from the data collected.	Original assessment: partially met 2024 assessment: partially met	Documentation describes methodologies used. By ensuring that the cost estimating methodology used to estimate interface costs uses applicable data, and wave costs are informed by requirements, among other things, this best practice can improve to be fully met.
	The cost estimate is adjusted properly for inflation.	Original assessment: minimally met 2024 assessment: N/A	This best practice was assessed as substantially met in 2022 during audit follow-up efforts for our original assessment. Thus, the 2023 cost estimate was not reevaluated for this best practice as part of this review.
	The cost estimate contains few, if any, minor mistakes.	Original assessment: partially met 2024 assessment: substantially met	Some mistakes we previously identified were corrected, showing improvement in the updated estimate. By correcting the remaining mistakes and ensuring that the cost estimate has consistent inputs and accurate normalization and escalation, among other things, this best practice can improve to be fully met.
	The cost estimate is regularly updated to ensure that it reflects program changes and actual costs.	Original assessment: substantially met 2024 assessment: N/A	This best practice was originally scored as substantially met and thus the 2023 cost estimate was not reevaluated for this best practice as part of this review.
	The cost estimate documents, explains, and reviews variances between planned and actual costs.	Original assessment: partially met 2024 assessment: partially met	In our 2024 and original analysis, we found that actual costs are captured in the cost model along with differences between planned and actual costs. This best practice can improve to be fully met by explaining a threshold variance such that all elements whose actual costs exceed the threshold are reviewed, and including lessons learned to inform future estimates.
	The cost estimate is based on a historical record of cost estimating and actual experiences from other comparable programs.	Original assessment: partially met 2024 assessment: partially met	In our 2024 and original analysis, we found that data sources such as awarded contracts were used to inform the cost estimate. VA stated that there were no comparable programs or data available in the department, despite the program collecting actual FMBT costs from 2018 forward. By addressing the reliability of the data sources used to estimate costs for the program, as well as knowledge about the data sources, this best practice can improve to be fully met.
Credible Original assessment: Minimally met Updated assessment: Minimally met	The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs.	Original assessment: minimally met 2024 assessment: minimally met	The 2023 life cycle cost estimate report documents a sensitivity analysis. By improving the cost estimate sensitivity analysis to vary input variables and underlying assumptions for the top three cost drivers examined, among other things, this best practice can improve to be fully met.
	The cost estimate includes a risk and uncertainty analysis that quantifies the imperfectly understood risks and identifies the effects of changing key cost driver assumptions and factors.	Original assessment: minimally met 2024 assessment: minimally met	We found that the 2023 life cycle cost estimate contains a risk analysis; however, issues remain. This best practice can improve to be fully met by including a risk and uncertainty analysis within the cost estimate that quantifies risks and uncertainties of individual cost element structure elements, assumptions, or data used; considers correlation; allocates the risk-adjusted estimate to cost element structure elements; and identifies contingency for achieving the desired confidence level.
	The cost estimate employs cross-checks—or alternate methodologies—on major cost elements to validate results.	Original assessment: not met 2024 assessment: not met	VA has not performed cross-checks on the major cost elements.
	The cost estimate is compared to an independent cost estimate conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.	Original assessment: not met 2024 assessment: partially met	VA provided a July 2024 independent cost estimate and supporting Cost Analysis Requirements Document that were both developed by the Institute for Defense Analysis. This best practice can improve to be fully met by developing an independent cost estimate that is reconciled to the cost estimate and shares the same technical basis and ground rules and assumptions, among other things.

Legend:

FMBT = Financial Management Business Transformation program

N/A = not applicable

VA = Department of Veterans Affairs

Met = VA provided evidence that satisfies the entire criterion

Substantially met = VA provided evidence that satisfies a large portion of the criterion

Partially met = VA provided evidence that satisfies about half of the criterion

Minimally met = VA provided evidence that satisfies a small portion of the criterion

Not met = VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

Note: To determine the overall assessment for each characteristic, we assigned each best practice assessment a score based on a five-point scale: not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. We calculated the average of the individual best practice assessment scores to determine the overall assessment rating for each characteristic as follows: not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0.

^aWBS is a framework for planning and assigning responsibility for work necessary to accomplish a program’s objectives. It deconstructs a program’s end product into smaller specific elements that are suitable for management control.

In December 2015, we issued the Schedule Assessment Guide. The guide identifies 10 best practices associated with a reliable schedule, which are summarized into four characteristics: (1) comprehensive, (2) well-constructed, (3) credible, and (4) controlled.[56] According to the Schedule Assessment Guide, a schedule is considered reliable if the assessment ratings for each of the four characteristics are substantially or fully met. If any of the characteristics are scored as not met, minimally met, or partially met, the schedule cannot be considered reliable.

In March 2021, we reported that the FMBT 2020 schedule was unreliable since it did not fully or substantially meet all characteristics associated with a reliable schedule.[57] In this review, we reevaluated those characteristics of a reliable schedule that were less than substantially or fully met: (1) comprehensive, (2) well-constructed, and (3) credible. We found that the FMBT 2024 schedule was unreliable since it did not fully or substantially meet all characteristics of a reliable schedule. See detailed summaries of our findings of each characteristic of a reliable schedule and their associated best practices in table 7.

Overall GAO characteristic assessment	Best practice	GAO best practice assessment	Summary
Comprehensive Original assessment: Partially met Updated assessment: Partially met	Capturing all activities	Original assessment: partially met 2024 assessment: substantially met	The integrated master schedule captures all activities and adheres to a hierarchical work breakdown structure. This best practice can improve to be fully met by improving the work breakdown structure to be fully consistent between projects and align with the program cost estimate.
	Assigning resources to all activities	Original assessment: minimally met 2024 assessment: minimally met	The program manages resources through their Agile processes. This best practice can improve to be fully met by incorporating processes that intend to have a fully resource-loaded schedule that includes resources within the integrated master schedule.
	Establishing the durations of all activities	Original assessment: substantially met 2024 assessment: N/A	This best practice was originally scored as substantially met and thus the 2024 schedule was not reevaluated for this practice as part of this review.
Well-constructed Original assessment: Partially Met Updated assessment: Substantially Met	Sequencing all activities	Original assessment: partially met 2024 assessment: substantially met	The 2024 schedule has a notable reduction in the number of tasks missing dependencies and tasks with date constraints. By correcting issues that remain in network logic, this best practice can improve to be fully met.
	Confirming that the critical path is valid	Original assessment: partially met 2024 assessment: partially met	We found several issues in project-level critical paths—the paths of longest duration through the sequence of activities. For example, the Veterans Health Administration individual project schedule included level of effort activities and differed from the manually calculated longest path. Additionally, two projects captured as planning tasks are not logically linked to other efforts, preventing a valid total program critical path. By addressing these issues, this best practice can improve to be fully met.
	Ensuring reasonable total float	Original assessment: partially met 2024 assessment: substantially met	The program made improvements in the 2024 schedule by reducing the amount of float within the schedule to reasonable levels. By continuing to make further improvements, this best practice can improve to be fully met.
Credible Original assessment: Partially Met Updated assessment: Partially Met	Verifying that the schedule can be traced horizontally and vertically	Original assessment: partially met 2024 assessment: partially met	The schedule should be horizontally traceable, meaning that it links products and outcomes associated with other sequenced activities, and it should be vertically traceable, meaning that varying levels of activities and supporting sub-activities can be traced within the schedule. We found that the program made improvements to the 2024 schedule’s horizontal traceability. By addressing remaining issues with vertical traceability and improving consistency between integrated master schedule dates and those in the program summary milestone chart, this best practice can improve to be fully met.
	Conducting a schedule risk analysis	Original assessment: minimally met 2024 assessment: partially met	We found that the program improved its process of identifying and managing schedule risk by conducting an initial schedule risk analysis for the 2024 schedule. By performing a complete schedule risk analysis on the entire FMBT effort that is not limited in scope, this best practice can improve to be fully met.
Controlled Original assessment: Substantially Met Updated assessment: N/A	Updating the schedule using actual progress and logic	Original assessment: substantially met 2024 assessment: N/A	The controlled characteristic was originally scored as substantially met. Thus, the 2024 schedule was not reevaluated for these best practices as part of this review.
	Maintaining a baseline schedule	Original assessment: substantially met 2024 assessment: N/A

Legend:

FMBT = Financial Management Business Transformation program

N/A = not applicable

VA = Department of Veterans Affairs

Met = VA provided evidence that satisfies the entire criterion

Substantially met = VA provided evidence that satisfies a large portion of the criterion

Partially met = VA provided evidence that satisfies about half of the criterion

Minimally met = VA provided evidence that satisfies a small portion of the criterion

Not met = VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

Note: To determine the overall assessment for each characteristic, we assigned each best practice assessment a score based on a five-point scale: not met = 1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. We calculated the average of the individual best practice assessment scores to determine the overall assessment rating for each characteristic as follows: not met = 1.0 to 1.4, minimally met = 1.5 to 2.4, partially met = 2.5 to 3.4, substantially met = 3.5 to 4.4, and met = 4.5 to 5.0.

We evaluated the Department of Veterans Affairs (VA) Financial Management Business Transformation program’s (FMBT) requirements development and management processes against the best practices of chapter 5, “Requirements Development and Management in Agile,” in GAO’s Agile Assessment Guide.[58] We also considered the FMBT Scaled Agile Framework policy document as part of our analysis.[59] We determined that the program fully met two, substantially met three, and partially met three of the requirements development and management best practices, as shown in table 8.

Best practice	GAO assessment
Refine requirements	●
Test and validate the system as it is being developed	●
Ensure work is contributing to the completion of requirements	◕
Elicit and prioritize requirements	◕
Manage and further refine requirements	◕
Ensure requirements are complete, feasible, and verifiable	◑
Maintain traceability in requirements decomposition	◑
Balance customer and user needs and constraints	◑

Legend:

FMBT = Financial Management Business Transformation program

VA = Department of Veterans Affairs

● = Fully met: VA provided evidence that satisfies the entire criterion.

◕ = Substantially met: VA provided evidence that satisfies a large portion of the criterion.

◑ = Partially met: VA provided evidence that satisfies about half of the criterion.

◔ = Minimally met: VA provided evidence that satisfies a small portion of the criterion.

○ = Not met: VA provided no evidence that satisfies any of the criterion.

Source: GAO analysis of VA FMBT documentation.  |  GAO‑25‑107256

Additional details on our assessments for each of these best practices are summarized below. A glossary of key terms associated with Agile development can be found in appendix II.

Refine requirements. FMBT fully met this best practice. This best practice is met when requirements are further refined as part of ongoing backlog refinement.

FMBT has a requirements management plan that states that Agile teams are to conduct backlog refinement. Backlog refinement involves breaking down and clarifying large, vague work items. The program also has a backlog refinement job aid that describes how to manage and refine backlog requirements. Additionally, the program provided evidence of this guidance being implemented. Specifically, FMBT Agile teams regularly hold backlog refinement meetings to review acceptance criteria and obtain product owner approval for user stories.

Test and validate the system as it is being developed. FMBT fully met this best practice. This best practice is met when (1) continuous integration and automated testing are used in the build process and (2) the product owner agrees and accepts the definition of done for each user story.

FMBT provided evidence of its testing policies and automated regression test results. For example, the program provided test strategy documents for multiple implementation waves, which visualize testing processes; provided overviews of test events, risks, and mitigations; and summarized various testing approaches (e.g., user acceptance testing). The program also provided detailed testing reports, which included the individual steps for each test script and any error messages generated.

The program provided its FMBT Scaled Agile Framework document, which includes standard, program-wide definitions of done and ready.[60] Additionally, this policy document states that the product owner is to collaborate with core teams and stakeholders to agree on a definition of done either at or beyond the FMBT standard definition.

According to program officials, product demonstrations are conducted at the end of sprint cycles to validate user stories with product owners and subject matter experts, and demonstrations for customers are held at various stages of development, allowing customers to observe functionality and provide feedback to refine the product.

Ensure work is contributing to the completion of requirements. FMBT substantially met this best practice. This best practice is met when (1) Agile teams are continuously working on tasks that directly contribute to the completion of user stories committed to for that iteration and (2) the product owner and Agile teams ensure that the committed user stories contribute to the commitments made to oversight bodies.

The FMBT Scaled Agile Framework document states that product owners are to evaluate acceptance criteria throughout a sprint, and acceptance criteria are not to be changed without agreement between the product owner and Agile team.[61] Additionally, while Agile teams are encouraged to commit to any amount of work for upcoming sprints, teams are expected to deliver on their commitments. Furthermore, capacity metrics can assist teams in committing to a realistic amount of work. FMBT provided evidence of regularly held sprint meetings in which product owners confirm acceptance criteria and team members agree on the work they will commit to for the sprint cycle.

The program’s Agile job aid on backlog refinement instructs teams to rank user story lists by priority and to continuously refine the product backlog across multiple sprint cycles. According to program officials, the VA Office of Acquisition and Logistics provides oversight and assists in prioritizing acquisitions-related work, while the VA Financial Services Center performs similar oversight for finance-related work. However, there is no apparent ranking of the requirements in the backlog, and the program lacks an Agile road map, which could be used to align user stories under development with the scope of a feature or epic. Both of these deficiencies prevent this best practice from being fully met.

Elicit and prioritize requirements. FMBT substantially met this best practice. This best practice is met when (1) a strong commitment exists to ongoing elicitation and refinement of new requirements to meet the changing needs of both the organization and the user, along with the evolving technical landscape, while managing requirements that have already been defined; (2) the process relies on surveys, forums, and other mediums in order to effectively understand the needs of the organization; and (3) nonfunctional requirements are accounted for using regulations or elicited through coordination with customers throughout the organization.[62]

There is an ongoing process to elicit and refine requirements. For example, FMBT provided a requirements management plan that outlines the approach to documenting and eliciting functional requirements, as well as obtaining stakeholder consensus. Additionally, the program provided a program management plan, which states that stakeholders are to be continuously updated on the program and potential impacts to their roles.

FMBT uses surveys to understand the needs of the organization and its customers. For example, customer surveys are conducted once users begin using the system. According to program officials, further customer feedback is gathered through sprint meetings, demonstrations, refinement sessions, and office hours. Additionally, the surveys link customer responses with possible action for leadership to take.

According to the program’s requirements management plan, software tools are used to manage processes and items (e.g., epics, features, user stories) that support the Agile life cycle; create the requirements traceability matrix (RTM); capture business requirements, nonfunctional requirements, data conversion, and new interfaces; and maintain traceability to user stories. According to program officials, identification and management of nonfunctional requirements is an iterative and ongoing activity; nonfunctional requirements are documented within epics and features. However, nonfunctional requirements are not identified as such in the requirements backlogs that the program provided, and it is not clear how they are considered, which prevents this best practice from being fully met.

Manage and further refine requirements. FMBT substantially met this best practice. This best practice is met when (1) additions and refinements to requirements are managed efficiently and effectively in an evolving, ranked backlog and (2) the backlog contains functional and nonfunctional requirements and bugs or defects representing revisions to existing functionality.

FMBT provided requirements backlogs that contained functional and nonfunctional requirements, as well as defects. The program additionally provided a job aid on backlog refinement, which states that Agile teams are to rank user stories in the backlog by priority and continuously refine the backlog across multiple sprint cycles. The program provided evidence of routine backlog refinement meetings throughout an implementation wave. According to program officials, backlog items are presented in order of priority. However, the backlogs provided do not indicate any ranking, which prevents this best practice from being fully met.

Ensure requirements are complete, feasible, and verifiable. FMBT partially met this best practice. This best practice is met when (1) prior to development, an overall definition of done and acceptance criteria for requirements are established and (2) a definition of ready may also be established as Agile teams work to set an expectation of the level of detail needed before teams can start development on a user story.

The FMBT Scaled Agile Framework document includes standard, program-wide definitions of done and ready.[63] This policy document also states that backlog items to be included in an upcoming sprint must have acceptance criteria and definitions of done and ready defined prior to the start of the sprint. FMBT also provided evidence of recurring backlog refinement meetings in which acceptance criteria were confirmed with the product owner.

However, across the user story lists provided by the program, some definitions of done and ready were left blank. Furthermore, the program did not provide a large-scale example of acceptance criteria inclusion across user stories. According to GAO’s Agile Assessment Guide, without clear acceptance criteria and definitions of done and ready, an Agile team may be working inefficiently and on requirements that are not high ranking.[64] FMBT provided updated data integrity standards and guidance, which requires Agile teams to include acceptance criteria and the definitions of done and ready when documenting user stories. Program officials stated that this updated guidance will lead to more consistent inclusion of acceptance criteria and the definitions of done and ready in FMBT user stories.

Maintain traceability in requirements decomposition. FMBT partially met this best practice. This best practice is met when (1) requirements can be traced from the source requirement (e.g., feature) to lower-level requirements (e.g., user story) and back again and (2) the program uses Agile artifacts, such as a road map, to ascertain requirements traceability.

FMBT maintains user story lists, requirements backlogs, and RTMs that include fields for epic, feature, user story, and use unique requirements identifiers for traceability. However, across each implementation wave’s respective user story lists, backlogs, and RTMs, traceability of requirements between these documents is irregular. For example, less than half of the user stories in the Consolidated Wave Stack user story list were present in that wave’s respective RTM. Furthermore, none of the user stories in the Veterans Benefits Administration Loan Guaranty program’s user story list were present in that wave’s respective requirements backlog. Without demonstrating requirements traceability, FMBT cannot justify whether it is meeting commitments made to oversight bodies and, in turn, contributing to the goals of the program, thereby providing value.

FMBT provided an implementation timeline, which it refers to as a road map. However, this timeline is not an Agile road map as defined in GAO’s Agile Assessment Guide, as it does not facilitate two-way traceability between high-level requirements (e.g., epics, features) and low-level requirements (e.g., user stories).[65] Without tools, like a road map, to facilitate frequent information dissemination, decision-makers may not have access to performance information and may not be able to make improvements or take corrective actions in a timely manner.

Balance customer and user needs and constraints. FMBT partially met this best practice. This best practice is met when (1) a consistent process is in place to measure the value of work to ensure that user stories are developed based on relative value and (2) backlog refinement is an ongoing, collaborative process between the product owner and the developers.[66]

FMBT provided job aids related to backlog refinement and measuring the value of work. For example, the program’s job aid on backlog refinement includes guidance and information on what backlog refinement is, how often to do it, who to involve, and how backlog items are organized hierarchically. This document also includes a backlog refinement checklist that can be used to increase the effectiveness of backlog refinement sessions. According to program officials, product owners, Agile teams, agency stakeholders, and subject matter experts work together to continuously refine the backlog and establish priorities through regular refinement sessions; the program provided evidence of sessions taking place.

Additionally, the program’s job aid on program increment objectives includes a business value scoring rubric, which is used to assess which objectives will provide the most value for customers.[67] According to program officials, assignment of value of work began with the Veterans Health Administration implementation wave. Although the program provided evidence that value of work is being assigned at a high level for this wave, there is no evidence that the value of work is being used to prioritize user stories. According to GAO’s Agile Assessment Guide, when the product owner does not consider the relative value of work, the team may develop functionality that is not immediately necessary to meet customer needs.[68] If the highest value requirements are not completed first, the customer may be left without necessary functionality.

Independent verification and validation (IV&V) (also referred to as independent review in this report) can provide management with an independent and objective assessment of a program’s processes, products, and risks throughout its life cycle. IV&V activities can help ensure the quality of program deliverables and improve business and requirements analysis, software development, and system and integration testing. Independent reviews can also help management detect and correct problems earlier on in the acquisition process. Overall, IV&V can assist acquisition programs with meeting their performance, schedule, and budget goals.

The Department of Veterans Affairs’ (VA) Office of Information and Technology, Compliance, Risk, and Remediation team, referred to as the IV&V team, plans, manages, and executes IV&V support for the Financial Management Business Transformation program (FMBT). The team conducts IV&V to determine whether the system or product meets the users’ needs and fulfills its intended purpose. The team provides independent assessment, testing, and status reports to FMBT leadership, which include program recommendations based on testing results.

The IV&V team is currently executing activities with a reduced scope in lieu of original plans. The team initially developed a comprehensive plan to define recommended activities to support FMBT. However, the team is now executing based on the agreed-upon FMBT IV&V Execution Plan developed in response to program funding cuts reported by the team. This current plan has a reduced scope for activities, consisting of independent testing of wave implementations, technical product reviews, and operations and maintenance (O&M).[69] According to officials from the team, they will reconsider full execution of the original plan when sufficient funding becomes available.

Evaluation of VA’s Implementation of IV&V

To assess the implementation of VA’s IV&V efforts, we first reviewed the five key elements and 10 sub-elements for effective IV&V identified in a prior GAO report.[70] To ensure the continued relevance and appropriateness of the key elements as evaluation criteria, we reviewed the latest versions of the relevant IV&V leading industry practices referenced in this prior report.[71] The five key elements are (1) establish decision criteria and process, (2) establish independence, (3) define program scope, (4) define program resources, and (5) establish management and oversight. We evaluated VA’s implementation of IV&V for FMBT against the 10 key sub-elements and determined that the department’s implementation met nine of these sub-elements and partially met one (see table 9).

Key elements and sub-elements of effective IV&V	GAO key sub-element assessment	Summary
(1) Establish decision criteria and process. A risk-based, decision-making process is defined to determine whether or the extent to which programs are to be subject to IV&V, to include:
1.     Establishing risk-based criteria for determining which programs, or aspects of programs, to subject to IV&V	◑	The IV&V team partially met this key element by establishing a risk-based process to determine which aspects of FMBT to subject to IV&V; however, the department’s overall decision to perform IV&V on the program was not based on documented, risk-based criteria.
2.     Establishing a process for using IV&V to improve the management of the IT acquisition/development program	●	FMBT leverages IV&V efforts for continuous improvement of the program through its implementation of a lessons learned process, a findings and recommendations process, and a risk management process, all of which provide mechanisms for identifying, communicating, and addressing areas for program enhancement based on IV&V efforts.
(2) Establish independence. The degree of technical, managerial, and financial independence required of the personnel or agents performing IV&V is defined, including:
3.     Technical, managerial, and financial independence requirements for the IV&V agent	●	The IV&V effort exhibits a sufficient degree of technical and managerial independence. Specifically, the IV&V efforts are managed and performed separately from FMBT development organizations, and the IV&V team is allowed to freely select areas to test and methods of testing, as well as report its findings to program management. Subsequent to our inquiries, in July 2024, VA transferred IV&V funding from FMBT directly to the IV&V team, establishing financial independence as well.
4.     A mechanism for reporting the results of IV&V to program oversight officials, as well as program management	●	The IV&V team has a documented process for reporting IV&V results and key findings to executive leadership, oversight officials, and program management. This process includes regular communication channels, such as weekly and biweekly reports, as well as the delivery of IV&V reports and other deliverables.
(3) Define program scope. The scope of IV&V activities is defined, including:
5.     A definition of the program activities subject to IV&V	●	The IV&V team sufficiently defines the scope of its IV&V activities and FMBT activities subject to IV&V through three primary documents: the FMBT IV&V Plan, the FMBT IV&V Execution Plan, and the FMBT IV&V Performance Work Statement. These documents collectively outline a comprehensive set of IV&V activities, encompassing areas such as system and software development, while also outlining specific tasks like independent testing and technical reviews.
6.     Validation and verification compliance criteria for each program activity subject to IV&V	●	The FMBT IV&V Plan, supplemented by comprehensive checklists, establishes specific compliance criteria for various program activities, including those related to software and system requirements and design, ensuring consistent IV&V assessments.
(4) Define program resources. The resources needed for IV&V are specified, including:
7.     The facilities, personnel, funding, tools, and techniques and methods	●	FMBT’s IV&V planning and budget documentation appropriately identify required resourcing needs for its FMBT IV&V efforts. The FMBT IV&V Plan, FMBT IV&V Execution Plan, and FMBT IV&V Performance Work Statement identify resource requirements for qualified personnel, tools and methods, and necessary infrastructure. Funding for these resources is secured through budgetary allocation detailed in the FMBT Budget Operating Plan and internal budget documentation.
(5) Establish management and oversight. The management and oversight to be performed are specified, including:
8.     The process for responding to issues raised by the IV&V effort	●	FMBT demonstrates effective management and oversight of its IV&V efforts by incorporating a structured process for addressing identified issues. This process, detailed in the integrated Financial and Acquisition Management System’s Technical Operations and Sustainment Development, Test and Release Procedure document, and the FMBT Defect Management Plan, ensures a systematic approach to defect identification, triage, and remediation.
9.     The roles and responsibilities of all parties involved in the program	●	The FMBT IV&V plan appropriately specifies the roles and responsibilities of key parties involved in FMBT management and oversight. Additionally, the FMBT Configuration Management Plan and FMBT IV&V Wave Process Framework specify the roles and responsibilities specific to FMBT configuration management and the teams involved with FMBT IV&V testing coordination.
10.   How the effectiveness of the IV&V effort will be evaluated	●	FMBT demonstrates effective management, oversight, and evaluation of its IV&V efforts. The program uses key performance indicators to track progress and assess the quality of IV&V deliverables. This information is communicated through regular leadership meetings, ensuring stakeholders remain informed of IV&V activities and their effectiveness.

Legend:

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

VA = Department of Veterans Affairs

● = Met: VA provided evidence that satisfies the entire criterion

◑ = Partially met: VA provided evidence that satisfies about half of the criterion

○ = Not met: VA provided no evidence that satisfies any of the criterion

Source: GAO analysis of VA FMBT IV&V documentation.  |  GAO‑25‑107256

FMBT IV&V Efforts for Consolidated Wave Stack and Operations and Maintenance

We compared recent independent review efforts against requirements outlined within the IV&V Performance Work Statement and IV&V Execution Plan and found that the team provided all FMBT deliverable requirements as part of its efforts for the Consolidated Wave Stack (CWS). The FMBT IV&V Performance Work Statement was developed using the original FMBT IV&V Plan and lists the specific FMBT IV&V tasks and deliverables that the contractor is to perform for both implementation waves and O&M.[72] The following tables list and describe the team’s completed deliverables stipulated within the FMBT IV&V performance work statement (table 10) and FMBT IV&V Execution Plan (table 11).

FMBT IV&V Performance Work Statement task and deliverable	Description
5.1.1 Contractor Project Management Plan
A.     Contractor Project Management Plan	Outlines the contractor’s approach, timeline, and tools to be used in execution of the task order effort.
B.     FMBT IV&V Integrated Master Schedule and Work Breakdown Structure	Encompasses all government and contractor IV&V activities and related deliverables.
5.1.2 Reporting Requirements
A.     IV&V Weekly Status Report	Covers all work completed during the reporting period and work planned for the subsequent reporting period and identifies and describes issues and their resolutions.
B.     IV&V Monthly Progress Report	Details the status of all work efforts and provides accurate, timely, and complete project information.
5.1.3 Technical Kickoff Meeting
A.     Draft Kickoff Meeting Agenda	Coordinated meeting that includes the contracting officer, contract specialist, contracting officer representative, VA program manager, and alternate program managers to discuss and collaborate on the details of the intended approach, work plan, staffing, resources, and project schedule for each effort. Meeting discussion notes on major issues, agreements, and disagreements are also documented and provided.
B.     Final Kickoff Meeting Agenda
C.    Draft Summary Technical Kickoff Report
D.    Final Summary Technical Kickoff Report
E.     Technical Kickoff Meeting Minutes
5.1.4 Onboarding
A.     Onboarding Status Report	Tracks the onboarding status of all contractor personnel and onboarding activities.
5.1.5 Staffing/Resource Planning
A.     Staffing/Resource Plan	Identifies the resource and staff requirements and includes an analysis of current and projected resource requirements.
5.1.6 Privacy Training
A.     Talent Management System Training Certificates of Completion for VA Privacy and Information Security Awareness Training	Training status for all individuals engaged on the task as part of the weekly/progress status report.
B.     VA Privacy and Information Security Awareness Signed Rules of Behavior
5.2.1.1 System Integrator Test Plan and Execution Assessment
A.     IV&V Test Plan and Execution Assessment Report	Contractor reviews of test plans, test scripts, test procedures, test design and analysis of test execution, test results, and defects for all levels of testing being verified and validated.
5.2.1.2 Product Reviews
A.     IV&V Product Review Report (IV&V Comment Tracker Spreadsheet)	Assess correctness, completeness, and consistency and validate deliverables against organization, agency, industry, or other specified acceptance criteria.
5.2.1.3 IV&V Requirements Evaluation
A.     IV&V Requirements Evaluation Report	Evaluates in-scope requirements for correctness, consistency, completeness, accuracy, readability, and testability.
5.2.1.4 IV&V Interface Requirements Analysis
A.     IV&V Interface Requirements Analysis Report	Verifies and validates that requirements for interfaces are correct, consistent, complete, accurate, and testable.
5.2.1.5 Traceability Analysis
A.     IV&V Traceability Analysis Report	Verifies traceability from requirements, design, configuration, and system integrator testing.
5.2.1.6 Test Readiness Review
A.     IV&V Implementation Wave Test Readiness Review Report	Determines if predefined criteria have been met to begin the planned test event.
5.2.1.7 Query Creation and Execution
A.     IV&V Database Queries	Supports verification and validation criteria within test scripts.
5.2.1.8 Implementation Waves Independent Testing
A.     IV&V Implementation Wave Test Plan	Includes the details necessary to support the scope of independent testing and evaluation planned for each approved set of wave requirements and business processes and all in-scope interfaces associated with the specific wave.
B.     IV&V Implementation Wave Test Procedures	Test procedures for each wave, such as order of execution, data requirements, and test data creation.
C.    IV&V Implementation Wave Test Scripts and Test Data	Manual and automated test scripts that include traceability, verification points, and expected results.
D.    IV&V Implementation Wave Requirements Traceability Matrix	Ensures IV&V testing artifacts trace to the approved requirements, associated business processes, designs, and configurations.
E.     IV&V Implementation Wave Test Executions and Defects	Ensures IV&V test execution results and uncovered defects are submitted and tracked.
F.     IV&V Implementation Wave Test Execution Summary Report	Includes the full results of the IV&V wave test execution, including specific details for each business process and interface execution and uncovered defects.
5.2.1.9 Test Planning and Execution Metrics
A.     IV&V Implementation Wave Test Planning and Execution Metrics (weekly and cumulative)	Weekly cumulative IV&V test planning and execution metrics for inclusion in the IV&V weekly status report and IV&V Implementation Wave Test Execution Summary Reports. Metrics include number of total tests per test event; number of tests executed, including pass/fail status; number of defects identified; number of requirements; user stories; and features tested.
5.2.1.10 JIRA/Xray Report Creation and Generation
A.     IV&V JIRA/Xray Reports	Reports developed in the JIRA tool that produce trending and historic data.
5.2.1.11 Implementation Readiness Review and Assessment
A.     IV&V Implementation Readiness Review and Assessment Report	Determines if entrance and exit criteria have been met and whether the product can support the operational mission. It is intended to determine the status of completion of specific actions that must be satisfactorily accomplished prior to execution of a production or deployment go-ahead decision.
5.2.1.12 Risk Management
A.     IV&V Implementation Wave Risk, Issues, Lessons Learned Submission and Tracking Report	Captures IV&V identified risks, issues, and lessons learned.
5.2.2 O&M Independent Test and Evaluation
A.     IV&V O&M Test Readiness Review Report	Determines if predefined criteria have been met to begin the planned test event.
B.     IV&V O&M Test Plan	Includes the details necessary to support the scope of independent testing and evaluation planned for each O&M release.
C.    IV&V O&M Test Procedures	IV&V test procedures for each O&M release, such as order of execution, data requirements, and test data creation.
D.    IV&V O&M Test Scripts and Test Data	Independent test scripts (both manual and automated) that include traceability, verification points, and expected results.
E.     IV&V O&M Requirements Traceability Matrix	Ensures IV&V testing artifacts trace to the approved release notes, VA-approved requirements, user stories, defects, or change request, as appropriate, following defined FMBT O&M processes and procedures.
F.     IV&V O&M Query Creation and Execution	Creates and executes queries to support verification and validation criteria within test scripts.
G.    IV&V O&M Test Executions and Defects	Ensures IV&V test execution results and uncovered defects are recorded.
H.    IV&V O&M Test Execution Summary Report	Includes the full results of the IV&V O&M test execution, including specific details for each item under test and uncovered defects.
I.      IV&V O&M Test Planning and Execution Metrics	Weekly cumulative IV&V test planning and execution metrics for inclusion in the IV&V weekly status report and the IV&V O&M Test Execution Summary Report. Metrics include number of total tests per test event; number of tests executed, including pass/fail status; number of defects identified; number of requirements; user stories; and features tested.
J.     JIRA/Xray Reports	Reports developed in the JIRA tool that use tool querying capabilities to produce trending and historic data.
K.     IV&V O&M Production Installation Verification Report	Confirms each iFAMS O&M release was successfully implemented in the iFAMS production environment.
L.     IV&V O&M Release Test Package	Includes release details specific to a planned release.
M.    IV&V O&M Risk, Issues, Lessons Learned Submission and Tracking Report	Identified and submitted risks specific to IV&V activities, findings, recommendations, and observations.

Legend:

FMBT = Financial Management Business Transformation program

iFAMS = Integrated Financial and Acquisitions Management System

IV&V = independent verification and validation

O&M = operations and maintenance

VA = Department of Veterans Affairs

Source: GAO analysis of VA FMBT IV&V efforts.  |  GAO‑25‑107256

Note: JIRA is an issue and project tracking software that the IV&V team uses for Agile and testing processes.

Table 11 lists the FMBT IV&V deliverables as stated in the FMBT IV&V Execution Plan. The FMBT IV&V Execution Plan has a limited scope of activities in comparison to the FMBT IV&V Plan.

FMBT IV&V Execution Plan deliverable	Description
5.1 FMBT IV&V O&M Test Summary and Recommendation Report	Provided at the conclusion of independent testing for each O&M sustainment release executed for iFAMS (including Performance Budget) to O&M decision-makers prior to Change Control Board voting and release into production.
5.2 FMBT IV&V Readiness Test Summary and Recommendation Report	Provided at the conclusion of all independent testing executed for the FMBT Wave as input for consideration into milestone decisions, such as go-live. All independent testing user stories, test plans, test scripts and executions are recorded in VA enterprise Agile and test tools.
5.3 FMBT IV&V Weekly Status Report	Updated and provided weekly to IV&V leadership and FMBT leadership. Independent testing status and metrics are incorporated into the weekly status report.
5.4 FMBT IV&V Bi-Weekly Leadership Report	Provided to FMBT leadership bi-weekly in advance of and discussed during the bi-weekly FMBT leadership meeting.

Legend:

FMBT = Financial Management Business Transformation program

iFAMS = Integrated Financial and Acquisitions Management System

IV&V = independent verification and validation

O&M = operations and maintenance

VA = Department of Veterans Affairs

Source: GAO analysis of VA FMBT IV&V efforts.  |  GAO‑25‑107256

FMBT IV&V Testing Results for CWS and O&M

The IV&V team conducted testing for CWS and O&M, and its results recommended that FMBT proceed with the next steps, including schedule deployment, as outlined in project and release management plans. The team reported reviewing the requirements, Agility[73] epics, Agility features, and Agility user stories to identify the features to be included within the CWS testing.[74] The team used requirements and expectations for CWS and Integrated Financial and Acquisitions Management System features from the Interface Control Document as testing input. The team reported that CWS testing yielded positive results, with the passing of 24 out of 27 manual tests and 392 out of 449 automated tests. See table 12 for an example of how the team linked requirements with each manual test, including the JIRA system requirement keys and corresponding summaries.[75]

2.1.9. Requirements linked with this test
Requirement key	Requirement summary
SQAS-6411	Initiative Notebook Epic
SQAS-6460	Verify spending controls – initiative Notebook
SQAS-6861	An appropriate error message was not displayed when project code is used without a notebook setup

Legend:

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

SQAS = Systems Quality Assurance Service

Source: FMBT IV&V Consolidated Wave Stack Test Execution Summary Report.  |  GAO‑25‑107256

Additionally, the team identified three defects through these CWS testing efforts.[76] As of June 21, 2024, the team reported that FMBT addressed these defects (see table 13).

Defect Identification	Defect summary	Initial priority	Status	Implementation wave
SQAS-6716 D-10969	New Initiative Notebook displays “Amendments” when user clicks on the Supervisors without saving first	Low	Invalid	CWS
SQAS-6816 D-10968	Sub project code search in project notebook has a required “project” field entry but allows user to search without entering required field information	Low	Done	CWS
SQAS-6958 D-10967	New Project Notebook is able to be created against an Inactive Initiative Notebook	Medium	Done	CWS

Legend:

CWS = Consolidated Wave Stack

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

SQAS = Systems Quality Assurance Service

Invalid = FMBT and the IV&V team determined the defect is no longer valid or applicable

Done = FMBT and the IV&V team determined the defect is resolved

Source: Department of Veterans Affairs FMBT IV&V Test Execution Summary Report for CWS.  |  GAO‑25‑107256

The team also identified one finding and made a related recommendation through these CWS testing efforts, which the team reported that FMBT addressed.[77] Specifically, the team found a duplicate organization name in the data entry rules for accounting templates and recommended consolidating it to a single entry or using a more general term.

Other FMBT IV&V Efforts and Results for CWS

The IV&V team conducted an IV&V Requirements Evaluation for CWS to evaluate in-scope requirements for correctness, consistency, completeness, accuracy, readability, and testability. As part of this evaluation, the team reported inspecting one or more requirements for each of the 129 features, covering a total of 365 user stories from the program’s Agility tool that represent all types of requirements and tasks. The evaluation resulted in 19 related findings and recommendations.

The team conducted a System Traceability Analysis for CWS to assess the draft functional requirements traceability matrix and validate that business requirements, user stories, and test cases are appropriately linked to one another and are correct, complete, and accurate. As part of this analysis, the team reported reviewing 205 out of the 205 related user stories and had nine related findings and recommendations.

The team also conducted an Interface Requirements Evaluation for CWS to verify and validate that the requirements for interfaces are correct, consistent, complete, accurate, and testable. As part of this evaluation, the team reported reviewing 157 out of 157 total requirements, resulting in eight related findings and recommendations.

Additionally, the team conducted an Implementation Readiness Review and Assessment for CWS to review, verify, validate, and evaluate the wave for deployment readiness. As part of this review, the team reported examining the integrated system testing agility backlog to validate that related efforts were completed and closed out. The team found that 476 tests passed, 246 were canceled, and seven failed. The team also reported evaluating the User Acceptance Testing Stories Agility backlog, identified 65 user stories, and found that 307 related tests passed and 12 failed. As a result, the team recommended that the CWS wave continue with its deployment as scheduled.

FMBT IV&V Team- Identified Defects and Findings and Recommendations

FMBT IV&V efforts have resulted in the identification and substantial resolution of FMBT defects, as well as findings and recommendations. IV&V support provides VA insight into the configured FMBT solution and identifies bugs and defects prior to user acceptance events and production release. We determined the extent to which VA addressed FMBT IV&V team-identified defects for implementation waves and O&M from November 2020 through November 2024. As of November 2024, the team reported that FMBT resolved 497, or 89 percent, of the 561 identified defects (see table 14).

	Defect status
	Backlog	Done	Invalid	Grand total
Operations and maintenance	42	492	21	555
Critical	0	7	0	7
High	0	19	0	19
Medium	34	434	20	488
Low	8	32	1	41
Wave	0	5	1	6
Medium	0	5	0	5
Low	0	0	1	1
Grand total	42	497	22	561

Legend:

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

Backlog = FMBT has not yet addressed, resolved, or assigned the defect to a development team

Done = FMBT and the IV&V team determined the defect is resolved

Invalid = FMBT and the IV&V team determined the defect is no longer valid or applicable

Source: GAO analysis of Department of Veterans Affairs FMBT IV&V defects tracking spreadsheet.  |  GAO‑25‑107256

Note: The defect priority indicates the urgency by which the defect should be fixed according to business need and resource availability. The four different priorities are: critical, high, medium, and low. Defect priorities can change over time, providing a means to balance and focus development efforts.

We determined the extent to which VA addressed FMBT IV&V findings and recommendations for implementation waves and O&M from April 2021 through November 2024. As of November 2024, the team reported that FMBT resolved 393, or 93 percent, of the identified 424 findings and recommendations (see table 15).

	Finding and recommendation status
	Assessing	Done	Open	Pending response	Ready for verification	Submitted	Grand total
Operations and maintenance	3	227	4	4	19	1	258
Finding	2	165	1	0	16	0	184
Recommendation	1	62	3	4	3	1	74
Wave	0	166	0	0	0	0	166
Finding	0	165	0	0	0	0	165
Recommendation	0	1	0	0	0	0	1
Grand total	3	393	4	4	19	1	424

Legend:

FMBT = Financial Management Business Transformation program

IV&V = independent verification and validation

Assessing = IV&V team is assessing FMBT’s disposition on the finding or recommendation

Done = IV&V team concurs with FMBT implementation or rejection of the finding/recommendation

Open = First status in the life cycle where IV&V team initially agrees on the finding and recommendation

Pending response = IV&V team has not received FMBT response in 5 or more business days

Ready for verification = FMBT accepts or partially accepts (with IV&V team concurrence) the finding/recommendation

Submitted = IV&V team provided the finding/recommendation to FMBT

Source: GAO analysis of Department of Veterans Affairs FMBT IV&V findings and recommendations tracking spreadsheet.  |  GAO‑25‑107256

GAO Contacts

Paula M. Rascona, (202) 512-9816 or rasconap@gao.gov

Brian P. Bothwell, (202) 512-6888 or bothwellb@gao.gov

Vijay A. D’Souza, (202) 512-7650 or dsouzav@gao.gov

Staff Acknowledgments

In addition to the contacts named above, Michael LaForge (Assistant Director), Eric Trout (Assistant Director), Jason Lee (Assistant Director), Lisa Rowland (Analyst in Charge), Seth Brewington, Giovanna Cruz, Jennifer Echard, Ash Harper, Jennifer Leotta, Cory Mazer, Christian Sullano, Mary Weiland, and Benjamin Wilder made key contributions to this report.

GAO’s Mission

The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO’s email updates to receive notification of newly posted products.

Order by Phone

The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm.

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO

Connect with GAO on Facebook, Flickr, X, and YouTube.
Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts.
Visit GAO on the web at https://www.gao.gov.

To Report Fraud, Waste, and Abuse in Federal Programs

Contact FraudNet:

Website: https://www.gao.gov/about/what-gao-does/fraudnet

Automated answering system: (800) 424-5454 or (202) 512-7700

Congressional Relations

A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548

Public Affairs

Sarah Kaczmarek, Managing Director, KaczmarekS@gao.gov, (202) 512-4800, U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548

Strategic Planning and External Liaison

Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707
U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548