The military services' depots averaged 5 to 10 months of carryover worth an average of $0.2 billion to $4.3 billion per year for fiscal years 2007 through 2018. Specifically, the Navy, Marine Corps, and Air Force averaged less than six months of carryover worth $1.0 billion, $0.2 billion, and $1.9 billion per year, respectively, during this period. The Army averaged 10 months of carryover per year for fiscal years 2007 through 2018 worth an average of $4.3 billion. The months and dollars of carryover by military service depots for fiscal years 2007 through 2018 are presented in figure 4.

Our prior work on DOD’s depot maintenance since fiscal year 2007 indicates that the top reasons for carryover was primarily due to planning issues.[25] Some carryover is acceptable to facilitate a smooth flow of work from one fiscal year to the next, and the depots may have planned carryover. However, since fiscal year 2007 we have reported seven reasons that contributed to unplanned carryover. The reasons for unplanned carryover may be related to one another or may overlap. For example, depots may accept work without the resources necessary to perform the work, and parts are listed as a resource within that category. At the same time, parts management is cited as a separate reason for unplanned carryover. According to the military service officials, the reasons for unplanned carryover that we have identified in our prior work remain valid. Figure 5 below lists the top reasons for unplanned carryover since fiscal year 2007.

• Parts management problems. We have reported on parts management issues at depots for many years, and it is a long-standing problem.[26] For example, we found in 2016 that part shortages occurring in 2011, 2013, and 2015 were a factor affecting the depots’ ability to complete maintenance and contributed to carryover.[27] In 2017, we found that DOD has made progress in forecasting for spare parts.[28] Specifically, the Defense Logistics Agency partnered with the military services to improve collaborative forecasting efforts through an analytical, results-oriented approach, such as regularly monitoring key performance metrics. The Air Force Air Logistics Complexes and the Navy Fleet Readiness Centers have transferred all retail supply, storage, and distribution functions to the Defense Logistics Agency, and integrated the Defense Logistics Agency into their parts management processes. The Marine Corps is taking steps to do so, and according to Army officials, the Army has collaborated with the Defense Logistics Agency on several projects to improve parts management performance.[29]
• Poorly defined scope of work. We found in 2013 that the lack of a well-defined scope of work was one of the causes contributing to carryover at Army depots.[30] For example, we found in 2013 that delays associated with poorly defined scope of work resulted in a depot experiencing problems with vehicle designs, reaching agreement with customers on the work to be performed, and performing tests on vehicles. In addition, we found in 2016 that the lack of a well-defined scope of work for 19 Army depot orders placed in fiscal years 2014 and 2015 resulted in $698 million of carryover.[31] The scope of work is a detailed statement of the specific work to be performed for the end item that helps determine the resources needed to perform the work including materials and spare parts, technical data, engineering drawings, equipment, facilities, and personnel.
• Changing customer requirements. In 2016, we found that the mix of customer orders at Army depots tends to change during the year as work is being performed because of operational decisions and changing customer requirements that can increase carryover at fiscal year-end.[32] For example, we found that in fiscal years 2014 and 2015 there were a total of 1,167 program changes to ordered work at Anniston Army Depot valued at $212.1 million and requiring about an additional 1.5 million work hours.[33] In addition, in 2012 we found that 25 of 60 orders contributing to Marine Corps carryover for fiscal years 2010 and 2011 involved amendments to orders accepted in the last quarter of the fiscal year that either increased order quantities or expanded the scope of work to be completed in the subsequent fiscal year.[34] According to Marine Corps officials, some of these orders or amendments to these orders were planned and funded in the fourth quarter of the fiscal year, and work could not start until it was funded according to DOD regulations.
• Work delayed until second half of the year. Since 2007, we found that the Army and Marine Corps started work on new orders later in the fiscal year because they were already performing work on other orders, which contributed to carryover.[35] For example, our analysis of Army depot carryover showed that two of the five Army depots we reviewed accepted more than 20 percent of their new orders in the last 3 months of the fiscal year, which contributed to carryover in fiscal year 2006.[36] Carryover is greatly affected by orders accepted late in the fiscal year that generally cannot be completed, and in some cases cannot even be started, prior to the end of the fiscal year.
• Increase in unplanned or underestimated orders. Accurately forecasting work is essential for ensuring that depots operate efficiently and complete work on orders as scheduled. However, we found that unplanned or underestimated orders contributed to carryover for the Army,[37] the Air Force,[38] and the Marine Corps.[39] For example, in 2011 we found that the Air Force depots received more work from the Air Force than planned. Further, the Air Force depots did not keep pace with the increase in new work orders received from year to year. In 2012, we found that Marine Corps officials cited unplanned workload as a primary driver for carryover, and that for 45 of the 60 orders for fiscal years 2010 and 2011 that we reviewed, customers increased quantities or added unanticipated workload requirements throughout the fiscal year that delayed completing work on existing orders.
• Depots accepting orders without resources. The Army, Navy, and Air Force contributed to carryover by accepting new orders without the resources available to perform the work, such as personnel, technical data, parts, or equipment. For example, in 2015 we found that the Navy accepted work on F/A-18 Hornet aircraft without enough engineers, depot artisans, support equipment, and facilities to perform the work, and additional work was needed for structural repairs with a high number of flying hours.[40] In addition, we found in 2011 that Air Force depot workforce reductions in fiscal year 2008 and during the first 4 months of fiscal year 2009 contributed to growth in carryover amounts for fiscal years 2008, 2009, and 2010.[41] Further, we found in 2008 that the Army depot maintenance budget had significantly underestimated the amount of new orders actually received from customers. Although the depots had increased the number of employees in the past to meet customer demands, the number of employees did not increase at the pace of the new orders actually received from customers, resulting in the large growth of carryover.[42]
• Battle or crash-damaged aircraft. The Army and Navy accepted orders involving work on aircraft with battle or crash damage that was difficult to predict, required nonstandard repairs, and necessitated long lead-time parts. For example, we found in 2016 that Corpus Christi Army Depot had $105 million and $71 million in carryover for fiscal years 2014 and 2015, respectively, on orders to repair crash-damaged aircraft.[43] In fiscal years 2013 and 2014, the Navy depots had $82.8 million and $81.1 million, respectively, in carryover on orders to repair crash-damaged aircraft.[44] According to Army officials, it is difficult to plan for battle and crash-damaged aircraft because the amount and precise type of work required cannot be predicted.

During the course of our review, officials stated that these reasons continue to affect the depots. They did not identify any additional reasons for carryover unrelated to those above. Since 2007, we have made 59 recommendations to address the reasons for carryover, including planning issues. At the time of our review, 46 of these recommendations have been implemented but further action is necessary to implement the remaining 13 recommendations. For example, in 2016 we recommended that the Army incorporate in its regulation provisions to improve communications and coordination with customers to address scope of work and parts issues to reduce carryover.[45] The Army has drafted a regulation to address our recommendation, but it has not been finalized.

DOD Considered Three Carryover Metrics in Its Report to Congress

DOD considered the following three options for calculating and determining allowable carryover in its report to the House Committee on Armed Services: the current metric, the OSD-proposed metric, and the Army-proposed metric.[46] Each carryover metric option and related results associated with applying them to depot data for fiscal years 2007 through 2018 are summarized below.

Current carryover metric. The current carryover metric determines allowable carryover based on multiple ways in which appropriations may be spent over time (i.e., outlay rates) requiring dozens of calculations.[47] DOD guidance allows workload to be exempted from carryover calculations if a written request is submitted and approved, and some workload is exempted from the carryover calculation on a regular basis, such as fourth quarter orders and crash-damaged aircraft. One military service may be granted exemptions that differ from exemptions granted to the other military services.[48] These exemptions can result in tens of millions of dollars of workload being excluded from the carryover calculation each year. The current metric for calculating carryover and comparing it to the allowable amount is depicted in figure 6.

Under DOD’s current carryover metric, the Army’s calculated carryover has consistently exceeded the allowable amount of carryover from fiscal year 2013 through fiscal year 2018, with calculated carryover for the other military services generally remaining within the allowable amount of carryover for the same period. DOD's current carryover metric results when applied to depot data for fiscal years 2007 through 2018 is depicted in figure 7.

OSD-proposed carryover metric. The OSD-proposed carryover metric compares a calculated carryover ratio with an allowable carryover ratio of 66 percent, which OSD states will allow for 6 months of carryover.[49] The calculated carryover ratio includes all available workload and is calculated by dividing the amount of money collected (collections) in the current year by the combined amount of the monies collected and not yet collected (available workload). Collections during the current fiscal year can be from depot work orders completed during the current or prior fiscal years. The OSD-proposed metric for calculating carryover and comparing it to the allowable amount is depicted in figure 8.

Our calculations indicate that under the OSD-proposed carryover metric the Navy, Marine Corps, and Air Force would have been within the allowable carryover amount more often than not for fiscal years 2007 through 2018, but the Army would have consistently exceeded the allowable amount. Results of applying the OSD-proposed carryover metric to budget data for fiscal years 2007 through 2018 are shown in figure 9.

Army-proposed carryover metric. The Army-proposed carryover metric is based on estimates of direct labor hours in available workload. DOD defines a direct labor hour as a common metric for measuring depot maintenance capability, workload, or capacity, representing 1 hour of direct work. The Army-proposed carryover metric includes direct labor hours for orders accepted by the depots along with projected workload to determine the total available workload, and then divides that figure by two to determine allowable carryover. The number of labor hours executed to generate revenue during the fiscal year is then subtracted from available workload to determine calculated carryover, which is then compared to allowable carryover. This approach’s focus on direct labor hours fundamentally differs from the current metric and the OSD-proposed metric. The Army-proposed metric for calculating carryover and comparing it to allowable carryover is depicted in figure 10.

Our calculations indicate that under the Army-proposed metric the Army would have been under the allowable amount of carryover consistently for fiscal years 2007 through 2018, as shown below in figure 11. However, we were unable to calculate carryover or determine the allowable amount for the other military services using the Army-proposed carryover metric because the calculations rely, in part, on projections that are not available to the other military services as well as data that has not been required under DOD regulations since December 2014 and is not produced by the other military services according to Navy and Air Force officials.[50]

The Metrics Considered by DOD Do Not Fully Address All Key Attributes of Quality Information

Based on our assessment, the metrics considered by DOD for calculating carryover do not fully address all key attributes of quality information—reliability, completeness, consistency, and appropriateness—although the Office of the Secretary of Defense-proposed carryover metric meets the most attributes as shown in figure 12.

According to Standards for Internal Control in the Federal Government management should use quality information to achieve the entity’s objectives and to inform decision making.[51] According to our analysis of those standards, leading practices in results-oriented management, and discussions with DOD officials, we determined that key attributes for providing quality information related to a carryover metric would incorporate reliable data and complete, consistent, and appropriate information, as highlighted below.

• Reliable. The metric uses reliable data if the data is readily available, is reasonably free from error or bias, and represents what it purports to represent. Data obtained through efforts that cannot be easily verified or based on estimates or projected amounts can be subject to inaccuracies.
• Complete. The metric uses complete information if it includes all relevant data needed by decision makers to assess the depot’s performance or make resource allocation decisions. Relevant data has a logical connection with, or bearing upon, the identified information requirements.
• Consistent. The metric is consistent if the military services use the same metric to calculate carryover from year-to-year.
• Appropriate. The metric is appropriate if it is communicated both internally and externally to support decisions related to depot operations and the allocation of resources. When selecting the appropriate metric, management considers the audience (intended recipients of the communication), nature of information (the purpose and type of information being communicated), and availability (information readily available when needed).

Below is a detailed comparison of each metric against the four attributes of quality information.

Current carryover metric. We found that the current carryover metric:

• partially addresses the attribute for providing reliable information. According to DOD officials, the inputs for the calculation for this metric come directly from verified data (e.g., new orders and revenue). However, if the military services do not compute carryover correctly (a process that requires dozens of calculations using rates that can change from year to year) it can lead to errors in budget materials. For example, according to DOD documents the Army had an error in its most recent budget materials because it failed to include updated fiscal year 2018 allowable carryover amounts, an error that resulted in understating Army’s allowable carryover by $48 million.[52] In addition, according to DOD’s carryover report to Congress, some of the inputs used to determine allowable carryover included financial data unrelated to depots.[53] Specifically, the outlay rates associated with different appropriations are unreliable for carryover calculations because they include expenditure information unrelated to depot maintenance.
• partially addresses the attribute for being complete. The information used includes all aspects of depot maintenance operations, such as labor and materials, in the carryover calculation. However, the current carryover metric allows some workload to be exempted, and removing any workload from the carryover calculation prevents it from being complete. Therefore, decision makers do not have all relevant data needed to assess depot performance or make resource allocation decisions.
• does not address the attribute for providing consistent information. The military services use the same methodology and reporting format, but exemptions are not applied to all military services in the same way and vary from year to year. For example, we previously reported that OSD approved three exemptions that allowed the Army to report carryover $363 million below the allowable amount at the end of fiscal year 2011 when the Army’s actual carryover was about $1 billion over the allowable amount without these exemptions.[54] These exemptions were not provided to the other services for the same fiscal year. Therefore decision makers lack consistent information on carryover to support comparison across military services.
• partially addresses the attribute for providing appropriate information. The metric expresses carryover in dollars, but it does not express carryover in months, nor does it clearly define workload exemptions and outlay rate exemptions in budget materials provided to Congress. Therefore, decision makers are not provided with sufficient disclosure about the nature of the information being presented.

According to OSD officials, they rejected the current carryover metric because, even if all exemptions were eliminated, outlay rates presented in DOD financial summary tables may not represent how funds are expended at the depots. The outlay rates provide a profile of how money appropriated for a program is expected to be spent over time according to the type of program, but according to DOD’s carryover report to Congress, the use of outlay rates is the chief weakness in the current carryover metric.[55]

OSD-proposed carryover metric. We found that the OSD-proposed metric:

• partially addresses the attribute for reliable data. The metric uses data that is readily available and is reasonably free from error or bias.[56] However, according to DOD officials, the metric does not represent actual production at the depots in a given fiscal year. Instead, the metric depends on collected amounts based on work depots may have completed in the current or prior fiscal years and the timing of payments made by the military services. Furthermore, collected amounts can be affected by transfers between funds, and accounting corrections and adjustments. On the other hand, revenue directly represents the dollar amount of work performed by depots in a single fiscal year. We found that the timing of collections may affect the carryover metric in a way that does not accurately represent carryover. For example, the Army depot collections increased by more than $3 billion from fiscal year 2015 to 2016, but revenue decreased by $0.2 billion during the same period. As a result, the OSD-proposed carryover metric would not have reflected the Army’s actual depot production if it were in use for fiscal year 2016. DOD officials agreed that basing a carryover calculation on revenue generated by the depots rather than collections would provide a more reliable approach for measuring carryover and that doing so would be supported by data provided under current DOD regulations.
• addresses the attribute for providing complete information. The metric includes all aspects of depot workload without exemptions.
• addresses the attribute for providing consistent information. The metric eliminates the exemptions that result in carryover calculation variations across the military services and allows for valid year-to-year comparisons of carryover across the military services.
• partially addresses the attribute for providing appropriate information. The metric shows whether the actual carryover ratio at depots exceeds the 66 percent benchmark, but does not provide a carryover dollar amount. The OSD-proposed carryover metric also uses an inverse ratio relationship that allows a higher amount of carryover to be expressed by a lower carryover ratio, which makes the concept of the ratio difficult to understand given that it is counter-intuitive. OSD officials acknowledged that using a 66 percent benchmark ratio that represents 6 months of carryover and having a higher ratio represent lower carryover might be confusing to decision makers, and therefore inappropriate. DOD officials also agreed that reporting carryover in terms of months and dollars with a direct relationship to actual amounts of carryover would provide the most appropriate information to support depot management and the allocation of scarce resources.

According to OSD officials, making adjustments to the OSD-proposed carryover metric may allow them to more fully address the key attributes for providing quality information on depot maintenance carryover. OSD, Navy, Marine Corps, and Air Force officials stated that this metric, by including all workload (without exemptions), provides the most complete view of carryover, and that establishing the same 66 percent benchmark for allowable carryover of 6 months across all military services is equitable and logical. OSD officials acknowledge that it may be difficult for the Army to comply with this benchmark and would plan to provide interim targets until the Army can reasonably reduce carryover to address it.

Army-proposed carryover metric. We found that the Army-proposed carryover metric:

• does not address the attribute for reliable data. Rather than basing the Army-proposed carryover metric on actual results, it is based on estimates and projections that are subject to inaccuracies and data that is not easily verifiable.
• does not address the attribute for providing complete information. The metric focuses on direct labor hours and does not include other aspects of depot maintenance carryover such as parts and materials. Therefore, the Army-proposed carryover metric does not include all relevant data needed to assess depot performance.
• does not address the attribute for consistent information. The metric is based on information that the other military services do not collect. Specifically, some of the information related to direct labor hours the Army used to perform its calculations is not required in DOD’s Financial Management Regulation, and the other military services do not collect or retain the data necessary to complete the Army-proposed metric calculations. Therefore, comparisons cannot be made across military services.
• does not address the attribute for appropriate information. The metric relies on direct labor costs and does not measure other aspects of carryover such as parts and material. Furthermore, the metric does not provide meaningful information to support decisions related to the allocation of resources.

According to Army officials, there are two key reasons they put forward an alternative carryover metric. First, having a metric that is based on direct labor hours provides a more relevant measure for what the depots can and have produced. According to the Army officials, direct labor hours are an input to the depot maintenance process that depots can readily influence, whereas the timing of customer orders and changes in customer requirements are beyond the control of depot management. However, the other services face these same conditions and have repeatedly been able to keep their carryover to around 6 months or less over much of the last 12 years. Moreover, direct labor hours are only one of many inputs of the depot maintenance process. Focusing solely on direct labor hours as an input is contrary to best practices for outcome-oriented performance metrics.[57] Specifically, outcomes are related to the extent to which a program achieves its objectives, but inefficiencies such as re-work can increase the number of labor hours expended without improving results.

Second, Army officials stated the Army recognizes revenue related to parts differently from the other military services. The Army recognizes revenue when a depot uses a part whereas the other military services recognize revenue from parts much earlier in the process and without regard for when they are used. Army officials stated this practice allows the other military services to artificially reduce reported carryover simply by receiving parts that were ordered. However, the portion of other military service’s carryover attributed to parts does not differ significantly from the Army, and any differences in the timing of revenue recognition for parts are not likely to be a principle reason the Army averages approximately 4 months more carryover than the other military services.

Congress and DOD decision makers need quality information about the amount of carryover that is acceptable to adequately address performance at depots and ensure that resources are being allocated appropriately. In developing a proposed metric, DOD’s carryover working group—comprised of representation from OSD and each of the military services—focused on addressing Congressional requirements, simplifying the carryover metric, and avoiding the need for additional resources to calculate carryover amounts.[58] However, according to DOD officials, DOD did not develop a systematic approach at the outset for evaluating a carryover metric for depot maintenance. For example, DOD did not identify key attributes, such as reliability, completeness, consistency and appropriateness, desired in its carryover metric. As a result, the working group developed the proposed carryover metric without determining if it would yield quality information. DOD officials stated that they did not discuss all the attributes of quality information appropriate for a carryover measure in their report to Congress because there was no requirement to do so. When we identified and discussed key attributes of quality information and performance measures with OSD officials, they agreed that the key attributes we identified should be considered when developing and adopting a carryover metric.

Unless DOD adopts a carryover metric for depot maintenance that results in reliable, complete, consistent, and appropriate information, decision makers may not have the quality information they need to make decisions. Specifically, DOD and Congressional leadership would not be well positioned to determine whether funds are directed to the highest priority while simultaneously assuring depots are managed as efficiently and effectively as possible.

The Metrics Considered by DOD Could Have Varied Depot Management Implications

The metrics considered by DOD could have varied depot management implications. DOD’s three carryover metric options—the current, OSD-proposed, and Army-proposed—measure carryover workload differently, with a variety of depot management implications and different results from each carryover metric.

Current carryover metric implications. Using DOD’s current metric, depot management lacks incentives to correct long-standing problems because exemptions may prevent the level of scrutiny and possible corrective actions if problem areas were more transparently reflected in higher carryover amounts. Allowing exemptions also undermines workload planning by eliminating incentives to engage in comprehensive workload planning. The reasons for carryover discussed in our prior reports since fiscal year 2007 often relate to workload uncertainties, such as changes to customer requirements and unplanned or underestimated workload. These issues occurred while the current metric for calculating carryover was in use by DOD.

OSD-proposed carryover metric implications. First, using the OSD-proposed metric sets the same benchmark for all of the military services, but Army depots are unlikely to achieve the OSD-proposed benchmark of representing 6 months of carryover in the near term based on our analysis of historical data. On the other hand, Navy and Marine Corps officials stated that a benchmark of 6 months of carryover may not provide a “stretch goal” and allowing 6 month of carryover may result in decreased management attention to depot operations and how it contribute to carryover. Second, including all workload in the calculation of carryover without exemptions under the OSD-proposed metric could provide incentives to improve workload planning, increase the level of scrutiny to workload previously excluded from calculated carryover, and prompt corrective actions to be developed. For example, including late-year workload from other military services in the carryover calculation (exempted under the current metric) may provide an incentive to improve planning for this type of work.

Army-proposed carryover metric implications. Carryover is expressed in direct labor hour dollars using the Army-proposed carryover metric, but allowable carryover amounts are not meaningful, in part, because carryover amounts are unlikely to exceed the ceiling according to our assessment of Army data. As a result, the Army-proposed carryover metric is not likely to provide an incentive to improve depot management. For example, as previously discussed, the Army had more than a year’s worth of actual carryover in 2011 and 2014 (see figure 4 above), but if this metric were in use, the Army would have remained within the allowable amount every year since 2007.

Officials of selected private industry companies and foreign militaries we met with stated they have workload that carries over across fiscal years, but they do not have a policy for determining an allowable amount of carryover. First, officials of the nine private companies we interviewed told us that their companies do not have a policy for determining an allowable amount of carryover. Officials also stated that there is no parallel to the role of carryover in the U.S. government budget process in the private sector. In the private sector, officials stated that companies generally perform work and customers pay for work according to the terms of a contract. Although not all work may be complete at the end of a year, there is no policy limiting the amount of work allowed to carry over from year to year. According to generally accepted accounting principles, funded orders or orders related to a binding agreement or contract for work not completed at year-end, can be accounted for as unearned (deferred) revenue and recorded as a liability on the entity's balance sheet. Private sector officials told us that there is no incentive to limit workload accepted if the customers’ needs can be met within the terms of the contract and the work is likely to be profitable. If private companies are not equipped to provide needed workload, they include the cost of additional resources in a business case analysis before accepting the work. Officials stated it is unlikely their customer’s resources would in any way be affected by the amount of work remaining to be completed at the end of a fiscal year.

Second, officials of the North Atlantic Treaty Organization and seven foreign militaries we interviewed told us that their organizations do not have a policy for determining an allowable amount of carryover, and their approach to depot maintenance and any carryover that may result differs in a variety of ways.

• Reliance on private industry or DOD. Officials from the North Atlantic Treaty Organization and five of the seven foreign militaries we interviewed told us that they do not have their own facilities to perform depot maintenance. Instead, weapon systems sustainment and overhaul work is outsourced (contracted out) to private industry or DOD depots.[59] French and Canadian officials stated that their militaries perform a limited amount of depot maintenance in government-owned facilities when compared to DOD depots. In addition, United Kingdom officials stated that their militaries have capabilities to manufacture and repair munitions and missiles but relies in most instances on contract support for depot-level maintenance and overhauls of ground vehicles, heavy equipment, or aircraft.
• Budgeting for depot maintenance. According to foreign military officials, other nations use a variety of different approaches to budgeting for depot maintenance workload based on their form of government. For example, all of the foreign militaries we interviewed have their defense budgets approved on a multi-year basis and most foreign militaries have limited latitude for spending outside the approved budget for each fiscal year.
• Approach. According to foreign military officials, they use approaches to depot maintenance that fundamentally differ from the approach used by U.S. military services. For example, Germany sets levels for overall weapon systems readiness and availability in the terms of a contract with a private sector provider according to the usage needs of the military, and then holds the provider accountable for addressing the terms of the contract. In contrast, the U.S. military services place orders to complete work on specific items that may or may not insure DOD’s weapon systems readiness and availability requirements are met. In other words, the U.S. military services purchase work from the depots in a batch or individual fashion to repair, overhaul, or modernize specific items, while Germany pays contractors to achieve a desired outcome.