Skip to main content
(G A O website.)

COAST GUARD:

Additional Actions Needed to Guide Training and Assessment Efforts

GAO-25-107025. Published: Oct 30, 2024. Publicly Released: Oct 30, 2024.

COAST GUARD

Additional Actions Needed to Guide Training and Assessment Efforts

Report to Congressional Committees

October 2024

GAO-25-107025

United States Government Accountability Office

GAO Highlights

View GAO‑25‑107025. For more information, contact Heather MacLeod at (202) 512-8777 or macleodh@gao.gov.

Highlights of GAO-25-107025, a report to congressional committees

October 2024

Coast Guard

Additional Actions Needed to Guide Training and Assessment Efforts

Why GAO Did This Study

In recent years, the Coast Guard has increasingly focused on emerging mission demands such as Arctic operations and cybersecurity. Meeting these mission demands requires enhanced skills and training for the Coast Guard workforce. Within the Coast Guard, FORCECOM is responsible for developing training for Coast Guard programs to meet these and other demands, and for assessing units’ compliance with policies and procedures.

The James M. Inhofe National Defense Authorization Act for Fiscal Year 2023 includes a provision for GAO to review FORCECOM’s efforts to train and assess the Coast Guard workforce.

This report examines 1) the extent that FORCECOM has addressed leading practices in its training efforts; 2) how FORCECOM has updated training to meet certain emerging mission demands; and 3) the extent that FORCECOM has assessed Coast Guard units’ compliance with policy.

GAO analyzed Coast Guard documentation and data on FORCECOM’s training and assessment activities. GAO also interviewed Coast Guard officials, including officials at several training centers.

What GAO Recommends

GAO recommends that FORCECOM (1) ensure the accuracy of certain training data, (2) assess its guidelines on course evaluations, and (3) develop a project plan to develop and implement its data dashboard. The Department of Homeland Security concurred with these recommendations.

What GAO Found

Force Readiness Command (FORCECOM) is responsible for a wide range of Coast Guard training programs. FORCECOM uses an established model for developing, implementing, and evaluating its training. GAO found that FORCECOM generally addressed five of seven leading practices for federal training efforts. However, FORCECOM did not fully address the remaining two: to ensure that 1) measures of effectiveness are included in training and 2) training is evaluated. Specifically, FORCECOM develops learning objectives for each course describing knowledge and skills to apply on the job. Training centers are to review these learning objectives to keep up with changing conditions, but FORCECOM does not have accurate data on the date when a course was last reviewed. For example, 63 percent of course review dates for fiscal year 2023 that GAO reviewed were blank. FORCECOM also has outdated guidance for evaluating the long-term effectiveness of its training programs. Fully addressing these two leading practices will help FORCECOM’s efforts to ensure its training is effectively preparing the Coast Guard workforce to meet mission demands.

Training Boat and Vessel Fire Simulator at a Coast Guard Training Center

FORCECOM has taken steps to update training for emerging mission demands, such as cybersecurity, and modernize its training efforts. When program offices identify a need for training, FORCECOM works with them to develop solutions. For example, FORCECOM has developed online cybersecurity trainings to address a knowledge gap among the Coast Guard’s marine inspectors. FORCECOM has also taken steps to modernize its training enterprise—for example, by expanding the use of virtual reality in certain trainings. 

FORCECOM has a long-standing assessment program to evaluate Coast Guard units’ compliance with policies and procedures. FORCECOM is updating the program to gain more transparency into each unit’s compliance status. For example, FORCECOM has created a data dashboard to centralize all the assessment program data. However, FORCECOM does not have a detailed, written project plan for fully developing and implementing the dashboard. Creating a plan to develop and implement the data dashboard would help guide FORCECOM’s efforts to increase assessment program compliance.

 

 

 

Abbreviations

 

 

 

ADDIE

Analysis, Design, Development, Implementation, and Evaluation

CIC

Continuous Improvement Cycle

FORCECOM

Force Readiness Command

IUU fishing

Illegal, Unreported, and Unregulated fishing 

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

Letter

October 30, 2024

The Honorable Maria Cantwell
Chair
The Honorable Ted Cruz
Ranking Member
Committee on Commerce, Science, and Transportation
United States Senate

The Honorable Sam Graves
Chairman
The Honorable Rick Larsen
Ranking Member
Committee on Transportation and Infrastructure
House of Representatives

The U.S. Coast Guard, a multi-mission, maritime military service within the Department of Homeland Security, serves as the principal federal agency responsible for maritime safety and security. In carrying out its long-standing statutory missions, in recent years the Coast Guard has increasingly focused on emerging mission demands, including Arctic operations, cybersecurity, and illegal fishing in the Western Hemisphere. Meeting these emerging mission demands, and successfully operating in complex maritime and security domains, requires enhanced skills and training for the Coast Guard workforce.

Within the Coast Guard, Force Readiness Command (FORCECOM) is responsible for developing and implementing training for a wide range of Coast Guard programs, from search and rescue operations to engine maintenance. In fiscal year 2023, FORCECOM delivered nearly 400 unique training courses to the Coast Guard workforce, with a combined enrollment of over 23,000 students.

The James M. Inhofe National Defense Authorization Act for Fiscal Year 2023 includes a provision for us to study the Coast Guard’s training and assessment activities.[1] This report examines (1) the extent to which FORCECOM has addressed leading practices for training development and evaluation in its training efforts, (2) how FORCECOM has updated training to meet certain mission demands, and (3) the extent that FORCECOM has assessed Coast Guard units’ compliance with policies and procedures.

To address the first objective, we reviewed FORCECOM’s policies and procedures for analyzing, developing, implementing, and evaluating Coast Guard training efforts and assessed them against selected leading practices we developed in prior work for developing and evaluating training.[2] We selected seven leading practices that are relevant to FORCECOM’s training responsibilities. For example, we selected a leading practice on evaluating training because FORCECOM is responsible for measuring the impact of its training on the Coast Guard workforce, and we did not select leading practices related to funding for training programs because FORCECOM is not responsible for obtaining training funds. We assessed the extent to which FORCECOM addressed all elements of our seven selected leading practices. We then assigned a category of “generally addressed,” “partially addressed,” or “not addressed” to describe the extent to which FORCECOM training information demonstrated that FORCECOM addressed the elements of each selected leading practice.

As part of our assessment, we analyzed FORCECOM data on its training course review process and course evaluation process for fiscal year 2023, the most recent data available. We assessed the reliability of the data by checking for errors and interviewing agency officials. We identified limitations in the data on the training course review process, which we discuss in our report. In addition, we interviewed training officials at FORCECOM headquarters and five selected training centers, as well as Coast Guard program office officials, to understand how they put training policies into practice.[3]

To address the second objective, we reviewed training-related documentation pertaining to emerging Coast Guard mission demands.[4] For example, we reviewed FORCECOM’s policy on cybersecurity training, the Coast Guard’s Western Hemisphere strategy, and the Coast Guard’s training modernization strategy. We interviewed officials from Coast Guard program offices with operational responsibilities related to each mission demand, such as the Office of Cyberspace Forces. We also interviewed FORCECOM training officials at five selected training centers, as mentioned above, to understand how FORCECOM coordinates with the program offices to update training to meet mission demands.

To address the third objective, we reviewed agency documentation related to assessing Coast Guard units’ compliance with policies and procedures, such as FORCECOM’s policy for its Assessment, Inspection, and Audit program (assessment program). We analyzed the most recently available data on the extent that Coast Guard units completed their required assessments. To assess the reliability of the data, we checked it for errors and interviewed agency officials. Although data for eight assessment areas were unavailable, we determined that the available data were sufficiently reliable to report how many Coast Guard units had completed their recent assessments for 21 of 29 assessment areas. We reviewed FORCECOM’s efforts to improve its assessment program by comparing them to leading practices for project management.[5] In addition, we interviewed FORCECOM Assessment Division officials to understand how they manage the assessment program.

Additional details on our scope and methodology can be found in appendix I.

We conducted this performance audit from September 2023 to October 2024 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Background

FORCECOM oversees more than a dozen training centers across the U.S. that offer basic and specialized training to support a wide range of the Coast Guard’s operational needs. See figure 1 for an example of training facilities at a FORCECOM training center. In fiscal year 2023, FORCECOM’s training centers employed almost 2,900 full-time staff and had operating budgets totaling almost $56 million. In addition to developing training for the Coast Guard’s needs, FORCECOM also helps procure training from other agencies, including the Department of Defense.

FORCECOM is also responsible for developing and updating training policy and guidance, including methods for assessing training needs and evaluating training effectiveness. To assess training needs, FORCECOM’s Training Division is required to analyze the knowledge and skills that the Coast Guard workforce needs to execute its missions. For example, FORCECOM is to develop lists of essential job tasks for certain positions and then develop training to meet those needs. To evaluate training effectiveness, FORCECOM is to work with training centers to administer surveys to training participants and observe classroom instruction, among other things.

Figure 1: Training Facilities at the U.S. Coast Guard’s Yorktown, Virginia, Training Center

In addition to its role as the primary training entity for the Coast Guard workforce, FORCECOM is responsible for assessing Coast Guard units’ readiness to accomplish their assigned missions. FORCECOM’s Assessment Division is required to monitor Coast Guard units’ compliance with policies and procedures via its Assessment, Inspection, and Audit program (assessment program).[6] The assessment program is comprised of 29 different assessment areas that monitor a range of Coast Guard programs, from command center operations to weapons safety. Within each assessment area, Coast Guard units are to complete checklists that assess their compliance with relevant policies and procedures, such as whether aircraft pilot training requirements were completed. FORCECOM is responsible for monitoring the units’ progress throughout the assessment process.

FORCECOM Uses an Established Model for its Training but Efforts Do Not Fully Address Selected Leading Practices

FORCECOM Uses an Established Model for Developing and Implementing Training

FORCECOM uses an established model for developing and implementing training, typically referred to as the ADDIE model. The ADDIE model is segmented into five broad, interrelated elements: (1) analysis, (2) design, (3) development, (4) implementation, and (5) evaluation (ADDIE). In our prior work, we have found that these five elements help to produce a strategic approach to federal agencies’ training and development efforts.[7] Figure 2 shows how FORCECOM aligns its training process with the ADDIE model.

Figure 2: U.S. Coast Guard Force Readiness Command (FORCECOM) Process for Developing and Implementing Coast Guard Training

Note: FORCECOM may also initiate an analysis in response to a directive from Coast Guard leadership, according to FORCECOM officials.

FORCECOM’s Training Efforts Address Five of Seven Selected Leading Practices

We found that FORCECOM generally addressed five of seven selected leading practices for training development and evaluation and partially addressed the other two leading practices. Table 1 provides an example of each selected leading practice and our assessment of the extent to which FORCECOM addressed them.

Table 1: GAO Assessment of Force Readiness Command (FORCECOM) Training Efforts Against Selected Leading Practices for Training Development and Evaluation

Selected leading practice

Example of FORCECOM training efforts

GAO assessment

Ensures alignment between training efforts and agency goals

FORCECOM uses organizational, occupational, and job task analyses to identify the causes of performance gaps and determine training solutions and requirements to meet agency goals.

Generally addressed

Provides the ability to adjust training based on changing conditions and, if necessary, quickly

FORCECOM uses a rapid analysis process to quickly develop a preliminary understanding of changing conditions and can deploy training quickly in response to a crisis.

Generally addressed

Includes mechanisms to assess whether training is appropriate for addressing a skill or competency gap

FORCECOM considers whether training is the appropriate solution to a skill or competency gap and may recommend solutions besides training, such as a policy clarification or an adjustment to available equipment.

Generally addressed

Includes mechanisms to assess how to best deliver training

FORCECOM considers and uses a variety of training delivery mechanisms, including formal resident courses, on-the-job training, virtual training, and job aids.

Generally addressed

Ensures measures of effectiveness are included in training programs

Training centers conduct reviews to ensure course learning objectives are kept up to date to enable accurate measures of effectiveness, but FORCECOM’s data on course review requirements is incomplete.

Partially addressed

Allows for and incorporates stakeholder feedback

FORCECOM incorporates stakeholder feedback throughout its training development process, including through regular communication with program managers and training centers to update training as needed.

Generally addressed

Includes mechanisms to ensure training programs are evaluated

FORCECOM uses an established model to evaluate its training efforts but does not have useful data for measuring long-term effectiveness.

Partially addressed

Legend:

Generally addressed = Program training materials and related documents and interviews with FORCECOM and training center officials demonstrated that FORCECOM addressed all or most leading practice elements and no deficiency was present.

Partially addressed = Program training materials and related documents and interviews with FORCECOM and training center officials demonstrated that FORCECOM addressed some, but not all, leading practice elements and there was some deficiency present.

Not addressed = Program training materials and related documents and interviews with FORCECOM and training center officials demonstrated that FORCECOM did not address any of the leading practice elements.

Source: GAO analysis of FORCECOM training information.  |  GAO‑25‑107025

Note: We selected leading practices from GAO’s guide for assessing strategic training efforts in the federal government that were relevant to FORCECOM’s training responsibilities. We assigned one of three categories to describe the extent to which FORCECOM training information demonstrated that FORCECOM addressed the elements of each selected leading practice (see legend).

Two examples that highlight how FORCECOM generally addressed selected leading practices are that FORCECOM (1) has the ability to quickly adjust training based on changing conditions and (2) has mechanisms to assess how to best deliver training.

Training adjustments. FORCECOM generally addressed the leading practice of adjusting training based on changing conditions by having mechanisms to make timely changes. One such mechanism is a rapid analysis process that FORCECOM uses to quickly develop a preliminary understanding of a performance gap. For example, in October 2020 FORCECOM conducted a rapid analysis to quickly identify the tasks and steps required to develop a cyber literacy course for remote delivery to the general workforce. In addition, FORCECOM officials told us that FORCECOM can also deploy training solutions quickly in response to a crisis. For example, officials told us they expedited just-in-time training for law enforcement officers who were reassigned to assist with migrant interdiction activities in the Florida Straits due to an increase in migrant arrivals.

Training delivery. FORCECOM generally addressed the leading practice of having mechanisms to assess how to best deliver training. FORCECOM considers and uses a comprehensive mix of training delivery methods, including offering resident in-person courses at training centers, training conducted at local units, virtual and e-Learning training, and providing job aids such as checklists. Before developing a training curriculum, FORCECOM conducts an analysis of the training need and training goals to determine the appropriate delivery mechanisms. FORCECOM standard operating procedures allow analysts to consider several possible training solutions and delivery methods. FORCECOM officials told us that a range of training solutions may be recommended as a result of their analysis. For example, we reviewed a course curriculum for culinary specialist training that resulted from FORCECOM’s latest occupational analysis and included self-paced learning, instructor-led classroom instruction, job aids such as a student recipe booklet, and practice performance tests.

FORCECOM Has Incomplete Data to Ensure All Learning Objectives are Up to Date

FORCECOM does not have complete, reliable data on required reviews of course learning objectives and, as a result, partially addressed the leading practice of ensuring measures of effectiveness are included in training programs. Specifically, FORCECOM develops learning objectives for each course that describe the knowledge and skills that students will be required to apply on the job.[8] FORCECOM uses these learning objectives to evaluate a training’s effectiveness. However, FORCECOM does not have reliable data on the date that a course’s learning objectives were last reviewed by the relevant training center to ensure they are up to date.

FORCECOM policy requires training centers to review each course within its purview every 3 years and submit the resulting updated curriculum outline to FORCECOM.[9] This triennial review includes an assessment of a course’s learning objectives to identify and recommend any needed changes. For example, training centers are to consider whether the learning objectives include conditions that match conditions found on the job. We reviewed FORCECOM data for 299 Coast Guard-owned courses for fiscal year 2023, the most recent data available, and found that the data on a course’s most recent review were not reliable. Specifically, 63 percent of course review dates were blank (187 of 299) and, of those with review dates, just one course had a date within triennial review requirements.

FORCECOM officials told us that the data on a course’s most recent review date are not reliable because there is no clearly defined responsibility for training center personnel to enter the data into their consolidated training management data system.[10] FORCECOM officials said that obtaining accurate data on course review dates would require a manual review of each course curriculum document to confirm dates. As a result, it is difficult for FORCECOM to oversee training center compliance with triennial review requirements. Further, officials said that they have data oversight processes in place, such as routine data checks with the training centers, but that FORCECOM has not typically provided oversight for course review dates because individual training centers, not FORCECOM, are responsible for tracking course data.

Officials we interviewed at two out of five selected training centers—training centers located in Cape May, NJ, and Yorktown, VA—told us that they were not meeting the requirement to review each course every 3 years. Officials from the Yorktown training center said that they prioritize reviewing courses that change more frequently, and they use other mechanisms to ensure training is valid, such as a job task analysis that identifies the performance required for a specific job.

The Yorktown training center is also developing a Continuous Improvement Cycle (CIC) dashboard that some training centers are using to monitor key aspects of course documentation and quality assurance, including the date of a course’s last review.[11] We reviewed data from the CIC dashboard as of May 2024 and found that 131 courses had review dates uploaded into the dashboard. Of those, 65 courses—or 49 percent—had a review date within the required triennial time frame. FORCECOM officials told us that they intend to take over the CIC effort from the Yorktown training center and apply it broadly to all courses.

According to leading practices on developing training, as part of efforts to establish measures to be used in assessing training contributions, agencies should consider available data and contemplate options for improving future data collection and analysis efforts.[12] Agencies should assess their capacity to provide relevant and reliable data for fact-based decision-making. Having reliable data on the date a course was last reviewed by individual training centers could better position FORCECOM to ensure that course learning objectives are valid, reliable, and kept up to date with changing conditions. Ensuring that learning objectives are valid and reliable could also better position FORCECOM to accurately measure the effectiveness of its training programs and provide Coast Guard personnel with the knowledge, skills, and abilities they need to perform effectively. Further, ensuring FORCECOM has accurate and complete training center data on triennial course review requirements could help FORCECOM comply with Coast Guard policy.

FORCECOM Evaluates its Training Programs but Does Not Collect Useful Data to Measure Long-Term Effectiveness

FORCECOM partially addressed the leading practice of having mechanisms to ensure training programs are evaluated. FORCECOM’s standard operating procedures establish guidelines for conducting standardized evaluations within the Coast Guard training system. Specifically, FORCECOM uses the Kirkpatrick model to evaluate the effectiveness of its training, which is a commonly accepted approach that includes four levels to evaluate a training’s impact.[13] For each course, training centers are to administer a Level 1 and Level 2 evaluation to every student and analyze the results to establish trends and identify possible changes. Level 1 evaluations entail training centers distributing surveys immediately following a course that ask about a student’s satisfaction with the instruction and course material. For Level 2, training centers assess the extent to which learning has occurred because of the training through an in-course progress quiz or a performance test in a simulated environment.

Level 3 evaluations measure the degree to which training impacts performance on the job. Training centers conduct Level 3 evaluations through a survey for their courses approximately 4 to 6 months after a training is completed to gather information from both the course graduate and their supervisor.[14] For example, Level 3 evaluations may ask supervisors if a training participant’s ability to perform each learning objective has improved since completing the training. According to FORCECOM evaluation guidelines, training centers should conduct a Level 3 evaluation annually for all courses.[15] The guidelines acknowledge that sending surveys to all graduates and supervisors can cause survey fatigue and include instructions on sending pre-notice and reminder emails to increase response rates. The guidelines also establish thresholds for responding to Level 3 evaluation data. For example, the guidelines say training centers should conduct further analysis and communicate with stakeholders when the data show that over 5 percent of graduates lacked the necessary skills or knowledge to perform course learning objectives on the job.

However, training center officials described several challenges with implementing FORCECOM’s Level 3 evaluation guidelines, including a lack of adequate resources and low survey response rates leading to unusable data. For example, officials at three training centers (Yorktown, Petaluma, and Cape May) told us they did not have adequate resources and staff capacity to conduct a Level 3 evaluation for all courses annually.

Officials at all five training centers that we interviewed said that Level 3 evaluations can provide helpful anecdotal information, but that it is difficult to reliably collect Level 3 evaluation data due to low survey response rates. For example, Cape May training center officials reported a Level 3 survey response rate of 16 percent for students, according to their most recently available data.[16] Our review of a random sample of Level 3 evaluation data for six courses offered in fiscal year 2023 showed similar low response rates for students, with response rates ranging from 5 to 24 percent for the three courses for which a Level 3 evaluation had been conducted.[17] Further, FORCECOM officials told us that because uncontrollable variables can influence performance between the time of training and evaluation, it can be challenging to attribute any changes in performance reported in the surveys solely to training.

Despite the challenges around conducting Level 3 evaluations, officials at all five selected training centers told us that they aim to meet FORCECOM’s guidelines, and some training centers reported that they were taking specific actions to do so. For example, Cape May officials said that they are developing a plan to reestablish their Level 3 evaluation program in line with the guidelines.

According to leading practices, agencies should select an evaluation approach that best measures the effect of a program while also considering what is realistic and reasonable given the broader context of the issue and resource constraints.[18] When establishing guidelines for when and how the agency’s training programs will be evaluated, each agency needs to consider the challenges associated with data collection. Factors to consider when deciding the appropriate level of evaluation include available resources, size of the training audience, management interest, program visibility, and the anticipated duration of the effort.

FORCECOM officials acknowledged the challenges in collecting Level 3 evaluation data. However, FORCECOM has not reevaluated its Level 3 evaluation guidelines since 2015 to determine if they remain applicable in the context of low response rates and lack of adequate resources. By assessing its Level 3 evaluation approach and considering new guidelines, FORCECOM could help training centers better determine how to prioritize Level 3 evaluations given the challenges and limitations in collecting and using the Level 3 data. For example, training centers could focus follow-up efforts on courses where the data could have a higher impact. This could result in a more efficient evaluation process and yield more useable information for improving future trainings.

FORCECOM Has Taken Steps to Update Training to Meet Mission Demands and Modernize its Approach to Training

FORCECOM Works with Program Offices to Update or Develop New Training

FORCECOM works directly with Coast Guard program offices to update existing training and develop new training for key mission demands to ensure training is responsive to agency needs.[19] According to Coast Guard policy, program offices are responsible for establishing training requirements for their members.[20] For example, the Office of Cyberspace Forces has developed training and qualification requirements that apply to all personnel who support the Coast Guard’s operational cybersecurity missions. Specific training courses are required for certain cybersecurity positions. For example, officials said that Cyber Mission Specialists must complete their foundational training and pass the Joint Cyber Analysis Course at the U.S. Navy’s Information Warfare Training Center. After completion, several advanced training options are available depending on the trainees’ position requirements. FORCECOM is responsible for ensuring that members have access to these training opportunities.

If the program offices identify a need for updated or new training, or a knowledge and skills gap that may be related to training, they work with FORCECOM to help identify specific skill needs and develop training solutions. According to FORCECOM officials, if a Coast Guard program office has new training requirements, including new requirements resulting from changing mission demands, FORCECOM analyzes the new requirements to convert them into a list of tasks that Coast Guard members will need to perform. FORCECOM then selects a training delivery method—such as on-the-job training or job aids—to address the new performance needs in accordance with FORCECOM standard operating procedures. FORCECOM refers to this process of working with program offices as the “training triangle” (see fig. 3).

Figure 3: U.S. Coast Guard Force Readiness Command (FORCECOM) “Training Triangle”

In particular, FORCECOM has worked with program offices to update or develop new training related to several emerging Coast Guard mission demands, including Coast Guard operations in the Arctic and Western Hemisphere regions, cybersecurity, and asset modernization.

Arctic. FORCECOM has worked to deliver training in support of Arctic mission demands, according to program officials that operate in the Arctic environment. For example, Coast Guard members stationed in the Anchorage, AK, sector can attend the Coast Guard’s National Ice Rescue School for training on tactics, techniques, and procedures for rescue efforts in the Arctic. FORCECOM has also taken steps to enact Arctic training-related strategic Coast Guard initiatives—for example, by developing a course on cold weather spill response for dealing with oil spills in an Arctic environment.

FORCECOM has also helped facilitate external training to support Arctic mission demands. For example, officials from the Coast Guard’s Office of Specialized Capabilities said that after sending their units to a U.S. Army training on hazardous technology, they requested FORCECOM to analyze the U.S. Army training for alignment with Coast Guard needs. FORCECOM assessed the Army-provided training and found that it did not fully meet the Coast Guard’s qualification standards. According to officials, FORCECOM is developing a contract for a new 80-hour course that will meet the program’s operational needs and will be applicable to Arctic operations.[21]

Western Hemisphere. FORCECOM has worked with several Coast Guard program offices on key issues in Western Hemisphere operations, in particular illegal, unreported, and unregulated (IUU) fishing.[22] For example, officials from one program office said that FORCECOM worked with them to create exportable training on countering IUU fishing, which can be delivered by FORCECOM’s regional fisheries training centers on an as-needed basis to Coast Guard units before they deploy.[23]

FORCECOM has also worked to update certain training related to international inspections. According to Pacific Area program officials, Coast Guard units conducting IUU fishing enforcement operations in the Pacific area must adhere to the requirements of several regional fisheries management organizations, depending in which jurisdiction the units operate. Coast Guard program officials asked FORCECOM to analyze the various regional organizations’ requirements and simplify the training. In May 2023, FORCECOM completed an analysis of the job tasks for Living Marine Resource Boarding Officers and developed a new training to cover all international boarding activities for the various Pacific regional organizations. These efforts help support the Coast Guard’s strategic plan for operating in the Western Hemisphere

Cybersecurity. FORCECOM has contributed to several efforts to update cybersecurity training for the Coast Guard workforce. For example, FORCECOM helped improve training opportunities for the newly created Cyber Mission Specialist rating for enlisted members and the Cyber Mission Management specialty for Warrant Officers. According to cybersecurity program officials, the Coast Guard collaborated with the U.S. Navy to obtain foundational training for the Coast Guard’s cybersecurity workforce at the Navy’s Information Warfare Training Center in Pensacola, FL, and other Department of Defense facilities—efforts which contribute to the Coast Guard’s cybersecurity strategy.[24]

In addition, FORCECOM has conducted several analyses to support new or modified cybersecurity training. For example, in July 2020, FORCECOM completed an analysis of cyber literacy requirements for Coast Guard Marine Transportation System personnel. They found that marine inspectors, who were increasingly required to assess vessels’ cybersecurity vulnerabilities, did not have adequate training in this area. To address this knowledge gap, FORCECOM developed a series of online trainings for marine inspectors, according to FORCECOM officials.

Asset modernization. FORCECOM is involved in the Coast Guard acquisition process to help ensure that members receive training on how to operate new assets, such as cutters.[25] For example, FORCECOM helps ensure that training requirements are included in acquisition contracts, which traditionally require the manufacturer to provide training on how to operate new assets, including emergency procedures and maintenance, according to program officials.

In addition, program offices can support FORCECOM’s training efforts by providing learning materials for new assets. For example, officials with the Office of Boat Forces said that their office provides new boat types to the training centers when there are significant changes to the operating platforms. Boat Forces officials also said that they are working to include an acquisition contract provision that will replace the training boats at the Yorktown Training Center so that members can train on the most current platform. Similarly, officials with the Office of Shore Forces said they provided a mock training deck to the training centers that simulates the experience of working in a command center. FORCECOM officials said that training aids such as these can provide long-term cost savings—for example, it is more cost-effective to train a pilot on certain tasks in a simulator versus an actual aircraft.

FORCECOM Developed a Strategy for Modernizing its Approach to Training

FORCECOM has developed a strategy for modernizing its training enterprise and taken steps toward implementing the strategy. In 2020, FORCECOM released the Modernized Ready Learning Strategy to address ongoing workforce challenges by modernizing its training enterprise.[26] The strategy calls for improving the training experience for the Coast Guard workforce by updating training delivery and tailoring training programs to individual learners, among other things. For example, the strategy emphasizes the importance of a blended approach to learning, with a mixture of virtual, self-paced, and traditional classroom training.

FORCECOM has taken some steps to implement the strategy. As part of the strategy’s initiative to improve classroom technology, FORCECOM has expanded Wi-Fi and internet access at several of its training centers and has adopted virtual reality training for some courses. FORCECOM is also making it easier for new recruits to get credit for their pre-existing skills and experiences as they move through the Coast Guard training system. For example, if a recruit enters the Coast Guard with a culinary degree, they might only need to complete a fraction of the required training to become a culinary specialist. Furthermore, at the FORCECOM training center in Petaluma, CA, instructors have adopted a flexible learning approach for certain courses, which allows students to advance more quickly through their required training modules as they master the course material—an example of the strategy’s goal of tailoring training to individual learners. According to FORCECOM officials, these new approaches to training can address workforce challenges by increasing training efficiency and filling key roles more quickly.

FORCECOM officials described resource and staffing challenges with initially implementing the Modernized Ready Learning Strategy. For example, officials said that creating individually tailored training—one of the strategy’s overarching goals—is a resource intensive effort that would require additional training staff within FORCECOM. To meet these challenges, in October 2023, FORCECOM assessed its implementation of the Modernized Ready Learning Strategy and developed several recommendations, including the creation of measurable goals and additional on-the-job training for instructors. Moreover, FORCECOM officials hired a program coordinator in May 2024 dedicated to guiding the Coast Guard’s Modernized Ready Learning Strategy efforts going forward.

FORCECOM Has an Established Assessment Program but Does Not Have a Plan for Implementing Updates

FORCECOM Has an Established Assessment Program for Evaluating Unit Compliance with Policies and Procedures

The FORCECOM Assessment Division uses an established assessment program to evaluate Coast Guard units’ compliance with relevant policies and procedures. FORCECOM oversees 29 different assessment areas that focus on various Coast Guard functions through its Assessment, Inspection, and Audit program (assessment program). These assessment areas provide critical information to program managers and operational commanders to maintain mission readiness.

As part of the assessment program, officials within each assessment area are responsible for developing and using checklists that assess Coast Guard units’ compliance with relevant policies, doctrines, and procedures. Figure 4 provides an overview of key steps in the assessment program.

Figure 4: Key Steps in U.S. Coast Guard’s Force Readiness Command’s (FORCECOM) Assessment Program

Assessment areas may have several checklists to address different policies and procedures relevant to the unit as well as specific functions within the unit. The assessments use a risk-based decision-making process to prioritize checklist items based upon the criticality of the control being checked, risk associated with noncompliance, complexity of the task, and the frequency with which the task is normally conducted in the field. For example, the Ordnance Safety Inspection assessment area uses a checklist to monitor higher-risk controls related to weapons storage and weapons training, such as whether a unit keeps an accurate inventory of arms and ammunition, and lower-risk controls, such as whether a unit took an annual inventory of ordnance guidance. Checklists can include training-related items to ensure personnel have the correct qualifications and may also test the knowledge of unit personnel. For example, the Command Center assessment area includes a knowledge test to all watchstanders as part of the unit assessment.[27] The results of these knowledge tests can be used to make recommendations for changes in training syllabi and other training materials. Figure 5 shows an example of a checklist used in the Mission Management System assessment area.

Figure 5: Example of an Assessment Checklist Used by the Mission Management System Assessment Area

Note: This is a simplified excerpt from an assessment program checklist. The full checklist contains additional information, such as page references, that are not displayed here.

FORCECOM officials said they use the number of unit assessments completed within a set time frame for each assessment area as a key performance measure. According to FORCECOM policy, each assessment area must establish its own standard operating procedures that include unit assessment time frames and assessment types.[28] Additionally, a unit may be required to complete multiple assessments for different assessment areas. For example, a cutter unit may have to complete assessments for the Shipboard Aviation assessment area and the Navy Type Navy Owned assessment area. The officials within the assessment areas compile their units’ data for FORCECOM to upload into a data system. Table 2 shows the number of Coast Guard units that have completed their assessments, by assessment area with available data, within the required time frame, as of April 2024.

Table 2: Number of U.S. Coast Guard Units Completing or Not Completing Last Required Assessment, Inspection, and Audit by Assessment Area Within Required Time Frame (as of April 2024)

Assessment area name

Number of Coast Guard units that completed their last required assessment

Number of Coast Guard units that did not complete their last required assessment

Percent of Coast Guard units that completed their last required assessment

Aeronautical Engineering

24

2

92%

Airframe

51

19

73%

Aviation Safety

25

4

86%

Badging and Credentialing

13

8

62%

Boat Forces

235

13

95%

Buoy Deck

25

5

83%

Countering Weapons of Mass Destruction/Chemical, Biological, Radiological, and Nuclear

111

94

54%

Comprehensive Law Enforcement Assessment Readiness

152

288

35%

Command Center

39

6

87%

Finance and Administration

84

76

53%

Housing Program Management

10

2

83%

Healthcare Process

34

9

79%

Independent Duty Health Services

54

68

44%

Intelligence

35

9

80%

Mission Management System

48

4

92%

Navy Type Navy Owned

76

0

100%

Ordnance Safety Inspection

524

65

89%

Pharmacy

81

56

59%

Shipboard-Helo Aviation

31

6

84%

Safety Mobility Assistance and Response Team

768

100

88%

Vessel Traffic Services

6

6

50%

Total

2,426

840

74%

Source: GAO analysis of U.S. Coast Guard information.  |  GAO‑25‑107025

Note: This table is limited to 21 of the 29 assessment areas. We excluded eight assessment areas because FORCECOM was not able to provide complete data for those areas. Data for the Navy Type Navy Owned assessment area are current as of April 25, 2024; data for all other assessment areas are current as of April 12, 2024. For more details on our data analysis, see appendix I.

FORCECOM Is Taking Steps to Improve its Assessment Program but Has Not Established Detailed Plans for Implementing All Efforts

FORCECOM is taking steps to improve the transparency of its assessment program, including developing a data dashboard; however, FORCECOM has not developed detailed plans for its implementation. To improve its assessment program, in April 2023, FORCECOM completed a master scheduling tool to gain more visibility into assessment schedules. In May 2024, FORCECOM established a new mechanism—the Discrepancy Improvement Initiative—to more systematically address discrepancies identified in Coast Guard units’ assessments. FORCECOM also created an assessment data dashboard as a central location for assessment areas and units to automatically upload assessment data and view compliance trends across units. However, FORCECOM has not established a detailed project plan to guide its development of the data dashboard.

Master scheduling tool. In April 2023, FORCECOM developed and implemented a master scheduling tool for more visibility into the assessment program schedule. According to FORCECOM policy, FORCECOM is required to facilitate scheduling unit assessments to minimize disruption to Coast Guard operations without creating undue risk.[29] Units use the scheduling tool to observe assessment schedules and anticipate their own operational and maintenance demands while planning their assessment events. For example, the command center assessment area must schedule assessments for 45 command centers (at all sectors, districts, and areas) and test roughly 2,500 watchstanders every 2 years.

Discrepancy Improvement Initiative. In May 2024, FORCECOM released new guidance which established the Discrepancy Improvement Initiative, directing assessment areas to identify discrepancies (i.e., checklist items that units determined to be in noncompliance) more systematically and meet with FORCECOM to discuss how to address them.[30] The Discrepancy Improvement Initiative connects FORCECOM with program managers to discuss the most frequent unit discrepancies across an assessment area to increase compliance at the unit level. FORCECOM officials said the long-term goal of this initiative is to review discrepancies for an assessment area each month. FORCECOM officials plan to meet with the program managers to discuss the top unit discrepancies shown in assessments and determine whether there is a training, funding, or staffing problem, and how to address the discrepancies. The discrepancies and actions to be taken to address them are to be recorded in a memo and shared internally for visibility and follow-up.

Assessment data dashboard. FORCECOM created a data dashboard to be a central location for all assessment data, but FORCECOM has not developed a detailed, written project plan for fully developing and implementing the dashboard. FORCECOM policy states that the assessment program will ultimately have a standardized process for reporting, managing, and responding to results of assessment activities.[31] To address this requirement, FORCECOM created the dashboard as a data repository for assessment activities. FORCECOM officials said they developed the dashboard to gain more transparency into how each assessment area is performing its assessment activities. Specifically, the dashboard provides an overview of how many units within an assessment area have completed their assessments on time and how many are overdue. The dashboard can also filter by program to look at individual unit performance, and some assessment areas have included checklist data to measure compliance and see discrepancy trends. FORCECOM data show, as reflected in table 2 above, that as of April 2024, Coast Guard units completed 2,426 out of 3,266 most recent assessments within the required time frame, with at least 840 assessments that were overdue.[32]

FORCECOM officials said tracking assessment time frames in the data dashboard has helped the assessment areas manage the number of overdue assessments. FORCECOM officials said they leveraged the data to motivate assessment areas to prioritize overdue assessments. For example, officials said there were six cutters that needed to reschedule their assessments due to flow and saltwater issues in the Mississippi River. FORCECOM’s data showed one cutter was 230 days overdue, and another was 60 days overdue, resulting in an increased risk. FORCECOM used the information to negotiate with the assessment area and operational commanders to prioritize assessments on the longer overdue cutters.

FORCECOM officials also said the transparency of information in the dashboard has helped units track their own assessment needs. For example, officials said that when operational commanders can more readily view that an assessment is overdue, they are more motivated to complete the assessment and address discrepancies. In the Ordnance Safety Inspection assessment area, for example, automating the assessment program for better integration into the data dashboard has helped increase the number of assessments completed. According to its year-end report for fiscal year 2023, the Ordnance Safety Inspection assessment area completed 88 percent of automated self-assessments—a substantial increase over the 40 percent of self-assessments completed for fiscal year 2022. Additionally, FORCECOM officials said the dashboard sends automatic email reminders to units with an overdue assessment, which can help improve overall assessment compliance. FORCECOM can also leverage the data on unit trends to help assessment areas and stakeholders identify compliance issues. If FORCECOM officials notice that multiple units fail a certain checklist item, the officials can provide the information to the assessment area and other stakeholders to address it through policy changes, training, staffing, or equipment.

Although FORCECOM has taken steps to develop the data dashboard to manage the assessment process, FORCECOM has not developed a project management plan to monitor dashboard development or plan long-term goals for future development capabilities. FORCECOM officials said that they plan for the dashboard to show additional data that can inform the comprehensive year-end reports for the assessment areas and the Discrepancy Improvement Initiative.[33] However, as of September 2024, FORCECOM has incorporated detailed compliance data for three out of the 24 assessment areas in the dashboard. FORCECOM officials working on the dashboard development said they are not subject matter experts in technology development or certified in the software used for the dashboard. For these reasons, FORCECOM officials estimate that it has taken them roughly three times longer to develop the dashboard than if there was a designated subject matter expert assigned to the task. Given these circumstances, FORCECOM officials acknowledged that having a project plan for the dashboard would help define development milestones and the time frames to complete implementation.

Generally recognized leading practices for project management from the Project Management Institute call for developing detailed plans, such as project plans, to provide a road map for systematic project execution.[34] Among other elements, these plans should define project goals, work activities, time frames, and interim milestones to gauge progress.

Given the dashboard’s centrality to FORCECOM’s efforts to update its assessment program, developing a detailed project plan can help FORCECOM determine the right level of resources and personnel to make continued progress. By developing a detailed written project plan that defines project goals, work activities, time frames, and interim milestones, FORCECOM will be better positioned to guide the dashboard implementation and monitor progress toward achieving its assessment program goals of increased transparency and compliance for Coast Guard units and assessment areas.

Conclusions

FORCECOM is responsible for preparing the Coast Guard to meet current and emerging mission demands by educating and training the Coast Guard workforce to ensure each member has the skills they need to perform effectively. FORCECOM uses an established model for developing and implementing training. However, FORCECOM does not have complete, reliable data on whether its courses are being reviewed every 3 years, as required by policy. Moreover, FORCECOM has not updated its guidance on conducting certain course evaluations since 2015. Ensuring that courses are routinely reviewed and updating guidance on conducting course evaluations will better position FORCECOM to ensure that its training adequately prepares the Coast Guard workforce for their missions.

Further, although FORCECOM has taken steps to improve its assessment functions, it has not developed a project plan or milestones for a key effort—the assessment data dashboard. Having a project plan will better position FORCECOM to fully implement the dashboard and achieve its goals of increased transparency and unit compliance for the assessment program.

Recommendations for Executive Action

We are making the following three recommendations to the Coast Guard:

The Commandant of the Coast Guard should ensure that the Commander of FORCECOM documents and implements an approach for ensuring the accuracy and completeness of training center data on triennial course review requirements. (Recommendation 1)

The Commandant of the Coast Guard should ensure that the Commander of FORCECOM assesses whether FORCECOM’s Level 3 evaluation guidelines are reasonable for training centers and considers whether new guidelines could provide more useful data. (Recommendation 2)

The Commandant of the Coast Guard should ensure that the Commander of FORCECOM develops a written project plan for developing and implementing the assessment program’s dashboard, including key items such as project goals, work activities, timeline for completion, and interim milestones. (Recommendation 3)

Agency Comments

We provided a draft of this report to DHS and the Coast Guard for review and comment. In its comments, reproduced in appendix II, DHS concurred with all three of our recommendations and described the Coast Guard’s planned actions to address them. DHS also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Homeland Security, the Commandant of the Coast Guard, and the Commander of FORCECOM. In addition, this report is available at no charge on the GAO website: https://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-8777 or MacLeodH@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Heather MacLeod
Director, Homeland Security and Justice

Appendix I: Objectives, Scope, and Methodology

This report addresses the following questions:

1.     the extent to which Force Readiness Command (FORCECOM) has addressed leading practices for training development and evaluation in its training efforts;

2.     how FORCECOM has updated training to meet certain mission demands; and

3.     the extent that FORCECOM has assessed Coast Guard units’ compliance with policies and procedures.

To address these objectives, we interviewed Coast Guard officials from FORCECOM’s Assessment Division, Training Division, and Business Division, as well as FORCECOM headquarters leadership in Norfolk, VA. In addition, we interviewed training and assessment officials at five of FORCECOM’s training centers.[35] We selected the training centers in Yorktown, VA, and Petaluma, CA, because they are responsible for training the highest numbers of students among all training centers, according to data available at the time of our review. We selected the training center in Cape May, NJ, because it delivers basic training to new recruits. We selected the Pacific Regional Fisheries Training Center in Alameda, CA, and the Maritime Law Enforcement Academy in Charleston, SC, because of their relevance to key mission demands such as countering illegal fishing. We also conducted a site visit to FORCECOM’s training center in Yorktown, VA, to observe training center operations and facilities and interview training center officials about their roles and responsibilities. We selected the Yorktown training center for the site visit because it delivers training for the highest number of students among FORCECOM’s training centers. The information presented in our report is not generalizable to all FORCECOM training centers.

To address the first objective, we reviewed FORCECOM’s policies and procedures for analyzing, developing, implementing, and evaluating Coast Guard training efforts. This included its Performance, Training, and Education Manual and its standard operating procedures for conducting analysis to identify potential new training, developing training programs, and evaluating the effectiveness of training.[36] To understand how FORCECOM puts its policies and procedures into practice, we analyzed data from FORCECOM’s Training Quota Management System on training course review dates and Level 3 course evaluations for active courses for fiscal year 2023, the most recent data available.[37] For training course review data, we reviewed data for 299 Coast Guard-owned courses and identified limitations with the data which we discuss in our report. For data on Level 3 course evaluations, we assessed a nongeneralizable random sample for six courses to obtain information on student response rates.

For data on training course review dates, we also analyzed data from the Continuous Improvement Cycle (CIC) dashboard, which is a training center data tool for monitoring certain key data elements related to the training course review process. At the time of our review, 201 courses had been uploaded into the CIC dashboard. We excluded 15 courses from our analysis because the data indicated that the courses were either a pilot or no longer active. We also excluded 54 courses that had a default review date. FORCECOM officials told us that training centers use this default date when they are unable to locate the necessary document and date. We also excluded one duplicate course entry. To assess the reliability of the CIC dashboard data, we reviewed it for errors and interviewed FORCECOM officials responsible for maintaining the data. We determined that the data were sufficiently reliable for reporting on the number of courses uploaded into the CIC dashboard that had course review dates within required time frames.

Additionally, we interviewed officials from FORCECOM’s Training Division, the five selected training centers mentioned above, and several Coast Guard program offices to understand how they put training policies into practice. We selected program offices to interview that have operational responsibilities for mission demands related to the Arctic, asset modernization, cybersecurity, the Western Hemisphere, and workforce issues.[38] For more details on these selections, see below.

We assessed the extent to which FORCECOM has addressed leading practices in its training efforts by comparing FORCECOM’s policies and procedures against GAO’s guide for developing and evaluating training.[39] To identify relevant leading practices, we reviewed the GAO guide and selected practices that were relevant to FORCECOM’s training roles and responsibilities. According to the GAO guide, not all elements of the guide are applicable to all federal training programs. Therefore, we selected leading practices that were relevant to FORCECOM’s training efforts, roles, and responsibilities. For example, we selected a leading practice on evaluating training because FORCECOM is responsible for measuring the impact of its training on the Coast Guard workforce. We did not select leading practices that were not applicable to FORCECOM, such as practices related to funding training programs and communication between agency leadership and employees. After identifying and summarizing seven relevant leading practices, we assigned a category of “generally addressed,” “partially addressed,” or “not addressed” to each leading practice by reviewing FORCECOM documentation and data and interviewing FORCECOM and program office officials.

We used the following categories to describe the extent to which the evidence demonstrated that FORCECOM addressed the elements of each selected leading practice: 1) Generally addressed—the evidence demonstrated that FORCECOM addressed all or most leading practice elements and no deficiency was present; 2) Partially addressed—the evidence demonstrated that FORCECOM addressed some, but not all, leading practice elements and there was some deficiency present, and 3) Not addressed—the evidence did not demonstrate that FORCECOM addressed any of the leading practice elements.

Specifically, one analyst reviewed the criteria and evidence and made a determination for each leading practice. Next, a second analyst repeated the process and made an independent determination for each leading practice. In instances where the two analysts reached different determinations, they discussed the relevant evidence and reached a consensus determination. Another analyst reviewed the final determinations to ensure they were adequately supported by documentary and testimonial evidence.

To address the second objective, we reviewed training-related documentation on mission demands related to the Arctic, asset modernization, cybersecurity, the Western Hemisphere, and workforce issues. For example, we reviewed FORCECOM’s policy on cybersecurity training, FORCECOM’s training modernization strategy, and FORCECOM analyses of potential training needs that were completed on behalf of the program offices. In addition, we reviewed the Coast Guard’s most recent strategic plans related to these mission demands to determine whether they contained information relevant to training.[40] If the strategic plans mentioned training-related goals, we obtained information from the relevant Coast Guard program offices and from FORCECOM on efforts they had taken to implement those goals as of June 2024. We also interviewed officials from Coast Guard program offices that have operational responsibilities for mission demands related to the Arctic, asset modernization, cybersecurity, the Western Hemisphere, and workforce issues.[41] In addition to meeting with program offices, we discussed the role of training and coordinating with program offices in meeting these mission demands with training officials at several FORCECOM training centers, such as the training center in Yorktown, VA, as well as FORCECOM headquarters.

To address the third objective, we reviewed agency documentation related to assessing Coast Guard units’ compliance with policies and procedures, such as FORCECOM’s Tactics, Techniques, and Procedures for its Assessment, Inspection, and Audit program (assessment program). We interviewed officials within FORCECOM’s Assessment Division to understand how FORCECOM manages its assessment program and puts its assessment policies into practice. We reviewed FORCECOM’s efforts to improve its assessment program by comparing them to leading practices for project management.[42]

In addition, we conducted a site visit to FORCECOM’s headquarters in Norfolk, VA, for a demonstration of FORCECOM’s assessment data dashboard, which it uses to monitor units’ progress in completing their required assessments, and to speak with FORCECOM Assessment Division officials about their efforts to modernize the assessment program.

To describe the extent to which Coast Guard units are completing their assessments, we obtained and analyzed FORCECOM data on the most recently completed unit assessment for 21 out of 29 assessment areas in the assessment program.[43] We excluded eight assessment areas because the data for those areas were not available. For each of the 21 assessment areas in our analysis, we identified the date of each unit’s most recently completed assessment and compared it to the unit’s schedule requirements—i.e., whether the unit was required to complete an assessment on an annual, biennial, or triennial basis—to determine if the assessment was completed or overdue.

To assess the reliability of the data, we reviewed it for errors and omissions and interviewed agency officials who are responsible for maintaining the data. Although data were not available for all 29 assessment areas, we determined that the data included in our report were sufficiently reliable for the purpose of showing the number of completed assessments by Coast Guard units.

We conducted this performance audit from September 2023 to October 2024 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Heather MacLeod, (202) 512-8777 or MacLeodH@gao.gov

Staff Acknowledgments

In addition to the contact above, Ben Atwater (Assistant Director), Ben Nelson (Analyst-in-Charge), Nasreen Badat, Kevin Copping, Michele Fejfar, Eric Hauswirth, Phoebe Iguchi, Matt Lowney, Cristina Norland, and Christopher Zubowicz all made key contributions to this report.

GAO’s Mission

The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO’s email updates to receive notification of newly posted products.

Order by Phone

The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm.

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO

Connect with GAO on Facebook, Flickr, X, and YouTube.
Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts.
Visit GAO on the web at https://www.gao.gov.

To Report Fraud, Waste, and Abuse in Federal Programs

Contact FraudNet:

Website: https://www.gao.gov/about/what-gao-does/fraudnet

Automated answering system: (800) 424-5454 or (202) 512-7700

Congressional Relations

A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548

Public Affairs

Sarah Kaczmarek, Managing Director, KaczmarekS@gao.gov, (202) 512-4800, U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, DC 20548

Strategic Planning and External Liaison

Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707
U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548

 



[1]Pub. L. No. 117-263, div. K, tit. CXII, subtit. F, § 11253, 136 Stat. 2395, 4053-54 (2022). Under this provision, the Comptroller General of the United States is to conduct a study of the performance of FORCECOM, which shall include an assessment of the following: (1) the actions FORCECOM has taken to develop and implement training for the Coast Guard workforce; (2) the extent to which FORCECOM—(A) has made an assessment of performance, policy, and training compliance across FORCECOM headquarters and field units, and the results of any such assessment; and (B) is modifying and expanding Coast Guard training to match the future demands of the Coast Guard with respect to growth in workforce numbers, modernization of assets and infrastructure, and increased global mission demands relating to the Arctic and Western Pacific regions and cyberspace.

[2]See GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO‑04‑546G (Washington, D.C.: Mar. 2004).

[3]We interviewed officials located at training centers in Yorktown, VA; Petaluma, CA; Cape May, NJ; Alameda, CA; and Charleston, SC. We selected the training centers in Yorktown, VA, and Petaluma, CA, because they are responsible for training the highest numbers of students among all training centers, according to data available at the time of our review. We selected the training center in Cape May, NJ, because it delivers basic training to new recruits. We selected the Pacific Regional Fisheries Training Center in Alameda, CA, and the Maritime Law Enforcement Academy in Charleston, SC, because of their relevance to key mission demands such as countering illegal fishing.

[4]We included mission demands related to the Arctic and cybersecurity, among others, because they are key mission demands that are related to the Coast Guard’s statutory missions and that were cited in the provision directing us to review the Coast Guard’s training and assessment activities. Appendix I has additional details on the scope of our review.

[5]Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Seventh Edition (2021). PMBOK® is a trademark of Project Management Institute, Inc.

[6]According to FORCECOM, the Assessment Division also applies readiness metrics to identify and analyze readiness gaps and their root causes and provides immediate assistance to correct discrepancies with the goal of driving systematic resolution of issues and improving readiness.

[7]GAO‑04‑546G. GAO’s framework is designed to serve as a flexible and useful guide in assessing how agencies plan, design, implement, and evaluate effective training programs. The framework summarizes elements of effective training programs within the components of the ADDIE model.

[8]FORCECOM develops both “terminal performance objectives” that describe the required task and “enabling objectives” that describe a measurable and observable skill that is a necessary step to accomplish the related “terminal performance objective.” We collectively refer to these as learning objectives.

[9]U.S. Coast Guard, Performance, Training, and Education Manual, Commandant Instruction 1500.10D (Norfolk, VA: Apr. 2023). To meet triennial review requirements, training centers conduct a document review (in the form of a checklist) of the instructional materials to ensure they were developed according to FORCECOM’s quality standards and provide assurance that training materials remain valid.

[10]The Training Quota Management System is FORCECOM’s central repository for its training course inventory and related data, including basic course information such as course title, delivery type, and student completions.

[11]The CIC dashboard shows five data fields, including the 1) date of a course’s last review, 2) date the curriculum outline was signed, 3) date of the course’s underlying analysis, 4) date of last classroom observation, and 5) date of last Level 3 evaluation. Level 3 evaluations measure the degree to which training impacts performance on the job.

[13]In the Kirkpatrick model, Level 1 measures participants’ immediate reaction to and satisfaction with the training program. Level 2 measures the extent to which learning has occurred. Level 3 measures how training affects changes in behavior on the job. Level 4 measures the impact of the training on the agency’s mission or organizational results. According to GAO’s guide for assessing training efforts in the federal government, Level 4 evaluations can be challenging to conduct because of the difficulty and costs associated with data collection and the complexity in directly linking training programs to improved organizational performance. FORCECOM officials told us that they conduct Level 4 evaluations periodically with smaller training programs and FORCECOM’s guidance for training centers includes Level 4 evaluation method considerations and best practices.

[14]Training centers may also conduct Level 3 evaluations via face-to-face interviews or phone interviews.

[15]A course may have multiple convenings per year. The guidelines recommend that training centers complete Level 3 evaluations for at least two course convenings per year.

[16]Cape May training center officials told us that the most recently available Level 3 evaluation data were from the fourth quarter of fiscal year 2016 because the training center lost some data when they transferred to a new data system in 2022 and since then, they have not systematically collected and maintained Level 3 evaluation data.

[17]Training centers did not conduct Level 3 evaluations for three of the six randomly selected courses in fiscal year 2023. Training center officials stated that Level 3 evaluations were not conducted for two courses because of expected course changes and for one course because it is managed by volunteer members of the Coast Guard Auxiliary.

[19]The need for updated training may also come from a Congressional mandate or agency leadership. For example, in the aftermath of Operation Fouled Anchor—an investigation into sexual assault at the Coast Guard Academy—the Commandant of the Coast Guard directed the Coast Guard to identify ways to prevent sexual assault among its workforce. In response, FORCECOM reviewed the Coast Guard’s portfolio of training on sexual assault prevention and, in September 2023, made several recommendations to improve the training.

[20]For the purposes of this report, the term “program offices” refers to those entities within the Coast Guard that carry out a range of operational responsibilities, such as the Office of Boat Forces, which provides boat operations to support Coast Guard missions such as search and rescue.  

[21]GAO has previously reported on the Coast Guard’s Arctic operations. For example, see GAO, Coast Guard: Complete Performance and Operational Data Would Better Clarify Arctic Resource Needs, GAO‑24‑106491 (Washington, D.C.: Aug. 13, 2024).

[22]According to the Coast Guard’s Western Hemisphere strategy and interviews with program officials, countering IUU fishing is a focus of increasing importance to the Coast Guard.  

[23]FORCECOM operates five regional fisheries training centers across the U.S. For our review, we met with officials from the Pacific Regional Fisheries Training Center located in Alameda, CA.  

[24]We have previously reported on the Coast Guard’s efforts to train its cybersecurity workforce. In 2022, we found that the Coast Guard had fully implemented several leading practices for training its cybersecurity workforce by establishing a training and development program to help achieve mission goals and establishing relationships with educational and professional organizations to expand the pipeline of cybersecurity talent, among other things. See GAO, Coast Guard: Workforce Planning Actions Needed to Address Growing Cyberspace Mission Demands, GAO‑22‑105208 (Washington, D.C.: Sept. 27, 2022).

[25]The term cutter identifies a Coast Guard vessel 65 feet in length or greater, with accommodations for a crew to live aboard.

[26]U.S. Coast Guard Force Readiness Command, Modernized Ready Learning Strategy (Washington, D.C.: May 5, 2020).  

[27]The command center watchstanders assist Coast Guard stations and cutters by relaying communications and positions for mariners in distress during search and rescue operations.

[28]U.S. Coast Guard, Force Readiness Command (FORCECOM), Instruction M5040.6A: FORCECOM Assessment, Inspection, and Audit Governance System (Norfolk, VA: May 24, 2019).

[29]U.S. Coast Guard, Instruction M5040.6A.

[30]U.S. Coast Guard, Force Readiness Command (FORCECOM), CGTTP 1-03.3A: Assessment, Inspection, and Audit (AIA) Governance System Tactics, Techniques, and Procedures (Norfolk, VA: May 1, 2024).

[31]U.S. Coast Guard, CGTTP 1-03.3A.

[32]This analysis excludes eight of the 29 assessment areas because complete data were not available for them. For more details on our analysis, see appendix I.

[33]Three assessment areas will not be included in the dashboard because they were either paused during COVID-19 and have not yet restarted, or they are exempt from reporting their results to FORCECOM, according to officials.

[34]Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Seventh Edition (2021). PMBOK® is a trademark of Project Management Institute, Inc. The Project Management Institute is a not-for-profit association that, among other things, provides standards for managing various aspects of projects, programs, and portfolios.

[35]FORCECOM operates thirteen training centers across the U.S. We interviewed officials located at five training centers: Yorktown, VA; Petaluma, CA; Cape May, NJ; Alameda, CA; and Charleston, SC.

[36]U.S. Coast Guard, Performance, Training, and Education Manual, Commandant Instruction 1500.10D (Norfolk, VA: Apr. 2023).

[37]The Training Quota Management System is FORCECOM’s central repository for its training course inventory and related data, including basic course information such as course title, delivery type, and student completions. Level 3 evaluations measure the degree to which training impacts performance on the job.

[38]These are key mission demands that are related to the Coast Guard’s statutory missions and that were cited in the statutory provision directing us to review the Coast Guard’s training and assessment activities. We were also directed to examine how FORCECOM modified and expanded training related to infrastructure modernization, but we did not identify training related to that topic.

[39]See GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO‑04‑546G (Washington, D.C.: Mar. 2004).

[40]U.S. Coast Guard, Arctic Strategic Outlook Implementation Plan (Washington, D.C.: Oct. 2023); U.S. Coast Guard, Coast Guard Cyber Strategic Outlook (Washington, D.C.: Aug. 2021); and U.S. Coast Guard, Western Hemisphere Strategy (Washington, D.C.: Sept. 2014).

[41]Specifically, we met with the following Coast Guard program offices. To discuss the Arctic, we met with the Office of Marine Environmental Response Policy, Office of Emergency Management, Office of Aviation Forces, Office of Specialized Capabilities, Office of Boat Forces, and Office of Cutter Forces. To discuss asset modernization, we met with the Office of Boat Forces and Office of Shore Forces. To discuss cybersecurity, we met with the Office of Cyberspace Forces and the Office of Cybersecurity Programs. To discuss the Western Hemisphere, we met with the Maritime Law Enforcement Academy and the Pacific Area office. To discuss workforce issues, we met with FORCECOM officials at several training centers and at FORCECOM headquarters in Norfolk, VA.

[42]Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Seventh Edition (2021).

[43]Our data analysis for 20 of the 21 assessment areas is current as of April 12, 2024—the date FORCECOM pulled the data. For the Navy Type Navy Owned assessment area, our analysis is current as of April 25, 2024—the date FORCECOM pulled the data. These were the most recently available data as of April 2024.