2017 Survey of Organizational Performance and Management Issues

United States Government Accountability Office

Nuclear Regulatory Commission (NRC)

This section of the supplement presents the responses provided by mid-level and upper-level managers and supervisors at NRC to the questions on the survey, along with the 95 percent confidence interval around those results. In addition to the agency responses, we also provide the overall government-wide estimates (these government-wide estimates include the responses of managers at NRC). Although respondents were asked to provide written comments to open-ended questions, these responses are not provided here. We obtained responses from 100 managers and supervisors at NRC, for a weighted response rate of approximately 82 percent of the eligible sample. Government-wide, 2,726 responded to our survey, a weighted response rate of 67 percent. However, because of variation in the skip patterns associated with some questions and respondents choosing not to answer certain questions, the number of people who responded to each question varied.

Contents of Survey Results

• Section 1 - Background (Note: Background results are intentionally not reported.)
• Section 2a - Performance Measurement (Performance Measures)
• Section 2b - Performance Measurement (Performance Information)
• Section 3 - Agency Climate
• Section 4 - Training
• Section 5a - Cross-Agency Priority Goals
• Section 5b - Agency Priority Goals
• Section 5c - Quarterly Performance Reviews
• Section 6 - Program Evaluation
• Section 7 - Comments and Final Response Submission (Note: Comments are intentionally not reported.)

(Return to main page)

Please note:

When we refer to "your agency" in a question, we are referring to the governmental component where you work. This could be:

• An independent agency (e.g., the Environmental Protection Agency).

• A cabinet-level department (e.g., the Department of Transportation). If you work for offices that support the Secretary of your department, please consider the cabinet-level department as "your agency."

• A major component of that cabinet-level department (e.g., the Federal Aviation Administration, the Army, or the Internal Revenue Service). If you are employed by a cabinet-level department but actually work in a major component of that department, please consider the major component as "your agency."

Section 1 - Background

The results in this section are intentionally not reported.

Section 2a - Performance Measurement
(Performance Measures)

Please note:

Performance measures track progress towards achieving agency goals or objectives or measure a program / operation / project's overall performance. These measures can be quantitative or qualitative.

For the following questions, please interpret "performance measures" in the broadest sense.

4. Are there performance measures for the program(s) / operation(s) / project(s) that you are involved with? (Select one.)

5. To what extent, if at all, do you agree with the following statements as they relate to performance measures for the program(s) / operation(s) / project(s) that you are involved with? (Select one answer in each row.)

5a. We have performance measures that tell us how many things we produce or services we provide (Output measures)

5b. We have performance measures that tell us if we are operating efficiently (Efficiency measures)

5c. We have performance measures that tell us whether or not we are satisfying our customers (Customer service measures)

5d. We have performance measures that tell us about the quality of the products or services we provide (Quality measures)

5e. We have performance measures that would demonstrate to someone outside of our agency whether or not we are achieving our intended results (Outcome measures)

Section 2b - Performance Measurement
(Performance Information)

Please note:

Performance information is data collected to measure progress toward achieving an agency's established mission or program-related goals or objectives.

Performance information can focus on performance measures, such as quality, timeliness, customer satisfaction, or efficiency. It can inform key management decisions such as setting program priorities, allocating resources, or identifying program problems and taking corrective action to solve those problems.

6. To what extent, if at all, do you agree with the following statements? (Select one answer in each row.)

6a. My agency's performance information is easily accessible to managers at my level

6b. My agency's performance information is easily accessible to employees

6c. My agency's performance information is easily accessible to the public, as appropriate

6d. My agency's performance information is available in a format that is easy to use

6e. My agency has sufficient analytical tools for managers at my level to collect, analyze, and use performance information

6f. I have access to the performance information I need to manage the program(s) / operation(s) / project(s) that I am involved with

6g. Performance information is available in time to manage the program(s) / operation(s) / project(s) that I am involved with

6h. The program(s) / operation(s) / project(s) that I am involved with have sufficient staff with the knowledge and skills needed to analyze performance information

7. For those program(s) / operation(s) / project(s) that you are involved with, to what extent, if at all, do you use the information obtained from performance measurement when participating in the following activities? (Select one answer in each row.)

7a. Developing program strategy

7b. Setting program priorities

7c. Allocating resources

7d. Identifying program problems to be addressed

7e. Taking corrective action to solve program problems

7f. Adopting new program approaches or changing work processes

7g. Coordinating program efforts with other internal or external organizations

7h. Refining program performance measures

7i. Setting new or revising existing performance goals

7j. Setting individual job expectations for the government employees I manage or supervise

7k. Rewarding government employees I manage or supervise

7l. Developing and managing contracts

7m. Identifying and sharing effective program approaches with others

7n. Informing the public about how programs are performing

7o. Streamlining programs to reduce duplicative activities

8. Based on your experience with the program(s) / operation(s) / project(s) that you are involved with, to what extent, if at all, have the following factors hindered measuring performance or using the performance information? (Select one answer in each row.)

8a. Difficulty determining meaningful measures

8b. Different parties are using different definitions to measure performance

8c. Difficulty obtaining valid or reliable data

8d. Difficulty obtaining data in time to be useful

8e. Lack of ongoing Congressional commitment or support for using performance information to make program / funding decisions

8f. Difficulty determining how to use performance information to improve the program

8g. Concern that performance information could be used against my program or agency

Section 3 - Agency Climate

9. To what extent, if at all, do you agree with the following statements? (Select one answer in each row.)

9a. Agency managers / supervisors at my level have the decision making authority they need to help the agency accomplish its strategic goals

9b. Agency managers / supervisors at my level are held accountable for agency accomplishment of its strategic goals

9c. Agency managers / supervisors at my level are held accountable for the results of the program(s) / operation(s) / project(s) they are responsible for

9d. Agency managers / supervisors at my level use performance information to recognize employees for their performance

9e. Employees in my agency receive positive recognition for helping the agency accomplish its strategic goals

9f. My agency is investing the resources needed to ensure that its performance data is of sufficient quality

9g. My agency's top leadership demonstrates a strong commitment to achieving results

9h. My agency's top leadership demonstrates a strong commitment to using performance information to guide decision making

9i. My agency is investing in resources to improve the agency's capacity to use performance information

9j. Agency managers / supervisors at my level take steps to align program performance measures with agencywide goals and objectives

9k. Agency managers / supervisors at my level effectively communicate performance information on a routine basis

9l. Agency managers / supervisors at my level take steps to ensure that performance information is useful and / or appropriate

9m. Agency managers / supervisors at my level use performance information to share effective program approaches with others

10. To what extent, if at all, do you agree with the following statements? (Select one answer in each row.)

10a. The individual I report to periodically reviews with me the results or outcomes of the program(s) / operation(s) / project(s) that I am responsible for

10b. Funding decisions for the program(s) / operation(s) / project(s) I am responsible for are based on results- or outcome-oriented performance information

10c. Changes by management above my level to the program(s) / operation(s) / project(s) I am responsible for are based on results- or outcome-oriented performance information

10d. I have sufficient information on the validity of the performance data I use to make decisions

11. To what extent, if at all, do you believe that the following persons or entities pay attention to your agency's use of performance information in management decision making? (Select one answer in each row.)

11a. NRC Chairman

11b. Head of your agency subcomponent (if applicable)

11c. The individual I report to

11d. Employees that report to me

11e. Office of Management and Budget (OMB)

11f. Federal agencies other than OMB

11g. Congressional committees (e.g., members, staff)

11h. The audit community (e.g., GAO, Inspectors General)

11i. The general public

12. Compared to three years ago, how much more or less attention do the following persons or entities pay to your agency's use of performance information in management decision making? (Select one answer in each row.)

12a. NRC Chairman

12b. Head of your agency subcomponent (if applicable)

12c. The individual I report to

12d. Employees that report to me

12e. Office of Management and Budget (OMB)

12f. Federal agencies other than OMB

12g. Congressional committees (e.g., members, staff)

12h. The audit community (e.g., GAO, Inspectors General)

12i. The general public

Section 4 - Training

13. During the past 3 years, has your agency provided, arranged, or paid for training that would help you to accomplish the following tasks? (Select one answer in each row.)

13a. Conduct strategic planning

13b. Set program performance goals

13c. Develop program performance measures

13d. Assess the quality of performance data

13e. Use program performance information to make decisions

13f. Link the performance of program(s) / operation(s) / project(s) to the achievement of agency strategic goals

Sections 5a and 5b - Cross-Agency and Agency Priority Goals

Questions 14 through 17 ask about cross-agency priority goals and agency priority goals, which are not applicable to NRC. Therefore, these questions have been intentionally not reported for this agency.

Section 5c - Quarterly Performance Reviews

Please Note:

Quarterly performance reviews, which some agencies refer to as "stat" meetings or data-driven performance reviews, are regular meetings held at the highest levels of your agency to examine progress toward accomplishing agency priority goals, and in some cases, other agency performance goals.

These reviews should, among other things, assess agencies' coordination efforts; activities and contributions to goals; and strategies for improvement, when applicable. Quarterly performance reviews are to involve key leadership and other relevant parties.

These reviews may be conducted more frequently than quarterly.

In the following questions, "quarterly performance reviews" refers to all aspects of these reviews, including preparation, review, and follow-up.

In the following question, "agency" refers only to your cabinet-level department (e.g., Department of Transportation) or independent agency (e.g., Environmental Protection Agency).

Please do not consider activities at the major component level (e.g., Federal Aviation Administration, Army, Internal Revenue Service).

18. How familiar are you with your agency's quarterly performance reviews? (Select one.)

19. To what extent, if at all, do you agree with the following statements as they relate to your agency's quarterly performance reviews? (Select one answer in each row.)

19a. Overall, the program(s) / operation(s) / project(s) that I am involved with has been the subject of these reviews

19b. These reviews are held on a regular, routine basis

19c. These reviews focus on goals and objectives that are aligned with my agency's strategic and performance plans

19d. Agency leadership actively participates in these reviews

19e. My agency uses these reviews to identify problems or opportunities associated with agency performance goals

19f. Agency leadership uses these reviews to help achieve agency performance goals

19g. These reviews include staff with relevant knowledge needed to facilitate problem solving and identify improvement opportunities

19h. These reviews include officials external to the agency who contribute to the goals discussed at the reviews

19i. My agency has the performance information needed for these reviews

19j. My agency has the capacity to analyze the performance information needed for these reviews

19k. Performance information for these reviews is communicated to relevant parties in an easy-to-understand, useful format

19l. My agency has a process in place for following up on problems or opportunities identified through these reviews

19m. Program managers / supervisors at my level are recognized for meeting performance goals discussed at these reviews

19n. Program managers / supervisors at my level are penalized for not meeting performance goals discussed at these reviews

19o. Discussion at these reviews provides a forum for honest, constructive feedback

19p. These reviews have led to similar meetings at lower levels

19q. I think my agency's quarterly performance reviews are worthwhile

Section 6 - Program Evaluation

Please note:

Program Evaluations are individual, systematic studies using research methods to assess how well a program, operation or project is achieving its objectives and the reasons why it may, or may not, be performing as expected. Program evaluations answer specific questions such as how well a program is operating, whether a program is reaching targeted recipients, why a program is not achieving its desired outcomes, or whether one approach is more effective than another.

Program evaluation as defined here does not include routine program monitoring activities or quarterly performance reviews.

20. Have there been any program evaluations completed within the past 5 years for any of the program(s) / operation(s) / project(s) that you were involved with? (Select one.)

21. For those program evaluations completed within the past 5 years, which of the following entities was primarily responsible for those studies? Note: By primarily responsible, we mean either directly conducted or contracted for those studies (Select one answer in each row.)

21a. My agency or the program itself

21b. An individual program grantee or project

21c. My agency's IG (Inspector General)

21d. GAO (Government Accountability Office)

21e. Another entity not specified above

If you answered "Another entity not specified above," please specify the other entity here.

22. For the program(s) / operation(s) / project(s) that you were involved with over the past 5 years that were evaluated, to what extent, if at all, have the program evaluations contributed to the following activities? (Select one answer in each row.)

22a. Implementing changes to improve program management or performance

22b. Developing or revising performance goals

22c. Supplementing or explaining performance results

22d. Assessing program effectiveness, value or worth

22e. Allocating resources within the program

22f. Increasing understanding about the program or topic

22g. Sharing what works or other lessons learned with others

22h. Designing or supporting program reforms

22i. Supporting program budget requests

22j. Informing the public about how programs are performing

22k. Streamlining programs to reduce duplicative activities

23. Based on your experience with the program(s) / operation(s) / project(s) you were involved with over the past 5 years that were evaluated, to what extent, if at all, have the following factors hindered the agency's use of the program evaluation(s)? (Select one answer in each row.)

23a. Concerns about the credibility (validity or reliability) of study results

23b. Difficulty obtaining study results in time to be useful

23c. Difficulty resolving differences of opinion among internal or external stakeholders

23d. Concerns that the evaluation did not address issues that are important to decision-makers

23e. Difficulty distinguishing between the results produced by the program and the results caused by other factors

23f. Lack of staff who are knowledgeable about interpreting or analyzing program evaluation results

23g. Difficulty determining how to use evaluation findings to improve the program

23h. Difficulty generalizing the results to other persons or locations

23i. Difficulty accepting evaluation findings that do not conform to expectations

23j. Lack of ongoing top executive commitment or support for using program evaluation to make program or funding decisions

23k. Lack of ongoing Congressional commitment or support for using program evaluation to make program or funding decisions

23l. Lack of resources to implement the evaluation findings

24. Based on your experience with the program(s) / operation(s) / project(s) you were involved with over the past 5 years that were evaluated, to what extent, if at all, have the following factors facilitated the agency's use of the program evaluation(s)? (Select one answer in each row.)

24a. Top executive commitment or support for using program evaluation to make program or funding decisions

24b. Congressional requests or mandates

24c. Consultation with stakeholders on the agency's evaluation agenda

24d. Importance of the evaluation's issues to agency decision makers

24e. Program staff involvement in planning and conducting evaluation studies

24f. Providing program staff or grantees with technical assistance on evaluation and its use

24g. Agency staff ability to make recommended program changes

24h. Agency policies or procedures to ensure evaluation quality

24i. Clear implications of results for improving program design or management

24j. Consideration of evaluation findings in agency quarterly performance reviews

24k. Internal presentations of evaluation methods or findings (e.g. brown bags)

24l. Tracking implementation of study recommendations

Section 7 - Comments and Final Response Submission

The results in this section are intentionally not reported.