10.6.3. Test, Training, and Exercise Requirements

(1) To ensure consistency, the IRS TT&E Program is requiring use of common language, various but standardized types of exercises, including customized and realistic scenarios, materials, and documents. National standards for the Federal Government have been set by the Homeland Security Exercise Evaluation Program (HSEEP) of the Department of Homeland Security (DHS).

(2) The IRS TT&E Program places emphasis on the following HSEEP principles:

1. Objectives designed to mitigate risks specific to IRS;

2. Progressive approach utilizing a common set of program-related priorities and objectives designed around IRS processes;

3. Reliance on a multi-year training and exercise plan.

(3) The HSEEP handbook is found at Homeland Security Exercise and Evaluation Program (HSEEP) (fema.gov).

(1) TT&E requirements are detailed in FCD-1 Federal Continuity Directive 1 - January 17, 2017 (fema.gov) dated January 17, 2017.

(2) Federal Continuity Directive 1 (FCD-1), as signed by the Administrator of the Federal Emergency Management Agency (FEMA), implements requirements, and establishes the framework, requirements, and processes that support the development of executive departments and agencies’ continuity programs and by specifying and defining elements of a continuity plan.

(3) IRS administers and executes its continuity program under the authority and direction of the Department of the Treasury (Treasury) Office of Emergency Preparedness.

(1) TT&Es are the responsibility of the SCR-CO:CO organization and all BU Heads of Office. The support of each BU and their attention to business continuity planning is critical to a successful Continuity Program at the IRS.

(2) BU Executives are required to assign knowledgeable key personnel to serve as continuity liaisons, including but not limited to:

1. National Continuity Points of Contact (NCPOC) - A primary and alternate geographically who are dispersed and are appointed to ensure continuity requirements are met within their respective BUs;

2. Local Continuity Representative (LCR) - A primary, first alternate, and second alternate who are geographically dispersed and are appointed to ensure representation for their BUs on Incident Management Teams (IMTs).

(3) Appointees must have knowledge of BU operations enabling them to perform all levels of related tasks. SCR-CO:CO will assist and guide them to ensure continuity responsibilities and requirements are met.

(4) Successful execution of the IRS continuity test and exercise program requirements involves collaboration between SCR-CO:CO and BU NCPOCs.

(5) At the beginning of each fiscal year, the IRS Continuity Coordinator issues a memorandum to all IRS BU Heads of Office that identifies the Annual Continuity Program RequirementsThe memorandum identifies the testing requirements and frequency for that year:

1. SCR-CO:CO in concert with other IRS Continuity Stakeholders pre-determine the testing schedule and/or frequency for each fiscal year;

2. The categories of Test and Exercises.

(1) All program reports are dependent on summaries of required information listed in IRM 10.6.3.9.4 After Action Report/Improvement Plan (AAR/IP) Development.

(2) Test and Exercise Completion:

1. Cumulative TT&E Completions by BU are reported monthly in the SCR-CO:CO Monthly Measures Report, SCR-CO:CO Monthly Report, and quarterly in the SCR-CO:CO Operational Review Managing BU Test, Training, and Exercise (% Complete);

2. Test and Exercise Completions by type of exercise are reported monthly in the SCR-CO:CO Monthly Report;

3. BU level completions are reported monthly in the Continuity Operations Working Group (COOPWG).

(3) Test and Exercise Corrective Actions:

1. Summary TT&E Corrective Action completions are reported monthly in the SCR-CO:CO Monthly Measures Report, the SCR-CO:CO Monthly Report, and the COOPWG.

(1) See IRM 10.6.1 Introduction to Continuity Planning for a general list of terms.

(2) A Situation Manual (SitMan) is document developed by the Exercise Planning Team to define the scenario(s) for the test or exercise.

(3) An Exercise Plan (ExPlan) is used to direct the phases or steps of the test or exercise.

(1) See IRM 10.6.1 Introduction to Continuity Planning for a general list of acronyms.

(2) Specific acronyms for Test, Training, and Exercises are:

1. AAR/IP - After Action Report/Improvement Plan;

2. ANA - Alert Notification Activation Test;

3. CAP - Corrective Action Plan;

4. GETS - Government Emergency Telecommunications Service;

5. HQRT - Headquarters Recovery Team;

6. WPS - Wireless Priority Service;

7. SAT - Satellite Phone Service.

(1) See IRM 10.6.1 Introduction to Continuity Planning, IRM 10.6.2 Continuity Plan Requirements, IRM 10.6.4 Incident Management Program, IRM 10.6.5 Annual Certification Requirements, and IRM 10.6.6 Pandemic Plan Requirements for related topics to testing and reporting.

(1) The purpose of the call-tree test is to validate the personnel notification process to be used in the event of an emergency on an annual basis. Managers should have the personnel call tree information accessible at all times. The test is to be conducted as quickly as possible to evaluate the time required to contact all personnel. The test requirements are for all personnel and should include calls from the Executive level down to the frontline personnel.

(2) This test will be performed as a stand-alone activity with all personnel currently at work. Personnel on authorized leave, or in training or travel status are exempt. When conducting the call tree, managers should use their documented contact list. Information to be validated includes: office location, office telephone number, contact number(s) for after hours, and out-of-area emergency contact names and number(s).

(1) The purpose of all Alert Notification and Activation (ANA) Tests is to authenticate all contact/notification information and validate the alert process.

(2) ANA Tests are primarily telephone based, but may include other communications devices, such as computer email or text chat, and mobile phone text messaging.

(3) Procedures are different for Headquarters (HQ) Recovery Teams and for IMTs.

(1) The purposes of the Communications Systems and Equipment Tests are to:

1. To validate the status of the continuity communications tools that can be used in the event of an emergency and/or during communications systems disruptions;

2. To promote operational readiness ensuring those responsible for using the Continuity Communication tool is familiar enough to use it during an emergency.

(2) There are three alternative means of communications available to the IRS in the event of an emergency

1. Government Emergency Telecommunication Service (GETS) subscribers receive priority access to the local/national telephone network during times of network congestion;

2. Wireless Priority Service (WPS) is a feature added to the approved government mobile device of a GETS subscriber. During an emergency, this feature allows priority access to the mobile telephone network during times of network congestion;

3. Satellite phones have a non-secure wireless voice capability and should not be used to discuss sensitive information. The phones work best outdoors with an unobstructed view of the sky to the horizon, and require a fully charged battery.

(1) Determining exercise scope enables planners to scale an exercise to meet the objectives while staying within the resource and personnel constraints of the exercising organizations. Key elements in defining exercise scope include exercise type, participation level, exercise duration, exercise location, and exercise parameters. Some of these elements are determined, or initially discussed, through program management activities. However, the exercise planning team finalizes the scope based on the exercise objectives. The planners should:

1. Encourage active participation by everyone. Active participation by appropriate staff and key leaders is paramount to meeting the exercise objectives successfully;

2. Determine how long it will take to address the exercise objectives effectively. Discussion-based exercises are generally shorter, ranging from a couple of hours to a full day;

3. Outline clearly the exercise parameters as to what should be included in an exercise scenario based on the objectives and scope, and what should not be exercised. Often there is a desire to add exercise activities that fall outside of the scope of the exercise in order to meet diverse planning and training requirements. While these activities may be useful to an organization, they may impact the ability of players to meet exercise objectives or may reduce the benefit of the exercise by diluting its focus;

4. Define clearly the exercise scope early in the design process will help exercise planners keep the exercise to a manageable and realistic level.

(1) The exercise planning team selects one or more exercise program priorities on which to focus an individual exercise. These priorities drive the development of exercise objectives, which are distinct outcomes that an organization wishes to achieve during an exercise.

(2) Exercise objectives should incorporate exercise participants’ plans and procedures, operating environment, and desired outcomes.

(3) Planners should select a reasonable number of specific, measurable, achievable, relevant, and time-bound exercise objectives to facilitate effective scenario design, exercise conduct, and evaluation.

(1) A scenario is an outline or model of the simulated sequence of events for the exercise.

(2) It can be written as a narrative or depicted by an event timeline.

(3) A scenario provides background information about the incident catalyst(s) of the exercise, drives players participation. and is contained in a SitMan or ExPlan. 

(4) Exercise planners should select and develop scenarios that enable an exercise to assess objectives and core capabilities. All scenarios should be realistic, plausible, and challenging; however, designers must ensure the scenario is not so complicated that it overwhelms players.

(5) A scenario consists of three basic elements:

1. The general context or comprehensive story;

2. The required conditions that will allow players to demonstrate proficiency and competency in conducting critical tasks, demonstrating core capabilities, and meeting objectives;

3. The technical details necessary to accurately depict scenario conditions and events.

(6) The exercise planning team ensures that the design effort is not characterized by a fixation on scenario development; rather, the scenario facilitates assessment of exercise objectives and core capabilities.

(7) Exercise planners should refrain from developing the scenario until after the scope and objectives of the exercise have been clearly defined.

(8) Scenarios should avoid any sensitivity that may arise, such as the use of real names of terrorist groups or sensitive venues.

(1) Comprehensive, organized exercise documentation is critical to ensure an accurate account of the exercise is preserved. This in turn allows for leverage of past documentation to support future exercises and, more importantly, ensures that all critical issues, lessons learned, and corrective actions are appropriately captured to support improvement efforts.

(2) It is important for the exercise sponsor(s) to understand the specific requirements for security marking rules and requirements, access and dissemination, storage, disposal, and incident reporting of sensitive documents. Additionally, all documents produced must be 508 compliant.

(3) A SitMan or an ExPlan provides the conceptual background for an exercise, presents the scenario narrative designed to drive participation during the exercise, and serves as the primary reference material for the exercise planning team. Typically, a SitMan or ExPlan contains exercise scope, objectives, and core capabilities; participant roles and responsibilities; rules of conduct; safety issues, notably real emergency codes and phrases; safety controller responsibilities, prohibited activities, and weapons policies; logistics; security of and access to the exercise site; communications; duration, date, and time of exercise and schedule of events; and maps or directions.

(4) A Master Scenario Events List (MSEL) is a chronological timeline of expected actions and scripted events (injects) to be inserted into exercise play by controllers in order to generate or prompt player activity. A MSEL ensures necessary events happen so that all exercise objectives are met.

(5) A Facilitator Guide is designed to help facilitators manage a discussion-based exercise. It usually outlines instructions and key issues for discussion during the event and provides background information to help the facilitator answer questions from participants or players.

(6) The Player Handout provides key information to exercise players. A Player Handout can supplement the SitMan or ExPlan by providing a quick-reference guide to logistics, agenda or schedule, and key contact data for players.

(7) A Participant Feedback Form allows players to provide input regarding observed strengths and areas for improvement that identified during the exercise.

(1) The design of the Test or Exercise is a crucial step to be able to define participants, goals, and measurements.

(2) The exercise design and development process builds on exercise program management. During this phase participants, are selected for an exercise planning team. They schedule planning meetings, identify and develop exercise objectives, design the scenario, create documentation, plan exercise conduct, select a focus evaluation, and coordinate logistics.

(3) Scenario provides the storyline that drives an exercise. The first step in designing a scenario is determining the type of threat/hazard (chemical, explosive, cyber, natural disaster) to be used in an exercise. The hazards selected for an exercise should realistically stress the resources an entity is attempting to improve through its exercise program. The scenario should also be a realistic representation of potential threats and hazards faced by the exercising entity. The next step in designing a scenario is to determine the venue (facility or site) in which exercise play will take place. Venue selection should reflect the hazard selected, allowing for realistic, exercise-based simulation of the hazard.

(1) The exercise planning team is responsible for the successful execution of all aspects of an individual exercise. The planning team works with exercise stakeholders to determine or refine exercise objectives. The exercise planning team also creates a realistic scenario to achieve exercise objectives and develops documentation to guide exercise conduct and evaluation.

(2) The exercise planning team should consist of both members of SCR-CO:CO Continuity Planning Team and the BU or function being tested.

(1) Effective exercise design and development involve a combination of exercise planning activities, often in the form of planning meetings. Planning activities bring together exercise stakeholders to discuss and agree on key aspects of the exercise’s design and development. Various factors, including exercise scope, type, and complexity are identified, and exercise planners tailor the planning schedule to suit the particular nature of the exercise.

(2) The core of the exercise planning activities begins and ends with the four planning meetings. These meetings serve as the key mechanism for executing the major steps of exercise design and will ensure all aspects of planning for the IFE are met.

(1) The Concept and Objectives Meeting is the formal beginning of the planning process. It is held to identify the type, scope, objectives, and purpose of the exercise. For less complex exercises and for entities with limited resources, the Concept and Objectives Meeting can be conducted in conjunction with the Initial Planning Meeting .

(2) Discussion points for this meeting include:

1. Exercise purpose;

2. Proposed exercise scenario, capabilities, tasks, and objectives;

3. Available exercise resources;

4. Proposed exercise location, date, and duration;

5. Exercise planning team and exercise participants;

6. Exercise assumptions and artificialities;

7. Exercise control and evaluation;

8. Exercise security organization and structure;

9. Local issues, concerns, and sensitivities;

10. Exercise logistics.

(3) The primary tools for the Concept and Objectives Meeting are the read-ahead packet, the agenda, and background/rationale for conducting the exercise. Briefings are useful for presenting the exercise background/rationale as well as the proposed exercise methodology for those unfamiliar with HSEEP.

(1) The Initial Planning Meeting marks the beginning of the exercise development phase. Unless a separate Concept and Objectives Meeting is conducted, the Initial Planning Meeting is typically the first official step in the planning process. Its purpose is to determine exercise scope by gathering: input from the exercise planning team; design requirements and conditions (assumptions and artificialities); objectives; extent of play; and scenario variables (time, location, hazard selection). The Initial Planning Meeting is also used to develop exercise documentation by obtaining the planning team’s input on exercise location, schedule, duration, and other relevant details. During the Initial Planning Meeting , exercise planning team members are assigned responsibility for activities associated with designing and developing exercise documents such as the MSEL and the Situation Manual (SitMan), and logistics, such as scene management and personnel.

(2) Activities during the Initial Planning Meeting include:

1. Clearly defined exercise objectives and aligned core capabilities;

2. Evaluation requirements, including capability targets and critical tasks;

3. Relevant plans, policies, and procedures to be tested in the exercise;

4. Exercise scenario;

5. Extent of play for each participating organization;

6. Ensuring clearly defined and measurable capabilities, tasks, and objectives;

7. Optimum duration of the exercise;

8. Exercise planners’ roles and responsibilities;

9. Decision to record exercise proceedings (audio or video);

10. Local issues, concerns, or sensitivities;

11. Consensus regarding the date, time, and location for the next meeting.

(1) Mid-Term Planning Meetings are typically used in more complex exercises. Mid-Term Planning Meetings provide additional opportunities to settle logistical and organizational issues that may arise during planning.

(2) The Mid-Term Planning Meeting is a working session to discuss exercise organization and staffing concepts, scenario and timeline development, scheduling, logistics, and administrative requirements. It is also a session to review draft documentation. A portion of the Mid-Term Planning Meeting should be devoted to developing the MSEL, as needed.

(3) Activities during the Mid-Term Planning Meeting include:

1. Comments on draft exercise documentation;

2. Identification of exercise venue artificialities and/or limitations;

3. Agreement on final logistical items;

4. Assignment of additional responsibilities;

5. Developing Facilitator Guides and handbooks.

(1) The Final Planning Meeting is the final forum for reviewing exercise processes and procedures. Prior to the Final Planning Meeting, the exercise planning team receives final drafts of all exercise materials. The Final Planning Meeting allows the exercise planning an opportunity to ensure the exercise is aligning with their intent, address any questions, and make any last-minute updates. A Final Planning Meeting should be conducted for all exercises to ensure that all elements of the exercise are ready for conduct. No major changes to the design or scope of the exercise, or its supporting documentation, should take place at the Final Planning Meeting. The Final Planning Meeting ensures that all logistical requirements have been met, all outstanding issues have been identified and resolved, and all exercise products are ready for printing.

(2) Activities during the Final Planning Meeting include:

1. Resolve any open issues related to exercise planning and identify last-minute concerns that may arise;

2. Review all exercise logistical activities (schedule, registration, attire, special needs);

3. Conduct a comprehensive, final review of, with approval for all exercise documents and presentation materials.

(3) By the conclusion of the Final Planning Meeting, planners should be able to:

1. Ensure attendees have a clear understanding of, and give final approval for, exercise processes and procedures;

2. Identify and resolve last-minute issues;

3. Confirm logistical elements, including audio visual equipment, room configuration and setup, refreshments, and schedule.

(1) The evaluator must be familiar with the exercise objectives, scenario for discussion, and key evaluation areas and have:

1. Reviewed appropriate plans, procedures, and protocols;

2. Attended required evaluator training and other briefings;

3. Reviewed appropriate exercise materials, including the exercise schedule and evaluator instructions.

(1) During the exercise the observer should listen closely to the players’ discussion and take detailed notes relative to the assigned area of evaluation:

1. The observer should be familiar with the activities and tasks for evaluation;

2. The observer should not jump ahead to analysis until the exercise has ended;

3. The observer should not prompt players with specific responses or interfere with player performance in any way.

(2) The primary duty is to document player discussion. After the exercise, that information will be used to determine whether the exercised objectives and plans were effectively demonstrated and to identify strengths and improvement items.

(3) Evaluators should not start to complete the evaluation form during this time.

(1) Following the exercise, owners should use notes and the participant feedback forms to develop the AAR/IP. In all IRS exercises, evaluators and players should provide copies of the forms to the exercise owners for the development of the AAR/IP. All players should completed these forms as they serve to identify strengths and areas for improvement for each activity.

(2) Below are some key reminders of evaluators:

1. Participate in the Hot Wash, and take notes on findings identified by players. Before the Hot Wash, do not discuss specific issues or problems with participants;

2. After the Hot Wash, summarize notes and prepare for the Evaluator Debrief. Have summary ready for the lead evaluator, if applicable;

3. Use data to determine whether the exercise objectives and plans were effectively implemented or demonstrated;

4. Use data to identify observations as strengths or areas for improvement.

(1) In this step, the detailed notes as well as the participant feedback forms from observing the exercise are provided to the exercise owners. At minimum the AAR/IP should contain detailed recommendations for continuous improvement of any issues identified.

(2) The exercise owners should use the data collected to develop the AAR/IP.  The AAR/IP will document whether the exercise objectives and plans were effectively implemented or demonstrated; and identify observations as strengths or areas for improvement.

(3) Form 14572 Test and Exercise After Action Report/Improvement Plan (AAR/IP) is the final exercise document that summarizes key exercise-related evaluation information. The length and format of AAR/IP depends on the exercise type and scope and is determined by the exercise planning team. Key elements of the AAR/IP include, but are not limited to:

1. Exercise Summary: This includes the exercise objectives and capabilities;

2. Exercise Overview: This includes identifying information, such as exercise name, date, duration, and participants;

3. Exercise Scenario: This includes a summary of key scenario events;

4. Observations, Strengths and Areas of Improvement: This includes an assessment of exercise performance in relation to exercise objectives and capabilities. It also highlights relevant areas for improvements, strengths, lessons learned, and gaps;

5. Corrective Actions: This identifies specific corrective actions from the exercise, assigns these actions to responsible parties and establishes target dates for completion.

(4) NCPOCs are responsible for ensuring that AAR/IPs are completed for their BU SLTs.

(5) The plan owners, usually the ICs are responsible for ensuring that AAR/IPs are completed for their IFEs.  Refer to the NARA approved disposition authority, General Records Schedule 4.1, Item 30 (DAA-GRS-2013-0002-0008) on agency records created and maintained relating to management of records and information including Vital or Essential Records.

(6) In both cases, within thirty calendar days of completing the exercise copies, of the Form 14572 Test and Exercise After Action Report/Improvement Plan (AAR/IP) should be submitted to the SCR-CO:CO Planning Team and the cos.scrco.continuity@irs.govSCR-CO:CO Continuity Mailbox identified on the form.

(1) FCD-1 requires that corrective actions identified during exercises are tracked to completion, ensuring that exercises yield tangible preparedness improvements. An effective corrective action program develops improvement plans that are dynamic documents, which are continually monitored and implemented as part of the larger system of improving preparedness.

(2) The IRS maintains a Corrective Action Plan (CAP), which includes deficiencies for both tests and exercises.

(3) The CAP must identify:

1. Continuity deficiencies and/or other areas requiring improvement;

2. Responsible parties and timelines for corrective action.

(4) SCR-CO is responsible for:

1. Managing and tracking all corrective actions;

2. Complying with all requirements described in SOP-FY18-CO-004 Test, Training, and Exercise Corrective Action Program or SOP-FY18-CO-004 Test, Training, and Exercise Corrective Action Program;

3. Ensuring the Corrective Action Tracking Report is kept current by identifying delinquent or incomplete issues and following-up with the individual BUs;

4. Maintaining a AAR/IP available for the Continuity Operations Program Manager upon request. This includes a listing of delinquent or incomplete corrective actions with aging information.

(5) NCPOCs and other Continuity Personnel are responsible for:

1. Documenting Corrective Actions including expected completion dates on applicable supporting documents;

2. Submitting supporting documentation timely;

3. Monitoring the corrective action plan in partnership with BU NCPOCs and engage leadership as needed to elevate challenges and issues;

4. Requesting extensions to corrective action completion dates, when necessary;

5. Reporting the completion of corrective actions to the SCR-CO:CO Planning Team via email.