Build a Smart Exercise and Respond to the Unexpected

November 16, 2020Tim Riecker, The Contrarian Emergency Manager Leave a comment

A few days ago I caught a documentary on NatGeo called Inside: 21^st Century Warship. The documentary was produced in 2013 and told of the cutting-edge design of the USS Freedom and USS Independence. One segment of the documentary captured an exercise the USS Freedom engaged in, with the objective of testing the ship’s firepower to destroy several remote-controlled fast attack boats in open seas. The Captain, well experienced in my opinion, was able to neutralize the boats through the massive wake created by the ship’s sizeable turbines. EndEx.

The lead controller was clearly upset with this. The objective of the exercise, after all, was to test the ship’s guns, which were not fired in this exercise. The controller vented his frustration with the Captain, needing to reemphasize the parameters of the exercise.

Who was at fault in this? Was the objective of the exercise communicated to the Captain? That wasn’t made clear in the documentary. If it was, perhaps it wasn’t made clear that use of the ship’s guns was the only means by which the Captain could engage the attacking boats. I do applaud the Captain’s initial defensive methods, which is perhaps what he was trained to do, though that obviously circumvented the intent of the exercise. Either way, there was a miscommunication or misunderstanding as to the intent and parameters of the exercise.

While this is a military example, the portability to emergency management and homeland security is pretty direct. How do we mitigate against this type of miscommunication or misunderstanding? It starts with a well-defined concept and objectives for our exercise. Those build the foundation from which the rest of the exercise is constructed. Part of exercise design is anticipating how players may respond to the information they are provided and the situations which they will face. This constant analysis helps us to ensure a well-designed exercise, especially in regard to reducing any and all ambiguity, particularly as information relates to the objectives of the exercise and the ‘rules of the game’. It helps us to craft clear injects and even contingency injects in the event players don’t respond the way in which we expect them. Finally, when it comes to deployment of the exercise, an effective player briefing is very important.

Can things still go wrong? Sure they can. That’s why it helps to have a well experienced Exercise Director and/or Lead Controller, and a proficient SimCell Manager (if you are using a SimCell). They can help get the exercise out of a rut. I’ve seen and performed all manner of intervention… most often it’s some ad-hoc development of contingency injects to help steer them down the right path. I’ve also engaged chief executives, who sometimes weren’t expected to participate in the exercise, to make a call, functioning in their own capacity but working for me as an actor, with clear direction to poke, prod, inquire, or otherwise re-direct to get players back into my sandbox. If necessary, it’s a conversation directly with the ‘leader’ of the players, pulling them out of the exercise for a moment and letting them know what they can or can’t be doing. If you have to call a time out and reset something, do it, but do it quickly.

It may be cliché but expect the unexpected. Sometimes players will do something you don’t anticipate. While this may be the circumstance, however, it could very well be on you. Either you didn’t communicate the rules or communicate them well enough. Ensure understanding in this communication. Certainly, ensure that during the exercise, there is good communication between controllers and the SimCell to identify when, if, and how players might be straying a bit. If it’s caught early enough, it will usually just take a gentle nudge to get them back on track. It’s important to recognize and address it as soon as possible – otherwise you will quickly lose your exercise, wasting time and money, and certainly frustrating the players.

Have you had an exercise go off the rails? How did you correct it?

Emergency Preparedness Solutions, LLC®

Improving the HSEEP Templates

March 9, 2020Tim Riecker, The Contrarian Emergency Manager Leave a comment

For years it has bothered me that the templates provided for the Homeland Security Exercise and Evaluation Program (HSEEP) are lacking. The way the documents are formatted and the lack of some important content areas simply don’t do us any favors. These templates go back to the origination of HSEEP in the early 2000s and they have seen little change since then. It gives me concern that the people who developed these have struggled with concepts of document structuring and don’t understand the utility of these documents.

I firmly believe that the documents we use in exercise design, conduct, and evaluation should be standardized. Many of the benefits of standardization that we (should) practice in the Incident Command System (ICS) certainly apply to the world of exercises, especially when we have a variety of different people involved in each of these key phases of exercises and entering at different times. Much like an incident, some people develop documents while others are users. Both should count on a measure of standardization so they don’t have to figure out what they are looking at and how to navigate it before actually diving into the content. That doesn’t mean, however, that standards can’t evolve to increase utility and function.

I’ve written in the past about the dangers of templates. While they are great guides and reminders of certain information that is needed and give us an established, consistent format in which to organize it, I still see too many people not applying some thinking to templates. They get lost in plugging their information into the highlighted text areas and lose all sense of practicality about why the document is being developed, who the target audience for the document is, and the information they need to convey.

Some of my bigger gripes…

Larger documents, such as ExPlans, SitMans, Controller/Evaluator Handbooks, and After-Action Reports MUST have a table of contents. These documents can get lengthy and a TOC simply saves time in finding the section you are looking for.
Some exercises are complex and nuanced. As such, key documents such as ExPlans, SitMans, and Controller/Evaluation Handbooks must have designated space for identifying and explaining those situations. This could be matters of multiple exercise sites and site-specific information such as different scopes of play for those sites, limited scopes of participation for some agencies, statements on the flow and execution of the exercise, and others.
Recognize that the first section of an EEG (Objective, Core Capability, Capability Target, Critical Tasks, and sources) is the only beneficial part of that document. The next section for ‘observation notes’ is crap. Evaluators should be writing up observation statements, an analysis of each observation, and recommendations associated with each observation. The information provided by evaluators should be easily moved into the AAR. The EEG simply does not facilitate capturing this information or transmitting it to whomever is writing the AAR.
The AAR template, specifically, is riddled with issues. The structure of the document and hierarchy of headings is horrible. The template only calls for documenting observations associated with observed strengths. That doesn’t fly with me. There should similarly be an analysis of each observed strength, as well as recommendations. Yes, strengths can still be improved upon, or at least sustained. Big missed opportunity to not include recommendations for strengths. Further, the narrative space for areas of improvement don’t include space for recommendations. I think a narrative of corrective actions is incredibly important, especially given the very limited space in the improvement plan; plus the improvement plan is simply intended to be an implementation tool of the AAR, so if recommendations aren’t included in the body of the AAR, a lot is missing for those who want to take a deeper dive and see specifically what recommendations correlate to which observations and with an analysis to support them.

Fortunately, strict adherence to the HSEEP templates is not required, so some people do make modifications to accommodate greater function. So long as the intent of each document and general organization remains the same, I applaud the effort. We can achieve better execution while also staying reasonably close to the standardization of the templates. But why settle for sub-par templates? I’m hopeful that FEMA’s National Exercise Division will soon take a look at these valuable documents and obtain insight from benchmark practitioners on how to improve them. Fundamentally, these are good templates and they have helped further standardization and quality implementation of exercises across the nation. We should never get so comfortable, though, as to let tools such as these become stagnant, as obsolesce is a regular concern.

I’m interested in hearing what you have done to increase the value and utility of HSEEP templates. How would you improve these? What are your pet peeves?

Emergency Preparedness Solutions, LLC®

Emergency Management Exercises: Not for the Inexperienced

June 6, 2018Tim Riecker, The Contrarian Emergency Manager Leave a comment

Many think exercise design is easy. I’ve seen agencies relegate it to interns and new staff with little supervision, or even performed by seasoned emergency managers with little concept of what the Homeland Security Exercise and Evaluation Program (HSEEP) is. Sadly, we have people completing HSEEP training and even FEMA’s Master Exercise Practitioner (MEP) program, thinking they are ready to conquer the world of preparedness exercises, but with little practical experience designing exercises under their belts. We all need to learn sometime.

Just as any organization or jurisdiction should be eased into their exercise program, exercise designers need to be eased into designing exercises. They should be starting small and with focused tasks, always under the mentorship of someone experienced, even if they aren’t within your own agency, to give some guidance and feedback. While HSEEP gives a lot of great guidance, exercise design can quickly become complex. It can be easy to lose track of tasks or have an oversight. There are political matters, organizational needs, safety issues, and simply good exercise practices that all need to be recognized and addressed. I’ve seen far too many exercises go off the rails due to a lack of awareness of these issues, poor exercise design, and poor exercise management.

Have partner agencies (even if not participating) been properly notified? Do notifications need to go out to the media or public so they are not alarmed? How about dispatch? Every exercise, especially operations-based exercises, should be periodically evaluated for risk throughout the design process. Identify what actions or lack thereof can cause things to go bad. Consider politics, the media, the public, and safety of participants, observers, and exercise staff. Do you need a weapons policy? How will you enforce it? Are there risks associated with traffic? How will exercise staff communicate? The template for the Exercise Plan (ExPlan) prompts you to address some things, but there may be additional needs.

What contingencies do you have for inclimate weather? Maybe you need to dip into the ICS tool box and conduct an incident (exercise) safety analysis, from that developing a safety plan (you can probably get a qualified/experienced safety officer to help you with this). Consider what operations will be conducted in the exercise, what can go wrong, how you will mitigate against them, and what resources are needed if something does go wrong. In the event of a real-world emergency, what needs to happen? Should you have EMS standing by? Should you have a rapid response team in reserve for a rescue situation? The information assembled in your risk assessment and safety plans should be provided to exercise staff prior to the exercise as part of their pre-exercise briefings.

As with exercise design, it can be a great learning experience for new staff to be part of the exercise support staff, but don’t put them in charge. You should have experienced staff serving in the key positions of exercise director, lead controller, and lead evaluator. If you are using a simcell, you want a strong and experienced simcell lead. Safety matters aside, the staff of an operations-based exercise need to have great awareness of what’s going on and excellent communication up their chain of command and with the simcell to ensure that the exercise is flowing properly. For a discussion-based exercise, your facilitators should be experienced as well. Participants in discussion-based exercises may take a discussion in a different direction. While this is generally not desired, sometimes it does bring great unintended results. An experienced facilitator should know how to properly handle this to ensure that participants and stakeholder agencies are getting the most benefit.

Far too many poorly designed exercises have gotten to execution, resulting in a failure to accomplish the exercise objectives, frustrating participants, and even resulting in inter-agency political issues or injuries. Even a well-designed exercise can be poorly conducted or facilitated, getting similar results. If you are new to emergency management and have little experience in the design, conduct, and evaluation of exercises, there is no shame in asking for help or at least another set of eyes to look over your exercise documents. While we want to encourage learning and growth, no one learns properly by being shoved into a situation with no guidance and so many pitfalls. Train people up properly, giving them mentored practical experience to compliment their classroom training. If you don’t have the personnel in place, there are a number of well experienced and well qualified firms (ahem…) that provide these services.

For more information on running an exercise program, take a look at this 10-part blog post.

What tips do you have for people new to the exercise world?

Emergency Preparedness Solutions, LLC™

Use of Pre-Developed Exercises – Proceed with Caution

April 16, 2018April 16, 2018Tim Riecker, The Contrarian Emergency Manager 2 Comments

I was recently asked by a client about my thoughts on pre-developed or ‘canned’ exercises. As it turns out, I have a lot of feelings about them, most of them negative. Pre-developed exercises, if properly understood and applied, can be a huge help, but the big problem is that we’re dealing with human nature, and some people are just damn lazy. Garbage in, garbage out.

We need to keep in mind that exercises, fundamentally, are developed to validate plans. Not my plans. Your own plans. While standards of practice mean that most plans have a high degree of commonality (i.e. a HazMat response plan for a jurisdiction in California will be largely the same as one for a jurisdiction in New York State), it’s often the deviations from the standards and the local applications that need to be tested most. So it doesn’t do well for anyone to replicate an exercise that doesn’t test your own plans. Similarly, the foundation of exercise design is objectives. While the pre-developed exercise may have a theme that coincides with what you want to test, sheltering, for example, there are a lot of different aspects of sheltering. The pre-developed exercise might not focus on what you need to exercise. With all this, anyone who wants a quality exercise from something pre-developed is going to have to do a lot of re-development, which might be more frustrating than starting from scratch.

If you want a quality exercise, you really can’t short cut the process. Not only might HSEEP be required for whatever grant funding you are using for the exercise, but it’s a best practice – and for good reason. So often people want to cut corners. If you do, the final product will look like you’ve cut corners. It might lack proper context, good reference documents, or meaningful evaluation. The exercise planning meetings have defined purpose, and the documents help capture that process and communicate the intent to specific audiences.

On the other hand, there are proper ways to use materials from a previously developed exercise to benefit your own exercise. The development of good questions in discussion-based exercises and injects for operations-based exercises can be a challenge. Reviewing other exercises, especially when there might be some similarity or overlap in objectives, can be a huge help, so long as they are properly contextualized and relate back to objectives for your exercise. This isn’t a copy and paste, though… as it all should still be applied within the exercise design process.

There are some exercises out there that might seem like exceptions to what I’ve written above. The first that comes to mind are FEMA’s Virtual Table Top Exercises (VTTX). The VTTX is a great program, conducted monthly, focusing on different themes and hazards. FEMA’s Emergency Management Institute (EMI) assembles a package of materials that go to each community registered for the event, allowing a measure of local customization. While jurisdictions may use this material differently, it is at least an opportunity to discuss relevant topics and hopefully capture some ideas for future implementation.

Similarly, my company, Emergency Preparedness Solutions, recently completed a contract with the Transportation Research Board for a project in which we developed a number of ‘generic’ exercises for airports. These functional exercises, facilitated through a web-based tool, can be easily customized to meet the needs of most airports across the nation and are written with objectives focused on the fundamentals of EOC management within the timeline of an incident. While specific plans aren’t directly referenced in the exercises, airport personnel are able to examine the structure of response in their EOC and can reflect on their own plans, policies, and procedures. Similar to FEMA’s VTTX series, they aren’t a replacement for a custom-developed exercise, but they can help examine some fundamentals and start some important discussions. I’m not able to get into much more detail on this project, as the final report has yet to be published, but look forward to future posts about it.

All in all, I tend to caution against using pre-developed exercises. I simply think that most people don’t use them with the right intent and perspective, which can severely limit, or even skew, outcomes. That said, there exists potential for pre-developed exercises to be properly applied, so proceed with caution and with your wits about you. Understanding that time, money, and other resources can be scarce, emergency management has always done well with ‘borrowing best practices’. While there is sometimes nothing wrong with that, short cutting the process will often short cut the benefits. Do it right. Use of a custom-developed exercise is going to maximize benefit to your community or organization.

HSEEP Training – Is it Required

February 28, 2018February 28, 2018Tim Riecker, The Contrarian Emergency Manager Leave a comment

Continuing from my previous blog post, I’ll answer a search phrase used to bring someone to my blog. Earlier this month, someone searched ‘Is HSEEP training mandatory?’. We speak, of course, of the Homeland Security Exercise and Evaluation Program, which is the DHS-established standard in exercise program and project management.

The short answer to the question: Maybe.

Generally speaking, if your exercise activities are funded directly or indirectly by a federal preparedness grant, then grant language usually requires that all exercises are conducted in accordance with HSEEP. While most federal grant guidance doesn’t explicitly state that exercise personnel must be formally trained in HSEEP, it’s kind of a no-brainer that the fundamental way to learn the standards of practice for HSEEP so you can apply them to meet the funding requirement is by taking an HSEEP course. If you are a jurisdiction awarded a sub-grant of a federal preparedness grant or a firm awarded a contract, there may exist language in your agreement, placed there by the principal grantee, that specifically requires personnel to be trained in HSEEP.

Beyond grant requirements, who you work for, who are you, and what you do generally don’t dictate any requirement for HSEEP training. Aside from the federal grant funding or contracts mentioned, there is no common external requirement for any organization to have their personnel trained in HSEEP. If your organization does require it, this is likely through a management-level decision for the organization or a functional part of it.

So, while HSEEP is a standard of practice, training in HSEEP, in general terms, is not a universal requirement. That said, I would certainly recommend it if you are at all involved in the management, design, conduct, or evaluation of exercises. FEMA’s Emergency Management Institute (EMI) offers HSEEP courses in both a blended learning and classroom format. The emergency management/homeland security offices of many states and some larger cities offer them as well.

Emergency Preparedness Solutions, LLC ^SM

2017 HSEEP Course Information

January 13, 2017Tim Riecker, The Contrarian Emergency Manager 3 Comments

Emergency Preparedness Solutions is regularly looking for Homeland Security Exercise and Evaluation Program (HSEEP) trained personnel to help us design, conduct, and evaluate exercises. The following training bulletin from FEMA’s Emergency Management Institute provides updated information on the course offerings and application process. Below the training bulletin is a listing of the webinar-based offerings of HSEEP (K0146).

Training Bulletin

Course: K/L0146 – Homeland Security Exercise Evaluation Program

(HSEEP): Basic Course

Emmitsburg, MD — You are subscribed to EMI News for FEMA. The following information has recently been updated, and is now available on http://training.fema.gov/EMI/

1263 – REVISED – Training Bulletin – K-L0146 – updated – Jan 12, 2017

The K/L0146 HSEEP is an intermediate-level course that provides a comprehensive overview of exercise design along with practical skill development in accordance with the HSEEP Doctrine. The course uses activities that will give participants an opportunity to interact with many of the templates and other materials that are provided by the National Exercise Division to ensure exercises are conducted in a consistent manner. Upon completion of this course, participants will gain a better understanding of what constitutes a HSEEP consistent exercise.

Course	Start	End
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	4/10/2017	4/13/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	5/8/2017	5/11/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	6/5/2017	6/8/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	7/10/2017	7/13/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	8/7/2017	8/10/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	9/18/2017	9/21/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	3/13/2017	3/16/2017
K0146: Homeland Security Exercise and Evaluation Program (HSEEP) Training Course (M,T,W,R 1:00-5:00 p.m. EST)	2/6/2017	2/9/2017

The Contrarian Emergency Manager

News, concepts, and opinions on emergency management and homeland security topics

Tag homeland security exercise and evaluation program