Emergency Management and Public Safety Should Prepare Like a Sports Team

When and how did a once-annual exercise become the standard for preparedness?  I suppose that’s fine for a whole plan, but most plans can be carved into logical components that can be not only exercised to various degrees, but training can also be provided to support and compliment each of those components.  There are a lot of elements and activities associated with preparedness.  Consider how sports teams prepare. They are in a constant yet dynamic state of readiness.

Sports teams will review footage of their opponents playing as well as their own games.  We can equate those to reviews of after-action reports, not only of their own performance, but also of others – and with high frequency.  How well does your organization do with this quiz?

  • Do you develop after action reports from incidents, events, and exercises?
  • Are they reviewed with all staff and stakeholders or just key individuals?
  • Are they reviewed more than once or simply archived?
  • Are improvements tracked and reviewed with staff and stakeholders?
  • Do your staff and stakeholders review after action reports from other incidents around the nation?

Planning is obviously important – it’s the cornerstone of preparedness.  Coaches look at standards of practice in the sport, best practices, and maybe come up with their own innovations.  They examine the capabilities of their players and balance those with the capabilities of the opposing team.  They have a standard play book (plan), but that may be modified based upon the specific opponent they are facing.  Their plans are constantly revisited based upon the results of practices, drills, and games.  Plans let everyone know what their role is.

  • Do your plans consider the capabilities of your organization or jurisdiction?
  • Do they truly include the activities needed to address all hazards?
  • Are your plans examined and updated based upon after action reports from incidents, events, and exercises?
  • Are your plans flexible enough for leadership to call an audible and deviate from the plan if needed?
  • Is your organization agile enough to adapt to changes in plans and audibles? How are ad-hoc changes communicated?

Training is a tool for communicating the plan and specific roles, as well as giving people the knowledge and skills needed to execute those roles with precision.  Sports players study their playbooks.  They may spend time in a classroom environment being trained by coaches on the essential components of plays.  Training needs are identified not only from the playbook, but also from after action reviews.

  • Is your training needs-based?
  • How do you train staff and stakeholders to the plan?
  • What training do you provide to help people staffing each key role to improve their performance?

Lastly, exercises are essential.  In sports there are drills and practices.  Drills are used to hone key skill sets (passing, catching, hitting, and shooting) while practices put those skill sets together.  The frequency of drills and practices for sports teams is astounding.  They recognize that guided repetition builds familiarity with plans and hones the skills they learned.  How well do you think a sports team would perform if they only exercised once a year?  So why do you?

  • What are the essential skill sets your staff and stakeholders should be honing?
  • What is your frequency of exercises?
  • Do your exercises build on each other?

I also want to throw in a nod to communication.  Even if you aren’t a sports fan, go attend a local game.  It could be anything… hockey, baseball, soccer, basketball, football… whatever.  It doesn’t necessarily have to be pro.  Varsity, college, or semi-pro would certainly suffice.  Even if you don’t stay for the whole game, there is a lot you can pick up.  Focus on the communication between and amongst players and coaches.  Depending on where you are sitting, you might not be able to hear or understand what they are saying, but what you will notice is constant communication.  Before plays, between plays, and during plays.  Sometimes that communication isn’t just verbal – it might be the tapping of a hockey stick on the ice, clapping of hands, finger pointing, or a hand wave or other silent signal.  Coaches are constantly talking to each other on the bench and with players, giving direction and encouragement.  There is a lot going on… strategy, tactics, offense, defense.  What lessons can you apply to your organization?

Lastly, accomplishments should be celebrated.  In public safety, we tend to ignore a lot of best practices not only of sports teams, but also in general employee relations.  Because of the nature of emergency management and other public safety endeavors, it’s easy to excuse getting stuck in the same rut… we get ready for the next incident, we respond to that incident, and we barely have time to clean up from that incident before the next one comes.  Take a moment to breathe and to celebrate accomplishments.  It’s not only people that need it, but also organizations as a whole.

What lessons can you apply from sports teams to your organization?

© 2019 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC℠

Emergency Management Exercises: Not for the Inexperienced

Many think exercise design is easy.  I’ve seen agencies relegate it to interns and new staff with little supervision, or even performed by seasoned emergency managers with little concept of what the Homeland Security Exercise and Evaluation Program (HSEEP) is.  Sadly, we have people completing HSEEP training and even FEMA’s Master Exercise Practitioner (MEP) program, thinking they are ready to conquer the world of preparedness exercises, but with little practical experience designing exercises under their belts.  We all need to learn sometime.

Just as any organization or jurisdiction should be eased into their exercise program, exercise designers need to be eased into designing exercises.  They should be starting small and with focused tasks, always under the mentorship of someone experienced, even if they aren’t within your own agency, to give some guidance and feedback.  While HSEEP gives a lot of great guidance, exercise design can quickly become complex.  It can be easy to lose track of tasks or have an oversight.  There are political matters, organizational needs, safety issues, and simply good exercise practices that all need to be recognized and addressed.  I’ve seen far too many exercises go off the rails due to a lack of awareness of these issues, poor exercise design, and poor exercise management.

Have partner agencies (even if not participating) been properly notified?  Do notifications need to go out to the media or public so they are not alarmed?  How about dispatch?  Every exercise, especially operations-based exercises, should be periodically evaluated for risk throughout the design process.  Identify what actions or lack thereof can cause things to go bad.  Consider politics, the media, the public, and safety of participants, observers, and exercise staff.  Do you need a weapons policy?  How will you enforce it?  Are there risks associated with traffic?  How will exercise staff communicate?  The template for the Exercise Plan (ExPlan) prompts you to address some things, but there may be additional needs.

What contingencies do you have for inclimate weather?  Maybe you need to dip into the ICS tool box and conduct an incident (exercise) safety analysis, from that developing a safety plan (you can probably get a qualified/experienced safety officer to help you with this).  Consider what operations will be conducted in the exercise, what can go wrong, how you will mitigate against them, and what resources are needed if something does go wrong.  In the event of a real-world emergency, what needs to happen?  Should you have EMS standing by?  Should you have a rapid response team in reserve for a rescue situation?  The information assembled in your risk assessment and safety plans should be provided to exercise staff prior to the exercise as part of their pre-exercise briefings.

As with exercise design, it can be a great learning experience for new staff to be part of the exercise support staff, but don’t put them in charge.  You should have experienced staff serving in the key positions of exercise director, lead controller, and lead evaluator.  If you are using a simcell, you want a strong and experienced simcell lead.  Safety matters aside, the staff of an operations-based exercise need to have great awareness of what’s going on and excellent communication up their chain of command and with the simcell to ensure that the exercise is flowing properly.  For a discussion-based exercise, your facilitators should be experienced as well.  Participants in discussion-based exercises may take a discussion in a different direction.  While this is generally not desired, sometimes it does bring great unintended results.  An experienced facilitator should know how to properly handle this to ensure that participants and stakeholder agencies are getting the most benefit.

Far too many poorly designed exercises have gotten to execution, resulting in a failure to accomplish the exercise objectives, frustrating participants, and even resulting in inter-agency political issues or injuries.  Even a well-designed exercise can be poorly conducted or facilitated, getting similar results.  If you are new to emergency management and have little experience in the design, conduct, and evaluation of exercises, there is no shame in asking for help or at least another set of eyes to look over your exercise documents.  While we want to encourage learning and growth, no one learns properly by being shoved into a situation with no guidance and so many pitfalls.  Train people up properly, giving them mentored practical experience to compliment their classroom training. If you don’t have the personnel in place, there are a number of well experienced and well qualified firms (ahem…) that provide these services.

For more information on running an exercise program, take a look at this 10-part blog post.

What tips do you have for people new to the exercise world?

© 2018 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC™

2018-2019 HSEEP Training

Based on my listing last year of Homeland Security Exercise and Evaluation Program (HSEEP) training, some have asked recently where they can find HSEEP training.  One of the most convenient sources is the web-based program run through FEMA’s Emergency Management Institute (EMI).  This course, K0146, is conducted in a live webinar over several blocks of time.

The schedule can be found at this link.  Just type K0146 into the course search field.

Also keep in mind that many state emergency management/homeland security offices offer the HSEEP course in a classroom setting.

-TR

 

Use of Pre-Developed Exercises – Proceed with Caution

I was recently asked by a client about my thoughts on pre-developed or ‘canned’ exercises.  As it turns out, I have a lot of feelings about them, most of them negative.  Pre-developed exercises, if properly understood and applied, can be a huge help, but the big problem is that we’re dealing with human nature, and some people are just damn lazy.  Garbage in, garbage out.

We need to keep in mind that exercises, fundamentally, are developed to validate plans.  Not my plans.  Your own plans.  While standards of practice mean that most plans have a high degree of commonality (i.e. a HazMat response plan for a jurisdiction in California will be largely the same as one for a jurisdiction in New York State), it’s often the deviations from the standards and the local applications that need to be tested most.  So it doesn’t do well for anyone to replicate an exercise that doesn’t test your own plans.  Similarly, the foundation of exercise design is objectives.  While the pre-developed exercise may have a theme that coincides with what you want to test, sheltering, for example, there are a lot of different aspects of sheltering.  The pre-developed exercise might not focus on what you need to exercise.  With all this, anyone who wants a quality exercise from something pre-developed is going to have to do a lot of re-development, which might be more frustrating than starting from scratch.

HSEEP1

If you want a quality exercise, you really can’t short cut the process.  Not only might HSEEP be required for whatever grant funding you are using for the exercise, but it’s a best practice – and for good reason.  So often people want to cut corners.  If you do, the final product will look like you’ve cut corners.  It might lack proper context, good reference documents, or meaningful evaluation.   The exercise planning meetings have defined purpose, and the documents help capture that process and communicate the intent to specific audiences.

On the other hand, there are proper ways to use materials from a previously developed exercise to benefit your own exercise.  The development of good questions in discussion-based exercises and injects for operations-based exercises can be a challenge.  Reviewing other exercises, especially when there might be some similarity or overlap in objectives, can be a huge help, so long as they are properly contextualized and relate back to objectives for your exercise.  This isn’t a copy and paste, though… as it all should still be applied within the exercise design process.

There are some exercises out there that might seem like exceptions to what I’ve written above.  The first that comes to mind are FEMA’s Virtual Table Top Exercises (VTTX).  The VTTX is a great program, conducted monthly, focusing on different themes and hazards.  FEMA’s Emergency Management Institute (EMI) assembles a package of materials that go to each community registered for the event, allowing a measure of local customization.  While jurisdictions may use this material differently, it is at least an opportunity to discuss relevant topics and hopefully capture some ideas for future implementation.

Similarly, my company, Emergency Preparedness Solutions, recently completed a contract with the Transportation Research Board for a project in which we developed a number of ‘generic’ exercises for airports.  These functional exercises, facilitated through a web-based tool, can be easily customized to meet the needs of most airports across the nation and are written with objectives focused on the fundamentals of EOC management within the timeline of an incident.  While specific plans aren’t directly referenced in the exercises, airport personnel are able to examine the structure of response in their EOC and can reflect on their own plans, policies, and procedures.  Similar to FEMA’s VTTX series, they aren’t a replacement for a custom-developed exercise, but they can help examine some fundamentals and start some important discussions.  I’m not able to get into much more detail on this project, as the final report has yet to be published, but look forward to future posts about it.

All in all, I tend to caution against using pre-developed exercises.  I simply think that most people don’t use them with the right intent and perspective, which can severely limit, or even skew, outcomes.  That said, there exists potential for pre-developed exercises to be properly applied, so proceed with caution and with your wits about you.  Understanding that time, money, and other resources can be scarce, emergency management has always done well with ‘borrowing best practices’.  While there is sometimes nothing wrong with that, short cutting the process will often short cut the benefits.  Do it right.  Use of a custom-developed exercise is going to maximize benefit to your community or organization.

© 2018 – Timothy Riecker, CEDP

EPS New logo

Public Health Preparedness as Part of Emergency Management

I’ve written in the past on the need for emergency managers, in the broadest definition, to become more familiar with public health preparedness.  As emergency management continues to integrate, by necessity, into and with other professions, this understanding is imperative.  We need to stop considering EMS as our only public health interface.  Public health incidents, of which this nation has yet to be truly and severely struck by in decades, require more than public health capabilities to be successfully managed – so we can’t just write off such an incident as being someone else’s responsibility.  We’ve also seen non-public health-oriented disasters take on a heavy public health role as concerns for communicable diseases, biological agents, or chemical agents become suspect.  If you are an emergency manager and you aren’t meeting regularly with public health preparedness officials for your jurisdiction, you are doing it wrong.

Aside from meeting with public health preparedness staff, you should also be reading up on the topic and gaining familiarity with their priorities, requirements, and capabilities.  (don’t skip either of those links… seriously.  They each contain more info on public health preparedness).  One of the best resources available is TRACIE.  TRACIE is a resource provided by the US Department of Health and Human Services (HHS) Assistant Secretary for Preparedness and Response (ASPR).  TRACIE stands for the Technical Resources, Assistance Center, and Information Exchange.  I’ve been digging around in ASPR TRACIE for the past several years and also receive their monthly newsletter.  I get a lot of newsletters from different sources… some daily, some weekly, some monthly.  I’ve recently unsubscribed to a bunch which seem to have information that has diminished in value, doesn’t seem to be timely, or are poorly written.  TRACIE is one of those that stays.  It has tremendous value, even if you aren’t directly involved in public health preparedness and response.  The information and resources provided here come from public health preparedness experts – these are emergency managers.

Recently, ASPR did a webinar on Healthcare Response to a No-Notice Incident, highlighting the Las Vegas shootings. Check it out.

But public health speaks a different language!  True.  So do cops, firefighters, and highway departments.  So what’s your point?  While public health certainly does have certain terminology that covers their areas of responsibility, such as epidemiology, med-surge, and others, that doesn’t mean their language is totally different.  In fact, most of the terminology is the same.  They still use the incident command system (ICS) and homeland security exercise and evaluation program (HSEEP), and can talk the talk of emergency management – they are just applying it to their areas of responsibility.  Are there some things they might not know about your job?  Sure.  Just like there are things you don’t know about theirs.  Take the time to learn, and make yourself a better emergency manager.

What have you learned from public health preparedness?  How do you interface with them?

© 2018 – Timothy Riecker, CEDP

EPS New logo

HSEEP Training – Is it Required

Continuing from my previous blog post, I’ll answer a search phrase used to bring someone to my blog.  Earlier this month, someone searched ‘Is HSEEP training mandatory?’.  We speak, of course, of the Homeland Security Exercise and Evaluation Program, which is the DHS-established standard in exercise program and project management.

The short answer to the question: Maybe.

Generally speaking, if your exercise activities are funded directly or indirectly by a federal preparedness grant, then grant language usually requires that all exercises are conducted in accordance with HSEEP.  While most federal grant guidance doesn’t explicitly state that exercise personnel must be formally trained in HSEEP, it’s kind of a no-brainer that the fundamental way to learn the standards of practice for HSEEP so you can apply them to meet the funding requirement is by taking an HSEEP course.  If you are a jurisdiction awarded a sub-grant of a federal preparedness grant or a firm awarded a contract, there may exist language in your agreement, placed there by the principal grantee, that specifically requires personnel to be trained in HSEEP.

Beyond grant requirements, who you work for, who are you, and what you do generally don’t dictate any requirement for HSEEP training.  Aside from the federal grant funding or contracts mentioned, there is no common external requirement for any organization to have their personnel trained in HSEEP.  If your organization does require it, this is likely through a management-level decision for the organization or a functional part of it.

So, while HSEEP is a standard of practice, training in HSEEP, in general terms, is not a universal requirement.  That said, I would certainly recommend it if you are at all involved in the management, design, conduct, or evaluation of exercises.  FEMA’s Emergency Management Institute (EMI) offers HSEEP courses in both a blended learning and classroom format.  The emergency management/homeland security offices of many states and some larger cities offer them as well.

© 2018 Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC SM

Exercises: Simple is Usually Better

I find often that people want to run exercises they aren’t quite ready for.  Sometimes those exercises are too complex, or they simply aren’t the appropriate type.  Most often, we run exercises to test plans, policy, and procedures; but sometimes those plans, policies, and procedures aren’t quite ready to be tested.  Last year I advised a client to run a workshop instead of a tabletop exercise.  The initial goal of the tabletop was to validate a new plan, but this plan wasn’t ready to be validated.  The problem was that many stakeholders hadn’t yet seen the plan, and the review of that plan by our team in preparedness for the exercise wasn’t favorable.  The plan had much of the needed content, but it was disjointed and didn’t have any logical flow.  By conducting a scenario-based workshop, we were able to identify not only the ideal flow of the plan by flagging benchmark activities, but we were also able to discuss expectations of and for each stakeholder agency in the plan.  The client was then able to apply the results of the workshop to restructure their plan and make some needed substantive changes.

Similarly, I’ve encouraged a current client to conduct a workshop instead of a tabletop.  The initial goal of this tabletop was to identify how a new group of stakeholders could integrate into an existing plan.  In this situation, the tabletop would have been less than effective as the new stakeholder group isn’t yet identified in the plan.  The outcome of the workshop will be to identify how this integration can occur.

I think that sometimes people gravitate to certain exercises simply because they are more popular in a certain application.  That preconceived notion might be too complex or simply a poor choice for what you really need to accomplish.  When it comes to discussion-based exercises, most people default to a tabletop.  With operations-based exercises, it can vary.  Drills are often used for tactical applications, but we don’t see them as much in EOCs.  Drills certainly have a place in an EOC if you are looking to test a very specific function or activity.  While full-scale exercises are fun and sexy, I’ve been to the site of plenty that are total chaos because the fundamental premise of certain plans hasn’t been worked out (or some stakeholders aren’t familiar with them), which perhaps should have been done through a discussion-based exercise or a drill or functional exercise first.  Running a drill to test and familiarize the process of setting up key equipment prior to doing it for the first time in a full scale will pay a lot of benefits, and certainly prevent dozens or hundreds of other people being held up in a full scale.

Another issue I often see with exercises is very long and complex Master Scenario Events Lists (MSELs).  The MSEL is essentially the timeline or script of the exercise.  Along with listing all injects, it also identifies all benchmarks in the management of the exercise, such as StartEx and EndEx, and the introduction of new elements or transition to a different segment.  While there is no particular rule of thumb for how many injects are needed for different exercise types, everything needs to associate back to the objectives of the exercise.  I hate injects that are crafted simply for ‘noise’ (unless it’s an intel exercise), or injects intended to just give someone something to do.  Arguably, if the participants take an exercise seriously, such as a functional exercise, and play out the situation as they would in real life, you can engage an entire EOC for a few hours with even ten well-crafted injects.  While some functions are very focused, consider that the vast majority of what we do in emergency management requires coordination among a variety of elements and functions.  Capitalize on that.  One inject may engage multiple agencies or functions because of the need to coordinate and problem solve.  It’s not enough to identify a solution to the problem, but work through where the resources will come from, how they will get to where they need to go, and what support is needed for them and how long.  That’s a lot of problems to solve and will often transcend every function within the incident command system.  Exercises don’t need to be complex to be effective.  Create a handful of objectives and make sure everything relates back to them.  Simplicity can work.

My last recommendation is to keep your exercise planning team a manageable size.  I’ve been the lead planner for some very large exercises.  These exercises, largely due to their sponsors, ended up involving massive exercise planning teams – and by massive I mean over five or six dozen people – or more.  These are just sheer insanity.  Not every agency or organization involved in the exercise needs to be directly represented, nor does each organization need to send a small army of people.  What you do need is consensus from those organizations on the objectives and their scope of play.  That doesn’t mean they have to be involved in every aspect of planning the exercise.  Just like any other meeting or group project, a large exercise planning team can be cumbersome and management by committee is never efficient.  If need be, stakeholder groups can be developed based upon function.  For example, a fire service exercise planning team would develop their contributions to the exercise.  Just make sure that these groups are well coordinated and the overall exercise planning effort is unified, otherwise you’ll end with a disjointed exercise effort.

In the end, simplicity rules.  As you begin planning your exercise, consider, in every step if it can or should be simplified.  Always refer back to your intent and your objectives.  Chances are you can create a simpler exercise that is just as impactful, or perhaps more impactful.  When our inclination is to make things overly complicated, we often miss the point entirely.

© 2017 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC

 

 

Measuring Preparedness – An Executive Academy Perspective

A recent class of FEMA’s Emergency Management Executive Academy published a paper titled Are We Prepared Yet? in the latest issue of the Domestic Preparedness Journal.  It’s a solid read, and I encourage everyone to look it over.

First off, I wasn’t aware of the scope of work conducted in the Executive Academy.  I think that having groups publish papers is an extremely important element.  Given that the participants of the Executive Academy function, presently or in the near future, at the executive level in emergency management and/or homeland security, giving others the opportunity to learn from their insight on topics discussed in their sessions is quite valuable.  I need to do some poking around to see if papers written by other groups can be found.

As most of my readers are familiar, the emphasis of my career has always been in the realm of preparedness.  As such, it’s an important topic to me and I tend to gravitate to publications and ideas I can find on the topic.  The authors of this paper bring up some excellent points, many of which I’ve covered in articles past.  They indicate a variety of sources, including literature reviews and interviews, which I wish they would have cited more completely.

Some points of discussion…

THIRA

The authors discuss the THIRA and SPR – two related processes/products which I find to be extremely valuable.  They indicate that many believe the THIRA to be complex and challenging.  This I would fully agree with, however I posit that there are few things in the world that are both simple and comprehensive in nature.  In particular regard to emergency management and homeland security, the inputs that inform and influence our decisions and actions are so varied, yet so relevant, that to ignore most of them would put us at a significant disadvantage.  While I believe that anything can be improved upon, THIRA and SPR included, this is something we can’t afford to overly simplify.

What was most disappointing in this topic area was their finding that only a scant majority of people they surveyed felt that THIRA provided useful or actionable information.  This leaves me scratching my head.  A properly done THIRA provides a plethora of useful information – especially when coupled with the SPR (POETE) process.  Regardless, the findings of the authors suggest that we need to take another look at THIRA and SPR to see what can be improved upon, both in process and result.

Moving forward within the discussion of THIRA and SPR, the authors include discussion of something they highlight as a best practice, that being New York State’s County Emergency Preparedness Assessment (CEPA).  The intent behind the CEPA is sound – a simplified version of the THIRA which is faster and easier to do for local governments throughout the state.  The CEPA includes foundational information, such as a factual overview of the jurisdiction, and a hazard analysis which ranks hazards based upon likelihood and consequence.  It then analyses a set of capabilities based upon the POETE elements.  While I love their inclusion of POETE (you all know I’m a huge fan), the capabilities they use are a mix of the current Core Capabilities (ref: National Preparedness Goal) and the old Target Capabilities, along with a few not consistent with either and a number of Core Capabilities left out.  This is where the CEPA falls apart for me.  It is this inconsistency with the National Preparedness Goal that turns me off.  Any local governments looking to do work in accordance with the NPG and related elements, including grants, then need to cross walk this data, as does the state in their roll-up of this information to their THIRA and SPR.

The CEPA continues with an examination of response capacity, along the lines of their response-oriented capabilities.  This is a valuable analysis and I expect it becomes quite a reality check for many jurisdictions.  This is coupled with information not only on immediate response, but also sustained response over longer periods of time.  Overall, while I think the CEPA is a great effort to make the THIRA and POETE analysis more palatable for local jurisdictions, it leaves me with some concerns in regard to the capabilities they use.  It’s certainly a step in the right direction, though.  Important to note, the CEPA was largely developed by one of the authors of the paper, who was a former colleague of mine working with the State of New York.

The Process of Preparedness

There are a few topic areas within their paper that I’m lumping together under this discussion topic.  The authors make some excellent points about our collective work in preparedness that I think all readers will nod their heads about, because we know when intuitively, but sometimes they need to be reinforced – not only to us as practitioners, but also to other stakeholders, including the public.  First off, preparedness is never complete.  The cycle of preparedness – largely involving assessment, planning, organizing, equipping, training, and exercising – is just that – a cycle.  It’s endless.  While we do a great deal of work in each of these, our accomplishments are really only temporary.

The authors also mention that our information is not always precise.  We base a lot of what we do in preparedness on information, such as a hazard analysis.  While there are some inputs that are factual and supported by science, there are many that are based on speculation and anecdote.  This is a reality of our work that we must always acknowledge.  As is other of their points – there is no silver bullet.  There is no universal solution to all our woes.  We must constantly have our head in the game and consider actions that we may not have ever considered before.

ICS Improvement Officer

The authors briefly discuss a conceptual position within the ICS Command Staff they call the ICS Improvement Officer.  The concept of this fascinating, if not a bit out of place in this paper given other topics of discussion.  Essentially, as they describe this position, it is someone at the Command Staff level who is responsible for providing quality control to the incident management processes and implementations of the organization.  While I’ve just recently read this paper and haven’t had a lot of time to digest the concept, I really can’t find any fault with the concept.  While the planning process itself is supposed to provide some measure of a feedback loop, there isn’t anyone designated in the organization to shepherd that process beginning to end and ultimately provide the quality control measures necessary.  In practice, I’ve seen this happen collaboratively, among members of the Command and General Staff of a well-staffed structure, as well as by the individual who has the best overall ICS insight and experience in an organization – often the Planning Section Chief.  The authors elude to this position also feeding an AAR process, which contributes to overall preparedness.  I like this idea and I hope it is explored more, either formally or informally.

Conclusion

There are a number of other topic areas of this paper which I haven’t covered here, but I encourage everyone to read on their own.  As mentioned earlier, I’d like to see more of the research papers that come from FEMA’s Emergency Management Executive Academy available for public review.  Agree or disagree with their perspectives, I think their discussions on various topics are absolutely worth looking at.  It’s these discussions like these which will ultimately drive bigger discussions which will continue to advance public safety.

I’m always interested in the perspectives of my readers.  Have you read the paper?  What do you think of the discussion topics they presented?

© 2017 – Timothy M Riecker, CEDP

Emergency Preparedness Solutions, LLC

 

Preparedness Exercises – The Building Block Concept Can Be Misleading

If you’ve been doing emergency management and homeland security exercises for even a little while, you are probably familiar with this graphic.  It’s included in a great many exercise training courses and other materials that talk about the different types of exercises out there.  This graphic looks correct at first glance.  It seems to make sense.  Intuitively, we assume that full-scale exercises are the most complex exercise type and that they test capabilities to the greatest extent.  When we put more thought into that, though, we realize that it can be wrong.

I’m happy to note that the most recent document on the Homeland Security Exercise and Evaluation Program (HSEEP) (April 2013) does NOT include this graphic.  They include a couple of paragraphs on using a ‘progressive approach’ to a multi-year exercise program, recommending that “each successive exercise building upon the previous one until mastery is achieved” is the path forward.  While I’m not too crazy about this statement, either, it’s not as misleading as the legacy ‘building block’ graphic.

Here’s the problem with the graphic – It assumes too much.  While in some instances this graphic can be correct, it shouldn’t be regarded as even occasional guidance.  I’ve witnessed, managed, designed, evaluated, or otherwise participated in a number of workshops, tabletops, drill, and functional exercises that are far more complex than some full-scale exercises I’ve likewise been involved in.  Similarly, I can attest to some ‘lower level’ exercises testing capabilities to a greater extent than the ‘higher level’ exercises.  The hierarchal structure is simply misleading.  The problem this can create is people being dismissive of the relevance, necessity, and value of conducting exercises other than functional and full-scale.

Making a training comparison, exercise types don’t really stick to a scaled taxonomy.  (Check out this article for a small introduction to Bloom’s Taxonomy which is used in instructional design.) While we can state that an awareness level course is more about knowing and understanding, and an operations-level course is more about applying, we can’t say the same for exercises as a spectrum generalization.  Consider that seminars are certainly geared toward orienting people with information, while workshops, on the other hand, require creation of a product, by definition.  Tabletop exercises can (and should) be incredibly in-depth, analyzing plans and policy, while drills are more about application.

So what exercise type should you choose?  It goes back to the foundations of exercise design – consider your objectives, then consider what exercise type best fits what was identified that you want to accomplish.  Reflecting, now, back on the progressive approach mentioned in the HSEEP document, it’s true that you probably don’t want to jump into a full-scale right away.  That has less to do with the perceived complexity of the exercise and more to do with how plans are properly tested – which is the main reason why we exercise.  First, we need to talk through aspects of our plans, policies, and procedures.  We need to make sure that our foundational assumptions are sound and that broader decisions and actions are in line with the documents created.  This is typically the reasoning behind conducting a table top exercise before any type of operations-based exercise.  Once we have established that the foundational premise and supporting policies of the plan is sound, then it can make sense to progress to a selection of operations-based exercises to test various tactile aspects of the plan and associated procedures.

That said, some procedures are so tactile that really the only way to test them is through an operations-based exercise.  Perhaps they are first built in a workshop-type of environment, then go straight to a drill for testing.  That might be all that’s needed.  Yes, they could be further integrated into a functional or full-scale later, if you choose to integrate them into other activities.

The bottom line here is that while the building-block approach to exercises makes sense at a glance, it is really quite more complex than the graphic eludes to.  Yes, it still has some relevance, but it should not be viewed as the rule.  Just as in training and emergency response, objectives drive everything in exercises.

© 2017 – Timothy M Riecker

Emergency Preparedness Solutions, LLC

 

Incident Evaluation

I’ve written at length about the importance of quality evaluation of exercises.  Essentially, if we don’t evaluate exercises, and do it well, the benefits of the exercises are quite limited.  Generally, we don’t see a benefit to incidents.  By their very nature, incidents threaten and impact life, property, and environment – things we don’t view as being beneficial.  However, benefits are often a product of opportunity; and we absolutely should take the opportunity to evaluate our responses.

Many incidents do get evaluated, but through research after the fact.  We retrace our steps, review incident documents (such as incident action plans), interview personnel, and examine dispatch logs.  These efforts usually paint a decent picture of intent and result (things that are often different), but often miss the delta – the difference between the two – as well as other nuances.  When we evaluate an exercise, we do so in real time.  Th evaluation effort is best done with preparation.  Our evaluation plans, methodologies, and personnel are identified in the design phase of the exercise.  Just as we develop emergency operations plans and train personnel to respond, we can develop incident evaluation plans and train personnel to evaluate incident responses.

Understandably, a hurdle we might have is the availability of personnel to dedicate solely to evaluation, especially on larger incidents – but don’t be afraid of asking for mutual aid just to support incident evaluation (just be sure to include them in your preparedness efforts).  Just as regional exercise teams should be developed to provide cooperative efforts in exercise design, conduct, and evaluation; incident evaluation teams should be developed regionally.  To me, it makes sense for many of these personnel to be the same, as they are already familiar with how to evaluate and write up evaluations.

In exercises, we often use Exercise Evaluation Guides (EEGs) to help focus our evaluation efforts.  These are developed based upon identified Core Capabilities and objectives, which are determined early in the exercise design process.  While we don’t know the specific objectives we might use in an incident, we can identify these in general, based upon past experiences and our preparedness efforts for future incidents.  Similarly, our emergency planning efforts should be based around certain Core Capabilities, which can help inform our incident evaluation preparedness efforts.  Job aids similar to EEGs, let’s call them incident evaluation guides (IAGs), can be drafted to prepare for incident evaluation, with adjustments made as necessary when an incident occurs.

Evaluating an incident, in practice, is rather similar to how we would evaluate an exercise, which is why the training for these activities is relatively portable.  Evaluation efforts should avoid evaluating individuals, instead focusing on the evaluation of functions and processes.  Don’t reinvent the wheel – evaluate based upon documented (hopefully!) plans and procedures and use the Homeland Security Exercise and Evaluation Program (HSEEP) standards to guide your process. Incident evaluation must be managed to ensure that evaluation gaps are minimized and that evaluation progresses as it should.  Observations should be recorded and, just as we would for an exercise, prepared for and eventually recorded in an after action report (AAR).

I favor honest after action reports.  I’ve seen plenty of after action reports pull punches, not wanting the document to reflect poorly on people.  Candidly, this is bullshit.  I’ve also heard many legal councils advise against the publication of an after action report at all. Similarly, this is bullshit.  If our actions and the need to sustain or improve certain actions or preparations is not properly recorded, necessary changes are much less likely to happen.  If an AAR isn’t developed, a corrective action plan certainly won’t be – which gives us no trackable means of managing our improvements and disavows our intent to do so.

As a profession, public safety must always strive to improve.  We have plenty of opportunity to assess our performance, not just through exercises, which are valuable, but also through the rigors of incident responses.  Prepare for incident evaluation and identify triggers in your emergency plans for when evaluation will be employed, how, and who is involved.  Begin evaluation as early as possible in an incident – there are plenty of lessons learned in the early, and often most critical moments of our incident response.  Finally, be sure to document lessons learned in an AAR, which will contribute to your overall continuous improvement strategy.

How does your agency accomplish incident evaluation?  If you don’t, why?

Need help with the evaluation of incidents?  We are happy to help!

© 2017 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC