Tag preparedness exercises

NEW: 2020 HSEEP Revision

Tim Riecker, The Contrarian Emergency Manager 1 Comment

Earlier today FEMA dropped the latest version of the Homeland Security Exercise and Evaluation Program (HSEEP) doctrine.  Doing a quick comparison between this new version and the previous (2013) version, I’ve identified the following significant changes:

  • They replaced the ‘Elected and Appointed Officials’ mentions within the document with ‘Senior Leaders’. This makes sense, since often the elected and appointed officials simply aren’t involved in many of these activities.  The previous terminology is also exclusionary of the private sector and NGOs.
  • The document specifically references the Preparedness Toolkit as a go-to resource.
  • A big emphasis through the document is on the Integrated Preparedness Cycle (see the graphic with this post). The Integrated Preparedness Cycle covers all POETE (Planning, Organizing, Equipping, Training, and Exercising) elements plus Evaluate/Improve.  The graphic also eludes to these activities not necessarily happing in a specific order, as well as the consideration of Preparedness Priorities and Threats, Hazards, and Risks.  Developing a preparedness plan is something I wrote about back in 2016.
  • Integrated Preparedness Cycle
    • Going along with the Integrated Preparedness Cycle, they have done away with the Training and Exercise Plan (TEP) and replaced it with the Integrated Preparedness Plan (IPP), which is developed through input obtained during an Integrated Preparedness Planning Workshop (IPPW). I serious HOPE this shift is successful, as I’ve mentioned in the past how often the training aspect of the TEP was ignored or phoned in.  This approach also does a lot to integrate planning, organizing, and equipping (but ESPECIALLY planning) into the effort.  This is all tied together even more if a jurisdiction has completed a THIRA.  The Integrated Preparedness Cycle and IPP are the things I’m happiest about with the updated document.
  • The new document provides easier to find and read layouts for information associated with exercise types and each of the planning meetings.
  • For years, HSEEP doctrine has suggested (though thankfully not required) an ICS-based organization for exercise planning. I’ve never used this as I found it awkward at best (though I know others often use it and have success in doing so).  The update provides a different suggestion (better, in my opinion) of a functionally organized planning team organization.  Consider that this is still a suggestion, and that you can use it, or a version of it, or an ICS-based one, or anything else you desire.
  • The update provides better delineation between the planning and conduct needs of discussion-based exercises vs those of operations-based exercises. Those of us who have been doing it for a while know, but for those who are new to exercises this should be very helpful.
  • Lastly, the document suggests making corrective actions SMART, as these are really objectives.

FEMA is hosting a series of webinars (listed on the HSEEP website) to discuss these changes.

I’m very happy with the changes made to the doctrine.  It’s a great continued evolution of HSEEP and preparedness as a whole.  For as much as I’m a champion of the Integrated Preparedness Plan, though, having it (thus far) only included in the HSEEP doctrine makes it easy to miss or dismiss by some.  I’m hopeful broader promotion of this concept, perhaps even including it as an emergency management performance grant requirement, will help adoption of this concept.

What are your thoughts?

© 2020 Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC®

Emergency Management Exercises: Not for the Inexperienced

Tim Riecker, The Contrarian Emergency Manager Leave a comment

Many think exercise design is easy.  I’ve seen agencies relegate it to interns and new staff with little supervision, or even performed by seasoned emergency managers with little concept of what the Homeland Security Exercise and Evaluation Program (HSEEP) is.  Sadly, we have people completing HSEEP training and even FEMA’s Master Exercise Practitioner (MEP) program, thinking they are ready to conquer the world of preparedness exercises, but with little practical experience designing exercises under their belts.  We all need to learn sometime.

Just as any organization or jurisdiction should be eased into their exercise program, exercise designers need to be eased into designing exercises.  They should be starting small and with focused tasks, always under the mentorship of someone experienced, even if they aren’t within your own agency, to give some guidance and feedback.  While HSEEP gives a lot of great guidance, exercise design can quickly become complex.  It can be easy to lose track of tasks or have an oversight.  There are political matters, organizational needs, safety issues, and simply good exercise practices that all need to be recognized and addressed.  I’ve seen far too many exercises go off the rails due to a lack of awareness of these issues, poor exercise design, and poor exercise management.

Have partner agencies (even if not participating) been properly notified?  Do notifications need to go out to the media or public so they are not alarmed?  How about dispatch?  Every exercise, especially operations-based exercises, should be periodically evaluated for risk throughout the design process.  Identify what actions or lack thereof can cause things to go bad.  Consider politics, the media, the public, and safety of participants, observers, and exercise staff.  Do you need a weapons policy?  How will you enforce it?  Are there risks associated with traffic?  How will exercise staff communicate?  The template for the Exercise Plan (ExPlan) prompts you to address some things, but there may be additional needs.

What contingencies do you have for inclimate weather?  Maybe you need to dip into the ICS tool box and conduct an incident (exercise) safety analysis, from that developing a safety plan (you can probably get a qualified/experienced safety officer to help you with this).  Consider what operations will be conducted in the exercise, what can go wrong, how you will mitigate against them, and what resources are needed if something does go wrong.  In the event of a real-world emergency, what needs to happen?  Should you have EMS standing by?  Should you have a rapid response team in reserve for a rescue situation?  The information assembled in your risk assessment and safety plans should be provided to exercise staff prior to the exercise as part of their pre-exercise briefings.

As with exercise design, it can be a great learning experience for new staff to be part of the exercise support staff, but don’t put them in charge.  You should have experienced staff serving in the key positions of exercise director, lead controller, and lead evaluator.  If you are using a simcell, you want a strong and experienced simcell lead.  Safety matters aside, the staff of an operations-based exercise need to have great awareness of what’s going on and excellent communication up their chain of command and with the simcell to ensure that the exercise is flowing properly.  For a discussion-based exercise, your facilitators should be experienced as well.  Participants in discussion-based exercises may take a discussion in a different direction.  While this is generally not desired, sometimes it does bring great unintended results.  An experienced facilitator should know how to properly handle this to ensure that participants and stakeholder agencies are getting the most benefit.

Far too many poorly designed exercises have gotten to execution, resulting in a failure to accomplish the exercise objectives, frustrating participants, and even resulting in inter-agency political issues or injuries.  Even a well-designed exercise can be poorly conducted or facilitated, getting similar results.  If you are new to emergency management and have little experience in the design, conduct, and evaluation of exercises, there is no shame in asking for help or at least another set of eyes to look over your exercise documents.  While we want to encourage learning and growth, no one learns properly by being shoved into a situation with no guidance and so many pitfalls.  Train people up properly, giving them mentored practical experience to compliment their classroom training. If you don’t have the personnel in place, there are a number of well experienced and well qualified firms (ahem…) that provide these services.

For more information on running an exercise program, take a look at this 10-part blog post.

What tips do you have for people new to the exercise world?

© 2018 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC™

Preparedness Exercises – The Building Block Concept Can Be Misleading

Tim Riecker, The Contrarian Emergency Manager Leave a comment

If you’ve been doing emergency management and homeland security exercises for even a little while, you are probably familiar with this graphic.  It’s included in a great many exercise training courses and other materials that talk about the different types of exercises out there.  This graphic looks correct at first glance.  It seems to make sense.  Intuitively, we assume that full-scale exercises are the most complex exercise type and that they test capabilities to the greatest extent.  When we put more thought into that, though, we realize that it can be wrong.

I’m happy to note that the most recent document on the Homeland Security Exercise and Evaluation Program (HSEEP) (April 2013) does NOT include this graphic.  They include a couple of paragraphs on using a ‘progressive approach’ to a multi-year exercise program, recommending that “each successive exercise building upon the previous one until mastery is achieved” is the path forward.  While I’m not too crazy about this statement, either, it’s not as misleading as the legacy ‘building block’ graphic.

Here’s the problem with the graphic – It assumes too much.  While in some instances this graphic can be correct, it shouldn’t be regarded as even occasional guidance.  I’ve witnessed, managed, designed, evaluated, or otherwise participated in a number of workshops, tabletops, drill, and functional exercises that are far more complex than some full-scale exercises I’ve likewise been involved in.  Similarly, I can attest to some ‘lower level’ exercises testing capabilities to a greater extent than the ‘higher level’ exercises.  The hierarchal structure is simply misleading.  The problem this can create is people being dismissive of the relevance, necessity, and value of conducting exercises other than functional and full-scale.

Making a training comparison, exercise types don’t really stick to a scaled taxonomy.  (Check out this article for a small introduction to Bloom’s Taxonomy which is used in instructional design.) While we can state that an awareness level course is more about knowing and understanding, and an operations-level course is more about applying, we can’t say the same for exercises as a spectrum generalization.  Consider that seminars are certainly geared toward orienting people with information, while workshops, on the other hand, require creation of a product, by definition.  Tabletop exercises can (and should) be incredibly in-depth, analyzing plans and policy, while drills are more about application.

So what exercise type should you choose?  It goes back to the foundations of exercise design – consider your objectives, then consider what exercise type best fits what was identified that you want to accomplish.  Reflecting, now, back on the progressive approach mentioned in the HSEEP document, it’s true that you probably don’t want to jump into a full-scale right away.  That has less to do with the perceived complexity of the exercise and more to do with how plans are properly tested – which is the main reason why we exercise.  First, we need to talk through aspects of our plans, policies, and procedures.  We need to make sure that our foundational assumptions are sound and that broader decisions and actions are in line with the documents created.  This is typically the reasoning behind conducting a table top exercise before any type of operations-based exercise.  Once we have established that the foundational premise and supporting policies of the plan is sound, then it can make sense to progress to a selection of operations-based exercises to test various tactile aspects of the plan and associated procedures.

That said, some procedures are so tactile that really the only way to test them is through an operations-based exercise.  Perhaps they are first built in a workshop-type of environment, then go straight to a drill for testing.  That might be all that’s needed.  Yes, they could be further integrated into a functional or full-scale later, if you choose to integrate them into other activities.

The bottom line here is that while the building-block approach to exercises makes sense at a glance, it is really quite more complex than the graphic eludes to.  Yes, it still has some relevance, but it should not be viewed as the rule.  Just as in training and emergency response, objectives drive everything in exercises.

© 2017 – Timothy M Riecker

Emergency Preparedness Solutions, LLC

 

Using Departmental Operations Centers for Incident Management Success

Tim Riecker, The Contrarian Emergency Manager Leave a comment

Increasingly, government agencies and departments are identifying the benefits of establishing and activating departmental operations centers (DOCs) to help manage their responses to incidents.  At the Vermont Vigilant Guard 2016 exercise, which concluded last week, I had some opportunity to discuss the benefits of DOCs, particularly with an agency who used theirs for the first time in this exercise.

For most agencies, a DOC can relieve agency representatives in an EOC from also having to manage and track their agency’s response activity.  In an EOC, an agency representative is largely a conduit for communication and they provide knowledge of their agency and their agency’s capabilities as they contribute to the greater discussions within the EOC.  According to NIMS/ICS, an agency representative should have some decision-making capability for their agency, although political and practical realities often dictate otherwise.  The overall scope of activity for an agency representative in an EOC largely precludes them from also managing the details of their agency’s response, particularly if that response is even moderately complex.

A DOC is the ideal location from which an agency can oversee and coordinate their own response to an incident.  They can deploy and track resources, address internal logistics matters, and coordinate external logistics matters back through their agency representative at the EOC.  DOCs are also an excellent application for large agencies, which may have a variety of technical functions organized throughout, such as a health department or transportation department.  Pulling together representatives from each organizational element within the agency to collectively troubleshoot, problem solve, and share resources, is excellent use of a DOC.  In a way, this application of a DOC could be considered similar to a multi-agency coordination center (MACC).

Does a DOC need to mirror NIMS/ICS (or the new Center Management System) standards?  While there is no set standard for organizing and managing a DOC, there are a lot of applications of ICS that can certainly be applied.  If you look at the main activities of your DOC, you will see where opportunities for integration of ICS principles exist.  Consider that a DOC should have and maintain good situational awareness.  While much of this can be provided by the EOC, the EOC may (should) be looking for some specific information from your agency.  A situation unit within your DOC would certainly be helpful.  Likewise, DOCs often address tactical or near-tactical application, by deploying and directing resources from throughout their agency.  Having a resource unit within your DOC will help tremendously in the tracking of these resources.  Depending on the size and scope, it may be prudent for your DOC to establish an incident action plan (IAP) of its own.

Logistics, mentioned earlier, may be another need within a DOC.  Certainly an element of finance is important for the approval of procurements and tracking of costs within the agency related to the incident.  If resources are being deployed, someone should be in charge of operations.  Lastly, any organization needs to maintain someone in charge.  A DOC Manager would be the ideal generic term for this position.

What are the draw backs of establishing a DOC?  First of all, it’s an additional layer of incident management.  While possibly necessary based on factors discussed earlier, adding layers of incident management can make incident management more complex, especially if roles of each layer and function are not well defined.  The best way to address this is pre-planning!  Staffing can also be a significant concern.  Many agencies may be too small to warrant, much less have staff available, for a DOC.  If such is the case, just as in other applications of ICS, you should be able to collapse down to a manageable size.

An often seen pitfall of DOCs is that they can quickly devolve into a management by committee type of structure; particularly with larger agencies where senior staff, who are used to regular meetings with each other, are now representing their functional interests within the DOC.  I’ve seen this result in what is essentially one endless meeting, interrupted by phone calls and emails which introduce new problems and perpetuate more discussion.  Strong leadership is absolutely required to ensure that a group such as this stays focused and on task, resolving issues on a timely basis.

Overall, the use of Departmental Operations Centers is a smart practice.  Work internally to plan their use, scoping out when and how it will be applied, where it will be located, the organization to be used, and how it is integrated into the overall incident management effort.  Once plans are developed and appropriate training is performed, exercise the plan to identify areas for improvement, turning those into corrective actions, and implementing them for continued success.

© 2016 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC – Your Partner in Preparedness

Five Guidelines for Creating Effective Disaster Exercise Injects

Tim Riecker, The Contrarian Emergency Manager 1 Comment

While there is a lot of important and necessary planning that takes place before the development of exercise injects can even be considered, injects themselves are where the proverbial rubber meets the road.  How we craft those injects can often times make or break the conduct of the exercise.  Injects provide context, as if the events of the exercise were occurring in real life.  While we try to avoid delivering injects that directly prompt player responses, injects will often provide information which will lead players to react to the information provided.  Here are five guidelines to help you develop effective exercise injects:

  1. Injects must be purposeful and each one must relate back to one or more exercise objectives. Far too often we see injects that have no real bearing on the objectives of the exercise.  These are simply distractions and lead to busy work.  Keep things focused.  Just a few well-crafted injects can engage a number of players in active discussion or activity.
  2. Realistic injects are a must. While there will always be a grumble from some people claiming that something would ‘never happen that way’, due diligence must go into ensuring that injects are as realistic and grounded as possible.
  3. Be aware of who an inject would actually originate from. A common mistake I see is injects being scripted to originate from inappropriate sources.  This distracts from reality.  Also, injects should never originate from a player.
  4. Be flexible and aware. Sometimes players accomplish what they need to without an inject.  In that event, there may not be a need to use that inject.  Similarly, players may not respond to an inject as expected, so further action on the part of the Facilitators/Controllers/Simcell may be needed.
  5. Always have backups! As you build your Master Scenario Events List (MSEL), maintain a side list of contingency injects that can be used to speed up or slow down the exercise, or to address occurrences where players did not respond as expected.

Thoughts and ideas on these and other guidelines are always welcome!

© 2016 – Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC – Your Partner in Preparedness!

Ebola Reflection Measures our Preparedness

Tim Riecker, The Contrarian Emergency Manager Leave a comment

NBC News recently posted an article citing a report published by the Presidential Commission for the Study of Bioethical Issues.  The link provided to the report in the NBC News article doesn’t seem to work, but I’ve found what I believe to be the report here.  The focus of the report is on the ethical challenges faced by the US in responding to this issue.  The report summarizes a variety of ethics related concerns and considerations in this ongoing response and paints a fairly accurate picture of our failures and what needs to be addressed – at least within the topics it discusses.

Photo credit: Forbes.com

Photo credit: Forbes.com

As you might expect from a report on bioethics, it is very public health focused.  While they do make mention of very public health centered topics such as clinical drug trials, they do cover topics which are much more broadly rooted in emergency management and homeland security, such as community and responder education, and ethics associated with quarantine.  This report, while fairly focused, opens a virtual Pandora’s box of issues related to our domestic response to Ebola.

Needless to say, our collective response to this matter was horrible.  Public health policy and guidance was a moving target for weeks; responders were ill prepared to handle potentially infected persons; and the collective of society, politicians, and public safety were largely reluctant to deal with matters of quarantine much less prepared for it.  Was this our first consideration of something like Ebola?  Of course not.  Didn’t we have preparations in place?  Kind of.

Back in the late 90s, pushed mostly by the Nunn-Lugar-Domenici act of 1996, preparedness efforts for state and local responders were funded to enhance our capabilities in dealing with WMD incidents.  Several years later, after 9/11 and the anthrax attacks, another surge of funding was pushed down to state and local governments from HHS/CDC for the purpose of bolstering public health preparedness including preparedness for WMD/weaponized biologicals and naturally occurring pandemic incidents.  These two programs alone, not including other related funding, fostered the creation of plans and organizations to support them, purchased entire stockpiles of equipment and supplies, trained tens of thousands of responders and public health workers, and encouraged exercises across the nation to test capabilities (it was actually these exercises which largely influenced the creation of what we now know as HSEEP).  A lot of good came from these programs, but when suddenly tested with the reality of implementation we seemed to fall apart.  Why?

First of all, many of these preparedness efforts occurred between 10 and nearly 20 years ago.  Many of the people initially trained in these programs have since retired from public service with their organizations losing a great deal of institutional knowledge.  While training programs have continued and still exist, there have been systemic gaps in tying this type of training to other preparedness efforts (planning, policy, equipment, etc.).  Some equipment purchased near the beginning of these programs has likely been retired as well.  Much of it still exists, but has been brought into the fold of other applications, such as HazMat – which is certainly appropriate, but yet again we see gaps, this time our ability to readily utilize equipment specifically for public health threats.

In my opinion many of the planning efforts we saw after 9/11 were misguided.  This started with the people who were doing the planning.  Many health organizations emphasized health care experience for these positions instead of EM or planning experience – which was their main function.  Certainly health care knowledge had some importance, but that could be supplemented through a good advisory committee (EM after all is a team effort).  Exacerbating poor hiring decisions was a lack of investing in the people that were hired.  Many organizations expected them to churn out pandemic influenza plans in short order, with little/no training on the planning process or integral systems that must be considered.  Further, much of the planning had been done in a vacuum – that is, it had been performed with little/no input from other stakeholders.  I had reviewed many of these plans, finding things such as inappropriate applications of ICS and wild assumptions of resource availability.  In no way were these plans realistic or applicable.

There were many exercises performed and most of them had great value.  The problem is that there were a lot of assumptions in these exercises and policy decisions made in the exercises were rarely challenged as they would be in reality.  The US Ebola response brought this all to light as decisions such as quarantine were being handled at the governor level and under significant controversy.  So in this recent response I ask why were decisions delayed and deferred to higher authorities?  Why were adequate local/regional plans not in place to address the care and handling of potentially infected persons?  Why did procedural issues take weeks to resolve?  The simple answer is that there was a lack of proper preparedness.

Back in October of last year, when Ebola was emerging in the US, I posted an article titled Preparing for Ebola – and Whatever Else May Come.  The article still has a great deal of relevancy since I’ve seen very little preparedness for future occurrences – only a harried response to the most recent incident.  There have certainly been a great deal of policies and procedures assembled for the current Ebola issue, but these have a feeling of being temporary, throwaway, or single-use documents, applied only for this instance instead of durable and lasting plans.  Many will keep them ‘on the books’, only to find that their hasty assembly wasn’t comprehensive enough for the next occurrence.  Emergency management and homeland security professionals, public health leaders, and elected and appointed officials need to take a step back and re-look at out preparedness efforts – especially in regard to public health issues.  While we should learn from what we have experienced, we also need to think comprehensively about what is needed.  Well considered policies need to be put in place, supported by our laws and responsibilities to protect the public while also considering protection of civil liberties.  Other preparedness efforts such as planning, training, and exercising need to continue to occur but must have their connections strengthened and intentional.  Exercises need to test plans and policies and challenge decision makers who are certainly making difficult decisions that may include ethics and moral issues in the consideration of caring for few while protecting the greater society.

These are not easy things to be done – which supports the need to work on them now, when we aren’t facing an imminent disaster.  While Ebola certainly wasn’t a health care crisis and there were a lot of things done right, there is always room for improvement – especially when the next biological occurrence could be a crisis.

What have you and your organization learned from the Ebola response?  What gaps have you addressed?  What do you feel still needs to be addressed?

© 2015 – Timothy Riecker

Are You Inviting the Right People to Your Exercises?

Tim Riecker, The Contrarian Emergency Manager Leave a comment

A couple of days ago I started reading Rumsfeld’s Rules – Leadership Lessons in Business, Politics, War, and Life.  Hopefully you have some familiarity with Donald Rumsfeld – the man was a naval aviator, US Congressman, aide to four US presidents, corporate CEO, and is the only person to ever serve as Secretary of Defense twice.  Politics aside, Mr. Rumsfeld has had quite a prolific career.  Throughout this career he has assembled a variety of mantra, proverbs, and sayings which he has used to help guide his career and serve as advice to others.

Early in the book, Mr. Rumsfeld talks about meetings.  What he mentions struck me as solid guidance not only for meetings but also for exercises.  He says “There is a balance that needs to be struck in determining who to invite to a meeting.  You want those who need to be there to contribute substance to the discussion.  But it can be useful to have people who may not be in a position to directly offer substantive input but will benefit from hearing how and why certain decision are being reached.”  Very often exercise offer great opportunity for people to learn – not only the participants but ‘shadowers’ as well.

Mr. Rumsfeld continues on to say “Including a range of people can also ensure that a variety of perspectives will be considered and help identify gaps in information and views.”  Consider that we build, conduct, and evaluate exercises primarily to test plans, polices, and procedures.  This testing is best performed by a spectrum of individuals giving different ideas and perspectives.  Someone may interpret a policy in a completely different way or have an approach to a problem that hasn’t been considered prior.  These fresh ideas, even if flawed, should be brought out into the open for discussion and consideration.

If you’ve followed this blog for any amount of time, you probably know that I prefer smaller meetings and have stressed that participants in exercises should be of a manageable number.  As Mr. Rumsfeld says, there is a balance that must be struck.  You want to be inclusive, but large numbers lend themselves to over-discussion and tangents.  For meetings do you expect the person to add value?  Should they be there given their area of responsibility?  Similarly in exercises is the individual associated with the objectives of the exercise?  (Recall that in exercises we should always reflect on the objectives throughout the entire design process).  When we add more participants to an exercise we need to ensure that they have something to participate in, so injects must be written for them and their activities must be evaluated.

A few years back my team was designing a table top exercise as a lead-in to a significant full-scale exercise.  We did not want to start the full-scale with the initial response, as so many often times are, as the objectives of that exercise were to test the extended response and to examine issues beyond the initial response.  That said, we felt it not fair for us to design such a large exercise by dictating what the first responders would do in the first 48 hours, rather we wanted them to tell us themselves.  So we designed a table top exercise to provide us with their actions both ‘boots on the ground’ as well as policy-level including emergency declarations, evacuation areas, and mutual aid requests.  We were quite fortunate that the design process for the exercise as a whole was very well received and many agencies wanted to participate – from federal, state, county, and local jurisdictions.  The exercise was centered on the state capital, which tends to garner even more attention and participation and included a scenario that most agencies have not participated in prior.  Needless to say, we had a lot of interest.  Nearly every agency invited to the table top wanted to bring not one or two additional people but often times three or four.  We discussed this matter with a few of the key agencies, asking of these were needed participants or observers.  The answer we got was that they were both.  Because of the technical nature of the incident, many agencies realized they needed their main spokesperson supported by one or more technical experts.  We realized this was a fair and reasonable request, but we still needed to figure out how to accommodate them all!

We decided to permit each representative to have a ‘second chair’ – someone seated directly behind them who could advise on technical matters.  Additional specialists were available to them in an adjacent room, which had the discussion live broadcast to them via closed circuit television.  Specialists could be ‘swapped out’ at any time based on the needs of the discussion.  This solution worked well for the exercise, keeping the number of direct participants manageable and meeting the needs of participants to have their specialists available to advise on technical matters – which truly helped inform their decisions and ultimately the outcome of the exercise.

Sometimes, though, you have to say ‘no’.  Realize that as an exercise designer you MUST set a firm deadline on additional participants.  Participants that are added late can set your design team back significantly by needing to ensure that they are written into the exercise and have sufficient activity to make their participation worth while for both them as well as the exercise as a whole – which can be particularly challenging if they are from a different jurisdiction or discipline altogether.  I’ve had to turn down several interested parties and while it’s often difficult to say no, it’s often for the better – and your design team will respect you for it.

What thoughts do you have on ‘right sizing’ your meetings and exercises?  Is there certain guidance that you use?

©2014 Timothy Riecker

Engaging a Nation in Preparedness – Learning from History

Tim Riecker, The Contrarian Emergency Manager Leave a comment

June 14, 1954 saw the first nation-wide civil defense drill conducted in the United States.  The Civil Defense Administration organized and promoted the event, which included operations in 54 cities around the country, including Puerto Rico, the US Virgin Islands, Alaska, and Hawaii.  Canada had also participated in the event.  The History Channel website has a nice write-up on the event.  The History Channel’s article explains some of the activities conducted during the event, which largely consisted of sheltering drills.

Today we do see some nation-wide exercises which engage citizens through the Shake Out earthquake drills.  Their website has a great deal of information on the program, including how you can participate.  The statistics on their site are great, showing not only the US regional exercises but also Shake Out exercises conducted in nations around the world (something I was not familiar with until visiting their site this morning).  The earthquake hazard in the US and around the globe is significant – in fact we just saw two large earthquakes late yesterday – a 7.9 near Alaska and a 7.2 near New Zealand.  While the core activity of the Shake Out exercises is the ‘Drop, Cover, and Hold on’ (similar to the ‘duck and cover’ of the civil defense days), their website also promotes preparedness activities including a ‘hazard hunt’ for items which may fall during an earthquake, a family disaster plan, business and organizational continuity planning, and emergency supply kits.  This is the type of preparedness activity we need to continue, but we also need to do more.  Unfortunately the message still isn’t getting through to many people.

How do you think we should get the message out?

 

© 2014 Timothy Riecker

 

 

Managing an Exercise Program – Part 10: Evalutation and Improvement Planning

Tim Riecker, The Contrarian Emergency Manager Leave a comment
New HSEEP graphic

New HSEEP graphic

For the previous parts in my series Managing an Exercise Program, please see below:

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

 

While conducting an exercise is often the most exciting part of the exercise cycle, evaluation and improvement planning lead us to the real reasons why we conduct exercises – to test plans, policies, and procedures.  Evaluation doesn’t just happen.  It must be a deliberate, planned activity within the design and conduct of an exercise.  There are key activities through the design and development of an exercise which link directly to evaluation – starting with the identification of core capabilities to be tested and exercise objectives to be accomplished.  These core capabilities and objectives should lead us directly back to specific plans, policies, and procedures which will be exercised and can help us determine the evaluation methodology and approach.  As we further develop our exercise, be it discussion based or operations based, the decisions we make influence evaluation and the resources needed for it.

When exercise planning begins, we identify staff to fill key roles – including exercise director, simcell manager, lead controller, and lead evaluator.  The lead evaluator doesn’t necessarily have to be involved in all aspects of exercise design, but they need to be informed of key points and should be reviewing draft materials (i.e. the explan or sitman) to become familiar with the details of the exercise and to help them assemble their portion of the control and evaluation plan.  The lead evaluator should ideally be present at exercise planning meetings to become even more familiar with the details, the people, and the facility(s) of exercise play.

Staffing evaluation has a number of options.  Often times evaluators will be selected based on matching their individual areas of expertise with the functional areas of the exercise (i.e. fire, EMS, public health, EOC management, shelter operations, etc.).  You may have a need for more than one evaluator per functional area, particularly if that function is a major component of the exercise and there is a lot to observe.  Remember that if there are multiple exercise venues, they each need to have evaluators assigned who can observe all critical areas.  The bottom line is that evaluation should respect the amount of time, effort, and resources invested in the design and conduct of the exercise.  If the exercise isn’t fairly and accurately evaluated, it’s simply disrespectful to all that effort and those involved.  These concepts apply to both discussion based and operations based exercises.  Members of the exercise planning team should be considered as possible evaluators.

Exercise Evaluation Guides (EEGs) give evaluators guidance in their activity.  The HSEEP website provides a multitude of EEG templates which you should absolutely modify for your own use.  EEGs are provided for each core capability.  Just like planning templates (see my rant about the use of planning templates here), these are guides for you to reach success, but can not and should not be used without modification and customization to ensure that they meet your own specific needs.  EEGs are crafted hand in hand with exercise design to ensure that all bench marks and expected actions are accounted for.  The rating guide on the last page of each EEG is particularly handy to ensure consistency.  You may want to consider including a ‘recommendations’ section for each observation as well, which provides an opportunity for you gain insight from your own subject matter experts on how to address a particular area for improvement.  This will help when it comes to the After Action Report.  I also encourage evaluators to use and ICS form 200 (a blank sheet of paper!) to take some freeform notes.  EEGs, no matter how well designed, simply won’t capture everything.  Personally, when I evaluate, I’ll refer to the EEGs, but make most of my notes on a blank sheet(s) of paper or a steno pad.  I like to mark specific observations, times and timelines, and even inject numbers if I happen to catch them.  After the exercise, I then transcribe my notes into the EEG in the appropriate format.

Training of evaluators is important.  You may have some evaluators which are new and some that are experienced.  Everyone should go through a controller/evaluator briefing prior to the exercise.  Those with little experience might require some extra training, which isn’t a bad thing.  Be sure that both your C/E plan and the briefing identify clearly what you expect from evaluators, how they should conduct themselves, the measure of allowable interaction with players, how they should take notes, and when you expect those notes to be turned in.  The HSEEP website, referenced prior, also has templates for C/E briefings, C/E plans, etc.  You’ve likely heard the saying ‘garbage in – garbage out’… well, exercise evaluation is exactly like that.  If you don’t invest in good training and job aids (EEGs) for your evaluators, you will not get good evaluations!  The Lead Evaluator also needs to lead!  They should check in periodically with each evaluator throughout the exercise to see how things are going and give a quick look at their notes and/or EEGs.  Good feedback will help your evaluators provide better observations.

Finally, at the end of exercise play, I expect evaluators to help with the hotwash.  Sometimes hotwashes are conducted as a single session with all players.  In such sessions, the exercise director or lead evaluator should facilitate and evaluators should take notes.  If the hotwash is tiered, where each functional area will identify their top strengths and areas for improvement; each functional area is facilitated by its evaluator who should also take notes.  This may be followed by a plenary session if possible.  Be sure to have your hotwash strategy planned and identified in the C/E plan!

The After Action Report (AAR) should ideally be written by one or two people based on the EEGs and notes of evaluators.  It may be possible for some observations to be combined, helping to make the document both more concise and easier to read.  Templates are provided on the HSEEP website for AARs and Improvement Plans.  Improvement plans are nothing more than a matrix identifying who will be responsible for what improvement actions, who will assist, what the major benchmarks are to improvement, and when it should be accomplished.  As with the rest of the AAR, these should be drafted and provided prior to the AAR meeting for comment – of which there is usually a great deal.  Once all this is finalized, don’t let this be the last of it!  The implementation of improvements is where organizations often fail!  Rarely do organizations follow through on improvements, resulting in similar observations being noted in exercises and incidents for years after.  The AAR/IP must be championed by someone at an executive level, and coordinated on an ongoing basis by someone who is responsible for tracking progress, coordinating solutions to problems, and reporting progress back to the executive.  For most organizations and jurisdictions, I would suggest a quarterly meeting to review improvement planning progress.  After a few exercises, you may find yourself addressing improvements of several exercises in one meeting.  Accomplishments should be noted, with the opportunity to test the ‘fixes’ in future exercises.  All this feeds back into the strategy planning phase of exercise program management – which is where we started in this series.

This is the end of my ‘Managing an Exercise Program’ series.  I appreciate all those who have read and provided feedback.  Please continue to do so, and best of luck on your next exercise!

I’d be remiss if I didn’t mention that my company – Emergency Preparedness Solutions, LLC – can help you with exercise program planning and management; training and exercise plans; and the design, conduct, and evaluation of exercises; as well as training in HSEEP, exercise design, and exercise evaluation.  Give us a call or email and we’ll be happy to discuss what we can do to help your organization’s preparedness.

EPS logo with tag line - Be proactive, be prepared.

EPS logo with tag line – Be proactive, be prepared.

 

 

Managing an Exercise Program – Part 9: Conducting an Exercise

Tim Riecker, The Contrarian Emergency Manager Leave a comment

This post is part of a 10-part series on Managing an Exercise Program. In this series I provide some of my own lessons learned in the program and project management aspects of managing, designing, conducting, and evaluating Homeland Security Exercise and Evaluation Program (HSEEP) exercises. Your feedback is appreciated!

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

Managing an Exercise Program – Part 10: Evaluation and Improvement Planning

Image

Since writing Part 8, the new HSEEP guidance (April 2013) was published. Thankfully this update didn’t dictate wholesale changes in the process we’ve all come to be familiar with. There were some subtle changes, such as changing ‘planning conferences’ to ‘planning meetings’, and stressing the involvement of elected officials in the exercise planning process. Similarly, I’m thankful that certain changes weren’t made – one of which was the proposed elimination of terminology of ‘discussion based exercises’ and ‘operations based exercises’. These general identifiers have become commonplace amongst exercise designers for decades. Upon having the opportunity to review a draft version of the new HSEEP guidance, I was quite opposed to that change.

Obviously conducting the exercise is the most exciting part of the process. It’s where all of your hard work comes together. That said, this is where it can all fall apart if you haven’t prepared property in the first eight parts and if you don’t pay attention to detail. I’m breaking exercise conduct down into four smaller parts: Set up, briefings, play, debrief. Remember that my intent with these postings is not to be fully comprehensive, but to emphasize areas where I have learned or have seen others struggle. In all facets of emergency management we need to continue a culture of sharing if we are to be successful.

Set up

The preparations for your exercise should have been finalized in Part 8 where you addressed logistical requirements. You should know where your exercise venue is, how people will be seated/where they will be working, the location and support for a SimCell (if you’re having one), and even matters such as parking, food, restrooms, and audiovisual equipment. Additionally, set up includes having print material available for all participants. When setting up, consider the flow of people, quite literally from the road to the parking lot, from the parking lot to the facility, into the facility through security, sign in, and to their first location for the exercise (this may be their only destination or it may be a large briefing room). Having conducted exercises in locations not familiar to some participants as well as these exercises being larger than the facility’s parking lot can handle, you must ensure that invitation packages have specific directions on how to get to the parking area and how to get from the parking area to the venue. Be sure that signage is prominently displayed. It may even help to have staff directing people. Be sure to also let them know what manner of identification is required to enter the facility. Again, signage is often necessary to get people from the building entrance to check in location and then to the meeting location. Signage for the refreshment area and restrooms may also be needed.

Have everything set up for people before they arrive. Use table tents, if needed, to help identify seating, and have copies of all necessary materials at their seating location. Materials like SitMans and ExPlans have a lot of content, so it’s nice to have these materials available for the early-comers to being reading through.

Be sure to set up early and test EVERYTHING. Make sure AV equipment and communications equipment are tested.

Briefings

Depending on the type of exercise you are conducting, there can be a number of briefings needed to take place before the exercise begins. Some of these briefings may need to take place well in advance, even a day or more before the exercise, depending on availability of audiences for each of them.

The new HSEEP guidance includes a specific briefing for elected and appointed officials. This is something to usually conduct well in advance of the actual exercise to outline the agenda/timeline of the exercise and what will be expected of them. They will likely want to interface with any media and should know what to say about the exercise. They may also want to ‘kick off’ the exercise at the player briefing.

Controllers, evaluators, and SimCell personnel should receive a briefing to ensure they are familiar with the exercise and their rolls. They all need to know what is expected of them and of the players, how and when to interact with the players, and what may need to be reported to their respective leaders and/or to the exercise director. Evaluators will need to become familiar with their assigned EEGs and what the expectations are for evaluating. Additional time will need to be spent with SimCell personnel to familiarize them with the equipment being used, how the MSEL is structured, and how injects will be delivered. If any actors are being used, they will also need to be briefed on their rolls and what their expectations are. Ensure that everyone is familiar with the safety word in the event of the necessity to immediately stop the exercise.

If observers are expected at the exercise, have a briefing ready for them. I suggest treating them like VIPs when possible (most of them usually are). Don’t just let them walk around to figure things out for themselves. Give them tour guides who can take them throughout the venue and talk about what is happening. Schedule their arrival when possible, and consider having an elected or appointed official there to greet them. Keep in mind that some observers will be very high tier VIPs. Vigilant Guard exercises are often visited by the head of the National Guard Bureau – a four star General. These visits are fantastic and very much emphasize the importance of the exercise, but they can also be a little disruptive. These high-tier VIPs will often come with their own entourage and/or security. They will want a brief tour and will stop often to shake hands and have pictures taken. They may even want a break in the exercise to speak. Try to be aware of these expectations up front if at all possible. If not, just go with the flow and be flexible.

The media is another form of observer. All media should be scheduled. Be sure to make it worth their while, where they can catch some video/picture footage of interesting activities. Maps and wall displays make for great footage as well. Remember operational security! If something is sensitive or classified, it should not be anywhere where it can be seen. The media should also be provided a tour and an opportunity to get a statement from the elected and/or appointed officials. They may also want to interview players. I generally don’t allow this without preparation of the specific player to ensure that the right messages are delivered. Also, be sure to let players know during the player brief that there will be observers and media coming through and what is expected of them.

Lastly, the player briefing. This briefing will introduce players to the purpose of the exercise, the ground rules, the facility, timeline, controllers and evaluators, and expectations. Players should be briefed on how and when to interact with the SimCell if you have one. It should always be reinforced that the exercise is ‘no fault’, and that they, individually, are not being evaluated, rather it is the plans, policies, and procedures they are using that are being evaluated. Lastly, players are briefed on the scenario. Be sure that the ExPlan/SitMan has the detailed scenario for their reference.

Exercise Play

Finally the moment of truth arrives! Remember that the purpose of the exercise is to accomplish the objectives, however, know that you may not accomplish all the objectives in the timeframe you have. Being flexible, and knowing you have to be flexible, are two very important aspects of running an exercise. Regardless of how well we think we’ve written an exercise, the responses to our prompts are entirely up to the players. They may accomplish the objectives faster or slower than you expected. Likewise, they may struggle a bit. This is where good exercise control comes in. Controllers/facilitators should observe the tempo of the exercise. Is everyone engaged? Are they needlessly overwhelmed and frustrated? Are they bored? Are they not following through on activities? All these observations should be reported back to the exercise director and SimCell so the tempo can be moderated.

Be sure to have contingencies in the event that players do or request the unexpected. Additional injects that are held back are always a good idea. You may need them to prompt certain activities in the event that players do not take the initiative to do so. Players may also ask questions or make requests of the SimCell that aren’t expected. While we can’t anticipate every need, we can be prepared for them. Have copies of plans, policies, procedures, and maps available to the SimCell. They may have to take some time to research and come up with an answer then get back to the individual. The SimCell manager should be smart when situations like this arise, however. They should always consider if a potentially inaccurate but reasonable answer is acceptable or if the answer must be completely accurate… or even if the answer is required for exercise play at all. There is no sense having SimCell personnel research an answer if the answer doesn’t much matter within the scope of the exercise.

Multi-day exercises may require the exercise staff to meet at the end of every day for a mini hotwash and to evaluate how the exercise is progressing. Where are the players in respect to where you expected them to be in your design of the exercise? Do you need to retool anything in the MSEL?

On large exercises, it helps for the exercise director to have an aide-de-camp, or assistant. Just like any individual in charge of a large operation, the exercise director is having their attention pulled in many directions and may simply not have the time to address everything, especially in large or multi-venue exercises. Having an assistant is also a great way to train up and coming exercise staff. Certainly consider the use of portable radios to help facilitate communication as well. The exercise director will spend most of their time managing and trouble shooting. They may also have to address some VIP concerns. Be sure to walk around to get a good sense for the exercise as a whole. Are evaluators properly positioned? Are controllers present and visible? Is the SimCell responding to questions adequately? Are players engaged and challenged? Perhaps most importantly, is the boss (the elected/appointed official) happy?

Evaluation is such an important aspect of exercise that I will cover it in its own section – Part 10.

Debriefs

Two significant debriefing activities should take place immediately following the exercise: 1) a player hotwash, 2) an exercise staff debriefing. Players should be led through a facilitated hotwash, reviewing the objectives of the exercise, with evaluators capturing their responses to if/how the objectives were accomplished. Encourage and capture responses both in the positive and the negative: i.e. what did we do well and what do we need to improve upon?

After the player hotwash, a similar process should take place with all exercise staff – controllers, evaluators, and SimCell. I like to not only capture their impressions of if/how objectives were accomplished, but I also like to discuss the conduct of the exercise, again capturing what went well and what needed to be improved upon.