Tag HSEEP

The Need for More Scenario-Based Learning

Tim Riecker, The Contrarian Emergency Manager 1 Comment

Think back through all the courses you’ve taken.  It’s a lot – I know.  What ones stand out the most?  I’m willing to be they are the ones that were the most engaging.  Not only did you enjoy them, but you learned a lot from them and still remember quite a bit of it.

It’s no secret that training adults can be challenging.  Training professionals in emergency services is certainly no different.  The challenges are even greater as the number of required training courses continually increase, requiring more and more ass-in-chair time every year for responders and other professions.  A great deal of training programs we see out there still seem to be holding out for the sake of traditional delivery styles, much to the detriment of our learners.  Why?  Designing traditional lecture-based learning is easy to do!  Figure out what people need to learn, develop content, slap together some PowerPoint, and voila!  Hell, even I’m guilty.

The fact of the matter is that we all know this is wrong.  Yes, it’s easy to do on our end, but the value and impact of the learning is pretty low.  People don’t want to be lectured to for hours on end.  We know that learning is most effective when we mix things up and when we increase interaction.  One of the best ways of engaging learners effectively is through scenario-based learning.

Now I’m not just talking about using a scenario at the end of the course to see if people can apply what they’ve learned over the past two days.  Yes, scenarios can be used as a test of sorts, but they are most effective for actual learning.  So when should you use scenarios?  Why not start the course with one?  It immediately gets people thinking, which is a good thing especially with an 8 am start time to the course.   If you use a lot of scenarios in a course, can they all be related?  Sure.  Maybe.  Maybe not.  It all depends on what the purpose of the scenario is.  In training responders, threading a common scenario through a course is usually helpful.  Scenarios can get complicated when we need to establish a common understanding of what is going on, where it is, what resources are available, etc.  As such, it helps to use the same foundational scenario throughout the course (or at least regularly revisit it), and continue to introduce new problems or a different focus based upon the path of the training.  Using a common foundational scenario saves time so you don’t have to start anew introducing all new information each time and it keeps learners comfortable.  That said, it may occasionally be valuable to change things up a bit.

Do you need to use HSEEP to develop course scenarios?  No.  While these aren’t exercises in the strictest sense, we can benefit considerably from many of the principles and concepts of HSEEP.  Develop what you need to give learners the information they need to participate and the information you and/or other instructors need to properly facilitate and evaluate.

Adult learners like to be challenged.  Lecturing them for hours on end will only challenge their ability to not fall asleep – which may only be accomplished by their challenge for a new high score on the new app they just put on their phone.  The best way to challenge adult learners is to give them problems to solve.  A well written scenario will help introduce these problems in a framework which is both familiar and challenging to them.  Depending on how the scenario is provided, such as a compelling background story or use of video, learners will establish an emotional connection to the scenario which prompts a visceral desire to solve these problems.  Even one scenario is powerful and can prompt a lot of interaction.  It can prompt individual responses to questions, group discussions, and group collaborations.

Finally, don’t forget to evaluate both your learners and the scenario itself.  At the conclusion of each scenario conduct a hotwash and feedback session with learners to discuss what they accomplished and possible areas for improvement.  Also be sure to gain feedback from them and other instructors on how well the scenario worked and what can be improved upon.

Just like any other aspect of instructional design, the integration of scenarios can be time consuming but it’s an investment that will pay off.  To capitalize on the value of your scenarios, make sure the activities and expected outcomes of each scenario are associated with the learning objectives of the course and engage learners to the proper degree (i.e. the proper level of Bloom’s Taxonomy).  Yes, scenarios also take a fair amount of class time to execute.  That time needs to be well accounted for in your instructional design and course planning.  However, if properly designed, learners can learn just as much content if not more through interactive scenarios as compared to lecture-based training.

What types of scenarios have you integrated into courses?  How did learners respond to them?  How can we do a better job of integrating more scenario-based learning into our courses?

Need help designing scenario-based learning?  Let EPS help!

© 2015 – Timothy Riecker

Emergency Preparedness Solutions, LLC

WWW.EPSLLC.BIZ

Effectiveness and Efficiency in Incident Management – Resource Tracking

Tim Riecker, The Contrarian Emergency Manager 4 Comments
Incident Check In

Incident Check In

I recently took part in the management of an exercise in which a Type 3 incident management team (IMT) was among the players.  As part of their initial set up they immediately recognized the importance of checking in and tracking resources.  This is an activity which is often overlooked at the onset of an incident and is a royal pain to catch up on once the need is realized.  There were a few things which they could have improved upon, though, which seriously impacted their effectiveness and efficiency.

  1. They spent time checking in each vehicle as equipment. Not every vehicle needs to be tracked in an incident.  Generally, the sedan, pick up, or SUV you come in on isn’t special enough that it requires tracking.  Huge waste of time, people, and effort.  Consider the nature and capability of the equipment that is coming through your access point.  Is it a specialized resource?  Will it be applied tactically?  Will it be supporting logistical needs?  Is it rented or leased?  These are the conditions that should be considered when deciding what equipment to track.
  2. They marked equipment using bottled shoe polish. Not a bad idea, except it rained all week, and within hours of application most of the markings couldn’t be read.  Windshield markers, similar to what car dealerships use, are cost effective, waterproof, and clean off easily with mild window cleaners.
  3. Equipment that was checked in was never logged in detail. What’s the difference between E-01234 and E-01235?  We will never know as no descriptions were entered into their tracking system.
  4. As vehicles flowed in to the staging area, people will directed to check in at the command post. This is obviously excellent, except to get to the command post people had to pass by the main access to the incident site.  This meant that many people did not check in as directed.  They got distracted by the incident and associated response activity and never made it to the command post to check in.  This severely impacted the effectiveness of accountability.

Sometimes people would try to explain these things away by saying “It’s just an exercise”, but exercises are an opportunity to do things the right way, not skimp and cut corners.  While their intent was good, their process and results were quite poor.  If we are supposed to train the way we fight, as they say, this team has a ways to go to be more effective with resource accountability.  On the surface resource tracking looks easy… but it’s not.  There is a lot of complexity, variables, and attention to detail that must all work together well in order to be successful.  The Resource Unit Leader has one of the hardest jobs in the Incident Command System.

Being who I am, I’m left wondering why this all happened.  I have little choice but to blame poor planning and training.  Planning is to blame for a lack of clear procedures, guidance, and decision models.  The training which people receive tends to be just as vague.  By now, most, if not all of you are familiar with my opinions on the current ICS training.  While the referenced article does not go into the IMT/position training curricula, from what I recall of the courses I’ve taken, there are certain things taken for granted.  It’s easy to put an item on a checklist that says ‘Establish check in’.  OK… how?  Where?  When?  What?  Why?  The answer to those questions, or guidance to help answer those questions, should be provided through training.  Let’s tell people not only why check in is important, but what people and resources should be checked in, where to establish check in (what to look for and what to avoid), etc.  Once we’ve trained people on it, let’s provide job aids… not just the ICS forms, but job aids that will actually help people do their jobs.  While it may seem like minutia and unnecessary detail, keep in mind that we are training people to operate in austere and chaotic environments which they are trying to establish order over and only do these activities on rare occasion.  Those conditions signal the need for detailed training and job aids to support sustained performance and limit the degradation of the training they received.

Bottom line – let’s take a step back, fix what we have to based upon what we’ve learned, and proceed forward so we can operate more effectively and efficiently.

Thoughts and comments are always appreciated.  What have you learned or observed from incidents or exercises that needs to be addressed foundationally?

© 2015 – Timothy Riecker

Emergency Preparedness Solutions, LLC

WWW.EPSLLC.BIZ

Emergency Managers Need to be More Like Engineers and Less Like Shopkeepers

Tim Riecker, The Contrarian Emergency Manager Leave a comment

I was inspired by this short (~1 minute) video from TrainingJournal.com.  In the presenter’s brief but pointed message, he describes many trainers as being akin to shopkeepers, providing organizations often times with rote solutions just as a shopkeeper will pull a product off their shelves. He goes on to say that this these solutions are usually effective, but only for a limited duration.  He offers, instead, that trainers need to be more like engineers, examining every facet of a problem and constructing lasting solutions.  As an experienced trainer and proponent of a detailed root cause analysis, I couldn’t agree more, but as I readied myself to write a post about the implications of this on training, my mind carried this metaphor to many of our practices in emergency management.

Consider how often we quickly dismiss identified gaps with an assumed solution.  Write a plan, conduct a training, install a bigger culvert.  Those are usually our solutions to an identified problem.  Are they wrong?  No – we’re correct more often than not.  Are these lasting solutions?  Rarely!  How often does the problem rear its head again within a relatively short span of time?  How do we address the re-occurrence?  As shop keepers we simply pull another solution off the shelf.  Can we do better?

The things we do in emergency management are often based upon best and current practices.  We address problems through the prevalent way of dealing with such things industry-wide.  Emergency management has a great community of practice.  I’ve mentioned in several previous blog posts the spirit of sharing we have and the benefits we see come of that.  It doesn’t seem often, though, that we engage in an industry-wide groupthink to solve various problems.  We use and adapt ideas of individuals and small groups, we see a steady and determined progression of the practices within our progression, but we rarely see ‘game changing’ ideas that revolutionize how prevent, prepare for, respond to, or recover from disasters.  Why is this?

Perhaps we need a greater collective voice locally, where practitioners are dealing with the problems directly?  Our methods of practice in emergency management are generally driven by the federal government (THIRA, NIMS, HSEEP, etc.).  I’m not saying any of these are bad – in fact they are excellent standards that we need to continue to refine and apply, but it’s generally not the federal government that is dealing directly with the constant flow of issues being dealt with at a state, and even more so, a local level.  We need to follow that metaphor of being engineers to apply more permanent solutions to these problems.  We need to create, innovate, and problem solve. Or do we?

Necessity, as they say, is the mother of invention.  We often miss the necessity of improving because we have current, functional solutions – we have things that work.  So why fix it if it’s not broken?  I say we can do better.  The realization of the need for lasting solutions is the necessity we need.  If the solutions we have on the shelf don’t work for us 100%, let’s figure out a better way.

I don’t know what or how, but I’m sure that as a community we can identify needs and prioritize what must be addressed.  Given the right people, time, and maybe a bit of money, we can be innovative and effect some lasting change.

I’d love to hear what others think on this topic.

© 2015 – Timothy Riecker

Emergency Preparedness Solutions, LLC

www.epsllc.biz

Ebola Reflection Measures our Preparedness

Tim Riecker, The Contrarian Emergency Manager Leave a comment

NBC News recently posted an article citing a report published by the Presidential Commission for the Study of Bioethical Issues.  The link provided to the report in the NBC News article doesn’t seem to work, but I’ve found what I believe to be the report here.  The focus of the report is on the ethical challenges faced by the US in responding to this issue.  The report summarizes a variety of ethics related concerns and considerations in this ongoing response and paints a fairly accurate picture of our failures and what needs to be addressed – at least within the topics it discusses.

Photo credit: Forbes.com

Photo credit: Forbes.com

As you might expect from a report on bioethics, it is very public health focused.  While they do make mention of very public health centered topics such as clinical drug trials, they do cover topics which are much more broadly rooted in emergency management and homeland security, such as community and responder education, and ethics associated with quarantine.  This report, while fairly focused, opens a virtual Pandora’s box of issues related to our domestic response to Ebola.

Needless to say, our collective response to this matter was horrible.  Public health policy and guidance was a moving target for weeks; responders were ill prepared to handle potentially infected persons; and the collective of society, politicians, and public safety were largely reluctant to deal with matters of quarantine much less prepared for it.  Was this our first consideration of something like Ebola?  Of course not.  Didn’t we have preparations in place?  Kind of.

Back in the late 90s, pushed mostly by the Nunn-Lugar-Domenici act of 1996, preparedness efforts for state and local responders were funded to enhance our capabilities in dealing with WMD incidents.  Several years later, after 9/11 and the anthrax attacks, another surge of funding was pushed down to state and local governments from HHS/CDC for the purpose of bolstering public health preparedness including preparedness for WMD/weaponized biologicals and naturally occurring pandemic incidents.  These two programs alone, not including other related funding, fostered the creation of plans and organizations to support them, purchased entire stockpiles of equipment and supplies, trained tens of thousands of responders and public health workers, and encouraged exercises across the nation to test capabilities (it was actually these exercises which largely influenced the creation of what we now know as HSEEP).  A lot of good came from these programs, but when suddenly tested with the reality of implementation we seemed to fall apart.  Why?

First of all, many of these preparedness efforts occurred between 10 and nearly 20 years ago.  Many of the people initially trained in these programs have since retired from public service with their organizations losing a great deal of institutional knowledge.  While training programs have continued and still exist, there have been systemic gaps in tying this type of training to other preparedness efforts (planning, policy, equipment, etc.).  Some equipment purchased near the beginning of these programs has likely been retired as well.  Much of it still exists, but has been brought into the fold of other applications, such as HazMat – which is certainly appropriate, but yet again we see gaps, this time our ability to readily utilize equipment specifically for public health threats.

In my opinion many of the planning efforts we saw after 9/11 were misguided.  This started with the people who were doing the planning.  Many health organizations emphasized health care experience for these positions instead of EM or planning experience – which was their main function.  Certainly health care knowledge had some importance, but that could be supplemented through a good advisory committee (EM after all is a team effort).  Exacerbating poor hiring decisions was a lack of investing in the people that were hired.  Many organizations expected them to churn out pandemic influenza plans in short order, with little/no training on the planning process or integral systems that must be considered.  Further, much of the planning had been done in a vacuum – that is, it had been performed with little/no input from other stakeholders.  I had reviewed many of these plans, finding things such as inappropriate applications of ICS and wild assumptions of resource availability.  In no way were these plans realistic or applicable.

There were many exercises performed and most of them had great value.  The problem is that there were a lot of assumptions in these exercises and policy decisions made in the exercises were rarely challenged as they would be in reality.  The US Ebola response brought this all to light as decisions such as quarantine were being handled at the governor level and under significant controversy.  So in this recent response I ask why were decisions delayed and deferred to higher authorities?  Why were adequate local/regional plans not in place to address the care and handling of potentially infected persons?  Why did procedural issues take weeks to resolve?  The simple answer is that there was a lack of proper preparedness.

Back in October of last year, when Ebola was emerging in the US, I posted an article titled Preparing for Ebola – and Whatever Else May Come.  The article still has a great deal of relevancy since I’ve seen very little preparedness for future occurrences – only a harried response to the most recent incident.  There have certainly been a great deal of policies and procedures assembled for the current Ebola issue, but these have a feeling of being temporary, throwaway, or single-use documents, applied only for this instance instead of durable and lasting plans.  Many will keep them ‘on the books’, only to find that their hasty assembly wasn’t comprehensive enough for the next occurrence.  Emergency management and homeland security professionals, public health leaders, and elected and appointed officials need to take a step back and re-look at out preparedness efforts – especially in regard to public health issues.  While we should learn from what we have experienced, we also need to think comprehensively about what is needed.  Well considered policies need to be put in place, supported by our laws and responsibilities to protect the public while also considering protection of civil liberties.  Other preparedness efforts such as planning, training, and exercising need to continue to occur but must have their connections strengthened and intentional.  Exercises need to test plans and policies and challenge decision makers who are certainly making difficult decisions that may include ethics and moral issues in the consideration of caring for few while protecting the greater society.

These are not easy things to be done – which supports the need to work on them now, when we aren’t facing an imminent disaster.  While Ebola certainly wasn’t a health care crisis and there were a lot of things done right, there is always room for improvement – especially when the next biological occurrence could be a crisis.

What have you and your organization learned from the Ebola response?  What gaps have you addressed?  What do you feel still needs to be addressed?

© 2015 – Timothy Riecker

Exercising Foundational Skills with Unorthodox Scenarios

Tim Riecker, The Contrarian Emergency Manager Leave a comment

Does the scenario of an exercise activity really matter?  Can we use a zombie scenario to exercise evacuation and sheltering?  Can we use a holiday food distribution to the needy to practice our POD (point of distribution) plan?  Do scenarios always have to be realistic or related to our jurisdiction’s hazards?

I’m a foodie.  As such I find myself occasionally watching shows like Cutthroat Kitchen and Chopped.  These are fun shows that strike a balance of cooking with game shows, including the cash prize in the end.  The competitors are legitimate cooks, some trained in culinary schools, some successful in their careers and earning the title of ‘chef’.  The competitors are given, on the spot, either a dish to create (Cutthroat Kitchen) or a box full of ingredients which must all be incorporated into a dish (Chopped), using a kitchen and pantry generally unfamiliar to them, within a relatively short amount of time – and make it better than their competitors.  Is competing on these shows anything like running a professional kitchen?  Hell no.  Does it make them better cooks?  From interviews I’ve heard, the answer is yes.

Can we recreate this in emergency management?  Of course we can, and we should.  How would this help emergency managers and other public safety professionals?  Recall that within the exercise design component of the HSEEP process the Core Capabilities to be focused on and the objectives to be tested are selected prior to determining the scenario.  This tells us that the activities to be performed are more important than the scenario in which they will be performed.  In these cooking competitions, the participants must fall back on their foundational skills to be successful.  It’s those foundational skills and the activities which they foster that we evaluate in our exercises.

Certainly a scenario has some importance.  It provides context, allowing the participants to get their head into what they are doing.  A scenario can be different, even a bit silly or fantastical (alien invasion, anyone?), but it still has to correlate to the objectives of the exercise; i.e. there must be a compelling reason to perform mass prophylaxis or to evacuate an area.  That said, the scenario is simply a vehicle to get our participants to perform what we intend to test.  Don’t we always tell our participants to not fight the scenario?  Well if it’s something they’ve never before experienced, they have little ground to stand on.

Another benefit to using an unfamiliar or alternate scenario is getting participants to break from the routine and face unexpected and new challenges.  What if digital communications fail?  What if they have to relocate to an alternate EOC? What if that alternate facility is likewise compromised?  Consider using the scenario to remove a critical resource from use.  How will the participants overcome this new problem?  In Cutthroat Kitchen, participants are faced with unseemly injects to their food preparation, such as replacing all cooking utensils with a Swiss Army knife or only being able to cook using a microwave.  Some of your participants may balk at such occurrences, but emergency management is about managing the unknown, the unfortunate, and the unexpected.

Regardless of the measure of reality we choose to base our exercises on, the scenarios we develop are really another level of fiction to help facilitate exercise participation.  Yes, often times we want to test hazard specific plans (a zombie apocalypse exercise can not replace the need of a hurricane exercise), but if the scenario itself doesn’t matter, consider using something ‘outside the box’.  Routine makes us complacent and complacency is very dangerous in emergency management.  We must always expect the unexpected and continually have the mindset to improvise, adapt, and overcome.

© 2014 – Timothy Riecker

Are You Inviting the Right People to Your Exercises?

Tim Riecker, The Contrarian Emergency Manager Leave a comment

A couple of days ago I started reading Rumsfeld’s Rules – Leadership Lessons in Business, Politics, War, and Life.  Hopefully you have some familiarity with Donald Rumsfeld – the man was a naval aviator, US Congressman, aide to four US presidents, corporate CEO, and is the only person to ever serve as Secretary of Defense twice.  Politics aside, Mr. Rumsfeld has had quite a prolific career.  Throughout this career he has assembled a variety of mantra, proverbs, and sayings which he has used to help guide his career and serve as advice to others.

Early in the book, Mr. Rumsfeld talks about meetings.  What he mentions struck me as solid guidance not only for meetings but also for exercises.  He says “There is a balance that needs to be struck in determining who to invite to a meeting.  You want those who need to be there to contribute substance to the discussion.  But it can be useful to have people who may not be in a position to directly offer substantive input but will benefit from hearing how and why certain decision are being reached.”  Very often exercise offer great opportunity for people to learn – not only the participants but ‘shadowers’ as well.

Mr. Rumsfeld continues on to say “Including a range of people can also ensure that a variety of perspectives will be considered and help identify gaps in information and views.”  Consider that we build, conduct, and evaluate exercises primarily to test plans, polices, and procedures.  This testing is best performed by a spectrum of individuals giving different ideas and perspectives.  Someone may interpret a policy in a completely different way or have an approach to a problem that hasn’t been considered prior.  These fresh ideas, even if flawed, should be brought out into the open for discussion and consideration.

If you’ve followed this blog for any amount of time, you probably know that I prefer smaller meetings and have stressed that participants in exercises should be of a manageable number.  As Mr. Rumsfeld says, there is a balance that must be struck.  You want to be inclusive, but large numbers lend themselves to over-discussion and tangents.  For meetings do you expect the person to add value?  Should they be there given their area of responsibility?  Similarly in exercises is the individual associated with the objectives of the exercise?  (Recall that in exercises we should always reflect on the objectives throughout the entire design process).  When we add more participants to an exercise we need to ensure that they have something to participate in, so injects must be written for them and their activities must be evaluated.

A few years back my team was designing a table top exercise as a lead-in to a significant full-scale exercise.  We did not want to start the full-scale with the initial response, as so many often times are, as the objectives of that exercise were to test the extended response and to examine issues beyond the initial response.  That said, we felt it not fair for us to design such a large exercise by dictating what the first responders would do in the first 48 hours, rather we wanted them to tell us themselves.  So we designed a table top exercise to provide us with their actions both ‘boots on the ground’ as well as policy-level including emergency declarations, evacuation areas, and mutual aid requests.  We were quite fortunate that the design process for the exercise as a whole was very well received and many agencies wanted to participate – from federal, state, county, and local jurisdictions.  The exercise was centered on the state capital, which tends to garner even more attention and participation and included a scenario that most agencies have not participated in prior.  Needless to say, we had a lot of interest.  Nearly every agency invited to the table top wanted to bring not one or two additional people but often times three or four.  We discussed this matter with a few of the key agencies, asking of these were needed participants or observers.  The answer we got was that they were both.  Because of the technical nature of the incident, many agencies realized they needed their main spokesperson supported by one or more technical experts.  We realized this was a fair and reasonable request, but we still needed to figure out how to accommodate them all!

We decided to permit each representative to have a ‘second chair’ – someone seated directly behind them who could advise on technical matters.  Additional specialists were available to them in an adjacent room, which had the discussion live broadcast to them via closed circuit television.  Specialists could be ‘swapped out’ at any time based on the needs of the discussion.  This solution worked well for the exercise, keeping the number of direct participants manageable and meeting the needs of participants to have their specialists available to advise on technical matters – which truly helped inform their decisions and ultimately the outcome of the exercise.

Sometimes, though, you have to say ‘no’.  Realize that as an exercise designer you MUST set a firm deadline on additional participants.  Participants that are added late can set your design team back significantly by needing to ensure that they are written into the exercise and have sufficient activity to make their participation worth while for both them as well as the exercise as a whole – which can be particularly challenging if they are from a different jurisdiction or discipline altogether.  I’ve had to turn down several interested parties and while it’s often difficult to say no, it’s often for the better – and your design team will respect you for it.

What thoughts do you have on ‘right sizing’ your meetings and exercises?  Is there certain guidance that you use?

©2014 Timothy Riecker

Managing an Exercise Program – Part 10: Evalutation and Improvement Planning

Tim Riecker, The Contrarian Emergency Manager 1 Comment
New HSEEP graphic

New HSEEP graphic

For the previous parts in my series Managing an Exercise Program, please see below:

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

 

While conducting an exercise is often the most exciting part of the exercise cycle, evaluation and improvement planning lead us to the real reasons why we conduct exercises – to test plans, policies, and procedures.  Evaluation doesn’t just happen.  It must be a deliberate, planned activity within the design and conduct of an exercise.  There are key activities through the design and development of an exercise which link directly to evaluation – starting with the identification of core capabilities to be tested and exercise objectives to be accomplished.  These core capabilities and objectives should lead us directly back to specific plans, policies, and procedures which will be exercised and can help us determine the evaluation methodology and approach.  As we further develop our exercise, be it discussion based or operations based, the decisions we make influence evaluation and the resources needed for it.

When exercise planning begins, we identify staff to fill key roles – including exercise director, simcell manager, lead controller, and lead evaluator.  The lead evaluator doesn’t necessarily have to be involved in all aspects of exercise design, but they need to be informed of key points and should be reviewing draft materials (i.e. the explan or sitman) to become familiar with the details of the exercise and to help them assemble their portion of the control and evaluation plan.  The lead evaluator should ideally be present at exercise planning meetings to become even more familiar with the details, the people, and the facility(s) of exercise play.

Staffing evaluation has a number of options.  Often times evaluators will be selected based on matching their individual areas of expertise with the functional areas of the exercise (i.e. fire, EMS, public health, EOC management, shelter operations, etc.).  You may have a need for more than one evaluator per functional area, particularly if that function is a major component of the exercise and there is a lot to observe.  Remember that if there are multiple exercise venues, they each need to have evaluators assigned who can observe all critical areas.  The bottom line is that evaluation should respect the amount of time, effort, and resources invested in the design and conduct of the exercise.  If the exercise isn’t fairly and accurately evaluated, it’s simply disrespectful to all that effort and those involved.  These concepts apply to both discussion based and operations based exercises.  Members of the exercise planning team should be considered as possible evaluators.

Exercise Evaluation Guides (EEGs) give evaluators guidance in their activity.  The HSEEP website provides a multitude of EEG templates which you should absolutely modify for your own use.  EEGs are provided for each core capability.  Just like planning templates (see my rant about the use of planning templates here), these are guides for you to reach success, but can not and should not be used without modification and customization to ensure that they meet your own specific needs.  EEGs are crafted hand in hand with exercise design to ensure that all bench marks and expected actions are accounted for.  The rating guide on the last page of each EEG is particularly handy to ensure consistency.  You may want to consider including a ‘recommendations’ section for each observation as well, which provides an opportunity for you gain insight from your own subject matter experts on how to address a particular area for improvement.  This will help when it comes to the After Action Report.  I also encourage evaluators to use and ICS form 200 (a blank sheet of paper!) to take some freeform notes.  EEGs, no matter how well designed, simply won’t capture everything.  Personally, when I evaluate, I’ll refer to the EEGs, but make most of my notes on a blank sheet(s) of paper or a steno pad.  I like to mark specific observations, times and timelines, and even inject numbers if I happen to catch them.  After the exercise, I then transcribe my notes into the EEG in the appropriate format.

Training of evaluators is important.  You may have some evaluators which are new and some that are experienced.  Everyone should go through a controller/evaluator briefing prior to the exercise.  Those with little experience might require some extra training, which isn’t a bad thing.  Be sure that both your C/E plan and the briefing identify clearly what you expect from evaluators, how they should conduct themselves, the measure of allowable interaction with players, how they should take notes, and when you expect those notes to be turned in.  The HSEEP website, referenced prior, also has templates for C/E briefings, C/E plans, etc.  You’ve likely heard the saying ‘garbage in – garbage out’… well, exercise evaluation is exactly like that.  If you don’t invest in good training and job aids (EEGs) for your evaluators, you will not get good evaluations!  The Lead Evaluator also needs to lead!  They should check in periodically with each evaluator throughout the exercise to see how things are going and give a quick look at their notes and/or EEGs.  Good feedback will help your evaluators provide better observations.

Finally, at the end of exercise play, I expect evaluators to help with the hotwash.  Sometimes hotwashes are conducted as a single session with all players.  In such sessions, the exercise director or lead evaluator should facilitate and evaluators should take notes.  If the hotwash is tiered, where each functional area will identify their top strengths and areas for improvement; each functional area is facilitated by its evaluator who should also take notes.  This may be followed by a plenary session if possible.  Be sure to have your hotwash strategy planned and identified in the C/E plan!

The After Action Report (AAR) should ideally be written by one or two people based on the EEGs and notes of evaluators.  It may be possible for some observations to be combined, helping to make the document both more concise and easier to read.  Templates are provided on the HSEEP website for AARs and Improvement Plans.  Improvement plans are nothing more than a matrix identifying who will be responsible for what improvement actions, who will assist, what the major benchmarks are to improvement, and when it should be accomplished.  As with the rest of the AAR, these should be drafted and provided prior to the AAR meeting for comment – of which there is usually a great deal.  Once all this is finalized, don’t let this be the last of it!  The implementation of improvements is where organizations often fail!  Rarely do organizations follow through on improvements, resulting in similar observations being noted in exercises and incidents for years after.  The AAR/IP must be championed by someone at an executive level, and coordinated on an ongoing basis by someone who is responsible for tracking progress, coordinating solutions to problems, and reporting progress back to the executive.  For most organizations and jurisdictions, I would suggest a quarterly meeting to review improvement planning progress.  After a few exercises, you may find yourself addressing improvements of several exercises in one meeting.  Accomplishments should be noted, with the opportunity to test the ‘fixes’ in future exercises.  All this feeds back into the strategy planning phase of exercise program management – which is where we started in this series.

This is the end of my ‘Managing an Exercise Program’ series.  I appreciate all those who have read and provided feedback.  Please continue to do so, and best of luck on your next exercise!

I’d be remiss if I didn’t mention that my company – Emergency Preparedness Solutions, LLC – can help you with exercise program planning and management; training and exercise plans; and the design, conduct, and evaluation of exercises; as well as training in HSEEP, exercise design, and exercise evaluation.  Give us a call or email and we’ll be happy to discuss what we can do to help your organization’s preparedness.

EPS logo with tag line - Be proactive, be prepared.

EPS logo with tag line – Be proactive, be prepared.

 

 

Managing an Exercise Program – Part 9: Conducting an Exercise

Tim Riecker, The Contrarian Emergency Manager Leave a comment

This post is part of a 10-part series on Managing an Exercise Program. In this series I provide some of my own lessons learned in the program and project management aspects of managing, designing, conducting, and evaluating Homeland Security Exercise and Evaluation Program (HSEEP) exercises. Your feedback is appreciated!

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

Managing an Exercise Program – Part 10: Evaluation and Improvement Planning

Image

Since writing Part 8, the new HSEEP guidance (April 2013) was published. Thankfully this update didn’t dictate wholesale changes in the process we’ve all come to be familiar with. There were some subtle changes, such as changing ‘planning conferences’ to ‘planning meetings’, and stressing the involvement of elected officials in the exercise planning process. Similarly, I’m thankful that certain changes weren’t made – one of which was the proposed elimination of terminology of ‘discussion based exercises’ and ‘operations based exercises’. These general identifiers have become commonplace amongst exercise designers for decades. Upon having the opportunity to review a draft version of the new HSEEP guidance, I was quite opposed to that change.

Obviously conducting the exercise is the most exciting part of the process. It’s where all of your hard work comes together. That said, this is where it can all fall apart if you haven’t prepared property in the first eight parts and if you don’t pay attention to detail. I’m breaking exercise conduct down into four smaller parts: Set up, briefings, play, debrief. Remember that my intent with these postings is not to be fully comprehensive, but to emphasize areas where I have learned or have seen others struggle. In all facets of emergency management we need to continue a culture of sharing if we are to be successful.

Set up

The preparations for your exercise should have been finalized in Part 8 where you addressed logistical requirements. You should know where your exercise venue is, how people will be seated/where they will be working, the location and support for a SimCell (if you’re having one), and even matters such as parking, food, restrooms, and audiovisual equipment. Additionally, set up includes having print material available for all participants. When setting up, consider the flow of people, quite literally from the road to the parking lot, from the parking lot to the facility, into the facility through security, sign in, and to their first location for the exercise (this may be their only destination or it may be a large briefing room). Having conducted exercises in locations not familiar to some participants as well as these exercises being larger than the facility’s parking lot can handle, you must ensure that invitation packages have specific directions on how to get to the parking area and how to get from the parking area to the venue. Be sure that signage is prominently displayed. It may even help to have staff directing people. Be sure to also let them know what manner of identification is required to enter the facility. Again, signage is often necessary to get people from the building entrance to check in location and then to the meeting location. Signage for the refreshment area and restrooms may also be needed.

Have everything set up for people before they arrive. Use table tents, if needed, to help identify seating, and have copies of all necessary materials at their seating location. Materials like SitMans and ExPlans have a lot of content, so it’s nice to have these materials available for the early-comers to being reading through.

Be sure to set up early and test EVERYTHING. Make sure AV equipment and communications equipment are tested.

Briefings

Depending on the type of exercise you are conducting, there can be a number of briefings needed to take place before the exercise begins. Some of these briefings may need to take place well in advance, even a day or more before the exercise, depending on availability of audiences for each of them.

The new HSEEP guidance includes a specific briefing for elected and appointed officials. This is something to usually conduct well in advance of the actual exercise to outline the agenda/timeline of the exercise and what will be expected of them. They will likely want to interface with any media and should know what to say about the exercise. They may also want to ‘kick off’ the exercise at the player briefing.

Controllers, evaluators, and SimCell personnel should receive a briefing to ensure they are familiar with the exercise and their rolls. They all need to know what is expected of them and of the players, how and when to interact with the players, and what may need to be reported to their respective leaders and/or to the exercise director. Evaluators will need to become familiar with their assigned EEGs and what the expectations are for evaluating. Additional time will need to be spent with SimCell personnel to familiarize them with the equipment being used, how the MSEL is structured, and how injects will be delivered. If any actors are being used, they will also need to be briefed on their rolls and what their expectations are. Ensure that everyone is familiar with the safety word in the event of the necessity to immediately stop the exercise.

If observers are expected at the exercise, have a briefing ready for them. I suggest treating them like VIPs when possible (most of them usually are). Don’t just let them walk around to figure things out for themselves. Give them tour guides who can take them throughout the venue and talk about what is happening. Schedule their arrival when possible, and consider having an elected or appointed official there to greet them. Keep in mind that some observers will be very high tier VIPs. Vigilant Guard exercises are often visited by the head of the National Guard Bureau – a four star General. These visits are fantastic and very much emphasize the importance of the exercise, but they can also be a little disruptive. These high-tier VIPs will often come with their own entourage and/or security. They will want a brief tour and will stop often to shake hands and have pictures taken. They may even want a break in the exercise to speak. Try to be aware of these expectations up front if at all possible. If not, just go with the flow and be flexible.

The media is another form of observer. All media should be scheduled. Be sure to make it worth their while, where they can catch some video/picture footage of interesting activities. Maps and wall displays make for great footage as well. Remember operational security! If something is sensitive or classified, it should not be anywhere where it can be seen. The media should also be provided a tour and an opportunity to get a statement from the elected and/or appointed officials. They may also want to interview players. I generally don’t allow this without preparation of the specific player to ensure that the right messages are delivered. Also, be sure to let players know during the player brief that there will be observers and media coming through and what is expected of them.

Lastly, the player briefing. This briefing will introduce players to the purpose of the exercise, the ground rules, the facility, timeline, controllers and evaluators, and expectations. Players should be briefed on how and when to interact with the SimCell if you have one. It should always be reinforced that the exercise is ‘no fault’, and that they, individually, are not being evaluated, rather it is the plans, policies, and procedures they are using that are being evaluated. Lastly, players are briefed on the scenario. Be sure that the ExPlan/SitMan has the detailed scenario for their reference.

Exercise Play

Finally the moment of truth arrives! Remember that the purpose of the exercise is to accomplish the objectives, however, know that you may not accomplish all the objectives in the timeframe you have. Being flexible, and knowing you have to be flexible, are two very important aspects of running an exercise. Regardless of how well we think we’ve written an exercise, the responses to our prompts are entirely up to the players. They may accomplish the objectives faster or slower than you expected. Likewise, they may struggle a bit. This is where good exercise control comes in. Controllers/facilitators should observe the tempo of the exercise. Is everyone engaged? Are they needlessly overwhelmed and frustrated? Are they bored? Are they not following through on activities? All these observations should be reported back to the exercise director and SimCell so the tempo can be moderated.

Be sure to have contingencies in the event that players do or request the unexpected. Additional injects that are held back are always a good idea. You may need them to prompt certain activities in the event that players do not take the initiative to do so. Players may also ask questions or make requests of the SimCell that aren’t expected. While we can’t anticipate every need, we can be prepared for them. Have copies of plans, policies, procedures, and maps available to the SimCell. They may have to take some time to research and come up with an answer then get back to the individual. The SimCell manager should be smart when situations like this arise, however. They should always consider if a potentially inaccurate but reasonable answer is acceptable or if the answer must be completely accurate… or even if the answer is required for exercise play at all. There is no sense having SimCell personnel research an answer if the answer doesn’t much matter within the scope of the exercise.

Multi-day exercises may require the exercise staff to meet at the end of every day for a mini hotwash and to evaluate how the exercise is progressing. Where are the players in respect to where you expected them to be in your design of the exercise? Do you need to retool anything in the MSEL?

On large exercises, it helps for the exercise director to have an aide-de-camp, or assistant. Just like any individual in charge of a large operation, the exercise director is having their attention pulled in many directions and may simply not have the time to address everything, especially in large or multi-venue exercises. Having an assistant is also a great way to train up and coming exercise staff. Certainly consider the use of portable radios to help facilitate communication as well. The exercise director will spend most of their time managing and trouble shooting. They may also have to address some VIP concerns. Be sure to walk around to get a good sense for the exercise as a whole. Are evaluators properly positioned? Are controllers present and visible? Is the SimCell responding to questions adequately? Are players engaged and challenged? Perhaps most importantly, is the boss (the elected/appointed official) happy?

Evaluation is such an important aspect of exercise that I will cover it in its own section – Part 10.

Debriefs

Two significant debriefing activities should take place immediately following the exercise: 1) a player hotwash, 2) an exercise staff debriefing. Players should be led through a facilitated hotwash, reviewing the objectives of the exercise, with evaluators capturing their responses to if/how the objectives were accomplished. Encourage and capture responses both in the positive and the negative: i.e. what did we do well and what do we need to improve upon?

After the player hotwash, a similar process should take place with all exercise staff – controllers, evaluators, and SimCell. I like to not only capture their impressions of if/how objectives were accomplished, but I also like to discuss the conduct of the exercise, again capturing what went well and what needed to be improved upon.

 

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Tim Riecker, The Contrarian Emergency Manager 9 Comments

This post is part of a 10-part series on Managing an Exercise Program. In this series I provide some of my own lessons learned in the program and project management aspects of managing, designing, conducting, and evaluating Homeland Security Exercise and Evaluation Program (HSEEP) exercises. Your feedback is appreciated!

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

Managing an Exercise Program – Part 10: Evaluation and Improvement Planning

As I forge ahead in this series on Managing an Exercise Program (thank you all for reading!!), I expect the revised Homeland Security Exercise and Evaluation Program (HSEEP) foundation document to be released soon from the US Department of Homeland Security (DHS).  Once that document is released, I’ll be sure to include a summary update in my blog.  Having been a reviewer of the draft document about a year ago, I don’t expect a lot of changes, but what does change will have some bits of significance on how we do business in the design, conduct, and evaluation of preparedness exercises.

This installment of Managing an Exercise Program gets us two steps away from actually conducting the exercise.  As you can see, putting an exercise together is no small feat.  I find that this particular step: Preparing Support, Personnel, and Logistical Requirements, is the one most often glossed over in documents and training.  As an example, HSEEP Volume I dedicates only one paragraph to exercise logistical support.  As Volume I states in its single paragraph, logistical elements ‘can make the difference between a smooth, seamless exercise and one that is confusing and ineffective.’  Let’s break down our considerations:

The location of the exercise is of significant concern.  Often times we are examining facilities, but some exercises are conducted outdoors with no use of facilities at all.  If outdoors, you still need to ensure the proper environment and support services, such as restrooms, being available.  If your exercise requires water for fire suppression, then proximity to hydrants is essential, unless you are looking to incorporate tanker operations into your exercise.  We’re looking for a location that is minimally disruptive to the surrounding area, including traffic and ensuring citizen safety.  Consider the need for public messaging, such as static displays, variable message signs (you can get these from your public works connections), and media releases to inform the public of the exercise.  Doing so will help satisfy their curiosity, will give you some positive media exposure, and will help you minimize disruption.  As an example, I’ll cite an urban search and rescue (USAR) component of the Vigilant Guard New York exercise which I led.

Working with local officials, our USAR specialist and a representative of the New York National Guard exercise team were able to select an appropriate cite for their activities.  Set up was extensive, involving multiple loads of building demolition debris and a few cars to be hauled in and specifically placed with the use of heavy equipment.  On one side of this lot were a number of three-story apartment buildings, which we sought to minimize impact to.  All hauling and set up operations took place during the day while exercise activities, which were 24 hour operations for several days, were minimized during the night.  USAR folks come with a lot of equipment… and I’m not just talking a few boxes of stuff, either.  Many have tractor trailers and cargo containers to transport their gear.  They set up tents where they can unload and unpack much of their gear and provide areas for briefing and down time for personnel.  This exercise brought in first responder and National Guard USAR assets from around the state, other states, and Canada.  An eating area needed to be on site as well as sanitation.  Obviously all these areas needed to be well out-of-the-way of operational areas of the exercise to ensure safety and allow room for the rescue activities.  Portable diesel-generated light towers were set up to support night-time operations.  A media time was scheduled to allow media to catch some of the action during the week as well.  Since some teams were only coming in to exercise for a day, a schedule needed to be established to ensure that they could be accommodated and a traffic plan had to be established to get them to the site.  The exercise, which included multiple venues, covered a period of time which included Election Day.  With caravans of first responder and National Guard equipment rolling through the area during this time period, we were sure to schedule movements off rush hour and I even had a conversation with the County Board of Elections.  In this conversation I briefed them on the locations and activity of the exercise to ensure that it didn’t interfere with their polling locations and provided them with my cell number which I told them to call if there was even the slightest hint of a problem or complaint.

Indoor exercises require the same measure of preparation.  You have to ensure that the spaces you use are safe and large enough to accommodate participants.  You may have a need for one or more break out rooms or meeting rooms, both for exercise management staff and for players.  Unless players are responsible for setting everything up themselves, ensure that power, internet, and telephonic communications are available for them… and can support their needs.  Back to Vigilant Guard, the EOC component of the exercise was significant.  Based on anticipated use, we actually brought in state emergency management capability for satellite digital communications to support the simcell with internet with phone so we wouldn’t draw on and degrade the in-house capability for players in the EOC.  Similar to an outdoor venue, you need to pay heed to needs for parking, restrooms, and food service.  It’s also a good media opportunity, so be sure to schedule that well in advance with the media and some VIPs.

In regard to personnel, we’ve touched upon the need for controllers, evaluators, and simulators in previous posts, mostly in regard to planning these needs and ensuring that they are covered with the necessary documents to help with their tasks, such as exercise evaluation guides (EEGs), controller/ evaluator plan, master scenario events list (MSEL), and Exercise Plan.  Identify the exercise leadership early – the exercise director, simcell and MSEL managers, and lead controller and evaluator.  These individuals, and the supporting staff for them, including simulators, controllers, and evaluators, are likely to come from your exercise planning team.  Some may have experience in these tasks, while others may not… something to keep in mind for development of the documents as well as the briefings you conduct for them just prior to the start of the exercise (that’ll be the next part of this series).  Don’t just assign folks randomly to positions, draw on their experience.  If someone has a strong EMS background, assign them to be controllers, simulators, or evaluators for that area of practice.  Be sure that your simulators also have some local experience as well if you are conducting this exercise for an area outside your own.  Local flavor brings realism and context to an exercise for the players.  Consider radios for controllers and evaluators, especially in large exercise areas.  This will allow the exercise director to speak with them and for them to interact with the simcell, letting them know if they need to speed up or slow down.  Also consider providing the exercise director with an assistant on large exercises.  Often times I’ve found the need for someone to aid me directly in resolving problems, gathering people, and handling miscellaneous tasks that are too much for any one person to handle.  It’s also a great learning experience for someone who wants to advance.

Overall, be sure to plan early for all logistical, support, and personnel needs.  Plan early for food contracts, ensure that all participants have the necessary supplies to conduct their jobs.  Plan ahead for safety as well, ensuring a safe work environment proactively and a good plan and personnel who can react to situations should they arise.  Be ready on-the-fly for changes and little or no-notice occurrences, as they almost always happen!  Make sure the players have everything they need for the exercise – if not, that lack of preparedness will be what they remember.

What experiences or ideas do you have with supporting an exercise?