Tag Tim Riecker

Creating Operational Emergency Plans

Tim Riecker, The Contrarian Emergency Manager 2 Comments

I was inspired for this article from an email I received earlier today from Lu Canton, a rather prolific emergency management consultant who has branched a bit into consulting consultants.  His email today (a forward from his blog) was about making emergency plans ‘real’.  His point was that many planners focus on checking the boxes of the list of planning requirements (those prescribed by law, regulation, etc.) rather than focusing on ensuring that you have a plan that can actually be implemented.  He conducted a webinar over a year ago which I had blogged about.

Planning requirements are important, as they largely stem from lessons learned from earlier incidents.  Granted, some of these requirements come about being translated through the eyes and ears of politicians whose staffers write the legislation and don’t understand emergency management at all – resulting in convoluted, contradictory, and poorly focused requirements.  Requirements lead to standards, helping to ensure that emergency managers are addressing the needs of their jurisdiction and best practices in the industry.  To help guide us through this, many higher level agencies provide templates.  I’ve pontificated in the past about the danger of templates, which have a place in reminding us of these requirements and help us with format and flow, but are often misused by individuals who simply seek to fill in the blank with the name of the jurisdiction and claim they have a finalized plan.

How do we avoid falling into this trap?  Follow the planning process!  FEMA’s Comprehensive Preparedness Guide (CPG) 101 provides an overview of this process for creating emergency operations plans.  The two initial steps – forming a planning team and conducting a hazard analysis – are absolutely critical to the integrity of the process and ensuring a quality plan that meets the needs of your jurisdiction by addressing your threats and hazards.  Planning teams then need to consider these threats and hazards, make reasonable assumptions about their impacts (using a credible worst case scenario), then identify resources and strategies the jurisdiction will undertake to solve the problems they will face.

Does all this mean that a plan needs to be written from scratch?  Of course not!  In fact I strongly encourage people against it.  It’s practically guaranteed that you will forget a critical element.  One of the greatest things in the emergency management community is how we learn from each other.  You can reference templates you find, examine plans of your neighboring jurisdictions or jurisdictions similar to you, check out what is on LLIS.  There is plenty of great content you can examine and apply for your own use.  Just ensure that you carefully review and consider how it applies to you.

As you write the plan, think the details through.  This will help ensure that your plan is operational, not just meeting requirements.  Discuss with your planning team what is expected of each assisting and cooperating agency for each incident type.  Who will be in charge?  What resources will be necessary and where will you get them from?  What would the objectives be and what processes and decision points must be conducted to accomplish those objectives.  As you create the plan, map out these processes and ensure that you’ve considered the who, what, where, when, and how of each step in each process.  Recall that you are planning at a strategic level, not a tactical level.  Planning at a tactical level is nearly impossible with a pre-incident (aka ‘deliberate’) plan.  Tactics will be addressed during the actual response, hopefully referencing the EOP/CEMP you are writing now, and implemented through an incident action plan (IAP).

Remember, though, the proof is in the pudding, as they say.  Your plan needs to be tested to ensure viability.  Use a table top exercise to test policy and decisions, then a functional exercise to test the implementation of the plan and higher level tactics.  Full scale exercises and drills can test the tactical implementation of plans.  Good evaluation of the exercises will lead to planning improvements.  For insight on the exercise process, you can check out my exercise management series of posts referenced here.

Remember: when it comes to planning – keep it real!

Tim Riecker

Critical Infrastructure Vulnerability – Water Delivery Systems

Tim Riecker, The Contrarian Emergency Manager 3 Comments
Picture of the Baltimore Water Main Break, July 18, 2001

40-inch Water Main Break, Baltimore July 18, 2001. Source: Baltimore Sun

I’ve written several posts in the past on the vulnerabilities of our electrical infrastructure – both to natural and human causes.  Yet, our electrical infrastructure is not the only element of critical infrastructure that is vulnerable to failures and attacks.   We have a very old water infrastructure in our nation, with many areas still maintaining Civil War era cast iron pipes, with an estimated useful life of 150 years (at the time of installation).  How often does your area experience a water main break?

According to the US Conference of Mayors “A major symptom of the aging water infrastructure includes 300,000 water main breaks in North America as result of the widespread corrosion problems adding up to a $50.7 billion annual drain on our economy. Leaking pipes are also losing an estimated 2.6 trillion gallons of treated drinking water annually (17 percent of all pumped water in the US), representing $4.1 billion in wasted electricity every year.”

This aging infrastructure has also failed us when we needed it most.  You might recall the Howard Street Rail Tunnel fire in Baltimore, MD on July 18, 2001.  I’ve designed exercise scenarios based upon this incident and have even received feedback from participants about the scenario being unlikely – they are rather surprised to learn that it is based on an actual event.

From the USFA Report on the incident:

“At 3:07 p.m. on Wednesday, July 18, 2001, a CSX Transportation train derailed in the Howard Street Tunnel under the streets of Baltimore, Maryland. Complicating the scenario was the subsequent rupture in a 40-inch water main      that ran directly above the tunnel. The flooding hampered extinguishing efforts, collapsed several city streets, knocked out electricity to about 1,200 Baltimore Gas and Electric customers, and flooded nearby buildings. The crash interrupted a major line associated with the Internet and an MCI WorldCom fiber optic telephone cable.

Throughout the incident, fire officials were plagued with three problems: fighting the fires in the tunnel; the presence of hazardous materials; and the weakening structural integrity of the tunnel and immediate surrounding areas.”

In reading the report you will see that the water main break both help and hurt the response.  The 40-inch main flooded streets and nearby businesses, but also was allowed to flow into the tunnel for a period of two hours, helping to decrease the temperature in the tunnel.  While no reports seem to indicate the impact of the water main break on nearby hydrants, I do include that impact in my exercise scenarios.

Water main breaks plague many areas around the nation.  The lack of potable water resulting from them creates a public health concern, resulting in many businesses and public buildings shutting down and households advised to boil water.  These breaks impact our ability to fight fires and, as a result of undermining, they can cause sink holes and damage to roadways.

In speaking to public works officials through the years, I’ve been told that every water system has leaks of varying severity.  Minor leaks often go undetected for a great period of time.

 

Securing our water supply is important as well.  Much of our water storage is in reservoirs, open and vulnerable to intentional contamination.  Most reservoirs have some measure of passive security (fences) and some even take more active security precautions.  However, we know that people who are determined can overcome these systems.  Luckily the sheer volume of water in most reservoirs would severely dilute any contamination introduced to them, but there may be agents so concentrated as to inflict harm.  The City of New York, for example, has a massive water supply system, with reservoirs as far north as the Catskills.  Their aqueducts, made famous in the third Die Hard movie, are massive.  The New York City Department of Environmental Protection is charged with securing the City’s water supply and does so through both active and passive security measures as well as active and on-demand water quality sampling.  Most areas, however, don’t have these law enforcement or public health resources available in such abundance.

Water is a critical component of our infrastructure and therefore must be protected.  It’s important not only to business and industry, but is also essential to human life, agriculture, and food production.  Similar to our roadways and electrical infrastructure, our water systems need a plan for restoration and funding to put that plan into action.  Beyond some more capable and financially stable municipalities, most water systems are implementing ad-hoc fixes and are only able to replace small sections of the system each year.

Does your plan account for water system failure?

Tim Riecker

Does Your Municipality Have an Emergency Management Handbook?

Tim Riecker, The Contrarian Emergency Manager Leave a comment

I recently came across an emergency management handbook assembled for municipalities in Connecticut.   First of all, I’m thrilled that officials in CT are openly sharing this handbook.  Doing so makes it easier for their municipal officials to find and, in the spirit of sharing found within emergency management, it is always great to share best practices.  There are a great deal of concepts, templates, checklists, and the sort which are benchmarked and shared throughout the emergency management community, by way of the Lessons Learned Information Sharing (LLIS) site and other means, which are of great help the collective profession.  If someone else has already done something and it seems to work, why not adapt it to your own needs instead of trying to create something completely different?

I recall early in my career when handbooks and reference guides assembled by states for their local governments were common practice.  For a period of time we seemed to get away from that for some reason, but the practice now is making a resurgence.  I consulted on a project a year ago for the creation of a UASI regional handbook and have seen others in use.  The CT handbook which I linked to at the beginning of the article is an excellent example of what should be included in a handbook and how it should be structured.  Recall from my training-related articles that my focus is always on the audience and their needs.  This audience-centered approach doesn’t only apply to training – it also applies perfectly well to document development. So who is the audience for these handbooks?  Certainly the chief elected officials of counties, cities, towns, and villages.  Their subordinates (department heads) can also benefit greatly from the content.  How about schools (primary and secondary education alike) and hospitals?  The more people who are familiar with the foundational concepts of emergency management and how it is applied in a particular state, the better.  With some modification to include more information for them, I’d also include private sector entities as well.

The structure of the CT document is also one to be benchmarked.  They make reference with brief explanations to federal and state laws and related authorities of individuals and agencies regarding emergency management and related topics, such as NIMS.  For these and other references throughout the document, they include external links where additional information can be found.  They include emergency management structure and flow, which helps local governments identify who they need to coordinate with and who they should request assistance from.  They break down the document into the phases of incident management and include information on each; including planning, training, exercises, and grants within Preparedness; communication, coordination, notification, resources, and other information within Response; debris management, PDAs, and recovery programs in Recovery; and the variety of common mitigation programs in Mitigation.  Appendices provide information sheets on things like the CT VOAD and 211, as well as templates and checklists for the critical elements of each phase of emergency management.

Recalling the needs of the audience, CT addresses them well with this document, which was a collaborative effort between their state emergency management department, emergency management association, state conference of municipalities, council of small towns, and association of regional planners.  These types of collaborative efforts help ensure that the right information is being conveyed.  Most new elected and appointed officials know little about emergency management and need to become familiarized with it before a disaster occurs.  While response is the most prominent phase of emergency management, the other phases are necessary to promote an effective response and a good, overall program.  Even for those experienced in emergency management, a guide such as this provides excellent references for the critical information.  The handbook is of reasonable size – only 90 pages – so it is easy to navigate.  While it has value in print, it is even more valuable electrically given the addition of hyperlinks which bring you to other information.  As a PDF, it’s handy to keep in the library of a iPad or other tablet where it can be referenced easily.  Overall, the information is succinct, giving only what is needed for foundational understanding.  The templates, checklists, and quick reference guide found in the appendices help turn the information into actionable content.

Lastly, the structure and content of the document lends itself well to a structured review (i.e. training), which gives people the opportunity to look at the document in-depth and ask questions.

Does your state or municipality have a handbook similar to this?  What are your thoughts on the one CT provides?  Would you like to see more content or less?  Have they missed something essential?

 

Tim Riecker

Is the Department of Defense Planning for a Zombie Attack?

Tim Riecker, The Contrarian Emergency Manager Leave a comment

No, not really.  But they are using the zombie scenario as highly effective training tool.

I reference, of course, the recent news releases (such as this one from ForeignPolicy.com) concerning the United States Strategic Command document titled: CONPLAN-8888 – Counter-Zombie Dominance (April 30, 2011).  You’ll see in the Disclaimer section of the document, beginning on the second page, that this scenario was selected because of its improbability of occurring, unlike other scenarios which may be used as the basis for training and exercises, which could be mis-perceived by the public.

The training they refer to is the Joint Operations Planning Process (JOPP).  Typically the word ‘Joint’ in military doctrine refers to activities which include two or more branches of military service. Not only do the US Military Services use joint planning and conduct joint operations, but the Army and Air National Guards do as well.  For as much as it makes sense, the concept of ‘jointness’ has really only seen large-scale attention in the last decade or so.  Having worked extensively with the National Guard in recent years, the ‘jointness’ has become the foundation of nearly all activity.  This is a great accomplishment as there is considerable competition between the branches, sometimes to the level of disdain, and certainly a great many differences, even in terminology and rank structure.  Despite the decades-long existence of the Joint Chiefs of Staff, true joint integration at an operational level rarely occurred and was generally not executed well.

Joint operations are simply a military version of the Incident Command System (ICS), where different disciplines and agencies – some often very different from each other – can function together within one organizational structure for the purpose of addressing an incident.  If you are interested in additional information on Joint Planning and Operations, I suggest you visit the DoD’s Joint Electronic Library where you can find documents which I have referenced often when working with the National Guard on preparedness and response activities – including the Universal Joint Task List (UJTL) (similar to the old DHS Universal Task List, in concept) and the Joint Publications (such as Joint Publication 5-0, Joint Operation Planning – which is the foundation for JOPP).

As mentioned in earlier posts, I love the integration of zombies and other apocalyptic lore into preparedness activities.  While tongue in cheek, they help planners and responders do away with pre-conceived notions they may have with other, more realistic scenarios.  In emergency management, we know that every incident is different.  While we learn from our experiences and hopefully apply what we learned to the new challenges we face, we sometimes go into the next incident with a bias.  The introduction of totally fictional scenarios, such as a zombie attack, help make the situation faced completely new – while many of the specific problems faced are things we do have plans for, such as mass fatalities, hazmat, contagious diseases, etc.  It’s a great approach which I strongly advocate.  The only caution we may have is that the premise of the scenario itself may cause some to not take the activity seriously or may cause a distraction.

Have you used any non-traditional or totally fictional scenarios?  What were they?  How were they received?

/s/

Tim Riecker

NYC Ready for Godzilla

Tim Riecker, The Contrarian Emergency Manager Leave a comment
Godzilla Movie Poster

Godzilla Movie Poster

If you’ve read some of my earlier posts, you know I love emergency management campaigns that draw on pop culture – especially post apocalyptic shows like The Walking Dead.  This article from the NY Daily News where NYC Office of Emergency Management Commissioner Joe Bruno indicates that the impacts of an attack on the city by a creature such as Godzilla would be manageable.  In essence, they are right.  Why?  Because NYC follows a model of all-hazard preparedness.  Do they have a Rampaging Radioactive Monster plan?  Of course not.  But they do have plans that address “fire, explosions, casualties, damage, debris, bridges and tunnels being out.  Roads being out, power issues…” as Commissioner Bruno said.  He continued on, joking that they don’t have a plan for slime.  (I don’t think Godzilla leaves slime, though)

A few weeks ago I blogged about all hazard planning and included some thoughts on asteroid strike and other low probability events that we don’t necessarily need a specific plan for – we simply needs plans that address the impacts and how we will respond to and manage them.

While the article is largely tongue in cheek, even including a link to an article about the US Air Force’s capability to take down Godzilla and referencing a study conducted by the Hollywood Reporter estimating the dollar cost of the damages in NYC from the Chitauri invasion featured in the Avengers movie, it still leaves a subliminal message to the reader that we (or at least NYC) is well prepared for the impacts of disaster.

Scenario-based Learning in Emergency Services – Success, how to, and the need for more!

Tim Riecker, The Contrarian Emergency Manager Leave a comment

Throughout all professions and specialties within the broad category of public safety and emergency services, we go through a lot of training.  Training within these various professions has become a profession itself, with many individuals, myself included, spending much of their career as trainers.  Through the vast quantity and variety of training we’ve received, we see many examples of training across the whole spectrum of quality and effectiveness and have endeavored to improve upon training and increase learner retention.

My Experiences

Much of the training that is most memorable to members of the public safety community is that which is hands-on.  Three classes in particular that stand out to me are the New York State Mask Confidence course, the DHS-sponsored Incident Response to Terrorist Bombings (IRTB) course conducted at the New Mexico Energetic Materials Research and Training Center (EMRTC), and the DHS-sponsored Enhanced Incident Management/Unified Command course conducted at the Texas A&M Engineering Extension Service (TEEX).

I attended the Mask Confidence Course about 15 years ago when I was active in the fire service.  Course instruction was led by Chuck, a well-respected fire officer in Central New York and an outstanding fire instructor (who, a number of years later, worked for me as an adjunct).  The premise of the course is for participants to become confident in their own abilities and limitations while wearing self contained breathing apparatus (SCBA).  Much of the course was conducted under blackout conditions, either in a darkened room or wearing a blackout mask, so you had to learn everything by touch – while wearing bulky firefighting gloves.  The course started simply, with a timed donning of all gear and SCBA in a darkened gymnasium; and ended with much more complexity several sessions later by navigating a dark, smoke-filled maze on your hands and knees, sometimes having to remove (then re-don) the SCBA pack to fit through tight spaces, all while still ensuring that you can breathe through the mask.  The course certainly increased my confidence in wearing SCBA and gave me important lessons learned which I was able to apply in fire operations later.

The IRTB course was an excellent program which instructed participants on the chemistry and physics of explosives – including those in common industrial and military use, as well as improvised devices – first in a classroom environment then in a field lab where we had the opportunity to see these explosives in action first hand.  The class was fun and insightful, but rigorous and more cerebral than expected.  As with many DHS consortium programs, the EMRTC has its own course managers and draws on expertise from around the nation to aid in instruction.

The Enhanced Incident Command/Unified Command course included a review of ICS concepts – mostly those at the ‘intermediate’ level, with emphasis on the management of a Type III (extended) incident and the necessity of transitioning to the formal planning process within ICS.  Learning was reinforced through participants taking on roles in an ICS structure and being challenged by a computer-based simulation.  Similar to the EMRTC course, this program drew upon the expertise of instructors from around the country.

Why Scenario-based Learning Works

Why do these three courses stand out for me?  Certainly they are topics of interest and had great instruction.  That alone should help anyone remember some of the course content, but these shine well above others.  Because they were hands-on?  Certainly Mask Confidence was hands-on, but to maintain safety, IRTB was mostly hands-off – literally.  They didn’t actually let us touch any of the explosives.  The TEEX course gave us a computer-based simulation, which is only somewhat hands-on.  So what was it that made these programs stick with me?  The commonality of these programs is that they were scenario-based.

In each course we first learned foundational material through traditional didactic instruction.  Certainly, the use of examples, pictures, and video enhanced our learning.  After the didactic portions, we were then challenged with a series of scenarios, each different from the previous and with increasing complexity.  Mask Confidence was largely an individual course – intended to build your own skills and confidence with your own life-sustaining equipment in adverse conditions.  While IRTB provided a group learning environment, we didn’t necessarily work together to solve problems; rather we were posed with scenarios to examine, akin to an amateur forensic level, to evaluate cause and effect.  The Enhanced Incident Command/Unified Command course was very much based upon individuals contributing to group success in solving problems.

Why is scenario-based learning so successful?  First, consider the graphic below.  We’ve all seen variations of this information.  The bottom line is that the increased degree to which a learner is actively engaged, the more information they will retain.  Many of the activities I described above for the three classes that stand out in my memory are contained in the 50%+ categories of retention.

Learner retention based upon delivery method

Learner retention based upon delivery method

How do we apply this concept to learning?  Bloom’s Revised Taxonomy is a regular reference I use when designing courses and can be a great starting point to determining if participants can benefit from scenario-based learning.  The essential meaning behind Bloom’s Taxonomy is: What level do participants need to be trained to?  In emergency services we often use the term ‘Awareness’ level training to identify a course at the lower levels of Bloom’s Taxonomy – those are the levels where we need to remember or understand information, but aren’t required to use the information.  In an ‘Operations’ level training course, learners are expected to apply what they have learned and perhaps do a lower level of analysis.  In a ‘Technician’ level course, higher levels of analysis as well as evaluation and creation are often expected as learner outcomes.  Compare the terms from the Bloom’s Taxonomy graphic below to those on the right side of the learner retention graphic above and you will see some similarity.  The correlation is that if we expect learners to apply, analyze, evaluate, or create, we need to ensure a higher rate of retention – therefore scenario-based learning is often an ideal strategy.

Bloom's Revised Taxonomy

Bloom’s Revised Taxonomy

How to Integrate Scenario-based Learning

Integrating activities into training courses can be tricky.  There are a great number of ‘training games’ out there – (Amazon has a huge listing of books on the subject), but every activity in a course has to be meaningful and drive toward one or more course objectives.  We’ve all seen gratuitous activities integrated into courses.  While some of these may be great, they do nothing to help us achieve the course objectives.  Also, simply having activities does not make a course scenario-based.  To achieve this, you need to include one or more scenarios.  Make sense?

The Bing dictionary defines ‘scenario’ as:

Definition of scenario (n)

  • sce·nar·i·o
  • [ sə nárree ō ]
  1. possible situation: an imagined sequence of possible events, or an imagined set of circumstances
  2. plot outline: an outline of the plot of a play or opera
  3. screenplay: a screenplay for a movie

We develop scenarios for exercises all the time, so when trying to integrate a scenario into a training course, just consider it a miniature version of an exercise.

First – where to start?  As mentioned earlier, a scenario must be associated with one or more learning (course) objectives.  It also must have (enabling) objectives of it’s own.  In other words, what are the expected outcomes of participating in the scenario for the learner?  Perhaps you want your learners to create an Incident Action Plan (course objective).  Scenario-based learning is great for this.  First, I would use didactic instruction to review the planning process and the tools (ICS forms) associated with this.  Then you can assign roles (within an ICS structure) and provide a scenario to help facilitate learning.  The scenario should generally be realistic, although you can always put a fun twist on it by using a zombie attack or some such thing.  This however, can be distracting – so you are better off sticking with something realistic and familiar to the learners.  HazMat incidents tend to work well as scenarios across the nation.  Add context by including detail – fixed facility or in transit (road?  rail?).  Time of day, day of the week, weather.  What were the immediate impacts?  What are the current threats?  The same concepts can be applied for something more hands-on, such as for water rescue training or a hostage negotiation training.

You have a choice of either placing the scenario in a location in which the learners are familiar (they know the roads, the resources, etc.), or providing a fictional location with supplemental information which they have to learn (maps, resource lists, etc.).  Both work with a fair amount of success, but you can be challenged if not all participants are familiar with the same area.  The HSEEP training course provides fictional jurisdictions I’ve used often for different training courses, as does FEMA and the EPA.  You can create a fictional jurisdiction yourself, but to do it right takes time and attention to detail.  Consider using a progressive scenario to facilitate several activities through the training program.

Be sure to give your participants clear instructions on what is expected of them.  Challenge them, but don’t frustrate them, which can impede learning.  Remember, this is NOT a formal HSEEP exercise as we know them.  Learners are not being tested, nor are policies, plans, or procedures.  You are providing a structured, experience-based learning environment.  Be sure they have the tools to succeed. With the Incident Action Planning scenario, an ICS Field Operations Guide (FOG) or text book guidance is a great reference for them.  Provide learners with the opportunity and a safe environment to ask questions, and even correct them if they stray too far from the desired path.  Remember – perfect practice makes perfect, so learners should be practicing ‘by the book’.

Finally, similar to exercises, hot wash the activity.  Ask learners how they feel the activity went.  What went well, what didn’t go so well?  What feedback do the instructors have?  While we aren’t testing learners in these scenarios, we should be evaluating them.  This open discussion feedback is important to their learning and can also help you improve your scenario for the next time.  The folks in TEEX actually capture video and audio of participants during activities which they use to help facilitate hot wash sessions.  This obviously takes time, equipment, and personnel which most don’t have available to them – but it’s great to experience.

Scenario-based learning takes a lot of preparation and forethought.  It also takes a lot of training time to implement.  When we’re fighting for training time and training dollars, we need to advocate for the value of scenario-based learning.  Make sure it’s done right, though… a poorly executed activity can have a negative impact on learning.

Scenario-based learning works.  There are best practices in the training of firefighters, police officers, EMTs, and dispatchers using scenarios.  They help learners ‘get their head into the task’, helping them bridge the mental gap between an activity and real-world application.  Consider how not only to integrate scenario-based training into your courses to reinforce learning, but also to substitute content we are currently delivering by other, less interactive means.

What successes have you found in scenario-based training?  What challenges have you encountered?  I’d like to hear from you.

 

Tim Riecker

App Review – The ASTD Trainer’s Toolkit

Tim Riecker, The Contrarian Emergency Manager Leave a comment
ASTD app home screen

ASTD app home screen

A few weeks ago I downloaded the new ASTD (the American Society for Training and Development – recently changed to the Association for Talent Development) iOS app, the ASTD Trainer’s Toolkit.  Surprisingly, as the ASTD doesn’t give much away, this app is FREE!  The ASTD info site for the app can be found here.  As their website states, the app includes 20 original classroom and virtual training activities to energize, motivate, and help learners to retain content.  The activities include closings, energizers, forming groups, and topical openings.  All of this content is internal to the app.  The app also includes links to external content such as articles on Training and Development (T&D) and a selection of T&D books through links to external content (the ASTD website for articles, and the iTunes Library – for iOS users – for the books).  There is an article native to the app titled ‘Intro to Facilitation’ and an ability to create your own notes, add your own activities (they provide you with a template), and add their activities to your own list of favorites.  While the app can function as a stand-alone on your device, you can also log in through Facebook or Google to access your data from anywhere.

This can be a handy app when you are stuck trying to think of a learning activity for a group.  They break each activity down with a variety of essential pieces of information, including the size of the group the activity is applicable for, the time of the activity, the person who contributed the activity, the goal of the activity, the materials and/or technology needed for the activity, search tags related to the activity, and a step by step process for conducting the activity.  They also provide some facilitator notes/tips for each activity.  The user can enter their own notes for each activity or use an in-app timer.  You can also tap the ‘favorite’ star in the upper right corner to add the activity to your list of favorites.  You can also use the in-app search function to find activities based on various metrics such as group size or time limit.

In typical ASTD fashion, however, there are plenty of opportunities for in-app purchases.  Not only the books, but you can also download additional activities from the app.  These additional activities include Icebreakers, Openings, Reviews and Teachbacks, Trainer Tools and Techniques – each for $1.99, or all of them for $5.99.  There does not appear to be a limit on the number of article views from the app – if you are familiar with ASTD’s website, non-members have a great deal of restrictions on viewing their articles.

All in all, it’s a good app with solid, easy to reference information.  The app did crash on me since my first download of it, resulting in a need to delete the app and reinstall it.  I’d also like to see more information provided in the app.  Activities are great – and we should all include more objective-driven activities into our training – but what about other areas of training?  Perhaps the steps of instructional design – be it the ADDIE or SAM process?  Perhaps a reference for Bloom’s Taxonomy or the Kirkpatrick levels of evaluation – two things I often reference when designing instruction or when writing a proposal involving instructional design (especially the action words associated with each level of Bloom’s Taxonomy!).  Remember that training is more than just what occurs in a classroom.  I appreciate the ASTD/ATD for providing us with this app, but challenge them to give us more (and for free, thank you very much!).

Give the app a try yourself and let me know what you think.

TR

Managing an Exercise Program – Part 10: Evalutation and Improvement Planning

Tim Riecker, The Contrarian Emergency Manager 1 Comment
New HSEEP graphic

New HSEEP graphic

For the previous parts in my series Managing an Exercise Program, please see below:

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

 

While conducting an exercise is often the most exciting part of the exercise cycle, evaluation and improvement planning lead us to the real reasons why we conduct exercises – to test plans, policies, and procedures.  Evaluation doesn’t just happen.  It must be a deliberate, planned activity within the design and conduct of an exercise.  There are key activities through the design and development of an exercise which link directly to evaluation – starting with the identification of core capabilities to be tested and exercise objectives to be accomplished.  These core capabilities and objectives should lead us directly back to specific plans, policies, and procedures which will be exercised and can help us determine the evaluation methodology and approach.  As we further develop our exercise, be it discussion based or operations based, the decisions we make influence evaluation and the resources needed for it.

When exercise planning begins, we identify staff to fill key roles – including exercise director, simcell manager, lead controller, and lead evaluator.  The lead evaluator doesn’t necessarily have to be involved in all aspects of exercise design, but they need to be informed of key points and should be reviewing draft materials (i.e. the explan or sitman) to become familiar with the details of the exercise and to help them assemble their portion of the control and evaluation plan.  The lead evaluator should ideally be present at exercise planning meetings to become even more familiar with the details, the people, and the facility(s) of exercise play.

Staffing evaluation has a number of options.  Often times evaluators will be selected based on matching their individual areas of expertise with the functional areas of the exercise (i.e. fire, EMS, public health, EOC management, shelter operations, etc.).  You may have a need for more than one evaluator per functional area, particularly if that function is a major component of the exercise and there is a lot to observe.  Remember that if there are multiple exercise venues, they each need to have evaluators assigned who can observe all critical areas.  The bottom line is that evaluation should respect the amount of time, effort, and resources invested in the design and conduct of the exercise.  If the exercise isn’t fairly and accurately evaluated, it’s simply disrespectful to all that effort and those involved.  These concepts apply to both discussion based and operations based exercises.  Members of the exercise planning team should be considered as possible evaluators.

Exercise Evaluation Guides (EEGs) give evaluators guidance in their activity.  The HSEEP website provides a multitude of EEG templates which you should absolutely modify for your own use.  EEGs are provided for each core capability.  Just like planning templates (see my rant about the use of planning templates here), these are guides for you to reach success, but can not and should not be used without modification and customization to ensure that they meet your own specific needs.  EEGs are crafted hand in hand with exercise design to ensure that all bench marks and expected actions are accounted for.  The rating guide on the last page of each EEG is particularly handy to ensure consistency.  You may want to consider including a ‘recommendations’ section for each observation as well, which provides an opportunity for you gain insight from your own subject matter experts on how to address a particular area for improvement.  This will help when it comes to the After Action Report.  I also encourage evaluators to use and ICS form 200 (a blank sheet of paper!) to take some freeform notes.  EEGs, no matter how well designed, simply won’t capture everything.  Personally, when I evaluate, I’ll refer to the EEGs, but make most of my notes on a blank sheet(s) of paper or a steno pad.  I like to mark specific observations, times and timelines, and even inject numbers if I happen to catch them.  After the exercise, I then transcribe my notes into the EEG in the appropriate format.

Training of evaluators is important.  You may have some evaluators which are new and some that are experienced.  Everyone should go through a controller/evaluator briefing prior to the exercise.  Those with little experience might require some extra training, which isn’t a bad thing.  Be sure that both your C/E plan and the briefing identify clearly what you expect from evaluators, how they should conduct themselves, the measure of allowable interaction with players, how they should take notes, and when you expect those notes to be turned in.  The HSEEP website, referenced prior, also has templates for C/E briefings, C/E plans, etc.  You’ve likely heard the saying ‘garbage in – garbage out’… well, exercise evaluation is exactly like that.  If you don’t invest in good training and job aids (EEGs) for your evaluators, you will not get good evaluations!  The Lead Evaluator also needs to lead!  They should check in periodically with each evaluator throughout the exercise to see how things are going and give a quick look at their notes and/or EEGs.  Good feedback will help your evaluators provide better observations.

Finally, at the end of exercise play, I expect evaluators to help with the hotwash.  Sometimes hotwashes are conducted as a single session with all players.  In such sessions, the exercise director or lead evaluator should facilitate and evaluators should take notes.  If the hotwash is tiered, where each functional area will identify their top strengths and areas for improvement; each functional area is facilitated by its evaluator who should also take notes.  This may be followed by a plenary session if possible.  Be sure to have your hotwash strategy planned and identified in the C/E plan!

The After Action Report (AAR) should ideally be written by one or two people based on the EEGs and notes of evaluators.  It may be possible for some observations to be combined, helping to make the document both more concise and easier to read.  Templates are provided on the HSEEP website for AARs and Improvement Plans.  Improvement plans are nothing more than a matrix identifying who will be responsible for what improvement actions, who will assist, what the major benchmarks are to improvement, and when it should be accomplished.  As with the rest of the AAR, these should be drafted and provided prior to the AAR meeting for comment – of which there is usually a great deal.  Once all this is finalized, don’t let this be the last of it!  The implementation of improvements is where organizations often fail!  Rarely do organizations follow through on improvements, resulting in similar observations being noted in exercises and incidents for years after.  The AAR/IP must be championed by someone at an executive level, and coordinated on an ongoing basis by someone who is responsible for tracking progress, coordinating solutions to problems, and reporting progress back to the executive.  For most organizations and jurisdictions, I would suggest a quarterly meeting to review improvement planning progress.  After a few exercises, you may find yourself addressing improvements of several exercises in one meeting.  Accomplishments should be noted, with the opportunity to test the ‘fixes’ in future exercises.  All this feeds back into the strategy planning phase of exercise program management – which is where we started in this series.

This is the end of my ‘Managing an Exercise Program’ series.  I appreciate all those who have read and provided feedback.  Please continue to do so, and best of luck on your next exercise!

I’d be remiss if I didn’t mention that my company – Emergency Preparedness Solutions, LLC – can help you with exercise program planning and management; training and exercise plans; and the design, conduct, and evaluation of exercises; as well as training in HSEEP, exercise design, and exercise evaluation.  Give us a call or email and we’ll be happy to discuss what we can do to help your organization’s preparedness.

EPS logo with tag line - Be proactive, be prepared.

EPS logo with tag line – Be proactive, be prepared.

 

 

FEMA Public Assistance Thresholds aka Do Your Research

Tim Riecker, The Contrarian Emergency Manager Leave a comment

I just received my April edition (number 627) of the Natural Hazards Observer (NHO), a bi-monthly publication of the Natural Hazards Center of the University of Colorado at Boulder. As of my posting of this article, they have not yet posted the April edition on their website, but they are likely to do so shortly.  My desire with this particular post is to encourage a thorough analysis of the topic at hand rather than an extremely narrow perspective as was published in the NHO.

First, I feel a need to mention that The Natural Hazards Observer and I have a tenuous on again, off again relationship. I’ve become rather disenchanted with their articles in the past; occasionally disagreeing completely with authors’ premise and approach (not that everyone is necessarily agreeable to mine). I’ve even written the NHO about the fact that a University center whose goal is to research disasters has presented articles that, to me, come across at times as unprofessional and not well thought out. I’m not really knocking the NHO nor is my intent to pick a fight, rather I’ve been trying to be constructive with them.  Given their foundation, I think their readers expect a more scholarly approach. While there is certainly plenty of good information to find in the NHO, some seems questionable. One of their tag lines is ‘Disaster Research News You Can Use’. I often find that their articles have little to do with research, such as the one I cite below, and are really more of a media regurgitation. There are other, more appropriate forums for that – like blogging – rather than a publication that advertises research.

That said, my particular beef with this edition is the first article, titled ‘All’s Not Fair: Illinois Battles Over Being Too Big To Feel FEMA’s Love’. I encourage you to read the article for yourself and form your own impressions. To sum up the general idea of the article, folks in Illinois are questioning FEMA’s Public Assistance (PA) program thresholds for disaster recovery. Specifically, they take issue with these thresholds being based on per-capita amounts. They argue that larger, more populous, states are discriminated against (their words); and further stating that a larger population doesn’t necessarily equate to a state’s ability to self-fund disaster recovery efforts.

Public Assistance (PA), by the way, is focused on recovering public infrastructure (roads, bridges, public buildings, etc.), whereas Individual Assistance (IA) provides direct assistance to individuals to help cover uninsured losses.

When first reading this, two particular issues stood out to me. 1) They don’t mention the fact that FEMA will also examine a county’s per-capita damages for potential declarations, and 2) High populated states quite frequently receive disaster declarations.

The Texas Department of Public Safety has a great document which summarizes the state and county thresholds for 2014. The NHO article makes no mention of the county threshold, which is where most PA declarations are decreed – for a county or counties, rather than the entire state. While the county threshold is higher, it would generally be easier for a localized disaster to be declared (PA) than the entire state, which is mostly unaffected.

As for state declarations, FEMA provides a great summary by state of major declarations (this is the specific number to look at in this case), emergency declarations, and fire management declarations. By scanning down the list, you’ll see that states like Texas, Florida, California, New York, and Louisiana are among the top contenders for declarations. Other large and more populous states, such as Illinois, are not far behind them. This pretty much blows a hole in the premise that larger, more populated states are ‘discriminated against’. Having been in this business for a while, I know that sometimes you win (the request for a declaration) and sometimes you lose. It makes a significant argument for states to have disaster reserve funds.

All that aside; are there issues of fairness in this traditional per-capita assessment of damages within the disaster declaration process? There may certainly be. The assumption of the current process is that a more populous state has a greater tax base from which to draw funds for disaster relief. This assumption, at a glance, makes sense, but may not be the most fair way of determining if a state warrants assistance.  The most significant arguments against this seem to be 1) A lower relative mean income as compared to other states translates into a lower tax base, and 2) Is the tax base even a fair measure of need/capability for a state?

An interesting read on this is a GAO report from 2012. It seems to me, a guy who is not a FEMA disaster recovery expert – but I’ve been around the block a few times – that there are some issues of fairness and perhaps better ways to assess a state’s true need for Federal assistance. A fair discussion of this topic, however, requires having a 360 degree perspective of all the issues involved. Perhaps it’s a good topic for a research journal?

Managing an Exercise Program – Part 9: Conducting an Exercise

Tim Riecker, The Contrarian Emergency Manager Leave a comment

This post is part of a 10-part series on Managing an Exercise Program. In this series I provide some of my own lessons learned in the program and project management aspects of managing, designing, conducting, and evaluating Homeland Security Exercise and Evaluation Program (HSEEP) exercises. Your feedback is appreciated!

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

Managing an Exercise Program – Part 10: Evaluation and Improvement Planning

Image

Since writing Part 8, the new HSEEP guidance (April 2013) was published. Thankfully this update didn’t dictate wholesale changes in the process we’ve all come to be familiar with. There were some subtle changes, such as changing ‘planning conferences’ to ‘planning meetings’, and stressing the involvement of elected officials in the exercise planning process. Similarly, I’m thankful that certain changes weren’t made – one of which was the proposed elimination of terminology of ‘discussion based exercises’ and ‘operations based exercises’. These general identifiers have become commonplace amongst exercise designers for decades. Upon having the opportunity to review a draft version of the new HSEEP guidance, I was quite opposed to that change.

Obviously conducting the exercise is the most exciting part of the process. It’s where all of your hard work comes together. That said, this is where it can all fall apart if you haven’t prepared property in the first eight parts and if you don’t pay attention to detail. I’m breaking exercise conduct down into four smaller parts: Set up, briefings, play, debrief. Remember that my intent with these postings is not to be fully comprehensive, but to emphasize areas where I have learned or have seen others struggle. In all facets of emergency management we need to continue a culture of sharing if we are to be successful.

Set up

The preparations for your exercise should have been finalized in Part 8 where you addressed logistical requirements. You should know where your exercise venue is, how people will be seated/where they will be working, the location and support for a SimCell (if you’re having one), and even matters such as parking, food, restrooms, and audiovisual equipment. Additionally, set up includes having print material available for all participants. When setting up, consider the flow of people, quite literally from the road to the parking lot, from the parking lot to the facility, into the facility through security, sign in, and to their first location for the exercise (this may be their only destination or it may be a large briefing room). Having conducted exercises in locations not familiar to some participants as well as these exercises being larger than the facility’s parking lot can handle, you must ensure that invitation packages have specific directions on how to get to the parking area and how to get from the parking area to the venue. Be sure that signage is prominently displayed. It may even help to have staff directing people. Be sure to also let them know what manner of identification is required to enter the facility. Again, signage is often necessary to get people from the building entrance to check in location and then to the meeting location. Signage for the refreshment area and restrooms may also be needed.

Have everything set up for people before they arrive. Use table tents, if needed, to help identify seating, and have copies of all necessary materials at their seating location. Materials like SitMans and ExPlans have a lot of content, so it’s nice to have these materials available for the early-comers to being reading through.

Be sure to set up early and test EVERYTHING. Make sure AV equipment and communications equipment are tested.

Briefings

Depending on the type of exercise you are conducting, there can be a number of briefings needed to take place before the exercise begins. Some of these briefings may need to take place well in advance, even a day or more before the exercise, depending on availability of audiences for each of them.

The new HSEEP guidance includes a specific briefing for elected and appointed officials. This is something to usually conduct well in advance of the actual exercise to outline the agenda/timeline of the exercise and what will be expected of them. They will likely want to interface with any media and should know what to say about the exercise. They may also want to ‘kick off’ the exercise at the player briefing.

Controllers, evaluators, and SimCell personnel should receive a briefing to ensure they are familiar with the exercise and their rolls. They all need to know what is expected of them and of the players, how and when to interact with the players, and what may need to be reported to their respective leaders and/or to the exercise director. Evaluators will need to become familiar with their assigned EEGs and what the expectations are for evaluating. Additional time will need to be spent with SimCell personnel to familiarize them with the equipment being used, how the MSEL is structured, and how injects will be delivered. If any actors are being used, they will also need to be briefed on their rolls and what their expectations are. Ensure that everyone is familiar with the safety word in the event of the necessity to immediately stop the exercise.

If observers are expected at the exercise, have a briefing ready for them. I suggest treating them like VIPs when possible (most of them usually are). Don’t just let them walk around to figure things out for themselves. Give them tour guides who can take them throughout the venue and talk about what is happening. Schedule their arrival when possible, and consider having an elected or appointed official there to greet them. Keep in mind that some observers will be very high tier VIPs. Vigilant Guard exercises are often visited by the head of the National Guard Bureau – a four star General. These visits are fantastic and very much emphasize the importance of the exercise, but they can also be a little disruptive. These high-tier VIPs will often come with their own entourage and/or security. They will want a brief tour and will stop often to shake hands and have pictures taken. They may even want a break in the exercise to speak. Try to be aware of these expectations up front if at all possible. If not, just go with the flow and be flexible.

The media is another form of observer. All media should be scheduled. Be sure to make it worth their while, where they can catch some video/picture footage of interesting activities. Maps and wall displays make for great footage as well. Remember operational security! If something is sensitive or classified, it should not be anywhere where it can be seen. The media should also be provided a tour and an opportunity to get a statement from the elected and/or appointed officials. They may also want to interview players. I generally don’t allow this without preparation of the specific player to ensure that the right messages are delivered. Also, be sure to let players know during the player brief that there will be observers and media coming through and what is expected of them.

Lastly, the player briefing. This briefing will introduce players to the purpose of the exercise, the ground rules, the facility, timeline, controllers and evaluators, and expectations. Players should be briefed on how and when to interact with the SimCell if you have one. It should always be reinforced that the exercise is ‘no fault’, and that they, individually, are not being evaluated, rather it is the plans, policies, and procedures they are using that are being evaluated. Lastly, players are briefed on the scenario. Be sure that the ExPlan/SitMan has the detailed scenario for their reference.

Exercise Play

Finally the moment of truth arrives! Remember that the purpose of the exercise is to accomplish the objectives, however, know that you may not accomplish all the objectives in the timeframe you have. Being flexible, and knowing you have to be flexible, are two very important aspects of running an exercise. Regardless of how well we think we’ve written an exercise, the responses to our prompts are entirely up to the players. They may accomplish the objectives faster or slower than you expected. Likewise, they may struggle a bit. This is where good exercise control comes in. Controllers/facilitators should observe the tempo of the exercise. Is everyone engaged? Are they needlessly overwhelmed and frustrated? Are they bored? Are they not following through on activities? All these observations should be reported back to the exercise director and SimCell so the tempo can be moderated.

Be sure to have contingencies in the event that players do or request the unexpected. Additional injects that are held back are always a good idea. You may need them to prompt certain activities in the event that players do not take the initiative to do so. Players may also ask questions or make requests of the SimCell that aren’t expected. While we can’t anticipate every need, we can be prepared for them. Have copies of plans, policies, procedures, and maps available to the SimCell. They may have to take some time to research and come up with an answer then get back to the individual. The SimCell manager should be smart when situations like this arise, however. They should always consider if a potentially inaccurate but reasonable answer is acceptable or if the answer must be completely accurate… or even if the answer is required for exercise play at all. There is no sense having SimCell personnel research an answer if the answer doesn’t much matter within the scope of the exercise.

Multi-day exercises may require the exercise staff to meet at the end of every day for a mini hotwash and to evaluate how the exercise is progressing. Where are the players in respect to where you expected them to be in your design of the exercise? Do you need to retool anything in the MSEL?

On large exercises, it helps for the exercise director to have an aide-de-camp, or assistant. Just like any individual in charge of a large operation, the exercise director is having their attention pulled in many directions and may simply not have the time to address everything, especially in large or multi-venue exercises. Having an assistant is also a great way to train up and coming exercise staff. Certainly consider the use of portable radios to help facilitate communication as well. The exercise director will spend most of their time managing and trouble shooting. They may also have to address some VIP concerns. Be sure to walk around to get a good sense for the exercise as a whole. Are evaluators properly positioned? Are controllers present and visible? Is the SimCell responding to questions adequately? Are players engaged and challenged? Perhaps most importantly, is the boss (the elected/appointed official) happy?

Evaluation is such an important aspect of exercise that I will cover it in its own section – Part 10.

Debriefs

Two significant debriefing activities should take place immediately following the exercise: 1) a player hotwash, 2) an exercise staff debriefing. Players should be led through a facilitated hotwash, reviewing the objectives of the exercise, with evaluators capturing their responses to if/how the objectives were accomplished. Encourage and capture responses both in the positive and the negative: i.e. what did we do well and what do we need to improve upon?

After the player hotwash, a similar process should take place with all exercise staff – controllers, evaluators, and SimCell. I like to not only capture their impressions of if/how objectives were accomplished, but I also like to discuss the conduct of the exercise, again capturing what went well and what needed to be improved upon.