In this last article of the Return on Investment series, I’ll be discussing the investments and benefits of preparedness exercises to help organizations determine their return on investment – or ROI. The series has followed the model of the five POETE elements (Planning, Organizing, Equipping, Training, Exercising). The inspiration for the series was a piece I wrote called Measuring Return on Investment in Emergency Management and Homeland Security: Improving State Preparedness Reports. If you haven’t had the opportunity to review the earlier articles in the series, they are linked below:
We conduct preparedness exercises for two main reasons: 1) to test plans and procedures, and 2) to provide people with an opportunity to practice their roles and responsibilities. Exercises can be stand-alone activities or integrated into training and education through scenario-based learning. In the US we use the Homeland Security Exercise and Evaluation Program (HSEEP) as our model for both the macro and micro levels of exercise management. If you are interested in an in-depth look on HSEEP and its components, you can check out an earlier series I did called Managing an Exercise Program.
As with most major activities we do, exercise-related tasks can be divided out into program management (the macro level) and project management (the micro level). Since we usually examine ROI for individual activities, we will focus on the micro level of exercises, that is the project management piece, or individual exercises, to identify specific costs (investments) and benefits.
Conducting an exercise takes a fair amount of preparation. The more complex the exercise, the greater period of time it should take to prepare. Complexity of an exercise is measured by a few different factors – the number of participants involved, the span of time for the exercise, the complexity of the tasks/plans being exercised, and the number of locations being exercised. Most of us who have been involved in emergency management and homeland security for a while have seen the full gamut of exercises – from discussion-based exercises like table top exercises, workshops, and seminars; to operations-based exercises like drills, functional, and full-scale exercises. Often we view functional and full scale exercises as being the most complex, however I’ve been involved in table top exercises and workshops which have involved significant efforts.
Up front, the most significant investment any organization can make in an exercise is personnel time. All exercise efforts will have a lead planner, and most will be supported by a planning team. If the exercise involves only one organization, that planning team will typically involve only internal people, while multi-organizational exercises should involve some measure of representation from every organization, either directly or indirectly. Planning an exercise requires a great attention to detail, drafting and editing of documents, and arranging of logistical matters. Experience helps, so those who do this less often will typically require more time to do it. This is why many organizations hire consultants (like me!) to help them with exercises.
During the planning phase for the exercise, you may have some associated costs, such as meeting space, food, and travel for planning meetings. You may also have these costs for the exercise itself which should be identified during the planning phase. It is also important to identify any costs associated with audio-visual equipment, communications equipment (including internet connectivity), and even things as simple as name badges and signage.
For the exercise itself, personnel costs are still significant. You must not only consider the time of all participants (as well as potential travel costs), but also the time of your exercise management staff – an exercise director, controllers, evaluators, and possibly staff for a simulation cell. Again, experience helps to support a successful exercise, so if you don’t have the depth of experience in your organization, consider hiring consultants for the conduct and evaluation of the exercise as well. Either way, exercises can be significant investments.
Once the exercise is complete, the activity isn’t over – and neither are the costs. The evaluation team needs to draft the after action report (AAR), and conduct an AAR meeting with the planning team and principal participants to ensure that everything was captured accurately. Once the AAR is finalized, action items identified in the AAR are assigned to responsible parties to address improvements. These improvements are generally not considered part of the cost of the exercise itself, but rather part of your general preparedness costs (these will all fall within the POETE elements).
While exercises come at no insignificant cost, the benefits are tremendous – if the exercise is done properly. A well designed, conducted, and evaluated exercise provides better outcomes and benefits. The AAR should reflect not only best practices that should be continued, but also areas for improvement which should be addressed to enhance preparedness. Any of these, as mentioned in the last paragraph, can fall within the five POETE elements – Planning, Organizing, Equipping, Training, and Exercising. While each of these certainly have costs associated with them, the benefit from the exercise was identification and documentation of need. Perhaps you are exercising a new active shooter response plan and through the exercise realize that a certain procedure was based on poor assumptions – if this plan was put in place without being exercised, the outcomes in a real life event could have been catastrophic. It’s better to identify these issues through an exercise so they can be addressed with much less cost.
As I mentioned earlier, another reason to exercise is to provide participants with an opportunity to practice plans, procedures, or skills in a safe and structured environment. While there is a great deal of routine to what we do in emergency management, homeland security, and public safety, there are certainly activities that we don’t do very often, resulting in degradation of skill over time. Many of these activities, though, are absolutely critical when needed, which means that we must give practitioners ample opportunity to practice and apply what they have learned through training. The benefits of this, depending on the activity, can include increased efficiency (time), reductions in injury and loss of life, and proper use of equipment and protocols. Additionally, there are benefits to getting people to work together in these activities, especially for those who don’t usually work together. Emergency management, after all, is about collaboration.
Because of the wide range of things we exercise, it’s up to you to examine what your investments and benefits might be. At EPS, we can help you with designing, conducting, and evaluating exercises; identifying potential costs and benefits of an exercise; and other preparedness activities. We’re happy to help!
Feedback from this return on investment series of posts has been very positive, which I greatly appreciate. We also got some good dialogue across all mediums including the blog home page (www.triecker.wordpress.com) and various LinkedIn discussion groups – some of which provided some excellent additional ideas on how to better capture information on investments and benefits. The challenge remains to not only identify these, but to convert them into meaningful information for decision makers, which usually involves currency values. Thank you, as always, for your time and attention.
© 2015 – Timothy Riecker
The Contrarian Emergency Manager 
One thought on “Gauging Return on Investment in Preparedness: Exercises”