Measuring Return on Investment Through Key Performance Indicators

Return on investment (ROI) is generally defined as a measurement of performance to evaluate the value of investments of time, money, and effort. Many aspects of preparedness in emergency management offer challenges when trying to gauge return on investment. Sure, it’s easy to identify that m number of classes were conducted and n number of people were trained, that x number of exercises were conducted with y number of participants, that z number of plans were written, or even that certain equipment was purchased. While those tell us about activity, they don’t tell us about performance, results, or outcomes.

More classes were conducted. So what?

We purchased a generator. So what?

The metrics of these activities are easy to obtain, but these are rather superficial and generally less than meaningful. So how can we obtain a meaningful measure of ROI in emergency preparedness?

ROI is determined differently based on the industry being studied, but fundamentally it comes down to identifying key performance indicators, their value, and how much progress was made toward those key performance indicators. So what are our key performance indicators in preparedness?

FEMA has recently began linking key performance indicators to the THIRA. The Threat and Hazard Identification and Risk Assessment, when done well, gives us quantifiable and qualifiable information on the threats and hazards we face and, based upon certain scenarios, the performance measures need to attain certain goals. This is contextualized and standardized through defined Core Capabilities. When we compare our current capabilities to those needed to meet the identified goals (called capability targets in the THIRA and SPR), we are able to better define the factors that contribute to the gap. The gap is described in terms of capability elements – planning, organizing, equipping, training, and exercises (POETE). In accordance with this, FEMA is now making a more focused effort to collect data on how we are meeting capability targets, which helps us to better identify return on investment.

2021 Emergency Management Performance Grant (EMPG) funding is requiring the collection of data as part of the grant application and progress reports to support their ability to measure program effectiveness and investment impacts. They are collecting this information through the EMPG Work Plan. This spreadsheet goes a long way toward helping us better measure preparedness. This Work Plan leads programs to identify for every funded activity:

  • The need addressed
  • What is expected to be accomplished
  • What the expected impact will be
  • Identification of associated mission areas and Core Capabilities
  • Performance goals and milestones
  • Some of the basic quantitative data I mentioned above

This is a good start, but I’d like to see it go further. They should still be prompting EMPG recipients to directly identify what was actually improved and how. What has the development of a new plan accomplished? What capabilities did a certain training program improve? What areas for improvement were identified from an exercise, what is the corresponding improvement plan, and how will capabilities be improved as a result? The way to get to something more meaningful is to continue asking ‘so what?’ until you come to an answer that really identifies meaningful accomplishments.

EMPG aside, I encourage all emergency management programs to identify their key performance indicators. This is a much more results-oriented approach to managing your program, keeping the program focused on accomplishing meaningful outcomes, not just generating activity. It’s more impactful to report on what was accomplished than what was done. It also gives us more meaningful information to analyze across multiple periods. This type of information isn’t just better for grant reports, but also for your local budgets and even routine reports to upper management and elected officials.

What do you think about FEMA’s new approach with EMPG? What key performance indicators do you use for your programs?

© 2021 Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC®

Emergency Management Budgets

Last week there were some posts circulating around Twitter expressing some considerable dismay about emergency management budgets. While I obviously agree that emergency management programs should be better funded, there is some important context to consider when looking at (most) emergency management agency budgets in the US.

While jurisdictions having emergency management programs provide some measure of funding, typically the largest quantity of funding comes from federal grant programs, with the most significant grant for operational expenses being the Emergency Management Performance Grant (EMPG). EMPG is part of the Homeland Security Grant Program (HSGP) and is budgeted each year in the federal budget with administrative responsibilities in the hands of FEMA. States are the grantees of EMPG. While a considerable amount of the funds are retained by states, there is a requirement for a certain percentage to be applied to local emergency management programs. States have different models for how the funds are allocated – some states award funds directly to county/local governments (subgrantees), others spend the funds on behalf of the subgrantees through the provision of direct services to county/local governments. Many states also use a hybrid of the two models. Those receiving an allocation of EMPG are ideally accounting for it in their published budgets, but we should be aware that some releases of budget information may not include EMPG numbers.

There are also additional grant funds available to county and local governments to support an array of emergency management and emergency management-related programs. These include hazard mitigation grants, the Urban Area Security Initiatives (UASI) grant, Secure the Cities, and others. Yes, a lot of these funds are targeted to more ‘homeland security’ types of activities, but we should also recognize the considerable overlap in a lot of EM and HS. I took a small sample of a few mid to large sized cities (mostly since they have established and funded emergency management offices), seeing ratios of 1:3 to 1:4 for local share funding compared to grant funding (this did not include COVID-related supplemental funding). Of course, you may see numbers significantly different in your jurisdiction.

I’ll also suggest that activities across many other local government agencies and departments support some measure of emergency management. While a lot of these expenditures may not have the input of an emergency management office, there are a variety of local infrastructure projects (hopefully contributing to hazard mitigation), health and human services investments (mitigation and preparedness), code enforcement (mitigation), and others that do contribute to the greater emergency management picture for the jurisdiction. In fact, some of the funding allocations received by these agencies may be through discipline-specific emergency management grant programs, such as those which may come from US DOT or CDC/HHS.

Overall, emergency management funding tends to be a lot larger than the casual observer may think, though even a budget analyst would require some time to identify how it all comes together, especially for a larger jurisdiction that tends to have larger departments, more complex expenditures, and more grant funding. As mentioned, I’d still love to see more direct funding allocations for emergency management programs, especially as emergency management can hopefully direct efforts where and how they are needed most within their communities. I’m also hopeful that officials leading different programs at the local level are coming together to jointly determine how best to allocate federal funds (obviously within the grant terms and conditions), even if they are coming from different federal and state agencies and being awarded to different local departments, with a goal of addressing local threats, hazards, and capabilities in the best ways possible for communities.

While what I wrote is a broad-brush example of how emergency funding is allocated across much of the US, different states do administer grants different. It can be as simple as I’ve outlined, or a lot more complex. We also have a lot of examples of the haves and have-nots, with many smaller jurisdictions being left woefully behind in funding. I’d love to hear what the funding situation looks like for your jurisdiction. Also, for those not in the US, how are your local programs funded?

© 2021 Timothy Riecker, CEDP

Emergency Preparedness Solutions, LLC®

Emergency Management Grants – Promoting Planning Standards

We know that good emergency plans are the cornerstone of preparedness.  Often times it is local governments that have difficulty putting quality plans in place because they don’t have knowledgeable personnel or funds available to make this happen.  This gap is critical since we know that all disasters begin and end locally, so quality local plans are an imperative.

States provide financial assistance to local governments through a local allocation of the Emergency Management Performance Grant (EMPG), which is an annual grant program through FEMA/DHS as a component of the Homeland Security Grant Program (HSGP).  While there is always some variance in the goals or focus of EMPG, the overall concept and allowable costs are fairly static and the emphasis is always on preparedness.

Preparedness, however, encompasses a lot of activities.  The best breakdown is POETE – Planning, Organizing, Equipping, Training, and Exercising.  Just from this we can see a lot of opportunity to spend money on a lot of needed activities.  Planning, however, regularly needs to be revisited.  While funding the other activities may be important, they mean very little without a quality, up to date plan.  All preparedness activities should relate somehow back to the plan, such as equipment and training efforts to shore up capabilities identified for need through the planning process.  This applies to everyone by the way – federal, state, and local governments; private sector; and not for profits.

How can states (or any other grant or budget managers) continue to emphasize the importance of planning?  I’ve recently seen a best practice by the State of New Hampshire which is similar to the federal administration of the Community Development Block Grant (CDBG) programs.  First, they make funds available for, and only for, planning.  This includes new plans and plan updates.  Once plans have been developed that meet their standards, then additional funds can be requested for supporting preparedness activities.  This building block preparedness approach helps provide targeted funds solely for plan improvements while helping to ensure that subsequent funds are provided for activities that associate with the plan and addressing or identifying (by way of exercises) gaps.  While it can be a bit cumbersome, I think it’s a great model for promoting preparedness the right way.

Thoughts?

©2014 – Timothy Riecker

A Disasterous Trend: Cuts in Preparedness Funding

This post was initially inspired by an article from CBS News on funding cuts to disaster preparedness programs.  These cuts go further and deeper than the current sequester cuts we are now seeing.  These cuts are a dangerous and disastrous trend.  To quote the article…

“In fiscal year 2010, Congress appropriated $3.05 billion to FEMA for preparedness grants designed to strengthen “our nation’s ability to prevent, protect, respond to, and recover from terrorist attacks, major disasters and other emergencies, …. In fiscal year 2012, that appropriation was less than half that figure – $1.35 billion. The same trend could be seen in FEMA pre-disaster mitigation grants, which fell from $100 million in 2010 to $35.5 million two years later.”

Have all the terrorists gone away?  Has Mother Nature stopped having temper tantrums?  Have stupid people stopped doing stupid things?  I don’t think so!  So why the cuts?

Let’s put some things in perspective… On one hand, we do need to have a bit of fiscal prudence and restraint.  GAO reports have repeatedly shown that many state and local governments are simply not spending down the grant funds they have been allocated.  DHS grants are backed up several grant years with unspent funds.  That said, as we peel back the layers of the onion, there are certain facts that need to be mentioned.  Why aren’t they spending the money they have been given?  First, grant periods have generally been too short.  The most significant reason for this is the inefficiency of bureaucracy we live in.  Follow this trail… The federal fiscal year begins October 1st.  The budget gets passed at some undermined point around that.  DHS, along with all the other agencies, get their allocations.  They then need time to formulate their grant guidance for the funds going to states and locals.  By the time states see this grant guidance and their respective allocations it’s usually close to the end of the second quarter of the federal fiscal year.  States then have to formulate their own grant guidance as they pass through funds to locals.  All this bureaucracy delays the grant year about six months.  Recognizing that nothing could be done about the bureaucracy, DHS finally extended grant years only recently, giving folks a more reasonable amount of time to spend the money.

Another reason why grant funds are slow to spend is that in most cases the grantees don’t actually ask for the money, therefore they don’t have a budget prepared beforehand.  DHS distributes funds based upon a formula.  While an application exists, it’s nothing more than an afterthought and formality.  That leaves states and locals with a pile of cash and no plan on how to spend it.  Here lies the beginning of the breakdown in accountability.  Now most folks will say that it’s easy to spend money.  In government, not so much.  Especially when you consider a few factors: 1) every level of government has spending rules (accountability is a good thing, but that can get in the way of efficiency when RFPs have to be issued for darn near everything); 2) a great deal of equipment was purchased in the big push of funds immediately surrounding 9/11 – what else do we need?; and 3) grants are restricting what funds can be spent on (i.e. there are limits on personnel (salary) expenses, and the purchase of disposables and maintenance costs of equipment – which are of particular importance for exercises).

So governments don’t have a lot of time to spend the money and face a few obstacles in getting the money spent.  But how is this a factor of cuts?  One reason for these cuts is that Congress is seeing that states and locals have a lot of money left over going back several grant years.  Failing to realize the whys and wherefores of it all, they are simply giving less money (because, to them, it’s not needed – but nothing could be further from the truth!).  They are also looking to reduce spending overall, as the article cites, and that’s a hit that will impact nearly everyone.

Taking a look at the grantees, however, there are a few criticisms.  Better and more proactive fiscal planning needs to be implemented.  Costs should be forecasted out several years to better anticipate needs.  They may, sadly, have to trim programs and streamline operations (although most emergency management programs certainly are not living in the lap of luxury).  They also need to be more creative with the declining funds they receive, especially through partnerships and regionalization.  An area doesn’t need to be regarded as a UASI or Catastrophic Planning Zone to work cooperatively as a region, which should include some pooling of funds for collective projects.

What can be approached regionally?  Most preparedness efforts fit well into that category: planning, training, and exercising.  Think about it, you work with your neighbors all the time and disasters don’t seem to stop at the county line, so why not make your cooperation more effective and efficient?  In the absence of regional catastrophic planning, which most areas don’t need to do, consider planning for some credible worst case scenarios and cascading impacts such as flooding and mass care.  Obviously regional mutual aid planning is essential.  How about working with your public health partners?  What about the private sector – how can you strengthen your relationships with them?  Regional planning conferences are a good start!  Regionalized training is obviously a no-brainer and regional exercises are essential making sure that the planning and training are effective and to give folks an opportunity to practice what they have learned.  Lastly, speaking as someone who has experience working for government and as a consultant, in many cases it’s actually more cost-effective and easier to coordinate regional preparedness activities by hiring a consulting firm, some of which have proven experience and expertise in working with the multiple stakeholders that a regional effort would include.

As we face reduced funding, we have to be more creative, cooperative, and communicate specific needs on a regular basis up the chain of government.  If you are with county or local government, let the state know what your needs are.  And don’t just tell them once – be sure to repeat yourself – not in an annoying wintertime house fly kind of way, but when the appropriate opportunity presents itself.  Make sure that you show justification for your needs through after action reports and documented strategies and plans.  Ask the State to take these needs up to federal partners – and when you have the opportunity to speak with these federal partners directly, take advantage of it; be they representatives of FEMA or your local representative of Congress or US Senator.  Remember to be specific and cite the need.  Don’t complain but be direct.  With funding that emergency management programs simply receive without asking being on the decline, we need to be proactive about receiving funds.

Emergency management and, to a greater degree homeland security, have been fortunate to have a good deal of funding over the last decade.  There has been so much money, though, with such short time lines, that things haven’t been done as well as they should have.  Now is the time to re-tool and reexamine how we do business.  Conduct needs assessments to determine what should be focused on and build upon community partnerships.  Consider what the community as a whole – the citizens – are willing to help in preparedness; as well as the private sector.  Whole-community partnerships have perhaps never been so important as they are now.

Managing an Exercise Program – Part 5: Securing Project Funding

This post is part of a 10-part series on Managing an Exercise Program. In this series I provide some of my own lessons learned in the program and project management aspects of managing, designing, conducting, and evaluating Homeland Security Exercise and Evaluation Program (HSEEP) exercises. Your feedback is appreciated!

Managing an Exercise Program – Part 1

Managing an Exercise Program – Part 2: Develop a Preparedness Strategy

Managing an Exercise Program – Part 3: Identify Program Resources and Funding

Managing an Exercise Program – Part 4: Conduct an Annual Training & Exercise Planning Workshop.

Managing an Exercise Program – Part 5: Securing Project Funding

Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences

Managing an Exercise Program – Part 7: Develop Exercise Documentation

Managing an Exercise Program – Part 8: Preparing Support, Personnel, & Logistical Requirements

Managing an Exercise Program – Part 9: Conducting an Exercise

Managing an Exercise Program – Part 10: Evaluation and Improvement Planning

Timothy Riecker

HSEEP Cycle

Nothing moves without funding – nothing.  Without funding, good ideas are nothing more than that – ideas.  I’ve seen many ideas and initiatives die before they even made it to the proverbial chopping block, simply because of a lack of money to support them.  This is the time of year when we see a lot of new ideas.  In public and private sectors alike, our leaders, motivated either by legislative writ or self driven compulsion, give us an annual speech to ring in the new year.  These speeches come with lofty ideas – many of which we never see get off the ground because funding is never allocated.  So where do we get money to conduct exercises?

Timothy RieckerAdmittedly, my expertise lies in government and the funds available to build and sustain emergency management programs – not so much in the private sector and not for profit areas, but I’ll give these a crack.  Public sector funds consist largely of the Homeland Security Grant Program (HSGP).  HSGP funds a myriad of emergency management and homeland security grants including the Emergency Management Performance Grant (EMPG), the Urban Area Security Initiative (UASI), Operation Stone Garden (OPSG), and others, including Citizen Corps which is now no longer a separate grant program, but instead an optional allocation which states may choose to provide.  Generally, exercises, and the expenses associated with all steps of the exercise program and project, are allowable expenses for all these grant programs.

How do you get these funds?  Well, the bad news is that if you don’t already receive them, you probably can’t.  There are some allocations, like Citizen Corps, which may be granted to jurisdictions by the state, but with this example you need to build a local Citizen Corps program and exercise only that program with any dollars you receive.  If you do receive some of these HSGP funds, the challenge is in reallocation of dollars that are probably needed elsewhere, and budget increases are probably out of the question.  So here’s where we have to get creative.  Reach out to the folks who were involved in your TEPW – all those agencies and organizations.  Try to gain consensus on the need for an exercise (or building-block series of them).  Most or all of these agencies may have an interest if you had a successful TEPW and managed to combine some exercise initiatives.  Don’t forget your private sector partners, either – especially if they are members of your Local Emergency Planning Committee (LEPC), as they may take special interest in preparedness.  Be sure to have a plan and make a business case.  The TEPW that you just conducted (see the previous post in this series) provides you with an excellent statement of need and a plan to address it.  You may have additional supporting documentation like after action reports, which can help add some context to your need for exercise funding.  Build a budget and know how much to ask for.  If each agency and organization can contribute a portion, that will all add up fairly quickly.  Don’t forget the possibility of sponsorships, as well.  I once managed to secure lunches to be provided for all exercise participants (about 150 of them!) in exchange for a medical supply vendor setting up in a near-venue area and giving a presentation during lunch, including the opportunity for folks to try out some of their equipment.

In the private sector, fighting for budget can be tough – especially when it’s not tied to a profit center.  My advice here (and again I have limited experience in this area, so if you have any ideas, please post them!) is, similar to the public sector, 1) make a good business case for it (i.e. improved safety, response coordination, and decreased down time all minimize the loss of revenue), 2) have a plan, and 3) if you are just starting an exercise program – start small.  Let the executives see the potential that can be gained from larger investments in your program.  Similarly, if you can partner with a local public safety exercise, be sure to invite your executives to see how it goes and be ready to explain the benefits to your company.

As for not for profits, largely it’s a combination of the public and private sector tips.  Also, consider seeking grants from foundations for the specific purpose of preparedness.  Don’t just limit yourself to local foundations, either.  Their may be companies that specialize in first responder or emergency equipment that may have a foundation.  I would guess that their foundations would have a particular interest in preparedness activities.

Overall, be sure to plan early.  Don’t expect to seek funding for an exercise that you have planned for a couple of months down the road.  It may take as long as a year to get your financial ducks in a row.

As always, if anyone has any additional thoughts or ideas, I’d love to see them!

Coming soon… Managing an Exercise Program – Part 6: Conducting Exercise Planning Conferences.  It’s more than just meetings!