Developing Metrics

determining where you are  

Check against the critical components  

Where we want to be: should be a well-defined, robust metric (or metric program) 

How we get there is using this book

--Designing Your Metric  

Developing the Root Question 

Defining the question 

Determining what type of answer is necessary 

Lost?  Check the Answer Key for help 

Remember not to throw away anything-byproducts are a valuable benefit 

Common errors 

Abstract Design 

Drawing the picture of the answer 

Test for comprehension and alignment 

Identify the information needed (check against Answer Key) 

Identify the measures needed (check against Answer Key) 

Identify the data needed

Common errors 

Using the Answer Key 

Customer view 

Worker view 

Leadership view 

Business view 

Common errors 

Finding the Sources of Data 

Quality Check:  Where does the draft metric fit in the Answer Key?

--References  

4.   Collection: An Implementation Guide

--Plan for Collection:  Using the Metric Development Plan 

Identify and fill gaps in design 

Determine players 

Determine availability of data;  work with data sources to ensure the data is available in the form needed 

Plan for collection and building of metrics           

Determine how you will deal with missing data 

Estimate? 

Create mechanism? 

Use other organization's data? 

Skip this measure?

--Collect Data (samples of each tool and help build out for particular use) 

Survey tools 

Check sheets 

Databases 

Spreadsheets

--References

--Using the Implementation Guide to Analyze Existing Metrics/measures            

--Example of Analyzing Existing Metrics                                       

--Most Metrics Are Actually Measures, and Some Unfortunately Merely Data 

--How It All Feeds Back to the Answer Key   

5.   Tools to Help You Create Metrics 

--The Answer Key 

--Software tools                                         

Tableau                                         

Excel

--Art Class

--Big Data

--Analytics

--Books  

6.       Analysis: Interpreting and Understanding the Results

--How to Chart/Graph Your Measures 

Ranges vs. targets 

What type of chart? 

Snapshot?  

Over time?  

Comparing month to month or running?    

--How to Communicate the Information

--How to Publish Your Metrics

--References

7.       Reporting I: Telling the Story

--Reporting Out 

Facts - indicators only, so make sure you're showing what's indicated 

Fiction - don't spin the data so it tells a particular story a particular way 

Nuances - the references, size, order, etc. your story can be spun or skewed.  Ensure you're telling the story accurately and honestly.

--Managing Up 

Critical factors to ensure metrics are not abused/misused 

Not showing data 

Protecting anonymity

--Letting Go 

Metrics aren't supposed to last forever 

Realize your work may serve its purpose and then go away 

It's not yours; you are not the key user, the process owner is 

Transitioning ownership to the process owner 

--Annual Reports: Where Do They Fit In? 

Same as a sales pitch 

Not a process improvement or problem resolution metric 

Could be a metric for gaining support, but normally it's purely a marketing tool

--Proactive Metrics vs. Reactive 

Real-time data flow--be careful of what you wish for 

Costs 

Harder to control reaction 

Ensure really needed 

Just because the data is "old" doesn't mean you're being reactive 

It's how you use the information

--References

8.   Reporting II: Dashboards, Scorecards, and Report Cards

--The Balanced Scorecard and Why It's Not the Best Place to Start        

What it is                                         

Who needs it                                       

Who doesn't need it

--Dashboards: Catch-all for the Lazy?                                           

What is normally meant by dashboard                                        

What a real dashboard should/could be 

--Report Cards                                          

Why Report Cards are different                                        

Modeled after education's report cards

--Combining All Three into a Complete Package

9.   Automation: Having Results Come to You

--Benefits of Automation                                          

How to benefit from automation                                        

Less human interaction, the less chance of human error

--Dangers of Automation                                         

Less oversight                                        

Changes cost more                                       

Reluctance to discard

--Big Data Analytics

--Success Stories 

--Current Tools                                          



10.   Analytics: Measuring the Metric

--What Is Analytics?                                          

Why another term?                                        

How is it making a difference?

--Can Our Data Tell Us What Decisions to Make and When?                                          

Data driven vs. data informed

--Looking for the Easy Way Out

--The Promise of Big Data

11.    On Metrics and Strategic Plans: Measure the Right Things

--Vision, Goals, and Objectives     

If "SMART" they have to be "measurable"     

Baselines to show progress, and finally success         

Measure success, progress, and alignment, comprehension, and necessity

--Measures of Success     

Progress indicators     

How the difference blurs the lower you get     

How progress indicators equate to measures of success (MoS) 

--Common Errors

--References 

12.       Troubleshooting: Being Prepared for the Normal Hiccups

--Those Who Already Have the Wrong Metrics 

Those who have already bought into a dashboard solution that doesn't fit their root question 

The perception of lacking authority and/or autonomy to move forward 

The perception of lacking resources

--Metric Abuse

--The Wrong Root Question

--Scattered vs. Focused 

--References

Appendix

--Forms, templates, checklists