Quality Management Risk Guide: Misinterpreting data presentation

The issue

Have you even seen a manager misunderstand data during a presentation? Such an obvious thing, of course you have! Why does this matter? With more virtual meetings and global communication challenges, there are fewer chances than ever to correct misunderstandings before resources are committed to the wrong activities.

There is an enterprise risk to not having quality professionals who are knowledgeable about such issues. Therefore, certifications are important. (See references at the end for more information.)

Learning objectives

Our intent here is to help illuminate the risks of misinterpreting data and give you references to use to avoid this kind of risk in your organization.

Kinds of risks

How do these risks appear? What do they look like? Being forewarned is forearmed, they say. Knowing what these risks look like will help you recognize them in time to make corrections. Remember your manager is human, and humans make mistakes, so being ready is the best way to keep your processes from being derailed.

Using statistics too soon

With all our technology today, one problem is applying statistical tools too soon. Once you have done some pre-screening of the data, and you suspect there is some kind of relationship (or some kind of disconnect) between sets of numbers, applied statistics provide the best way to confirm that your visual evaluation is right or wrong.  The number one rule in statistics is ‘plot your data’. Once plotted, you can look at it and pre-screen it to judge that something is either good or bad about it.

Since we humans often have trouble with seeing what we want to see and not seeing what we do not expect, solid and unforgiving statistical analysis will allow us to avoid the hard-wired biases we are all born with (see biases in the references at the bottom). Doing such analysis after visual review is important, as you do not want to waste your limited human resources on frivolous analyses.

Charting problems

Too many times an employee labors over a presentation of data using graphs and charts, only to have the executive audience misinterpret the data. Perhaps you have been the presenter, perhaps you have been the executive. Regardless, there are some common risks of poor data representation outlined by Edward R. Tufte (See references).

Unmarked chart items

Having items on the chart or graph that are not identified creates friction in the minds of the observers. Instead of looking at your data, they are asking themselves, “What is that thing there? Why is it on the chart? Is it important? If so, why isn’t it identified?”

Avoiding this risk is easy: if it makes a point you think is important, put it on the chart and label it! Otherwise, leave it off.

Comparing differently sized time slices

Have you ever had the experience of seeing two graphs for comparison purposes, and then find out they represent two different periods of time? For example, comparing last year (full year) to this year (year-to-date). This is often done without identification. Even if it is identified, why on Earth would comparing two different time-window-sizes show anything useful? This kind of comparison is likely to misrepresent the data and cause poor decisions.

Nonalignment of Image vs. data

Have you ever seen a pictographic representation of one-dimensional data with two dimensional pictures? Here is the issue: data values increase in a linear fashion (call it x), but the image enlarges in area, as in x-squared. To the eye, double the size of ‘x’ from 2 to 4, and you change the image size ‘x-squared’ from 4 to 16! You eye sees a four-fold increase, but the data only has a two-fold increase. This is misrepresentation of data by a chart. It may be unintentional, but it is misleading. (See references on Misleading graphs)

Exaggerating variation

Often a chart is used to show the level of a metric like gradually rising sales. While this is needed, BOTH the ‘level’ and the variation of a metric may be important. Rarely are both ‘level’ and variation shown. And while the trends in ‘level’ are obvious, increasing or decreasing variation is less obvious. If you are about to invest $100,000 in a stock, you need to be interested in both what has been the level of stock value, and how variable is the valuation.

Consequently, making a chart that diminishes or exaggerates the variation of a metric can mislead. There are several metrics of variation, such as range or standard deviation. Used well, these can help to put variability in perspective.

Color problems

This is as simple as it is obvious. Someone using mostly muted colors on a chart with one item picked out in a bold color will misrepresent the data. Our eyes and attention are drawn to the different color, and we can miss other more important aspects of the chart.

Using red and green may have to be avoided, since color blindness occurs in 8% of men and 0.5% of women. Red and Green color blindness is the most common, with 99% of the color blind having this type. Using red and blue can avoid confusion here.

Distracting chart junk

This is a term Edward R. Tufte uses extensively, and it is highly descriptive. Chart junk are items that add nothing to the message of the data and distract from content. Excess lines, useless details, background colors, all are chart junk. Having lines to allow proper reading of the chart is good and cannot be considered chart junk. Tufte recommends against using a legend, suggesting using lines to label portions of the chart instead of a boxed legend. A legend can be hard to read, and cause eye fatigue looking back and forth over a chart to interpret it.

Which way is good?

Moving beyond Tufte’s tips on charting problems, a tip I learned from one large aerospace firm is to always mark which direction on the chart is ‘good’ with an arrow. Sometimes up is best, sometimes down is best, sometimes moving farther from the middle is bad. Do not expect your audience of executives to instantly know which direction is good. Tell them.

Use time on the horizontal axis

This is expected, so common that if a chart is based on variation of a slice in time and time is not charted, people will expect the horizontal axis to be ‘time.’ Histograms do this, they show variation from the middle, but are based on a slice of time and do not show time anywhere. (See histograms in references.) If time is shown make it horizontal; if time is not shown, say that.

Summary

So, you now know how data may be misinterpreted. Having a certified quality staff may not eliminate these kinds of issues, but it will reduce them. It is harder to fool people who have studied how to present data in a quality manner. Think about how much these misinterpretations can cost you and decide if you want to have data presentation expertise available on demand.  

References

·         ASQ Certified Manager of Quality/Organizational Effectiveness (CMQ/OE) link: https://asq.org/cert/manager-of-quality

·         Cognitive biases: https://thedecisionlab.com/biases

·         Edward R. Tufte’s book on “The Visual Display of Quantitative Information (2nd Edition)”: https://www.amazon.com/gp/product/0961392142/ref=dbs_a_def_rwt_hsch_vapi_taft_p1_i0

·         Misleading graphs: https://www.statisticshowto.com/misleading-graphs/

·         Histograms: https://en.wikipedia.org/wiki/Histogram