Research work · 2016
The Value of Scatter Plots
Kristin Lynn Sainani
PM&R
Abstract
Authors frequently report correlation or regression statistics to quantify the linear association between 2 numeric variables. These statistics always should be accompanied by scatter plots. Scatter plots visually display the relationship between 2 variables and instantly provide the reader with key information, such as the shape of the relationship, the sample size involved, and the influence of outlying data points. "Among all the forms of statistical graphics, the humble scatterplot may be considered the most versatile, polymorphic, and generally useful invention in the entire history of statistical graphics," write Michael Friendly and Daniel Denis 1. This article gives tips for drawing scatter plots and also gives 3 examples of how scatter plots can reveal problems with correlation and regression statistics. Lines on plots can fool the eyes, as illustrated in Figure 1. This figure shows 2 variables (x and y) that are completely unrelated; to fool your eye, I added arbitrary diagonal lines to the plot. The lines makes it appear as if there is a positive relationship between x and y (top panel) or a negative relationship (middle panel). But when you remove the lines (bottom panel), it becomes easier to see that x and y have no relationship. Commonly, when drawing scatter plots, authors will superimpose a linear regression line to help visualize the relationship between the variables. These regression lines, however, can trick your eyes in the same way—causing you to perceive stronger relationships than actually exist. Lines superimposed on a scatter plot can be misleading. A +90° line has been added to the scatter plot in the top panel, which causes your eye to perceive a positive relationship. A −90°line has been added to the scatter plot in the middle panel, which causes your eye to perceive a negative relationship. There is no relationship, however, between x and y, as shown in the bottom panel. I advise authors to examine scatter plots without overlay lines initially and then to have the computer superimpose smoothing lines rather than linear regression lines. Smoothing lines, such as LOESS (or locally weighted scatter plot smoothing) curves, are readily available in most modern statistical packages. Smoothing lines are flexible and conform to local patterns in the data. If the smoothing line reveals that the relationship between variables is not a straight line, then the authors should present the plot with the smoothing line in their paper. If the smoothing line reveals a straight-line relationship between variables, then the authors can present a plot with a linear regression line. When the smoothing line or regression line is roughly horizontal, this indicates that the variables are unrelated. Smoothing lines require the user to input a degree of curvature; generally a medium amount of curvature (not too much, not too little) is recommended. For more information on smoothing lines, see the accompanying In-Depth box. Authors also should avoid manipulating the axes in ways that visually exaggerate the effect. Examples of such manipulations include stretching or truncating the X or Y axes. Correlation coefficients and linear regression statistics assume that the relationship between 2 variables is linear (follows a straight line). This assumption is often violated, however, and should be checked by plotting. In an excellent example of the use of scatter plots, the authors of a cross-sectional study of 187 cognitively normal adults 2 revealed a strong quadratic relationship between brain white matter volume and age. I used an online digitizer (http://arohatgi.info/WebPlotDigitizer/app/) to extract the values of the data points from their graph so that I could recreate the plot and reanalyze their data. Had the authors simply calculated a correlation coefficient (or linear regression coefficient) without adequately plotting their data, they would have concluded erroneously that white matter volume and age are unrelated, because there is no linear correlation between the 2 variables (r = .03, P = .70). Figure 2 shows plots of their data. There is a strong quadratic relationship, with white matter volume increasing in volume until about 50 years of age, and then decreasing thereafter (top panel). When the authors added an age-squared term to their regression model, the age-squared term was highly significant (P < .0001), thus confirming the quadratic relationship between age and white matter volume. Because in this study the white matter volume did not strictly increase or decrease with age, a straight line fit to the data misses the parabolic relationship between the variables (bottom panel). Example 1: Data were extracted from a cross-sectional study of 187 cognitively normal adults 2. When plotting the data with a smoothing line, a strong quadratic relationship is revealed (P < .001). The bottom panel shows that there is no linear correlation between the variables (r = .03, P = .70). Unlike box plots or bar charts, scatter plots show each individual data point, which gives readers more insight. Scatter plots visually remind readers of sample size limitations and reveal instances when a purported correlation rests on just a handful of observations. In a 2014 study 3, 13 acne-prone young men consumed varying amounts of cocoa at baseline; a week later, a dermatologist measured their facial acne. The authors concluded that there was a "medium-strength positive Pearson correlation coefficient between the amount of chocolate each subject consumed and the number of lesions each subject developed on day 7 (r = .314)." The authors failed to mention that this correlation was not statistically significant. Commendably, they did provide a scatter plot of their data—from which I was able to extract the individual values and replicate their plot (Figure 3). Example 2: Data extracted from 3 show the relationship between the number of facial acne lesions at day 7 and the amount of chocolate consumed at baseline for 13 acne-prone men. The panels superimpose a linear regression line (top); no line (middle); and a smoothing line (bottom). The bottom 2 panels reveal that the relationship between acne and chocolate is unconvincing. At first glance, there appears to be some moderate correlation between the 2 variables (top panel, Figure 3), but removing the regression line (middle panel, Figure 3) and adding a smoothing line (bottom panel, Figure 3) reveal that the relationship is unimpressive. Visually, it is apparent that without the 2 data points in the lower left corner—the 2 men who had no chocolate and developed little or no acne—there would be no relationship between acne and chocolate. Indeed, when I recalculate the correlation coefficient between acne and chocolate excluding these two data points, the correlation is essentially zero (r = .03). Note that the authors made the X-axis longer than the Y-axis, which may also have visually exaggerated the effect. Outliers in the data can inflate correlation coefficients (and linear regression statistics) even when the sample size is relatively large. When there is an extreme outlier, the data are effectively reduced to 2 points in space: the outlier and the cloud of other data points. Connecting these 2 "points" forms a line; however, the line is determined by the position of the outlier rather than the true relationship between x and y. In a 2016 study of 29 patients who underwent open liver resection, the authors reported a statistically significant correlation between body mass index (BMI) and blood loss during surgery (r = .573, P < .001) 4. The authors should be commended for plotting their data, which allowed me to replicate their graph (Figure 4). In their paper, however, they failed to point out that the correlation between BMI and blood loss is driven by 2 outliers. By superimposing a smoothing line rather than a straight line, it is easy to see that there is no clear association between BMI and blood loss for the majority of patients. Without the outliers, the correlation coefficient is greatly attenuated: r = .24, P = .23. Example 3: Data were extracted from a study of 29 patients who underwent open liver resection 3. Two outliers exert undue influence on the regression line, resulting in an inflated correlation coefficient (r = .573, P < .001, top panel). Patterns in the data are better captured by a smoothing line (bottom panel). Absent the 2 outliers, there is no clear relationship between blood loss and BMI (r = .24, P = .23). These 2 outliers may be meaningful—perhaps, risk of blood loss jumps dramatically above a certain BMI threshold. But with only 2 patients in this high-BMI range, it is impossible to know whether these data points represent a larger trend or merely chance findings. Scatter plots are a fundamental graphing tool and are essential when reporting linear correlation and regression statistics. These statistics evaluate linear associations and will miss nonlinear relationships—which scatter plots can reveal. Scatter plots also can reveal when a statistic is unduly influenced by a few data points because of a small sample sizes or the presence of outliers. Authors are advised to superimpose smoothing lines on scatter plots rather than the traditional linear regression lines, at least when first inspecting the data. Readers should be skeptical when authors report correlation and linear regression coefficients but fail to provide scatter plots. When plots are provided, readers should examine them carefully to judge whether the reported statistics accurately reflect the true relationship between the variables or are misleading due to nonlinear relationships, small sample sizes, or outliers. Most modern statistical graphing tools have built-in algorithms that allow the computer to find and draw the line or curve that best fits the cloud of data points. When the user asks for a linear regression line, the computer finds the straight line that is closest to the data points, meaning it minimizes the total squared distance between the points and the line (squaring is used to avoid negative values). A LOESS (or locally weighted scatter plot smoothing) smoothing line is more flexible. Rather than assuming a straight line relationship between x and y, LOESS curves can take on any shape that the data dictate. The algorithm works by sliding along the X-axis, fitting lines* to local subsets of the data, and then connecting these lines into a smooth curve. For example, the computer might find the line that best fits the 20 percent of data points nearest to X = −1, and then the 20% of data points nearest to X = −0.5, and then the 20 percent of data-points nearest to X = 0, and so on. These overlapping lines are then "smoothed" together into a single curve. The computer does most of the work, but the user does have to input a "smoothing parameter." The smoothing parameter determines the size of the data windows—a value of 0.20 means that each window will span 20% of the data, whereas 0.80 means that each window will encompass 80 percent of the data. The larger the smoothing parameter, the less the curve will conform to local noise in the data, and the lower the curvature. Smoothing parameters between 0.50 and 0.75 usually give the optimal curvature. You can see that when the curvature is too high (upper panels), the curve "overfits" the data, meaning it is too responsive to individual data points. But when the curvature is too low, the curve "under fits" the data, meaning it oversimplifies the relationship between x and y (lower right panel). A medium amount of curvature (lower left panel) gives the best sense of the true relationship between x and y. Authors can try smoothing parameters between 0.50 and 0.75 and pick the value at which the line becomes relatively smooth—that is, it loses the bumps where it seems to just be fitting noise in the data—but not necessarily straight. Smoothing lines give authors and readers a sense of the true relationship between x and y. A straight line indicates a linear relationship; a U-shaped line indicates a quadratic relationship; more complex relationships are also possible. Finally, when the smoothing line appears close to a horizontal line (even if there are a few minor bumps in the line), this indicates that x and y are unrelated. ∗It is also possible to fit quadratic or cubic functions, rather than straight lines, to the local subsets of data; these also can then be joined together into a single smooth curve.