Mathematics in Contemporary Society: A Work in Progress

The standard deviation, is a measure of variation that approximates how far, on average, the data values deviate, or are different from, the mean. The mean and standard deviation are paired to provide a measure of center and spread for symmetric distributions. We use the letter \(s\) for the standard deviation of a sample and the lowercase Greek letter \(\sigma\) (sigma) for the standard deviation of a population.

We will go through the whole process for calculating the standard deviation of Section D from above.

The mean quiz score, like in Sections A, B and C, is 5 points.

The first step in finding the standard deviation is to find the deviation, or difference, of each data value from the mean. We will do this in a table. You could also use a spreadsheet to do these calculations.

We would like to get an idea of the “average” deviation from the mean, but if we find the average of the values in the second column, the negative and positive values cancel each other out (this will always happen), so to prevent this we square the deviations which results in a postive number.

Next, we add the squared deviations and we get

Ordinarily, we would then divide by the number of scores, \(n\text{,}\) (in this case, 10) to find the mean of the deviations, but the division by \(n\) is only done if the data set represents a population. When the data set represents a sample (as it almost always does), we divide by \(n–1\) (in this case, 9). The reason for this is that we should assume the population is more diverse than the sample we are looking at, and dividing by a smaller number gives us a larger value for the standard deviation to better represent the variability. (The reason we use \(n-1\) exactly has to do with degrees of freedom which is beyond the scope of this text.)

We assume Section D represents a sample, so we will divide by \(10-1=9\text{.}\) (If this had been a population, we would have divided by 10.) Note that our units are now points-squared since we squared all of the deviations. It is much more meaningful to use the units we started with, so to convert back to points we take the square root.

The sample standard deviation for Section D is

For comparison, here is the standard deviation for each section listed above:

\(S_{A}=0\) points

\(S_{B}=5.27\) points

\(S_{C}=0.82\) points

\(S_{D}=2.36\) points

For the standard deviation, we usually use two more decimal places than the original data. This tells us that on average, scores were approximately 2.36 points away from the mean of 5 points.

One technical side note: the sample standard deviation is the square root of an inflated average of the squared deviations from the mean, which isn’t exactly the same as the average distance from the mean, but it is a good approximation and has some nice mathematical properties that make it the most commonly used measure of variation.

In summary, here are the steps to calculate the standard deviation by hand.

Calculating the standard deviation by hand can be quite a nuisance when we are dealing with a large data set, so we can also use technology. We use the spreadsheet function =STDEV.S to find the sample standard deviation. Notice that this is different from the population standard deviation, which uses the function =STDEV.P.

Just like the spreadsheet functions =AVERAGE and =MEDIAN, we can either list the individual data values in the formula, or we can enter the data values into a row or column and use the row or column range in the formula.

=AVERAGE(A1:A36)

=MEDIAN(A1:A36)

=STDEV.S(A1:A36)

The standard deviation is the measure of variation that we pair with the mean for approximately symmetric distributions. This pairing should make sense because the standard deviation uses the mean in its calculation. But what about the median? What measure of variation do we pair with it?

One candidate is the range. The range tells us the spread or width of the entire data set. We calculate the range as the difference between the maximum and minimum value.

However, the range is not a very good measure of variation since it is very strongly affected by skew and outliers. Consider, for example, the distribution of full time salaries in the United States. Many people earn a minimum wage salary, while others like Jeff Bezos (Amazon) and Bill Gates (Microsoft) earn millions (if not billions!). A range this large does very little to help us get a sense of the spread where most of the data values lie.

Instead, the measure of variation that we pair with the median is the interquartile range (IQR). The IQR tells us the width of the middle 50% of data values. By cutting off the lower and upper 25% of data values, we are able to ignore extreme values and provide a more accurate sense of how spread out the distribution is.

The IQR is calculated as the difference between the third quartile (Q3) and the first quartile (Q1). Before we can calculate the interquartile range, though, we need to learn how to find the first and third quartiles.

As the name implies, quartiles are values that divide the data into quarters. The first quartile (Q1) is the value that 25% of the data lie below. The third quartile (Q3) is the value that 75% of the data lie below. As you might have guessed, the second quartile is the same as the median since 50% of the data values lie below it. Then the interquartile range is the difference between the third and first quartiles; it tells us how large the spread is of the middle 50% of data values.

There are several acceptable methods for finding quartiles, and you may see different methods used in different textbooks, spreadsheet programs, and statistical software. The resulting values may be slightly different depending on the method used. In this book, we will use the locator method.

Suppose, for example, we have measured the height, in inches, of 12 people who identify as female. The data values, in numerical order, are listed below.

The second quartile is the median which occurs between the second and third groups of values. This falls between 64 and 66, and averaging those values we get the median to be Q2=65.

The third quartile occurs at the dividing line between the third and fourth groups of values which is between 67 and 69, so we take the average of these two values to get Q3=68 inches. Therefore, the interquartile range is Q3-Q1=68-62=6 inches. This means that the middle 50% of people in this data set are all within 6 inches of each other in height.

What if the dataset cannot be divided evenly into four groups? For example, if we had 13 data values instead of 12, there would be \(13\div 4 = 3.25\) numbers in each group, which is a problem. There are several valid approaches to solve this problem, and we will use an approach called the locator method. Our locator is \(L=0.25\cdot 13=3.25\text{.}\) Any time the locator is not a whole number, we go up to the next whole number, which is 4 in this case. Then the first quartile is the data value in the fourth position as seen in the next example.

We can summarize the locator method for finding quartiles as follows.

The locator method can also be used to find the median, which is Q2, using \(L=0.50n\text{.}\) In the 13-number dataset above, the locator of the median (Q2) is \(L=0.5\cdot13=6.5\text{.}\) Since 6.5 is not a whole number, we round up to 7 and find that the 7th value, the median, is 66 inches. This approach can be broadened to find the value of particular percentiles, too!

The five-number summary consists of the minimum, first quartile, median, third quartile, and maximum, separated by commas. These five numbers give us a good summary of the center and spread of the data. For example, the five-number summary of the dataset of test scores in the previous example is: 0, 68.1, 82.5, 93.75, 100.

A boxplot, also called a box and whisker plot, is a visual representation of the five-number summary.

Boxplots are often displayed vertically, and outliers, which are values significantly greater or lower than the other values, may be displayed as points and excluded from the five-number summary. The boxplots below were created using Microsoft Excel on the left and Google Sheets on the right. They represent the same test score data.

In addition to the use of dots to indicate outliers, Microsoft Excel boxplots include an X to indicate the mean. Boxplots created with Google Sheets provide less information: the do not indicate outliers or the locations of the mean or even the median.

The primary application of boxplots is in comparing multiple datasets. When multiple boxplots are displayed side-by-side of stacked above each other using the same scale, we can compare the datasets quickly.