Organizing Data

Measures of Position &Exploratory Data Analysis

Essentials: Measures of Position(Better understanding distribution shapes.)

Know the types of measures used to look at specific positions within a data distribution.

Be able to calculate the inter-quartile range, three quartiles, Pearson’s Index of Skewness, z-score, Coefficient of Variation.

Be familiar with symmetry vs. skewness and distribution shapes.

Be able to build both traditional and modified box plots (aka: box-and-whiskers plot).

Measures of Position Measures of position are points

within the data that are used to describe characteristics of the data. Percentiles, Deciles, Quartiles, Minimum and Maximum are among these points.

We will focus on 5 specific values...

The Five-Number-Summary

Five numbers are frequently used to indicate positions within a data set and much more...

These points are: Minimum - Q1 - Median - Q3 -

Maximum

Components of the

Five-Number Summary The Minimum and

Maximum values represent the extremes in the data set.

Obtaining these points is a simple matter of looking for them.

Q1, Median (Q2), and Q3 represent points within the data. They are the 25th, 50th, and 75th percentiles, respectively.

Formulas are used to locate these positions.

The Quartiles The Quartiles are obtained using the

following three formulas. Q1(25th percentile) = (n+1)/4 Median (Q2, 50th percentile) =

(n+1)/2 Q3 (75th percentile) = [3(n+1)]/4

(Where n = number of observations. Note that these formulas identify POSITIONS within the data, not values; the data must be placed in numeric order)

The Interquartile Range The Interquartile Range (IQR)

describes the middle 50% of the data. It is obtained by the formula:

IQR = Q3 - Q1

The Interquartile Range is used as a measure of variation around the Median of a set of data.

Anatomy of Measures of Position

Quartiles divide a data set that has been ordered from smallest to largest into four

sections, each containing 25% of the data values.A term more familiar to you might be percentile.

For example, the 25th percentile,50th percentile, or 75th percentile.

440 481 482 483 483 514 514 554 554 554 562 612 623 631 638 664 671 677 690 707

Recall that the median in this data set is found using the formula (n+1)/2 to obtain the POSITION of the median

(here the POSITION is (20 + 1)/2 = 10.5). Determining the value half way between the 10th and 11th values yields a Median of 558: [(554 + 562)/2 = 558].

Another name for the median, is the second quartile (Q2) . It is the value in the data set such that 50% of the values are lower than it,

and 50% of the values are higher than it.

The first quartile (Q1) is the valuesuch that 25% of the values are lower

than it, and 75% of the values are higher than it.To find this value apply the formula (n+1)/4 to obtain

the POSITION of the first quartile. Here the formula yields POSITION 5.25: [(20+1)/4 = 5.25]. Determine the value ¼ of the way between the 5th value (483) and the 6th value (514)

by determining the difference between these two numbers (514 – 483 = 31); multiplying the

difference by .25 (31 * .25 = 7.75); and adding this value to the smaller number

(483 + 7.75 = 490.75 = Q1)

The third quartile (Q3) is the valuesuch that 75% of the values are lower

than it, and 25% of the values are higher than it.To find this value apply the formula [3(n+1)/4] to obtain

the POSITION of the third quartile. Here the formula yields POSITION 15.75: [(3(20+1))/4 = 15.75]. Determine the value ¾

of the way between the 15th value (638) and the 16th value (664) by determining the difference between these two

numbers (664 - 638 = 26); multiplying the difference by .75 (26 * .75 = 19.5); and adding

this value to the smaller number (638 + 19.5 = 657.5 = Q3)

The Interquartile Range is the difference between the third quartile,

and the first quartile. Here, theinterquartile range is

657.5 - 490.75 = 166.75

The Minimum value, First Quartile (Q1), Median (Q2), Third Quartile (Q3) and Maximum value

comprise what is referred to as the Five Number Summary. Each component of this summary

represents a measure of position within a set of data. Together, these five values are referred to as the

Five Number Summary.

Exploratory Data Analysis (EDA): EDA is the process of using statistical tools (graphs, measures of center, variation, and position) to investigate data sets in order to understand their important characteristics.

Exploring and Comparing Data

Box-and-Whiskers Plot a.k.a. Box plot A Box plot is used to display the

five-number-summary. One can also examine the shape of the distribution with a Box plot.

Data Presentation: Box plot: Traditional vs. Modified Box

plot Display of Outliers and Adjacent Values

20 N=20 Flour (in lbs.) Used

800

700

600

500

400

Anatomy of a Traditional Box plot

DUDLEY’S DOUGHNUTS(flour in pounds, used on 20 consecutive days during the

month of Dec. 1999)This is Q3, thethird quartile. Here,

Q3 is 657.5.

This is theMedian (Q2). Here, the

median is 558.

This is Q1, the firstquartile. Here, Q1 is 490.75.

This is the UpperWhisker. It is the

maximum value in thedata set. Here, the

maximum value is 707.

This is theLower Whisker. It is theminimum value in the data

set. Here, the minimumvalue is 440.

Flour (in lbs.) Used

440, 481, 482, 483, 483, 514, 514, 554, 554, 554, 562, 612, 623, 631, 638, 664, 671, 677, 690, 707

Title

Historical NoteWho invented this useful tool for quick data

analysis?John TukeyStatistician

Outliers, Limits, and Adjacent Points

An Outlier is a data point found at one of the extremes of the data, and is well outside the general pattern of the data.

Upper and Lower Limits: Used as a tool to identify observations that may be outliers.

Lower Limit = Q1 - 1.5(IQR) Upper Limit = Q3 + 1.5(IQR)

Adjacent Points: The last data value that occurs before (or at) the Upper or Lower Limit. In modified Box plots the whisker would stop at this data value rather than being drawn out to 1.5(IQR).

Mean Price of a Movie Ticket for a Sample of 12 U.S. Cities

US

987654

8

Example of a Modified Box Plot

A grouped box plot is a good way to visualize differences/similarities between groups.

Grouped Box plots

SKEWED LEFT(negatively)

SKEWED RIGHT(positively)

Mode = Mean = MedianSYMMETRIC

Mean Mode Median

Mean Mode Median

Symmetry

Symmetry – a distribution is symmetric if the left half of the distribution is roughly a mirror image of its right half.

Skewness – a distribution is skewed if it is not symmetric and if it extends more to one side than the other

Pearson’s Index of Skewness

Skewness can be measured using Pearson’s Index of Skewness.

Example: Given a set of date whose statistics include: mean = 40, median = 41, S.D = 4, determine if this distribution id skewed.

I = (3(40 - 41))/4 = -.75 Given that the value is within the range from -1 to 1,

inclusive, this distribution would not be considered to be significantly skewed.

s

medianxI

)(3

Pearson’s Index of Skewness

When symmetric, I = 0. Values usually range from –3 to +3. A distribution is considered symmetric if the

index value is between -1 and +1 If the index value (I) is less than -1 the data

are negatively (left) skewed. If the index value is greater than +1 The data

are positively (right) skewed.

Pearson’s Index of Skewness: Example

Use Pearson’s Index of Skewness to determine if the distribution of 406 automobileweights is approximately normally distributed or does it display a degree of skewness?

Mean Wt: 2969.56 lb.Median Wt: 2811.00 lb.Standard Deviation: 849.83 lb.

Measures of Position Standard Scores: a standard score,

or z-score is the number of standard deviations that a given value x is above or below the mean. To find a z-score

For populations: For samples:

(Always round z to two decimal places.)

xz s

xxz

Standard Scores (z-score)

Example: Ozzie just took two tests. Given his scores, the mean for the tests, and the standard deviations, on which test did Ozzie perform better relative to the other students?

Calculus Exam: Grade = 65, class mean = 50, S.D. = 10 z = (65 - 50)/10 = 1.5 (or 1.5 standard deviations above the mean)

History exam: Grade = 30, class mean = 25, S.D. = 5 z = (30 - 25)/5 = 1 (or 1 standard deviation above the mean)

Since the z-score for the calculus exam is larger, Ozzie’s relative position is higher in the calculus class than it is in the history class.

sxxz

Recall, a z-score is a measure of a value’s distance away from a distribution’s mean as measured in standard deviations.

Coefficient of Variation Allows us to compare standard

deviations. The result is expressed as a percentage.

Example:Trinity’s test statistics included: Anthropology test - mean of 50 and S.D. of 10; Music test - mean of 40 and S.D. of 5. Which test showed greater variation in test scores?

Anthropology: (10/50)*100 = 20% History: (5/40)*100 = 12.5% Thus, there was greater variation in test scores for the Anthropology

test.

100_ x

sCVar

Coefficient of Variation: Example

The heights and weights and ages of the starting members of the 2008 World Champion Boston Celtics are noted below. Determine the coefficient of determination for these variables to determine which has the greatest variation.

Ray Allen: 77 in., 205 lb., 33 yrs. Rajon Rondo: 73 in., 171 lb., 21 yrs. Paul Pierce: 79 in., 235 lb., 30 yrs. Kevin Garnett: 83 in., 253 lb., 32 yrs. Kendrick Perkins: 82 in., 280 lb., 23 yrs.