By the end of this lesson, you will be able to calculate and interpret measures of central tendency (mean, median, mode), assess distribution shape using skewness, and quantify variability using standard deviation.
What it is
Descriptive statistics summarize large datasets into single values. Central Tendency identifies the "center" of data: the mean (average), median (middle value), and mode (most frequent). Variability describes how spread out the data is, commonly measured by standard deviation. Asymmetry, or skewness, indicates if the distribution leans left or right. When mean and median differ significantly, the data is skewed; when they are close, the data is likely symmetric.
Why it matters
- Outlier Detection: Comparing mean vs. median reveals if extreme values are distorting your average.
- Data Cleaning: High standard deviation relative to the mean suggests inconsistent data quality.
- Model Selection: Many machine learning algorithms assume normal distributions; skewness checks validate this assumption.
- Business Reporting: Median income is often more representative than mean income in unequal economies.
Syntax or steps
To analyze a dataset, follow these steps:
- Load your numerical data into an array or list.
- Calculate the mean (
sum / count) and median (sort, then pick middle). - Compute variance (average squared difference from mean) and take the square root for standard deviation.
- Determine skewness by comparing mean and median: if Mean > Median, positive skew; if Mean < Median, negative skew.
Example
This Python example uses the statistics module to demonstrate these concepts on a small salary dataset containing one outlier.
import statistics
# Dataset with an outlier (100k)
salaries = [50000, 52000, 48000, 51000, 100000]
# Central Tendency
mean_val = statistics.mean(salaries)
median_val = statistics.median(salaries)
# Variability
stdev_val = statistics.stdev(salaries)
# Asymmetry Check
if mean_val > median_val:
skew_direction = "Positive (Right-skewed)"
elif mean_val < median_val:
skew_direction = "Negative (Left-skewed)"
else:
skew_direction = "Symmetric"
print(f"Mean: {mean_val}")
print(f"Median: {median_val}")
print(f"Std Dev: {stdev_val:.2f}")
print(f"Skewness: {skew_direction}")Explanation: The mean is pulled upward by the $100,000 outlier, resulting in a value higher than the median. This discrepancy signals positive skewness. The standard deviation quantifies the wide spread caused by that single high value.
Common mistakes
- Using Mean for Skewed Data: Always check skewness first. If data is heavily skewed, report the median instead of the mean.
- Ignoring Sample Size: Standard deviation behaves differently for samples vs. populations. Use
stdevfor samples andpstdevfor entire populations. - Confusing Mode with Mean: In multimodal distributions (multiple peaks), the mode may not represent the center at all.
- Assuming Normality: Do not apply parametric tests (like t-tests) without checking if skewness and kurtosis are within acceptable limits.
When to use it
| Metric | Best Used When... | Avoid When... |
|---|---|---|
| Mean | Data is symmetric and free of outliers. | Data has extreme values or is highly skewed. |
| Median | Data is skewed or contains outliers. | You need to perform further algebraic calculations. |
| Mode | Categorical data or identifying most common items. | Data is continuous and unique (no repeats). |
Practice
Guided Exercise: Modify the code above to add a second low-value outlier ($10,000). Observe how the mean changes compared to the median.
Challenge: Calculate the Interquartile Range (IQR) for the original dataset. Hint: IQR = Q3 - Q1. You can find quartiles using statistics.quantiles(salaries, n=4).
Quick check
Question: If a dataset's mean is 10 and its median is 15, what does this indicate about the distribution?
Answer: It indicates a negative (left) skew. The lower mean suggests the presence of low-value outliers pulling the average down below the middle value.
Summary
Central tendency provides a baseline, while variability and skewness reveal the structure and reliability of that baseline. Always compare mean and median to detect asymmetry before drawing conclusions from averages.
