A random variable is a variable whose value is the numeric outcome of an uncertain process. It's the bridge between "probability" as an abstract idea and the concrete numbers a model actually computes with.
Definition and Notation
A random variable, usually written as a capital letter like \(X\), assigns a number to each possible outcome of an experiment. It's discrete if it takes a countable set of values (e.g. a coin flip: 0 or 1), or continuous if it can take any value in a range (e.g. a pixel's brightness, a model's confidence score).
| Type | Example | Deep Learning Example |
|---|---|---|
| Discrete | Number of heads in 3 coin flips: {0,1,2,3} | Which class a classifier predicts |
| Continuous | Height of a randomly selected person | A single pixel's intensity, a model's raw logit output |
Numerical Example
Let \(X\) be the outcome of rolling a fair six-sided die. \(X\) is discrete, taking values \(\{1,2,3,4,5,6\}\), each with probability \(P(X=k) = \frac{1}{6}\). This simple case already illustrates the key idea: a random variable comes paired with a rule (here, uniform) describing how likely each of its possible values is — that rule is a probability distribution (next note).
Code — Sampling from a Random Variable
import numpy as np
# Simulating a random variable: rolling a fair die 10,000 times
rolls = np.random.randint(1, 7, size=10000)
print(np.mean(rolls)) # should be close to 3.5 (the theoretical expected value)
print(np.unique(rolls, return_counts=True)) # roughly equal counts per outcomeWhere This Shows Up in Deep Learning
- A dataset's labels are treated as random variables — training assumes each labeled example is a sample drawn from some underlying (unknown) data distribution.
- A network's weights are initialized by sampling from a random variable (e.g. a normal or uniform distribution) — see He/Xavier initialization in later categories.
- Dropout treats each neuron's "keep or drop" decision as a random variable, typically a Bernoulli one (see Dropout).
Common Mistakes
- Confusing a random variable (the abstract object, e.g. "the outcome of this die roll") with one specific realized value (e.g. "I rolled a 4") — notation distinguishes these as \(X\) vs \(x\).
- Assuming all random variables in deep learning are discrete — pixel intensities, model confidences, and most real-valued data are continuous random variables, requiring probability density rather than probability mass (a subtle but important distinction covered in the next note).
Interview Relevance
Q: "In what sense are the labels in a training dataset 'random variables'?" Each labeled example is treated as one sample drawn from an underlying, unknown joint distribution over inputs and labels. This framing is what justifies using techniques like maximum likelihood estimation to fit a model — you're estimating parameters that make the observed data most probable under that assumed distribution.
Practice Question
Is "the number of typos in a randomly selected email" a discrete or continuous random variable? Justify your answer.
