NumPy for Data Science

By the end of this lesson, you will understand how NumPy arrays enable fast, vectorized mathematical operations and how broadcasting allows these operations to work seamlessly across differently shaped data.

What it is

NumPy (Numerical Python) is the foundational library for scientific computing in Python. Its core object is the ndarray, a homogeneous, fixed-size array that stores data more efficiently than standard Python lists. Unlike lists, which store pointers to objects scattered in memory, arrays store contiguous blocks of raw data, enabling CPU-level optimizations. The mental model shifts from "looping over items" to "applying an operation to the entire dataset at once," known as vectorization. Key related terms include dtype (data type), shape (dimensions), and broadcasting (the rule set for handling arrays of different shapes).

Why it matters

• Performance: Vectorized operations are 10-100x faster than equivalent Python loops because they bypass the interpreter overhead and leverage optimized C/Fortran libraries.
• Conciseness: Complex mathematical expressions can be written in single lines, reducing code clutter and potential logic errors.
• Ecosystem Compatibility: Pandas, Scikit-Learn, TensorFlow, and PyTorch all rely on NumPy arrays as their underlying data structure.
• Memory Efficiency: Arrays use less memory than lists of objects, allowing you to work with larger datasets on limited hardware.

Syntax or steps

The smallest useful pattern involves creating an array and applying a scalar or element-wise operation.

1. Import the library: import numpy as np.
2. Create an array using np.array() or generation functions like np.zeros().
3. Apply arithmetic operators (+ - * /) directly to the array variable.
4. Rely on broadcasting when operands have compatible but unequal shapes.

Example

import numpy as np

# Create a 2D array representing sales data (rows=products, cols=months)
sales = np.array([[100, 200, 300], 
                  [400, 500, 600]])

# Vectorized multiplication: Apply a 1.1 tax rate to every element
taxed_sales = sales * 1.1

# Broadcasting: Add a monthly bonus (1D array) to each row (2D array)
bonus = np.array([10, 20, 30])
final_sales = taxed_sales + bonus

print(final_sales)

Explanation: 1. sales is a 2x3 matrix. 2. sales * 1.1 multiplies every individual number by 1.1 without writing a loop. 3. bonus is a 1D array of shape (3,). When added to taxed_sales (shape 2x3), NumPy broadcasts bonus across both rows. It effectively treats bonus as [[10, 20, 30], [10, 20, 30]].

Common mistakes

• Mixed Data Types: Creating an array with integers and strings forces NumPy to cast everything to strings, breaking math operations. Always ensure numeric consistency.
• Incompatible Shapes: Broadcasting fails if dimensions do not align from right to left. For example, adding a (2,3) array to a (3,2) array raises an error unless one is transposed.
• Modifying Views: Slicing an array often returns a view, not a copy. Changing the slice changes the original array. Use .copy() if you need independent data.
• Using Python Lists for Math: Multiplying a list by an integer repeats the list; multiplying an array scales the values. Ensure you are operating on ndarray objects.

When to use it

Compare NumPy with standard Python lists to determine the appropriate tool.

Feature	Python List	NumPy Array
Data Homogeneity	Heterogeneous allowed	Homogeneous required
Math Operations	Manual loops required	Vectorized built-in
Best For	General purpose storage	Numerical computation

Use NumPy whenever you perform statistical analysis, linear algebra, or large-scale numerical transformations. Use lists only for small, mixed-type collections where no math is involved.

Practice

Guided Exercise: Create a 1D array of numbers 1 through 5. Square each number using vectorization (not a loop). Hint: Use the ** operator or np.square(). Challenge: Create two arrays: A (shape 3x1) and B (shape 1x3). Add them together. What is the resulting shape? Solution Hint: The result is a 3x3 matrix due to outer-product-style broadcasting.

Quick check

Question: Why does [1, 2] * 2 produce [1, 2, 1, 2] while np.array([1, 2]) * 2 produces [2, 4]? Answer: The first uses Python's list repetition behavior, while the second uses NumPy's element-wise multiplication defined for arrays.

Summary

NumPy replaces slow Python loops with fast, vectorized array operations, serving as the backbone of the data science stack. Mastering broadcasting rules allows you to manipulate complex multidimensional data structures intuitively and efficiently.