The practical reference for the tensor manipulation operations used throughout every code example in this hub — indexing, reshaping, and combining tensors.
Indexing and Slicing
x = torch.arange(12).reshape(3, 4)
print(x[0]) # first row
print(x[:, 1]) # second column, across all rows
print(x[1:3, 0:2]) # a sub-block
print(x[-1]) # last rowReshaping — view() vs reshape()
x = torch.arange(12)
print(x.view(3, 4)) # reinterprets the SAME underlying memory -- fails if not contiguous
print(x.reshape(3, 4)) # like view(), but falls back to copying if the data isn't contiguous
print(x.unsqueeze(0).shape) # (1, 12) -- adds a dimension of size 1
print(x.squeeze().shape) # removes dimensions of size 1.view() requires the tensor's data to be contiguous in memory (e.g. after certain operations like .transpose(), it may not be) — .reshape() is the safer general-purpose choice, since it copies data automatically when needed.
Combining Tensors
a = torch.zeros(2, 3)
b = torch.ones(2, 3)
torch.cat([a, b], dim=0) # (4, 3) -- concatenate ALONG an existing dimension
torch.stack([a, b], dim=0) # (2, 2, 3) -- combine along a NEW dimensionThe distinction between cat and stack is one of the most common sources of shape-mismatch confusion: cat joins tensors along an already-existing dimension (the result has the same number of dimensions as the inputs); stack creates a brand new dimension (the result has one more dimension than the inputs).
Element-Wise vs Matrix Operations
a = torch.tensor([[1., 2.], [3., 4.]])
b = torch.tensor([[5., 6.], [7., 8.]])
print(a * b) # element-wise multiplication
print(a @ b) # matrix multiplication -- see Matrix Multiplication
print(a.T) # transposeCommon Mistakes
- Using
.view()on a non-contiguous tensor (e.g. right after.transpose()) and hitting a runtime error — call.contiguous()first, or just use.reshape()instead. - Confusing
catandstackwhen combining a batch of individually-processed tensors — using the wrong one produces a shape one dimension off from what's expected, a very common bug.
Interview Relevance
Q: "What's the difference between torch.cat and torch.stack?" cat concatenates tensors along an existing dimension, so the result has the same number of dimensions as the inputs. stack combines tensors along a brand-new dimension, so the result has one more dimension than the inputs — used when you want to preserve each input as a distinct "slice" rather than merging them along an existing axis.
Practice Question
You have 5 tensors, each of shape (3, 4), representing 5 separate samples. How would you combine them into one tensor of shape (5, 3, 4)?
