There are many neural network libraries. Three of the most popular are TensorFlow, Keras, and PyTorch. PyTorch is rapidly growing in popularity among my colleagues.
The fundamental data structure in PyTorch is the tensor. A PyTorch tensor is a one-dimensional (i.e., a vector) or multidimensional (i.e., a matrix) that can be handled by a GPU.
Working with PyTorch tensors can be mildly frustrating for beginners. Based on my learning path, I think it’s important to have a solid understanding of tensor basics, starting with different techniques for tensor creation.
Here’s the introductory example I use when I teach a PyTorch workshop.
x = pt.tensor([[0,0,0],[0,0,0]], dtype=pt.float32)
creates a 2×3 tensor of zeros where each cell is a 32-bit floating point value. Notice the lower-case ‘t’. I like to use “pt” as an alias but almost all my colleagues spell out “torch”.
x = pt.zeros(2, 3, dtype=pt.float32)
is a shortcut to do the same thing using the special zeros() function. I’m not a fan of most programming language shortcuts like this.
Here’s a third way to do the same thing:
x = pt.FloatTensor([[0,0,0],[0,0,0]])
which is a different shortcut. Other shortcuts include functions like LongTensor().
And now a fourth way is:
x = pt.Tensor([[0,0,0],[0,0,0]])
which is based on the idea that 32-bit float is the default numeric type for neural networks.
A fifth way, which is a bit different, is to create a PyTorch tensor from a NumPy array like:
a = np.array([[0,0,0],[0,0,0]], dtype=np.float32)
x = pt.from_numpy(a)
The from_numpy() function is especially useful when reading data from a text file using np.loadtxt().
The moral of the story is that when learning PyTorch, you have to move slower than you’d like because mastering tensor basics is a bit trickier than you might expect.