Sometimes it’s useful to assign fixed weight and bias values to a neural network. To do so requires a knowledge of how those values are stored.
I wrote a short demo program to illustrate the technique. The demo creates a 3-4-2 neural network. The single hidden layer is named hid1 and has a total of 3 x 4 = 12 weights and 4 biases. PyTorch sores the weight values in a 4×3 shaped matrix named self.hid1.weight.data. The biases values are stored in self.hid1.bias.data.
Similarly, the output layer is named oupt and has a total of 4 x 2 = 8 weights and 2 biases. They’re stored in a 2×4 shaped matrix named self.oupt.weight.data and self.oupt.bias.data.
The demo code iterates through the weights and biases and stores 0.01, 0.02, . . 0.26 into the network.
The diagram above shows the conceptual view of the neural network, and a representation of the weight and bias data structures.
Important note: My demo sets the values of the weights and biases in the Net class __init__() method. If you want to modify the weights and biases of a neural network after the network has been instantiated, you need to do so inside a torch.no_grad() block so that the gradients don’t get messed up.
Software system conceptual diagrams are a facade over the reality and complexity of the underlying code. Here are three remarkable building facades that give a flat wall the appearance of 3D complexity.
# layer_assign_wts.py # PyTorch 1.10.0-CPU Anaconda3-2020.02 Python 3.7.6 # Windows 10 import torch as T device = T.device("cpu") # apply to Tensor or Module class Net(T.nn.Module): def __init__(self): super(Net, self).__init__() self.hid1 = T.nn.Linear(3, 4) # 3-4-2 self.oupt = T.nn.Linear(4, 2) v = 0.01 for i in range(4): # hid1 4x3 for j in range(3): self.hid1.weight.data[i][j] = v v += 0.01 for i in range(4): self.hid1.bias.data[i] = v v += 0.01 for i in range(2): # oupt 2x4 for j in range(4): self.oupt.weight.data[i][j] = v v += 0.01 for i in range(2): self.oupt.bias.data[i] = v v += 0.01 def forward(self, x): z = T.tanh(self.hid1(x)) z = self.oupt(z) # no softmax for CrossEntropyLoss() return z def main(): print("\nBegin ") T.manual_seed(1) print("\nCreating a 3-4-2 network with fixed wts and biases ") net = Net().to(device) print("\nhid1 wts and biases: ") print(net.hid1.weight.data) print(net.hid1.bias.data) print("\noupt wts and biases: ") print(net.oupt.weight.data) print(net.oupt.bias.data) print("\nEnd ") if __name__ == "__main__": main()
You must be logged in to post a comment.