Sometimes it’s useful to assign fixed weight and bias values to a neural network. To do so requires a knowledge of how those values are stored.

I wrote a short demo program to illustrate the technique. The demo creates a 3-4-2 neural network. The single hidden layer is named hid1 and has a total of 3 x 4 = 12 weights and 4 biases. PyTorch sores the weight values in a 4×3 shaped matrix named self.hid1.weight.data. The biases values are stored in self.hid1.bias.data.

Similarly, the output layer is named oupt and has a total of 4 x 2 = 8 weights and 2 biases. They’re stored in a 2×4 shaped matrix named self.oupt.weight.data and self.oupt.bias.data.

The demo code iterates through the weights and biases and stores 0.01, 0.02, . . 0.26 into the network.

The diagram above shows the conceptual view of the neural network, and a representation of the weight and bias data structures.

Important note: My demo sets the values of the weights and biases in the Net class __init__() method. If you want to modify the weights and biases of a neural network after the network has been instantiated, you need to do so inside a torch.no_grad() block so that the gradients don’t get messed up.

*Software system conceptual diagrams are a facade over the reality and complexity of the underlying code. Here are three remarkable building facades that give a flat wall the appearance of 3D complexity.*

Demo code.

# layer_assign_wts.py
# PyTorch 1.10.0-CPU Anaconda3-2020.02 Python 3.7.6
# Windows 10
import torch as T
device = T.device("cpu") # apply to Tensor or Module
class Net(T.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.hid1 = T.nn.Linear(3, 4) # 3-4-2
self.oupt = T.nn.Linear(4, 2)
v = 0.01
for i in range(4): # hid1 4x3
for j in range(3):
self.hid1.weight.data[i][j] = v
v += 0.01
for i in range(4):
self.hid1.bias.data[i] = v
v += 0.01
for i in range(2): # oupt 2x4
for j in range(4):
self.oupt.weight.data[i][j] = v
v += 0.01
for i in range(2):
self.oupt.bias.data[i] = v
v += 0.01
def forward(self, x):
z = T.tanh(self.hid1(x))
z = self.oupt(z) # no softmax for CrossEntropyLoss()
return z
def main():
print("\nBegin ")
T.manual_seed(1)
print("\nCreating a 3-4-2 network with fixed wts and biases ")
net = Net().to(device)
print("\nhid1 wts and biases: ")
print(net.hid1.weight.data)
print(net.hid1.bias.data)
print("\noupt wts and biases: ")
print(net.oupt.weight.data)
print(net.oupt.bias.data)
print("\nEnd ")
if __name__ == "__main__":
main()

### Like this:

Like Loading...

*Related*

You must be logged in to post a comment.