I regularly use the Keras (over TensorFlow), CNTK, and PyTorch neural network libraries. I use plain TensorFlow less often. A researcher might tell you that the different libraries are all quite similar in many respects, or perhaps talk about architectural differences like static and dynamic computational graphs. But from a developer’s point of view, the libraries feel dramatically different — to me at least.

I was looking at binary classification using PyTorch. Unlike the other libraries, PyTorch does not have a built-in function to compute binary accuracy. PyTorch is only in version 0.4.1 as I write this post, so it’s very immature.

I go a working version of an accuracy() function but it uses an explicit for-loop:

def accuracy(model, data_x, data_y): # data_x and data_y are numpy matrices n_feat = len(data_x[0]) # number features n_items = len(data_x) # number items n_correct = 0; n_wrong = 0 for i in range(n_items): X = T.Tensor(data_x[i]) # Y = T.Tensor(data_y[i]) # not needed oupt = model(X) pred_y = oupt.item() if pred_y = 0.5 and data_y[i] == 1: n_correct += 1 else: n_wrong += 1 return (n_correct * 100.0) / (n_correct + n_wrong)

This approach is OK, especially if you want to insert print() statements to investigate exactly which data items are being correctly, and incorrectly, predicted. But I wanted to see if I could use PyTorch meta-operations that work on an entire Tensor, for improved performance.

After about 30 minutes of wrestling with the code, I came up with:

def akkuracy(model, data_x, data_y): # data_x and data_y are numpy matrices X = T.Tensor(data_x) Y = T.ByteTensor(data_y) # a Tensor of 0s and 1s oupt = model(X) # a Tensor of floats pred_y = oupt >= 0.5 # a Tensor of 0s and 1s num_correct = T.sum(Y==pred_y) # a Tensor acc = (num_correct.item() * 100.0 / len(data_y)) # scalar return acc

There are a ton of details here, but if you work with PyTorch and examine the code closely, it should make sense. The main point is that I’ve avoided the explicit for-loop and the accuracy function operates on entire Tensors which is much more efficient.

The moral of the story is that even though most of the blog posts I read that compare Keras, CNTK, and PyTorch state that they’re all quite similar to use, I disagree. PyTorch works at a much lower lever of abstraction than Keras and CNTK. You get more flexibility with PyTorch, but coding with PyTorch is quite a bit more difficult in my opinion than coding with Keras or CNTK. I do think however, that coding with PyTorch is about the same level of difficulty as coding with TensorFlow.

*Artist Frank Wagtmans paints interesting abstract portraits. What’s not apparent from these images is that his paintings are quite large — typically five feet wide by three feet tall. I suspect that, just as for a programmer in computer science, for an artist, finding the right level of abstraction is difficult.*