PyTorch Accuracy Function for Binary Classification

I regularly use the Keras (over TensorFlow), CNTK, and PyTorch neural network libraries. I use plain TensorFlow less often. A researcher might tell you that the different libraries are all quite similar in many respects, or perhaps talk about architectural differences like static and dynamic computational graphs. But from a developer’s point of view, the libraries feel dramatically different — to me at least.

I was looking at binary classification using PyTorch. Unlike the other libraries, PyTorch does not have a built-in function to compute binary accuracy. PyTorch is only in version 0.4.1 as I write this post, so it’s very immature.

I go a working version of an accuracy() function but it uses an explicit for-loop:

def accuracy(model, data_x, data_y):
  # data_x and data_y are numpy matrices
  n_feat = len(data_x[0])  # number features
  n_items = len(data_x)    # number items
  n_correct = 0; n_wrong = 0
  for i in range(n_items):
    X = T.Tensor(data_x[i])
    # Y = T.Tensor(data_y[i])  # not needed
    oupt = model(X)
    pred_y = oupt.item()

    if pred_y = 0.5 and data_y[i] == 1:
      n_correct += 1
      n_wrong += 1

  return (n_correct * 100.0) / (n_correct + n_wrong)

This approach is OK, especially if you want to insert print() statements to investigate exactly which data items are being correctly, and incorrectly, predicted. But I wanted to see if I could use PyTorch meta-operations that work on an entire Tensor, for improved performance.

After about 30 minutes of wrestling with the code, I came up with:

def akkuracy(model, data_x, data_y):
  # data_x and data_y are numpy matrices
  X = T.Tensor(data_x)
  Y = T.ByteTensor(data_y)   # a Tensor of 0s and 1s
  oupt = model(X)            # a Tensor of floats
  pred_y = oupt >= 0.5       # a Tensor of 0s and 1s
  num_correct = T.sum(Y==pred_y)  # a Tensor
  acc = (num_correct.item() * 100.0 / len(data_y))  # scalar
  return acc

There are a ton of details here, but if you work with PyTorch and examine the code closely, it should make sense. The main point is that I’ve avoided the explicit for-loop and the accuracy function operates on entire Tensors which is much more efficient.

The moral of the story is that even though most of the blog posts I read that compare Keras, CNTK, and PyTorch state that they’re all quite similar to use, I disagree. PyTorch works at a much lower lever of abstraction than Keras and CNTK. You get more flexibility with PyTorch, but coding with PyTorch is quite a bit more difficult in my opinion than coding with Keras or CNTK. I do think however, that coding with PyTorch is about the same level of difficulty as coding with TensorFlow.

Artist Frank Wagtmans paints interesting abstract portraits. What’s not apparent from these images is that his paintings are quite large — typically five feet wide by three feet tall. I suspect that, just as for a programmer in computer science, for an artist, finding the right level of abstraction is difficult.

This entry was posted in Machine Learning, PyTorch. Bookmark the permalink.