I wrote an article titled “Multi-Class Classification Accuracy by Class Using PyTorch” in the January 2023 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2023/01/03/accuracy-by-class.aspx.
A multi-class classification problem is one where the goal is to predict a discrete value where there are three or more possibilities. For example, you might want to predict the political leaning (conservative, moderate, liberal) of a person based on their sex, age, state where they live and income.
A naive approach for evaluating the effectiveness of a trained multi-class model is to compute the accuracy of the model on all test data. But suppose your data is skewed toward a particular class. For example, if most of the data items are class moderate (say, 900 out of 1,000) and only a few are class conservative (say, 40 out of 1,000) and class liberal (60 out of 1,000), then a model that predicts class moderate for any input will score 90 percent accuracy. My article shows how to compute classification accuracy class-by-class.
The key function is:
def do_acc(model, dataset, n_classes): X = dataset[0:len(dataset)] # all X values Y = dataset[0:len(dataset)] # all Y values with T.no_grad(): oupt = model(X) # [40,3] all logits for c in range(n_classes): idxs = np.where(Y==c) # indices where Y is c logits_c = oupt[idxs] # logits corresponding to Y == c arg_maxs_c = T.argmax(logits_c, dim=1) # predicted class num_correct = T.sum(arg_maxs_c == c) acc_c = num_correct.item() / len(arg_maxs_c) print("%0.4f " % acc_c)
This version is efficient because it processes all data items of a particular class at once, rather than iterating through each data item one at a time.
“From Russia, with Love” is the fifth James Bond novel by Ian Fleming. The novel was first published in 1957 and went through many editions. The story was all about Cold War politics. The Soviet counter-intelligence agency hatched a plot to assassinate Bond in a way that discredits both him and his MI6 organization. The Russians use a beautiful, unwitting cipher clerk, Tatiana Romanova, as bait.
The movie version of “From Russia with Love” was the second Bond film (following “Dr. No” in 1962), and was released in 1963. It’s regarded by many movie critics as the best of all Bond films.