Author Archives: jamesdmccaffrey

Using PyTorch to Generate Synthetic Data for a Regression Problem

There are a lot of good datasets for experimenting with machine learning classification. But there are very few datasets for ML regression experiemnts. Creating a synthetic dataset for regression is relatively easy and effective. In high-level pseudo-code: create PyTorch network … Continue reading

Posted in PyTorch | Leave a comment

Refactoring My PyTorch Hyperparameter Search Using Evolutionary Optimization Demo

One of the advantages that experienced developers have compared to early-career developers is subjective intuition. Several days ago, I implemented a program that searches for PyTorch neural network hyperparameter values (number of hidden nodes, batch size, and so on) using … Continue reading

Posted in PyTorch | Leave a comment

Why Isn’t PyTorch Training Code Placed Inside the Network Class Definition?

I’ve been using PyTorch as my neural code library of choice for the past several years. Every PyTorch example I’ve ever seen has a program structure where there’s a class that defines the network, a class that defines a Dataset, … Continue reading

Posted in PyTorch | Leave a comment

Example of Principal Component Regression Using the scikit Library

The goal of a regression problem is to predict a single numeric value. For example, you might want to predict the selling price of a house based on its square footage, number of bedrooms, age, and so on. The simplest … Continue reading

Posted in Scikit | Leave a comment

Hyperparameter Search Using Evolutionary Optimization for a PyTorch Binary Classifier

Bear with me for a moment — it’s difficult to explain the topic of this blog post. I’ll work backwards from the output of a demo program: . . . End evolution Final best soln found: [9 6 4 9 … Continue reading

Posted in PyTorch | Leave a comment

“Researchers Create a Computer Program that has Superhuman Skill at Stratego” on the Pure AI Web Site

I contributed to an article titled “Researchers Create a Computer Program that has Superhuman Skill at Stratego” on the May 2023 edition of the Pure AI web site. See https://pureai.com/articles/2023/05/01/stratego.aspx. Researchers at Google have demonstrated a new algorithm that achieves … Continue reading

Posted in Machine Learning | Leave a comment

Hyperparameter Search Using Evolutionary Optimization for a PyTorch Multi-Class Classifier

Neural networks can create powerful prediction systems. But NNs have a lot of hyperparameters that must be specified and the values of the hyperparameters have a huge effect on the effectiveness of a NN. Architecture hyperparameters include number of hidden … Continue reading

Posted in PyTorch | Leave a comment

Understanding the Name of the scikit-learn Python Machine Learning Library

I was presenting a talk recently. One of my examples used the scikit-learn code library for machine learning. One of the attendees asked about the difference between “scikit” and “scikit-learn”. Briefly, scikit-learn is a Python language code library for machine … Continue reading

Posted in Scikit | Leave a comment

Using PairwiseKernel in scikit GaussianProcessRegressor for Polynomial Kernel

Briefly, I put together a demo of a scikit Gaussian process regression model that uses the polynomial kernel from the sklearn.metrics.pairwise module via the PairwiseKernel class. Whew, what a mouthful. A Gaussian process regressor model predicts a single numeric value … Continue reading

Posted in Scikit | Leave a comment

Mapping an Integer Vector to PyTorch Hyperparameter Values

I’ve been investigating the idea of using evolutionary optimization to find good values for the hyperparameters of a PyTorch neural network. Evolutionary optimization sets up a population of possible solutions and then: create poulation of possible solutions loop several times … Continue reading

Posted in PyTorch | 1 Comment