Category Archives: Transformers

A regression problem is one where the goal is to predict a single numeric value. I’ve been looking at incorporating a Transformer component into a PyTorch neural network to see if the technique works or not. Bottom line: I got … Continue reading →

A binary classification problem is one where the goal is to predict a discrete value where there are only two possibilities. For example, you might want to predict the sex of a person based on their age, income, and so … Continue reading →

I’ve been looking at unsupervised anomaly detection using a PyTorch Transformer module. My first set of experiments used the UCI Digits dataset because the inputs (64 pixels with values between 0 and 16 — a scaled down MNIST) are all … Continue reading →

I’d been experimenting with the idea of using a Transformer module as the core of a multi-class classifier. The idea is rather weird because Transformer systems were designed to accept sequential information, such as a sequence of words, where order … Continue reading →

I had recently been exploring the idea of creating a PyTorch neural network layer that’s not fully connected. After many hours of experiments, I got a demo working for the Iris dataset where the neural architecture was (4–8)-7-3 meaning there … Continue reading →

The vast majority of my blog posts show a successful program of some sort. But behind the scenes, every successful program is preceded by many, many failures. For example, I’ve been looking at PyTorch Transformer modules. Transformer architecture is the … Continue reading →

Transformer architecture neural networks are very complex. Unlike some of my colleagues, I’m not a naturally brilliant guy. But my primary strength is persistence. I continue to probe the complexities of transformer systems, one example at a time, day after … Continue reading →

Computing a measure of the similarity of two datasets is a very difficult problem. Several months ago, I worked on a project and devised an algorithm based on a neural autoencoder. See jamesmccaffrey.wordpress.com/2021/04/02/dataset-similarity-sometimes-ideas-work-better-than-expected/. I wondered if I could adapt my … Continue reading →

Several months ago I devised an algorithm that computes a value that represents the distance (difference) between two datasets. Computing the distance between two datasets is a remarkably difficult task — I consider it one of the unsolved fundamental problems … Continue reading →

I’ve been exploring PyTorch Transformer Architecture models sequence-to-sequence problems for several months. TA architecture systems are among the most complicated software things I’ve ever worked with. I recently completed a demo implementation of my idea of the simplest possible sequence-to-sequence. … Continue reading →