Banknote Authentication Example Using Keras

I recently upgraded my Keras library installation to version 2.6 and so I’ve been revisiting my three basic examples: Iris Dataset (multi-class classification), Boston Housing (regression), and Banknote Authentication (binary classification). In older versions of Keras, you would install the TensorFlow engine first and then the separate Keras library second, but now Keras is included in TensorFlow.

The Banknote Authentication dataset has 1372 items. Each item represents a banknote (think Euro or dollar bill) that is authentic (class 0) or a forgery (class 1). Each line of data has four predictor values: variance of image of banknote, skewness of image, kurtosis of image, entropy of image.


A graph of some of the Banknote Authentication data — just kurtosis and entropy for the first 40 class 0 (authentic) and first 40 class 1 (forgery) items.

The Banknote data can be used as-is because all predictor values are roughly in the same range, however, I normalized each predictor value by dividing by 20 so that all values are between -1 and +1. Then I wrote a little script to randomly split the data into a 1000-item set for training (about 70% of the data) and a 372-item set for testing (about 30%).

For the neural network binary classifier, I used a 4-(8-8)-1 architecture. I used tanh() activation on the two hidden layers but I could have used relu() activation instead. I used sigmoid activation on the output node so that output values are between 0.0 and 1.0, and then an output value less than 0.5 indicates class 0 = authentic, and an output value greater than 0.5 indicates class 1 = forgery.

import numpy as np
import tensorflow as tf
from tensorflow import keras as K

print("Creating 4-(8-8)-1 neural network ")
g_init = K.initializers.glorot_uniform(seed=1)
model = K.models.Sequential()
model.add(K.layers.Dense(units=8, input_dim=4,
  activation='tanh', kernel_initializer=g_init, 
  bias_initializer='zeros')) 
model.add(K.layers.Dense(units=8,
  activation='tanh', kernel_initializer=g_init,
  bias_initializer='zeros')) 
model.add(K.layers.Dense(units=1,
  activation='sigmoid', kernel_initializer=g_init,
  bias_initializer='zeros'))

I used explicit Glorot initialization for layer weights and explicit zero-initialization for layer biases. These are the default initialization schemes used so I could have omitted the explicit initialization. I prefer explicit initialization — I think it’s more clear, and guards against confusion if the default initialization scheme changes.

I hit a few minor glitches as expected (deprecated parameter names, etc.) but I was ale to fix these glitches quickly. This is a big advantage of experience with Keras or any other machine learning library — you make fewer mistakes, but more importantly, over time you learn how to correct mistakes quickly.

Good fun!



Dealing with mistakes is a part of any kind of software development, including the development of machine learning systems. According to Wikipedia, cellophane was invented as a result of a mistake.

Cellophane is a thin, transparent sheet made of regenerated wood or cotton cellulose. Cellophane was invented by Jacques Brandenberger in 1900. He was inspired by seeing wine spill on a restaurant tablecloth, and he decided to create a cloth that could deal with that type of mistake.

Cellophane is sometimes used for contemporary women’s fashion (left and right images) but has been around for a long time (center image is from 1933).


Code below.

# banknote_tfk.py
# Banknote classification
# Keras 2.6.0 in TensorFlow 2.6.0 ("_tfk")
# Anaconda3-2020.02  Python 3.7.6  Windows 10

import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'  # suppress CPU warn

import numpy as np
import tensorflow as tf
from tensorflow import keras as K

class MyLogger(K.callbacks.Callback):
  def __init__(self, n):
    self.n = n   # print loss and acc every n epochs

  def on_epoch_end(self, epoch, logs={}):
    if epoch % self.n == 0:
      curr_loss =logs.get('loss')
      curr_acc = logs.get('accuracy') * 100
      print("epoch = %4d  loss = %0.6f  acc = %0.2f%%" % \
        (epoch, curr_loss, curr_acc))

def main():
  print("\nBanknote Authentication using Keras example ")
  np.random.seed(1)
  tf.random.set_seed(1)

  # 1. load data
  print("Loading Banknote data into memory ")
  train_file = ".\\Data\\banknote_train.txt"
  train_x = np.loadtxt(train_file, delimiter='\t',
    usecols=[0,1,2,3], dtype=np.float32)
  train_y = np.loadtxt(train_file, delimiter='\t',
    usecols=[4], dtype=np.float32)

  test_file = ".\\Data\\banknote_test.txt"
  test_x = np.loadtxt(test_file, delimiter='\t',
    usecols=[0,1,2,3], dtype=np.float32)
  test_y = np.loadtxt(test_file, delimiter='\t',
    usecols=[4], dtype=np.float32)

  # 2. define 4-(x-x)-1 deep NN model
  print("\nCreating 4-(8-8)-1 neural network ")
  g_init = K.initializers.glorot_uniform(seed=1)
  model = K.models.Sequential()
  model.add(K.layers.Dense(units=8, input_dim=4,
    activation='tanh', kernel_initializer=g_init, 
    bias_initializer='zeros')) 
  model.add(K.layers.Dense(units=8,
    activation='tanh', kernel_initializer=g_init,
    bias_initializer='zeros')) 
  model.add(K.layers.Dense(units=1,
    activation='sigmoid', kernel_initializer=g_init,
    bias_initializer='zeros'))  

  # 3. compile model
  opt = K.optimizers.SGD(learning_rate=0.01)  
  model.compile(loss='binary_crossentropy',
    optimizer=opt, metrics=['accuracy'])  

  # 4. train model
  max_epochs = 100
  log_every = 10
  my_logger = MyLogger(log_every)
  print("\nStarting training ")
  h = model.fit(train_x, train_y, batch_size=32,
    epochs=max_epochs, verbose=0, callbacks=[my_logger]) 
  print("Training finished ")

  # 5. evaluate model
  # np.set_printoptions(precision=4, suppress=True)
  eval_results = model.evaluate(test_x, test_y, verbose=0) 
  print("\nLoss, accuracy on test data: ")
  print("%0.4f %0.2f%%" % (eval_results[0], \
eval_results[1]*100))

  # 6. save model
  print("\nSaving trained model as banknote_model.h5 ")
  # mp = ".\\Models\\banknote_model.h5"
  # model.save(mp)

  # 7. make a prediction
  np.set_printoptions(formatter={'float': '{: 0.4f}'.format})
  inpts = np.array([[0.5, 0.5, 0.5, 0.5]], dtype=np.float32)
  pred = model.predict(inpts)
  print("\nPredicting authenticity for: ")
  print(inpts) 
  print("Probability of class 1 (forgery) = %0.4f " % pred)

if __name__=="__main__":
  main()
This entry was posted in Keras. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s