Getting Familiar with Keras

Neural networks are computing systems loosely inspired by brain connectivity. In short, neural networks make a series of transformations to input, the results of which are used as features during learning. Keras is an open-source library in python that enables easy experimentation with neural networks. Keras provides many of the building blocks of neural networks including objective functions, optimizers, activation functions, various types of layers, and a host of additional tools. In this post, we will build three regression neural network models using the Keras library.

The two regression problems we will focus on are as follows:

1. Predicting the Value of a Car ( data )
2. Predicting Fuel Efficiency ( data )

Let’s get started!

Predicting the Value of a Car

To start, let’s import pandas and read the data into a data frame:

import pandas as pd 
df = pd.read_csv("cars.csv")

Let’s print the first five rows:

print(df.head())

Next, we will define our input and output variables. We will use ‘age’, ‘gender’, ‘mile’, ‘debt’, and ‘income’ to predict ‘sales:

import numpy as np
X = np.array(df[['age', 'gender', 'miles', 'debt', 'income']])
y = np.array(df['sales'])

We now need to split our data for training and testing. We will import the ‘train_test_split’ method from ‘sklearn’:

from sklearn.model_selection import train_test_split

We then define our test set as a random sample (20% of our data):

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 42)

Next, we need to reshape our array of labels:

y_train = np.reshape(y_train, (-1,1))

We now need to scale our data for training. Specifically, we need to ensure our input and output training values have both negative and positive values. This makes the optimization step, where we minimize error, more flexible.

To scale our data we do the following:

X_scaled = MinMaxScaler()
y_scaled = MinMaxScaler()X_scaled.fit(X_train)
X_scaled = X_scaled.transform(X_train)y_scaled.fit(y_train)
y_scaled = y_scaled.transform(y_train)

Now we can define our model. First, we will need to import a few packages:

from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense

Next, we define a sequential model object:

model = Sequential()

Let’s build a simple neural network with an input, hidden layer, and output layer. The input and hidden layers will have 32 neurons:

model.add(Dense(32, input_dim=5, kernel_initializer='normal', activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(1, activation='linear'))

Next, we will compile the model. We will use the ‘adam’ optimizer and mean square error loss function:

model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'])

Finally, we fit our model. Let’s fit with 100 epochs and a batch size of 10:

model.fit(X_scaled, y_scaled, epochs=100, batch_size=10)

We see that both mean squared error and mean absolute error are both pretty low which is good.

Now, let’s make predictions on the test set:

y_pred = model.predict(X_test)

Next, we can visualize our results. Let’s import matplotlib and seaborn and display a scatter plot of true values vs. predictions:

import matplotlib.pyplot as plt 
plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Predictions')

This doesn’t exactly look like a straight line, which is the relationship we want. Let’s try more neurons. I’ll use 64 neurons:

model = Sequential()
model.add(Dense(64, input_dim=5, kernel_initializer='normal', activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'])
model.fit(X_scaled, y_scaled, epochs=100, batch_size=10)y_pred = model.predict(X_test)plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Predictions')

I encourage you to play our with the number of layers, neurons, epochs and the batch size to improve performance. You can also try some other optimizers, instead of ‘adam’ , like ‘rmsprop’, or ‘sgd’.

Predicting Fuel Efficiency

Now, let’s move on to another problem. Here, we will build a neural network used to predict fuel efficiency.

Let’s import the data and print the first five rows:

df = pd.read_csv("auto-mpg.csv")
print(df.head())

Next we will define our input and output. We will use ‘Cylinders’, ‘Displacement’, ‘Horsepower’, ‘Weight’, ‘Acceleration’, ‘Model Year’, and ‘Origin’ to predict miles per gallon (MPG):

X = np.array(df[['Cylinders','Displacement','Horsepower','Weight',
 'Acceleration', 'Model Year', 'Origin']])
y = np.array(df['MPG'])

We now need to split our data for training and testing. We will define our test set as a random sample of our data:

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 42)

Next, we need to reshape our array of labels:

y_train=np.reshape(y_train, (-1,1))

We now scale our data:

X_scaled = MinMaxScaler()
y_scaled = MinMaxScaler()X_scaled.fit(X_train)
X_scaled = X_scaled.transform(X_train)y_scaled.fit(y_train)
y_scaled = y_scaled.transform(y_train)

Now let’s define our model. Let’s start with a neural network with input and hidden layers with 64 neurons. The input dimension is 7, the optimizer is ‘adam’ and the loss function is ‘mse’. We will also use 1000 epochs and a batch size of 10 :

model = Sequential()
model.add(Dense(64, input_dim=7, kernel_initializer='normal', activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'], validation_split = 0.2)
model.fit(X_scaled, y_scaled, epochs=1000, batch_size=10)

Now, let’s visualize our results:

y_pred = model.predict(X_test)
plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Predictions')

It turns out scaling the data in this case worsens performance. Let’s try with the original triaining samples:

model = Sequential()
model.add(Dense(64, input_dim=7, kernel_initializer='normal', activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'], validation_split = 0.2)
model.fit(X_train, y_train, epochs=1000, batch_size =10)
y_pred = model.predict(X_test)import matplotlib.pyplot as plt 
plt.scatter(y_test, y_pred)
plt.xlabel('True Values')
plt.ylabel('Prediction')

We can see that we achieve much better results here. In the previous example, scaling the data improved performance while in the subsequent example performance was worse with scaled data. This demonstrates that there is rarely a golden bullet solution to building accurate models. Much of the model building process is driven by trial and error guided by experience. I encourage you to apply these models to other prediction problems. For example, you may be interested in using neural networks to predict wildfire size using US wildfire data .

To summarize, in this post we walked through the model building process for two regression problems: predicting the value of a car and predicting fuel efficiency. We discussed how to initialize sequential model objects, add layers, add neurons, specify optimizers, specify epochs and specify batch sizes. We also went over model validation using true vs. predicted scatterplots. I hope you found this post useful. The code from this post is available on GitHub . Thank you for reading and happy machine learning!