Sunday, December 17, 2017

tf.estimator of TensorFlow lets us concisely write deep neural network

Overview


On this article, I’ll re-write the simple deep neural network model to iris data by tf.estimator. From official page,
TensorFlow’s high-level machine learning API (tf.estimator) makes it easy to configure, train, and evaluate a variety of machine learning models.
By comparing with the original code, I’ll check how much it becomes concise and how to use tf.estimator.



Deep neural network model to iris data


On the following article, I made a simple deep neural network model for regression by TensorFlow. Here, I’ll use the code of that as original code and re-write by tf.estimator.

Simple tutorial to write deep neural network by TensorFlow

On this article, I'll show simple deep neural network(DNN) model for regression by TensorFlow. TensorFlow is open source library from Google. From the official web site, TensorFlow™ is an open source software library for numerical computation using data flow graphs. TensorFlow is an open-source software library for machine learning across a range of tasks.

Re-write by tf.estimator

Write model


Basically, I’ll follow the official tutorial about the usage of tf.estimator.


As a premise, you need to import same libraries as original codes.
At first, the data and pre-processing, normalization, are common. Concretely, the common part is following. 

# set random number
seed = 42
tf.set_random_seed(seed)
np.random.seed(seed)

# data
iris = datasets.load_iris()
x = np.array([x[0:3] for x in iris.data])
y = np.array([x[3] for x in iris.data])

x_train, x_test, y_train, y_test = train_test_split(x, y, train_size=0.7)

# normalization
mms = MinMaxScaler()
x_train = mms.fit_transform(x_train)
x_test = mms.transform(x_test)

From here, everything look different from the original. Without tf.estimator, we usually set the things below.

  • placeholders for data
  • variables
  • loss function
  • how to decrease the loss
  • loop for training

Usually, each of them takes some or many lines.
By tf.estimator, we can write those concisely in the form of options like below. 

# Specify that all features have real-value data
feature_columns = [tf.feature_column.numeric_column("x", shape=[3])]

# Hidden layers
regressor = tf.estimator.DNNRegressor(feature_columns=feature_columns,
                                        hidden_units=[6, 4],
                                        model_dir="/tmp/iris_model")

# Define the training inputs
train_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": x_train},
    y=y_train,
    num_epochs=None,
    shuffle=True)

regressor.train(input_fn=train_input_fn, steps=50)

Roughly, I wrote input data’s shape, hidden layers information, data information and training information. And it takes just few lines.

Evaluation


Basically, on evaluation, we just need to write data information and use evaluate() method. As you can see, this is almost same code as the model writing. 

# Define the test inputs
test_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": x_test},
    y=y_test,
    num_epochs=1,
    shuffle=False)

# Evaluate by test data.
eva = regressor.evaluate(input_fn=test_input_fn)
print(eva)
INFO:tensorflow:Starting evaluation at 2017-12-17-01:10:01
INFO:tensorflow:Restoring parameters from /tmp/iris_model/model.ckpt-50
INFO:tensorflow:Finished evaluation at 2017-12-17-01:10:02
INFO:tensorflow:Saving dict for global step 50: average_loss = 0.614151, global_step = 50, loss = 27.6368
{'average_loss': 0.61415124, 'global_step': 50, 'loss': 27.636806}

The code is simple. It just changed the model writing part’s data information. On the model writing, we used train data of course. Here we just need to change that to test data.
But on evaluation, you need to be careful about the num_epochs option. From the official page,

Note: The num_epochs=1 argument to numpy_input_fn is important here. test_input_fn will iterate over the data once, and then raise OutOfRangeError. This error signals the classifier to stop evaluating, so it will evaluate over the input once.

Prediction


We can predict on just same way as evaluation. 

predict_input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": x_test},
    num_epochs=1,
    shuffle=False)

predictions = [p['predictions'][0] for p in regressor.predict(input_fn=predict_input_fn)]

After the re-writing


Now just after re-writing the simple deep neural network by tf.estimator, I’m thinking about the situation where I may use this tf.estimator. As I wrote, tf.estimator lets us write deep neural network by few lines. But as a choice, we can also use high-level neural networks APIs like Keras. Personally, if there is no restriction of libraries, when I write not so complex model, I want to use Keras, not TensorFlow with tf.estimator.
I usually use the both of TensorFlow and Keras. When the model is relatively simple, I use Keras. When the model needs some tricks, I use TensorFlow. About those points, I have more or less same opinion as the one showed on the article below.


Anyway, when I use TensorFlow, on my use case, usually the model is something complex and I don’t think tf.estimator fulfill its role that makes the code concise. I’m just after finishing simple tutorial and still going on exploring how to use tf.estimator. Through some examples, I want to try to use it more to know the advantages better.
Labels: ,