Simple example of how to use TensorBoard

Overview

On this article, through the simple regression, I’ll show how to observe the parameter’s behavior on TensorBoard.

TensorBoard is cool visualizing tool and by using it, our debug to model can be easier.

Regression

The purpose of this article is to observe parameters by visualization on TensorBoard. So, I just adapted simple regression model to iris data.

import numpy as np
from sklearn import datasets
import tensorflow as tf
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt

# data
iris = datasets.load_iris()
x = np.array([x[3] for x in iris.data]).reshape(iris.data.shape[0], 1)
y = np.array([x[0] for x in iris.data]).reshape(iris.data.shape[0], 1)

The model is to predict y by x as predictor. The both of predicted and predicting variables are metric.

# placeholder
x_ph = tf.placeholder(tf.float32, [None, 1])
y_ph = tf.placeholder(tf.float32, [None, 1])

# parameters
w = tf.Variable(tf.random_normal(shape=[1, 1]))
b = tf.Variable(tf.random_normal(shape=[1, 1]))

# model
output = tf.add(tf.matmul(x_ph, w), b)

The model is as below.
$y = w \times x + b$
The w is weight and the b is bias.

# loss
loss = tf.reduce_mean(tf.square(y_ph - output))

# optimizer
opt = tf.train.GradientDescentOptimizer(0.01)
train_step = opt.minimize(loss)

The loss and optimizer is like above.
After initializing the variables, let’s make them train.

sess = tf.Session()

# initialize
init = tf.global_variables_initializer()
sess.run(init)

batch_size = 25
# training
for i in range(1000):
    random_index = np.random.choice(150, size=batch_size)
    sess.run(train_step, feed_dict={x_ph: x[random_index], y_ph: y[random_index]})

By the training, the variables w and b are updated. Roughly, we can check those updated variables by plotting.

W = sess.run(w, feed_dict={x_ph: x[random_index], y_ph: y[random_index]})[0][0]
B = sess.run(b, feed_dict={x_ph: x[random_index], y_ph: y[random_index]})[0][0]

plt.scatter(x, y, color='blue')
x_base = np.linspace(np.min(x), np.max(x))
plt.plot(x_base, W * x_base + B, color='red')
plt.show()

More or less, it looks okay. Usually, we should be more careful to the validity. But for the purpose of the article, just go on.

To observe the Behavior on TensorBoard

When we want to observe parameter’s behaviors on TensorBoard, It needs two preparations.

• set the directory for the log
• point the target variables

On the code below, if the directory named iris exists, it deletes that and re-choosing that for the log store.

# visualizer
if tf.gfile.Exists('./iris'):
    tf.gfile.DeleteRecursively('./iris')
summary_writer = tf.summary.FileWriter('./iris', tf.get_default_graph())

On the model, we usually have some parameters to fix by training. To visualize those on TensorBoard, it’s necessary to write about those by the function tf.summary.scalar().

# summary
tf.summary.scalar('weight', tf.squeeze(w))
tf.summary.scalar('bias', tf.squeeze(b))

summary_op = tf.summary.merge_all()

The whole code is on the bottom of the article.
By executing that, the parameters are trained by the data and the log is on ./iris directory.

Check on TensorBoard

To check the parameters on TensorBoard, on terminal, you need to execute the following command.

tensorboard --logdir='iris'

On the browser, you can get access on http://localhost:6006 to check the output.



On this case, the output on TensorBoard is like the image above. About the parameters, we can see how the training went on.

Code

import numpy as np
from sklearn import datasets
import tensorflow as tf
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt

# data
iris = datasets.load_iris()
x = np.array([x[3] for x in iris.data]).reshape(iris.data.shape[0], 1)
y = np.array([x[0] for x in iris.data]).reshape(iris.data.shape[0], 1)


# placeholder
x_ph = tf.placeholder(tf.float32, [None, 1])
y_ph = tf.placeholder(tf.float32, [None, 1])

# parameters
w = tf.Variable(tf.random_normal(shape=[1, 1]), name='W')
b = tf.Variable(tf.random_normal(shape=[1, 1]), name='B')

# model
output = tf.add(tf.matmul(x_ph, w), b)

# loss
loss = tf.reduce_mean(tf.square(y_ph - output))

# optimizer
opt = tf.train.GradientDescentOptimizer(0.01)
train_step = opt.minimize(loss)


sess = tf.Session()

# visualizer
if tf.gfile.Exists('./iris'):
    tf.gfile.DeleteRecursively('./iris')
summary_writer = tf.summary.FileWriter('./iris', tf.get_default_graph())

# summary
tf.summary.scalar('weight', tf.squeeze(w))
tf.summary.scalar('bias', tf.squeeze(b))

summary_op = tf.summary.merge_all()

# initialize
init = tf.global_variables_initializer()
sess.run(init)

batch_size = 25
# training
for i in range(1000):
    random_index = np.random.choice(150, size=batch_size)
    _, summary = sess.run([train_step, summary_op], feed_dict={x_ph: x[random_index], y_ph: y[random_index]})

    log_writer = tf.summary.FileWriter('./iris')
    log_writer.add_summary(summary, i)