Tuesday, June 13, 2017

Breakout by tensorflow model

Overview

I made a Tensorflow model of breakout by the data which is from my playing.
The purpose of this is to visually observe how outcome of the prediction works. So this time ‘theoretical accuracy’ should be left behind.
I just made simple and easy model without thinking about details and tried to make the model play breakout like the following image.


The one I used as breakout is from address.
breakout



Motivation


Sometimes I see the news and code to make machine learning model play games. I just want to try that not by using much time.
And in my opinion, the combination of machine learning and games is very nice to feel the effect of the model. So, by this, I wanted to know my model is really working.

Premise


The summary of the breakout is like these.


  • you can move paddle to reflect the ball
  • the mouse position corresponds with the paddle x-coordinate


This means when the ball's y-coordinate becomes same as paddle's y-coordinate, if x-coordinates of the both are same, the ball is reflected.
So this is not difficult task.

Make data


By actual play, I made data for model. This time, the concept of the trial is 'simple and easy(logically)'. So, the set of explaining and explained variables are prepared not in good manner.
I set the explaining and explained variables as followings.
When the frame of the game updates itself, the coordinate of paddle and ball is written out. The ball's x-coordinate and y-coordinate are explaining variables. The paddle position is explained variable.

Make model


# import library
import tensorflow as tf
import pandas as pd

# read data
data_x = pd.read_csv('/somewhere/ball_movement_x.csv', header=None)
data_y = pd.read_csv('/somewhere/ball_movement_y.csv', header=None)
movement = pd.read_csv('/Users/shu/Train/qiita/mouse_movement.csv', header=None)

data = pd.DataFrame({'x_data': data_x.ix[:, 0],
                     'y_data': data_y.ix[:, 0]
                     })
teacher = pd.DataFrame({'teacher':movement.ix[:, 0]})

# make model
# set placeholder
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

# set variable
W = tf.Variable(tf.zeros([2,1]))
B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

# loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

# how to update parameters
train_step = tf.train.GradientDescentOptimizer(0.001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

Let's look one by one.

# make model
# set placeholder
X = tf.placeholder(tf.float32, shape = [None, 2])
Y = tf.placeholder(tf.float32, shape = [None, 1])

# set variables
W = tf.Variable(tf.zeros([2,1]))
B = tf.Variable(tf.zeros([1]))

y_hypo = tf.matmul(X, W) + B

The data is assigned on placeholder on the training phase and it update variables(parameters).

# loss function
cost = tf.reduce_mean(tf.square(y_hypo - Y))

# how to update parameters
train_step = tf.train.GradientDescentOptimizer(0.001).minimize(cost)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):
        sess.run(train_step, feed_dict={X:data, Y:teacher})
    parameter_W = sess.run(W)
    parameter_B = sess.run(B)

By training, the algorithm updates parameters to fix.

  • parameter_W: parameter
  • parameter_B: parameter
  • X: input data
  • tf.matmul(X, parameter_W) + parameter_B: predicted paddle x-coordinate


execute


You can add the model information to the breakout script to get predicted x-coordinate of the paddle.
It works as following.

Labels: , , ,