Overview
With magenta and Slack’s Bots application, I made an image style change bot.This works as following gif.
When we post the image to the bot, it adds the style to the image and posts the images to the slack channel.
Reason of making this
Probably, some of the readers of this article already have husband or wife. And some of you have maybe experienced the following conversation with the partner.
I'm getting married and as possible as I can, I want to avoid the situation above.
Approach
As a data scientist and a machine learning engineer, the adventure to get fiancee’s understanding to my area just started. Anyway at first, to reach the understandings, I re-thought about my fiancee.She is
- art-oriented
- love museum
- illustrator/digital artist?
- on some services, selling drawings
art heronita (@heronita.art) * Instagram photos and videos
67 Followers, 79 Following, 58 Posts - See Instagram photos and videos from art heronita (@heronita.art)
Okay. There are some approaches.
This time, I decided to try to do something to her drawings by machine learning knowledge. And by the combination of the word “art and machine learning”, magenta hit on me. About magenta, later I’ll show the detail. Anyway, with magenta, I can add some styles to the drawings.
Next, I thought about the flow. If whenever she wants to do style change, she needs to send the image to me and wait for my response, this is not nice. This kind of dull-witted flow can lead to a misunderstanding to the technology. So, as a preparation, I adopted Slack between us. By using Slack as front-end, we can make clean flow.
magenta
As I wrote above, I used magenta to add style. magenta is machine learning library for art area. From the official page,Magenta is a research project exploring the role of machine learning in the process of creating art and music.There is a nice presentation about magenta.
When I checked the information about magenta, I found more information about music than about image. In a day, I want to try the function about music.
Anyway, this time, I used the image style changer. The image below is one of the examples.(This image is from the fiancee’s instagram account, with her permission.)
This is official page link.
How to make
At first, I thought for the purpose, I could use Slack's outgoing webhook and incoming webhook. But actually, Slack already has Bots application and I used this. On the web page, we can make configurations and get API token.On the server side, I used Python’s library slackclient and slacker. Actually, I can probably make just by slackclient. But about image posting to slack, I was stuck and from the viewpoint of fast prototype, I chose to use the both to avoid the problem for now. In a day, I’ll fix.(I know on this pattern, I sometimes won’t fix.)
About magenta’s stylization, I checked the official page and the following link. On my bot, I changed the following example way to the class and used it.
This is the code about magenta part. I need to fix some points. Anyway, as a prototype, it is enough.
import os
import random
import urllib2
import tensorflow as tf
from magenta.models.image_stylization import model
class Style():
def __init__(self):
self.original_image = None
self.style = 'varied'
self.checkpoints = 'check_points/multistyle-pastiche-generator-varied.ckpt'
self.num_styles = 32
def change_style(self, style):
self.style = style
if self.style == 'monet':
self.checkpoints = 'check_points/multistyle-pastiche-generator-monet.ckpt'
self.num_styles = 10
def generate_image(self, image):
styles = range(self.num_styles)
random.shuffle(styles)
which_styles = styles[0:6]
with tf.Graph().as_default(), tf.Session() as sess:
stylized_images = model.transform(tf.concat([image for _ in range(len(which_styles))], 0),
normalizer_params={'labels': tf.constant(which_styles),
'num_categories': self.num_styles, 'center': True,
'scale': True})
model_saver = tf.train.Saver(tf.global_variables())
model_saver.restore(sess, self.checkpoints)
stylized_images = stylized_images.eval()
return stylized_images
@staticmethod
def download_checkpoints(checkpoint_dir):
"""
This method makes a directory and downloads checkpoints data to that.
"""
# check directory
if not os.path.isdir(checkpoint_dir):
os.mkdir(checkpoint_dir)
url_prefix = 'http://download.magenta.tensorflow.org/models/'
checkpoints = ['multistyle-pastiche-generator-monet.ckpt', 'multistyle-pastiche-generator-varied.ckpt']
for checkpoint in checkpoints:
full_checkpoint = os.path.join(checkpoint_dir, checkpoint)
if not os.path.exists(full_checkpoint):
print 'Downloading', full_checkpoint
response = urllib2.urlopen(url_prefix + checkpoint)
data = response.read()
with open(full_checkpoint, 'wb') as fh:
fh.write(data)
And the prototype bot is the image below.
Later I'll upload whole code to my github account. The code is too big for blog and is still messy.
Little by little, I want to add the functions to the bot.