Kuzushiji-MNIST exploring

Overview

Kuzushiji-MNIST is MNIST like data set based on classical Japanese letters.
The following image is part of the data set. As you can see, this is composed of visually complex letters.



On this article, I’ll do simple introduction of Kuzushiji-MNIST and classification with Keras model.

How to write Dense block of DenseNets: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Dense block of DenseNets. DenseNets is one of the convolutional neural network models. If you have an experience of using fine-tuning or frequently tackle with image recognition tasks, probably you have heard that before.
DenseNets is composed of Dense blocks. It is expressed as the image below, which is quoted from https://arxiv.org/abs/1608.06993.

On the context of the history of convolutional neural network, ResNet helps the network to be deeper without degradation problem by the shortcut path to the output of the Residual module. DenseNets and Dense block is near concept from the different approach.

This article is to help to understand the basic concept of Dense block of DenseNets and how to write that. For coding, I’ll use Python and Keras.
About the ResNet and Residual module, please read the article below.

• How to write Residual module: understanding and coding with Keras

If you want to know the detail of DenseNets and Dense block, I recommend you read the article below.

• https://arxiv.org/abs/1608.06993

When you find a mistake, please let me know by comment or mail.

How to write Residual module: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Residual module. If you have experience of using fine tuning or frequently tackle with image recognition tasks, probably you have heard the network name, ResNet. ResNet is composed of Residual module, whose structure is expressed as below.

The image above is from https://arxiv.org/abs/1512.03385.
Basically, deeper neural network contributes to the better outcome. If you have enough computational resource(unfortunately, I don't have), for difficult task, you can approach it with really deep neural network. However, with deeper neural network, the problem of degradation comes, which makes it difficult to train the model. Residual module offers one of the solutions to this problem, meaning that with this, we can make deeper neural network by softening the difficulty of training.
For precise and better understanding, I recommend that you read the paper below. Here, I'll just show summary for simple and concise understanding and coding with Keras.

• https://arxiv.org/abs/1512.03385

If there are strange or wrong points, please let me know by comment or message.

How to write Inception module: understanding and coding with Keras

Abstract

This article covers the basic understanding and coding of Inception module.
GoogLeNet, which is composed by stacking Inception modules, achieved the state-of-the-art in ILSVRC 2014. And probably, many people already touched the models which have the name “Inception” by fine-tuning. Here, on this article, I'll deal with the Inception module.
To write the model, I'll use Keras with Python.
To deepen your knowledge, you can use the following paper.

• https://arxiv.org/abs/1409.4842

Object detection by CAM with Keras

Abstract

On this article, I'll try CAM(Grad-CAM) to high resolution images. Cam has the potential for object-detection. So, I will make CNN model and by CAM, check if it really works.
About CAM(Grad-CAM) itself, I'll recommend the theses below.

• Learning Deep Features for Discriminative Localization
• Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization

Some Fine tuning models with Keras: vgg16, vgg19, inception-v3 and xception

Overview

On this article, I'll try four image classification models, vgg16, vgg19, inception-v3 and xception with fine tuning. With Keras, we can easily try this.
About fine tuning itself, please check the article below.

• How to make Fine tuning model by Keras

The purpose of this article is to understand the points of fine tuning by doing some fine tuning models at once.

How to convert a Keras model to a TensorFlow Estimator

Overview

TensorFlow has the the function of converting Keras model to TensorFlow Estimator. On this article, I checked how to use it.
About the TensorFlow Estimator. Please read the article below and official pages.

• tf.estimator Quickstart
• tf.estimator of TensorFlow lets us concisely write deep neural network

Anyway, Keras lets us write neural network model relatively easily. But sometimes we need a model following TensorFlow. On this kind of cases, we write the model by Keras at first and after that can convert it to TnsorFlow's one.

Simple case of tf.keras

Overview

On this article, I rewrote the Keras code by tf.keras. From TensorFlow, we can use Keras by tf.keras. But I had never used this. So I checked and it was very simple and easy.

Fashion-MNIST exploring using Keras and Edward

Overview

On the article, Fashion-MNIST exploring, I concisely explored Fashion-MNIST dataset.

We can get access to the dataset from Keras and on this article, I’ll try simple classification by Edward.

Image generator of Keras: to make neural network with little data

Keras has image generator which works well when we don’t have enough amount of data. I’ll try this by simple example.

Overview

To make nice neural network model about images, we need much amount of data. In many cases, the shortage of data can be one of the big obstacles for goodness.
Keras has image generator and it can solves the problem.

InceptionV3 Fine-tuning model: the architecture and how to make

Overview

InceptionV3 is one of the models to classify images. We can easily use it from TensorFlow or Keras.
On this article, I’ll check the architecture of it and try to make fine-tuning model.

There are some image classification models we can use for fine-tuning.
Those model’s weights are already trained and by small steps, you can make models for your own data.

About the fine-tuning itself, please check the followings.

• How to make Fine tuning model by Keras
• VGG19 Fine-tuning model

Or TensorFlow and Keras have nice documents of fine-tuning.

From TensorFlow

• How to Retrain Inception’s Final Layer for New Categories

From Keras

• Applications

VGG16 Fine-tuning model

Overview

On the article, VGG19 Fine-tuning model, I checked VGG19’s architecture and made fine-tuning model. On the same way, I’ll show the architecture VGG16 and make model here.

There are some image classification models we can use for fine-tuning.
Those model's weights are already trained and by small steps, you can make models for your own data.

About the fine-tuning itself, please check the followings.

• How to make Fine tuning model by Keras
• VGG19 Fine-tuning model

VGG19 Fine-tuning model

Overview

On the article, How to make Fine tuning model, I made fine-tuning models by some pre-trained models. At that time, I didn't write about the pre-trained model's architecture and the train target area based on it.

This time, I foucused on the VGG19 as pre-trained model. And in a nutshell, I tried to make fine-tuning model in better manner, checking some important points.

How to make Fine tuning model by Keras

Overview

Fine-tuning is one of the important methods to make big-scale model with a small amount of data.

Usually, deep learning model needs a massive amount of data for training. But it is not always easy to get enough amount of data for that. To be added, in many cases, it takes much time to make model from the viewpoint of training. I know you don’t like to see one epoch of training using the time from sunrise to sunset. In some areas like image classification, you can use fine-tune method to solve this situation.

For example, when you try to make image classification model, very deep CNN model works well(sometimes and other time not). To make that kind of model, it is necessary to prepare a huge amount of data. However, by using the model trained by other data, it is enough to add one or some layers to that model and train those. It saves much time and data.

Here, I show this type of method, fine-tuning, by Keras.

Understand how to use keras's functional API

Overview

keras is awesome tool to make neural network. Being compared with Tensorflow, the code can be shorter and more concise. If you want to enter the gate to neural network, deep learning but feel scary about that, I strongly recommend you use keras.

keras has two types of writing ways. Here I introduce one of them, functional API.

Re-try CNN + KNN model

Overview

When I tried CNN + KNN model before, the training epoch was not enough(50) to check the characteristics. This time I trained 200 epoch on the CNN phase.

How to write diverged type neural network by keras

How to write Diverged neural network

Overview

Simple style neural network as below is easy to write by deep learning frame work. 

This time, I make diverged neural network whose route to output is diverged and merged. The image of this is like below.

The purpose of this article is following two points.

• see how to write diverged type neural network
• see how accurate and good this type of model is

I used keras to write. Although it is bit annoying to write this kind of neural network compared with simple type, keras makes diverged type of model in relatively easy manner.
About the model's characteristics and accuracy, it’s difficult to judge, because there is no simple model which is relevant to the diverged model. So, we can just check roughly.

Convolutional neural network scale experiment by keras

Overview

It is not easy to understand about convolutional neural network how the goodness changes when the nodes each layer has, layer’s number and other factors change.

For practical use of convolutional neural network, I experimented some types of convolutional neural network.

Convolutional neural network by keras

Make convolutional neural network model for mnist in keras

Overview

Convolutional neural network is one of the best solutions about image classification. In keras, it is relatively easy to make model.

Simple keras trial

Simple keras trial

Overview

By making simple newral network, I try to use keras.