Various ways of writing Neural Network with Flux: to write complex model

Abstract

Flux is one of the deep learning packages in Julia. It is flexible and easy to use. But, there are not enough examples to grasp the points, although the official documents and model zoo somehow work. So, here, on this article, I'll write down some types of model and the points where I was caught. I'm still on the phase of exploring Flux by reading the source code and trial-error. So, if you find something strange or mistake, please let me know.
On this article, I'll use Julia version 0.6.2.

Convolutional Neural Network with Julia: Flux

Abstract

Here, I'll make a convolutional neural network model by Flux with Julia. In the article, Deep learning with Julia: introduction to Flux, I made simple neural network with Flux. Neural network, especially convolutional neural network, is quite efficient in image classification area. So, this time, I'll make the convolutional neural network model to image classification.

Deep learning with Julia: introduction to Flux

Abstract

On this article, I'll try simple regression and classification with Flux, one of the deep learning packages of Julia.

Practical hack to make deep learning model

Overview

Neural network has a lot of flexibility in its design. You can choose and set many components and options. Because of that, to make more optimized network, you need to know and care about the procedures to adjust those to update your network efficiently.
Here, I arranged neural network’s components and in which procedure those should be adjusted.

Re-try CNN + KNN model

Overview

When I tried CNN + KNN model before, the training epoch was not enough(50) to check the characteristics. This time I trained 200 epoch on the CNN phase.

CNN + KNN model accuracy

Overview

On the contest site like Kaggle, we can see many trials and good scores by the combination of some methods.
For example, you can get scores by logistic regression and lasso regression. You can make xgboost model by using those scores.
This time, about cifar-10, I make CNN model. And by using the score, I check KNN scores.

The pragmatic procedure of making CNN model

Overview

On the image classification modeling, you need to understand how good your model is, meaning not absolute accuracy itself but relative meaning of the accuracy.

This is the example. Your first trial model's validation accuracy is 0.6. How do you think about it?
Without knowing the unique label number, ratio, and the data's difficulty, only answer you can return is "I don't know".
To evaluate the model, not only the absolute accuracy but also the base score to compare with are necessary.

If the modeling trial and error don't take much time, you can feel the scale of goodness and accuracy by many trial samples. But usually image classification model takes much time to make themselves.
How do we do the shortcut?

How to write diverged type neural network by keras

How to write Diverged neural network

Overview

Simple style neural network as below is easy to write by deep learning frame work. 

This time, I make diverged neural network whose route to output is diverged and merged. The image of this is like below.

The purpose of this article is following two points.

• see how to write diverged type neural network
• see how accurate and good this type of model is

I used keras to write. Although it is bit annoying to write this kind of neural network compared with simple type, keras makes diverged type of model in relatively easy manner.
About the model's characteristics and accuracy, it’s difficult to judge, because there is no simple model which is relevant to the diverged model. So, we can just check roughly.

Googles's Tensorflow Object Detection API trial

Try Google’s TensorFlow Object Detection API

Overview

Google sent to the world awesome object detector.
When I tried object detection before by myself, I strongly felt it was hard job and even small trial took much time.
Not to be late to the growing technology about image detection, I tried object detection tutorial today.

Method for efficient neural network

Overview

Usually, neural network’s training takes much time and doesn’t go well. There are some ways to make that efficiently go.
Here, I list up those and summarize.
By using those method, the training go well and good model can be made.

How to use Inception v3

Low cost image classification by CNN, convolutional neural network

Overview

These days, CNN(convolutional neural network) is almost regarded as the best answer to classify images.
But it has many rules.

• it needs huge amount of images
• it takes much time to train
• it needs slow and gradual steps to find good network model to attein the goal
• it needs high spec environment to do try-and-error

Of course, there are free data sets and kinda sample network which works easily in short time. But at the case you practically make model to solve some practical problem personaly or oficilally, those restrictions can face with you.
The combination of Inception v3 model and fine tune can solve the point.

Convolutional neural network scale experiment by keras

Overview

It is not easy to understand about convolutional neural network how the goodness changes when the nodes each layer has, layer’s number and other factors change.

For practical use of convolutional neural network, I experimented some types of convolutional neural network.

Breakout by tensorflow model

Overview

I made a Tensorflow model of breakout by the data which is from my playing.
The purpose of this is to visually observe how outcome of the prediction works. So this time ‘theoretical accuracy’ should be left behind.
I just made simple and easy model without thinking about details and tried to make the model play breakout like the following image.

The one I used as breakout is from address.
breakout

Simple keras trial

Simple keras trial

Overview

By making simple newral network, I try to use keras.