Tuesday, July 17, 2018

How to write Dense block of DenseNets: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Dense block of DenseNets. DenseNets is one of the convolutional neural network models. If you have an experience of using fine-tuning or frequently tackle with image recognition tasks, probably you have heard that before.
DenseNets is composed of Dense blocks. It is expressed as the image below, which is quoted from https://arxiv.org/abs/1608.06993.



On the context of the history of convolutional neural network, ResNet helps the network to be deeper without degradation problem by the shortcut path to the output of the Residual module. DenseNets and Dense block is near concept from the different approach.

This article is to help to understand the basic concept of Dense block of DenseNets and how to write that. For coding, I’ll use Python and Keras.
About the ResNet and Residual module, please read the article below.
If you want to know the detail of DenseNets and Dense block, I recommend you read the article below.
When you find a mistake, please let me know by comment or mail.

Monday, July 9, 2018

How to write Residual module: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Residual module. If you have experience of using fine tuning or frequently tackle with image recognition tasks, probably you have heard the network name, ResNet. ResNet is composed of Residual module, whose structure is expressed as below.


The image above is from https://arxiv.org/abs/1512.03385.
Basically, deeper neural network contributes to the better outcome. If you have enough computational resource(unfortunately, I don't have), for difficult task, you can approach it with really deep neural network. However, with deeper neural network, the problem of degradation comes, which makes it difficult to train the model. Residual module offers one of the solutions to this problem, meaning that with this, we can make deeper neural network by softening the difficulty of training.
For precise and better understanding, I recommend that you read the paper below. Here, I'll just show summary for simple and concise understanding and coding with Keras.
If there are strange or wrong points, please let me know by comment or message.

Friday, June 29, 2018

An insight into AUC objective function from the viewpoint of evaluation

Abstract

This article is to think about the model which is with AUC objective function by some evaluation methods. Also, I can say this is to think about the AUC objective function from the viewpoint of evaluation as the title of the article shows.
On the article, AUC as an objective function: with Julia and Optim.jl package, I made a model with AUC objective function. The predicted score by that was distributed in really narrow area, because AUC objective function is based on the order without caring the distance from explained variable. With some evaluation norms, the model's score seems not nice.
 
About this point, just in case, I'll leave the simple experiment.

Julia: version 0.6.3


Thursday, June 28, 2018

AUC as an objective function: with Julia and Optim.jl package

Abstract

On this article, I'll do AUC optimization on logistic regression. With Julia's Optim package, relatively easily, we can optimize AUC objective function.

Julia: Version 0.6.3


Sunday, June 24, 2018

Follow simple analysis workflow with Julia

Abstract

On this article, with Julia I'll roughly reproduce the simple analysis I did on Simple analysis workflow to data about default of credit card clients.
After I wrote that article, I thought to write the following ones. But, I want to follow the same flow with Julia at first. So, I'll do.


Thursday, June 21, 2018

Simple analysis workflow to data about default of credit card clients

Abstract

These days I had opportunity of reading some papers about finance data analysis, meaning credit score, default rate and so on. Personally, I want to tackle with cutting-edge way as soon as possible. But, it is important to see from basic flow on this kind of case. So, here, on this article, I'll follow the basic work flow like univariate analytics with Logistic Regression.
To focus on basic flow and some characteristics, I'll ignore some manner to the data and modeling.
This article more or less follows the chapter 2 and 3 of the following article.

Sunday, June 10, 2018

How to write Inception module: understanding and coding with Keras

Abstract

This article covers the basic understanding and coding of Inception module.
GoogLeNet, which is composed by stacking Inception modules, achieved the state-of-the-art in ILSVRC 2014. And probably, many people already touched the models which have the name “Inception” by fine-tuning. Here, on this article, I'll deal with the Inception module.
To write the model, I'll use Keras with Python.
To deepen your knowledge, you can use the following paper.

Tuesday, June 5, 2018

Various ways of writing Neural Network with Flux: to write complex model

Abstract

Flux is one of the deep learning packages in Julia. It is flexible and easy to use. But, there are not enough examples to grasp the points, although the official documents and model zoo somehow work. So, here, on this article, I'll write down some types of model and the points where I was caught. I'm still on the phase of exploring Flux by reading the source code and trial-error. So, if you find something strange or mistake, please let me know.
On this article, I'll use Julia version 0.6.2.


Wednesday, May 30, 2018

Convolutional Neural Network with Julia: Flux

Abstract

Here, I'll make a convolutional neural network model by Flux with Julia. In the article, Deep learning with Julia: introduction to Flux, I made simple neural network with Flux. Neural network, especially convolutional neural network, is quite efficient in image classification area. So, this time, I'll make the convolutional neural network model to image classification.

Sunday, May 27, 2018

Deep learning with Julia: introduction to Flux

Abstract

On this article, I'll try simple regression and classification with Flux, one of the deep learning packages of Julia.