sum function with divide and conquer algorithm

Overview

By divide and conquer algorithm, I’ll write sum function. This article is my personal memo about a book.

Functional Kotlin: Book review

These days I've been on the journey of studying Functional programming and as one of the books, read Functional Kotlin.

I'll leave the review of it.

Awesome keyboard: My best buying

I think there are many people who are on the journey of finding the best keyboard for themselves. I was on that. On this post, I’ll introduce my personal best buying of keyboard.


As a data scientist, I spend a lot of time in front of my PC. Although I'm not a work-environment geek, I more or less care about some points. One of those is Keyboard.

Kuzushiji-MNIST exploring

Overview

Kuzushiji-MNIST is MNIST like data set based on classical Japanese letters.
The following image is part of the data set. As you can see, this is composed of visually complex letters.



On this article, I’ll do simple introduction of Kuzushiji-MNIST and classification with Keras model.

Data Science with Functional Programming on Python

Data Science with Functional Programming

Overview

On this article, I’ll show some functional programming approach to data science with Python. With functional approach, some pre-processing can be concise. Especially when you are reluctant to use pandas library on some situation, this kind of approach can lead to code-readability.

How to write Dense block of DenseNets: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Dense block of DenseNets. DenseNets is one of the convolutional neural network models. If you have an experience of using fine-tuning or frequently tackle with image recognition tasks, probably you have heard that before.
DenseNets is composed of Dense blocks. It is expressed as the image below, which is quoted from https://arxiv.org/abs/1608.06993.

On the context of the history of convolutional neural network, ResNet helps the network to be deeper without degradation problem by the shortcut path to the output of the Residual module. DenseNets and Dense block is near concept from the different approach.

This article is to help to understand the basic concept of Dense block of DenseNets and how to write that. For coding, I’ll use Python and Keras.
About the ResNet and Residual module, please read the article below.

• How to write Residual module: understanding and coding with Keras

If you want to know the detail of DenseNets and Dense block, I recommend you read the article below.

• https://arxiv.org/abs/1608.06993

When you find a mistake, please let me know by comment or mail.

How to write Residual module: understanding and coding with Keras

Abstract

This article covers basic understanding and coding of Residual module. If you have experience of using fine tuning or frequently tackle with image recognition tasks, probably you have heard the network name, ResNet. ResNet is composed of Residual module, whose structure is expressed as below.

The image above is from https://arxiv.org/abs/1512.03385.
Basically, deeper neural network contributes to the better outcome. If you have enough computational resource(unfortunately, I don't have), for difficult task, you can approach it with really deep neural network. However, with deeper neural network, the problem of degradation comes, which makes it difficult to train the model. Residual module offers one of the solutions to this problem, meaning that with this, we can make deeper neural network by softening the difficulty of training.
For precise and better understanding, I recommend that you read the paper below. Here, I'll just show summary for simple and concise understanding and coding with Keras.

• https://arxiv.org/abs/1512.03385

If there are strange or wrong points, please let me know by comment or message.

An insight into AUC objective function from the viewpoint of evaluation

Abstract

This article is to think about the model which is with AUC objective function by some evaluation methods. Also, I can say this is to think about the AUC objective function from the viewpoint of evaluation as the title of the article shows.
On the article, AUC as an objective function: with Julia and Optim.jl package, I made a model with AUC objective function. The predicted score by that was distributed in really narrow area, because AUC objective function is based on the order without caring the distance from explained variable. With some evaluation norms, the model's score seems not nice.
 
About this point, just in case, I'll leave the simple experiment.

Julia: version 0.6.3

AUC as an objective function: with Julia and Optim.jl package

Abstract

On this article, I'll do AUC optimization on logistic regression. With Julia's Optim package, relatively easily, we can optimize AUC objective function.

Julia: Version 0.6.3

Follow simple analysis workflow with Julia

Abstract

On this article, with Julia I'll roughly reproduce the simple analysis I did on Simple analysis workflow to data about default of credit card clients.
After I wrote that article, I thought to write the following ones. But, I want to follow the same flow with Julia at first. So, I'll do.

Simple analysis workflow to data about default of credit card clients

Abstract

These days I had opportunity of reading some papers about finance data analysis, meaning credit score, default rate and so on. Personally, I want to tackle with cutting-edge way as soon as possible. But, it is important to see from basic flow on this kind of case. So, here, on this article, I'll follow the basic work flow like univariate analytics with Logistic Regression.
To focus on basic flow and some characteristics, I'll ignore some manner to the data and modeling.
This article more or less follows the chapter 2 and 3 of the following article.

• https://edoc.ub.uni-muenchen.de/17143/1/Kraus_Anne.pdf

How to write Inception module: understanding and coding with Keras

Abstract

This article covers the basic understanding and coding of Inception module.
GoogLeNet, which is composed by stacking Inception modules, achieved the state-of-the-art in ILSVRC 2014. And probably, many people already touched the models which have the name “Inception” by fine-tuning. Here, on this article, I'll deal with the Inception module.
To write the model, I'll use Keras with Python.
To deepen your knowledge, you can use the following paper.

• https://arxiv.org/abs/1409.4842

Various ways of writing Neural Network with Flux: to write complex model

Abstract

Flux is one of the deep learning packages in Julia. It is flexible and easy to use. But, there are not enough examples to grasp the points, although the official documents and model zoo somehow work. So, here, on this article, I'll write down some types of model and the points where I was caught. I'm still on the phase of exploring Flux by reading the source code and trial-error. So, if you find something strange or mistake, please let me know.
On this article, I'll use Julia version 0.6.2.

Convolutional Neural Network with Julia: Flux

Abstract

Here, I'll make a convolutional neural network model by Flux with Julia. In the article, Deep learning with Julia: introduction to Flux, I made simple neural network with Flux. Neural network, especially convolutional neural network, is quite efficient in image classification area. So, this time, I'll make the convolutional neural network model to image classification.

Deep learning with Julia: introduction to Flux

Abstract

On this article, I'll try simple regression and classification with Flux, one of the deep learning packages of Julia.

How to make HTTP server for prediction of machine learning model with Julia

Abstract

On this article, I'll try Julia's HTTP server. Concretely, the goal is to make HTTP server that execute k-means’s prediction. About machine learning task, it is usual to set the learned model to HTTP server and post the data to that. So, as a first step of it on Julia, I'll try it. The package used here is HTTP.jl.
Here, I'll just touch the initial step and won’t follow the good or proper manner. When you make the HTTP server for machine learning task, I strongly recommend that you read the official document after this article.

Image segmentation and compression by K-means

Abstract

On this article, I'll try image segmentation and compression by K-means.
This is shown on the popular book, Pattern Recognition And Machine Learning, as an example of K-means. I've never used K-means with setting the segmentation and compression of images as a main purpose, because it is not practical way. But it looks fun on the book. So, I'll try.

Here, for experiment, the code is written in Julia.

EM algorithm with Initialization by K-means

Abstract

On this article, I'll check the EM algorithm with the initialized values by k-means. In many algorithms, initial values are very important theme. On EM algorithm, with inappropriate initial values, it takes much time for convergence and if the algorithm is naively written, it stops with error because of non-positive-definite.

So, I'll do experiment to check how much the accuracy, the number of iteration and time change with and without initialized values by k-means. Here, I'll just touch EM algorithm about the mixture of Gaussian case.
If there is a mistake or inappropriate points, please let me know by comment.

k-means++: Introduction and small experiment with Julia

Overview

On this article, I'll write about k-means++. To say precisely, I'll explain what k-means++ is and do small experiment with Julia. For the people who have experience of Python, Julia code is easy to read. So, basically no problem.
k-means++ is regarded as the algorithm to give nice initialization for k-means and sometimes can be used in other algorithm like EM algorithm.
If you find a mistake, please let me know on comment.

Probabilistic Clustering with EM algorithm: Algorithm and Visualization with Julia from scratch

Abstract

On this article, I'll write probabilistic clustering by EM algorithm from scratch with Julia. Here, I'll touch only about mixture of Gaussian case.
The outcome of clustering becomes below. To the simple artificial data, it is working.

Introduction to K-medoids: Algorithm and Visualization with Julia from scratch

Abstract

On this article, I'll write K-medoids with Julia from scratch.
Although K-medoids is not so popular algorithm if you compare with K-means, this is simple and strong clustering method like K-means. So, here, as an introduction, I'll show the theory of K-medoids and write it with Julia.
As a goal, I'll make animation like below.

Introduction to K-means: Algorithm and Visualization with Julia from scratch

Abstract

On this article, I'll write K-means with Julia from scratch and show animation to see how the algorithm works.
K-means is very simple unsupervised algorithm for clustering. So, when I start to study new programming language, I always use K-means as the theme for writing from scratch.
The following GIF shows how data points are classified into clusters on the way of algorithm going. Relatively easily, we can write K-means code and plot this kind of animation with Julia.



On this article, I used Julia with version 0.6.2.

Note for speeding up Julia's code

Abstract

On this article, I'll re-write the Julia code, which I wrote before, for speed up.
One of the huge advantages of Julia is its performance, speed. But to make advantage of it, we need to write the code in proper rule. So, here, I’ll re-write the code to more efficient one.

kNN by Julia from scratch

Abstract

On this article, I'll write naive kNN algorithm with Julia. Recently, I started to use Julia and need to practice. kNN is relatively simple algorithm and nice for practice. So, here, I'll write simple kNN with Julia.

Object detection by CAM with Keras

Abstract

On this article, I'll try CAM(Grad-CAM) to high resolution images. Cam has the potential for object-detection. So, I will make CNN model and by CAM, check if it really works.
About CAM(Grad-CAM) itself, I'll recommend the theses below.

• Learning Deep Features for Discriminative Localization
• Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization

Class Activation Map with Keras

Abstract

On this article, I'll try CAM, Class Activation Map, to mnist dataset on Keras.

Simple note for Julia DataFrame

Overview

On this article, I'll write down the Julia's basic dataframe manipulation.
I just started to use Julia. I want to continue to use this language as DS and ML tool. For that, at first I'll focus on the basic dataframe manipulation.
Here, the version of Julia is 0.6.2.

Try Julia: from install to simple mathematical operation

I started to use Julia. Although more or less one year ago, I touched Julia, it didn't become my daily tool. But these days I frequently hear about Julia in data science field and also the news about version 1.0 hit on me. So, I decided earnestly to tackle with it.
On this article, I'll arrange some information of Julia and show some mathematical and matrix operations.

Some Fine tuning models with Keras: vgg16, vgg19, inception-v3 and xception

Overview

On this article, I'll try four image classification models, vgg16, vgg19, inception-v3 and xception with fine tuning. With Keras, we can easily try this.
About fine tuning itself, please check the article below.

• How to make Fine tuning model by Keras

The purpose of this article is to understand the points of fine tuning by doing some fine tuning models at once.

How Regularization works

Overview

On this article, I'll write about regularization.
Regularization is important method to prevent from overfitting and used in many algorithms. If I try to write accurately in good manner, it will be so long article. So here, I don’t stick to mathematical accuracy so much and in rough way I’ll try to show the system.

As a different approach to overfitting, Dropout is also nice. About this, please check the article below.

• How Dropout works on Neural Network

Simple Bayesian Modeling on Edward by Hamiltonian Monte Carlo

Overview

On this article, I'll re-write by Edward the simple bayesian model that was written on Stan.
This time, my target is from the following article.

• Simple Bayesian modeling by Stan

Actually, I already wrote the article, Simple regression by Edward: variational inference. There, I re-wrote the model on Edward. But at that time, I used variational bayesian method for inference.
Stan uses Hamiltonian Monte Carlo. So, this time, I'll use Hamiltonian Monte Carlo on Edward and re-write the model.

Bayesian prediction interval from pystan output

On this article, I’ll leave the simple memo to get bayesian prediction interval from the sampled points of PyStan output and visualize it.
Basically, I’ll use the code from the article, Simple Bayesian modeling by Stan.

Nice video for time series data analysis on Python

These days, I have watched some videos about time series analysis. I'll write here about one of my recommendations.
Roughly, the video is about time series data manipulation on Python, to say more concretely, on Pandas.



On this video, you can know

• how to deal with time stamp data type
• simple time series data processing

Those are fundamental and necessary knowledges to deal with time series data on Python. This video lets us know those manipulation on Pandas.
If you are new to time series analysis on Python, you can get the knowledge about basic manipulation by this video.

Slack bot for image stylization by magenta

Overview

With magenta and Slack’s Bots application, I made an image style change bot.
This works as following gif.



When we post the image to the bot, it adds the style to the image and posts the images to the slack channel.

How to make the local level model with seasonal effect

Overview

On time-series analytics, we frequently need to think about a recurring pattern. In the context of time-series analytics, a recurring pattern is referred to as a seasonal effect.

For example, please see the image below. This image is the plotting of a time series data. As you can see, it has recurrent pattern.


On this article, I'll make the local level model with seasonal effects on Stan.

Simple trial of various types of time series analytics

By air passengers data, which is typical time-series data, I'll try some time-series analytics methods. Actually, about some points, I'm not sure if it is really appropriate or not. So, if you find some wrong or incorrect points, please let me know.

How to check autocorrelation on Python

To time series data, we usually check autocorrelation. As a memo, I’ll write down how to get the autocorrelation and the plot of it on Python.

Anti-pattern of the local level model with explanatory variable

Overview

On this article, I'll try anti-pattern of the local level model with explanatory variable.
Before, on the article, Local level model with explanatory variable to time series data on Stan , I made the local level model with explanatory variable to time series data. From the article, I set the model as followings.
$u_t \sim Normal(u_{t-1}, \sigma_ξ^2)$
$β_t \sim Normal(β_{t-1}, \sigma_τ^2)$
$y_t \sim Normal(u_t + β_t \times x_t, \sigma_e^2)$
There, I set state disturbances $σ_τ^2$ as zero. That’s because on the book, An Introduction to State Space Time Series Analysis (Practical Econometrics), it is written that these state disturbances are usually fixed on zero to establish a stable relationship between $y_t$ and $x_t$. This time, as experiment, I’ll try the model without setting $σ_τ^2$ as zero, meaning I’ll set $β_t$ as changeable.

Summary of local level model and local linear trend model to time series data

Overview

On this article, I’ll leave the summary about local level model and local linear trend model.
The both models are for time series analysis. Those are too simple to adapt for real data as they are. But those are very fundamental in many cases and by adding some other factors, those can become practical. So, here, I’ll leave rough memos about those.
As a text book, I’m using the following book. This article responds to chapters two and three.

Local Linear Trend Model for time series analysis on Stan

Overview

On this article, I’ll make the local linear trend model to artificial time series data by Stan. Before, I made a local level model on Stan on the article, Local level model to time series data on Stan.
By adding the slope variable to that model, I’ll make the local linear trend model.
I used this book as reference.

TensorBoard's graph visualization with tf.variable_scope

Overview

On this article, I’ll write down the note about the example of tf.variable_scope(), meaning how to arrange the graph for TensorBoard.
The target code is from the article below.

• Simple tutorial to write deep neural network by TensorFlow

On a tensorboard, without using namespace, the graph information becomes complex. Namespace will solve the problem and makes it easy to debug.

Edward modeling to an artificial data

Overview

On the article below, I switched method on Edward model from variational method to Hamiltonian Monte Carlo.

• Hamiltonian Monte Carlo on TensorFlow and Edward

As an another example, I'll try same thing to the model of the following article.

• Edward modeling to artificial data with random effects

In a nutshell, I'll make model for an artificial data and get sample by Hamiltonian Monte Carlo.

Hamiltonian Monte Carlo on TensorFlow and Edward

Overview

On this article, I tried Hamiltonian Monte Carlo algorithm to the simple data by TensorFlow and Edward.
Edward lets us use variational inference, Gibbs sampling and Monte Carlo method. And by relatively small changes, we can switch the methods. So I'll try simple HML model here.
About Hamiltonian Monte Carlo itself, I'll write another article for it.

How to convert a Keras model to a TensorFlow Estimator

Overview

TensorFlow has the the function of converting Keras model to TensorFlow Estimator. On this article, I checked how to use it.
About the TensorFlow Estimator. Please read the article below and official pages.

• tf.estimator Quickstart
• tf.estimator of TensorFlow lets us concisely write deep neural network

Anyway, Keras lets us write neural network model relatively easily. But sometimes we need a model following TensorFlow. On this kind of cases, we write the model by Keras at first and after that can convert it to TnsorFlow's one.

Simple case of tf.keras

Overview

On this article, I rewrote the Keras code by tf.keras. From TensorFlow, we can use Keras by tf.keras. But I had never used this. So I checked and it was very simple and easy.

Get started with golearn: Machine Learning with Go

Overview

On this article, I’ll try golearn package on Golang.
These days, typical environment sets for data science and machine learning are Python, R, R-studio and Jupyter, although it depends on the purposes and phases. When I personally do something, I always use Python and Jupyter. But of course other programming languages have machine learning libraries and those are sometimes used.
Here, I’ll try golearn package, which is the package for machine learning.
This is the official page.

• GoLearn

As a first step, through kNN algorithms, I’ll follow the basic step of that.

Local level model with explanatory variable to time series data on Stan

Overview

On this article, I'll make the local level model with explanatory variable to time series data on Stan.
Before, I made the simple local level model on Stan. In the practical situation, we frequently need to make model with some explanatory variables. So, I'll make simple local level model with explanatory variables here.
As a reference, I’m using the following book. This article is dealing with the chapter 5 of the book.