Overview
keras is awesome tool to make neural network. Being compared with Tensorflow, the code can be shorter and more concise. If you want to enter the gate to neural network, deep learning but feel scary about that, I strongly recommend you use keras.
keras has two types of writing ways. Here I introduce one of them, functional API.
What is functional API?
The Keras functional API is the way to go for defining complex models, such as multi-output models, directed acyclic graphs, or models with shared layers.
The explanation is from keras's official document. Shortly, I can say when you want to make complex model, functional API is necessary.
Why is functional API necessary?
Let’s think about the reason that knowledge of functional API is necessary when you make a complex model.
What kind of model do you make by keras?
At first, you need to know what you make by keras.
From the official document
Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.So, keras is for neural networks. When you need to make neural network model, you can choose keras for that.
Next, why do you make not other machine learning models but neural network?
In my opinion, neural network has following characters.
- good prediction power to high dimension data
- it receives flexible input and output
Of course, about the choice of methods, anyone can have his/her own restriction. But basically, you should use neural network only under the circumstances that the data are too complex to solve by other machine leaning methods or the problem forces you to set flexible input or output on the model.
In a nutshell, the problem you try to solve by neural network is complex. It means the model also can be complex easily.
Functional API is necessary for complex model
keras makes model by piling layers. Although you can make easy model by sequential model, when you try to make model which has shared layers or some input or output points, functional API is necessary.
In short, the problems which should be solved by neural network tends to have high dimension or some complexities, leading the model to be complex easily. So, you need to use functional API to make that model.
Check functional API
Let’s check how to write model by functional API. The code below is from keras tutorial.
from keras.layers import Input, Dense
from keras.models import Model
inputs = Input(shape=(784,))
x = Dense(64, activation='relu')(inputs)
x = Dense(64, activation='relu')(x)
predictions = Dense(10, activation='softmax')(x)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(data, labels) # starts trainingMain characteristics are as followings.
- By Input() function, you need to set input data's dimension
- Take the layer one before as layer function's input and assign it on the variable
- Set input and output by Model() function
There are many other characteristics. But from the viewpoint of comparison with Sequential model, the points above are important.
Though Sequential model is easier to use, functional API's style is not difficult and looks concise to express networks.
Because without functional API you can't write complex neural network model, functional API is necessary. So, we should see the complex pattern which can be written only by functional API.
Here, as one of examples, I made diverged type model by functional API.
from keras.layers import Input, Dense, merge
from keras.models import Model
inputs = Input(shape=(784,))
x = Dense(64, activation='relu')(inputs)
x_1 = Dense(64, activation='relu')(x)
x_2 = Dense(32, activation='relu')(x)
x_m = merge([x_1, x_2], mode='concat')
predictions = Dense(10, activation='softmax')(x_m)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(data, labels) # starts trainingThe model above gets input and sends to 64 nodes layer and there, it is diverged to 2 types of layers which are composed of 64 and 32 nodes. And it is merged.
There are so many things you can try by functional API and keras's functional API page has those examples.
Actually how do you deal with functional API?
keras has two types of writing styles, sequential model and functional API. The point is when you make model by Keras, when do you use functional API?More concretely, you have two stances as followings and which one is better?
One, you start to make model by functional API. Two, you start with sequential model and at the timing you notice complex model is necessary, change to use functional API.
I think there is no definite answer about that. It depends on the person's work flow styles. But I want to recommend that you use functional API from the first.
If you know the model should be complex at the stage before starting modeling, of course functional API is only choice.
If it’s not, you can start with sequential model. But neural network can easily get complex and frequently functional API becomes necessary. So, it frequently happens that you need to re-write model by functional API if you start with sequential model. It takes more time to re-write sequential model to functional API model than to start with functional API model and make it more complex.
Usually, deep leaning modeling is based on trial and error. Although simple model writing is important, the writing style which makes trial and error easy to go is much more important.