Tensorflow Advanced : Part 1

Machine Learning › Tensorflow
,

Functional API

An example of sequential architecture

1
2
3
4
5
seq_model = Sequential([
		Flatten(input_shape=(28,28)),
		Dense(128,activation='relu'),
		Dense(10,activation='softmax')
])

Now we will use functional API instead of the sequential one

3 steps for writing in functional API :

define input

1
2
3
from tensorflow.keras.layers import Input

input = Input(shape=(28,28))

define layers

  • connect each layers using python functional syntax
  • python functional syntax : current layer is a function and previous layer is a parameter to this function
1
2
3
x = Flatten()(input)
x = Dense(128,activation='relu')(x)
pred = Dense(10,activation='softmax')(x)

define model

  • give only input and output layers
1
2
3
from tensorflow.keras.layers import Model

func_model = Model(inputs=input,outputs=pred)

Layer Stacking

1
2
3
4
5
6
7
8
9
10
# instantiate the input Tensor
input_layer = tf.keras.Input(shape=(28, 28))

# stack the layers using the syntax: new_layer()(previous_layer)
flatten_layer = tf.keras.layers.Flatten()(input_layer)
first_dense = tf.keras.layers.Dense(128, activation=tf.nn.relu)(flatten_layer)
output_layer = tf.keras.layers.Dense(10, activation=tf.nn.softmax)(first_dense)

# declare inputs and outputs
func_model = Model(inputs=input_layer, outputs=output_layer)

this is just a convention . we can also write like this

1
2
flatten_layer = tf.keras.layers.Flatten()
flatten_layer(input_layer)

but writing in one line is just convenient

Branching Model

Using the functional API architecture we can do stuffs like branching and concatenating easily 

1
2
3
4
5
6
7
8
layer1 = Dense(32)

layer2_1 = Dense(32)(layer1)
layer2_2 = Dense(32)(layer1)
layer2_3 = Dense(32)(layer1)
layer2_4 = Dense(32)(layer1)

merge = Concatenate([layer2_1,layer2_2,layer2_3,layer2_4])

Also we can have multiple inputs and outputs

1
func_model = Model(inputs=[input1,input2],outputs=[output1,output2])

Inception model is built using this strategy . Details here

Multi-output Model

1
2
3
4
5
6
7
8
9
10
input_layer = Input(shape=(len(train.columns),))
first_dense = Dense(units = '128', activation = 'relu')(input_layer)
second_dense = Dense(units = '128', activation = 'relu')(first_dense)

y1_output = Dense(units = '1', name = 'y1_output')(second_dense)

third_dense = Dense(units = '64', activation = 'relu')(second_dense)
y2_output = Dense(units = '1', name = 'y2_output')(third_dense)

model = Model(inputs = input_layer , outputs = [y1_output,y2_output])
1
2
3
4
5
6
7
# regression problem : https://archive.ics.uci.edu/ml/datasets/Energy+efficiency
# Specify the optimizer, and compile the model with loss functions for both outputs
optimizer = tf.keras.optimizers.SGD(lr=0.001)
model.compile(optimizer=optimizer,
              loss={'y1_output': 'mse', 'y2_output': 'mse'},
              metrics={'y1_output': tf.keras.metrics.RootMeanSquaredError(),
                       'y2_output': tf.keras.metrics.RootMeanSquaredError()})

Multi Input Model (Siamese Network)

Example

correlation / difference / similarity between 2 image 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# base model : similar for both inputs

def initialize_base_network():
    input = Input(shape=(28,28,), name="base_input")
    x = Flatten(name="flatten_input")(input)
    x = Dense(128, activation='relu', name="first_base_dense")(x)
    x = Dropout(0.1, name="first_dropout")(x)
    x = Dense(128, activation='relu', name="second_base_dense")(x)
    x = Dropout(0.1, name="second_dropout")(x)
    x = Dense(128, activation='relu', name="third_base_dense")(x)

    return Model(inputs=input, outputs=x)

base_network = initialize_base_network()

base network

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# siamese network

# create the left input and point to the base network
input_a = Input(shape=(28,28,), name="left_input")
vect_output_a = base_network(input_a)

# create the right input and point to the base network
input_b = Input(shape=(28,28,), name="right_input")
vect_output_b = base_network(input_b)

# measure the similarity of the two vector outputs
# using Lamda we can use custom function
output = Lambda(euclidean_distance, name="output_layer", output_shape=eucl_dist_output_shape)([vect_output_a, vect_output_b])

# specify the inputs and output of the model
model = Model([input_a, input_b], output)

# plot model graph
plot_model(model, show_shapes=True, show_layer_names=True, to_file='outer-model.png')