Cloud Stack Ninja

I'm studying DL by myself and trying to perform a CNN classification model on the CIFAR10 dataset. I built a ResNet model and used it to classify the data. I got 0.96 accuracy but my val_accuracy started oscillating around 0.77 and val_loss around 1.15 from the 20th epoch (of 100).

(X_trn, y_trn), (X_test, y_test) = keras.datasets.cifar10.load_data()

# Data Preprocessing
# Create training and val sets
(X_train, y_train) = X_trn[:40000, ...], y_trn[:40000, ...]
(X_val, y_val) = X_trn[40000:50000, ...], y_trn[40000:50000, ...]

# Import the libraries
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import to_categorical

# one-hot encode
y_train = to_categorical(y_train)
y_val = to_categorical(y_val)
y_test = to_categorical(y_test)
# Data augmentation
train_datagen = ImageDataGenerator(shear_range=0.1, zoom_range=0.1, horizontal_flip=True)
train_datagen.fit(X_train)

# Create model architecture, and compile it
from ResNetModel import ResNet
model = ResNet.build(32, 32, 3, 10, [3, 4, 6], [32, 64, 128, 256])
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
    initial_learning_rate=1e-2,
    decay_steps=10000,
    decay_rate=0.9)
optimizer = keras.optimizers.SGD(learning_rate=lr_schedule)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
# Begin training
hist = model.fit(train_datagen.flow(X_train, y_train, batch_size=32),
          validation_data=(X_val, y_val), epochs = 100, shuffle=True)

Epoch 1/100
1250/1250 [==============================] - 48s 38ms/step - loss: 2.1462 - accuracy: 0.3027 - val_loss: 1.8949 - val_accuracy: 0.3959
Epoch 2/100
1250/1250 [==============================] - 47s 37ms/step - loss: 1.8299 - accuracy: 0.4201 - val_loss: 1.7263 - val_accuracy: 0.4601
Epoch 3/100
1250/1250 [==============================] - 46s 37ms/step - loss: 1.6957 - accuracy: 0.4770 - val_loss: 1.6404 - val_accuracy: 0.4969
Epoch 4/100
1250/1250 [==============================] - 46s 37ms/step - loss: 1.6044 - accuracy: 0.5134 - val_loss: 1.6401 - val_accuracy: 0.5086
Epoch 5/100
1250/1250 [==============================] - 46s 37ms/step - loss: 1.5251 - accuracy: 0.5427 - val_loss: 1.4406 - val_accuracy: 0.5708
.
.
.
Epoch 95/100
1250/1250 [==============================] - 46s 37ms/step - loss: 0.3495 - accuracy: 0.9604 - val_loss: 1.1415 - val_accuracy: 0.7780
Epoch 96/100
1250/1250 [==============================] - 46s 37ms/step - loss: 0.3463 - accuracy: 0.9628 - val_loss: 1.1519 - val_accuracy: 0.7763
Epoch 97/100
1250/1250 [==============================] - 46s 37ms/step - loss: 0.3464 - accuracy: 0.9624 - val_loss: 1.1413 - val_accuracy: 0.7786
Epoch 98/100
1250/1250 [==============================] - 48s 39ms/step - loss: 0.3422 - accuracy: 0.9625 - val_loss: 1.1566 - val_accuracy: 0.7769
Epoch 99/100
1250/1250 [==============================] - 48s 38ms/step - loss: 0.3384 - accuracy: 0.9645 - val_loss: 1.1552 - val_accuracy: 0.7762
Epoch 100/100
1250/1250 [==============================] - 47s 38ms/step - loss: 0.3454 - accuracy: 0.9622 - val_loss: 1.1548 - val_accuracy: 0.7731

val_accuracy stat

val_loss stat

I tried to set the optimizer to 'adam' without any decay but the model got worse (80% accuracy, 70% val_accuracy) What can I do? What is the cause?

Thank you?



Read more here: https://stackoverflow.com/questions/64406769/keras-cnn-resnet-model-cifar10-improving-val-accuracy-val-loss

Content Attribution

This content was originally published by Noy Rahmani at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: