딥러닝으로 영화 리뷰의 긍정/부정 이진 분류 모델 만들기

ML PV:

IMDB 데이터셋을 이용한 영화 리뷰 분류 : Binary Classification

머신러닝은 크게 분류와 회귀 문제로 나누어 지게 됩니다.
저희는 분류 문제에서도 이진분류를 진행해볼건데요.
이진분류라는건 예를들어 메일을 받았을 때 이 메일이 스팸이냐 아니냐, 말을 했을 때 욕이냐 아니냐와 같이 문제에 대해 예/아니오 형태로 구분되는 문제에 적합합니다.

이번에는 IMDB 데이터셋을 이용해 해당 리뷰가 긍정적인지 부정적인지 예측하는 이진 분류 모델을 만들어 보겠습니다.

Dataset

IMDB 데이터셋이란 영화에 대한 평점과 리뷰들이 들어가있는 데이터인데요, (영어로 되어 있습니다.)

추후에는 한글로 된 데이터셋을 직접 구하고 만들어서 분류기를 만들어보시면 재밌지 않을까요?

Download

데이터셋은 텐서플로의 패키지 안에 다운로더 및 로더가 내장되어 있습니다.
(포스팅 시각 기준으로 저는 r1.12 버전의 텐서플로 패키지를 사용중이며, 너무 오래된 버전의 경우 keras 패키지가 내장되어 있지 않을 수 있습니다.)

1
from tensorflow.keras.datasets import imdb
1
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

Dataset shape 확인

1
2
print("Train Datas Shape : {}".format(train_data.shape))
print("Train Labels Shape : {}".format(train_labels.shape))
Train Datas Shape : (25000,)
Train Labels Shape : (25000,)

데이터 확인

25000개의 훈련용 데이터셋이 존재하며, 각 인덱스는 단어 인덱스의 리스트를 가지고 있습니다.

1
2
display(train_data[0][:10])
display(train_labels)
[1, 14, 22, 16, 43, 530, 973, 1622, 1385, 65]



array([1, 0, 0, ..., 0, 1, 0])

단어 인덱스를 단어로 치환

1
2
3
4
word_index = imdb.get_word_index()
indexes = dict([(value, key) for (key, value) in word_index.items()])
decoded_review = ' '.join(indexes.get(i - 3, '?') for i in train_data[0])
print(decoded_review)
? this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there robert ? is an amazing actor and now the same being director ? father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it was released for ? and would recommend it to everyone to watch and the fly fishing was amazing really cried at the end it was so sad and you know what they say if you cry at a film it must have been good and this definitely was also ? to the two little boy's that played the ? of norman and paul they were just brilliant children are often left out of the ? list i think because the stars that play them all grown up are such a big profile for the whole film but these children are amazing and should be praised for what they have done don't you think the whole story was so lovely because it was true and was someone's life after all that was shared with us all

데이터 변환

단어의 개수를 10,000개로 지정해두었고, 이 단어 인덱스를 원핫인코딩으로 변환하여 10,000차원의 벡터로 변경시키도록 합니다. 

1
2
3
4
5
6
7
import numpy as np

def vectorize_sequences(sequences, dimension=10000):
    results = np.zeros((len(sequences), dimension)) # 크기가 들어온 리스트 (단어개수, 전체단어개수)이고, 모든 원소가 0인 행렬을 생성
    for i, sequence in enumerate(sequences):
        results[i, sequence] = 1.
    return results
1
2
3
4
5
x_train = vectorize_sequences(train_data)
x_test = vectorize_sequences(test_data)

display(x_train.shape)
display(x_test.shape)
(25000, 10000)



(25000, 10000)

1
2
3
4
5
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')

display(y_train)
display(y_test)
array([1., 0., 0., ..., 0., 1., 0.], dtype=float32)



array([0., 1., 1., ..., 0., 0., 0.], dtype=float32)

신경망 구성

신경망 네트워크 구축

1
2
from tensorflow.keras import models
from tensorflow.keras import layers
1
2
3
4
5
6
model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 16)                160016    
_________________________________________________________________
dense_1 (Dense)              (None, 16)                272       
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 17        
=================================================================
Total params: 160,305
Trainable params: 160,305
Non-trainable params: 0
_________________________________________________________________

모델 컴파일

모델을 사용하기 위해선 네트워크를 구성한 모델을 컴파일하는 과정이 필요합니다.
rmsprop 옵티마이저를 사용하고,
확률을 출력하는 모델을 사용할 때는 크로스엔트로피가 최선의 선택인데,
이진 분류로 각 확률을 구하는 모델이니 binary crossentropy를 사용합니다.

1
2
3
4
5
6
7
8
9
10
11
12
from tensorflow.keras import optimizers
from tensorflow.keras import metrics
from tensorflow.keras import losses

model.compile(
#     optimizer='rmsprop',
    optimizer=optimizers.RMSprop(lr=0.001),
#     loss='binary_crossentropy',
    loss=losses.binary_crossentropy,
#     metrics=['accuracy']
    metrics=[metrics.binary_accuracy]
)

검증 데이터 준비 (Validation)

훈련하는 동안 처음 본 데이터에 대한 모델의 정확도를 측정하기 위해
원본 훈련 데이터에서 10,000개의 샘플을 떼내어 검증 데이터 세트를 만들겠습니다.

1
2
3
4
x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]

모델 학습

512의 샘플씩 미니배치를 만들어 20번의 에폭동안 훈련시키고,
앞에서 떼어놓은 10,000개의 데이터를 이용해 손실과 정확도를 측정하겠습니다

1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=20,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/20
15000/15000 [==============================] - 2s 139us/step - loss: 0.5883 - binary_accuracy: 0.7147 - val_loss: 0.5165 - val_binary_accuracy: 0.7686
Epoch 2/20
15000/15000 [==============================] - 1s 90us/step - loss: 0.4284 - binary_accuracy: 0.8771 - val_loss: 0.3974 - val_binary_accuracy: 0.8760
Epoch 3/20
15000/15000 [==============================] - 1s 85us/step - loss: 0.3157 - binary_accuracy: 0.9189 - val_loss: 0.3314 - val_binary_accuracy: 0.8880
Epoch 4/20
15000/15000 [==============================] - 1s 86us/step - loss: 0.2407 - binary_accuracy: 0.9349 - val_loss: 0.2963 - val_binary_accuracy: 0.8893
Epoch 5/20
15000/15000 [==============================] - 1s 85us/step - loss: 0.1912 - binary_accuracy: 0.9473 - val_loss: 0.2806 - val_binary_accuracy: 0.8906
Epoch 6/20
15000/15000 [==============================] - 1s 85us/step - loss: 0.1587 - binary_accuracy: 0.9555 - val_loss: 0.2811 - val_binary_accuracy: 0.8880
Epoch 7/20
15000/15000 [==============================] - 1s 85us/step - loss: 0.1303 - binary_accuracy: 0.9654 - val_loss: 0.2927 - val_binary_accuracy: 0.8850
Epoch 8/20
15000/15000 [==============================] - 1s 87us/step - loss: 0.1108 - binary_accuracy: 0.9707 - val_loss: 0.2976 - val_binary_accuracy: 0.8859
Epoch 9/20
15000/15000 [==============================] - 1s 87us/step - loss: 0.0930 - binary_accuracy: 0.9762 - val_loss: 0.3311 - val_binary_accuracy: 0.8771
Epoch 10/20
15000/15000 [==============================] - 1s 87us/step - loss: 0.0795 - binary_accuracy: 0.9802 - val_loss: 0.3353 - val_binary_accuracy: 0.8808
Epoch 11/20
15000/15000 [==============================] - 1s 87us/step - loss: 0.0652 - binary_accuracy: 0.9855 - val_loss: 0.3555 - val_binary_accuracy: 0.8786
Epoch 12/20
15000/15000 [==============================] - 1s 88us/step - loss: 0.0556 - binary_accuracy: 0.9881 - val_loss: 0.3749 - val_binary_accuracy: 0.8768
Epoch 13/20
15000/15000 [==============================] - 1s 91us/step - loss: 0.0442 - binary_accuracy: 0.9919 - val_loss: 0.4263 - val_binary_accuracy: 0.8694
Epoch 14/20
15000/15000 [==============================] - 1s 91us/step - loss: 0.0373 - binary_accuracy: 0.9934 - val_loss: 0.4213 - val_binary_accuracy: 0.8753
Epoch 15/20
15000/15000 [==============================] - 1s 90us/step - loss: 0.0305 - binary_accuracy: 0.9947 - val_loss: 0.4821 - val_binary_accuracy: 0.8657
Epoch 16/20
15000/15000 [==============================] - 1s 92us/step - loss: 0.0263 - binary_accuracy: 0.9955 - val_loss: 0.4871 - val_binary_accuracy: 0.8704
Epoch 17/20
15000/15000 [==============================] - 1s 91us/step - loss: 0.0178 - binary_accuracy: 0.9978 - val_loss: 0.5176 - val_binary_accuracy: 0.8693
Epoch 18/20
15000/15000 [==============================] - 1s 88us/step - loss: 0.0155 - binary_accuracy: 0.9982 - val_loss: 0.5890 - val_binary_accuracy: 0.8639
Epoch 19/20
15000/15000 [==============================] - 1s 91us/step - loss: 0.0136 - binary_accuracy: 0.9980 - val_loss: 0.5645 - val_binary_accuracy: 0.8679
Epoch 20/20
15000/15000 [==============================] - 1s 89us/step - loss: 0.0086 - binary_accuracy: 0.9990 - val_loss: 0.5974 - val_binary_accuracy: 0.8672

모델의 훈련 정보 그리기

위에서 fit의 반환으로 받은 history는 각각의 훈련 데이터세트와 검증 데이터세트에 대한 매 에폭마다의 손실율과 정확도를 가지고 있습니다.
해당 지표를 matplot를 이용해 시각화 하도록 해보겠습니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
import matplotlib.pyplot as plt

def show_graph(history):
    history_dict = history.history
    accuracy = history_dict['binary_accuracy']
    val_accuracy = history_dict['val_binary_accuracy']
    loss = history_dict['loss']
    val_loss = history_dict['val_loss']

    epochs = range(1, len(loss) + 1)
    
    plt.figure(figsize=(16, 1))
    
    plt.subplot(121)
    plt.subplots_adjust(top=2)
    plt.plot(epochs, accuracy, 'ro', label='Training accuracy')
    plt.plot(epochs, val_accuracy, 'r', label='Validation accuracy')
    plt.title('Trainging and validation accuracy and loss')
    plt.xlabel('Epochs')
    plt.ylabel('Accuracy and Loss')

    plt.legend(loc='upper center', bbox_to_anchor=(0.5, -0.1),
              fancybox=True, shadow=True, ncol=5)
#     plt.legend(bbox_to_anchor=(1, -0.1))

    plt.subplot(122)
    plt.plot(epochs, loss, 'bo', label='Training loss')
    plt.plot(epochs, val_loss, 'b', label='Validation loss')
    plt.title('Training and validation loss')
    plt.xlabel('Epochs')
    plt.ylabel('Loss')
    plt.legend(loc='upper center', bbox_to_anchor=(0.5, -0.1),
          fancybox=True, shadow=True, ncol=5)
#     plt.legend(bbox_to_anchor=(1, 0))

    plt.show()
1
show_graph(history)

png

훈련 데이터셋의 그래프를(점선) 먼저 확인해보면,
각 에폭이 돌때마다 정확도가 오르고, 손실은 줄어드는 형태로 제대로 학습이 된것 처럼 보이나, 검증 데이터셋(실선)을 보게 되면 그렇지 않습니다.
각 에폭마다도 정확도는 오르지 않고, 손실이 늘어나는걸 확인할 수 있는데, 이런 경우 훈련 데이터셋에 과대적합(overfitting) 되었다고 합니다.
과대적합이 된 경우 모델이 새로운 데이터셋을 만났을 때 제대로 분류를 하지 못하게 됩니다.

모델 재학습하기

아까와 동일한 형태의 모델을 구성하고,
학습과 관련된 하이퍼파라미터만 변경하여 과대적합을 피해보겠습니다.

1
2
3
4
5
6
model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_3 (Dense)              (None, 16)                160016    
_________________________________________________________________
dense_4 (Dense)              (None, 16)                272       
_________________________________________________________________
dense_5 (Dense)              (None, 1)                 17        
=================================================================
Total params: 160,305
Trainable params: 160,305
Non-trainable params: 0
_________________________________________________________________

1
2
3
4
5
model.compile(
    optimizer=optimizers.RMSprop(lr=0.001),
    loss=losses.binary_crossentropy,
    metrics=[metrics.binary_accuracy]
)
1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=8,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/8
15000/15000 [==============================] - 2s 111us/step - loss: 0.5347 - binary_accuracy: 0.7908 - val_loss: 0.4127 - val_binary_accuracy: 0.8663
Epoch 2/8
15000/15000 [==============================] - 1s 89us/step - loss: 0.3256 - binary_accuracy: 0.9005 - val_loss: 0.3316 - val_binary_accuracy: 0.8697
Epoch 3/8
15000/15000 [==============================] - 1s 88us/step - loss: 0.2393 - binary_accuracy: 0.9232 - val_loss: 0.2822 - val_binary_accuracy: 0.8906
Epoch 4/8
15000/15000 [==============================] - 1s 85us/step - loss: 0.1881 - binary_accuracy: 0.9410 - val_loss: 0.2801 - val_binary_accuracy: 0.8875
Epoch 5/8
15000/15000 [==============================] - 1s 85us/step - loss: 0.1525 - binary_accuracy: 0.9524 - val_loss: 0.2770 - val_binary_accuracy: 0.8885
Epoch 6/8
15000/15000 [==============================] - 1s 87us/step - loss: 0.1263 - binary_accuracy: 0.9611 - val_loss: 0.2856 - val_binary_accuracy: 0.8880
Epoch 7/8
15000/15000 [==============================] - 1s 87us/step - loss: 0.1035 - binary_accuracy: 0.9701 - val_loss: 0.3127 - val_binary_accuracy: 0.8846
Epoch 8/8
15000/15000 [==============================] - 1s 89us/step - loss: 0.0863 - binary_accuracy: 0.9754 - val_loss: 0.3270 - val_binary_accuracy: 0.8835

1
show_graph(history)

png

아까보단 모델이 상대적으로 과대적합 되지 않았습니다.
손실 그래프를 확인했을 때 2에폭과 3에폭이 훈련 세트와 검증 세트가 가장 근접한 손실을 갖고있는걸 확인할 수 있고,
정확도 또한 2,3에폭이 가장 근접한걸 확인할 수 있습니다.
즉, 이 모델의 경우 2에폭 혹은 3에폭을 돌렸을 때 과대적합을 가장 피할 수 있는 학습상태가 된다는걸 확인할 수 있습니다.
이렇게 학습에 파라미터를 조작하는 것 이외에도 과대적합을 피하는 기법이 많이 존재합니다.

모델의 평가

모델의 정확도를 측정합니다.

1
2
loss, accuracy = model.evaluate(x_test, y_test)
print('accuracy : {acc}, loss : {loss}'.format(acc=accuracy, loss=loss))
25000/25000 [==============================] - 2s 67us/step
accuracy : 0.86852, loss : 0.35010315059185027

모델의 예측

긍정이거나 부정일 확률 (높으면 긍정, 낮으면 부정)

1
model.predict(x_test[:10])
array([[0.2251153 ],
       [0.9999784 ],
       [0.98094064],
       [0.94734573],
       [0.97099954],
       [0.9737046 ],
       [0.9995834 ],
       [0.01185756],
       [0.9645392 ],
       [0.99970514]], dtype=float32)

번외. 레이어 변경하여 정확도 개선해보기

레이어를 한개 더 추가하여 테스트 (Deep)

1
2
3
4
5
6
7
8
9
10
11
12
13
model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

model.compile(
    optimizer=optimizers.RMSprop(lr=0.001),
    loss=losses.binary_crossentropy,
    metrics=[metrics.binary_accuracy]
)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_6 (Dense)              (None, 16)                160016    
_________________________________________________________________
dense_7 (Dense)              (None, 16)                272       
_________________________________________________________________
dense_8 (Dense)              (None, 16)                272       
_________________________________________________________________
dense_9 (Dense)              (None, 1)                 17        
=================================================================
Total params: 160,577
Trainable params: 160,577
Non-trainable params: 0
_________________________________________________________________

1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=8,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/8
15000/15000 [==============================] - 2s 113us/step - loss: 0.5264 - binary_accuracy: 0.7816 - val_loss: 0.4348 - val_binary_accuracy: 0.8122
Epoch 2/8
15000/15000 [==============================] - 1s 87us/step - loss: 0.2989 - binary_accuracy: 0.9006 - val_loss: 0.2936 - val_binary_accuracy: 0.8870
Epoch 3/8
15000/15000 [==============================] - 1s 85us/step - loss: 0.2103 - binary_accuracy: 0.9263 - val_loss: 0.2941 - val_binary_accuracy: 0.8812
Epoch 4/8
15000/15000 [==============================] - 1s 86us/step - loss: 0.1550 - binary_accuracy: 0.9481 - val_loss: 0.2963 - val_binary_accuracy: 0.8817
Epoch 5/8
15000/15000 [==============================] - 1s 89us/step - loss: 0.1294 - binary_accuracy: 0.9543 - val_loss: 0.2956 - val_binary_accuracy: 0.8850
Epoch 6/8
15000/15000 [==============================] - 1s 86us/step - loss: 0.0964 - binary_accuracy: 0.9709 - val_loss: 0.3357 - val_binary_accuracy: 0.8745
Epoch 7/8
15000/15000 [==============================] - 1s 86us/step - loss: 0.0814 - binary_accuracy: 0.9739 - val_loss: 0.3678 - val_binary_accuracy: 0.8714
Epoch 8/8
15000/15000 [==============================] - 1s 84us/step - loss: 0.0598 - binary_accuracy: 0.9834 - val_loss: 0.3910 - val_binary_accuracy: 0.8714

1
show_graph(history)

png

유닛을 추가하여 테스트 (Wide)

1
2
3
4
5
6
7
8
9
10
11
12
model = models.Sequential()
model.add(layers.Dense(64, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

model.compile(
    optimizer=optimizers.RMSprop(lr=0.001),
    loss=losses.binary_crossentropy,
    metrics=[metrics.binary_accuracy]
)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_10 (Dense)             (None, 64)                640064    
_________________________________________________________________
dense_11 (Dense)             (None, 64)                4160      
_________________________________________________________________
dense_12 (Dense)             (None, 1)                 65        
=================================================================
Total params: 644,289
Trainable params: 644,289
Non-trainable params: 0
_________________________________________________________________

1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=8,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/8
15000/15000 [==============================] - 2s 135us/step - loss: 0.4850 - binary_accuracy: 0.7656 - val_loss: 0.3620 - val_binary_accuracy: 0.8517
Epoch 2/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.2538 - binary_accuracy: 0.9058 - val_loss: 0.2754 - val_binary_accuracy: 0.8902
Epoch 3/8
15000/15000 [==============================] - 2s 103us/step - loss: 0.1857 - binary_accuracy: 0.9340 - val_loss: 0.2826 - val_binary_accuracy: 0.8874
Epoch 4/8
15000/15000 [==============================] - 2s 103us/step - loss: 0.1417 - binary_accuracy: 0.9499 - val_loss: 0.3328 - val_binary_accuracy: 0.8734
Epoch 5/8
15000/15000 [==============================] - 2s 104us/step - loss: 0.1128 - binary_accuracy: 0.9601 - val_loss: 0.3275 - val_binary_accuracy: 0.8826
Epoch 6/8
15000/15000 [==============================] - 2s 104us/step - loss: 0.0795 - binary_accuracy: 0.9743 - val_loss: 0.3473 - val_binary_accuracy: 0.8802
Epoch 7/8
15000/15000 [==============================] - 2s 103us/step - loss: 0.0544 - binary_accuracy: 0.9832 - val_loss: 0.3840 - val_binary_accuracy: 0.8780
Epoch 8/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.0514 - binary_accuracy: 0.9849 - val_loss: 0.4150 - val_binary_accuracy: 0.8789

1
show_graph(history)

png

깊고 넓게 구성하기 (Deep and wide network)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
model = models.Sequential()
model.add(layers.Dense(64, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

model.compile(
    optimizer=optimizers.RMSprop(lr=0.001),
    loss=losses.binary_crossentropy,
    metrics=[metrics.binary_accuracy]
)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_13 (Dense)             (None, 64)                640064    
_________________________________________________________________
dense_14 (Dense)             (None, 64)                4160      
_________________________________________________________________
dense_15 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_16 (Dense)             (None, 32)                1056      
_________________________________________________________________
dense_17 (Dense)             (None, 1)                 33        
=================================================================
Total params: 647,393
Trainable params: 647,393
Non-trainable params: 0
_________________________________________________________________

1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=8,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/8
15000/15000 [==============================] - 2s 142us/step - loss: 0.5094 - binary_accuracy: 0.7512 - val_loss: 0.3875 - val_binary_accuracy: 0.8442
Epoch 2/8
15000/15000 [==============================] - 2s 105us/step - loss: 0.2693 - binary_accuracy: 0.8983 - val_loss: 0.4223 - val_binary_accuracy: 0.8310
Epoch 3/8
15000/15000 [==============================] - 2s 105us/step - loss: 0.1949 - binary_accuracy: 0.9275 - val_loss: 0.5629 - val_binary_accuracy: 0.7950
Epoch 4/8
15000/15000 [==============================] - 2s 104us/step - loss: 0.1534 - binary_accuracy: 0.9434 - val_loss: 0.2965 - val_binary_accuracy: 0.8854
Epoch 5/8
15000/15000 [==============================] - 2s 105us/step - loss: 0.1106 - binary_accuracy: 0.9613 - val_loss: 0.3647 - val_binary_accuracy: 0.8718
Epoch 6/8
15000/15000 [==============================] - 2s 104us/step - loss: 0.0797 - binary_accuracy: 0.9733 - val_loss: 0.4042 - val_binary_accuracy: 0.8748
Epoch 7/8
15000/15000 [==============================] - 2s 110us/step - loss: 0.0802 - binary_accuracy: 0.9775 - val_loss: 0.4029 - val_binary_accuracy: 0.8815
Epoch 8/8
15000/15000 [==============================] - 2s 106us/step - loss: 0.0656 - binary_accuracy: 0.9827 - val_loss: 0.4207 - val_binary_accuracy: 0.8809

1
show_graph(history)

png

손실함수 변경

1
2
3
4
5
6
7
8
9
10
11
12
13
14
model = models.Sequential()
model.add(layers.Dense(64, activation='relu', input_shape=(10000, )))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

model.compile(
    optimizer=optimizers.RMSprop(lr=0.001),
    loss=losses.MSE,
    metrics=[metrics.binary_accuracy]
)
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_18 (Dense)             (None, 64)                640064    
_________________________________________________________________
dense_19 (Dense)             (None, 64)                4160      
_________________________________________________________________
dense_20 (Dense)             (None, 32)                2080      
_________________________________________________________________
dense_21 (Dense)             (None, 32)                1056      
_________________________________________________________________
dense_22 (Dense)             (None, 1)                 33        
=================================================================
Total params: 647,393
Trainable params: 647,393
Non-trainable params: 0
_________________________________________________________________

1
2
3
4
5
6
7
history = model.fit(
    partial_x_train,
    partial_y_train,
    epochs=8,
    batch_size=512,
    validation_data=(x_val, y_val),
)
Train on 15000 samples, validate on 10000 samples
Epoch 1/8
15000/15000 [==============================] - 2s 138us/step - loss: 0.1701 - binary_accuracy: 0.7462 - val_loss: 0.1152 - val_binary_accuracy: 0.8508
Epoch 2/8
15000/15000 [==============================] - 2s 103us/step - loss: 0.0794 - binary_accuracy: 0.8991 - val_loss: 0.0829 - val_binary_accuracy: 0.8917
Epoch 3/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.0580 - binary_accuracy: 0.9281 - val_loss: 0.0892 - val_binary_accuracy: 0.8833
Epoch 4/8
15000/15000 [==============================] - 2s 111us/step - loss: 0.0375 - binary_accuracy: 0.9544 - val_loss: 0.0858 - val_binary_accuracy: 0.8876
Epoch 5/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.0359 - binary_accuracy: 0.9560 - val_loss: 0.0874 - val_binary_accuracy: 0.8863
Epoch 6/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.0216 - binary_accuracy: 0.9751 - val_loss: 0.0927 - val_binary_accuracy: 0.8822
Epoch 7/8
15000/15000 [==============================] - 2s 109us/step - loss: 0.0189 - binary_accuracy: 0.9782 - val_loss: 0.0946 - val_binary_accuracy: 0.8812
Epoch 8/8
15000/15000 [==============================] - 2s 107us/step - loss: 0.0068 - binary_accuracy: 0.9931 - val_loss: 0.1091 - val_binary_accuracy: 0.8678

1
show_graph(history)

png

Powered by Hexo and Hexo-theme-hiker

Copyright © 2019 - 2019 Commit once a day All Rights Reserved.

UV : | PV :