关于添加的Bilstm_atttention代码的问题？

[garboa]

你好，运行代码出现这样的问题？请问如何修改？
Tensor("dense_1/Tanh:0", shape=(?, 100), dtype=float32)

---

Layer (type) Output Shape Param #

input_1 (InputLayer) (None, 1000) 0

---

embedding_1 (Embedding) (None, 1000, 100) 8056700

---

bidirectional_1 (Bidirection (None, 1000, 200) 160800

---

attention_layer_1 (Attention (None, 1000, 200) 40200

---

dense_1 (Dense) (None, 1000, 100) 20100

---

dense_2 (Dense) (None, 1000, 2) 202

Total params: 8,278,002
Trainable params: 221,302
Non-trainable params: 8,056,700

---

IMDB_Classifier_BiLSTM_Attention.py:124: UserWarning: The nb_epoch argument in fit has been renamed epochs.
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)
Traceback (most recent call last):
File "IMDB_Classifier_BiLSTM_Attention.py", line 124, in
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1522, in fit
batch_size=batch_size)
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 1382, in _standardize_user_data
exception_prefix='target')
File "/home/a/.local/lib/python2.7/site-packages/keras/engine/training.py", line 132, in _standardize_input_data
str(array.shape))
ValueError: Error when checking target: expected dense_2 to have 3 dimensions, but got array with shape (20000, 2)

[mpk001]

你这个应该是网络最后层的输出与你传进去的label数据不对应引起的。按理说attention层的输出就应该是2维的了，不知道你这里怎么还是3维的，你看一下你的网络结构有没有问题。

[garboa]

这份代码我什么都没有改跑的，这个包里的lstm的代码是跑通了的，这个文件相比起lstm的是加了attention层，是否单独写attention层的代码需要改一下？
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
l_gru = Bidirectional(LSTM(100, return_sequences=True))(embedded_sequences)
l_att = Attention_layer()(l_gru)
dense_1 = Dense(100,activation='tanh')(l_att)
print(dense_1)
dense_2 = Dense(2, activation='softmax')(dense_1)
dense_2 = tf.reshape(dense_2)

model = Model(sequence_input, dense_2)

model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['acc'])

model.summary()
model.fit(x_train, y_train, validation_data=(x_val, y_val), nb_epoch=10, batch_size=50)

[kuhung]

老哥你的问题解决了吗？ @mpk001 可否提供一下您的keras版本

[kuhung]

我解决了这个问题keras 1.2.2版本 @garboa @mpk001 在输出层前加一个flatten，代码如下

sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
l_gru = Bidirectional(LSTM(100, return_sequences=True))(embedded_sequences)
dense_1 = Dense(100,activation='tanh')(l_gru)
flatten = Flatten()(dense_1)
dense_2 = Dense(2, activation='softmax')(flatten)

model = Model(sequence_input, dense_2)

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['acc'])

model.summary()

[garboa]

请问大家在跑程序的时候，会不会出现头两个epoch时，val_acc>acc的情况？一般情况下测试集的准确率不会比训练集大吧？

[garboa]

@mpk001 @kuhung

[Eurus-Holmes]

@kuhung
我最开始出现的问题和 @garboa 一样，按你的代码运行以后原来错误没有了，有了新的错误。。。
ValueError: Input 0 is incompatible with layer flatten_1: expected min_ndim=3, found ndim=2

[vvvictorlee]

keras 2.1.5 tensorflow 1.7环境 在Attention_layer.py中增加下面代码可以解决ValueError: Error when checking target: expected dense_2 to have 3 dimensions,问题
def compute_output_shape(self, input_shape):
# output_dim = K.int_shape(self.u)
return (input_shape[0], input_shape[-1])

[vvvictorlee]

跟在评论里说一样，关键方法 返回shape在高版本，跟之前不一样了 keras 2.1.5 tensorflow 1.7环境 在Attention_layer.py中增加下面代码可以解决ValueError: Error when checking target: expected dense_2 to have 3 dimensions,问题 def compute_output_shape(self, input_shape): # output_dim = K.int_shape(self.u) return (input_shape[0], input_shape[-1]) Sent from Outlook<http://aka.ms/weboutlook>

…

________________________________ From: wjian <notifications@github.com> Sent: Friday, May 4, 2018 11:26 AM To: mpk001/Sentiment_analysis Cc: lisheng; Comment Subject: Re: [mpk001/Sentiment_analysis] 关于添加的Bilstm_atttention代码的问题？ (#2) @Eurus-Holmes<https://github.com/Eurus-Holmes> 你好，我的问题和你差不多，请问你解决了没 ― You are receiving this because you commented. Reply to this email directly, view it on GitHub<#2 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AjcYDMQKNiY1dcnF1IkIFGzIqrIYNCugks5tu8ptgaJpZM4PkTTv>.