|
输入:正面,d_model,面具,q, k, v |
|
输出:输出,经办人 |
| (1) |
进口tensorflow tf, keras.layers。密集,keras.callbacks.K |
| (2) |
dk = dv = d_model / /头 |
| (3) |
qs_layer =密集(正面dk), ks_layer =密集(正面dk), vs_layer =密集(正面dv) |
| (4) |
q = qs_layer (q), ks = ks_layer (k), vs = vs_layer (v) |
| (5) |
qs⟵重塑(qs), ks←重塑(ks)和←重塑(vs) |
| (6) |
脾气= tf.sqrt (tf.shape (ks) [−1]) |
| (7) |
为我= 1做 |
| (8) |
为j = 1头做 |
| (9) |
a_ij = K。batch_dot (qs:,:,:, ks: j:,:,轴= [2]) |
| (10) |
如果面具不是没有然后: |
| (11) |
mmask = (−1 e + 9)(1−面具) |
| (12) |
a_ij⟵K.Add ([a_ij mmask]) |
| (13) |
如果 |
| (14) |
a_ij←K。扩大退去(a_ij,轴= 1) |
| (15) |
attn.append (a_ij) |
| (16) |
结束了 |
| (17) |
结束了 |
| (18) |
j = 0 |
| (19) |
而真正的做 |
| (20) |
如果j != 0和j % = = 0然后 |
| (21) |
打破 |
| (22) |
如果 |
| (23) |
为我= 1做 |
| (24) |
=注意我头+j] |
| (25) |
temp_a.append (a) |
| (26) |
结束了 |
| (27) |
sm = K.Activation (“softmax K.Add (temp_a)) |
| (28) |
输出= K.batch_dot (sm [0,::,:], vs [j::,:],轴= [1,2]) |
| (29) |
输出←K。expand_dims(输出轴= 0) |
| (30) |
sm←K。permute_dimensions (sm (1 0、2、3) |
| (31) |
outpus.append(输出) |
| (32) |
attns.append (sm) |
| (33) |
j←+ 1 |
| (34) |
结束时 |
| (35) |
输出= K。concatente(输出轴= 0) |
| (36) |
经办人= K。concatente(经办人,轴= 0) |
| (37) |
输出←重塑(输出),经办人←重塑(经办人) |
|
