p47068915
/
mindspore
forked from mindspore-Ecosystem/mindspore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

!9192 reset the default activation function for bert 

From: @shibeiji
Reviewed-by: @c_34,@liangchenghui
Signed-off-by: @c_34
This commit is contained in:
mindspore-ci-bot 2020-11-30 11:24:23 +08:00 committed by Gitee
parent 0694c2d31c bde36af04c
commit bebc37b389
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
model_zoo/official/nlp/bert/src/config.py
View File

@ -103,7 +103,7 @@ if cfg.bert_network == 'large':
num_hidden_layers=24, num_hidden_layers=24,
num_attention_heads=16, num_attention_heads=16,
intermediate_size=4096, intermediate_size=4096,
hidden_act="fast_gelu", hidden_act="gelu",
hidden_dropout_prob=0.1, hidden_dropout_prob=0.1,
attention_probs_dropout_prob=0.1, attention_probs_dropout_prob=0.1,
max_position_embeddings=512, max_position_embeddings=512,