非非不在家
非非不在家

非非不在家

学霸进行时

60 Following
2 Followers
·

Highlighted by 非非不在家

See more

From Checklist for debugging neural networks by Cecelia Shao

If your gradient values are zero, it could mean that the learning rate might be too small in the optimizer or that you’re encountering Error #1 above with incorrect expressions for the gradient updates.

From Why you should care about byte-level sequence-to-sequence models in NLP by Tom Kenter

To read the sentence “Where is Amsterdam”, a word-level model would need 3 steps, one for each word. A byte- or character-level model would need 18. Average word length varies across languages, but it unavoidable for character-level models to need…

From Why you should care about byte-level sequence-to-sequence models in NLP by Tom Kenter

Just to be clear on one detail here, as this sometimes is a source of confusion, the bytes themselves are not the embeddings. Rather, they are used as indices to the entries of an embedding matrix. So, byte 0000 0010 correspo…