l350.yaml « model « configs - zbp-scaling.git - Unnamed repository; edit this file 'description' to name the repository.

summary refs log tree commit diff

path: root/configs/model/l350.yaml

blob: cd36f078a63e7fb2bb5a98e3ba1d8284ff388208 (plain)

d: 1024
layers: 24
heads: 16
vocab: 50257
seq_len: 1024
ffn_mult: 8/3
pos: learned