m124.yaml « model « configs - zbp-scaling.git - Unnamed repository; edit this file 'description' to name the repository.

summary refs log tree commit diff

path: root/configs/model/m124.yaml

blob: ad3b5fab97bfd226a1708c8a4d4cd31029b68546 (plain)

d: 768
layers: 12
heads: 12
vocab: 50257
seq_len: 1024
ffn_mult: 8/3
pos: learned