Isn’t this last layer analysis basically saying that after a bunch of Chinese tokens you get a Chinese token, and after some Latin tokens you get a Latin token? That seems expected given the setup. Would be interested in seeing what translations look like in the middle layers.