Small but time-aware!
At the " #LLM for for the History, Philosophy, and Sociology of Science workshop" @tuberlin Jochen Büttner presented a compact small language model (39M parameters) that integrates a temporal dimension for historical text analysis.
A proof of concept using historical UK weather reports shows the model successfully reflects both seasonal patterns and linguistic shifts
Wanna try it? github.com/j-buettner/time_transformer