The effects of generational learning on language models were described in the Supplementary Materials of wikitext2. The study found that over the generations, models tend to produce more probable sequences from the original data and start introducing their own improbable sequences, that is, errors. A random 10% of the original data points have been trained…