Statistics for Novel Methods for Natural Language Modeling and Pretraining