Statistics for Pretrained Transformers for Efficient and Robust Information Retrieval