Statistics for Stackelberg Multi-Agent Reinforcement Learning for Hierarchical Environments