Statistics for Towards Robust Control in Visual Generation and Manipulation