Statistics of Taylor-Series Expansion Based Numerical Methods: A Primer, Performance Benchmarking and New Approaches for Problems with Non-smooth Solutions