For nonparametric regression, where the regression function has discontinuity points, the kernel regression estimator and cross-validation are known to be affected by discontinuity.

Viet Ngo, A Structure Theorem for Discontinuous Derivations of Banach Algebras of Differential Functions, Proceedings of the American Mathematical Society, Vol. 102, No. 3 (Mar., 1988), pp. 507-513 ...