An optimization problem is ill-posed if its solution is not unique or is acutely sensitive to data perturbations. A common approach to such problems is to construct a related problem with a well-behaved solution that deviates only slightly from the original solution set. The strategy is often used in data fitting applications, and also within optimization algorithms as a means for stabilizing the solution process. This approach is known as regularization, and deviations from solutions of the original problem are generally accepted as a trade-off for obtaining solutions with other desirable properties. In fact, however, there exist necessary and sufficient conditions such that solutions of the regularized problem continue to be exact solutions of the original problem. We present these conditions for general convex programs, and give some applications of exact regularization. (Joint work with Paul Tseng.)