This paper investigates the benefits of intentionally adding noise to a Bayesian estimator, which comprises a linear combination of a number of individual Bayesian estimators that are perturbed by mutually independent noise sources and multiplied by a set of adjustable weighting coefficients. We prove that the Bayes risk for the mean square error (MSE) criterion is minimized when the same optimum weighting coefficients are assigned to the identical estimators in the combiner. This property leads to a simplified analysis of the noise benefit to the MSE of the combined Bayesian estimator even when the number of individual estimators tends to infinity. It is shown that, for a sufficiently large number of individual estimators, the MSE of the designed Bayesian estimator approaches a plateau for a wide range of added noise levels. This robust feature facilitates the incorporation of the added noise into the design of Bayesian estimators without tuning the noise level. For an easily implementable Bayesian estimator composed of quantizers, the benefit of the symmetric scale-family noise is demonstrated, and the optimal noise probability density function is approximated by solving a constrained nonlinear optimization problem. We further extend this potential Bayesian estimator to the nonlinear filter design. Finally, examples of the noise benefits in random parameter estimation and nonlinear filtering support the theoretical analyses.