We give a proof of the Universality Conjecture in Random Matrix Theory for orthogonal $(beta=1)$ and symplectic $(beta=4)$ ensembles in the scaling limit for a class of weights $w(x)=exp(-V(x))$ where V is a polynomial. For such weights the associated equilibrium measure is supported on a single interval. Our starting point is Widom's representation of the correlation kernels for the beta=1,4 cases in terms of the unitary $(beta=2)$ correlation kernel plus a correction term which involves orthogonal polynomials $(OP's)$ with respect to the weight w introduced above. We do not use skew orthogonal polynomials. In the asymptotic analysis of the correction terms we use amongst other things differential equations for the derivatives of OP's due to Tracy-Widom, and uniform Plancherel-Rotach type asymptotics for OP's due to Deift-Kriecherbauer-McLaughlin-Venakides-Zhou. The problem reduces to a small norm problem for a certain matrix of a fixed size that is equal to the degree of the polynomial potential.