This paper develops a new computationally attractive procedure for estimating dynamic discrete choice models that is applicable to a wide range of dynamic programming models. The proposed procedure can accommodate unobserved state variables that are neither additively separable nor follow generalized extreme value distribution, and its implementation is straightforward in terms of computer programming. Our estimation algorithm sequentially updates the parameter estimate and the value function estimate. It builds upon the idea of the iterative estimation algorithm proposed by Aguirregabiria and Mira (2002, 2007) but conducts iteration using the value function mapping rather than the policy iteration mapping. We analyze the convergence property of the proposed algorithm and derive the conditions for its convergence. We further extend our sequential procedure to estimation of dynamic programming models with an equilibrium constraint, which include models of dynamic game and dynamic macroeconomic models.