This paper extends the class of deterministic volatility Heath-Jarrow-Morton models to a Markov chain stochastic volatility framework allowing for jump discontinuities and a variety of deformations of the term structure of forward rate volatilities. Analytical solutions for the dynamics of the volatility term structure are obtained. Semimartingale decompositions of the interest rates under a spot and forward martingale measures are identified. Stochastic volatility versions of the continuous time Ho-Lee and Hull-White extended Vasicek models are obtained. Introducing a regime shift in volatility that is an exponential function of time to maturity leads to a Vasicek dynamics with regime switching coefficients of the short rate.