Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning