New in softcover. Hardcover was published in 2023. This book presents recent findings and results concerning the solutions of especially finite state-space Markov decision problems and determining Nash equilibria for related stochastic games with average and total expected discounted reward payoffs. In addition, it focuses on a new class of stochastic games: stochastic positional games that extend and generalize the classic deterministic positional games.