← Back to Research Index

Stochastic Markov Decision Processes and Expectiminimax Optimization in Classical Dice Games

By Tabletop Systems Lab • Published: October 2026 • Reading time: 7 min • Subject: Stochastic Dynamic Programming & Probability

1. Introduction: Imperfect Decision-Making Under Environmental Stochasticity

While combinatorial games like chess and draughts represent deterministic systems where state transitions occur with probability \( P = 1 \), stochastic board games introduce environmental randomness into the state transition kernel. In games such as Backgammon, ancient Roman Tabula, and the Royal Game of Ur, players operate under complete information regarding the physical board state, yet all forward projections must account for discrete probabilistic distributions generated by randomized dice outcomes.

This structural characteristic transforms the underlying game-theoretic framework from an alternating minimax tree into a Markov Decision Process (MDP), or more specifically, a two-player zero-sum stochastic game. Solving or optimizing play within these systems requires mathematical formulations that balance tactical foresight with probabilistic expectation, absorbing Markov chain analysis, and financial option pricing models for the doubling cube.

2. Discrete Probability Space of Polyhedral Dice

In standard backgammon, transitions are driven by the rolling of two fair six-sided dice (\( d_1, d_2 \in \{1, 2, 3, 4, 5, 6\} \)). The sample space \( \Omega \) contains \( 6 \times 6 = 36 \) equiprobable permutations, each occurring with probability \( P(\omega) = \frac{1}{36} \).

Because the physical rules of backgammon treat non-double rolls as unordered pairs \( \{x, y\} \) where \( x \ne y \), and grant double rolls \( \{x, x\} \) four moves of value \( x \) instead of two, the 36 microscopic outcomes collapse into 21 unique macroscopic events:

The expected movement yield of a random turn \( E[M] \) can be calculated rigorously:

\( E[M] = \sum_{x < y} \frac{2}{36}(x + y) + \sum_{x=1}^{6} \frac{1}{36}(4x) = \frac{1}{36} \left[ 2 \times 210 + 4 \times 21 \right] = \frac{420 + 84}{36} = \frac{504}{36} = 14.0 \text{ pips} \)

Thus, each player advances an average of exactly 14 pips per completed turn, establishing the baseline velocity against which all positional races are measured.

3. The Expectiminimax Search Architecture

To compute optimal moves in a stochastic environment, minimax decision trees must be augmented with intermediate chance nodes. The resulting algorithm, known as expectiminimax, alternates between three distinct node categories:

  1. Max Nodes: Player 1 selects the action \( a \in A(s) \) that maximizes expected utility.
  2. Min Nodes: Player 2 selects the action \( a' \in A(s') \) that minimizes Player 1's utility.
  3. Chance Nodes: Nature rolls the dice according to the discrete probability distribution \( P(d) \), computing the weighted expected value of child states:

    \( V(s_{\text{chance}}) = \sum_{d=1}^{21} P(d) \cdot V(s_d) \)

A profound algorithmic challenge arises when applying alpha-beta pruning to expectiminimax trees. In deterministic trees, a branch can be pruned the moment a single refutation is found. At a chance node, however, a very bad outcome on one branch might be compensated by exceptional outcomes on other branches. Pruning can only occur when the mathematical upper bound of the unexplored probability mass cannot mathematically overcome the current alpha threshold, an optimization known as Star-Alpha-Beta pruning.

4. Absorbing Markov Chains and Effective Pip Count (EPC)

When both players have successfully disengaged their checkers and entered the non-contact pure race phase, the game can be modeled exactly as two independent, competing absorbing Markov chains. Each checker configuration \( C \) represents a discrete state, and the terminal absorbing state corresponds to bearing off all 15 checkers.

While the raw pip count (the sum of distances of all checkers to the home tray) provides a rough initial heuristic, it ignores pip wastage—the inefficiency of using high dice rolls to bear off checkers from low home-board points. To resolve this, mathematicians and computer theorists developed the Effective Pip Count (EPC):

\( \text{EPC} = 14.0 \times E[T] - 7.0 \)

where \( E[T] \) is the expected number of turns required to bear off all checkers, calculated using backward induction through the transition matrix. For an ideal, unblemished checker distribution, \( \text{EPC} \approx \text{Raw Pips} + 7.0 \). Poorly distributed boards with severe wastage (such as excessive checkers clustered on the 1-point) incur wastage penalties of 15 to 25 additional effective pips, drastically shifting winning equity in tournament racing positions.

5. Mathematical Modeling of the Doubling Cube: Janowski's Formulas

The introduction of the doubling cube into backgammon in the 1920s transformed the game from a linear race into a dynamic exercise in risk management and financial option valuation. Possessing the doubling cube grants a player an American put/call option: the right, but not the obligation, to double the stakes of the match prior to rolling.

In a money game (infinite session) without gammon risks, the basic take-point threshold can be derived directly from zero-sum expectation. If Player A offers a double to Player B:

Player B should mathematically accept whenever \( U(\text{take}) \ge U(\text{drop}) \):

\( 4p - 2 \ge -1 \implies 4p \ge 1 \implies p \ge 0.25 \)

Thus, 25% winning probability represents the fundamental risk-neutral take point. In match play with finite score targets, the calculation expands into Match Equity Tables (MET). Rick Janowski developed empirical formulas to calculate the exact double and take thresholds factoring in market volatility and cube ownership leverage:

\( \text{Take Point} = \frac{\text{Risk}}{\text{Risk} + \text{Reward}} = \frac{E(2, -1) - E(1, -1)}{E(2, 2) - E(1, -1)} \)

6. Conclusion: The Synthesis of Randomness and Optimal Strategy

The mathematical architecture of dice-based board games presents a sophisticated convergence of discrete probability, absorbing Markov chains, expectiminimax search, and economic game theory. Far from reducing the role of analytical skill, environmental stochasticity exponentially expands the strategic search space, requiring players and automated engines to master long-term expected utility and variance management. As AI engines continue to evolve, the stochastic principles proven in classical dice games remain central to real-world decision-making under uncertainty.