We model each driver as an agent in a finite-horizon discrete Mean-Field Game. The key objects are:
$V(z,t)$ is the expected total future reward starting from zone $z$ at time $t$, assuming you act optimally and the crowd follows $\mu$.
Terminal condition โ no future left at the horizon:
Recursive Bellman equation โ the value at $t$ is the best Q-value achievable:
The expression inside the max defines the Q-value:
So compactly:
$V$ is computed by sweeping from $t = T$ back to $t = 0$. Each step uses the already-computed $V(\cdot,\,t+1)$ โ you need the future before you can value the present.
The optimal (greedy) policy is extracted at each step:
In practice a soft policy replaces the hard argmax for numerical stability โ a sigmoid over the Q-value gap:
The temperature $\tau$ controls selectivity: $\tau \to 0$ recovers the hard argmax; $\tau \to \infty$ gives a uniform 50/50 policy.
Given policy $\pi^*$, propagate $\mu$ forward from $t = 0$ to $t = T$:
In words: the fraction of drivers in zone $z'$ at $t{+}1$ equals the sum over all origin zones $z$ and all actions $a$ of (fraction of drivers there) ร (probability of choosing $a$) ร (probability of transitioning to $z'$).
This propagation yields a new implied distribution $\hat{\mu}$ โ what the crowd actually looks like when everyone follows $\pi^*$.
The backward pass assumed $\mu^{(k)}$; the forward pass produced $\hat{\mu}^{(k+1)}$. If they differ, we blend and repeat. A damped update prevents oscillation:
Iterate until the L1 change falls below tolerance $\varepsilon$:
At convergence $\mu^* \approx \hat{\mu}^*$ โ the distribution each driver assumes is exactly the distribution produced when everyone best-responds to it. This is the Mean-Field Nash Equilibrium.
Phase 1 uses $\alpha = 0.05$, $\varepsilon = 0.01$. Phase 2 uses $\alpha = 0.02$ (smaller, to handle spiky real demand) and $\varepsilon = 1.0$ (calibrated to the integer noise floor of real trip counts).
Backward โ "what should I do?" given the crowd ยท Forward โ "what does the crowd do?" given $\pi^*$ ยท Equilibrium โ both answers agree
| Symbol | Meaning |
|---|---|
| $z,\; z'$ | Current zone and next zone (state space $\mathcal{Z}$) |
| $t$ | Discrete time step, $t \in \{0,\ldots,T\}$ |
| $a$ | Action: accept or reject a ride offer |
| $\mu_t(z)$ | Mean field โ fraction of idle drivers in zone $z$ at time $t$ |
| $r(z,a,\mu_t)$ | Instantaneous reward; depends on zone, action, and crowd density |
| $P(z'\mid z,a)$ | Transition probability: land in $z'$ from $z$ after action $a$ |
| $V(z,t)$ | Value function โ optimal expected future reward from $(z,t)$ |
| $Q(z,a,t)$ | Q-value โ expected reward of taking action $a$ in $(z,t)$ |
| $\pi^*(z,t)$ | Optimal policy โ action distribution at $(z,t)$ |
| $\hat{\mu}$ | Implied mean field โ distribution produced by forward simulation |
| $\alpha$ | Mixing coefficient for damped update |
| $\varepsilon$ | L1 convergence tolerance |
| $\tau$ | Softmax temperature for the accept/reject policy |
| $\sigma(\cdot)$ | Sigmoid function $\sigma(x) = 1/(1+e^{-x})$ |