Theory of Probability and Mathematical Statistics
A variational characterization of the optimal exit rate for controlled diffusions
Ari Arapostathis, Vivek S. Borkar
Link
Abstract: The main result in this paper is a variational formula for the exit rate from a bounded domain for a diffusion process in terms of the stationary law of the diffusion constrained to remain in this domain forever. Related results on the geometric ergodicity of the controlled $Q$-process are also presented.
Keywords: Killed diffusions, exit rate, principal eigenvalue, $Q$-process, quasi-stationarity
Bibliography: 1. V. Anantharam and V. S. Borkar, A variational formula for risk-sensitive reward, SIAM J. Control Optim. 55 (2017), no. 2, 961-988. MR3629428.
2. A. Arapostathis and A. Biswas, Risk-sensitive control for a class of diffusions with jumps, ArXiv e-prints 1910.05004 (2019), available at https://arxiv.org/abs/1910.05004.
3. A. Arapostathis and A. Biswas, A variational formula for risk-sensitive control of diffusions in R^d, SIAM J. Control Optim. 58 (2020), no. 1, 85-103. MR4048004
4. A. Arapostathis, A. Biswas, V. S. Borkar, and K. S. Kumar, A variational characterization of the risk-sensitive average reward for controlled diffusions in R^d, ArXiv e-prints 1903.08346 (2019), available at https://arxiv.org/abs/1903.08346.
5. A. Arapostathis and V. S. Borkar, 'Controlled' versions of the Collatz-Wielandt and Donsker-Varadhan formulae. Applied Probability and Stochastic Processes, Springer Nature, Singapore, 2020, to appear, available at https://arxiv.org/abs/1903.10714.
6. A. Arapostathis, H. Hmedi, and G. Pang, On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin-Whitt regime, Math. Oper. Res. (2020), to appear, available at https://arxiv.org/abs/1812.03528.
7. A. Arapostathis and A. Biswas, Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions, Stochastic Process. Appl. 128 (2018), no. 5, 1485-1524. MR3780687.
8. A. Arapostathis, A. Biswas, and S. Saha, Strict monotonicity of principal eigenvalues of elliptic operators in R^d and risk-sensitive control, J. Math. Pures Appl. (9) 124 (2019), 169-219. MR3926044.
9. A. Arapostathis, V. S. Borkar, and M. K. Ghosh, Ergodic control of diffusion processes, Encyclopedia of Mathematics and its Applications, vol. 143, Cambridge University Press, Cambridge, 2012. MR2884272.
10. A. Arapostathis, V. S. Borkar, and K. S. Kumar, Risk-sensitive control and an abstract Collatz-Wielandt formula, J. Theoret. Probab. 29 (2016), no. 4, 1458-1484. MR3571250.
11. G. K. Befekadu and P. J. Antsaklis, On the asymptotic estimates for exit probabilities and minimum exit rates of diffusion processes pertaining to a chain of distributed control systems, SIAM J. Control Optim. 53 (2015), no. 4, 2297-2318. MR3376771.
12. V. E. Benes, Existence of optimal strategies based on specified information, for a class of stochastic decision problems, SIAM J. Control 8 (1970), 179-188. MR0265043.
13. H. Berestycki, L. Nirenberg, and S. R. S. Varadhan, The principal eigenvalue and maximum principle for second-order elliptic operators in general domains, Comm. Pure Appl. Math. 47 (1994), no. 1, 47-92. MR1258192.
14. A. Biswas and V. S. Borkar, On a controlled eigenvalue problem, Systems Control Lett. 59 (2010), no. 11, 734-735. MR2767905.
15. N. Champagnat and D. Villemonais, Exponential convergence to quasi-stationary distribution and Q-process, Probab. Theory Related Fields 164 (2016), no. 1-2, 243-283. MR3449390.
16. N. Champagnat and D. Villemonais, Uniform convergence to the Q-process, Electron. Commun. Probab. 22 (2017), Paper No. 33, 7. MR3663104
17. L. Collatz, Einschliessungssatz fur die charakteristischen Zahlen von Matrizen, Math. Z. 48 (1942), 221-226. MR8590.
18. P. Collet, S. Martinez, and J. San Martin, Quasi-stationary distributions: Markov chains, diffusions and dynamical systems, Probability and its Applications, Springer, Heidelberg, 2013. MR2986807.
19. A. Dembo and O. Zeitouni, Large deviations techniques and applications, Stochastic Modelling and Applied Probability, vol. 38, Springer-Verlag, Berlin, 2010. Corrected reprint of the second (1998) edition. MR2571413.
20. M. D. Donsker and S. R. S. Varadhan, On a variational formula for the principal eigenvalue for operators with maximum principle, Proc. Nat. Acad. Sci. U.S.A. 72 (1975), 780-783. MR361998.
21. D. Down, S. P. Meyn, and R. L. Tweedie, Exponential and uniform ergodicity of Markov processes, Ann. Probab. 23 (1995), no. 4, 1671-1691. MR1379163.
22. P. Echeverria, A criterion for invariant measures of Markov processes, Z. Wahrsch. Verw. Gebiete 61 (1982), no. 1, 1-16. MR671239.
23. S. N. Ethier and T. G. Kurtz, Markov processes. characterization and convergence, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1986. MR838085.
24. W. H. Fleming, Exit probabilities and optimal stochastic control, Appl. Math. Optim. 4 (1977/78), no. 4, 329-346. MR512217.
25. W. H. Fleming and M. R. James, Asymptotic series and exit time probabilities, Ann. Probab. 20 (1992), no. 3, 1369-1384. MR1175266.
26. W. H. Fleming and P. E. Souganidis, A PDE approach to asymptotic estimates for optimal exit probabilities, Stochastic differential systems (Marseille-Luminy, 1984), pp. 281-285. Lect. Notes Control Inf. Sci., 69, Springer, Berlin, 1985. MR798331.
27. W. H. Fleming and C. P. Tsai, Optimal exit probabilities and differential games, Appl. Math. Optim. 7 (1981), no. 3, 253-282. MR635802.
28. G. L. Gong, M. P. Qian, and Z. X. Zhao, Killed diffusions and their conditioning, Probab. Theory Related Fields 80 (1988), no. 1, 151-167. MR970476.
29. I. Gyongy and N. Krylov, Existence of strong solutions for It^o's stochastic equations via approximations, Probab. Theory Related Fields 105 (1996), no. 2, 143-158. MR1392450.
30. N. Ichihara, Large time asymptotic problems for optimal stochastic control with superlinear cost, Stochastic Process. Appl. 122 (2012), no. 4, 1248-1275. MR2914752.
31. R. Knobloch and L. Partzsch, Uniform conditional ergodicity and intrinsic ultracontractivity, Potential Anal. 33 (2010), no. 2, 107-136. MR2658978.
32. S. Meleard and D. Villemonais, Quasi-stationary distributions and population processes, Probab. Surv. 9 (2012), 340-410. MR2994898.
33. R. G. Pinsky, On the convergence of diffusion processes conditioned to remain in a bounded region for large time to limiting positive recurrent diffusion processes, Ann. Probab. 13 (1985), no. 2, 363-378. MR781410.
34. H. Wielandt, Unzerlegbare, nicht negative Matrizen, Math. Z. 52 (1950), 642-648. MR35265.
