Optimal Control of Stochastic Dynamic Systems with Semi-Markov Parameters

optimal control; semi-Markov parameter; stochastic system; Computer Science (miscellaneous); Chemistry (miscellaneous); Mathematics (all); Physics and Astronomy (miscellaneous)

Abstract :

[en] This paper extends classical Markov switching models. We introduce a generalized semi-Markov switching framework in which the system dynamics are governed by an Itô stochastic differential equation. Of note, the optimal control synthesis problem is formulated for stochastic dynamic systems with semi-Markov parameters. Further, a system of ordinary differential equations is derived to characterize the Bellman functional and the corresponding optimal control. We investigate the case of linear dynamics in detail, and propose a closed-form solution for the optimal control law. A numerical example is presented to illustrate the theoretical results.

Disciplines :

Mathematics

Author, co-author :

LUKASHIV, Taras ^✱; University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > Clinical and Translational Informatics ; Department of Mathematical Problems of Control and Cybernetics, Yuriy Fedkovych Chernivtsi National University, Chernivtsi, Ukraine

Malyk, Igor V. ; Department of Mathematical Problems of Control and Cybernetics, Yuriy Fedkovych Chernivtsi National University, Chernivtsi, Ukraine

HEMEDAN, Ahmed ; University of Luxembourg > Luxembourg Centre for Systems Biomedicine > Bioinformatics Core > Team Reinhard SCHNEIDER

SATAGOPAM, Venkata ^✱; University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > Clinical and Translational Informatics

^✱ These authors have contributed equally to this work.

External co-authors :

yes

Language :

English

Title :

Optimal Control of Stochastic Dynamic Systems with Semi-Markov Parameters

Publication date :

April 2025

Journal title :

Symmetry

eISSN :

2073-8994

Publisher :

Multidisciplinary Digital Publishing Institute (MDPI)

Volume :

Issue :

Pages :

498

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://www.mdpi.com/2073-8994/17/4/498/pdf

Funders :

ELIXIR-LU

Funding text :

This research was supported by ELIXIR-LU https://elixir-luxembourg.org/ (accessed on 24 March 2025), the Luxembourgish node of ELIXIR, with funding and infrastructure provided by the Luxembourg Centre for Systems Biomedicine (LCSB). LCSB\u2019s support contributed to the computational analyses and methodological development presented in this study.

Available on ORBilu :

since 19 February 2026

Statistics

Number of views

45 (0 by Unilu)

Number of downloads

11 (0 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

WoS citations^™

Bibliography

Lindquist A. Optimal control of linear stochastic systems with applications to time lag systems Inf. Sci. 1973 5 81 124
Kumar P.R. van Schuppen J.H. On the optimal control of stochastic systems with an exponential-of-integral performance index J. Math. Anal. Appl. 1981 80 312 332
Buckdahn R. Labed B. Rainer C. Tamer L. Existence of an optimal control for stochastic control systems with nonlinear cost functional Stochastics 2010 82 241 256
Dragan V. Popa I.-L. The Linear Quadratic Optimal Control Problem for Stochastic Systems Controlled by Impulses Symmetry 2024 16 1170 10.3390/sym16091170
Das A. Lukashiv T.O. Malyk I.V. Optimal control synthesis for stochastic dynamical systems of random structure with the markovian switchings J. Autom. Inf. Sci. 2017 4 37 47 10.1615/JAutomatInfScien.v49.i4.40
Antonyuk S.V. Byrka M.F. Gorbatenko M.Y. Lukashiv T.O. Malyk I.V. Optimal Control of Stochastic Dynamic Systems of a Random Structure with Poisson Switches and Markov Switching J. Math. 2020 2020 9457152 10.1155/2020/9457152
Dynkin E.B. Markov Processes Academic Press New York, NY, USA 1965
Koroliuk V. Limnios N. Stochastic Systems in Merging Phase Space World Scientific Publishing Co., Pte Ltd. Hackensack, NJ, USA 2005
Ibe O. Markov Processes for Stochastic Modelling 2nd ed. Elsevier London, UK 2013
Gikhman I.I. Skorokhod A.V. Introduction to the Theory of Random Processes W. B. Saunders Philadelphia, PA, USA 1969
Feldman R. Optimal Replacement With Semi-Markov Shock Model J. Appl. Probab. 1976 13 108 117
Love C. Zhang G. Zitron M. Renkuan G. Discrete semi-Markov decision model to determine the optimal repair/replacement policy under general repairs Eur. J. Oper. Res. 2000 125 398 409 10.1016/S0377-2217(99)00009-0
Cheng J. Xie L. Park J. Yan H. Protocol-Based Output-Feedback Control for Semi-Markov Jump Systems IEEE Trans. Autom. Control 2022 67 4346 4353 10.1109/TAC.2022.3175723
Xu Z. Su H. Shi P. Wu Z.-G. Asynchronous H∞ control of semi-Markov jump linear systems Appl. Math. Comput. 2019 349 270 280
Jallow M.A. Nafiu L. Weke P. Stochastic Forecasting of Stock Prices of Capital Assets Using Semi-Markov Model Int. J. Stat. Econ. 2022 23 50 61
Øksendal B. Stochastic Differential Equation Springer New York, NY, USA 2013
Kolmanovskii V.B. Shaikhet L.E. Control of Systems with Aftereffect American Mathematical Society Providence, RI, USA 1996
Jacod J. Shiryaev A.N. Limit Theorems for Stochastic Processes Fizmatlit Moscow, Russia 1994 Volumes 1 and 2 (In Russian)
Lukashiv T. One Form of Lyapunov Operator for Stochastic Dynamic System with Markov Parameters J. Math. 2016 2016 1694935
Stewart I.A.N. Broken symmetries and biological patterns On Growth, Form and Computers Kumar S. Bentley P.J. Elsevier Amsterdam, The Netherlands 2003 181 202
Bellman R. Dynamic Programming Princeton University Press Princeton, NJ, USA 1972
Narayan P. Popp S. A New Unit Root Test with Two Structural Breaks in Level and Slope at Unknown Time J. Appl. Stat. 2010 37 1425 1438 10.1080/02664760903039883