In this paper, optimal control of an inverted pendulum on a cart system is studied. Since the nonlinear structure of the system is complex, and in the presence of random disturbances, optimization and control of the motion of the system become more challenging. For handling this system, a discrete-time stochastic optimal control problem for the system is described, where the external force is considered as the control input. By defining a loss function, namely, the mean squared errors to be minimized, the stochastic approximation (SA) approach is applied to estimate the state dynamics. In addition, the Hamiltonian function is defined, and the first-order necessary conditions are derived. The gradient of the cost function is determined so that the SA approach is employed to update the control sequences. For illustration, considering the values of the related parameters in the system, the discrete-time stochastic optimal control problem is solved iteratively by using the SA algorithm. The simulation results show that the state estimation and the optimal control law design are well performed with the SA algorithm, and the motion of the inverted pendulum cart is addressed satisfactorily. In conclusion, the efficiency of the SA approach for solving the inverted pendulum on a cart system is verified.
Citation: Xian Wen Sim, Sie Long Kek, Sy Yi Sim. State estimation and optimal control of an inverted pendulum on a cart system with stochastic approximation approach[J]. Applied Computing and Intelligence, 2023, 3(1): 79-92. doi: 10.3934/aci.2023005
Related Papers:
[1]
Saima Rashid, Sobia Sultana, Bushra Kanwal, Fahd Jarad, Aasma Khalid .
Fuzzy fractional estimates of Swift-Hohenberg model obtained using the Atangana-Baleanu fractional derivative operator. AIMS Mathematics, 2022, 7(9): 16067-16101.
doi: 10.3934/math.2022880
[2]
Sajjad Ali Khan, Kamal Shah, Poom Kumam, Aly Seadawy, Gul Zaman, Zahir Shah .
Study of mathematical model of Hepatitis B under Caputo-Fabrizo derivative. AIMS Mathematics, 2021, 6(1): 195-209.
doi: 10.3934/math.2021013
[3]
Maysaa Al-Qurashi, Saima Rashid, Fahd Jarad, Madeeha Tahir, Abdullah M. Alsharif .
New computations for the two-mode version of the fractional Zakharov-Kuznetsov model in plasma fluid by means of the Shehu decomposition method. AIMS Mathematics, 2022, 7(2): 2044-2060.
doi: 10.3934/math.2022117
[4]
Muammer Ayata, Ozan Özkan .
A new application of conformable Laplace decomposition method for fractional Newell-Whitehead-Segel equation. AIMS Mathematics, 2020, 5(6): 7402-7412.
doi: 10.3934/math.2020474
[5]
Azzh Saad Alshehry, Naila Amir, Naveed Iqbal, Rasool Shah, Kamsing Nonlaopon .
On the solution of nonlinear fractional-order shock wave equation via analytical method. AIMS Mathematics, 2022, 7(10): 19325-19343.
doi: 10.3934/math.20221061
[6]
Ihtisham Ul Haq, Nigar Ali, Hijaz Ahmad, Taher A. Nofal .
On the fractional-order mathematical model of COVID-19 with the effects of multiple non-pharmaceutical interventions. AIMS Mathematics, 2022, 7(9): 16017-16036.
doi: 10.3934/math.2022877
[7]
Amjad Ali, Iyad Suwan, Thabet Abdeljawad, Abdullah .
Numerical simulation of time partial fractional diffusion model by Laplace transform. AIMS Mathematics, 2022, 7(2): 2878-2890.
doi: 10.3934/math.2022159
[8]
Gulalai, Shabir Ahmad, Fathalla Ali Rihan, Aman Ullah, Qasem M. Al-Mdallal, Ali Akgül .
Nonlinear analysis of a nonlinear modified KdV equation under Atangana Baleanu Caputo derivative. AIMS Mathematics, 2022, 7(5): 7847-7865.
doi: 10.3934/math.2022439
[9]
Ihsan Ullah, Aman Ullah, Shabir Ahmad, Hijaz Ahmad, Taher A. Nofal .
A survey of KdV-CDG equations via nonsingular fractional operators. AIMS Mathematics, 2023, 8(8): 18964-18981.
doi: 10.3934/math.2023966
[10]
Yousef Jawarneh, Humaira Yasmin, Abdul Hamid Ganie, M. Mossa Al-Sawalha, Amjid Ali .
Unification of Adomian decomposition method and ZZ transformation for exploring the dynamics of fractional Kersten-Krasil'shchik coupled KdV-mKdV systems. AIMS Mathematics, 2024, 9(1): 371-390.
doi: 10.3934/math.2024021
Abstract
In this paper, optimal control of an inverted pendulum on a cart system is studied. Since the nonlinear structure of the system is complex, and in the presence of random disturbances, optimization and control of the motion of the system become more challenging. For handling this system, a discrete-time stochastic optimal control problem for the system is described, where the external force is considered as the control input. By defining a loss function, namely, the mean squared errors to be minimized, the stochastic approximation (SA) approach is applied to estimate the state dynamics. In addition, the Hamiltonian function is defined, and the first-order necessary conditions are derived. The gradient of the cost function is determined so that the SA approach is employed to update the control sequences. For illustration, considering the values of the related parameters in the system, the discrete-time stochastic optimal control problem is solved iteratively by using the SA algorithm. The simulation results show that the state estimation and the optimal control law design are well performed with the SA algorithm, and the motion of the inverted pendulum cart is addressed satisfactorily. In conclusion, the efficiency of the SA approach for solving the inverted pendulum on a cart system is verified.
1.
Introduction
In the last few decades, the researchers have shown significant interest in fractional calculus in different areas, such as edge detection, electromagnetic, engineering, viscoelasticity, electrochemistry, cosmology, turbulence, diffusion, signal processing material science, physics and acoustics. It has been proved by many researchers that the integer-order models are generalized by fractional-order portray the complex phenomena in a very effective approach [1,2,3]. Various dynamical systems in physics and engineering are also modeled by using fractional-order differential equations. Many researchers have contributed a lot to provide an outstanding history of the derivative of fractional-order, such as Caputo [4], Yin et al.[5], Rashid et al., Arife et al. [6] and Oldham and Spanier [7,8].
The majority of science problems are nonlinear and it isn't easy to find its analytical solutions. The physical problems are mostly designed by using higher nonlinear partial differential equations (PDEs). It is challenging to find the exact or analytical solutions for such problems. However, in the last several centuries, many scientists have made significant progress and adopted various techniques to solve nonlinear PDEs. Recently in [9,10,11,12], Arqub et al. have implemented the Hilbert space reproduction kernel technique along with the iterative computational method to extended the semi-analytical approaches to other time fractional-order PDEs. In [13], the author examined approximate methods for certain groups of Robin time-fractional PDEs by fitting the kernel reproduction algorithm [14]. Sadly, many nonlinear problems can not be solved by analytical techniques. Moreover, the traditional numerical methods have to provide linearization, transformation, discretization and perturbation to solve the nonlinear models [15,16,17,18,19,20].
In 1980, Adomian developed an effective technique is known as Adomian Decomposition Method (ADM) to solve differential equations describing physical phenomena [21]. Further, ADM is modified by using Laplace transformation. As a result, a new method is developed, known as Laplace-ADM (LADM). LADM possesses the basic properties of both Laplace transformation and ADM. S.A Khuri [22] introduced LADM. This technique works effectively for PDEs of both initial and boundary value problems. LADM is also used in several research article to achieve an analytical solution, such as Whitham-Broer-Kaup equations [23], third-order dispersive fractional PDEs [24], fractional-order telegraph equations [25], fisher's equation [26], time-fractional model of Navier-Stokes equation [27] and Keller-Segel equation [28].
In the current article, the analytical solution of the following Swift-Hohenberg (SH) equation.
∂βu∂tβ=γu−(1+∂2∂x2)u−u3,γ∈R,0<β≤1,
(1.1)
where a and b are parameters. This is among the most fundamental equations suggested by Jack Swift and Pierre Hohenberg in 1977, which is related to the temperature and thermal convection of fluid dynamics [29]. A nonlinear parabolic equation is the SH equation, which also explains the formation process in liquid surfaces bounded along a horizontally well-conducting boundary. The broad field of SH equation is used in engineering and science problems from hydrodynamics of fluids to complex patterns [30,31]. Density gradient-driven liquid convection occurs in streams of geophysical fluid in the atmosphere, oceans and mantle of the earth. The mathematical model for the convection of Rayleigh-Benard includes the equations of Navier-Stokes combined with the temperature formula for transport. The S-H equation plays a vital role in the various branches of science, including fluid mechanics, such as laser studies [32] and Taylor-Couette flow [30] nonlinear optics.
The SH equation's solution was analyzed by some researchers using various methods, such as Vishal et al., that used the Homotopy Perturbation Transform Method (HPTM) [33]. The Homotopy Analysis Method (HAM) [34], Merdan, who used the method of variational iteration utilizing the Riemann-Liouville derivative [35], Khan et al., who used the differential transform method [36], Prakasha et al., applied is residual power series method and others [37,38,39,40].
In this research work, the analytical solution of SH equation is investigated by using Laplace Adomian decomposition method. In the prior literature, the solution of SH equation was handled by using Variational iteration method and Homotopy analysis method. It is observed that the proposed method is simple and straightforward as compared to other existing techniques. Moreover, the suggested method has greater accuracy and requires fewer calculations than other methods. The method is applied easily to achieve the solutions of fractional problems. The representation has confirmed the closed relation between the exact and obtained results. The fractional-order solutions are convergent to the solution of the integer-order problem.
2.
Definitions and preliminaries concepts
2.1. Definition
The Caputo definition of the fractional derivative of order β is described as [1,2,3,7]
The Laplace transform of G(s) of g(t) is defined by [22]
G(s)=L[g(t)]=∫∞0e−stg(t)dt.
2.3. Definition
The Laplace transform of fractional derivative is [22]
L(Dβtg(t))=sβG(s)−m−1∑k=0sβ−1−kg(k)(0),m−1<β<m.
3.
LADM idea for FPDEs
In this section, LADM is implemented to solve FPDEs [22].
For this first consider the following FPDE, that is
∂βu∂tβ+Lu(x,t)+Nu(x,t)=q(x,t),x,t≥0,0<β≤1,
(3.1)
with initial condition
u(x,0)=k(x).
(3.2)
The fractional derivative in equation (3.1) is expressed in Caputo sense. The linear and nonlinear terms are denoted by L and N respectively. The q is the sources term.
Taking both sides Laplace transformation of Eq (3.1),
In Table 1, the VIM and LADM solutions of example 9.1 are compared at different fractional orders β=0.5 and 1. The corresponding results are seems to be very similar and confirmed the validity of the present technique.
Table 1.
Comparison of LADM and VIM [35] of different fractional-order of β, l=3 and t=1.
In Figure 1, the LADM solutions at fractional orders β=0.75 and 1 are plotted. The graph (a) represent the solution of example 1 at β=1 while graph (b) is the plot of LADM solution at β=0.75. In Figure 2, the LADM solutions at fractional orders β=0.75 and 1 are plotted at t=1,2,3 and 4 in the domain −10<x<10. The graphs (c) and (d) of Figure 2 represent the LADM solutions at fractional orders β=1 and 0.75 respectively.
Figure 1.
LADM solution for example 1 of a=1, l=10 at (a) β=1 and (b) β=0.75.
In Table 2, the solutions comparison of LADM and HPM is given at fractional orders β=0.5 and 1. It has been observed that LADM has the higher degree of accuracy as compare to HPM.
Table 2.
Comparison of LADM and HPM of different fractional-order of β, l=10 and t=1.
In Figure 3, the LADM solutions at fractional orders β=0.75 and 1 are plotted. The graph (a) represent the solution of example 2 at β=1 while graph (b) is the plot of LADM solution at β=0.75. In Figure 4, the LADM solutions at fractional orders β=0.75 and 1 are plotted at t=1,2,3 and 4 in the domain −10<x<10. The graphs (c) and (d) of Figure 4 represent the LADM solutions at fractional orders β=1 and 0.75 respectively.
Figure 3.
LADM solution for example 2 of a=1, b=0.5 at (a) β=1 and (b) β=0.75.
In Figure 5, the LADM solutions of 3D graph at different fractional-orders β=1,0.8,0.6 and 0.4 are plotted. In Figure 6, the LADM solutions graph of 2D at different fractional-orders β=1,0.8,0.6 and 0.4 are plotted at t=1 in the domain −1<x<1.
Figure 5.
The LADM solution graph of different fractional-order of example 1.
In this research paper, the Laplace-Adomian decomposition method is found to be a simple and effective technique to solve the Swift-Hohenberg equations of fractional-order. The implementation of the proposed method has been done for some numerical examples of Swift-Hohenberg equations. The solution of these examples are in good contact with other analytical techniques such as the variational iteration method and the homotopy perturbation method. The present method has simple, accurate and straightforward implementation to solve fractional-order Swift-Hohenberg equations. It is also investigated that the proposed method has simple procedure and required a small volume of calculations as compared to other analytical methods. In conclusion, the suggested approach is considered to be a sophisticated tool for the solution of other fractional-order differential equations.
Conflict of interest
The authors declare no conflict of interest.
Authors contributions
Conceptualization, H.K. and R.S.; Methodology, H.K.; Software, R.S.; Validation, J.X. and D.B.; Formal Analysis, A.A.; Investigation, H.K. J.X. and D.B.; Resources, H.K. and S.A.; Writing-Original Draft Preparation, R.S.; Writing-Review and Editing, H.K. and D.B; Visualization, J.X.; Supervision, D.B.; Project Administration, A.A, Funding, J.X.
Acknowledgements
Natural Science Foundation of Neijiang Normal University (No.14ZB06).
The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University, Saudi Arabia for founding this work through Research Groups program under grant number (R.G.P2./99/41).
References
[1]
A. Ghanbari, M. Farrokhi, Decentralized neuro-fuzzy controller design using decoupled sliding-mode structure for two-dimensional inverted pendulum, 2006 IEEE International Conference on Engineering of Intelligent Systems, (2006), 1‒6. https://doi.org/10.1109/ICEIS.2006.1703155 doi: 10.1109/ICEIS.2006.1703155
[2]
P. Kumar, K. Chakraborty, R. R. Mukherjee, S. Mukherjee, Modelling and controller design of inverted pendulum, International Journal on Advanced Research in Computer Engineering & Technology (IJARCET), 2 (2013), 200‒206.
[3]
O. Boubaker, The inverted pendulum: a fundamental benchmark in control theory and robotics, Journal of Electronic Systems, 2 (2012), 154‒164. https://doi.org/10.1109/ICEELI.2012.6360606 doi: 10.1109/ICEELI.2012.6360606
[4]
S. M. Azimi, H. Miar-Naimi, Designing an analog CMOS fuzzy logic controller for the inverted pendulum with a novel triangular membership function, Scientia Iranica D, 26 (2019), 1736‒1748.
[5]
S. Hasnain, K. D. Kallu, M. H. Nawaz, N. Abbas, C. I. Pruncu, Dynamic response of an inverted pendulum system in water under parametric excitations for energy harvesting: a conceptual approach, Energies, 13 (2020), 5215. https://doi.org/10.3390/en13195215 doi: 10.3390/en13195215
[6]
S. J. Mohammadi Doulabi Fard, S. Jafari, Fuzzy controller structures investigation for future gas turbine aero-engines, International Journal of Turbomachinery Propulsion and Power, 6 (2021), 1‒23. https://doi.org/10.3390/ijtpp6010002 doi: 10.3390/ijtpp6010002
[7]
R. Eide, P. M. Egelid, A. Stamsø, H. R. Karimi, LQG control design for balancing an inverted pendulum mobile robot, Intelligent Control and Automation, 2 (2011), 160‒166. https://doi.org/10.4236/ica.2011.22019 doi: 10.4236/ica.2011.22019
[8]
A. N. K. Nasir, M. A. Ahmad, M. F. A. Rahmat, Performance comparison between LQR and PID controllers for an inverted pendulum system, AIP Conference Proceedings, American Institute of Physics, 1052 (2008), 124‒128. https://doi.org/10.1063/1.3008655
[9]
S. Dhobaley, P. Bhopale, A. Pandey, LMI based control for balancing an inverted pendulum mobile robot, International Journal of Engineering Research, 3 (2014), 2023‒2026.
[10]
J. C. Spall, Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control, Hoboken: John Wiley & Sons, 2003. https://doi.org/10.1002/0471722138
[11]
L. B. Prasad, B. Tyagi, H. O. Gupta, Optimal control of nonlinear inverted pendulum system using PID controller and LQR: performance analysis without and with disturbance input, Int. J. Autom. Comput., 11 (2014), 661‒670. https://doi.org/10.1007/s11633-014-0818-1 doi: 10.1007/s11633-014-0818-1
[12]
K. György, The LQG control algorithms for nonlinear dynamic systems, Procedia Manufacturing, 32 (2019), 553‒563. https://doi.org/10.1016/j.promfg.2019.02.252 doi: 10.1016/j.promfg.2019.02.252
[13]
A. E. Bryson, Y. C. Ho, Applied Optimal Control. Hemisphere, Washington, DC, USA, 1975.
[14]
E. K. P. Chong, S. H. Zak, An Introduction to Optimization. 4th Ed. John Wiley & Sons, Inc., Hoboken, New Jersey, 2013.
[15]
D. E. Kirk, Optimal Control Theory: An Introduction. Mineola, NY: Dover Publications, New York, 2004.
[16]
F. L. Lewis, V. Vrabie, V. L. Symos, Optimal Control. 3rd Ed. John Wiley & Sons, Inc., New York, 2012.
[17]
K. L. Teo, B. Li, C. Yu, V. Rehbock, Applied and Computational Optimal Control: A Control Parametrization Approach, Springer, 2021.
This article has been cited by:
1.
Naveed Iqbal, Ali Akgül, Rasool Shah, Abdul Bariq, M. Mossa Al-Sawalha, Akbar Ali, Hemen Dutta,
On Solutions of Fractional-Order Gas Dynamics Equation by Effective Techniques,
2022,
2022,
2314-8888,
1,
10.1155/2022/3341754
2.
Ismail Gad Ameen, Rasha Osman Ahmed Taie, Hegagi Mohamed Ali,
Two effective methods for solving nonlinear coupled time-fractional Schrödinger equations,
2023,
70,
11100168,
331,
10.1016/j.aej.2023.02.046
3.
Hegagi Mohamed Ali, Hijaz Ahmad, Sameh Askar, Ismail Gad Ameen,
Efficient Approaches for Solving Systems of Nonlinear Time-Fractional Partial Differential Equations,
2022,
6,
2504-3110,
32,
10.3390/fractalfract6010032
4.
Hassan Khan, Poom Kumam, Qasim Khan, Fairouz Tchier, Kanokwan Sitthithakerngkiet, Ioannis Dassios,
A New Modified Analytical Approach for the Solution of Time-Fractional Convection–Diffusion Equations With Variable Coefficients,
2022,
10,
2296-424X,
10.3389/fphy.2022.900502
5.
Meshari Alesemi, Naveed Iqbal, Ahmed A. Hamoud, C. Rajivganthi,
The Analysis of Fractional-Order Proportional Delay Physical Models via a Novel Transform,
2022,
2022,
1099-0526,
1,
10.1155/2022/2431533
6.
Humaira Yasmin,
Numerical Analysis of Time-Fractional Whitham-Broer-Kaup Equations with Exponential-Decay Kernel,
2022,
6,
2504-3110,
142,
10.3390/fractalfract6030142
7.
Khursheed Jamal Ansari, Fatima Ilyas, Kamal Shah, Aziz Khan, Thabet Abdeljawad,
On new updated concept for delay differential equations with piecewise Caputo fractional-order derivative,
2023,
1745-5030,
1,
10.1080/17455030.2023.2187241
8.
NEHAD ALI SHAH, PRAVEEN AGARWAL, JAE DONG CHUNG, SAAD ALTHOBAITI, SAMY SAYED, A. F. ALJOHANI, MOHAMED ALKAFAFY,
ANALYSIS OF TIME-FRACTIONAL BURGERS AND DIFFUSION EQUATIONS BY USING MODIFIEDq-HATM,
2022,
30,
0218-348X,
10.1142/S0218348X22400126
9.
Noufe H. Aljahdaly, Ravi P. Agarwal, Thongchai Botmart,
The analysis of the fractional-order system of third-order KdV equation within different operators,
2022,
61,
11100168,
11825,
10.1016/j.aej.2022.05.032
10.
Divya Raghavan, J. F. Gómez-Aguilar, N. Sukavanam,
Analytical approach of Hilfer fractional order differential equations using iterative Laplace transform method,
2023,
61,
0259-9791,
219,
10.1007/s10910-022-01419-7
11.
Mohammed Kbiri Alaoui, Rabia Fayyaz, Adnan Khan, Rasool Shah, Mohammed S. Abdo, Fathalla A. Rihan,
Analytical Investigation of Noyes–Field Model for Time-Fractional Belousov–Zhabotinsky Reaction,
2021,
2021,
1099-0526,
1,
10.1155/2021/3248376
12.
Hassan Khan, Adnan Khan, Rasool Shah, Dumitru Baleanu,
A novel fractional case study of nonlinear dynamics via analytical approach,
2024,
39,
1005-1031,
276,
10.1007/s11766-024-4148-y
13.
Muhammad Arshad, hassan Khan, Qadim Khan, Samia Bushnaq,
A new and efficient numerical algorithm to solve fractional boundary value problems,
2025,
0037-8712,
10.5269/bspm.75377
Xian Wen Sim, Sie Long Kek, Sy Yi Sim. State estimation and optimal control of an inverted pendulum on a cart system with stochastic approximation approach[J]. Applied Computing and Intelligence, 2023, 3(1): 79-92. doi: 10.3934/aci.2023005
Xian Wen Sim, Sie Long Kek, Sy Yi Sim. State estimation and optimal control of an inverted pendulum on a cart system with stochastic approximation approach[J]. Applied Computing and Intelligence, 2023, 3(1): 79-92. doi: 10.3934/aci.2023005