Fuzzy gradient descent for the linear fuzzy real number system

1.   Introduction

Problems in production planning and scheduling, location, transportation, finance, and engineering design sometimes require that decisions be made under uncertainty. In education, there exist Fuzzy Educational datasets, which are useful sources of information for educational software developers, students, parents, researchers and other educational stakeholders ^[2]. Fuzzy decision making was initially introduced by Bellman and Zadeh ^[1]. This concept was then adopted to mathematical programming ^[14], uncertain programming ^[12,14], fuzzy posets ^[6], linear programming ^{[3,7,8,10,13,14]} and many other concepts.

Linear Fuzzy Real (LFR) numbers are a system of numbers that have properties comparable to the set of real numbers and the set of fuzzy numbers ^[9]. LFR numbers are used in the study of fuzzy random variables ^[4,9] and in the linear optimization problem ^[8]. The set of LFR numbers is a set that shows intermediate properties which are unique to the set and not to those of either the real numbers or the "general" fuzzy numbers. Because of the unique properties of LFR numbers, we can solve fuzzy linear and nonlinear problems ^[11].

At the core of machine learning techniques, there are many types of cost functions. The process of increasing the accuracy of our model by iterating over a training data involves searching for the lowest error in the cost function. The cost function is used to check the error in predictions of a Machine Learning model. In many learning algorithms the Gradient Descent is used to minimize the cost function ^[5].

In this paper, the Gradient Descent method is used to find both the extreme values and find the solutions of fuzzy equations.

The Crisp Gradient Descent algorithm begins with an initial value x⁰ and generates the sequence $\left\{ {{x^n}} \right\}_{n = 0}^\infty$, by

The algorithm finds the minimum of a function f by taking small steps proportion to the negative of the gradient of the function f. The symbol γ is the measure of the small steps and $\nabla F$ is the gradient of f.

Solving fuzzy equations also implies that the Gradient Descent method can be used to minimize fuzzy cost functions since the values of x that satisfies

Will also be the global minimizer of

The paper is outlined as follows: Linear Fuzzy Real numbers are reviewed in Section 2. In Section 3, we look at nonlinear equations in LFR. In Section 4, we illustrate LFR Gradient Descent method to solve systems of nonlinear equations in LFR. Finally, conclusions are made in Section 5.

2.   Linear fuzzy real numbers

In this section, we describe the Linear Fuzzy Real numbers ^[7,10]. Considering the set of all real numbers R, one way to associate a fuzzy number with a fuzzy subset of real numbers is as a function μ: R → [0, 1], where the value μ(x) is to represent a degree of belonging to the subset of R.

Definition 2.1. (Linear fuzzy real number) Let μ: R → [0, 1] be a function such that:

1. μ(x) = 1 if x = b;

2. μ(x) = 0 if x ≤ a or x ≥ c;

3. μ(x) = (x - a)/(b - a) if a < x < b;

4. μ(x) = (c - x)/(c - b) if b < x < c.

Then μ(a, b, c) is called a linear fuzzy real number with associated triple of real numbers (a, b, c) where a ≤ b ≤ c shown in Figure 1.

We let LFR be the set of all linear fuzzy real numbers. We note that any real number b can be written as a linear fuzzy real number, r(b), where r(b) = μ(b, b, b) and so R $\subseteq$ LFR. As a linear fuzzy real number, we consider r(b) to represent the real number b itself. A linear fuzzy real number μ(a, b, c) is defined to be positive if a > 0, negative if c < 0, and zeroic if a ≤ 0 and c ≥ 0. The hybrid nature of LFR allows one to map the set to an isomorphic set where we can use inverse operations to solve fuzzy linear equations ^[10].

Operations, functions, and linear equations are also defined in LFR as follows:

Definition 2.2. (Operations in LFR) For given two linear fuzzy real numbers μ₁ = μ(a₁, b₁, c₁) and μ₂ = μ(a₂, b₂, c₂), we define addition, subtraction, multiplication, and division by

1. μ₁ + μ₂ = μ(a₁ + a₂, b₁ + b₂, c₁ + c₂);

2. μ₁ - μ₂ = μ(a₁ - c₂, b₁ - b₂, c₁ - a₂);

3. μ₁ · μ₂ = μ(min{a₁a₂, a₁c₂, a₂c₁, c₁c₂}, b₁b₂, max{a₁a₂, a₁c₂, a₂c₁, c₁c₂}).;

4. $\frac{\mu_{1}}{\mu_{2}} = \mu_{1} * \frac{1}{\mu_{2}}$ where $\frac{1}{\mu_{2}} = \mu\left(\min \left\{\frac{1}{a_{2}}, \frac{1}{b_{2}}, \frac{1}{c_{2}}\right\}, \operatorname{median}\left\{\frac{1}{a_{2}}, \frac{1}{b_{2}}, \frac{1}{c_{2}}\right\}, \max \left\{\frac{1}{a_{2}}, \frac{1}{b_{2}}, \frac{1}{c_{2}}\right\}\right)$.

Definition 2.3. (Function in LFR) Given real-valued function f: R → R and an LFR μ(a, b, c), the LFR-valued function ${\dddot f}$: LFR → LFR is defined as

where a^* = min{f(a), f(b), f(c)}, b* = median{f(a), f(b), f(c)}, c* = max{f(a), f(b), f(c)}. We note that if a = b = c then a^* = b^* = c*, i.e., ${\bar f}$ (r(b)) = r(f(b)). Hence ${\dddot f}$ is an extension of the function f.

Definition 2.4. (Linear equation in LFR) A linear equation over LFR is an equation of the form

where the μ_i are LFR's for i = 1, 2, 3 and μ_x = μ(α, β, γ) is an unknown LFR with a triple of unknown real numbers (α, β, γ).

Definition 2.5. (Ordering Properties of LFR) Given μ₁, μ₂ ∈ LFR, μ₁ ≤ μ₂ provided that

If $0 \le \mu (a, b, c)$ then 0 ≤ a ≤ b ≤ c, hence μ is a non-negative linear fuzzy real number. Therefore, if μ is non-negative and zeroic, then a = 0 precisely.

(LFR, ≤) is a complete ordered set. However, it is not linearly ordered. If we let μ₁ = μ(3, 4, 5) and let μ₂ = μ(a, 5, 6) and state that a < 3, then it is not true that μ₁ ≤ μ₂ nor is it true that μ₁ ≥ μ₂. Therefore, μ₁ and μ₂ are incomparable in this order ^[10].

3.   Nonlinear equations in LFR

Nonlinear equations can be found in many applications, all the way from light diffraction to planetary orbits for example ^[10]. In this section, we discuss how to solve nonlinear equations in LFR so that we may locate the maximum or minimum of nonlinear functions. It is important that we discuss locating fuzzy zeros of nonlinear functions. Thus, we will find a μ_x in LFR such that a nonlinear equation f(μ_x) = 0, where f: LFR → LFR is a nonlinear function.

3.1.   Support for LFR gradient descent for solving nonlinear equations

Solving and optimizing nonlinear equations over linear fuzzy real numbers is possible with a modification of the Gradient Descent method. This method is also known as the Steepest Descent method over real numbers.

Definition 3.1. A function f satisfies Lipschitz if there is a real number N such that |f(x)-f(y)| $≤$ N|x-y|.

We can apply Definition 3.1 to the function $g = \nabla f$, such that there is a real number P such that |g(x)-g(y)| $≤$ P|x-y|. Thus, the function g satisfies Lipschitz.

Proposition 3.1. The function f satisfies Lipschitz condition in the LFR environment for a real number N.

Proof: Given the existence of f and f'( μ) where f is continuous in a compact interval. Let us suppose that $\left|f^{\prime}(\mu)\right| \leq k$ for all $\mu_{x} \in\left[\mu_{a}, \mu_{b}\right]$ such that μ_x≠μ_y,

Thus, we have $\left|\frac{f\left(\mu_{x}\right)-f\left(\mu_{y}\right)}{\left(\mu_{x}-\mu_{y}\right)}\right| = \left|f^{\prime}\left(\mu_{c}\right)\right| \leq k$.

For some $\mu_{c} \in\left[\mu_{x}, \mu_{y}\right]$ by the mean value theorem. Then $\left|f\left(\mu_{x}\right)-f\left(\mu_{y}\right)\right| \leq k\left|\mu_{x}-\mu_{y}\right|$.

Therefore, the function $f$ satisfies Lipschitz condition on any interval $\left[\mu_{a}, \mu_{b}\right]$

Since we can apply Taylor's theorem to f in LFR ^[10], Proposition 3.1 coupled with the application of the mean value theorem, there is a constant M, such that f is Lipschitz continuous in LFR.

Proposition 3.2. Suppose the function ${\dddot f}$ : $LF{R^n} \to LFR$, where ${\dddot f}$ is an extension of the real function f. ${\dddot f}$ is convex and differentiable and the gradient ${\dddot f}$ is an extension of the gradient and real function $\nabla f$. $\nabla f$ is Lipschitz continuous with constant K > 0. Then if we run gradient descent for j iterations with a fixed step ${\rm{t}} \le \frac{1}{K}$, it will yield a solution that satisfies

Where ${\dddot f}(\mu_{x}^{*})$ is the optimal value. $\mu_{x}^{*}$ is the value that minimizes the function, ${\dddot f}$.

Proof

Because $\nabla{\dddot f}$ is Lipschitz continuous with constant K, $\nabla^{2} \dddot{f}\left(\mu_{x}\right)$ -KI is a negative semidefinite fuzzy matrix. $\nabla^{2} \dddot{f}$ is an extension of the hessian and real function ${\nabla ^2}f$. The identity matrix, I, is a matrix in LFR^[10].

Because of LFR's hybrid nature we can expand around ${\dddot f}(\mu_{x})$ apply the rules of differential calculus^[10]. This yields the following inequality.

In general, we wish to minimize the increase in the expansion of the area of a fuzzy number. For this reason, in the case of the expansion, we use b, the crisp maximum value of $\mu_{x}$ in the Jacobian and Hessian. Thus, we use a modified expansion

Where ${\nabla ^2}{\dddot f}\left(b \right)$ -KI is a semidefinite matrix. Thus

Taking the next step in the gradient descent update by letting ${\mu _y} - {\mu _x}t{\nabla {\dddot f}}\left(b \right)$

Then

If we set ${{t}} \le \frac{1}{K}$, we find that $-\left(1-\frac{1}{2} K t\right) \leq-\frac{1}{2}$, thus we have

This inequality implies that the objective value decreases with each iteration of the gradient descent. Since ${\dddot f}$ is convex.

which implies that

Thus

By factoring, we have

And thus, the triangle inequality yields

If we note that another step of the gradient descent, by definition yields ${\mu _y} - {\mu _x}t\nabla {\dddot f}\left(b \right)$

Then ${\mu _y} - {\mu _x}t\nabla {\dddot f}\left(b \right)$ and

This holds for ${\mu _y}$ on every iteration of the modified gradient descent. If we sum over the iterations.

Given the update (6) and using the fact that f is decreasing. We can conclude the following:

Which is what we sought to prove. Thus, the minimal value ${\dddot f}(\mu_{x}^{*})$ is reached where both $\mu_{x}^{*}$ and ${\dddot f}(\mu_{x}^{*})$ are Linear Fuzzy Real (LFR) number as defined by Definition 2.1. It follows that Proposition 3.2 applies to (1). As well as the modified form (3).

3.2.   LFR gradient descent for searching for extreme values of nonlinear equations

The Modified Gradient Descent method begins with an initial approximation LFR, ${\mu ^{(0)}}_x = \mu \left({{a^{(0)}}, {b^{(0)}}, {c^{(0)}}} \right)$, and generates the sequence $\left\{ {\mu _x^{(n)}} \right\}_{n = 0}^\infty$ with $\mu \left({{a^{({\rm{n}})}}, {b^{({\rm{n}})}}} \right)$, by

The stopping criterion of this method is $\left| {{b^n} - {b^{n - 1}}} \right|$ < $\epsilon$, where $\epsilon$ is a preset small value.

Example 3.1. Find the local minimum of an LFR nonlinear equation ${\mu ^4}_x - 3{\mu ^3}_x + 2$.

This is an equation with fuzzy coefficients. With typical calculation, it is expected that the local minimum occurs "around" 2.25.

If we set f (μ_x) = μ⁴_x - 3 μ³_x +2, then f '(μ_x) = 4μ³_x - 9μ²_x . If we use an initial approximation μ⁽⁰⁾_x = μ(4.5, 5, 5.5). If we set $\gamma = \frac{1}{\alpha }$ . Then we can set $\gamma = 0.001$ and $\epsilon = 0.00001$ . Using Python, after 356 iterations, we find a fuzzy solution μ^(*)_x = μ(1.7505, 2.2505, 2.7505).

Example 3.2. Find the extreme values of an LFR nonlinear equation

This is a multivariable equation with fuzzy coefficients. With typical calculation, it is expected that the optimal solution occurs "around" (1, 1).

Let us set ${\rm{F}}({\rm{x}}) = 5*{\left({{\mu _{x1}}} \right)^\mathit{2}} + {\left({{\mu _{x2}}} \right)^\mathit{2}} + 4{\mu _{x1}}*{\mu _{x2}} - 14{\mu _{x1}} - 6{\mu _{x2}} + 20$

then we have

If we use an initial approximation $\mu _{x1}^{(0)} = \mu (- 0.5, 0, 0.5)$, and $\mu _{x2}^{(0)} = \mu (9.5, 10, 10.5)$. If we set $\gamma=0.02$ and $\epsilon = 0.000001$ . Using Python, after 59583 iterations, we find fuzzy solutions μ^(*)_x1 = μ(0.44704, 0.944244, 1.44704) and μ^(*)_x2 = μ(0.63582, 1.13461, 1.63582).

4.   LFR gradient descent for solving system of nonlinear equations

In this section, we discuss using Gradient Descent to solve system of nonlinear equations in LFR. To illustrate, suppose we solve the following system of two LFR nonlinear equations:

Let the vector function F and the Jacobian matrix $J = \nabla F$ be the followings:

And

Then LFR Gradient Descent method generates the solution sequences $\left\{\mu_{x 1}^{(n)}\right\}_{n = 0}^{\infty} \text { and }\left\{\mu_{x 2}^{(n)}\right\}_{n = 0}^{\infty}$ such as:

Where $\mu_{x 1}^{(n)} = \mu\left(a_{x 1}^{(n)}, b_{x 1}^{(n)}, c_{x 1}^{(n)}\right) \quad \text { and } \quad \mu_{x 2}^{(n)} = \mu\left(a_{x 2}^{(n)}, b_{x 2}^{(n)}, c_{x 2}^{(n)}\right)$.

The stopping criterion of this method is $\min \left\{\left|b_{x 1}^{(n)}-b_{x 1}^{(n-1)}\right|, \left|b_{x 2}^{(n)}-b_{x 2}^{(n-1)}\right|\right\} < \epsilon$, where $\epsilon$ is a preset small value.

Example 4.1. Solve an LFR system of nonlinear equations:

We easily find F and J as

And

With typical calculation, it is expected that the optimal solution occurs "around" (2, 1, 3). If we use initial approximations $\mu_{x 1}^{(0)} = \mu_{x 2}^{(0)} = \mu_{x 3}^{(0)} = \mu(-1, 2, 4)$. And set $\gamma = 0.02$ and $\epsilon = 0.000001$.

Then in Python after 121193 iterations, LFR Gradient Descent Method yields

5.   Conclusion

We often do not know the consequence of each possible decision with precision, sometimes we can only know this case with uncertainty. In many cases the LFR approach may be a good approach. The simplest case for example would be tolerance, where all possible values within the interval are equally likely. LFR presents an advantage where it may represent an interval, a crisp point, or a fuzzy number that can be written discretely or continuously. This is an advantage in the case of certain problems where it is already known that "crisp optima" in the purest sense do not exist, but projecting to the middle μ(a, b, c) → μ(b, b, b) = b produces a "crisp good choice" for an optimal value in optimization ^[12] and machine learning. It is also possible to approximate other fuzzy number types, using LFR. In this paper, we used LFR coupled with the Gradient Descent to solve and optimize Fuzzy Nonlinear problems and systems of fuzzy problems. As has been shown, LFR is a viable environment for machine learning techniques.

Acknowledgments

This work has been financially supported by Project Engage: ROAR a grant funded by the United States Department of Education. Grant number: P120A160122.

Conflict of interest

The author declares that there is no conflict of interests.