Initial coefficient bounds for certain new subclasses of bi-univalent functions with bounded boundary rotation

1.   Introduction

Let $\mathcal{A}$ be the class of all functions defined by

normalized by the conditions $f(0) = 0$ and $f^{\prime }(0)-1 = 0$, which are analytic in $\mathbb{D} = \left\{z : |z| < 1\right\}$. Furthermore, let us denote by $\mathcal{S}$ the subclass of $\mathcal{A}$ where the functions in $\mathcal{S}$ are also univalent in $\mathbb{D}$. Let $\mathcal{S}^{*}(\gamma)$ and $\mathcal{C}(\gamma)$ be the subclasses of $\mathcal{S}$ consisting of functions that are starlike of order $\gamma$ and convex of order $\gamma$, $0\leq \gamma < 1$. The analytic descriptions of the above two classes are respectively given by

and

Note that $\mathcal{S}^{*}(0)\equiv \mathcal{S}^{*}$ and $\mathcal{C}(0)\equiv \mathcal{C}$, the class of all starlike and convex functions. Let $0 \leq \gamma < 1$. A function $f(z) \in \mathcal{A}$ given in (1.1) with $f^{\prime}(z) \neq 0$ on $\mathbb{D}$ is said to be in the class of the close-to-convex function of order $\gamma$ if there exists a function $\phi \in \mathcal{S}^*$ such that

The class of all close-to-convex functions of order $\gamma$ are denoted by $\mathcal{K}(\gamma)$. For $0 \leq \gamma < 1$, a function $f \in \mathcal{A}$ of the form given in (1.1) with $f^{\prime}(z) \neq 0$ on $\mathbb{D}$ is said to be in the class of the close-to-star function of order $\gamma$ if there exists a function $\phi(z) \in \mathcal{S}^*$ such that

The class of all close-to-star functions of order $\gamma$ are denoted by $\mathcal{C}\mathcal{S}^*(\gamma)$. For details on close-to-convex functions and close-to-star functions, one may refer to ^[20] and ^[28,29] (see ^[2] also).

It is already known that every function $f \in \mathcal{S}$ has an inverse $f^{-1}$ that is defined by

and

(for details see ^[12]). It is to be remarked here that for $f \in \mathcal{S}$ and of the form (1.1), the inverse $f^{-1}$ may have an analytic continuation to $\mathbb{D}$, where

Let $\mathcal{A}_\sigma$ denote the class of functions of the form (1.1) defined on $\mathbb{D}$, for which the function $f \in \mathcal{A}$ and its inverse $f^{-1}\equiv g$ with Taylor series expansion as in (1.4), and both are univalent in $\mathbb{D}$. A function $f \in \mathcal{S}$ is said to be bi-univalent in $\mathbb{D}$ if there exists a function $g \in \mathcal{S}$ such that $g(z)$ is an univalent extension of $f^{-1}$ to $\mathbb{D}$. Let $\sigma$ denote the class of all bi-univalent functions in $\mathbb{D}$. The functions $\frac{z}{1-z}$, $\frac{1}{2} \log\left(\frac{1+z}{ 1-z}\right)$ and $-\log(1-z)$ are in the class $\sigma$. It is interesting to note that the famous Koebe function $\frac{z}{(1-z)^2}$ is not bi-univalent. Lewin ^[21] investigated the class of bi-univalent functions $\sigma$ and obtained a bound $|a_2| < 1.51$. Further, Brannan and Clunie ^[6] and Brannan and Taha ^[7] also worked on certain subclasses of the bi-univalent function class $\sigma$ and obtained the bounds for their initial coefficients. The study of bi-univalent functions gained concentration as well as thrust, mainly due to the investigation of Srivastava et al. ^[32]. Brannan and Taha ^[7] defined the classes $\mathcal{S} _{\sigma}^{*}(\gamma)$ and $\mathcal{C}_{\sigma}(\gamma)$ of bi-starlike functions of order $\gamma$ and bi-convex functions of order $\gamma$. The bounds on $|a_n|\, (n = 2, 3)$ for the class $\mathcal{S}_{\sigma}^{*}(\gamma)$ and $\mathcal{C}_{\sigma}(\gamma)$ (for details see ^[7]) were established and non-sharp. Subsequent to Brannan and Taha ^[7], lots of researchers (^[1], ^[8], ^{[10,11,12,13,14,15,16,17,18]}, ^[22], ^[24], ^[35,36,37]) in recent times have introduced and investigated several interesting subclasses of the class $\sigma$. They have obtained the bounds on the initial two Taylor-Maclaurin coefficients for the new bi-univalent classes, which they introduced and identified as non-sharp.

For $0 \leq \gamma < 1$, let $\mathcal{N}(\gamma)$ denote the class of all functions of the form (1.1) and satisfy the condition $\Re(f^{\prime}(z)) > \gamma$. This is called the class of functions whose derivatives have a positive real part of order $\gamma$.

For $0 \leq \gamma < 1$, a function $f \in \sigma$ given in (1.1) with $f^{\prime}(z) \neq 0$ on $\mathbb{D}$ is said to be in the class $\mathcal{N}_\sigma(\gamma)$ if

and

The class $\mathcal{N}_\sigma(\gamma)$ was discussed in ^[32]. For $0 \leq \gamma < 1$, let $\mathcal{F}_\sigma^\varrho(\gamma)$ denote the class of all functions $f \in \sigma$ and of the form (1.1) and satisfy the conditions

and

For $\varrho = 0$, $\mathcal{F}_\sigma^\varrho(\gamma) \equiv \mathcal{N}_\sigma(\gamma)$. This class $\mathcal{F}_\sigma^\varrho(\gamma)$ involving complex order was considered in ^[33].

For $0 \leq \gamma < 1$ and $\tau \geq0$, let $\mathcal{G}_\sigma^{\tau}(\gamma)$ denote the class of all functions $f \in \sigma$ and of the form (1.1) and satisfy the conditions

and

For $\tau = 1$, $\mathcal{G}_\sigma^1(\gamma) \equiv \mathcal{N}_\sigma(\gamma)$. The class $\mathcal{G}_\sigma^{\tau}(\gamma)$ was investigated in ^[13]. As a matter of fact, it is to be mentioned that the class of bi-close-to-convex functions of order $\gamma$ in the sense of Kaplan was studied by ^[31], and the class of bi-close-to-convex functions was also studied by Cho et al. ^[3].

Let $k \geq 2$ and $0 \leq \gamma < 1$. Let $\mathcal{P}_k(\gamma)$ denote the class of functions $p$, which are analytic and normalized with $p(0) = 1$, satisfying the condition

where $z = re^{i \theta} \in \mathbb{D}$. The class $\mathcal{P}_k (\gamma)$ was introduced by Padmanabhan and Parvatham ^[26] (see also ^[22]). If $\gamma = 0$, we denote $\mathcal{P}_k(0)$ as $\mathcal{P}_k$. Hence, the class $\mathcal{P}_k$ (defined by Pinchuk ^[27]) represents the class of analytic functions $p(z)$ with $p(0) = 1$, and the function $p \in \mathcal{P}_k$ will be having a representation

where $\mu$ is a real-valued function with a bounded variation satisfying

Remark 1. $\mathcal{P} \equiv \mathcal{P}_2$ is the class of analytic functions with a positive real part in $\mathbb{D}$, familiarly called as the class of Carathéodory functions.

For the class $\mathcal{P}_k$, the following lemma was proved.

Lemma 1. ^[27] For $p \in \mathcal{P}_k$, there exists $p_1, p_2 \in \mathcal{P}$ such that

Let $\mathcal{R}_k(\gamma)$ represent the class of analytic functions $h(z)$ in $\mathbb{D}$ with $h(0) = 0$, $h^{\prime }(0) = 1$ and satisfying

This class generalizes the class $\mathcal{S}^*(\gamma)$ of starlike functions of the order $\gamma$, investigated by Robertson ^[30]. For $\gamma = 0$, we get the class $\mathcal{R}_k(0)\equiv \mathcal{R}_k$, the class of all functions of bounded radius rotation. Therefore, the functions $h \in \mathcal{R}_k$ will be having a representation

where $\mu$ is a real-valued function with a bounded variation satisfying

Let $\mathcal{V}_k(\gamma)$ denote the class of all analytic functions $h(z)$ in $\mathbb{D}$ normalized by $h(0) = 0$ and $h^{\prime }(0) = 1$, satisfying

For $\gamma = 0$, we get the class $\mathcal{V}_k(0)\equiv \mathcal{V}_k$, the class of all analytic functions of a bounded boundary rotation studied by Paatero ^[25]. Therefore, the functions $h \in \mathcal{V}_k$ will be having a representation

where $\mu$ is a real-valued function with a bounded variation satisfying

The class $\mathcal{V}_k(\gamma)$ generalizes the class of all convex functions $\mathcal{C}(\gamma)$ of order $\gamma$, introduced by Robertson ^[30]. An interesting connection for the classes $\mathcal{V} _k(\gamma)$ and $\mathcal{R}_k(\gamma)$ with $\mathcal{P}_k(\gamma)$ was established by Pinchuk ^[27] and are given by

and

Let $\mathcal{S}_k$ be the subclass of $\mathcal{V}_k$ whose members are univalent in $\mathbb{D}$. It was pointed out by Paatero ^[25] that $\mathcal{V}_k$ coincides with $\mathcal{S}_k$ whenever $2 \leq k \leq 4$. Pinchuk ^[27] also proved that functions in $\mathcal{V}_k$ are close-to-convex in $\mathbb{D}$ if $2\leq k\leq4$ and, hence, are univalent. Brannan ^[5] showed that $\mathcal{V}_k$ is a subclass of the class $\mathcal{K}(\gamma)$ of the close-to-convex of order $\gamma = \frac{k}{2}-1$. If $f \in \mathcal{V}_k(\gamma)$ and $n = 2, 3$, then the sharp results are $|a_2|\leq \frac{k}{2}$ and $|a_3|\leq \frac{k^{2}+2}{6}$ (see ^[34]).

Lemma 2. ^[4,22] If $\Psi(z) = 1+\sum_{n = 1}^{\infty} B_n z^n$, $z \in \mathbb{D}$ be such that $\Psi \in \mathcal{P} _k(\gamma)$, then

Let us consider that the functions $p, q \in \mathcal{P}_k(\gamma)$, with

and

Then, from Lemma 2, we have

and

Lemma 3. [12, Theorem 2.14, p.44] If $\phi(z) = z+\sum_{n = 2}^{\infty} g_n z^n$, $z \in \mathbb{D}$ is a starlike function, then

Lemma 4. ^[19] If $\phi(z) = z+\sum_{n = 2}^{\infty} g_n z^n$, $z \in \mathbb{D}$ is a starlike function, then for $\mu \in \mathbb{R}$,

In the current article, we introduce new classes of bi-univalent functions with bounded boundary rotation. For these new classes, the authors first obtain two initial coefficient bounds. They also verify the special cases where the familiar Brannan and Clunie's conjecture are satisfied. Furthermore, the famous Fekete-Szegö inequality is obtained for these new classes of functions. The results of this article gives few interesting corollaries. Apart from a few of the results that generalize the earlier results existing in the literature, it also improvises the results of Srivastava et al. ^[32] and Frasin and Aouf ^[13].

Presume that if $f$ is given by (1.1), then

where $g = f^{-1}$.

For

one may get

Here, $\phi^{-1}(w) = \psi(w)$.

Throughout this article, unless or otherwise stated, $g, \phi$ and $\psi$ will have Taylor expansions as in (1.12), (1.13) and (1.14).

2.   Coefficient bounds for $\mathcal{K}_\sigma(k, \gamma)$

Definition 1. Suppose $0\leq\gamma < 1,$ $2 \leq k\leq 4$ and $\eta \geq 0$. Let $f \in\mathcal{A}_\sigma$ given by (1.1) such that $f^{\prime }(z) \neq 0$ on $\mathbb{D}$. Then, $f$ is said to be $\eta$-bi-close-to-star with bounded boundary rotation of order $\gamma$ if there exists functions $\phi, \psi \in \mathcal{S}^*$ satisfying

and

where $g$ is the analytic continuation of $f^{-1}$ to $\mathbb{D}$. The class of all such functions is denoted by $\mathcal{K}^\eta_\sigma(k, \gamma)$.

Remark 2. $({\rm{i}})$ For $\eta = 1$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^1_\sigma(k, \gamma) \equiv \mathcal{K}_\sigma(k, \gamma)$, the class of bi-close-to-convex functions with bounded boundary rotation of order $\gamma$.

$({\rm{ii}})$ For $\eta = 1$ and $\gamma = 0$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^1_\sigma(k, 0) \equiv \mathcal{K}_\sigma(k)$, the class of bi-close-to-convex functions with bounded boundary rotation.

$({\rm{iii}})$ For $\eta = 1$ and $k = 2$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^1_\sigma(2, \gamma) \equiv \mathcal{K}_\sigma(\gamma)$, the class of bi-close-to-convex functions of order $\gamma$.

$({\rm{iv}})$ For $\eta = 0$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^0_\sigma(k, \gamma) \equiv \mathcal{C}\mathcal{S}^*_\sigma(k, \gamma)$, the class of bi-close-to-star functions with bounded boundary rotation of order $\gamma$.

$({\rm{v}})$ For $\eta = 0$ and $\gamma = 0$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^1_\sigma(k, 0) \equiv \mathcal{C}\mathcal{S}^*_\sigma(k)$, the class of bi-close-to-star functions with bounded boundary rotation.

$({\rm{vi}})$ For $\eta = 0$ and $k = 2$, we get $\mathcal{K}^\eta_\sigma(k, \gamma) \equiv \mathcal{K}^0_\sigma(2, \gamma) \equiv \mathcal{C}\mathcal{S}^*_\sigma(\gamma)$, the class of bi-close-to-star functions of order $\gamma$.

Next, we obtain the initial coefficient bounds and $\left| a_3 - \mu a_2^{2} \right|$ for the class $\mathcal{K}^\eta_\sigma(k, \gamma)$.

Theorem 1. Let $f$ given by (1.1) be in the class $\mathcal{K}^\eta_\sigma(k, \gamma)$, $0\leq\gamma < 1$ and $2 \leq k\leq 4$, then

Further, if $\mu$ is real, then

Proof. Let $g, \phi$ and $\psi$ be given in the form (1.12), (1.13) and (1.14). Since $f \in \mathcal{K}^\eta_\sigma(k, \gamma)$, there exists analytic functions $p, q \in \mathcal{P}_k(\gamma)$ with

and

satisfying

and

Therefore,

and

From (2.10) and (2.11), we obtain

and

Then, from (2.12) and (2.14), we get $p_1 = -q_1$. The addition of (2.13) and (2.15) implies

By the relation $p_1 = -q_1$ and using Lemma 3, (1.8), (1.9) and applying in (2.16), we get

Equations (2.12) and (2.17) essentially gives (2.3). Using Lemma 3, (1.8), (1.9) and applying in (2.13), we have (2.4).

Now, by (2.13) and (2.16) and for all $\mu \in \mathbb{R}$,

Hence,

By using Lemma 4, we get (2.5). This completes the proof of Theorem 1. □

For the particular choice of $\eta = 1$, Theorem 1 gives the following coefficient estimates for the class $\mathcal{K}_\sigma(k, \gamma)$ and is stated as a corollary below.

Corollary 1. Let $0 \leq \gamma < 1$ and $2 \leq k \leq 4$. Let $f$ given by (1.1) be in the class $\mathcal{K}_\sigma(k, \gamma)$, then

and

Remark 3. It is evident to note that the familiar Brannan and Clunie's conjecture is true for $\mathcal{K}_\sigma(k, \gamma)$, the class of bi-close-to-convex functions with a bounded boundary rotation of order $\gamma$, for all $2 \leq k \leq 4$ if $\frac{3k-2}{3k} \leq \gamma < 1$.

For the particular choice of $\eta = 0$, Theorem 1 gives the following coefficient estimates for the class $\mathcal{C}\mathcal{S}^{*}_\sigma(k, \gamma)$ and is stated as a corollary below.

Corollary 2. Let $f$ given by (1.1) be in the class $\mathcal{C}\mathcal{S}^*_\sigma(k, \gamma)$. Further, if $0\leq\gamma < 1$ and $2 \leq k \leq 4$, then

and

For the particular choice of $\gamma = 0$, we denote ${\mathcal{K}}^{\eta}_\sigma(k, \gamma)$ by ${\mathcal{K}}^{\eta}_\sigma(k)$. For the class ${\mathcal{K}}^{\eta}_\sigma(k)$, Theorem 1 reduces the following corollary.

Corollary 3. If $f$ given by (1.1) belong to the class ${\mathcal{K}}^{\eta}_\sigma(k)$ and $2 \leq k \leq 4$, then

and

If we choose the function $\phi(z) = z$, we can get the following Theorem 2 very similar to that of Theorem 1. For the choice of $\phi(z) = z$, let us denote the class $\mathcal{K}^{\eta}_\sigma(k, \gamma)$ by ${\mathcal{K}}^{\eta}_\sigma[k, \gamma]$. Indeed, the class ${\mathcal{K}}^{\eta}_\sigma[k, \gamma]$ will consist of all functions $f \in \sigma$ of the form (1.1) and satisfy the conditions

and

where $g$ is the analytic continuation of $f^{-1}$ to $\mathbb{D}$. However, for obtaining the bounds for the class ${\mathcal{K}}^{\eta}_\sigma[k, \gamma]$, the calculation needs to be reworked and we omit the details involved.

Theorem 2. If $f$ given by (1.1) belong to the class ${\mathcal{K}}^{\eta}_\sigma[k, \gamma]$, $0\leq\gamma < 1$ and $2 \leq k\leq 4$, then

and

Remark 4. For the choice of $k = 2$, Theorem 2 improves the bound of $|a_3|$ and verifies the bound of $|a_2|$, obtained by Frasin and Aouf ^[13].

For the particular choice of $k = 2$, we denote ${\mathcal{K}}^{\eta}_\sigma[k, \gamma]$ by ${\mathcal{K}}^{\eta}_\sigma[\gamma]$. For the class ${\mathcal{K}}^{\eta}_\sigma[\gamma]$, Theorem 2 reduces the following corollary.

Corollary 4. If $f$ given by (1.1) belong to the class ${\mathcal{K}}^{\eta}_\sigma[\gamma]$ and $0\leq\gamma < 1$, then

and

Remark 5. $({\rm{i}})$ Since

Corollary 4 verifies that the bound of $|a_3|$ is less than that of the bound given by Srivastava et al. ^[32].

$({\rm{ii}})$ For the particular choice of $\eta = 1$ in Theorem 2, we have the class ${\mathcal{K}}^{1}_\sigma[k, \gamma] \equiv \mathcal{N}_\sigma(k, \gamma)$, consisting of all functions $f \in \sigma$ of the form (1.1) and satisfying the conditions

and

Finally, we will verify whether the Brannan and Clunie's conjecture is satisfied for the class $\mathcal{K}^\eta_\sigma(k, \gamma)$, and it is stated in the following corollary.

Corollary 5. If $f \in \mathcal{K}^\eta_\sigma(k, \gamma)$, then for $\frac{3k+2-4\eta}{3k}\leq \gamma < 1, \, \eta \geq 1$ and $2 \leq k\leq 4$,

3.   Coefficient bounds for $\mathcal{M}^\alpha_\sigma(k, \gamma)$

We start the section by introducing the definition of $\alpha$-bi-convex function of a bounded boundary rotation of order $\gamma$.

Definition 2. Suppose $0\leq\gamma < 1$ and $2 \leq k \leq 4$. Let $\alpha$ be real. A function $f$ given by (1.1) is said to be $\alpha$-bi-convex function with a bounded boundary rotation of order $\gamma$ with $f(z)\cdot f^{\prime}(z) \neq 0$ if for $z \in \mathbb{D}$,

and

where $g$ is the analytic continuation of $f^{-1}$ to $\mathbb{D}$. The class of all $\alpha$-bi-convex functions with a bounded boundary rotation of order $\gamma$ are denoted by $\mathcal{M}^\alpha_\sigma(k, \gamma)$.

Remark 6. $({\rm{i}})$ For $k = 2$, we get $\mathcal{M}^\alpha_\sigma(2, \gamma) \equiv \mathcal{M}^\alpha_\sigma(\gamma)$, the class of $\alpha$-bi-convex functions of order $\gamma$.

$({\rm{ii}})$ For $\alpha = 0$, we get $\mathcal{M}^\alpha_\sigma(k, \gamma) \equiv \mathcal{M}^0_\sigma(k, \gamma) \equiv \mathcal{S}^*_\sigma(k, \gamma)$ ^[22], the class of bi-starlike functions with bounded boundary rotation of order $\gamma$.

$({\rm{iii}})$ For $\alpha = 0$ and $k = 2$, we get $\mathcal{M}^\alpha_\sigma(k, \gamma)\equiv \mathcal{M}^0_\sigma(2, \gamma) \equiv \mathcal{S}^*_\sigma(\gamma)$, the class of bi-starlike functions of order $\gamma$.

$({\rm{iv}})$ For $\alpha = 1$, we get $\mathcal{M}^\alpha_\sigma(k, \gamma)\equiv \mathcal{M}^1_\sigma(k, \gamma) \equiv \mathcal{C}_\sigma(k, \gamma)$ ^[22], the class of bi-convex functions with bounded boundary rotation of order $\gamma$.

$({\rm{v}})$ For $\alpha = 1$ and $k = 2$, we get $\mathcal{M}^\alpha_\sigma(k, \gamma)\equiv \mathcal{M}^1_\sigma(2, \gamma) \equiv \mathcal{C}_\sigma(\gamma)$, the class of bi-convex functions of order $\gamma$.

Theorem 3. If $f$ given by (1.1) belong to the class $\mathcal{M}^\alpha_\sigma(k, \gamma)$, $0\leq\gamma < 1$ and $2 \leq k\leq 4$, then

and

where $\rho = \frac{3+5\alpha}{2(1+2\alpha)}$ and $\delta = \frac{1+3\alpha}{2(1+2\alpha)}.$

Proof. Let $g$ be given in the form (1.12). Since $f \in \mathcal{M}^\alpha_\sigma(k, \gamma)$, there exists analytic functions $p, q \in \mathcal{P}_k(\gamma)$ with

and

satisfying

and

Therefore,

and

From (3.11) and (3.12) we get

and

Then, from (3.13) and (3.15), we get $p_1 = -q_1$. The addition of (3.14) and (3.16) implies

Now, using (1.8) and (1.9) in (3.17) we have

This essentially gives (3.3). An application of (3.18), (1.8) and (1.9) in (3.14) at once gives (3.4). Now, (3.14) can be written as

Furthermore,

where

Now, (3.16) can be written as

Furthermore,

where

This completes the proof of Theorem 3. □

For the particular choice of the function $\alpha = 0$ in Theorem 3, we have the following coefficient bounds for the class $\mathcal{S}^*_\sigma(k, \gamma)$.

Corollary 6. Let $f$ given by (1.1) be in the class $\mathcal{S}^*_\sigma(k, \gamma)$, $0\leq\gamma < 1$ and $2 \leq k \leq 4$, then

and

Remark 7. For the choice of $k = 2$, Corollary 6 verifies the coefficient bounds of $|a_2|$ and $|a_3|$, obtained by Mishra and Soren ^[23], for the class of bi-starlike functions of order $\gamma$.

For the particular choice of the function $\alpha = 1$ in Theorem 3, we have the following coefficient estimates for the class $\mathcal{C}_\sigma(k, \gamma)$.

Corollary 7. Let $f$ given by (1.1) be in the class $\mathcal{C}_\sigma(k, \gamma)$, $0\leq\gamma < 1$ and $2 \leq k \leq 4$, then

and

4.   Coefficient bounds for $\mathcal{S}^*_\sigma(k, \beta, \gamma)$

Definition 3. Suppose $0\leq\gamma < 1$ and $2 \leq k\leq 4$. Let $\beta$ be real. A function $f$ given by (1.1) is said to be in the class $\mathcal{S}^*_\sigma(k, \beta, \gamma)$ if it satisfies the conditions

and

where $g$ is the analytic continuation of $f^{-1}$ to $\mathbb{D}$.

A similar type of the class with the left hand side expression involving subordination was studied in ^[1], ^[9] and ^[38].

Remark 8. $({\rm{i}})$ For $\beta = 0$, we get $\mathcal{S}^*_\sigma(k, \beta, \gamma) \equiv \mathcal{S}^*_\sigma(k, 0, \gamma)\equiv \mathcal{S}^*_\sigma(k, \gamma)$, the class of bi-starlike functions with bounded boundary rotation of order $\gamma$.

$({\rm{ii}})$ For $\beta = 0$ and $k = 2$, we get $\mathcal{S}^*_\sigma(k, \beta, \gamma) \equiv \mathcal{S}^*_\sigma(2, 0, \gamma)\equiv \mathcal{S}^*_\sigma(\gamma)$, the class of bi-starlike functions of order $\gamma$.

$({\rm{iii}})$ For $\beta = 0$, $k = 2$ and $\gamma = 0$, we get $\mathcal{S}^*_\sigma(k, \beta, \gamma) \equiv \mathcal{S}^*_\sigma(2, 0, 0)\equiv \mathcal{S}^*_\sigma$, the class of bi-starlike functions.

Theorem 4. Let $f$ given by (1.1) be in the class $\mathcal{S}^*_\sigma(k, \beta, \gamma)$, $0\leq\gamma < 1$ and $2 \leq k\leq 4$, then

and

where $\chi = \frac{3+10\beta}{2(1+3\beta)}$ and $\nu = \frac{1+2\beta}{2(1+3\beta)}.$

Proof. Let $g$ be given in the form (1.12). Since $f \in \mathcal{S}^*_\sigma(k, \beta, \gamma)$, there exists analytic functions $p, q \in \mathcal{P}_k(\gamma)$ with

and

satisfying

and

Therefore,

and

From the Eqs (4.11) and (4.12), we obtain

and

Then, from (4.13) and (4.15), we get $p_1 = -q_1$. By an addition of (4.14) and (4.16), we get

Now, applying relation $p_1 = -q_1$ in (4.17) and using Eq (4.13), we get

Now, using (1.8) and (1.9) in (4.18), we have

This essentially gives (4.3). An application of (4.13), (1.8) and (1.9) in (4.14) at once gives (4.4). Now, Eq (4.16) can be written as

Furthermore,

where

Thus, Eq (4.14) can be written as

Furthermore,

where

This completes the proof of Theorem 4. □

Remark 9. $({\rm{i}})$ For $\beta = 0$, $\mathcal{S}^*_\sigma(k, 0, \gamma) \equiv \mathcal{S}^*_\sigma(k, \gamma)$ and Theorem 4 reduce to Corollary 6.

$({\rm{ii}})$ For $\beta = 0$ and $k = 2$, Theorem 4 reduces to the coefficient bounds of $|a_2|$ and $|a_3|$, given by Mishra and Soren ^[23].

For $\beta = 0$, $\gamma = 0$ and $k = 2$, Theorem 4 reduces to the following corollary as stated below.

Corollary 8. Let $f$ given by (1.1) be in the class $\mathcal{S}^*_\sigma(2, 0, 0) \equiv \mathcal{S}^*_\sigma$, then

and

5.   Coefficient bounds for class $\mathcal{F}_\sigma^\varrho(k, \gamma)$

Definition 4. Suppose $0\leq\gamma < 1$, $\varrho \geq 0$ and $2 \leq k\leq 4$. Let $\mathcal{F}_\sigma^\varrho(k, \gamma)$ denote the class of all functions $f \in \sigma$ of the form (1.1) and satisfy the conditions

and

where $g$ is the analytic continuation of $f^{-1}$ to $\mathbb{D}$.

Theorem 5. Let $f$ given by (1.1) be in the class $\mathcal{F}_\sigma^\varrho(k, \gamma)$, $\varrho \geq 0$, $0\leq\gamma < 1$ and $2 \leq k\leq 4$, then

and

Proof. Let $g$ be given in the form (1.12). Since $f \in \mathcal{F}_\sigma^\varrho(k, \gamma)$, there exists analytic functions $p, q \in \mathcal{P}_k(\gamma)$ with

and

satisfying

and

From (5.8) and (5.9), we obtain

and

Then, from (5.11) and (5.13), we get

Now, using (1.8) and (1.9) in (5.14), we have

This essentially yields (5.3). An application of (1.9) in (5.11) at once gives (5.4). Now, Eq (5.13) can be written as

An application of (5.12) in (5.16) gives (5.5). This completes the proof of Theorem 5. □

Remark 10. For $k = 2$, Theorem 5 verifies the $|a_2|$ bound and improves the bound of $|a_3|$ obtained in ^[33].

Concluding remarks and observations

In this article, we investigated the estimates of second and third Taylor–Maclaurin coefficients for new subclasses of bi-univalent functions of order $\gamma$ with bounded boundary rotation. Also, interesting Fekete-Szegö coefficient estimates for functions in these subclasses were obtained. The authors have verified the special cases where the familiar Brannan and Clunie's conjecture were satisfied. Interesting remarks on the main results including improvements of the earlier bounds were also given. Apart from these remarks, which are given in the present article, more corollaries and remarks can be stated for the choice of $\gamma = 0$, and those details are omitted.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The work of third author is supported by the Basic Science Research Program through the National Research foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (No.2019R1I1A3A01050861).

Conflict of interest

Prof. Dr. Cho is the Guest Editor of special issue "Geometric Function Theory and Special Functions" for AIMS Mathematics. Prof. Dr. Cho was not involved in the editorial review and the decision to publish this article.

All authors declare there are no conflicts of interest in this paper.