Citation: Edward S. Canepa, Alexandre M. Bayen, Christian G. Claudel. Spoofing cyber attack detection in probe-based traffic monitoring systems using mixed integer linear programming[J]. Networks and Heterogeneous Media, 2013, 8(3): 783-802. doi: 10.3934/nhm.2013.8.783
[1] | Shaofang Hong, Rongjun Wu . On deep holes of generalized Reed-Solomon codes. AIMS Mathematics, 2016, 1(2): 96-101. doi: 10.3934/Math.2016.2.96 |
[2] | Jing Huang, Jingge Liu, Dong Yu . Dimensions of the hull of generalized Reed-Solomon codes. AIMS Mathematics, 2024, 9(6): 13553-13569. doi: 10.3934/math.2024661 |
[3] | Xiaofan Xu, Yongchao Xu, Shaofang Hong . Some results on ordinary words of standard Reed-Solomon codes. AIMS Mathematics, 2019, 4(5): 1336-1347. doi: 10.3934/math.2019.5.1336 |
[4] | Xiaofan Xu, Yongchao Xu . Some results on deep holes of generalized projective Reed-Solomon codes. AIMS Mathematics, 2019, 4(2): 176-192. doi: 10.3934/math.2019.2.176 |
[5] | Claude Carlet . Identifying codewords in general Reed-Muller codes and determining their weights. AIMS Mathematics, 2024, 9(5): 10609-10637. doi: 10.3934/math.2024518 |
[6] | Xuesong Si, Chuanze Niu . On skew cyclic codes over $ M_{2}(\mathbb{F}_{2}) $. AIMS Mathematics, 2023, 8(10): 24434-24445. doi: 10.3934/math.20231246 |
[7] | Wei Qi . The polycyclic codes over the finite field $ \mathbb{F}_q $. AIMS Mathematics, 2024, 9(11): 29707-29717. doi: 10.3934/math.20241439 |
[8] | Guanghui Zhang, Shuhua Liang . On the construction of constacyclically permutable codes from constacyclic codes. AIMS Mathematics, 2024, 9(5): 12852-12869. doi: 10.3934/math.2024628 |
[9] | Adel Alahmadi, Tamador Alihia, Patrick Solé . The build up construction for codes over a non-commutative non-unitary ring of order $ 9 $. AIMS Mathematics, 2024, 9(7): 18278-18307. doi: 10.3934/math.2024892 |
[10] | Ismail Aydogdu . On double cyclic codes over $ \mathbb{Z}_2+u\mathbb{Z}_2 $. AIMS Mathematics, 2024, 9(5): 11076-11091. doi: 10.3934/math.2024543 |
Data-intensive machine learning has become widely used, and as the size of training data increases, distributed methods are becoming increasingly popular. However, the performance of distributed methods is mainly determined by stragglers, i.e., nodes that are slow to respond or are unavailable.
Raviv et al. [11] used coding theory and graph theory to reduce stragglers in distributed synchronous gradient descent. A coding theory framework for straggler mitigation, called gradient coding, was first introduced by Tandon et al. [14]. Gradient coding consists of a system with one master and $ n $ worker nodes, where the data are partitioned into $ k $ parts, and one or more parts are assigned to each worker. In turn, each worker computes the partial gradients on each given partition, combines the results linearly according to a predefined vector of coefficients, and sends this linear combination back to the primary node. By choosing the coefficients at each node appropriately, it can be guaranteed that the primary node can reconstruct the full gradient even if a machine fails to do its job.
The importance of straggler mitigation is demonstrated in [8,16]. Specifically, it was shown by Tandon et al. [14] that stragglers run up to 5 times slower than the performance of typical workers (8 times in [16]). In [11], for gradient calculations, a cyclic maximum distance separable (MDS) code is used to obtain a better deterministic construction scheme than existing solutions, both in the range of parameters that can be applied and in the complexity of the algorithms involved.
One well-known family of MDS codes is generalized Reed-Solomon (GRS) codes. GRS codes have interesting mathematical structures and many real-world applications, such as mass storage systems, cloud storage systems, and public-key cryptosystems. On the other hand, although more complex than cyclic codes, quasi-cyclic codes satisfy the condition of the Gilbert-Varshamov lower bound at minimum distances, as shown in [6]. Quasi-cyclic codes are also equivalent to linear codes with circulant block generator matrices. This type of matrix has circular blocks of the same size, such as $ m $, which denotes the co-indexes of the associated quasi-cyclic code. From this point of view, one way to generalize quasi-cyclic codes is to let the generator matrix have circular blocks of different sizes. This code is called a generalized quasi-cyclic code with shared indices $ (m_1, m_2, \dots, m_k), $ where $ m_1, m_2, \dots, m_k $ represents the size of the circular block in the generator matrix.
In [10], a generalized quasi-cyclic code without block length limitations is studied. By relaxing the conditions on block length, several new optimal codes with small lengths could be found. In addition, the code decomposition and dimension formulas given by [3,12,13] have been generalized.
In this paper, we describe the construction of generalized quasi-cyclic GRS codes over totally real number fields, as well as their application in exact gradient coding. The construction method is derived by integrating known results from the inverse Galois problem for totally real number fields. Furthermore, methods in [2,4,11,14] will be adapted to generalized quasi-cyclic GRS codes to mitigate stragglers.
Let $ \mathbb{F} $ be a Galois extension of $ \mathbb{Q} $ and choose non-zero elements $ v_1, \dots, v_n $ in $ \mathbb{F} $ and distinct elements $ a_1, \dots, a_n $ in $ \mathbb{F}. $ Also, let $ \mathbf{v} = \left(v_1, \dots, v_n\right) $ and $ \mathbf{a} = \left(a_1, \dots, a_n\right). $ For $ 1\leq k\leq n, $ define the GRS codes as follows:
$ GRS_{n,k}\left(\mathbf{a},\mathbf{v}\right) = \left\{\left(v_1f(a_1),\dots,v_nf(a_n)\right)\left|\right.f(x)\in\mathbb{F}[x]_k\right\}, $ |
where $ \mathbb{F}[x]_k $ is the set of all polynomials over $ \mathbb{F} $ with degree less than $ k. $ The canonical generator of $ GRS_{n, k}\left(\mathbf{a}, \mathbf{v}\right) $ is given by the following matrix:
$ G=(v1v2⋯vj⋯vnv1a1v2a2⋯vjaj⋯vnanv1a21v2a22⋯vja2j⋯vna2n⋮⋮⋱⋮⋱⋮v1ai1v2ai2⋯vjaij⋯vnain⋮⋮⋱⋮⋱⋮v1ak−11v2ak−12⋯vjak−1j⋯vnak−1n) $
|
(2.1) |
Theorem 2.1. [7] Let $ \mathbf{v}\in\mathbb{F}^n $ be a tuple of non-zero elements in $ \mathbb{F} $ and $ \mathbf{a}\in\mathbb{F}^n $ be a tuple of pairwise distinct elements in $ \mathbb{F} $; then,
a) The $ GRS_{n, k}\left(\mathbf{a}, \mathbf{v}\right) $ is a $ [n, k, n-k+1] $ code, i.e., GRS codes are MDS codes.
b) The dual code of $ GRS_{n, k}\left(\mathbf{a}, \mathbf{v}\right) $ is as follows:
$ GRS_{n,k}\left(\mathbf{a},\mathbf{v}\right)^\bot = GRS_{n,n-k}\left(\mathbf{a},\mathbf{u}\right), $ |
where $ \mathbf{u} = \left(u_1, \dots, u_n\right) $ with
$ u_i^{-1} = v_i\prod\limits_{j\not = i}\left(a_i-a_j\right). $ |
Proof. (a) See the proof of [7, Theorem 6.3.3]. (b) See the proof of [7, Theorem 6.5.1].
Let $ \overline{\mathbb{F}} = \mathbb{F}\cup\{\infty\} $ and $ \mathbf{a} $ be an $ n $-tuple of mutually distinct elements of $ \overline{\mathbb{F}}, $ and let $ \mathbf{c} $ be an $ n $-tuple of non-zero elements of $ \mathbb{F}. $ Also, define
$ [a_i,a_j] = a_i-a_j,\quad [\infty,a_j] = 1\quad [a_i,\infty] = -1\quad \text{for all}\;a_i,a_j\in\mathbb{F}. $ |
Definition 2.2. ([9]) Let $ B(\mathbf{a}, \mathbf{c}) $ be the $ k\times (n-k) $ matrix with the following entries:
$ {\frac{c_{j+k}}{c_i[a_{j+k},a_i]}}, \text{for}\;1\leq i\leq k,1\leq j\leq n-k. $ |
The generalized Cauchy code $ C_k(\mathbf{a}, \mathbf{c}) $ is an $ [n, k, n-k+1] $ code defined by the generator matrix $ \left(I_k\left|\right.B(\mathbf{a}, \mathbf{c})\right). $
The following proposition shows that the GRS codes are also generalized Cauchy codes.
Proposition 2.3. [9, Proposition C.2] Let $ \mathbf{a} $ be an $ n $-tuple of mutually distinct elements of $ \overline{\mathbb{F}}, $ and let $ \mathbf{c} $ be an $ n $-tuple of non-zero elements of $ \mathbb{F}. $ Also, let
$ c_i = \left\{ bi∏kt=1,t≠i[ai,at],if1≤i≤k;bi∏kt=1[ai,at],ifk+1≤i≤n. \right. $
|
Then, $ GRS_{n, k}(\mathbf{a}, \mathbf{b}) = C_k(\mathbf{a}, \mathbf{c}). $
Let $ Gal(\mathbb{F}/\mathbb{Q}) $ be the Galois group of $ \mathbb{F} $ over $ \mathbb{Q} $ and $ P\Gamma L(2, \mathbb{F}) $ denote the group of semilinear fractional transformations given by
$ f:¯F⟶¯Fx⟼aγ(x)+bcγ(x)+d, $
|
where $ ad-bc\not = 0 $ and $ \gamma\in Gal(\mathbb{F}/\mathbb{Q}). $ Let $ S_n $ be the symmetric group on a set of $ n $ elements and $ Per(C) = \{\xi\in S_n\left|\right. \xi(C) = C\}, $ where $ n $ is the length of the code $ C. $ The set $ Per(C) $ is called the permutation group of the code $ C. $ We have the following theorem that is related to the permutation group of a Cauchy code.
Theorem 2.4. [1, Corollary 2] Let $ C = C_k(\mathbf{a}, \mathbf{y}) $ be a Cauchy code over $ \mathbb{F}, $ where $ 2\leq k\leq n-2 $ and $ \mathbf{a} = \left(a_1, \dots, a_n\right). $ Also, let $ L = \{a_1, \dots, a_n\}. $ Then, the map
$ ω:{f∈PΓL(2,F)|f(L)=L}⟶Per(C)f⟼σ, $
|
where $ a_{\sigma(i)} = f(a_i) $ for $ i = 1, \dots, n $ is a surjective group homomorphism.
A number field $ \mathbb{F} $ is a finite Galois extension of the rational field $ \mathbb{Q}. $ In this section, we describe a way to construct a number field $ \mathbb{F} $ with $ Gal(\mathbb{F}/\mathbb{Q})\cong \langle\sigma\rangle $ for $ \sigma\in S_n, $ where $ \langle\sigma\rangle $ is a cyclic subgroup generated by $ \sigma. $
Let $ \sigma = \sigma_1\sigma_2\cdots\sigma_t $ be a permutation in $ S_n, $ where $ \sigma_1, \sigma_2, \dots, \sigma_t $ are disjoint cycles. Also, let $ \langle\sigma\rangle $ be the cyclic group generated by $ \sigma. $ Let $ l(\sigma_j) $ be the length of the cycle $ \sigma_j, $ and define a set $ \mathcal{P} = \left\{p:p\; \text{prime and}\; \exists j\in\{1, \dots, t\}\ni p|l(\sigma_j)\right\}. $ Since $ \mathcal{P} $ is finite, assume that $ p_1 < p_2 < \cdots < p_{|\mathcal{P}|} $ are all elements in $ \mathcal{P}. $ For any $ j, $ we have
$ l(σj)=|P|∏i=1pαiji, $
|
(3.1) |
where $ \alpha_{ij}\in\mathbb{Z}_{\geq0}. $ Based on Eq (3.1), we have
$ ord(σ)=|⟨σ⟩|=|P|∏i=1pmaxj{αij}i, $
|
(3.2) |
where $ ord(\sigma) $ is the order of the permutation $ \sigma. $ Since $ \langle\sigma\rangle $ contains the element of order $ p_i^{\max_j\{\alpha_{ij}\}} $ for all $ i = 1, \dots, |\mathcal{P}|, $ by the structure theorem for finite Abelian groups, we have
$ ⟨σ⟩≅|P|∏i=1Zpmaxj{αij}iZ≅Z∏|P|i=1pmaxj{αij}iZ. $
|
(3.3) |
Let $ \zeta_p $ be the primitive $ p $-th root of unity and $ \mathbb{Q}(\zeta_p) $ be the corresponding cyclotomic extension of $ \mathbb{Q}. $ The following theorem shows a Galois extension of $ \mathbb{Q}, $ where its Galois group is isomorphic to $ \langle\sigma\rangle. $ The proof of the theorem is similar to the proof of [5, Theorem 3.1.11]. We write the proof here to give a sense of how to construct the related Galois extension.
Theorem 3.1. There exists a totally real Galois extension $ K $ of $ \mathbb{Q} $ such that $ Gal(K/\mathbb{Q})\cong \langle\sigma\rangle. $
Proof. By Eq (3.3), we have
$ {\langle\sigma\rangle\cong\prod\limits_{i = 1}^{|\mathcal{P}|}\frac{\mathbb{Z}}{p_i^{\max\limits_j\{\alpha_{ij}\}}\mathbb{Z}}\cong\frac{\mathbb{Z}}{\prod_{i = 1}^{|\mathcal{P}|}p_i^{\max\limits_j\{\alpha_{ij}\}}\mathbb{Z}}}. $ |
Now, choose a prime $ p $ such that
$ p\equiv 1 \mod 2\prod\limits_{i = 1}^{|\mathcal{P}|}p_i^{\max\limits_j\{\alpha_{ij}\}}. $ |
Let $ \zeta_p $ be the $ p $-th root of unity. By [5, Theorem C.0.3], $ \mathbb{Q}(\zeta_p) $ is a Galois extension of $ \mathbb{Q}, $ with its corresponding Galois group being isomorphic to $ G = \left(\mathbb{Z}/p\mathbb{Z}\right)^\times, $ where $ \left(\mathbb{Z}/p\mathbb{Z}\right)^\times $ is the multiplicative group of $ \mathbb{Z}/p\mathbb{Z}-\left\{\overline{0}\right\}. $ Since $ p $ is a prime number, $ G $ is a cyclic group. Moreover, we can find a unique subgroup $ H $ of $ G $ such that
$ |H| = {\frac{p-1}{\prod_{i = 1}^{|\mathcal{P}|}p_i^{\max_j\{\alpha_{ij}\}}}}. $ |
Let $ \mathbb{Q}(\zeta_p)^H $ be a subset of $ \mathbb{Q}(\zeta_p) $ which is invariant under the action of $ H. $ By the fundamental theorem of Galois theory ([15, Theorem 25]), $ \mathbb{Q}(\zeta_p)^H $ is also a Galois extension of $ \mathbb{Q}, $ with the corresponding Galois group isomorphic to $ G/H. $ Moreover, $ \left|G/H\right| = \prod_{i = 1}^{|\mathcal{P}|}p_i^{\max_j\{\alpha_{ij}\}}, $ and, as a consequence,
$ G/H\cong \frac{\mathbb{Z}}{\prod_{i = 1}^{|\mathcal{P}|}p_i^{\max_j\{\alpha_{ij}\}}\mathbb{Z}} \cong\langle\sigma\rangle. $ |
Also, by using a similar argument as in the proof of [5, Theorem 3.1.11], we have that $ \mathbb{Q}(\zeta_p)^H $ is a totally real Galois extension of $ \mathbb{Q}. $ The following algorithm provides a way to construct $ \mathbb{Q}(\zeta_p)^H $ in the proof of Theorem 3.1. The algorithm is based on Theorem 3.1 and [5, Proposition 3.3.2].
Algorithm 3.2. Suppose that $ \sigma\in S_n $ and $ G = Gal(\mathbb{Q}(\zeta_p)/\mathbb{Q})\cong \left(\mathbb{Z}/p\mathbb{Z}\right)^\times, $ where $ p $ is a prime number such that $ p\equiv 1\mod 2\cdot ord(\sigma). $
1) Choose $ H\subseteq G, $ where $ H $ is the subgroup of $ G $ with order $ {\frac{p-1}{ord(\sigma)}}. $
2) Calculate
$ \alpha = \sum\limits_{\lambda\in H}\lambda(\zeta_p). $ |
3) Find minimal polynomial $ m_\alpha(x) $ of $ \alpha $ over $ \mathbb{Q}. $
4) Construct the splitting field $ \mathbb{F} $ of $ m_\alpha(x) $ by using Algorithm A.1.
5) Then, $ \mathbb{F} = \mathbb{Q}(\zeta_p)^H. $
In this section, we describe a way to construct an invariant GRS code under a given permutation in $ S_n. $ We call this GRS code the GRS generalized quasi-cyclic (GQC) code. Let $ \sigma = \sigma_1, \sigma_2, \cdots, \sigma_t $ be a permutation in $ S_n, $ where $ \sigma_1, \sigma_2, \dots, \sigma_t $ are disjoint cycles. Also, let $ G = \langle\sigma\rangle $ be a cyclic group generated by $ \sigma. $
Theorem 4.1. If $ \sigma $ is a permutation in $ S_n, $ then there exists a GQC $ GRS_{n, k}(\overline{\alpha}, \mathbf{b}) $ over $ \mathbb{F}, $ with its corresponding permutation being $ \sigma $ for some totally real number field $ \mathbb{F}. $
Proof. We can find the number field $ \mathbb{F} $ and its corresponding minimal polynomial $ m_\alpha(x) $ with $ Gal(\mathbb{F}/\mathbb{Q})\cong \langle \sigma\rangle $ by using Algorithm 3.2. Since $ Gal(\mathbb{F}/\mathbb{Q})\cong \langle \sigma\rangle, $ there exists $ \gamma\in Gal(\mathbb{F}/\mathbb{Q}) $ to be associated with $ \sigma\in \langle\sigma\rangle. $ Let $ L = \{\alpha_1, \dots, \alpha_n\} $ be the roots of $ m_\alpha(x) $ and some additional elements from linear combinations of the roots. We can see that $ \gamma $ is a permutation on $ L, $ i.e., $ \gamma(L) = L. $ Note that the orbit of $ L $ under $ H $ can be used to rearrange the elements of $ L $ such that
$ γ(αi)=ασ(i), $
|
(4.1) |
for all $ i = 1, 2, \dots, n. $ Let $ \overline{\alpha} = (\alpha_1, \alpha_2, \dots, \alpha_n) $ and $ \mathbf{b} = (b_1, b_2, \dots, b_n) $ be an $ n $-tuple of non-zero elements in $ \mathbb{F}. $ Define a Cauchy code $ C_k(\overline{\alpha}, \mathbf{c}), $ where $ \mathbf{c} = (c_1, c_2, \dots, c_n), $ with
$ ci={bi∏kt=1,t≠i[αi,αt],if1≤i≤k;bi∏kt=1[αi,αt],ifk+1≤i≤n. $
|
(4.2) |
Then, by Proposition 2.3, $ C_k(\overline{\alpha}, \mathbf{c}) $ is a $ GRS_{n, k}(\overline{\alpha}, \mathbf{b}) $ code. Moreover, according to Theorem 2.4 and Eq (4.1), $ \omega(\gamma) = \sigma $ is an element in $ Per\left(C_k(\overline{\alpha}, \mathbf{c})\right). $
Consider the following example.
Example 4.2. Let $ \sigma = (1, 2, 3, 4)(5, 6) $ in $ S_6. $ We would like to construct a GRS code of length $ 6 $ over a totally real number field that is invariant under the action of $ \sigma. $ We can see that $ ord(\sigma) = 4 $ and $ \langle\sigma\rangle = \mathbb{Z}/4\mathbb{Z}. $ Choose $ p = 17 $ so that $ p\equiv 1\mod 2\times 4. $ The corresponding subgroup $ H $ of $ Gal(\mathbb{Q}(\zeta_{17})/\mathbb{Q}) $ will have the order equal to $ 4. $ Since the unique subgroup of $ \left(\mathbb{Z}/17\mathbb{Z}\right)^\times $ with order $ 4 $ is $ \{1, 4, 13, 16\}, $ we have
$ H = \{\lambda_k|k = 1,4,13,16\}, $ |
where $ \lambda_k:\zeta_{17}\mapsto \zeta_{17}^k. $ Then, we have
$ \alpha = \sum\limits_{\lambda\in H}\lambda(\zeta_{17}) = \zeta_{17}+\zeta_{17}^{4}+\zeta_{17}^{13}+\zeta_{17}^{16}. $ |
From [5, Example 3.3.3], the minimal polynomial of $ \alpha $ is as follows:
$ m_\alpha(x) = x^4+x^3-6x^2-x+1. $ |
The roots of $ m_\alpha(x) $ given by
$ r_1 = {\frac{1}{4}\left(-1-\sqrt{17}-\sqrt{34+\sqrt{17}}\right)},\quad r_2 = {\frac{1}{4}\left(-1-\sqrt{17}+\sqrt{34+\sqrt{17}}\right)}, $ |
$ r_3 = {\frac{1}{4}\left(-1+\sqrt{17}-\sqrt{34-\sqrt{17}}\right)}.,\quad r_4 = {\frac{1}{4}\left(-1+\sqrt{17}+\sqrt{34-\sqrt{17}}\right)}. $ |
Let $ \gamma $ be a map such that
$ r_1\mapsto r_2, \quad r_2\mapsto r_3,\quad r_3\mapsto r_4,\quad r_4\mapsto r_1. $ |
We can see that $ \langle \gamma\rangle = Gal(\mathbb{Q}(\zeta_{17})^H/\mathbb{Q})\cong \mathbb{Z}/4\mathbb{Z}. $
Choose $ L = \{\alpha_1, \dots, \alpha_6\}, $ where $ \alpha_i = r_i $ for $ i = 1, 2, 3, 4, \; \alpha_5 = r_1+r_3, $ and $ \alpha_6 = r_2+r_4. $ We can check that
$ \gamma(\alpha_i) = \alpha_{\sigma(i)}, $ |
for all $ i = 1, \dots, 6. $ Take $ \overline{\alpha} = (\alpha_1, \dots, \alpha_6) $, any $ n $-tuple of non-zero elements $ \mathbf{b} $ (from the set of linear combinations of roots of $ m_\alpha(x) $), and $ \mathbf{c} = (c_1, \dots, c_6), $ where $ c_i $ is as in Eq 4.2. We have that $ C_k(\overline{\alpha}, \mathbf{c}) $ is a GQC GRS code with corresponding permutation $ \sigma. $
In Section 4, we described the construction of GRS code, which is invariant under the action of a given permutation in $ S_n. $ Moreover, the alphabet for the corresponding codes is a totally real number field, not a complex number field. This feature can be useful for bandwidth reduction in exact gradient coding schemes.
Algorithm 1 describes the process of gradient coding. The algorithm is a slight modification of [11, Algorithm 1].
Algorithm 1 Gradient coding |
Input:
Data $ \mathcal{S} = \left\{z_i = (x_i, y_i)\right\}_{i = 1}^m, $ number of iterations $ t > 0, $ learning rate $ \{\eta\}_{r = 1}^t, $ straggler tolerance parameter $ \{s_r\}_{r = 1}^t, $ a matrix $ \mathbf{B}\in\mathbb{C}^{n\times n}, $ a function $ \Lambda:\mathcal{P}(n)\rightarrow \mathbb{C}^n, $ a vector of non-zero elements $ \overline{\beta} = (\beta_1, \dots, \beta_n)\in\mathbb{C}^n $ Initialize: $ \mathbf{w}^{(1)} \gets (0, 0, \dots, 0) $ Partition $ \mathcal{S} = \bigcup_{i = 1}^n\mathcal{S}_i $ and send $ \{\mathcal{S}_j|j\in supp(\mathbf{b}_i)\} $ to $ W_i $ for every $ i\in [n] $ for $ r = 1 $ to t do $ M $ broadcasts $ \mathbf{w}^{(r)} $ to all nodes Each $ W_j $ sends $ \sum_{i\in supp(\mathbf{b}_j)}b_{j, i}\frac{\nabla L_{\mathcal{S}_i}(\mathbf{w}^{(r)})}{\beta_i} $ to $ M $ $ M $ waits until at least $ n-s_r $ nodes have responded $ M $ computes $ \mathbf{v}_r = \Lambda\left(\mathcal{K}_r\right)\cdot\mathbf{C}, $ where the $ i $-th row of $ \mathbf{C} $ is $ \frac{1}{n} $ times the response from $ W_i $ if it has responded, and $ 0 $ otherwise; also, $ \mathcal{K}_r $ is the set of non-stragglers in the current iteration $ r $ $ M $ updates $ \mathbf{w}^{(r+1)}\gets \mathbf{w}^{(r)}-\eta_r\mathbf{v}_r $ end for return $ \frac{1}{t}\sum_{r = 1}^t\mathbf{w}^{(r+1)} $ |
Algorithm 1 works in the following way. In order to execute the gradient descent process, the master node $ M $ distributes a particular partition of the training set $ \mathcal{S} $ to all worker nodes $ W_j, $ where $ j = 1, \dots, n. $ In the $ r $-th iteration of the gradient descent process, the master $ M $ broadcasts the parameter $ \mathbf{w}^{(r)} $ to all worker nodes. Using the received parameter $ \mathbf{w}^{(r)}, $ the worker node $ W_j $ calculates the partial gradient $ \nabla L_{\mathcal{S}_i}(\mathbf{w}^{(r)}) $ and sends its linear combination $ \sum_{i\in supp(\mathbf{b}_j)}b_{j, i}\frac{\nabla L_{\mathcal{S}_i}(\mathbf{w}^{(r)})}{\beta_i} $ to $ M. $ The linear combination is chosen from the entries $ b_{j, i} $ of a particular matrix $ \mathbf{B}. $ In this work, $ \mathbf{B} $ is constructed by using GRS codes which are invariant under the action of a particular permutation. After $ M $ has received the linear combinations of partial gradients from some number of worker nodes, $ M $ updates the parameter $ \mathbf{w} $ by using the decoding vector $ \Lambda\left(\mathcal{K}_r\right), \; \mathbf{w}^{(r)}, $ and some other additional vectors (mentioned in the algorithm). Note that we will see later that the decoding vector $ \Lambda\left(\mathcal{K}_r\right) $ can be computed by using Algorithm 2[11, Algorithm 2].
Definition 5.1. A matrix $ \mathbf{B}\in\mathbb{C}^{n\times n} $ and a function $ \Lambda:\mathcal{P}(n)\rightarrow \mathbb{C}^n $ satisfy the exact computation (EC) condition with respect to $ \overline{\beta}\in\mathbb{C}^n, $ where $ \overline{\beta} $ is an $ n $-tuple of non-zero elements in $ \mathbb{C}^n $ if, for all $ \mathcal{K}\subseteq [n] $ such that $ |\mathcal{K}|\geq \max_{r\in [t]}s_r, $ we have that $ \Lambda(\mathcal{K})\cdot \mathbf{B} = \overline{\beta}. $
Note that Definition 5.1 is a slight modification of [11, Definition 2]. Let $ \overline{\beta} = (\beta_1, \dots, \beta_n) $ be an $ n $-tuple of non-zero elements of $ \mathbb{C}^n $ and
$ \mathbf{N}_{\overline{\beta}}(\mathbf{w}) = \frac{1}{n}\left( ∇LS1(w)β1∇LS2(w)β2⋮∇LSn(w)βn \right). $
|
Lemma 5.2. If $ \Lambda $ and $ \mathbf{B} $ satisfy the EC condition with respect to $ \overline{\beta}, $ then, for all $ r\in [t], $ we have that $ \mathbf{v}_r = \nabla L_{\mathcal{S}}(\mathbf{w}^{(r)}). $
Proof. Given $ r\in [t], $ let $ \mathbf{B}' $ be the matrix whose $ i $-th row $ \mathbf{b}_i' $ equals to $ \mathbf{b}_i $ if $ i\in\mathcal{K}_r, $ and $ \mathbf{0} $ otherwise. The matrix $ \mathbf{C} $ in Algorithm 1 can be written as $ \mathbf{C} = \mathbf{B}'\cdot \mathbf{N}_{\overline{\beta}}(\mathbf{w}^{(r)}). $ Since $ supp\left(\Lambda\left(\mathcal{K}_r\right)\right)\subseteq \mathcal{K}_r, $ we have that $ \Lambda\left(\mathcal{K}_r\right)\cdot \mathbf{B}' = \Lambda\left(\mathcal{K}_r\right)\cdot \mathbf{B}. $ Therefore, we have
$ vr=Λ(Kr)⋅C=Λ(Kr)⋅B⋅Nβ(w(r))=β⋅Nβ(w(r))=1n∑ni=1∇LSi(w(r))=1n∑ni=11m/n∑z∈Si∇l(w(r),z)=1m∑z∈S∇l(w(r),z)=∇LS(w(r)). $
|
For a given $ n $ and $ s, $ let $ C = GRS_{n, n-s}(\overline{\alpha}, \overline{\beta}) $ GQC code over a number field $ \mathbf{F} $ with corresponding permutation $ \pi $ of order $ n. $ Clearly, the vector $ \overline{\beta} $ is in $ C. $ Moreover, by [11, Lemma 8], there exists a codeword $ \mathbf{c}_1 $ in $ C $ whose support is $ \{1, 2, \dots, s+1\}. $ Let $ \mathbf{c}_i = \pi^{i-1}(\mathbf{c}_1) $ for $ i = 2, \dots, n $ and $ \mathbf{B} = \left(\mathbf{c}_1^T, \mathbf{c}_2^T, \dots, \mathbf{c}_n^T\right). $
Theorem 5.3. The matrix $ \mathbf{B} $ satisfies the following properties:
a) Each row of $ \mathbf{B} $ is a codeword in $ \sigma(C), $ where $ \sigma $ is a permutation such that
$ σ−1=(123⋯i⋯nnπn−1(n)πn−2(n)⋯πn−(i−1)(n)⋯π(n)). $
|
(5.1) |
b) $ w_H(\mathbf{b}) = s+1 $ for each row $ \mathbf{b} $ in $ \mathbf{B}. $
c) The column span of $ \mathbf{B} $ is the code $ C. $
d) Every set of $ n-s $ rows of $ \mathbf{B} $ are linearly independent over $ \mathbb{F}. $
Proof. (a) Let $ \mathbf{c}_1 = (c_1, \dots, c_n). $ Notice that the $ i $-th row of $ \mathbf{B} $ is as follows:
$ \left(c_i,c_{\pi^{n-1}(i)},c_{\pi^{n-2}(i)},\dots,c_{\pi(i)}\right). $ |
Since $ ord(\pi) = n, $ the $ i $-th row of $ \mathbf{B} $ is a permutation of $ \mathbf{c}_1 $ for all $ i = 1, \dots, n. $ Moreover, by considering the last row of $ \mathbf{B}, $ we can see that all rows of $ \mathbf{B} $ constitute a codeword in $ \sigma(C), $ where $ \sigma $ is the permutation as in Eq (5.1).
(b) By part (a), we have that the Hamming weight of every row of $ \mathbf{B} $ is $ s+1.
(c) Let $ \sigma = \left(1, 2, \dots, n\right) $ be a cyclic permutation and $ G_1 $ be a cyclic group generated by $ \sigma. $ Also, let $ G_2 $ be a cyclic group generated by $ \pi. $ Define $ \overline{S}_1 = span(G_1\mathbf{c}_1) $ and $ \overline{S}_2 = span(G_2\mathbf{c}_1), $ where $ G\mathbf{c}_1 = \{\lambda(\mathbf{c}_1)|\lambda\in G\}. $ Since $ ord(\sigma) = ord(\pi) = n, $ we have that $ G_1\cong G_2 $ by the following group isomorphism:
$ τ:G1→G2σi↦πi. $
|
Define the following map:
$ ¯τ:¯S1→¯S2∑ni=1αiσi(c1)↦∑ni=1αiπi(c1). $
|
The map $ \overline{\tau} $ is a linear map. Since it is induced by $ \tau, \; \overline{\tau} $ is a bijective map. So, $ \overline{S}_1\cong \overline{S}_2. $ By [11, Lemma 12 B3], $ \overline{S}_1 = C. $ Since $ \overline{S}_2\subseteq C $ and $ dim\; \overline{S}_2 = n-s, $ we have that $ \overline{S}_2 = C. $
(d) Similar to [11, Lemma 12 B4].
Let $ \mathbf{G} $ be the canonical generator for the $ C = GRS_{n, n-s}(\overline{\alpha}, \overline{\beta}) $ GQC code, as in Eq (2.1). By Theorem 2.1(b), the canonical generator for the dual code $ C^\perp $ is $ \mathbf{G}^\perp = \mathbf{G}\cdot \mathbf{D}, $ where $ \mathbf{D} = diag(u_1, \dots, u_n), $ with
$ {u_i = \frac{1}{\beta_i^2\prod_{j\not = i}\left(\alpha_i-\alpha_j\right)}} $ |
for all $ i = 1, \dots, n. $ Using this setting, Algorithm 2[11, Algorithm 2] can be used to compute the decoding vector $ a\left(\mathcal{K}\right). $
The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.
This research was funded by Hibah PPMI KK Aljabar Institut Teknologi Bandung 2023.
The authors declare no conflict of interest.
The following algorithm provides a way to construct the unique splitting field of a given polynomial $ f(x) $ in $ \mathbb{Q}[x]. $
Algorithm A.1. Given a polynomial $ f(x) $ in $ \mathbb{Q}[x], $ we will construct the splitting field $ L $ of $ f(x) $ based on the construction of a chain of number fields:
$ K_0 = \mathbb{Q}\subset K_1\subset K_2\subset\cdots\subset K_{s-1}\subset K_s = L $ |
such that $ K_i $ is an extension of $ K_{i-1} $ containing a new root of $ f(x). $
$ 1) $ Factorize $ f(x) $ over $ K_i $ into irreducible factors $ f_1(x)f_2(x)\cdots f_t(x). $
$ 2) $ Choose any non linear irreducible factor $ g(x) = f_j(x) $ for some $ j\in\{1, \dots, t\}. $
$ 3) $ Construct the field extension $ K_{i+1} = \frac{K_i[x]}{\langle g(x)\rangle}. $
$ 4) $ Repeat the process for $ K_{i+1} $ until $ f(x) $ completely factors.
The following algorithm can be used to compute the decoding vector in the exact gradient coding scheme [11, Algorithm 2].
Algorithm 2 Computing decoding vector $ \Lambda\left(\mathcal{K}\right) $ |
Data: any vector $ \mathbf{x}'\in\mathbb{C}^n $ such that $ \mathbf{x}'\mathbf{B} = \mathbf{\beta} $
Input: A set $ \mathcal{K}\subseteq [n] $ of $ n-s $ non-stragglers Output: a vector $ \Lambda\left(\mathcal{K}\right) $ such that $ supp\left(\Lambda\left(\mathcal{K}\right)\right)\subseteq \mathcal{K} $ and $ \Lambda\left(\mathcal{K}\right)\mathbf{B} = \mathbf{\beta} $ find $ \mathbf{f}\in\mathbb{C}^s $ such that $ \mathbf{f}\mathbf{G}_{\mathcal{K}^c} = -\mathbf{x}'_{\mathcal{K}^c}\mathbf{D}_{\mathcal{K}^c}^{-1} $ $ \mathbf{y}\gets \mathbf{f}\mathbf{G}\mathbf{D} $ return $ \Lambda\left(\mathcal{K}\right)\gets \mathbf{y}+\mathbf{x}' $ |
[1] |
S. Amin, A. Cardenas and S. Sastry, Safe and secure networked control systems under denial-of-service attacks, in "Proceedings of the 12th International Conference on Hybrid Systems: Computation and Control," Springer-Verlag, (2009), 31-45. doi: 10.1007/978-3-642-00602-9_3
![]() |
[2] |
S. Amin, X. Litrico, S. Sastry and A. Bayen, Stealthy deception attacks on water scada systems, In "Proceedings of the 13th ACM International Conference on Hybrid Systems: Computation and Control," ACM, (2010),161-170. doi: 10.1145/1755952.1755976
![]() |
[3] | J.-P. Aubin, "Viability Theory," Systems and Control: Foundations and Applications, Birkhäuser, Boston, MA, 1991. |
[4] |
J.-P. Aubin, A. M. Bayen and P. Saint-Pierre, Dirichlet problems for some Hamilton-Jacobi equations with inequality constraints, SIAM Journal on Control and Optimization, 47 (2008), 2348-2380. doi: 10.1137/060659569
![]() |
[5] |
M. Bardi and I. Capuzzo-Dolcetta, "Optimal Control and Viscosity Solutions of {Hamilton-Jacobi-Bellman} Equations," Birkhäuser, Boston, MA, 1997. doi: 10.1007/978-0-8176-4755-1
![]() |
[6] |
E. N. Barron and R. Jensen, Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians, Communications in Partial Differential Equations, 15 (1990), 1713-1742. doi: 10.1080/03605309908820745
![]() |
[7] | E. S. Canepa and C. G. Claudel, Exact solutions to traffic density estimation problems involving the Lighthill-Whitman-Richards traffic flow model using Mixed Integer Linear Programing, In "Proceedings of the 15th International IEEE Conference on Intelligent Transportation Systems", (2012), 832-839. |
[8] |
P. D. Christofides, "Nonlinear and Robust Control of Partial Differential Equation Systems: Methods and Applications to Transport-Reaction Processes," Birkhäuser, Boston, MA, 2001. doi: 10.1007/978-1-4612-0185-4
![]() |
[9] |
C. G. Claudel and A. M. Bayen, Lax-Hopf based incorporation of internal boundary conditions into Hamilton-Jacobi equation. Part I: Theory, IEEE Transactions on Automatic Control, 55 (2010), 1142-1157. doi: 10.1109/TAC.2010.2041976
![]() |
[10] |
C. G. Claudel and A. M. Bayen, Lax-Hopf based incorporation of internal boundary conditions into Hamilton-Jacobi equation. Part {II: Computational methods}, IEEE Transactions on Automatic Control, 55 (2010), 1158-1174. doi: 10.1109/TAC.2010.2045439
![]() |
[11] |
C. G. Claudel and A. M Bayen, Convex formulations of data assimilation problems for a class of Hamilton-Jacobi equations, SIAM Journal on Control and Optimization, 49 (2011), 383-402. doi: 10.1137/090778754
![]() |
[12] |
M. G. Crandall and P.-L. Lions, Viscosity solutions of {Hamilton-Jacobi equations}, Transactions of the American Mathematical Society, 277 (1983), 1-42. doi: 10.1090/S0002-9947-1983-0690039-8
![]() |
[13] |
C. Daganzo, The cell transmission model: a dynamic representation of highway traffic consistent with the hydrodynamic theory, Transportation Research, 28B (1994), 269-287. doi: 10.1016/0191-2615(94)90002-7
![]() |
[14] |
C. F. Daganzo, A variational formulation of kinematic waves: basic theory and complex boundary conditions, Transportation Research B, 39B (2005), 187-196. doi: 10.1016/j.trb.2004.04.003
![]() |
[15] |
C. F. Daganzo, On the variational theory of traffic flow: well-posedness, duality and applications, Networks and Heterogeneous Media, 1 (2006), 601-619. doi: 10.3934/nhm.2006.1.601
![]() |
[16] |
H. Frankowska, Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations, SIAM Journal of Control and Optimization, 31 (1993), 257-272. doi: 10.1137/0331016
![]() |
[17] |
J. C. Herrera, D. B. Work, R. Herring, X. J. Ban, Q. Jacobson and A. M. Bayen, Evaluation of traffic data obtained via GPS-enabled mobile phones: The Mobile Century field experiment, Transportation Research Part C: Emerging Technologies, 18 (2010), 568-583. doi: 10.1016/j.trc.2009.10.006
![]() |
[18] |
B. Hoh, M. Gruteser, R. Herring, J. Ban, D. Work, J. C. Herrera, A. M. Bayen, M. Annavaram and Q. Jacobson, Virtual trip lines for distributed privacy-preserving traffic monitoring, in "Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services," ACM, (2008), 15-28. doi: 10.1145/1378600.1378604
![]() |
[19] |
M. Krstic and A. Smyshlyaev, Backstepping boundary control for first-order hyperbolic pdes and application to systems with actuator and sensor delays, Systems & Control Letters, 57 (2008), 750-758. doi: 10.1016/j.sysconle.2008.02.005
![]() |
[20] |
P. E. Mazare, A. Dehwah, C. G. Claudel and A. M. Bayen, Analytical and grid-free solutions to the lighthill-whitham-richards traffic flow model, Transportation Research Part B: Methodological, 45 (2011), 1727-1748. doi: 10.1016/j.trb.2011.07.004
![]() |
[21] | K. Moskowitz, Discussion of "freeway level of service as influenced by volume and capacity characteristics' by D.R. Drew and C. J. Keese, Highway Research Record, 99 (1965), 43-44. |
[22] | G. F. Newell, A simplified theory of kinematic waves in highway traffic, Part (I), (II) and (III). Transporation Research B, 27B (1993), 281-313. |
[23] |
R. C. Smith and M. A. Demetriou, "Research Directions in Distributed Parameter Systems," SIAM, Philadelphia, PA, 2000. doi: 10.1137/1.9780898717525
![]() |
[24] |
I. S. Strub and A. M. Bayen, Weak formulation of boundary conditions for scalar conservation laws, International Journal of Robust and Nonlinear Control, 16 (2006), 733-748. doi: 10.1002/rnc.1099
![]() |
[25] | D. Work, S. Blandin, O. Tossavainen, B. Piccoli and A. Bayen, A distributed highway velocity model for traffic state reconstruction, Applied Research Mathematics eXpress (ARMX), 1 (2010), 1-35. |
[26] | http://traffic.berkeley.edu/. |
[27] | http://pems.dot.ca.gov. |