DyCARS: A dynamic context-aware recommendation system

1.   Introduction

With the rapid development of the Internet and big data technology, recommendation systems have become an integral part of technical tools and application solutions in multiple industries and fields, particularly in e-commerce ^[1], social media ^[2], and online education ^[3]. These systems demonstrate significant application value in these areas. Moreover, they also play an important role in public safety warning ^[4,5,6], environmental monitoring ^[7], and other domains of social welfare. Traditional recommendation systems are mostly built on static models, which means they typically process historical interaction data using one-time or batch updates, making it difficult to capture the dynamic changes in user interests and item attributes ^{[8,9,10,11,12]}. In contrast, dynamic recommendation systems, as a subset within the context-aware recommendation system domain ^[13,14], focus on considering time-related contextual information, such as the interaction history of users and items. They employ more complex models, like Recurrent Neural Networks (RNNs), to capture the dynamic interaction patterns between users and items ^[15,16]. Due to their ability to accurately reflect and adapt to the dynamic changes in users and items, dynamic recommendation systems are emerging as a cutting-edge direction in the research and application of recommendation algorithms ^[17,18].

In this paper, the two types of entities, users and items, can cover a variety of notions, e.g., users could be customers in an e-commerce system, or accounts on social media; items could be products, posts, or media produced or consumed by users. To more comprehensively learn the dynamic changes in user interests and item attributes, and thereby provide precise personalized recommendations, we propose a novel dynamic recommendation approach. This method captures the dynamic relationships between users and items by mining the most relevant delayed interaction patterns within the dynamic interaction history, aiming to predict the items with which users are likely to interact in the future ^[6,18]. Dynamic graphs naturally handle graph data structures with timestamps, where each edge represents a user-item interaction at a specific time point ^[19]. Therefore, we construct our model based on dynamic graphs, enabling it to effectively capture the evolving interaction patterns between users and items over time.

Currently, several methods have been proposed to predict which items users are likely to interact with in the future ^{[16,20,21,22,23,24,25]}. However, these methods often suffer from a common limitation: they typically generate corresponding embeddings only when users perform specific actions, thereby overlooking the rich information contained in the historical interaction sequences ^[8]. Therefore, RNNs and other models optimized for sequential data ^{[15,26,27,28]} have been employed to model the historical interaction sequences and capture the long-term dependency relationships among items in the sequence. Recent studies have demonstrated significant improvements over traditional methods by jointly modeling users and items using dynamic graphs ^[16,21]. One key challenge lies in accurately capturing the evolution of user interests and item attributes over time. This is because users may interact with different items in a sequential manner, and their interests can change within a certain time frame. Similarly, item attributes also undergo continuous changes and are heavily influenced by user behavior. In contrast to previous research, JODIE ^[16], Dynamic Graph Collaborative Filtering (DGCF) ^[17], and several other dynamic evolution models ^[29,30,31] employ mutually recursive RNNs to model the dynamic interaction between users and items. However, these methods heavily rely on the embeddings at time t-1 to recursively compute the dynamic embeddings at time t. As the network scale increases, this recursive training approach has been shown to be a performance bottleneck. Although recent studies have attempted to alleviate this burden by introducing a batch processing algorithm called t-Batch, the computational overhead has not been fundamentally reduced ^[16]. In contrast, DeePRed proposes a simple and efficient approach to model short-term interaction behavior, effectively eliminating the aforementioned recursive dependency issue and achieving a more efficient training process ^[18].

Although there is great potential in constructing dynamic embeddings by capturing the dynamic interaction relationships between users and items for dynamic recommendation, there is still room for further improvement in modeling the long-term dependency structures of dynamic interaction histories and extracting the most relevant delayed interaction patterns. In dynamic recommendation systems, delayed interaction patterns refer to the interactions between users and items that occurred during a certain period in the past. Although these interactions happened at earlier points in time, they still significantly influence the user's current interests and preferences. The key to mining delayed interaction patterns lies in identifying and learning those historical interactions that continue to have a significant impact on current recommendation decisions. For instance, a user might have frequently watched a certain genre of movies or purchased specific types of products over a past period. Although such behavior may have decreased recently, these long-term interests and preferences still influence their current choices. Therefore, effectively mining and utilizing delayed interaction patterns can help recommendation systems to understand the evolution of user interests more comprehensively, thereby providing more accurate and personalized recommendations. To this end, we propose a Dynamic Context-Aware Recommendation System (DyCARS) for dynamic recommendation. By treating the dynamic interaction history of users and items as the dynamic context, this method utilizes a Gated Multi-Layer Perceptron (GaMLP) encoder to learn the long-term dependency structure of user-item interaction sequences. Moreover, it employs an Attention Pooling (AP) network to extract the most relevant delayed interaction patterns from the dynamic interaction history. In summary, our main contributions are as follows:

1) We propose a novel non-recursive dynamic recommendation model, DyCARS, which effectively captures the long-term dependency structure in the dynamic interaction history using the GaMLP encoder introduced in this paper and further extracts high-level features. Additionally, the AP network is introduced to learn similarity scores between high-level features of user and item dynamic interaction histories. By computing bidirectional attention weights, the model is able to identify the most relevant delayed interaction patterns from the historical interaction sequence. As a result, the model can effectively predict the dynamic embeddings of users and items.

2) We emphasize the importance of the loss function in dynamic recommendation and accordingly propose the Pairwise Cosine Similarity (D-PCS) loss function to jointly optimize the static and dynamic embeddings of two types of nodes. In the ${L}_{PCS}$ term of the D-PCS loss function, we utilize cosine similarity to calculate the similarity between user-item pairs and allow the model to emphasize the learning of hard negative samples, which aids in learning more discriminative features and enhances the model's ability to extract useful information.

3) Extensive experimental analyses were conducted on two public datasets, LastFM and the Global Terrorism Database (GTD). Compared to the state-of-the-art baseline models, the experimental results confirm that our proposed DyCARS achieves the best performance.

The remaining parts of this paper are organized as follows: Section 2 formally describes the dynamic recommendation problem addressed in this paper. Section 3 provides a brief overview of the related research in the field of dynamic recommendation systems. Section 4 elaborates on the overall architecture and individual components of the proposed model. In Section 5, experiments are conducted on two real-world datasets to validate the effectiveness of the proposed model. Finally, Section 6 concludes the paper and outlines future directions.

2.   Problem definition

The purpose of this study is to learn representations of users and items based on their current interaction behavior and historical records, and use these representations to predict the items that users may be interested in the future. This "interest" is the relationship between the user and the recommended item. Dynamic graph structures provide a natural and effective way to depict the evolving dynamic interaction relationship between users and items over time ^[17,32]. Therefore, we utilize dynamic graph structures to model the problem of dynamic recommendation.

Definition 1: Dynamic graph. Essentially, the dynamic graph in this context is a bipartite graph, where users and items are considered as nodes, and all interaction behaviors occur between user nodes and item nodes. If at time $t\in {\mathbb{R}}^{+}, \forall t\in \left[0, T\right]$, an interaction occurs between a user and an item, a dynamic graph ${G}_{t} = (U\cup V, {E}_{t})$ is formed. Here, U and V represent the sets of users and items, respectively, and $\left|U\right|$ and $\left|V\right|$ denote the number of nodes in the user and item sets. $U\cup V$ represents a finite set containing all user and item nodes, satisfying $U\cap V = \mathrm{\varnothing }$. ${E}_{t}$ represents a finite set of all interactions in ${G}_{t}$, which essentially refers to the collection of all interactions between users and items up until time t. At the initial time ${t}_{0}$, the dynamic graph ${G}_{{t}_{0}} = (U\cup V, {E}_{{t}_{0}})$ consists of isolated nodes or snapshots of the dynamic graph, where the initial embeddings of users and items are initialized as either initial feature vectors or random vectors.

Definition 2: User-item interaction events. In the context of dynamic recommendation, the interaction between a user and an item can be represented as a triplet $\text{i} = \left(u, v, t\right)$, where at a specific time t, the user node $u\in U$ interacts with the item node $v\in V$, denoted as $\left(u, v\right)\in {E}_{t}$. From this perspective, a dynamic graph can be understood as a sequential arrangement of user-item interaction events $I = \left\{{\left(u, v, t\right)}_{n}:n = \mathrm{1, 2}, \cdots, \left|{E}_{T}\right|\right\}$. Let ${I}_{u}$ denote the set of all interaction events for user u, which can be expressed as ${I}_{u} = \{{i}_{1}^{u}, {i}_{2}^{u}, \cdots, {i}_{N}^{u}\}$. Similarly, ${I}_{v}$ represents the set of all interaction events for item v, written as ${I}_{v} = \{{i}_{1}^{v}, {i}_{2}^{v}, \cdots, {i}_{M}^{v}\}$. For a specific user u and item v, ${I}_{u}$ and ${I}_{v}$ respectively capture their complete interaction history.

Definition 3: Static and dynamic embeddings. When predicting future interactions between users and items, long-term and short-term interaction patterns are often considered ^[21,33,34]. We adopt this assumption and model the users and items from both long-term and short-term perspectives. From the user's point of view, although it is commonly believed that user interests evolve over time ^[15], we assume the existence of a stable long-term interest that remains unchanged. Additionally, the user's interest may also exhibit a corresponding short-term trend based on recent behavior changes. For example, in music recommendation systems, users may have a tendency towards a specific genre of music at any given moment. However, individuals may also be influenced by emotions or life events, leading to an interest in different genres during specific time periods. To encode the long-term static attributes and short-term dynamic attributes of users and items, we allocate static and dynamic embeddings to all users and items.

Static embeddings. We define the static embeddings of user u and item v at time t as $\stackrel{-}{u}\in {\mathbb{R}}^{d}$ and $\stackrel{-}{v}\in {\mathbb{R}}^{d}$, respectively. These embeddings do not change over time. In other words, we use an embedding matrix $E\in {\mathbb{R}}^{\mathrm{d}\times (\left|U\right|+\left|V\right|)}$ as the static embedding matrix, which is trained when observing user-item interactions.

Dynamic embeddings. We assign dynamic embeddings to each user u and item v at time t, represented as $u\left(t\right)\left|{I}_{u}({t}^{ < }, n)\right.\in {\mathbb{R}}^{k}$ and $v\left(t\right)\left|{I}_{v}({t}^{ < }, n)\right.\in {\mathbb{R}}^{k}$, respectively. These embeddings effectively capture the evolving user interests and item attributes over time. ${I}_{u}({t}^{ < }, n)$ and ${I}_{v}({t}^{ < }, n)$ denote the n most recent interaction events for each user u and item v before time t:

where ${\Delta }_{i} = t-{t}_{i}$ represents the time interval between the current moment t and the previous interaction time ${t}_{i}$. It effectively captures the temporal decay effect of recent interaction events on the current interaction event.

We adopt the approach consistent with previous studies ^[33,35] to capture the short-term dynamic characteristics of users or items by incorporating the n most recent interaction events. However, unlike the research in ^{[20,24,26,36]}, we consider that the n recent interaction events of both users and items have an impact on the next user-item interaction. In previous studies, the dynamic embeddings of users and items often recursively depend on their respective previous dynamic embeddings ^[16,21]. The recursive nature of these algorithms increases computational costs, necessitating the use of specialized batch processing algorithms to avoid sequential processing. Table 1 summarizes the mathematical notation used in this paper.

Research Problem. The main problem addressed in this study is: Given a known ordered sequence of user-item interaction events I, can we design an algorithm to effectively predict future interaction behavior in the dynamic graph?

3.   Related work

The main task of dynamic recommendation models is to capture the dynamic changes of users and items from historical and current interactions, so as to accurately predict the embedding trajectory of users or items over time to recommend items that users may interact with in the future. In the field of dynamic recommendation research, conventional methods typically employ standard deep neural network (DNN) architectures that integrate manually designed features with learned features into the recommendation model. For instance, Covington et al. ^[20] divided YouTube's recommendation system into two separate stages: candidate generation and ranking. Both stages utilized structurally similar deep neural networks, resulting in significant improvements in recommendation performance.

To accurately capture the repetitive patterns in user-item interaction sequences, Recurrent Neural Networks (RNNs) and their variants have been widely applied in dynamic recommendation. Specifically, Hidasi et al. ^[26] were the first to apply RNNs to session-based recommendation, particularly in scenarios that rely on short-term session data for recommendations. Similarly, Wu et al. ^[15] employed Long Short-Term Memory (LSTM) networks to model the dynamic changes in users and movies, thereby predicting future behavioral trajectories. Beutel et al. ^[33] effectively incorporated various types of contextual data into a Gated Recurrent Unit (GRU)-based recommendation system by performing element-wise multiplication on the model's hidden states.

Activities closer in time to a specific event are often more likely to trigger that event. Therefore, modeling the time differences in historical behavior sequences has shown significant advantages in improving recommendation system performance. Although standard RNNs and their variants can capture repetitive patterns in sequences, they lack the ability to encode the time intervals between historical events. To address this limitation, extensive research has been conducted to enhance RNNs and their variants by fully considering the factor of time intervals ^[34,35]. For instance, Zhu et al. ^[34] designed dedicated temporal gates based on LSTM, enabling the model to more effectively capture the time interval factors in user behavior sequences.

Although the aforementioned methods have successfully simulated the evolution of user interests through user-item interaction sequences, they fall short in fully consider the changing trends of items themselves. To alleviate this limitation, several studies have attempted to combine point process models with RNNs to jointly learn dynamic embeddings of users and items for next-item prediction. Specifically, Dai et al. ^[21] employed RNNs to define the intensity function of the point process, enabling the capture of complex interdependencies and the temporal evolution of features between users and items. Kumar et al. ^[16] proposed a coupled recursive neural network recommendation model that included two steps of embedding update operations and an embedding projection function to predict the dynamic embeddings of users and items. Li et al. ^[17] first applied dynamic graphs to dynamic recommendation by integrating second-order neighborhood information from dynamic interaction networks to extend JODIE. Hou et al. ^[6] simultaneously introduced Hawkes processes and dynamic graphs into dynamic recommendation scenarios to model the dynamic interactions and evolution between users and items.

However, these methods primarily rely on recursively calculating the dynamic embeddings at time t based on the embeddings at time t-1. As the network scale increases, this recursive training approach has been proven to be a performance bottleneck. Although recent research attempts to alleviate this issue by introducing a batch processing algorithm called t-Batch, the computational cost has not been fundamentally reduced. To address this problem, Kefato et al. ^[18] successfully avoid the recursion and high computational costs between consecutive dynamic embeddings by using static embeddings as proxies when modeling dynamic embeddings. Although many existing methods have achieved success in modeling the dynamic interaction between users and items, they still perform poorly in capturing long-term dependency structures in the historical interaction sequences between users and items. It is challenging to accurately mine and model the most relevant delayed interaction patterns from the dynamic interaction history. To address this limitation, we propose a novel non-recurrent dynamic recommendation model called DyCARS. This model considers the dynamic interaction history between users and items as a dynamic context and effectively addresses the aforementioned shortcomings by utilizing the GaMLP encoder and the AP network. Table 2 summarizes the comparison of DyCARS with other dynamic recommendation system methods.

4.   Research method

In this section, we will detail the design and implementation of DyCARS. Initially, an overview of the model architecture is presented, elucidating the overall framework of DyCARS and its core components. Subsequently, we delve into a key component of DyCARS: the GaMLP encoder. This part will explain in detail the working mechanism of the GaMLP encoder, including its handling of the embedding matrix of recent interaction history and the design and functionality of the spatial gating module. These components collectively capture the long-term dependencies in the dynamic interaction history and extract high-level features. Following this, the section introduces the design and application of the AP network. The AP network plays a pivotal role in DyCARS, learning the similarities and bidirectional attention weights among high-level features in the dynamic interaction history of users and items. It identifies the most relevant delayed interaction patterns from the historical interaction sequences, further enhancing the model's recommendation capabilities. Lastly, we discuss the D-PCS loss function and model optimization strategies. This part will demonstrate how the specially designed loss function effectively co-optimizes static and dynamic embeddings, achieving more efficient and accurate recommendations.

4.1.   Model architecture

Figure 1 illustrates the overall architecture of DyCARS. The interactions between users and items form a dynamic graph over time. The input of DyCARS is determined by the observed user-item interaction events and the hyper-parameter n of the model. Whenever a new user-item interaction event is observed, the model parameters are updated accordingly. DyCARS is built on dynamic graphs and considers the n most recent interaction events of users and items (i.e., ${I}_{u}\left({t}^{ < }, n\right)$ and ${I}_{v}\left({t}^{ < }, n\right)$) prior to time t as dynamic contexts. These contexts are captured through static embeddings (${\stackrel{-}{u}}_{i}$ or ${\stackrel{-}{v}}_{i}, i = 1, \cdots n$). Furthermore, we employ the proposed GaMLP encoder to model long-term dependency structures in the n recent interaction histories and extract high-level features. Subsequently, an AP network is utilized to compute similarity scores between the high-level features of user-item dynamic interaction histories. By calculating bidirectional attention weights, we are able to discover the most relevant delayed interaction patterns from the historical sequences, thus predicting the dynamic embeddings ($u\left(t\right)$ and $v\left(t\right)$) of users and items at time t. Finally, we jointly optimize the static and dynamic embeddings of users and items using the D-PCS loss function to make more accurate predictions for the next item recommendation.

4.2.   The GaMLP encoder

To model the long-term dependency structure in the user-item dynamic interaction history, we propose the GaMLP encoder. In DyCARS, the user encoder and item encoder share the same architecture. Therefore, we primarily focus on the user side to illustrate the structural design of the GaMLP encoder.

As illustrated in Figure 2, the GaMLP encoder consists of L modules with the same size and structure. Let ${X}_{u}\left(t\right)$ denote the embedding matrix of the recent n interaction histories for user u at time t (see Section 4.2.1 for details). Each module is defined as follows:

where $\sigma$ is the activation function, e.g., Gaussian error linear unit (GeLU) ^[37]. W and V define linear projections along the channel dimension. For brevity, the above equations omit residual connections, normalization operations, and bias terms.

One key function in Eq (3) is $\mathrm{S}\mathrm{G}\left( \cdot \right)$, which aims to capture the mutual interactions between dynamic interaction histories (see Section 4.2.2 for details). When $\mathrm{S}\mathrm{G}\left( \cdot \right)$ behaves as an identity mapping, the $\mathrm{S}\mathrm{G}\left( \cdot \right)$ transformation degenerates into a regular Feedforward Neural Network (FFN), where individual interaction history embeddings are processed independently without any cross interaction history embeddings communication. Therefore, one of our focuses is to design an $\mathrm{S}\mathrm{G}\left( \cdot \right)$ that can effectively capture complex interactions between dynamic interaction histories. The overall architectural inspiration comes from inverted bottlenecks ^[38], where $\mathrm{S}\mathrm{G}\left( \cdot \right)$ is defined as spatial depthwise convolutions.

Finally, the output of the GaMLP encoder, ${Z}_{u}\left(t\right)\in {\mathbb{R}}^{k\times n}$, represents the high-level feature matrix of the recent n interaction histories for user u at time t. It captures the interactions between dynamic interaction histories, effectively modeling the long-term dependency structure in the dynamic interaction history. Similarly, the encoding process for the recent interaction histories of items is identical to that of users. The output, ${Z}_{v}\left(t\right)\in {\mathbb{R}}^{k\times n}$, represents the high-level feature matrix of the recent n interaction histories for item v at time t. For instance, in a movie recommendation system, the GaMLP encoder can be used to capture the dynamic changes and long-term dependencies in users' movie-watching history and the history of movies being watched. Specifically, we take the most recent n movie-watching records of user u as input. The GaMLP encoder transforms these viewing records into a high-level feature matrix ${Z}_{u}\left(t\right)\in {\mathbb{R}}^{k\times n}$, where each column represents a specific viewing event, encompassing high-level information related to that event. By analyzing these high-level features, the GaMLP encoder can understand the dynamic changes in user viewing preferences, such as whether a user is increasingly favoring a certain genre of movies. Similarly, for a specific movie, we can also employ the GaMLP encoder to analyze its viewing history. In this scenario, the encoder's output ${Z}_{v}\left(t\right)\in {\mathbb{R}}^{k\times n}$ represents the high-level feature matrix of the most recent n viewings of movie v at time t. Through this approach, the GaMLP encoder can effectively model the long-term dynamic interaction history of users and items, providing essential input information for the AP network.

4.2.1.   Embedding matrix of recent interaction history

A simple yet expressive technique employed by the GaMLP encoder is to utilize the embedding matrix ${X}_{u}\left(t\right)\in {\mathbb{R}}^{\mathrm{n}\times \left(\mathrm{d}+1\right)}$ (or ${X}_{v}\left(t\right)\in {\mathbb{R}}^{\mathrm{n}\times (\mathrm{d}+1)}$) of the user's (or item's) recent n interaction histories as input to the model, which is defined as follows:

In Eq (4), ${X}_{u}\left(t\right)$ is dependent on the static embeddings of the n most recent interacted with items by user u. Similarly, in Eq (5), the static embeddings of the n users who have recently interacted with item i are used to compute ${X}_{v}\left(t\right)$. The key assumption here is that the static embeddings of multiple recently interacted with items capture strong signals of a user's recent interests, as each static embedding ${\stackrel{-}{v}}_{i}\in {X}_{u}\left(t\right)$ captures the intrinsic attributes or context of item ${v}_{i}$. It is important to note that the embedding matrix at time t only contains information from the past (i.e., before time t), as our objective is to predict the current (i.e., at time t) state.

Furthermore, research has also indicated that the time difference between user-item interactions plays a crucial role in predicting future interactions. Therefore, in the embedding matrix of recent interaction history, each static embedding is concatenated with the time difference $\Delta$ to capture the temporal decay effect of recent interaction events on the current interaction event. It is important to note that some studies use decay functions instead of the time difference $\Delta$, such as $f\left(\Delta \right) = 1/\mathrm{log}(e+\Delta)$ ^[33,34,35]. However, through our experimental testing, we have found no significant difference in these methods, and thus we opted to directly utilize $\Delta$.

4.2.2.   Spatial gating module

As shown in Figure 2, to capture the interactions between the user-item dynamic interaction history, the Spatial Gating (SG) module incorporates projection operations on the spatial dimension. Taking the user side as an example, we denote $\mathrm{S}\mathrm{G}\left( \cdot \right)$ as:

where $\odot$ denotes the element-wise multiplication operation. $U\in {\mathbb{R}}^{n\times n}$ represents a spatial projection matrix of the same size as the length n of the recent interaction history, and b represents the bias term. To ensure training stability, we have found it crucial to initialize U with values close to zero and initialize b as 1. This implies that, at the beginning of the training process, ${h}_{U, b}\left({Y}_{u}\left(t\right)\right)$ is approximately equal to 1, resulting in $\mathrm{S}\mathrm{G}\left({Y}_{u}\left(t\right)\right)$ being close to ${Y}_{u}\left(t\right)$. This initialization method ensures that each GaMLP module behaves similarly to a conventional FFN during the initial stages of training. In this setup, each interaction history embedding is independently processed, gradually incorporating spatial information across the interaction history embeddings during the course of learning.

Furthermore, we have also found that introducing a residual mechanism (Eq (8)) or splitting ${Y}_{u}\left(t\right)$ into two separate parts along the channel dimension for gating functions and multiplication bypasses (Eq (9)) is effective:

In addition, we also normalized the input of ${h}_{U, b}\left( \cdot \right)$, a strategy that has been proven in practice to enhance the stability of the model.

Relation to Existing Networks. The overall formulation of the SG module bears resemblance to Gated Linear Units (GLU) ^[39] as well as early works such as Highway Networks ^[40] and LSTM ^[41]. However, a key distinction lies in the fact that our SG module operates on the spatial dimension for computations, rather than the channel dimension. In terms of element-wise multiplication operations, there is a connection between the SG module and SE blocks ^[42]. However, the SG module does not include cross-channel projections nor does it enforce permutation invariance. The spatial projection in the SG module can theoretically learn to express superficial depthwise convolutions; unlike typical depthwise convolutions with channel-specific filters, the SG module learns a single transformation shared across channels.

4.3.   The AP network

Static embeddings of users and items are both located in the same space, from which the derived high-level features ${Z}_{u}\left(t\right)$ and ${Z}_{v}\left(t\right)$ are based. The AP network enables the direct influence of information from these two high-level features on each other's computations. The main idea behind the AP network is to calculate similarity scores between high-level features based on the dynamic interaction history of users and items, and determine bidirectional attention weights according to these similarity scores of interaction histories ^[43].

Figure 1 illustrates how the AP network is applied to the output of the GaMLP encoder to predict the dynamic embeddings of users and items ($u\left(t\right)$ and $v\left(t\right)$) at time t. Considering the input pair $({Z}_{u}\left(t\right), {Z}_{v}(t\left)\right)$, the similarity score matrix $S\left(t\right)\in {\mathbb{R}}^{\mathrm{n}\times \mathrm{n}}$ between high-level features in the most recent n interaction histories of users and items is represented as:

where $\gamma$ is the parameter matrix. However, based on our empirical observations, setting $\gamma$ as the identity matrix yields superior results. Therefore, in DyCARS, the only parameters that need to be determined are the static embedding matrix E and the parameters associated with the GaMLP encoder, including W, V, U, and bias terms.

Next, we apply max pooling (or average pooling) along the columns and rows of matrix $S\left(t\right)$, resulting in the generation of vectors $\stackrel{~}{u}\left(t\right)\in {\mathbb{R}}^{\mathrm{n}}$ and $\stackrel{~}{v}\left(t\right)\in {\mathbb{R}}^{\mathrm{n}}$, respectively. Formally, the computation of the i-th element of vectors $\stackrel{~}{u}\left(t\right)$ and $\stackrel{~}{v}\left(t\right)$ is as follows (using max pooling as an example):

The i-th element of vector $\stackrel{~}{u}\left(t\right)$ can be interpreted as the importance score of the i-th interaction history of user u relative to the most recent n interaction histories of item v. Similarly, the i-th element of vector $\stackrel{~}{v}\left(t\right)$ can be interpreted as the importance score of the i-th interaction history of item v relative to the most recent n interaction histories of user u.

Next, we apply the Softmax function to vectors $\stackrel{~}{u}\left(t\right)$ and $\stackrel{~}{v}\left(t\right)$ to construct attention vectors ${Att}_{u}\left(t\right)$ and ${Att}_{v}\left(t\right)$. Taking the i-th element of ${Att}_{u}\left(t\right)$ as an example, its calculation formula is as follows:

Finally, the dynamic embeddings of the user and item at time t ($u\left(t\right)$ and $v\left(t\right)$) are calculated as follows:

Both of the above equations can be seen as feature selectors based on bidirectional attention mechanisms. Specifically, the Softmax function provides a weight distribution for different events in the dynamic interaction history of user u and item i, allowing the recent interaction events with larger attention weights to dominate the projection operation. Therefore, DyCARS, by learning the allocation of attention weights, is able to extract more accurately the most relevant delayed interaction patterns from the historical sequence, thus predicting the dynamic embeddings of users and items.

4.4.   D-PCS loss and model optimization

Learning the parameters of DyCARS requires defining an objective function for model optimization. In this study, we propose a novel loss function, D-PCS, suitable for dynamic recommendation tasks. The D-PCS loss function is defined as:

where B represents batch size, and ${I}_{train}$ represents the observed interaction events in the training set. In the D-PCS loss function, the objective of the first term of the loss is to jointly train dynamic embeddings and static embeddings to make the projections of frequently interacted items as close as possible. Therefore, we minimize the ${L}_{2}$ distance between the dynamic embeddings of users and items.

To avoid the trivial solution of the D-PCS loss function, we introduce a regularization loss, ${L}_{REG}$. This strategy is inspired by Laplacian eigenmaps, which avoids collapse by adding the constraint ${u\left(t\right)}^{T}v\left(t\right) = 1$. Therefore, we define ${L}_{REG}$ as:

where $\alpha$ represents the regularization coefficient, and ${I}_{identity}$ denotes the identity matrix.

For the third term in the D-PCS loss function, given a positive user-item pair $(u, v)$ and a set of negative sample items $N\left(u\right)$ randomly sampled for user u, ${L}_{PCS}$ is defined as:

where $\left|N\left(u\right)\right|$ represents the number of negative samples; M is the threshold for filtering negative samples, typically ranging from 0.4 to 0.6 and may vary depending on the specific experiment; $\tau$ is used to control the relative weight of positive and negative sample losses, typically taking values of 1, 2, or 4; ${r}_{uv}$ represents the preference matching score between user u and item v; and $Cosine\left(\cdot \right)$ denotes the cosine similarity between the static embeddings of users and items, defined as follows:

The objective of ${L}_{PCS}$ is to maximize the preference matching score between positives and minimize the preference matching score between negatives. In random probability sampling, we sample based on the occurrence frequency of each item among all negative samples. The mathematical expression of this can be formulated as:

where $f\left(v\right)$ represents the frequency of item v appearing in all negative samples, and $P\left(v\right)$ denotes the sampling probability of item v. This frequency-based sampling method balances the occurrence probabilities of different items during the sampling process by utilizing probability weights. Consequently, it increases the chances of selecting low-frequency items while reducing the sampling probability of high-frequency items. This strategy effectively alleviates biases caused by significant frequency differences, thereby enhancing the effectiveness and diversity of the sampling process.

The design inspiration of ${L}_{PCS}$ originates from the widely used contrastive loss in computer vision tasks ^[44,45]. In contrast to common loss functions in the field of recommendation systems, we made several choices in the design of ${L}_{PCS}$ to facilitate model training and achieve better performance.

First, we chose to use cosine similarity to compute the similarity (or preference matching degree) between user-item pairs, as opposed to using the dot product (e.g., LightGCN ^[46]), Euclidean distance (e.g., Collaborative metric learning ^[47]), or multi-layer perceptron (e.g., Dual sIde deepContext-awarEmodulation for socialRecommendation ^[48]). The reason for this choice is that, similar to the calculation of word similarity in word2vecv ^[49], cosine similarity effectively measures the similarity between vectors and avoids the issue of large value ranges that may arise with the dot product or other methods. Additionally, cosine similarity can handle cases where the vector lengths of users and items differ, making it more suitable for feature vectors of users and items in recommendation systems. For example, in recommendation tasks, the embedding size of users or items may be influenced by their popularity, thus making the advantages of cosine similarity particularly relevant in such scenarios.

Second, we consider the issues caused by the increase in the number of negative samples. Typically, the increased negative samples contain a large amount of redundant and ineffective data, which may interfere with the model's training process, leading to decreased model performance and slower training speed. To address this problem, ${L}_{PCS}$ introduces a threshold M to filter out these useless negative samples. When the cosine similarity of a negative sample is too small and falls below M, they will be assigned a loss of zero. Therefore, ${L}_{PCS}$ can automatically identify the hard negative samples with cosine similarity greater than M and optimize the model training based on them. This strategy is similar to the widely used hard example mining technique in the field of computer vision ^[50].

Finally, for dynamic recommendation systems, there is often a severe imbalance between positive and negative samples. Summing or averaging the loss of all negative samples directly may have adverse effects on the model's performance. Therefore, we introduce a weight factor $\tau$ to adjust the balance between the loss of positive and negative samples. This helps improve the model's performance, especially when there is a large number of negative samples. Moreover, this is a common approach to addressing class imbalance issues.

To optimize the objective function, we choose to use the Adaptive Moment Estimation (Adam) as the optimizer. Adam is a gradient-based optimization algorithm that automatically adjusts the learning rate during the training phase and performs well in handling sparse gradients and non-stationary objective functions. Additionally, to mitigate the overfitting issue, we employ the Dropout strategy ^[51]. Dropout is a widely accepted regularization technique that randomly sets a portion of the neuron outputs to zero during the training phase, thereby preventing the model from overfitting to the training data excessively. By combining the Dropout optimization strategies, our model can effectively handle the training data and enhance recommendation performance.

5.   Experiment

A series of experiments were conducted for a comprehensive evaluation of DyCARS, aiming to address the following research questions:

Q1: How does DyCARS perform compared to state-of-the-art dynamic recommendation methods?

Q2: Are the core components of the GaMLP encoder, gating mechanism, bidirectional attention mechanism, and ${L}_{PCS}$ loss essential for improving the recommendation performance of DyCARS?

Q3: What is the operational efficiency of DyCARS?

Q4: How diverse are the recommendations provided by DyCARS?

Q5: How do the hyper-parameters in DyCARS affect the recommendation performance?

5.1.   Experiment setup

In this section, we will provide a detailed introduction to the experimental setup, including the selection and preprocessing of datasets, the determination of baseline methods and evaluation metrics, as well as the configuration of experimental parameters and the description of the experimental environment.

5.1.1.   Datasets and preprocessing

In order to comprehensively evaluate the performance of the model, a series of experiments were conducted on two real-world datasets. The details of the datasets are shown in Table 3.

1) LastFM: This dataset is widely adopted and consists of music play records within a month ^[52]. We selected a total of 1000 users and the top 1000 most played songs for our study, resulting in 1,293,103 interactions. Note that users only listened to the same song continuously in 8.6% of the interactions. For the experiments, we divided the data into training, validation, and testing sets in proportions of 80, 10, and 10%, respectively.

2) GTD: The GTD is widely recognized as the most comprehensive database covering global terrorist incidents, encompassing over 200,000 terrorist activities carried out by nearly 3000 terrorist organizations worldwide since 1970. This study aims to achieve early warning of terrorist attacks by using DyCARS to predict the next province or state where a terrorist organization is likely to strike at a specific time. During the experiment, we excluded events with unclear dates, unknown perpetrating organizations and attack locations, as well as events involving terrorist organizations and attack locations with fewer than 10 occurrences. Finally, a total of 99,043 events were screened, of which a terrorist organization interacts with the same location of attack consecutively in 22.7% interactions. The data was partitioned based on time, with 70% of the interactions used for model training, 10% for validation, and the remaining interactions for testing purposes.

5.1.2.   Baselines and evaluation metrics

To evaluate the performance of DyCARS, we compare it with the following seven baselines:

1) LSTM ^[41]: This model is a special type of RNN that can capture long-term dependencies.

2) Time-LSTM ^[34]: This model extends the standard LSTM by adding two time gates to model the influence of time intervals on both the user's current and long-term behaviors.

3) RRN ^[15]: This model is based on the concept of matrix factorization and utilizes an RNN to learn dynamic embeddings of users and items.

4) DeepCovolve ^[21]: It is based on co-evolutionary point process algorithms. According to ^[16], 10 negative samples are used in each interaction.

5) JODIE ^[16]: This model is a coupled RNN model used to learn dynamic feature representations of users and items.

6) DGCF ^[17]: This model is a novel framework that captures both the collaborative relationships and historical sequence relationships between users and items using dynamic graphs.

7) DeePRED ^[18]: This model employs a non-recurrent approach to model the short-term preferences of users and items based on their recent interaction history, which represents the state-of-the-art model in dynamic recommendation problems.

We use the following two evaluation metrics for experiments:

1) Mean Reciprocal Rank (MRR) is the average of the reciprocal rank of the first positive example in all user recommendation lists. The performance of a model on the item ranking list can be evaluated using this metric, where a higher score indicates that the ground truth items are ranked higher in the recommendation list. In certain cases, such as search result ranking for specific users, improving MRR can significantly enhance user experience, especially for items with higher importance in the ranking results. The calculation formula is as follows:

where $i\in I$ represents traversing all interactions and $ran{k}_{i}$ represents the position of the first ground truth item in the recommendation list for the i-th interaction.

2) Recall@k is the fraction of interactions in which the ground truth item is ranked in the top k. This metric focuses on evaluating the comprehensiveness and diversity of recommendation results, making it highly applicable in recommendation systems. This metric is used because the recommendation system aims to cover a wide range of items within the user's interests, ultimately enhancing user satisfaction and participation. In certain application environments, such as e-commerce and video recommendations, improving Recall@k can significantly optimize the practical effectiveness of recommendation systems, thereby increasing sales or user stickiness. Formally, Recall@k is defined as:

where ${n}_{hit}$ is the number of ground truth items that are among the top-k recommendation list, and $\left|I\right|$ denotes the total number of interactions.

5.1.3.   Experiments details

The proposed DyCARS in this paper is implemented based on Pytorch and employs Xavier initialization ^[53] to initialize the model parameters. For all algorithms, we adopt 128-dimensional dynamic embeddings and randomly initialize the embeddings of users and items from a Gaussian distribution with mean 0 and variance 1. During the training process, we utilize the validation set to tune the hyper-parameters of DyCARS through Bayesian optimization.

For the LastFM dataset, we set the learning rate of the Adam optimizer to $1\times {10}^{-4}$, dropout rate to 0.8, and the length n of the user-item recent interaction history to 200. The number of layers L for the GaMLP encoder is set to 3, and we select max pooling as the pooling strategy for the AP network. The regularization coefficient $\alpha$ in the D-PCS loss function is set to 0.7, the number of negative samples is set to 10, the threshold M for filtering negative samples is set to 0.4, and the hyper-parameter $\tau$ is set to 1 to control the relative weight between positive and negative sample losses. For the GTD dataset, we set the dropout rate to 0.7 and the number of layers L for the GaMLP encoder to 1. Additionally, we choose average pooling as the pooling strategy for the AP network, and set the regularization coefficient $\alpha$ in the D-PCS loss function to 0.5. It is worth noting that, except for the adjustments mentioned above, the settings of the remaining hyper-parameters are consistent with those for the LastFM dataset.

During the testing phase, for a given real user-item interaction $(u, v, t)$, DyCARS predicts the top-k items that user u will interact with at time t based on the dynamic interaction history of the user and item, denoted as ${I}_{u}\left({t}^{ < }, n\right)$ and ${I}_{v}\left({t}^{ < }, n\right)$, respectively. It should be emphasized that the prediction target of DyCARS is dynamic embedding rather than interaction probability, allowing us to predict the top-k items in the recommendation list through an effective nearest neighbor search strategy.

We carefully tune the parameters to ensure that all baseline models achieve optimal performance. All experiments are run independently in the same experimental environment with an Intel(R) Xeon(R) Platinum 8358P host and NVIDIA Tesla A40-48GB GPU.

5.2.   Next item prediction experiment (Q1)

For next item prediction, we compared DyCARS with seven baselines on two datasets, and the results are presented in Table 4. The best and second-best results are highlighted in bold and underlined, respectively. "Improvement" denotes the performance improvement of DyCARS over the best baseline. Based on the experimental findings, we can draw the following conclusions:

1) On both datasets, DyCARS, DeePRED, and DGCF demonstrate superior performance compared to other baselines, indicating the crucial importance of modeling topological structure information for learning dynamic graphs.

2) DyCARS outperforms all baselines in terms of all metrics on both datasets. Particularly on the LastFM dataset, compared with the best baseline, the improvement of DyCARS on MRR is 39.95% and the improvement on Recall@10 is 28.88%. Furthermore, the performance of DyCARS gradually improves compared to state-of-the-art baselines on both the GTD and LastFM datasets, which aligns with the repetitive patterns observed in user behavior within the datasets. These substantial improvements can be attributed to two key factors. First, DyCARS leverages the recent interaction history of users and items as dynamic context, and models long-term dependency structures in the dynamic interaction history through the GaMLP encoder. Second, we introduce the AP network to learn similarity scores between high-level features of user and item dynamic interaction histories, extracting the most relevant delayed interaction patterns from the historical sequences by calculating bidirectional attention weights. As a result, DyCARS is capable of effectively handling scenarios with low action repetition.

3) In DyCARS, there is a significant reduction in the gap between the MRR and Recall@10 metrics compared to other baselines. This indicates that DyCARS exhibits higher precision in ranking ground truth items, whereas other baseline models often only achieve high rankings within the top ten predicted items.

5.3.   Detailed model analysis (Q2)

In this section, we will conduct a detailed analysis of the model to gain a deeper understanding of the impact of each core component of DyCARS on its overall performance. This analysis will focus on four key aspects: the influence of different encoders, the role of the gating mechanism, the contribution of the bidirectional attention mechanism, and the effectiveness of the ${L}_{PCS}$ loss function. Through these analyses, our aim is to uncover the internal workings of DyCARS and how each component collaboratively enhances the performance of the recommendation system.

5.3.1.   Impact of different encoders

Several variants of DyCARS were implemented, and comparative experiments on two datasets were conducted to demonstrate the effectiveness of the GaMLP encoder. The specific settings for these variants are as follows:

1) DyCARS-GRU: In this variant, the GaMLP encoder is replaced with a GRU ^[54]. GRU is a commonly used RNN variant for sequence modeling. It employs gate mechanisms that allow the network to selectively retain and update information when processing sequential data, enabling better capture of long-term dependencies.

2) DyCARS-Trm: In this variant, the GaMLP encoder is replaced with a Transformer model ^[55]. The Transformer model is a deep learning architecture based on self-attention mechanisms, specifically designed for handling sequential data. The self-attention mechanism allows the model to attend to different positions within the input sequence, capturing richer contextual information without relying on complex recursive or convolutional structures. Additionally, the Transformer model supports parallel computation, greatly improving the efficiency of processing long sequential data.

According to the experimental results in Table 5, the following can be observed:

1) The performance of DyCARS-GRU is superior to DyCARS-Trm. This result may be due to several factors. First, models based on RNNs (such as GRU) often perform better on smaller datasets. Conversely, Transformer models typically require large-scale datasets to achieve optimal performance. Second, when dealing with dynamic recommendation problems, certain characteristics of GRUs may contribute to its more prominent performance. For instance, the adoption of gate mechanisms in GRUs allows for explicit storage and retrieval of user-item interaction history. Additionally, due to its recursive nature, GRUs can maintain the sequential order of input sequences. These two characteristics are crucial for addressing dynamic recommendation problems.

2) The performance of DyCARS surpasses that of variants using GRUs or Transformers, and several key factors can account for this. First, the GaMLP encoder utilizes an embedding matrix of the user's (or item's) recent n interaction histories as input. This embedding matrix concatenates the static embeddings of the most recent interacting items (or users) with the temporal differences between user-item interactions, thereby capturing user behavior patterns and variations in intrinsic item properties more comprehensively. It also effectively considers the time decay effect of recent interaction events on the current event. Second, the GaMLP encoder employs the SG module to capture the interactions between user-item dynamic interaction histories, enabling more effective exploration of correlations and patterns among interaction events, thus achieving modeling of long-term dependency structures in dynamic interaction histories. In contrast, the modeling capabilities of DyCARS-GRU and DyCARS-Trm may be constrained by their respective architectures, potentially limiting their ability to fully capture the complexity of dynamic interactions.

5.3.2.   Impact of gating mechanisms

One notable feature of the GaMLP encoder is its adoption of the SG module to capture the interactions between user-item dynamic interaction history, enabling the modeling of long-term dependency structures within the dynamic interaction history. Table 6 demonstrates the impact of different gating mechanisms on model performance on two datasets. Taking the user side as an example, the mathematical representations of various gating mechanisms are as follows:

1) Variant a (Linear GaMLP): ${\mathrm{S}\mathrm{G}\left({Y}_{u}\left(t\right)\right) = h}_{U, b}\left({Y}_{u}\left(t\right)\right)$

2) Variant b (Additive GaMLP): ${\mathrm{S}\mathrm{G}\left({Y}_{u}\left(t\right)\right) = {Y}_{u}\left(t\right)+h}_{U, b}\left({Y}_{u}\left(t\right)\right)$

3) Variant c (Multiplicative GaMLP): ${\mathrm{S}\mathrm{G}\left({Y}_{u}\left(t\right)\right) = {Y}_{u}\left(t\right)\odot h}_{U, b}\left({Y}_{u}\left(t\right)\right)$

4) Variant d (Multiplicative, Split GaMLP): ${\mathrm{S}\mathrm{G}\left({Y}_{u}\left(t\right)\right) = {Y}_{u}^{1}\left(t\right)\odot h}_{U, b}\left({Y}_{u}^{2}\left(t\right)\right), {Y}_{u}\left(t\right) = {Y}_{u}^{1}\left(t\right)‖{Y}_{u}^{2}\left(t\right)$

The model architectures of the four variants are illustrated in Figure 3.

Except for the variant b used on the GTD dataset in Section 5.2, variant d is employed throughout the rest of this paper. From Table 6, it can be observed that the impact of different gating mechanisms on model performance varies depending on the dataset. For the LastFM dataset, its low user action repetition and higher number of interaction events indicate that the dataset exhibits stronger dynamism and is more challenging to predict. In this scenario, variant d performs the best. This approach may have increased the complexity and flexibility of the model by splitting the input embeddings into two parts and processing them separately, enabling the model to better capture complex user behavior patterns and long-term dependencies. In contrast, for the GTD dataset, its higher user action repetition suggests a weaker level of dynamism, making the prediction relatively simpler. In this scenario, variant b performed the best, possibly due to the provision of moderate complexity offered by variant b.

5.3.3.   Impact of bidirectional attention mechanism

DyCARS leverages the AP network to learn bidirectional attention weights for extracting the most relevant delayed interaction patterns from user-item interaction history, enabling the prediction of dynamic embeddings for users and items. To investigate the impact of the bidirectional attention mechanism on model performance, we conducted comparative experiments on two datasets. Specifically, "DyCARS w/o Att" means that we pool the high-level feature matrices of the recent interaction history of users and items by rows and columns, respectively, to directly generate dynamic embeddings of users and items.

The results in Table 7 clearly demonstrate a significant decline in the performance of DyCARS across both metrics when lacking a bidirectional attention mechanism. This observation underscores the inadequacy of solely relying on pooling operations for extracting high-level feature matrices of users and items. Such a method fails to capture the complexity and dynamics in the user-item interaction history, potentially overlooking crucial delayed interaction patterns that have decisive impacts. Moreover, this finding underscores the critical role of bidirectional attention mechanisms in DyCARS.

5.3.4.   Impact of ${L}_{PCS}$ loss

Currently, most research on dynamic recommendation focuses primarily on constructing recommendation models, often overlooking the crucial role of the loss function in model learning. To investigate the impact of the ${L}_{PCS}$ loss on model performance, we conducted comparative experiments on two datasets. Specifically, "DyCARS w/o ${L}_{PCS}$" denotes training DyCARS without the ${L}_{PCS}$ loss by removing it from the D-PCS loss function.

Table 8 presents the experimental results. It should be noted that each model has been trained for sufficient iterations to achieve convergence, and the best results have been reported. The results demonstrate a significant improvement in both evaluation metrics when the ${L}_{PCS}$ loss is included in the D-PCS loss function. This finding underscores the crucial role of the ${L}_{PCS}$ loss in model parameter optimization and performance enhancement. By utilizing cosine similarity to compute the similarity between user-item pairs, the ${L}_{PCS}$ loss avoids the issue of large value ranges that may arise in dot product or other methods. Additionally, the ${L}_{PCS}$ loss allows the model to emphasize learning from hard negative samples, resulting in more discriminative features and enhancing the model's ability to extract useful information.

5.4.   Runtime experiment (Q3)

To empirically evaluate the efficiency of DyCARS, we conducted measurements to assess its runtime. In Figure 4, we present a comparison of the computational time (in seconds) required to complete an epoch using the GTD dataset. The following figure clearly demonstrates that DyCARS exhibits a significant advantage in terms of the computational time compared to recursive baseline models such as JODIE and DGCF. Moreover, the computational time of DyCARS is comparable to that of the non-recursive baseline model, DeePRed. Recursive models typically incur additional computational costs as they rely on their previous states to predict dynamic embeddings. However, DyCARS circumvents this additional overhead by leveraging its non-recursive architecture that relies on interaction history rather than previous dynamic embeddings. Notably, DyCARS achieves both high efficiency and excellent predictive accuracy. This is primarily attributed to the powerful expressive capabilities of the GaMLP encoder, the AP network, and the optimized design of the D-PCS loss function.

5.5.   Diversity analysis in recommendations (Q4)

In practical applications, an enhancement in diversity within recommendation systems often accompanies a reduction in prediction accuracy. This phenomenon partially stems from the inherent uncertainty in user interests and the influence of certain unobservable factors in recommendation systems, which limit the real-time prediction capabilities for users' next actions. Moreover, most existing experimental simulations do not fully reflect the impact of real-time recommendation systems on user behavior. However, this does not imply an irreconcilable contradiction between diversity and accuracy. In fact, a key goal of recommendation systems is to guide user choices, rather than merely matching users' historical data. Increasing the exposure of different items in the recommendation list aims to provide users with a broader range of information, assisting them in discovering new items that may be of interest. Considering this, our study has adopted the widely used diversity metric in the field of recommendation systems, Diversity ^[56], to measure the degree of variation among items in the recommendation list. The formula for calculating this metric is as follows:

In this formula, K represents the length of the recommendation list, and $sim(i, j)$ denotes the cosine similarity between item i and item j.

Figure 5 displays the results of the diversity analysis of DyCARS on the GTD dataset. The experimental data indicate that, compared to the state-of-the-art baseline model DeePRED, DyCARS not only maintains personalized recommendations in line with user interest patterns but also effectively enhances the diversity of the recommendations.

5.6.   Parameter analysis (Q5)

In this section, we analyzed the impact of various hyper-parameters of DyCARS on the next item prediction. Throughout the subsequent experimental process, unless otherwise stated, we employ average pooling as the pooling strategy for the AP network on the LastFM dataset.

5.6.1.   Impact of embedding size

Table 9 illustrates the impact of embedding size on model performance on two datasets. From the experimental results, it can be observed that for the LastFM dataset, increasing the embedding size yields improved performance. In contrast, for the GTD dataset, the model achieves optimal performance when the embedding size is set to 128. However, increasing the size of embeddings also results in increased computational complexity. Therefore, we need to choose a suitable embedding size to balance the model performance and efficiency.

5.6.2.   Impact of the number of layers L in the GaMLP encoder

The experimental results in Figure 6 demonstrate the impact of the number of layers L in the GaMLP encoder on model performance, indicating that this effect is dependent on dataset characteristics. For the LastFM dataset, the model exhibits improved performance with an increase in the number of layers L. This could be attributed to the fact that the LastFM dataset possesses more complex interaction dynamics and structures, requiring multiple layers of GaMLP encoders to capture deep-level interaction patterns and long-term dependencies. Each layer of the GaMLP encoder can extract different levels of information from the user-item interaction history, and the stacking of multiple layers aids the model in learning higher-order representations.

However, on the GTD dataset, the model achieves optimal performance when the number of layers is set to 1. This could be due to the lower complexity or smaller sample size of the GTD dataset. Therefore, increasing the model's complexity, such as adding more layers, may result in overfitting on the training data and a decrease in the generalization performance on the test data. Additionally, if the dataset has low dynamics, multiple layers of GaMLP may not bring additional performance improvement but instead increase computational costs and model complexity.

5.6.3.   Impact of the length n of user-item recent interaction history

We conducted comparative experiments on two datasets to assess the effect of different lengths of recent user-item interaction history on model performance. As depicted in Figure 7, it can be observed that, as the length n of recent interaction history increases, the model's performance gradually improves on both datasets. This can be attributed to several potential reasons: First, appropriately increasing the length of recent interaction history allows the model to acquire richer contextual information. This helps the model better understand user behavior patterns and changes in interests, thereby making more accurate recommendations. Second, as the length n of recent interaction history increases, the model can encompass a wider time span, enabling it to effectively capture long-term dependency structures and better model the long-term trends in user behavior. Finally, in the short-term, user behavior may be influenced by certain transient factors such as emotions or seasonal activities, which can introduce noise. By increasing the history length, the model can consider user behavior across multiple time periods, reducing the impact of these transient factors and improving recommendation stability.

However, it is important to note that an excessively long length of recent interaction history may lead to decreased computational efficiency and introduce an abundance of irrelevant information, which can have a negative impact on the model's performance. Therefore, it is necessary to carefully select an appropriate length to strike a balance between model performance and efficiency.

5.6.4.   Impact of different pooling strategies in the AP network

Comparative experiments were conducted on two datasets to assess the impact of various pooling strategies on model performance. As observed from Table 10, the max pooling strategy exhibits superior performance for the LastFM dataset. However, for the GTD dataset, the average pooling strategy demonstrates better performance. This indicates that the choice of pooling strategy directly influences the model's ability to capture and extract features. The effects of max pooling and average pooling may vary when dealing with datasets of different types.

Max pooling is a strategy that emphasizes the most salient features. The LastFM dataset features complex and diverse user-item interactions. Within this dataset, certain key features closely related to the prediction outcome play a significant role in predicting user behavior. Therefore, by selecting the maximum value, max pooling can capture these crucial features, thereby enhancing the performance of the model.

In contrast, average pooling takes into account the average effect of all features, enabling it to smooth the data and reduce the influence of noise and outliers. On the GTD dataset, there may not be significant key features, and the overall behavioral patterns are more crucial for prediction. In such cases, average pooling can capture the global behavioral patterns, leading to superior performance.

6.   Discussion

In this section, we first explore the potential impact of Session-Based Recommendation Systems (SBRSs) and the Dynamic Recommendation Systems (DRSs) used in this study on personalized recommendations, as well as their differences. Then, we discuss the prospects of applying recent large-scale time series language models in the field of DRSs. Additionally, we address the limitations of this study.

6.1.   Session-based recommendation systems and dynamic recommendation systems

The potential impact and differences between SBRSs and DRSs on personalized recommendations are primarily evident in data processing and recommendation strategies. 1) Data processing: SBRSs typically handle session data, which consists of a series of interactions with clear start and end boundaries, focusing on intra-session interactions. DRSs, on the other hand, process continuous sequences of user behavior, emphasizing long-term user preferences and behavioral patterns. 2) Recommendation strategy: SBRSs concentrate on short-term user behavior and interests within the current session, which may lead to more immediate and current activity-related recommendations. DRSs, however, provide deeper levels of personalized recommendations by analyzing long-term data to understand a user's overall interests. Therefore, SBRSs focus more on short-term, immediate personalization, while DRSs emphasize the analysis of long-term user behavior and the depth of personalized recommendations.

In DyCARS, the following characteristics of the model design determine its potential impact on personalized recommendations: 1) Capturing dynamic interests: Through the dynamic graph and the GaMLP encoder, DyCARS can more accurately capture the evolving interests of users over time, thereby offering more tailored personalized recommendations that fit current needs. 2) Long-term dependency modeling: The model considers users' long-term interaction history, which aids in understanding and predicting their interests and preferences. 3) Real-time updates: As the model emphasizes dynamic interaction history, it can respond more swiftly to the latest user behaviors, providing real-time updated personalized recommendations. These features make DyCARS more precise and timely in delivering personalized recommendations, catering to the ever-changing needs of users.

6.2.   Time series large language models and dynamic recommendation systems

Recent time series large language models, such as those based on the Transformer architecture, have provided significant insights for the field of DRSs. These models excel in processing long sequence data, particularly in capturing temporal dependencies and understanding complex contexts. In DRSs, these capabilities can be leveraged to analyze and predict users' behaviors and interests as they evolve over time more accurately.

Furthermore, the natural language processing capabilities of these models can also: 1) Analyze user-generated content, such as reviews or social media posts, to enrich user profiles and recommendation logic. 2) By analyzing the emotional tendencies in users' statements and interactions, DRSs can more accurately capture changes in user mood, thereby offering recommendations that better align with the user's current emotional state. 3) Language models can assist in identifying and analyzing trending topics in social media and online content, enabling recommendation systems to respond promptly to current hot topics and user interests. Therefore, combining research on time series large language models with DRSs could lead to new breakthroughs in personalized recommendations.

6.3.   Limitations of this study

Despite the excellent performance of DyCARS in multiple aspects, we also recognize its limitations. 1) Dataset scale and complexity: The current DyCARS model has primarily been tested on datasets of limited scale. Its performance and efficiency may be challenged on larger and more complex datasets. 2) Limited utilization of contextual information: Although the DyCARS model considers the dynamic interaction history of users and items, it does not fully utilize other potential contextual information, such as users' social networks, browsing behavior, and emotional states. 3) Challenges in real-time recommendations: In handling real-time data streams and rapidly changing user preferences, the model may require further optimization to improve response speed and accuracy.

7.   Conclusions

This study proposes a novel non-recursive dynamic recommender model, called DyCARS, and successfully applies it to dynamic recommendation. DyCARS considers the dynamic interaction history of users and items as dynamic context. This approach utilizes the GaMLP encoder to capture long-term dependency structures in user-item interaction sequences. Additionally, we introduce the AP network to extract the most relevant delayed interaction patterns from the dynamic interaction history. Specifically, our model is built on dynamic graphs and leverages the static embeddings of recent user-item interactions as dynamic context. Additionally, we construct a GaMLP encoder to capture the long-term dependencies in the dynamic interaction history and extract high-level features. Furthermore, by applying the AP network, we learn similarity scores between the high-level features of users and items in the dynamic interaction history. By computing the similarity score matrix of high-level features, we obtain bidirectional attention weights and uncover the most relevant delayed interaction patterns from the historical sequence to predict the dynamic embeddings of users and items. Finally, we propose a D-PCS loss function suitable for dynamic recommendation, aiming to jointly optimize the static and dynamic embeddings of both types of nodes. We conducted extensive experiments on two real-world datasets, and the results demonstrate that DyCARS achieves state-of-the-art performance. In future research, we plan to integrate more fine-grained contextual factors into DyCARS, such as considering users' social network information, browsing behavior, and emotional states. The inclusion of these pieces of information may further enhance the accuracy and diversity of recommendations. Next, we aim to test and optimize the DyCARS model on larger and more diverse datasets to validate its performance and applicability in different scenarios. Moreover, in response to real-time data streams and rapidly changing user demands, we will explore new lightweight model architectures and real-time data preprocessing techniques to improve the response speed and accuracy of DyCARS.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This work was supported by the Double First-Class Innovation Research Project for People's Public Security University of China (No.2023SYL08).

Conflict of interest

The authors declare there is no conflict of interest.