Stimulating the processes of attracting investments in industry of the constituent entities of the Russian Federation

1.   Introduction

With the spread of COVID-19, the emergence of online learning ^[1] platform has effectively solved the dilemma that universities and educational institutions cannot carry out offline teaching and ensured the safety and reliability of learning scenarios, which has become an important way for people to acquire knowledge. How to use the Internet and various artificial intelligence technologies to select the appropriate resource list from massive learning resources and accurately recommend the learning content to provide personalized learning experience for people has become one of the research hotspots in the field of online learning.

Personalized recommendation system can not only satisfy the preferences of learners, but also effectively improve the satisfaction of online learning. The recommendation system ^[2] generates the appropriate resource recommendation list mainly according to learning time ^[3], knowledge ^[4], learning preference ^[5], demand ^[6] and social situation ^[7]. The methods of learning resource recommendation can be basically divided into content-based resource recommendation ^[8], collaborative filtering-based resource recommendation ^[9], knowledge-based resource recommendation methods ^[10] and session-based resource recommendation ^[11]. The content-based learning resource recommendation makes similarity recommendation by comparing the characteristics between learners and resources, which is intuitive but ignores the correlation between resources. The collaborative filtering-based learning resource recommendation makes recommendation through the similarity matrix between learners or resources. This method has the defects of cold start and high time complexity, and the calculation method of similarity will affect the results. The knowledge-based learning resource recommendation considers the relevant knowledge of learners and resources, and matches the knowledge demands of learners with resource knowledge. This method conforms to the law of psychology and can save a lot of learning time, but its recommendation process requires large amount of manual intervention, which has subjective influence. The session-based learning resource recommendation relies on the interaction information between learners and resources, which can effectively improve the recommendation accuracy of resources. However, it requires the internal structure processing and the modeling of relationship between sessions on session data, and there are few researches on this aspect.

Learners have multiple attributes, and there are associations between resources, while users' demands for learning resources are multi-dimensional. For example, different users have different requirements on cognitive ability, media preference, content preference and other aspects. Therefore, the problem of learning resource recommendation with multidimensional constraints can be regarded as an NP-hard problem, and the main solutions include ant colony algorithm ^[12], genetic algorithm ^[13] and particle swarm optimization algorithm ^[14]. In addition, the $\varepsilon$-constraint method ^[15] and the minimum-maximum method ^[16] solve the multi-objective optimization problem by constraining other objective functions and optimizing one objective function. These research methods all need to set complex weights. In order to avoid the above problems, NSGA-Ⅱ and its improved algorithm ^{[17,18,19,20,21,22,23,24,25]}, as a solution to the multi-objective optimization problem, have successfully solved the multi-objective optimization problem in several research fields. However, there are few related researches in the field of online learning resource recommendation.

From the view of balancing the various demands of learners, this paper proposed a learning resource recommendation model based on the improved NSGA-Ⅱ algorithm. The learning resources were filtered with background fitness, cognitive fitness and diversity as objective functions. The Tabu search algorithm was applied to improve local search ability. At the same time, the dynamic updating of crowding degree and the adaptive changing of crossover rate and mutation rate were adopted to ensure individual diversity. The main contributions of this paper are as follows:

1) In order to solve the problem that low-density individuals are easy to be deleted and homogeneous populations, dynamic updating crowded distance and adaptive crossover mutation operators are proposed.

2) According to the multi-dimensional needs of learners, a three-objective optimization model of background fitness, cognitive fitness and diversity is established.

3) While retaining the advantages of individual diversity and uniformity of NSGA-Ⅱ algorithm, Tabu search strategy is added to improve the search ability of solution set and increase the diversity.

The rest of this paper is structured as follows: Section 2 describes the algorithm and modeling process of this paper; Section 3 details the solution process of model. Section 4 discusses the experiments and results. Section 5 presents conclusion and future research direction.

2.   Problem description and model framework

This section first makes conventions on the symbols used in the full text, and formalizes the selection of personalized learning resources, including the necessary symbol definition. After that, the factors that affect the results of personalized resource recommendation are briefly analyzed to clarify the rationality of the three factors considered in this paper, i.e., background fitness, difficulty fitness and content preference.

2.1.   Problem description

The online resource recommendation technology in online teaching judges the preferences of students according to their pre-test characteristics, so as to provide students with resources that meet their demands ^[26]. The characteristics of students in online teaching mainly include age, gender, educational background, cognitive ability and other information, as well as the acceptable learning time of learners. The characteristics of online resources mainly include the average age of learning groups ^[27], female proportion, educational threshold, difficulty level, the feature of resource content and time to complete learning resources.

This paper proposes an online learning resource recommendation model based on improved NSGA-Ⅱ algorithm, which does not convert the multi-objective problem into a single objective problem by the weighted method, but uses the multi-objective method to improve the online learning resources recommendation. The improved NSGA-Ⅱ algorithm adopts the ternary combination mode, and takes background fitness, cognitive fitness and diversity as the selection targets.

Definition 1. Characteristics of learner: The personalized characteristics of learners include background information (BI), cognitive ability (CA) and time limit (T). The learner set is expressed as $\mathrm{U} = \{{\mathrm{u}}_{1}, {\mathrm{u}}_{2}, ..., {\mathrm{u}}_{\left|\mathrm{U}\right|}\}$, where the $\left|\mathrm{U}\right|$ represents the size of learner set. For a specific learner ${\mathrm{u}}_{\mathrm{i}}$, its background information (${\mathrm{B}\mathrm{I}}_{\mathrm{i}}$) is defined as ${\mathrm{B}\mathrm{I}}_{\mathrm{i}} = \{\mathrm{A}\mathrm{I}, \mathrm{G}\mathrm{I}, \mathrm{E}\mathrm{I}\}$, which is composed of age information (AI), gender information (GI) and education Information (EI). According to the relevant researches on the influencing factors of cognitive ability in recent years, it is concluded that learners' cognitive ability in the field of education has relation to their age ^[28], gender ^[29] and educational background ^[30].

Definition 2. Characteristics of learning resource: The characteristics of learning resources include preference information (PI), difficulty information (DI), content information (CI), and time (t). The learning resource set is expressed as $\mathrm{L} = \{{\mathrm{L}\mathrm{O}}_{1}, {\mathrm{L}\mathrm{O}}_{2}, ..., {\mathrm{L}\mathrm{O}}_{\left|\mathrm{L}\mathrm{O}\right|}\}$, of which |LO| denotes the size of learning resource set. The preference information (PI) of specific learning resource ${\mathrm{L}\mathrm{O}}_{\mathrm{i}}$ is defined as ${\mathrm{P}\mathrm{I}}_{\mathrm{i}} = \{\mathrm{M}\mathrm{A}, \mathrm{W}\mathrm{P}, \mathrm{U}\mathrm{P}\}$, which is composed of mean age (MA) of learner group, women proportion (WP) and undergraduates percentage (UP).

Definition 3. Content information of learning resource: content information (CI) of learning resources is expressed as $\mathrm{C}\mathrm{I} = \{\mathrm{P}\mathrm{D}, \mathrm{A}\mathrm{D}, \mathrm{N}\mathrm{D}\}$, which determines the selection quality of learning resource recommendation list. It is codetermined by popularity degree (PD), authority degree (AD) and novelty degree (ND). The background fitness (BF) between learners and learning resources is decided by learners' background information (BI) and learning resources' preference information (PI). The difficulty fitness (DF) of learner to learning resources is determined by the cognitive ability (CA) of learners and the difficulty information (DI) of learning resources (DI).

The background fitness, cognitive fitness and diversity screening of learner affect the final resource recommendation list from three dimensions. Based on the above three characteristics, a multi-dimensional online learning resource recommendation model combined with time constraints is constructed. According to the background fitness and the difficulty fitness of learners, the diversity of resources is regarded as the auxiliary screening objective to provide suitable online learning resources list to learners.

2.2.   Construction of multi-objective recommendation model

Aimed at the above problems, an online learning resource recommendation model based on the improved NSGA-Ⅱ algorithm is proposed, as shown in Figure 1.

In the above recommendation model, the three-dimensional objectives of resource screening include background fitness, cognitive fitness and diversity. The screening conditions of diversity contain popularity degree, authority degree and novelty degree. The three-dimensional objectives are constrained by learning time limit. The three-dimensional objectives and time constraints jointly determine the final result, the higher the fitness of result, the better the experience of learners.

2.3.   Mathematical modeling of multi-objective programming model

The background characteristics of learners include age, gender and education, which affect the choice of learning resources to a large extent. The background information is expressed as $\mathrm{B}\mathrm{I} = [{\mathrm{B}\mathrm{I}}_{1}, {\mathrm{B}\mathrm{I}}_{2}, ..., {\mathrm{B}\mathrm{I}}_{\left|\mathrm{U}\right|}]$, where ${\mathrm{B}\mathrm{I}}_{\mathrm{i}}$ is a three-dimensional vector. The background characteristics of the ith learner are represented by ${\mathrm{B}\mathrm{I}}_{\mathrm{i}} = \{{\mathrm{A}\mathrm{I}}_{\mathrm{i}}, {\mathrm{G}\mathrm{I}}_{\mathrm{i}}, {\mathrm{D}\mathrm{I}}_{\mathrm{i}}\}$. The preference information of learning resources are described by $\mathrm{P}\mathrm{I} = [{\mathrm{P}\mathrm{I}}_{1}, {\mathrm{P}\mathrm{I}}_{2}, ..., {\mathrm{P}\mathrm{I}}_{\left|\mathrm{L}\mathrm{O}\right|}]$, of which ${\mathrm{P}\mathrm{I}}_{\mathrm{j}}$ is a three-dimensional vector, ${\mathrm{P}\mathrm{I}}_{\mathrm{j}} = [{\mathrm{M}\mathrm{A}}_{\mathrm{j}}, {\mathrm{W}\mathrm{P}}_{\mathrm{j}}, {\mathrm{U}\mathrm{P}}_{\mathrm{j}}]$ shows the preference information of the jth learning resource. The background fitness of the ith learners to the jth learning resource is formulated as:

where, ${\mathrm{A}\mathrm{I}}_{\mathrm{i}}$ indicates the age information of learners, values for actual age. ${\mathrm{G}\mathrm{I}}_{\mathrm{i}}$ figures the gender information of learner, with value of 0 or 1, 0 for female and 1 for male. ${\mathrm{D}\mathrm{I}}_{\mathrm{i}}$ denotes the education information of learners. The value can be 0 or 1, indicating that the education of learner is below the bachelor degree and above the bachelor degree, respectively. ${\mathrm{M}\mathrm{A}}_{\mathrm{j}}$ represents the mean age of learner that choose the resource. ${\mathrm{W}\mathrm{P}}_{\mathrm{j}}$ shows the proportion of women that study the resource, and ${\mathrm{U}\mathrm{P}}_{\mathrm{j}}$ represents the undergraduate proportion of the population studying the resource.

The cognitive ability of learner is expressed as $\mathrm{C}\mathrm{A} = [{\mathrm{C}\mathrm{A}}_{1}, {\mathrm{C}\mathrm{A}}_{2}, ..., {\mathrm{C}\mathrm{A}}_{\left|\mathrm{U}\right|}]$, where ${\mathrm{C}\mathrm{A}}_{\mathrm{i}}$ shows the learning ability of the ith learner, value for ${\mathrm{C}\mathrm{A}}_{\mathrm{i}}\in \left(\mathrm{0, 1}\right)$. The difficulty of learning resource is related to the learning stage and the percentage of people who have got achievement. The difficulty ${\mathrm{D}\mathrm{I}}_{\mathrm{j}}$ of the jth learning resource can be calculated by

where, ${\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}}_{\mathrm{j}}$ represents the course duration of learning resource, ${\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}}_{\mathrm{j}}\in \left[\mathrm{0, 4}\right]$ and ${\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}}_{\mathrm{j}}$ is an integer. ${\mathrm{c}\mathrm{o}\mathrm{m}}_{\mathrm{j}}$ denotes the number of people who have completed the learning task and successfully took the exam; $\mathrm{M}\mathrm{a}\mathrm{x}\left(\mathrm{c}\mathrm{o}\mathrm{m}\right)$ indicates the maximum number of participants for all learning resources. The difficulty fitness ${\mathrm{D}\mathrm{F}}_{\mathrm{i}\mathrm{j}}$ of learners is the difference between the cognitive ability of the ith learner ${\mathrm{C}\mathrm{A}}_{\mathrm{i}}$ and the difficulty of the jth learning resource ${\mathrm{D}}_{\mathrm{j}}$:

If the difficulty of learning resources exceeds the cognitive ability of learners too much, learners will not be able to learn smoothly. If the difficulty of learning resources is too simple, there will be time waste of learners. Therefore, the difficulty of learning resources should be as close as possible to the cognitive limit of learners, the calculation result ${|\mathrm{D}\mathrm{F}}_{\mathrm{j}}|$ the closer to zero, the better.

The content preference CP of learning resource is determined by its popularity degree, authority degree and novelty degree. The popularity degree of learning resources is related to the percentage of participants, the viewing frequency and score. The percentage of learning participants, the viewing frequency and the comprehensive score of resources can be expressed as:

where, ${\mathrm{S}}_{\mathrm{j}}$ indicates the participant number of learning resource ${\mathrm{L}\mathrm{O}}_{\mathrm{j}}$ that exceeds 50%. Max(S) indicates the maximum visits time of learning resource. ${\mathrm{V}}_{\mathrm{j}}$ represents the viewing times of learning resource ${\mathrm{L}\mathrm{O}}_{\mathrm{j}}$. Max(V) denotes the maximum viewing times of resource. ${\mathrm{P}}_{\mathrm{i}\mathrm{j}}$ shows the score of ith learner for the jth learning resource. $\stackrel{-}{{\mathrm{P}}_{{\mathrm{L}\mathrm{O}}_{\mathrm{j}}}}$ is the average score of learning resources. $\stackrel{-}{{\mathrm{P}}_{{\mathrm{u}}_{\mathrm{i}}}}$ is the average score of all learners.

The popularity degree (PD) of learning resource is calculated by weighted ${\mathrm{f}}_{1}\left({\mathrm{L}\mathrm{O}}_{\mathrm{j}}\right)$, ${\mathrm{f}}_{2}\left({\mathrm{L}\mathrm{O}}_{\mathrm{j}}\right)$ and ${\mathrm{f}}_{3}\left({\mathrm{L}\mathrm{O}}_{\mathrm{j}}\right)$ in the above formula, with formula as follows:

The authority of learning resources will affect the trust of learner in the resources. The authoritative resources can improve the learning motivation of learner. The influences of issuing institution and lecturer affect the authority of resources. H-index is a method to evaluate academic achievements. The H-index of a scientific researcher means that he has H papers cited at least H times. The influence of institution can be related to the H-index of its staff, so the influence of agency can be expressed as:

where, ${\mathrm{H}}_{\mathrm{P}}$ represents the H-index of institution. $\mathrm{M}\mathrm{i}\mathrm{n}\left({\mathrm{H}}_{\mathrm{P}}\right)$ and $\mathrm{M}\mathrm{a}\mathrm{x}\left({\mathrm{H}}_{\mathrm{P}}\right)$ are the minimum and maximum H-index of the staff in institution. The influence of resource lecturers is expressed as:

where, ${\mathrm{H}}_{\mathrm{a}}$ is the H-index of resource lecturers. $\mathrm{M}\mathrm{i}\mathrm{n}\left({\mathrm{H}}_{\mathrm{a}}\right)$ and $\mathrm{M}\mathrm{a}\mathrm{x}\left({\mathrm{H}}_{\mathrm{a}}\right)$ represent the minimum and maximum H-index of lecturers. Their H-index is closely related to the citation rate of their published scientific papers, which can be found and calculated from research platforms like CNKI. The authority degree AD of learning resources is expressed as:

Novel learning resources can improve the positivity of learners. The novelty degree of learning resources $\mathrm{N}\left({\mathrm{L}\mathrm{O}}_{\mathrm{i}}\right)$ is related to the release time of resource ${\mathrm{T}\mathrm{i}\mathrm{m}\mathrm{e}}_{{\mathrm{L}\mathrm{O}}_{\mathrm{i}}}$, and the novelty degree of resources is expressed as:

where, E(Time) represents the earliest release time of the knowledge point contained in the learning resource, and T represents the current year. By integrating the PD, AD and ND of learning resources, the content selection of learning resources can be expressed as:

3.   Model solution

In this paper, a new hybrid algorithm is proposed, which combines the Tabu search algorithm with excellent local search efficiency with the NSGA-Ⅱ algorithm. The children obtained from the Tabu search algorithm are inserted into the population selection of NSGA-Ⅱ algorithm to enhance the diversity of population. The basic idea of NSGA-Ⅱ algorithm is to sort the population by non-dominated sorting and calculate the crowding distance to maintain the diversity of population. The improved NSGA-Ⅱ algorithm takes the resources as the parent population ${\mathrm{P}}_{0}$ with the size of N, and carries out non-dominated sorting on them. Each resource individual obtains the fitness, then doing tournament selection and genetic operator operation on each resource individual to generate a child population ${\mathrm{G}}_{\mathrm{t}}$ with the size of N.

In the elitist strategy, the parent population ${\mathrm{P}}_{\mathrm{t}}$ of generation t, the children population ${\mathrm{G}}_{\mathrm{t}}$ generated by crossover mutation and the new individual ${\mathrm{T}}_{\mathrm{t}}$ generated by Tabu search are combined into the new population ${\mathrm{R}}_{\mathrm{t}}$. The non-dominated Pareto solution set with decreasing rank is obtained by fast non-dominated sorting of ${\mathrm{R}}_{\mathrm{t}}$, expressed as ${\mathrm{F}}_{1}, {\mathrm{F}}_{2}, \cdots$. The first N solutions are selected into the population ${\mathrm{P}}_{\mathrm{t}+1}$ until there are ${\mathrm{F}}_{1}+{\mathrm{F}}_{2}+\cdots +{\mathrm{F}}_{\mathrm{i}}\ge \mathrm{N}$, where ${\mathrm{F}}_{\mathrm{i}}$ is called the critical layer. Calculating the crowding distance of the critical layer, the individuals with high crowding degree is preferentially selected until the maximum number of iterations. The flow chart is shown in Figure 2.

3.1.   Calculation of crowding distance

The sorting mechanism of NSGA-Ⅱ algorithm is based on the calculation of crowding distance of individual resources. Crowding distance represents the density of resource solutions near a specific resource individual. In order to maintain diversity, individuals with large crowding distance at the same layer are preferentially selected. Although this strategy can retain individuals with low crowding degree, there is a risk that individuals with low crowding degree in the same area will be deleted at the same time.

In order to solve the above problems, the dynamical updating of crowding distance is adopted. The calculation process is as follows:

Step 1: Calculate the Euclidean distance between adjacent individuals. Let the crowding distance between the kth individual and the k+1th individual be:

where, $m$ is the target number, ${\mathrm{f}}_{\mathrm{i}}\left(\mathrm{k}\right)$ is the value of the kth individual on the ith target. ${\mathrm{f}}_{{\mathrm{i}}_{\mathrm{m}\mathrm{i}\mathrm{n}}}$ is the minimum value of the ith target, and ${\mathrm{f}}_{{\mathrm{i}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}$ is the maximum value of the ith target.

Step 2: Find the minimum crowding distance $I[k, k+1]$ and compare $I[k-1, k]$ with $I[k+1, k+2]$. If $I[K-1, k]$ is smaller, select individual $k+1$; otherwise, select individual $k$.

Step 3: Update the crowding distance.

Step 4: Determine whether the number of individuals is greater than or equal to $N$, otherwise return to Step 2.

3.2.   Crossover mutation operator

When the genetic algorithm approaches the termination generation, there will be duplicate individuals with good fitness in the mating pool, and the population is close to homogeneity. Although the mutation operation can improve this situation, traditional mutation operations would set a fixed value and the probability of mutation is very small. Even with the mutation operation, premature population convergence will occur.

The performance of genetic algorithm is closely related to the selection of crossover rate ${\mathrm{H}}_{\mathrm{c}}$ and mutation rate ${\mathrm{H}}_{\mathrm{m}}$. Therefore, an adaptive genetic algorithm (AGA) is proposed. The optimal crossover rate ${\mathrm{H}}_{\mathrm{c}}$ and mutation rate ${\mathrm{H}}_{\mathrm{m}}$ are allocated adaptively according to the adaptive level of individuals. This algorithm ensures the convergence of genetic algorithm and the diversity of population:

where, ${\mathrm{G}\mathrm{e}\mathrm{n}}_{\mathrm{a}\mathrm{v}\mathrm{g}}$ represents the mean remaining generations of individuals. ${\mathrm{G}\mathrm{e}\mathrm{n}}_{\mathrm{m}\mathrm{a}\mathrm{x}}$ and ${\mathrm{G}\mathrm{e}\mathrm{n}}_{\mathrm{m}\mathrm{i}\mathrm{n}}$ represent the minimum and maximum remaining generations of the population, respectively; ${\mathrm{G}\mathrm{e}\mathrm{n}}^{{{'}}}$ denotes the larger residual generation of the two individuals participating in the crossover; Gen expresses the remaining generations of individuals involved in the mutation; ${\mathrm{H}}_{{\mathrm{c}}_{\mathrm{m}\mathrm{i}\mathrm{n}}}$ and ${\mathrm{H}}_{{\mathrm{c}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}$ refer to the minimum and maximum crossover rates set by the algorithm, respectively. ${\mathrm{H}}_{{\mathrm{m}}_{\mathrm{m}\mathrm{i}\mathrm{n}}}$ and ${\mathrm{H}}_{{\mathrm{m}}_{\mathrm{m}\mathrm{a}\mathrm{x}}}$ are the minimum and maximum mutation rates set by the algorithm, respectively.

3.3.   Tabu search strategy

The candidate solutions generated by Tabu search algorithm during the search process are not random, which can accept sub-optimal solutions, so it has strong search ability. Therefore, the advantages of individual diversity and uniformity in NSGA-Ⅱ algorithm are retained while maintaining the genetic operator operation of NSGA-Ⅱ algorithm. The individual variables in this paper are generated from the neighborhood of current solution, rather than randomly generated. When the optimal solution meets the aspiration criterion, we have Pm = 1, and the current individual is added to the children population as a mutant; otherwise, Pm = 0.

The solution set after merging is set as ${\mathrm{R}}_{\mathrm{t}} = {\mathrm{P}}_{\mathrm{t}}\cup {\mathrm{G}}_{\mathrm{t}}\cup {\mathrm{T}}_{\mathrm{t}}$, where ${\mathrm{P}}_{\mathrm{t}}$ is the parent individual, ${\mathrm{G}}_{\mathrm{t}}$ is the individual obtained by genetic operator, and ${\mathrm{T}}_{\mathrm{t}}$ is the individual obtained by Tabu search operator. Secondly, the first $N$ individuals are selected as the next generation parent individuals ${\mathrm{R}}_{\mathrm{t}+1}$ by non-dominated sorting and crowding distance. Through this operation, the advantage of NSGA-Ⅱ algorithm that avoids losing excellent individuals in the parent generation can be retained and the local search ability can be improved, so as to reduce the number of duplicate individuals in the offspring and enhance the diversity of solution set.

The Tabu search strategy is as follows:

1) Initial solution

Tabu search algorithm is highly dependent on the initial solution, so the initial solution is randomly selected from the individuals whose parent individual non-dominated order equal to 1.

2) Neighborhood solution

During the initial iteration, the search scope should be enlarged to increase the probability of finding the optimal solution. As the iteration continues, the obtained solution gradually approaches the optimal solution, so the search scope should be narrowed to avoid repeated search. The objective functions (1), (3) and (12) are used as fitness functions to calculate the fitness values of each candidate solution.

3) Candidate solutions

For a specific resource ${\mathrm{L}\mathrm{O}}_{\mathrm{j}}$, it takes a lot of computation to calculate all the neighborhood solutions, so some strategies are required for choosing the candidate solutions. In this paper, a number of solutions N (X) are selected from a given set of candidate solutions by using non-dominated sorting and crowding operators.

4) Aspiration criterion

The solution with the best fitness is selected from the candidate solutions and compared with the current optimal solution. If it is better than the current optimal solution, the best candidate solution is applied to update the current optimal solution regardless of whether it is taboo or not. It is also used as the current solution for the next iteration. Then, the corresponding operation is inserted to the taboos table. If it is not better than the current optimal solution, the optimal solution that is not in the Tabu state is selected from all candidate solutions as the new current solution, and then the corresponding operation is added to the Tabu table.

4.   Experimental performance and analysis

In order to evaluate the efficiency of the proposed course personalization algorithm, three evolutionary algorithms (NSGAII, IBEA, PESA2) are compared with the proposed algorithm, so as to verify the effectiveness of the personalized learning resource recommendation proposed in this paper.

4.1.   Dataset

The study was based on 290 online courses from Harvard and MIT on the edX platform. The dataset contains 250,000 certifications, 4.5 million participants, and 28 million hours of participant data. The feasibility and accuracy of the proposed algorithm are verified on this dataset. The dataset of this online public course contains data like issuing institution, course number and release date, which fully meet the research requirements of this paper. Table 1 shows the description of parameters in the edx dataset, and Table 2 presents the information of four datasets generated by course subject according to the edx dataset, denoted as S1, S2, S3 and S4, respectively.

4.2.   Evaluation index

This section introduces the performance evaluation index of the algorithm, which mainly evaluates learners' satisfaction with the resource list. We determined the weight value through comparative analysis of several experiments, and then compared the accuracy, mean fitness, recall rate, F1 mean, HV, GD and IGD of each algorithm in different datasets.

1) Accuracy refers to the proportion of resources in a given resources recommendation list that can correctly predict the demand of learners. The computation formula of accuracy is as follows:

2) Mean fitness refers to the fitness between learners and learning resources. The mean fitness with different scales reflects the satisfaction of learners with the recommendation list. The computation formula is as follows:

3) Recall rate refers to the proportion of samples that accurately predict resources required by learners. The computation formula is as follows:

4) F1 value is used to better represent the accuracy rate and recall rate of learning resource recommendation. The computation formula is as follows:

5) GD measures the average minimum distance between each point in solution set P and reference set P. The smaller the GD value is, the better the convergence effect will be. The computation formula is as follows:

6) IGD measures the average distance between each reference point and the nearest solution. The smaller the IGD value is, the better the comprehensive performance of the algorithm is. The computation formula is as follows:

4.3.   Parameter verification experiment

The weight in Eq (1) is the weight factor to calculate the background fitness between learners and resources, which is used to control the proportion of age, gender and educational background when calculating the background similarity. In order to analyze the influence of weight ${\rm{ \mathsf{ ω} }}_{1}$ and ${\rm{ \mathsf{ ω} }}_{2}$ on the algorithm in this paper, we calculate the accuracy values under different values of weight ${\rm{ \mathsf{ ω} }}_{1}$ and ${\rm{ \mathsf{ ω} }}_{2}$ in the datasets with different sizes. By comparing the accuracy values under three different weight values in Figure 3, it can be seen that when ${\rm{ \mathsf{ ω} }}_{1}$ = 0.38 and ${\rm{ \mathsf{ ω} }}_{2}$ = 0.1, the fitness between learners and resources has nothing to do with gender, and the importance of age and education for learning resources is much higher than gender.

The weight in Eq (7) is the weight factor to calculate the popularity degree of learning resources, which is used to control the proportion of the learning participant percentage, viewing frequency and score when calculating the popularity degree. In order to analyze the influence of weight ${\rm{ \mathsf{ ω} }}_{3}$ and ${\rm{ \mathsf{ ω} }}_{4}$ on the algorithm in this paper, we calculate the accuracy values of resources under different values of weight ${\rm{ \mathsf{ ω} }}_{3}$ and ${\rm{ \mathsf{ ω} }}_{4}$ in datasets with different size. By comparing the accuracy values under three different weight values in Figure 4, it can be seen that when ${\rm{ \mathsf{ ω} }}_{3}$ = 0.47 and ${\rm{ \mathsf{ ω} }}_{4}$ = 0.3, the average popularity degree of learning resources is the highest.

The weight in Eq (10) is the weight factor to calculate the authority degree of learning resources, which is used to control the proportion of the influence of institutions and the influence of lecturers when calculating the authority degree of resources. In order to analyze the influence of weight ${\rm{ \mathsf{ ω} }}_{5}$ on the algorithm in this paper, we calculate the accuracy values of resources under different values of weight ${\rm{ \mathsf{ ω} }}_{5}$ in datasets with different sizes. By comparing the accuracy values under two different weight values in Figure 5, it can be seen that when ${\rm{ \mathsf{ ω} }}_{5}$ = 0.65, the average authority degree of learning resources is the highest.

The weight in Eq (12) is the weight factor to calculate the content quality of learning resources, which is used to control the proportion of popularity degree, authority degree and novelty degree when calculating the content quality of learning resources. In order to analyze the influence of weight ${\rm{ \mathsf{ ω} }}_{6}$ and ${\rm{ \mathsf{ ω} }}_{7}$ on the algorithm in this paper, we calculated the accuracy value of resources under different values of weight ${\rm{ \mathsf{ ω} }}_{6}$ and ${\rm{ \mathsf{ ω} }}_{7}$ in datasets with different sizes. By comparing the f accuracy values under three different weight values in Figure 6, it can be seen that when ${\rm{ \mathsf{ ω} }}_{6}$ = 0.22 and ${\rm{ \mathsf{ ω} }}_{7}$ = 0.52, the average accuracy value of learning resources is the highest.

4.4.   Performance verification

The parameter setting of the multi-objective algorithm is an important step, which has huge impact on the results. Therefore, we configured probabilities for the basic MOEA operations, including mutation and crossover, population size, and generation. Table 3 summarizes the parameter settings of four algorithms used in the experiment, and Figure 7 shows the Pareto frontier of a certain user.

Each point in Figure 7 represents the recommendation lists for target users. The algorithm is based on the tradeoff between learner fitness and resource diversity, which is contradictory. In order to verify the effectiveness of the algorithm, the proposed algorithm is compared with the other three algorithms. Table 4 shows the performance results of each algorithm in each dataset.

In this paper, 30 independent operation (iterative) experiments were performed to verify the efficiency of four algorithms in the generation of personalized learning resource recommendation list, so as to obtain the average accuracy of the model. For each iteration, we evaluated 10 different data partitions that were cross-validated.

Figure 8 demonstrates the accuracy results of the algorithm for generating personalization rules. The improved NSGA-Ⅱ provides optimal accuracy in 85 to 98%, with an average accuracy of 92%, slightly higher than the average accuracy of NSGA-Ⅱ algorithm. However, IBEA and PESA2 are less accurate.

The evaluation of multi-objective evolutionary algorithm needs to consider some evaluation indexes, so various performance indicators are introduced. This paper focuses on only three performance indicators, GD and IGD, to evaluate the efficiency of the algorithm. The IGD algorithm that provides minimum GD value is considered as the best algorithm.

Table 8 describes the results achieved when applying performance indicators. It can be seen that the improved NSGA-Ⅱ algorithm provides the minimum value of 0.010 and 0.065 for IGD and GD respectively. and its superior convergence and diversity are verified by calculating the distance between the individuals in the real individuals of non-dominated front. The indicator GD is used to determine the number of solutions placed at the intersection of the obtained Pareto frontier and the true Pareto frontier, which is achieved by calculating the distance between each recommended resource. To sum up, the improved NSGA-Ⅱ can generate a personalized resources recommendation list better than INSGA-Ⅱ, IBEA and PESA2 due to its good performance in indicators.

5.   Conclusions

Online learning is an emerging education method that aims to meet the preferences of each learner by engaging with them and providing personalized resources. Ultimately, the goal of personalized learning is to provide appropriate learning resources according to the demands of learners. Therefore, this paper utilizes the online learning resource recommendation model based on the improved NSGA-Ⅱ algorithm to transform the personalized screening of massive learning resources into an optimization problem, and defines two competing objectives. The quality of the resource list gets maximum improvement by maximizing the difficulty and the learning threshold. Besides, the credibility also gets enhanced by maximizing the popularity degree, authority degree, and novelty degree of each learning resource. The improved NSGA-Ⅱ algorithm takes the dataset, personalized characteristics of learner and the resource characteristic of learner as input. In the process of model construction, a large number of weights are set for the feature processing of the learning resource model in the early stage, which is not intelligent enough. In future work, we plan to add correlation researches between resources to build a model that can customize learning paths according to learners' preferences.

Acknowledgments

The research is supported by the National Natural Science Foundation of China (No. 12105120, No. 72174079 and No. 72101045), Natural Science Foundation of the Jiangsu Higher Education Institutions of China (No. 21KJB520033), Jiangsu Qinglan Project Teaching Group, Lianyungang "521 projects", The Research Project of Graduate Education and Teaching Reform of Jiangsu Ocean University (YJG202201).

Conflict of interest

The authors declare there is no conflict of interest.