Research on the evaluation system of operations behavioral norms for off-campus tutoring institutions based on AHP

1.   Introduction

Under China's current higher education system, the main criterion for education achievement is still the academic performance of students. As a result, to improve academic performance, students in the compulsory education stage participate in extracurricular tutoring at their own expense ^[1,2], resulting in a dual-track phenomenon of "on-campus classes and off-campus tutoring". However, in the process of off-campus tutoring, institutions cater to the increasingly diverse educational needs of students and parents ^[3,4]. Due to the capital attributes of off-campus tutoring institutions, the "exam-oriented" off-campus tutoring approach is inconsistent with the requirements of fostering virtue through quality education and all-round development, essentially violating the norms of healthy growth of young people. This has had a somewhat negative impact on primary and secondary school students and has interfered with the effective implementation of national education policies. If the behavior of off-campus tutoring institutions was not standardized, it would be detrimental to the healthy growth of primary and secondary school students and would endanger the promotion and equity of education. Off-campus tutoring institutions have had different ways of running schools, and their school administration behavior has been questioned. Issues that infringe upon the legitimate rights and interests of parents, such as disputes over refunds, false advertising and uneven quality of training products and services ^[5], have long been the focus of questions and inquiries by the public ^[6]. This kind of chaotic operations behavior that accompanies the development of the off-campus tutoring industry has not only increased the financial burden of parents and family anxiety about off-campus training ^[7] but has also not met the real expectations of parents about investing in their children's education ^[8].

According to the survey data from the Chinese Society of Education, approximately 70% of primary and secondary school students in first-tier cities such as Beijing, Shanghai, and Guangzhou participated in extracurricular tutoring before the implementation of the "Double Reduction" policy ^[9]. In 2016, the market size of the off-campus tutoring industry exceeded 800 billion, and the number of participating students reached 137 million ^[10]. The nonstandard school administration behaviors of off-campus tutoring institutions were generally concentrated in four aspects: a lack of basic safety guarantees in training locations, a lack of scientific basis of training content, a lack of unified job qualification requirements for training teachers and various training methods ^[11]. In addition to the basic nonnormative behaviors, teaching actions that violated the laws of education and teaching by the training institutions were a deep and acute problem, such as training activities that went beyond the teaching syllabus and advancing teaching and mechanical homework training ^[12].

In recent years, the norms and school management behavior of off-campus tutoring institutions have attracted the attention of experts and scholars. In-depth discussions on practice and theory have been carried out on how to convince the government to issue policies and regulate the school management of off-campus tutoring institutions, that are currently in disarray ^[13]. Based on different perspectives, most scholars have put forward suggestions to improve the school administration standards of off-campus tutoring institutions. For example, Haifeng Lin et al. proposed the strategy of eliminating the troubles in the governance of off-campus tutoring institutions based on the logic of multiple systems ^[14]. Subin Wang and Yiming Zhu's research found that a comprehensive governance model by parents, schools, society, government and institutions is required to standardize the school administration behavior of institutions and ensure the lasting effect of off-campus education ^[12]. Cheng Yang et al. emphasized the importance of governance by law and believed that when off-campus tutoring entered a cycle of benign development, legal governance was needed. Therefore, it is important to do a good job in the legislation, interpretation, popularization and law enforcement of off-campus tutoring governance and develop a legal training environment through the ideas and methods that reflect the rule of law ^[7].

With the support of the whole society, the State Council proposed focusing on establishing and improving the supervision mechanism of off-campus tutoring institutions and striving to build a long-term mechanism for the standardized and orderly development of off-campus tutoring institutions. It is important to adhere to strict governance and comprehensively standardize off-campus tutoring behavior ^[15]. Therefore, important issues such as regulating the behavior of off-campus tutoring institutions were quickly put on the government's agenda ^[16]. According to the "Double Reduction" data released by the Ministry of Education, the number of offline tutoring institutions for primary and middle school students has been slashed by 92 percent, from 124,000 to 9728, and online institutions have been cut by 87 percent, from 263 to 34. Even though some cities have reached 100%, the number of off-campus tutoring institutions offering academic tutoring services to primary and middle school students has been greatly reduced. All remaining tutoring institutions have been converted to nonprofit organizations, and all of their prepaid tutoring fees are under government supervision. The provinces and cities have formulated standards for fees for off-campus tutoring institutions that offer academic tutoring services to primary and middle school students. "Standardize tutoring service behavior, prohibit overstandard and advanced tutoring". The competent government department has defined a negative list of overstandard and advanced tutoring for primary- and middle-school students. At the same time, various localities and departments have rectified the of school management behavior of off-campus tutoring institutions in practice and have achieved some results ^[17]. However, what needs to be considered is that only a complete evaluation and monitoring system for behavioral norms of off-campus tutoring institutions can be established. The standardized development of off-campus education must take place through the strengthening of an evaluation and monitoring system of off-campus tutoring. Only then can the "Double Reduction" policy be truly implemented to promote the healthy and rapid development of the country's education mission.

Many countries have also undergone the same process in the management of management codes of conduct for off-campus tutoring institutions. These countries have also implemented a series of governance policies for off-campus tutoring in response to problems in the process of off-campus tutoring. In addition to general governance policies, it would include institutional registration and basic requirements, employment of practitioners, tuition fees, taxation, supervision, rewards and punishments ^[18]. The governments of various countries have formulated their own governance measures according to local conditions. Of them, the governance of off-campus tutoring in South Korea, Japan and the United States are the most representative. In South Korea, the off-campus tutoring industry first experienced a stage of prosperity, followed by a total ban, and then gradual liberalization. To control the development scale of off-campus tutoring, the Korean government has steadily improved the governance system by promoting public education, improving the quality of public school education, providing free private off-campus tutoring services, and establishing nonprofit after-school education websites ^[19,20]. The Japanese government has led the development of a three-pronged governance system featuring multidepartmental collaboration, legal supervision and industry self-discipline ^[21,22]. At the same time, off-campus tutoring institutions have received guidance on conducting educational cooperation with schools to provide more high-quality educational resources. The core of the American off-campus tutoring governance concept is to improve the quality of off-campus training through the intervention of the federal government and to develop a variety of off-campus tutoring types. For example, the federal government leads after-school education and advocates and organizes "voluntary tutoring" projects. The governance measures of off-campus tutoring in South Korea, Japan and the United States reflected the basic trend of after-school tutoring governance in Eastern and Western countries: the key to the governance of off-campus tutoring is to make more robust, rather than block, the top-down governance model led by the government, and the continuous refinement of off-campus tutoring governance.

In summary, efforts have been made at home and abroad on the issue of school administration behavioral norms for off-campus tutoring institutions. However, research on the governance of off-campus tutoring has mostly focused on macrolevel aspects such as governance measures, the division of responsibilities of departments and how to correct problems. Although these guiding reforms have had some significance, they were rather vague at the operational level. For example, although some researchers proposed a five-dimensional framework of analysis for the standardization and governance of off-campus tutoring institutions, they limited the discussions to the level of laws, regulations and competent authorities ^[23]. There was no mention of the norms of specific school management behavior. To accurately resolve the governance problems of off-campus tutoring institutions, some researchers proposed coordinated actions by logical stakeholders such as the government, tutoring institutions, teachers and the public. Although it involved the government's supervision of training institutions in regulating the operations of schools, it only put forward suggestions from the aspects of supervision mechanisms and supervision platforms ^[24] but did not study the specific school operational behavior to be monitored. For the competent departments of off-campus tutoring and their personnel, there was no suitable or feasible reference against which school operations would be checked, what the standards were, or how to judge the overall school administration behaviors of off-campus tutoring institutions. That is, at present, there is no mature monitoring and management index evaluation system on the issue of the code of conduct for off-campus tutoring institutions at home and abroad. In addition, there is still a lack of relevant theoretical research on the off-campus tutoring industry and off-campus tutoring institutions. However, it is worth noting that the evaluation of the school management code of conduct of off-campus tutoring institutions can provide a decision-making reference for the normative development of these institutions. In view of this, this research will construct a hierarchical structure model through the analytic hierarchy process (AHP) and calculate the index weights to construct an evaluation system of school administration norms for off-campus tutoring institutions. The AHP is an important theory of operations research. It decomposes decision-related factors into multiple goals and criteria and then combines qualitative and quantitative analysis to understand the impact of various goals and criteria on target factors. The AHP is suitable for dealing with decision-making problems, by identifying the influencing factors that affect the behavioral norms of off-campus tutoring institutions and designing these influencing factors into evaluation indicators at different levels. First-level, second-level and third-level indicators are constructed from different dimensions into a hierarchical structural model. This study uses AHP to construct an evaluation system of school operations behavioral norms for off-campus tutoring institutions to promote the establishment of relevant standards and to provide a reference for the governance and monitoring of off-campus tutoring.

2.   Research ideas and methods

2.1.   Research ideas

According to the dependence on the target means of monitoring the school operations behavior of the off-campus tutoring institution, the measurability of the self-efficacy of the practitioners and other behavioral characteristics ^[25], this study referred to the existing supervision tools and the system model construction and index weight calculation of the AHP. The result is the construction of a system of evaluation of school administration behavioral norms for off-campus tutoring institutions, which is composed of four dimensions: "operations foundation, organization construction, operations process, and operations results" ^[26].

The indicators of the four dimensions considered the entire process of the school operations behavior of off-campus tutoring institutions from the perspectives of occurrence, development, effectiveness and results. The underlying logic is guided by representational indicators, which deeply influence and regulate off-campus tutoring institutions, including their intentions, original intentions, starting points, goals and other value orientations. The construction and application of the evaluation system of school behavioral norms for off-campus tutoring institutions are to objectively reflect the standard of the institution's operations behavior, help rectify the disarray in off-campus training institutions, and promote the orderly development of the industry. According to the tomographic analysis method, the index weight is calculated and assigned, and the corresponding formula is established to calculate the performance score that captures the institution's behavior. Through this kind of accurate measurement of the institution's operations behavior, the opinions and requirements to correct the institution's future behavior and improve performance are proposed.

2.2.   Research methods

The AHP is a hierarchical weight decision analysis method proposed by American operations researchers and mathematician T. L. Saaty in the 1970s. AHP refers to taking a complex multiobjective decision-making problem as a system, decomposing the objective into multiple objectives or criteria, and then decomposing them into several levels of multiple indicators and criteria. Then, through analysis and research, the priority weight of each element at each level and to an element in the previous level is obtained, and the research method of the best solution is finally selected. The specific principle of AHP is to decompose the problem into different components according to the nature of the problem and the overall goal to be achieved. In addition, the factors are aggregated and combined at different levels according to the interrelated influence and affiliation between the factors to form a multilevel analysis structure model. Ultimately, the problem boils down to the determination of the relative importance weights of the lowest layer relative to the highest layer or the order of relative superiority and inferiority. According to this principle, we must first clarify the nature of the research problem, the influencing factors and its internal relationship, and decompose the systematic and complex problem that affects the standardization of off-campus tutoring institutions into several levels and elements.

The indicator weight is the expression of expert opinion and decision-making, which greatly affects the accuracy of multifactor decision-making. At the same time, the consistent matrix method proposed by Saaty et al. does not compare all factors together but uses relative scales to minimize the difficulty of comparing factors with different properties and improve accuracy. In other words, it compares each of the two factors and finally obtains the weight of the different elements of the solution target. In this way, less quantitative information can be used to digitize the decision-making process, thereby providing a simple and convenient decision-making method for complex problems with multicriteria or unstructured characteristics; it is also the most commonly used system evaluation method.

The AHP has the following advantages. First, it is a systematic approach to analysis. The AHP takes the research object as a system and makes decisions according to the thinking mode of decomposition, comparison and judgment, and synthesis. It has become an important tool for system analysis developed after mechanism analysis and statistical analysis. The second is a concise and practical decision-making method. It neither simply pursues advanced mathematics nor pays one-sided attention to behavior, logic, and reasoning but organically combines qualitative and quantitative methods to decompose complex systems and mathematicize and systematize people's thinking processes. Third, less quantitative data information is needed. It mainly starts from the evaluator's understanding of the nature and elements of the evaluation problem and is more stressful for qualitative analysis and judgment than the general quantitative method. It is precisely because of these advantages of AHP that it is widely used in academic research. For example, the construction of the evaluation index system for interschool balanced development in countries with compulsory education, the evaluation index system of the implementation effect of education poverty alleviation policy and the construction of the county basic education policy evaluation index system are all calculated and completed based on the stratified analysis method. Aiming at the research problem of constructing the evaluation system of operations behavioral norms for off-campus tutoring institutions, it involves many factors and is relatively complex, and it is difficult to quickly find an entry point to solve the problem. At the same time, the behavioral norms for off-campus tutoring institutions involve more qualitative factors than quantitative factors, and it is impossible to explain everything with simple numbers. However, the advantages of AHP are that it can solve these difficulties and provide the optimal solution. Therefore, AHP is a very suitable research method for constructing an evaluation system of school administration behavioral norms for off-campus tutoring institutions. Of course, AHP has two drawbacks. One is that there are few quantitative data and many qualitative components, which are not convincing; the other is that when there are too many indicators, the data statistics are large, and the weights are difficult to determine. In the research process, researchers have focused on the above two points and have tried to choose quantitative data or quantify the research questions to improve the scientific nature of the research. In addition, researchers have strictly controlled the number of secondary and tertiary indicators and repeatedly calculated and verified the weights of the indicators to ensure the accuracy of the weight values.

The specific steps in the AHP involve decomposing the decision-making problem into different hierarchical structures in the order of the overall objective, subobjectives, evaluation criteria and specific investment plans. Then, by solving the eigenvector of the judgment matrix, the priority weight of each element of each level to an element of the previous level is obtained. Finally, the reweighted sum method hierarchically merges the final weights of the alternatives to the overall goal. To use AHP, a hierarchical structure must be established to analyze complex hierarchical problems. At the theoretical level, through the construction of an evaluation system of school management behavioral norms for off-campus tutoring institutions, the overall goal of "evaluation of the degree of behavioral norms of off-campus tutoring institutions" is decomposed into different operable components. Then, each element is hierarchically reorganized according to the degree of association and affiliation. Finally, a multilevel analysis model consisting of "4 first-level indicators, 11 second-level indicators and 34 third-level indicators" is established.

Construction of the evaluation index system:

To construct an evaluation system of off-campus tutoring institutions, we must fully consider the reality of their operations behavior and the objective requirements of social development. It is necessary to take into account the commonalities between off-campus tutoring institutions and formal school education, as well as the special features of the off-campus tutoring institutions. What they have in common is that off-campus tutoring institutions follow the educational purpose, teaching methods, and development goals of school education. There are 4 special features of these institutions. First, off-campus tutoring institutions are basically nonprofit and for-profit institutions organized by social forces. Second, the tutoring institutes are essentially rented houses, and the basic security needs to be scientifically evaluated and judged. Third, in terms of teacher recruitment, off-campus tutoring institutions do not have a fixed source channel, making it difficult to form a stable labor-management relationship. This makes it difficult to unify teacher standards, teacher training, and teacher assessment. Fourth, the tutoring content, charging standards, and tutoring forms vary greatly among the institutions, and the quality of tutoring services is difficult to measure. In response to these problems, it is necessary to formulate a set of indicators that evaluate the performance of off-campus tutoring institutions based on the requirements of policies and regulations established for their governance. Only by making a procedural judgment on the degree of school operations behavior norms of off-campus tutoring institutions can we effectively supervise and regulate them. Real-time analysis and judgment of whether the administration of off-campus tutoring institutions is standardized, which functions are standardized, and which behaviors are yet to be standardized are achieved by comparing the actual situation of the whole process of running an off-campus tutoring institution with what is required by the government to provide a basis for improving supervision and regulations. According to the design of the above research method path, a corresponding hierarchical research structure model is established. The specific model is shown in Figure 1.

1) First-level indicators

Combined with the evaluation characteristics and needs of the degree of behavioral norms of off-campus tutoring institutions, this study uses the CIPP evaluation model as the theoretical basis for the selection of indicators. The evaluation follows the four elements of "background-input-process-result" to construct an evaluation index system ^[27]. These four elements reflect the internal mechanism and logic mechanism of the evaluation of the degree of operations behavior norms ^[28]. To evaluate the standard of behavior of off-campus tutoring institutions, it is necessary to evaluate every important link covered by these institutions. It is necessary to pay attention not only to the foundation of the institution but also to the process of running the institution. It is necessary to pay attention not only to the organizational construction of the institution but also to the operational results of the institution. On the basis of clarifying the evaluation objectives and the construction ideas of the evaluation index system, the four principles of systematicness, objectivity, operability and universality of the index system construction are followed. Combined with policy texts and references, this paper determines four first-level indicators of "operations foundation—organizational construction—operations process—operations results" (see Figure 1).

① Operations foundation. The foundation of running a school is the most basic requirement for running an off-campus training institution. Basic conditions, such as ideas, facilities and personnel, must meet the standards for running a school ^[29]. Regulating the operations behavior of off-campus tutoring institutions must be checked from the beginning of the establishment of the school, and the threshold for operating off-campus tutoring institutions must be strictly observed ^[30] to eliminate nonstandard behavior. The establishment and specification of the basic indicators of running a school require a basic positive orientation from the original intention of operating a school, the direction of running a school, humanistic feelings, and economic strength. From the perspective of the standardized operation of off-campus tutoring institutions, the foundation of running schools plays a key role in the follow-up operations and development of tutoring institutions, and a good foundation for running schools is an important guarantee for the smooth development of enrollment, teaching and other functions. Referring to the requirements of the operations standards for primary and secondary schools, the operations foundation can be subdivided into directions, hardware facilities and institutional personnel.

② Organizational construction. Organization construction entails the behavior and software construction based on the hardware construction of operations foundations. Specifically, the order and regulations established for off-campus tutoring institutions and institutional practitioners ensure that institutional operations are planned and will run smoothly. ^[31]. Without rules, the orderly development of various functions of off-campus tutoring institutions would require a scientific and effective mechanism to understand and coordinate the overall direction of the institutions. Tutoring institutions as a whole organization, in addition to the basic functions of student learning and teacher teaching, involve system construction to ensure the long-term development of the institution and management of daily operations. Therefore, combined with the current practices and the supervision content of the competent department, "organizational construction" is selected as a first-level indicator, and three second-level indicators of "institutional system", "teaching system" and "management system" are established.

③ Operations process. The process of running a school entails all the behaviors involved in every aspect of operating off-campus tutoring institutions, from enrollment to teaching to achieving goals. The process of running a school is the core manifestation of the degree of standardization of off-campus tutoring institutions, and it is the key to considering whether the evaluation index system of the degree of standardization is scientific, reasonable and correctly oriented. The operating a school represents the most direct contact with students and parents, has the closest relationship with the rights and interests of students and parents and is the most problematic aspect of off-campus tutoring institutions ^[32]. Therefore, the standardization of the operations process is the key guarantee of the long-term development of off-campus tutoring institutions. Off-campus tutoring institutions currently have the following characteristics. First, unlike public primary and secondary schools in the compulsory education stage, off-campus tutoring institutions do not have a fixed source of students and funding. These institutions mainly attract students in the form of programs and other features that cater to the needs of parents and students. Second, due to the lack of trust between institutions and students and their parents, when students sign up to participate in tutoring services, the rights and interests of both parties generally involve written contracts to clarify specific responsibilities and rights. Third, because the teaching content of off-campus tutoring institutions is considered supplementary to formal school education, the quality of the training services they provide is difficult to measure centrally and standardize. Therefore, it is necessary to assess whether these institutions follow the laws of education and teaching and of the healthy development of young people from a cost-benefit perspective and whether they have complemented and strengthened school education. Feedback from investors in off-campus education should be followed. Therefore, "the process of operating a school" is considered a first-level indicator from the perspective of qualitative research and has three second-level indicators: "admissions enrollment", "classroom teaching" and "tutoring feedback".

④ Performance results. The results of running a school are the most direct manifestation of the value of tutoring activities carried out by off-campus institutions; assessing results is also the fundamental purpose of evaluating the degree of meeting operations behavioral norms. The "performance results" is also the core factor that determines the long-term development of the off-campus tutoring industry. After an off-campus tutoring institution has been in operation for a certain period of time, students, parents and education authorities have the criteria to judge its performance. From the perspective of evaluation subject and content, this kind of evaluation can be divided into the evaluation of the competent education department and the word-of-mouth evaluation by society mainly based on student and parent opinions. Based on this, the "performance results" is a first-level indicator, which includes the two second-level indicators of "supervision benefit" and "social benefit".

2) The second-and third-level indicators

After the first-level indicators are determined, based on the current situation of off-campus tutoring governance and supervision and the actual situation of off-campus training institutions, a third-level index for evaluating the degree of behavioral norms for off-campus tutoring institutions is constructed, taking into account the key difficulties in the process of institutional governance and supervision and the requirements of the "Double Reduction" policy, combined with expert suggestions.

① Under the first-level operations foundation indicators, three second-level indicators are set up: "operations direction", "hardware facilities" and "institutional personnel". Of them, the "operations direction" mainly examines the implementation of the Party's education policy, the implementation of quality education standards and fostering of virtue through education. The "hardware facilities" mainly examine the construction and configuration of hardware facilities of the institution, considering whether the average teaching space per student and the teaching equipment meet the teaching needs as representative indicators. The "institutional personnel" indicator mainly examines the establishment of standardized labor relations between institutions and employees, as well as the qualifications and the structural situation of employees.

② Under the first-level indicator of organizational construction, three second-level indicators are established: "institutional system", "teaching system" and "management system". Of them, the "institutional system" mainly examines whether the institution has established and implemented rules and regulations to ensure operations and whether the institution has standardized and posted the certificates and teaching staff information that need to be publicized. The "teaching system" mainly examines the guarantee of teaching quality and selects the complete teaching plan, the development of teaching and research activities, and the teacher education and training as representative indicators. The "management system" mainly examines fire safety compliance, training material specifications, training time and duration compliance, precharge supervision and financial details, and file and student information management.

③ Under the first-level indicator of the operations process, three second-level indicators are set up: "admissions enrollment", "classroom teaching" and "tutoring feedback". Of them, "admissions enrollment" mainly examines whether tutoring institutions and consumers have established an equal partnership on the basis of truthfulness, honesty, openness and transparency. The indicators consider the content and usage of the Service Contract, whether the charging standard is within the range of the government's guide on prices, the refund standard and process clarity, and false advertising. Classroom teaching is the central part of institutional training services. The examination points focus on whether the institutional teaching follows general teaching principles, the teaching progress is consistent with that of local schools, the teaching content does not exceed standards, and the teaching features are rich. "Tutoring feedback" mainly examines the feedback on the tutoring service experience by consumers, namely, the students and their parents. The main indicators of consideration are the satisfaction rate of students, the satisfaction rate of parents, and the smoothness of the feedback channels and procedures.

④ Under the first-level indicators of operations results, two second-level indicator layers are set up: "supervision benefit" and "social benefit". Since the establishment of off-campus tutoring institutions, competent government departments have carried out supervision and various inspections of their institutions operations. This inspection result is a measure of whether the institutions meet operations behavioral norms. Therefore, there are good reasons to equate the qualification of the supervision inspection and special sampling inspection and the situation of whitelists as supervision benefits. At the same time, taking into account the feelings that the government's supervision and inspection evoke in the employees of the institution, it is necessary to increase the awareness of practitioners to regulate the main aspects of running a school. The "social benefit" mainly examines the social reputation and industry image of off-campus tutoring institutions. The most distinctive sign is whether the students participating in the training have recruited new students to the institution and whether the training institution has actively participated in social welfare activities. Therefore, two indicators, the student recommendation ratio and participation in volunteer service, are selected as the indicators.

Calculation process of the AHP:

The importance of each consideration index is expressed as the index weight value in the research. At the practical level, the weight of each indicator is measured. The relative importance of the specific scheme layer relative to the criterion layer index is ranked according to the weight value. Three steps are required to calculate the indicator weight value.

1) Constructing a judgment matrix for pairwise comparison

The evaluation system of operations behavioral norms for off-campus tutoring institutions is relatively complex, the evaluation of the degree of meeting behavior norms involves many factors, and the weight value is difficult to obtain directly. It is necessary to gradually judge the importance of the indicators through pairwise comparison. Therefore, to avoid subjective assumptions and value preferences of the researchers themselves, this study selected 10 experts to collect their scores on the importance of the two elements in the form of questionnaires and used a scale of 1–9 to assign the importance of the indicators. Finally, according to the geometric mean of the questionnaire data, a judgment matrix for pairwise comparison is constructed at each level, such as the criterion layer, the element layer, and the scheme layer (see Table 1).

If the upper level is denoted as A and the next level is B, then Table 2 below shows the relevance of the B-th layer index relative to the A-th layer. For A, the relative importance of indicators ${B}_{i}$ and ${\mathrm{B}}_{\mathrm{j}}$ is assigned as ${\mathrm{b}}_{\mathrm{i}\mathrm{j}}$, and ${\mathrm{b}}_{\mathrm{i}\mathrm{j}}\ne \frac{1}{{\mathrm{b}}_{\mathrm{j}\mathrm{i}}}$ is satisfied.

2) Indicator relative weight vector and consistency test

To ensure the desirability and validity of the calculated weight values, it is necessary to carry out a consistency check on the matrix to check whether the decision-maker's thinking process is consistent.

Calculate the product of the elements of each row in matrix A-B; the formula is:

Calculate the nth root of each element to obtain a new matrix ${(\overline{{w}_{1}}, \overline{\overline{{w}_{2}}}, \dots, \overline{{w}_{n}})}^{T}$, of which:

The matrix ${(\overline{{w}_{1}}, \overline{\overline{{w}_{2}}}, \dots, \overline{{w}_{n}})}^{T}$ is normalized to obtain the feature (weight) vector $W$ of each factor. The calculation formula is:

Calculate the largest eigenroot of matrix $W$; the formula is:

After that, calculate the consistency index $CI$, and the formula is:

In the formula, n is the order of the comparison matrix.

Then, the consistency ratio $CR$ is calculated using the following formula:

In the formula, $RI$ is the random consistency index, which varies with the order of the judgment matrix (see Table 3).

If $CR$ < 0.1, it proves that the judgment matrix has good consistency. The smaller the $CR$ value is, the better the consistency of the judgment matrix.

3) The evaluation score of the degree of code of conduct for running schools in off-campus tutoring institutions

The evaluation scores for the degree of the operations code of conduct for off-campus tutoring institutions are:

In the formula, $p$ is the evaluation score of the management code of conduct of off-campus education and training institutions, ${w}_{i}$ is the weight of each item in the criterion layer, ${w}_{ij}$ is the weight of each indicator in the indicator layer, and ${x}_{ij}$ is the value of each indicator in the indicator layer.

3.   Results

3.1.   Basic information of respondents

According to the research on this topic, pedagogical experts from universities and research institutions were invited to assist in filling out the questionnaire. The selection criteria for experts are: ① Working experience more than 10 years; ② Deputy senior and above titles; ③ Engaging in educational research; ④ Able to actively complete questionnaires. A total of 32 experts from Shandong, Zhejiang, Jiangsu, Liaoning, and Tibet were surveyed in this study, and 32 questionnaires were returned. Of them, there are 11 professors of education, 7 associate professors from universities, 8 researchers and 6 associate researchers from research institutions. Of the invited experts, 21 were male and 11 were female, accounting for 65.63 and 34.37%, respectively. Their average tenure is 14.11 years.

3.2.   Judgment matrix

According to the index system, the abovementioned scaling method is used, and experts in the field are selected through the questionnaire survey of the expert consultation method to score the importance of the indicators. According to each expert's scoring of the judgment matrix for the pairwise comparison of the indicators, the scoring results are then discussed and summarized internally, and the pairwise judgment matrix is obtained as shown in Tables 4–19.

The maximum eigenvalue of the judgment matrix of operations foundation, organizational construction, operations process, and operations results ${\lambda }_{max}$ = 0.0472, $CI$ = 0.0157, $CR$ = 0.0175 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the operations foundation${\lambda }_{max}$ = 3.0809, $CI$ = 0.0157, $CR$ = 0.0175 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the organizational construction ${\lambda }_{max}$ = 3.0292, $CI$ = 0.0146, $CR$ = 0.0252 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the operations process${\lambda }_{max}$ = 3, $CI$ = 0, $CR$ = 0 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the operations results${\lambda }_{max}$ = 2.

The maximum eigenvalue of the judgment matrix of each index of the operation directions ${\lambda }_{max}$ = 3, $CI$ = 0, $CR$ = 0 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the hardware facilities${\lambda }_{max}$ = 2

The maximum eigenvalue of the judgment matrix of each index of the institutional personnel ${\lambda }_{max}$ = 3.0292, $CI$ = 0.0146, $CR$ = 0.0252 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the institutional system${\lambda }_{max}$ = 2.

The maximum eigenvalue of the judgment matrix of each index of the teaching system${\lambda }_{max}$ = 3, $CI$ = 0, $CR$ = 0 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the management system${\lambda }_{max}$ = 5.3366, $CI$ = 0.0841, $CR$ = 0.0751 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of admissions enrollment ${\lambda }_{max}$ = 4, $CI$ = 0, $CR$ = 0 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of classroom teaching ${\lambda }_{max}$ = 4.0329, $CI$ = 0.0110, $CR$ = 0.0122 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the tutoring feedback ${\lambda }_{max}$ = 3, $CI$ = 0, $CR$ = 0 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the supervision benefit${\lambda }_{max}$ = 3.0292, $CI$ = 0.0146, $CR$ = 0.0252 < 0.10.

The maximum eigenvalue of the judgment matrix of each index of the social benefit${\lambda }_{max}$ = 2.

3.3.   Indicator weights

As shown in Table 20, the weight of each indicator is calculated by using the AHP. The weight of "operations results" is 0.455, which is the most important indicator in the evaluation of the operation codes of conduct. The lowest weight value is that of "operations foundation", which is only 0.078. To a certain extent, this reflects that the "operations results" are more representative of the degree of behavioral norms of off-campus tutoring institutions than the "operations foundation". This also indicates that the approval of off-campus tutoring is only the starting point of supervision. At the same time, this is also in line with the procedural and dynamic characteristics of evaluation. The relative weights of the five indicators of level three, such as "the ratio of student recommendation", "the awareness of practitioners to regulate running a school", "the satisfaction rate of students with tutoring services", "the satisfaction rate of parents with tutoring services" and "participation in volunteer services", are ranked in the top five. They have a greater impact on the degree of operations of off-campus tutoring institutions.

4.   Discussion

Based on the above research and analysis results, the weight assignments in the four first-level indicators are, from high to low, "operations results", "operations process", "operations foundation" and "organizational construction".

The weight of "operations results" ranks first among the four first-level indicators. Of them, "the ratio of student recommendations" under "social benefits" and "the awareness of practitioners to regulate running a school" under "supervision benefits" have the highest weights among the 34 third-level indicators. Specifically, "the ratio of student recommendation" better reflects the recognition and word-of-mouth of students and parents who have participated in off-campus tutoring. Students and parents who have participated in off-campus tutoring assess the training service based on their real experience and then judge whether it is worth recommending to other students or parents. This kind of feedback similar to the user's immersive experience not only objectively reflects the degree of meeting behavioral norms of off-campus tutoring institutions but also reduces the probability of off-campus tutoring prospective participants "stepping into the pit". At the same time, this is also positive feedback and an incentive for the standardized off-campus tutoring institutions. "The awareness of practitioners to regulate running a school" is the embodiment of the governance objective of complying with the relevant policies of off-campus tutoring, and it is also the acceptance and recognition of the standardization of off-campus tutoring. Off-campus tutoring practitioners maintain the code of conduct of off-campus tutoring institutions conceptually and practice it in action. Furthermore, they made every effort to improve the nonstandard operations behavior in off-campus tutoring institutions and improve the standardization of operations behavior. Therefore, as far as "social benefits" are concerned, it is required to guide off-campus tutoring institutions to compete and evaluate the level of operations through the examination indicators. At the same time, they pay attention to the rewards and publicity of off-campus tutoring institutions with a high degree of meeting operations behavioral norms. In terms of "supervisory benefits", the guidance and reinforcement of off-campus tutoring practitioners' standardized operations concepts and awareness should be strengthened, and seminars and salons such as centralized training seminars should be regularly held to continuously improve the awareness of operations behavior. Off-campus tutoring institutions are required to inculcate the responsibility of individual practitioners to meet the required operations behavior.

The weight of "operations process" ranks second among the four first-level indicators, second only to "operations results". "The satisfaction rate of students with tutoring services" and "the satisfaction rate of parents with tutoring services" under "tutoring feedback" have higher weights. This is consistent with the conclusion that the "the ratio of student recommendation" has the highest weight under "social benefit". Second, there is "the teaching progress being consistent with that of local schools" under "classroom teaching", which refers to the content that the current education authorities focus on and rectify and is consistent with the current governance goals of off-campus tutoring institutions. In addition, the sum of the weights of "operations results" and "operations process" is greater than 0.8, which is a key first-level indicator for evaluating the degree of operations behavioral norms of off-campus tutoring institutions and is also an important part of their regulation and supervision. Although a number of regulations have been developed around the two in policy documents, they are still fragmented and have not yet formed a system. We should build systematic and standardized supervision content and effectively improve the standard of operations behavior of off-campus tutoring institutions.

Although the weights of "operations foundation" and "organization construction" are slightly lower, it does not mean that they are not important, especially the secondary and tertiary indicators they contain. "Operations directions" under "operations foundation" and "teaching system" under "organization construction" have higher weights. As an auxiliary institution of school education, off-campus tutoring institutions are also important in terms of educational goals and methods. Of the third-level indicators, the comprehensive weights of "the structural situation of institutional employees" under "institutional personnel" and "the complete teaching plan", "the development of teaching and research activities" and "the situation of teacher education and training" under "teaching system" are relatively high. These four third-level indicators represent the quality of teachers in off-campus tutoring institutions. The education plan is teacher-oriented, and the same is true for off-campus tutoring institutions. At the beginning of the establishment of off-campus tutoring institutions, it is necessary to strictly control the qualifications and matching of teachers and continuously improve the quality of teachers' teaching through training, collective lesson preparation, lectures and other forms.

5.   Conclusions

Based on the survey on the operations behavior of off-campus tutoring institutions, this research uses the AHP to construct a more scientific and easy-to-operate evaluation system of operations behavioral norms for these institutions. By embodying the evaluation target of the normative degree of operations behavior of off-campus tutoring institutions and then decomposing it, the abstract target is broken down into multiple monitorable indicators. Finally, an evaluation system of operations behavioral norms for off-campus tutoring institutions is constructed, which includes 4 first-level indicators, 11 second-level indicators and 34 third-level indicators. At the same time, we incorporated the practical experience and suggestions of the group of experts. The constructed evaluation system of operations behavioral norms for off-campus tutoring institutions fully reflects the institutional characteristics and the degree with which they meet norms. This evaluation system of operations behavioral norms for off-campus tutoring institutions has been proven to be scientific and applicable.

Operating an off-campus tutoring institution is complex and involves much content. If only quantitative or qualitative indicators are used to conduct an evaluation, the results are not comprehensive. Therefore, the index system should be constructed so as to adhere to the principle of combining quantitative and qualitative to measure and evaluate the degree of meeting behavioral norms of off-campus tutoring institutions more scientifically and effectively. The significance of this study lies in that while a qualitative analysis of the relevant indicators is carried out, what cannot be achieved by qualitative analysis can be converted into quantifiable results to achieve the target results of the analysis and research. As far as this research is concerned, the evaluation index system is used to comprehensively evaluate the normative degree of operations behavior of off-campus tutoring institutions from multiple dimensions. This not only helps to determine the causes of various problems from different dimensions and helps to implement improvement measures for different problems but also improves the standard of operations behavior and optimizes the standardized development of off-campus tutoring institutions. Importantly, off-campus tutoring institutions can participate in the improvement and direct intervention to achieve self-control and regulation through this method. Through the implementation of standardized evaluation index system monitoring, off-campus tutoring institutions can determine deficiencies in their operations behaviors in a targeted manner. Due to the lack of current data, there are still many difficulties in the comprehensive evaluation of the operations behavioral norms of off-campus tutoring institutions. The scientific nature of the evaluation index system and monitoring index designed in this study needs to be further studied and discussed, but the value of this study to this field of research is unquestionable.

In terms of the application of the research results, first, it is recommended that local education authorities further establish and improve data platforms and related mechanisms for off-campus tutoring. We also established and improved the basic database of our country's off-campus tutoring to make up for the limitations and deficiencies in the statistical reports. Second, experts and scholars in the industry should further study the key indicators commonly used in the evaluation of operational codes of conduct for off-campus tutoring institutions. In addition, they should develop reasonable regulations for the qualitative evaluation indicators of behavioral norms of off-campus tutoring institutions and improve the theoretical monitoring results. Third, they should regularly publish research reports on the monitoring of the degree of meeting operations behavioral norms of off-campus tutoring institutions. This can provide a scientific basis for the government and education authorities to formulate planning and decision-making about off-campus tutoring, lay a solid foundation for the next steps in making decisions related to off-campus tutoring institutions, and provide support for improving the standards of conduct of off-campus tutoring institutions.

Acknowledgements

This study was supported by the Key Project of Chinese National Social Science Foundation (Grant No. AFA190018).

Conflict of interest

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.