Span-based model for overlapping entity recognition and multi-relations classification in the food domain

1.   Introduction

Considering the food field, the number of electronic texts has increased owing to the rapid growth of computer technology and the response to changes in people's dietary habits. Texts in the food field contain a number of precious knowledge. For example, considering the literature, we can learn what microorganisms are used for in food preservation. Moreover, regarding recipes, we can know how a dish is prepared, etc. This study is a completion of the project of the 'Construction of Knowledge Graph for Safety Risk Analysis and Identification of Food Ingredient for the Beijing 2022 Olympic and Paralympic Winter Games'. We consider Chinese dishes as examples to show more details. Chinese dishes are rich in ingredients and complex in preparation, and the recipes contain a wealth of information. The dish name may even contain information about ingredients and cooking methods. An entity can be the subject or object of multiple entity pairs in the same sentence. The aim of IE is to extract triple facts from a given sentence, and Figure 1 provides an example from the Recipe-IE dataset. The triple facts are in the form of (subject, relation, object) or (head entity, relation, tail entity). However, manually annotating useful information from a number of texts requires an enormous amount of labor and is time-consuming ^[1]. Deep learning (DL), a branch of machine learning (ML), supports more effective extraction of entities and their relations because of its ability to learn representations of data via multiple processing layers ^[2]. Deep learning-based IE is a process of extracting structured knowledge from unstructured text. This is usually composed of two subtasks: 1) recognizing named entity from raw unstructured text and predicting which pre-defined entity category they belong to ^[3]; 2) identifying the relationship between pairs of entities mentioned and classifying them into predefined relation categories ^[4].

Conventional approaches train one model to extract entities and another model to determine the relationship between them. This is known as the pipeline approach. This method is easy to implement, and the two models have high flexibility. From 2014 (Miwa and Sasaji ^[5]), the most promising end-to-end approach has been used to jointly model these two tasks. The two tasks make predictions in a unified structural framework or share representations. Although joint models can be used to mitigate error propagation issues, it does not mean that it always has better performance than the pipeline method. One main argument about why the joint model is superior to the pipeline model is that its capable of capturing the interactions between entities and relations. However, in 2019, Zhong et al. ^[6] used a pipeline approach for the IE task and discovered that entity information is significant for relation prediction, while they have not found that the relation model can improve the entity model. In addition, the two tasks need different formats and features to predict entities and relations, while the joint model shares the encoder, which cannot better learn task-specific features.

In this study, a pipeline model known as SpIE is presented to handle IE from complex Chinese texts. The SpIE models all possible spans and uses span representation to perform entity recognition on all the spans in parallel. The fine-tuned span representation (with added entity mentions and type information) was subsequently used to perform the relation classification on the pair of recognized entities. The contributions of our study are summarized below.

● This study offers a novel approach for complex information extraction from unstructured Chinese texts, using a suitable language model to generate span representation for overlapping entity recognition and multi-relations classification. Considering the Beijing 2022 Olympic and Paralympic Winter Games, we established a new dataset known as Recipe-IE.

● The SpIE considers the impact of its own entity attributes and innovatively feeds the fusion features of entity mentions and types into the RC task.

● The experimental results on the publicly DuIE dataset and the self-constructed Recipe-IE dataset demonstrate that SpIE markedly outperforms the traditional neural approaches for complex IE and remains competitive in normal IE tasks.

2.   Related work

Information extraction is primarily oriented towards open linked data and extracts usable triples (knowledge units) through automated techniques. When a sentence with well-labeled entities and relations are provided, the goal of IE is to obtain triples from the sentence. These triples are the basis for a series of high-quality factual representations to further build the upper-level model. They also support various downstream natural language processing tasks, such as query answering ^[7] and recommendation systems ^[8]. Recently, the construction of knowledge graphs in vertical fields ^{[9,10,11,12,13]}, such as the biomedical domain and the field of urban traffic, has shown great application prospects and commercial value.

Manually annotating knowledge from various unstructured texts based on predefined rules ^[14] is labor-intensive, expensive, and difficult to adapt to data changes. Several existing approaches use traditional ML-based models, such as conditional random fields (CRF) and hidden Markov models (HMM), to handle this task, which rely on hand-crafted rules. These ML-based models are too inflexible to scale into a new dataset. To overcome this shortcoming, DL-based approaches have been proposed to capture features automatically because of their ability to learn representations ^[15]. When raw data are fed to the machine, it can automatically discover the potential representations needed for classification or detection ^[16]. We divided the literature on the application of DL-based approaches to the IE task. In this study, we classify the existing studies on IE into four major categories: table-filling, multi-task learning, sequence labeling, and span-based approaches.

2.1.   Table-filling

Miwa et al. ^[5] and Gupta et al. ^[17] transformed NER and RC tasks into a simple table-filling problem and modeled these two tasks jointly through table representation. The main difference between the two is that the former used a history-based entity and relation table, and the latter labeled each word pair using a bidirectional recurrent neural network (Bi-CNN).

2.2.   Multi-task learning

This family of models is essentially the same as the pipeline method. Bekoulis et al. ^[18] modeled the NER and RC tasks as multi-head selection problems, which did not require external NLP tools. The contextualized representation of each word was generated from a long short-term memory (LSTM) layer. Miwa and Bansal ^[19] used a Bi-LSTM to tag the entities using a begin inside-last outside unit (BI-LOU) scheme and a bidirectional tree-structured RNN model for relation extraction. The attention mechanism can also be introduced in joint models. Furthermore, Nguyen and Verspoor ^[20] adopted a pipeline approach. A Bi-LSTM-CRF-based model for NER and a bi-affine attention mechanism was presented for RC. The aforementioned approaches relied heavily on the LSTM and its variants.

2.3.   Sequence labeling

Sequence labeling requires additional features, which is essentially a distributed representation for input. Zheng et al. ^[21] proposed an end-to-end model with tagging scheme for IE and experimental results showed the effectiveness of their method. Ma et al. ^[22] introduced a NN architecture using the combination of LSTM, CNN and CRF, which required no feature engineering or data processing. Cao et al. ^[23] and Lv et al.^[24] adopted a Bi-LSTM-CRF model to extract information from biomedical and software engineering context respectively. The difference between the two is that the former based on the BIOES (Begin, Inside, Outside, End, Single) scheme with an additional M tag to indicate the relation type that the entity took part in whereas the later extract information from multi-source software knowledge. Extra knowledge has also been used for IE: Zhou et al. ^[25] introduced an attention-based Bi-LSTM model to acquire the semantic information from sentences without NLP tools. The attention layer was used to generate a weight vector, and then merged each word-level feature into a sentence-level feature vector in order. Li et al. ^[26] proposed a multi-grained information and external linguistic knowledge for Chinese information extraction. Polysemy and ambiguity will be avoided by introducing external linguistic knowledge. Sequence labeling can also be used for key-phrase extraction ^[27] from scholarly documents, which is similar to IE.

2.4.   Span-based approaches

The above approaches tackle both entity and relation extraction as sequence labeling tasks which are processed by token-level models with the begin inside outside (BIO) scheme or its variant. However, this task is still challenging because the structure of Chinese texts is more complex, and the texts are rich in information. For example, "腊肉" in entities "豉椒蒸腊肉" (an subject) and "腊肉" (an object) has the same token representation, this situation with overlapping entity cannot be handled well. Here, we propose a span-based model that is beneficial for covering overlapping entity by providing different representations for all spans. The idea of using span representations to replace the token-level is not entirely new because it has been studied in neural coreference resolution ^[28]. Recently, the idea of span representation has been improved by the DYGIE model ^[29]. It introduced a general architecture for IE tasks and captured the interaction of spans by using dynamically constructed span graphs to update the span representation. The DYGIE++ (Wadden et al.) ^[30] replaced LSTM encoding with BERT-based representations to capture the semantic information within and across sentences. Different from DYGIE++, PURE ^[6] used separate encoders for both NER and RC model, and took the entity labels gotten from NER model as the input feature. PURE reported better performance than previous work. However, this approach is not suitable for Chinese and ignores the role of entity attributes in predicting the relation between two entities. We propose an improved span-based model for Chinese IE in the food domain. In contrast with PURE, 1) we use the Chinese pre-trained BERT ^[31] model known as "BERT-wwm-ext1" ^[32] to generate the raw token representations. A suitable pre-trained language model can deliver further gains. 2) We use entity attributes (fusion information of entity mention and type) as the input feature for the RC task. 3) We used a unique representation for each span in RC model regardless of which entity pairs it belongs to in one sentence, due to the certainty of the entity. In addition, SpIE uses a feed-forward neural network (FFNN) and a softmax layer for entity and relation classification. The results of the experiment show that SpIE markedly outperforms the baseline approaches.

3.   Materials and methods

3.1.   Overview of the proposed framework

This study presents a novel framework known as SpIE to solve the problem of overlapping entity recognition and multi-relations classification. Figure 2 displays the workflow of our framework which is divided into three modules. 1) Regarding the embedding on the original corpus, BERT-wwm-ext1 ^[32] was used to obtain the token representation, and span representation generated from the token representation. 2) Considering the entity recognition model, the model is used to detect both the boundaries and types of entities. 3) Regarding the relation classification model, the model is used to classify each pair of entities into predefined relation types.

Figure 3 illustrates the SpIE architecture. Figure 3(a) is an entity model for recognizing the named entity, and Fihure 3(b) is a relation model for classifying each span pair into predefined relation types. In the subsequent subsections, we explain the main components of the SpIE in detail.

3.2.   Different part of the framework

3.2.1.   Span representation generation

The input is a sentence $X = {x_1, x_2, ..., x_n}$ with n tokens. Because of the remarkable performance of the recent transformer networks, this study adopts BERT-wwm-ext1 ^[32] to obtain the contextualized embedding $\bf{x_n}$. Our model constructs all the possible spans $S = {s_1, s_2, ..., s_m}$, where $m = \dfrac{n(n+1)}{2}$. Following Zhong's model ^[6], $s_i$ is defined as all the tokens from $START(i)$ to $END(i)$ for $1\leqslant i\leqslant$ m. This indicates that $START(i)$ and $END(i)$ denote the start and end indices of $s_i$, respectively. The spans are sorted based on the order of $START(i)$, and spans with the same start index are sorted by $END(i)$. Given a span $s_i \in S$, $s_i$'s representation $\bf{g_i}$ is obtained by concatenating the vector representation of the start, end, and span width features:

Considering Eq (3.1), $\bf{x}_{START(i)}$ and $\bf{x}_{END(i)}$ are the head and tail token embedding of $s_i$, respectively. Moreover, $\bf{ \pmb{\mathsf{ ϕ}}}(i)$ is the embedding of the span width feature.

3.2.2.   Named entity recognition

Considering this step, the task is to predict the best entity category for each candidate span. Regarding each $s_i$, we predict an entity category $y_e(s_i)\in\varepsilon \cup NONE$. The output of the task is $Y_E = {(s_i, e)}$, where ${s_i \in S, e\in\varepsilon \cup NONE}$. We use $\varepsilon$ to represent a set of predefined entity categories. $y_e(s_i) = NONE$ indicates that $s_i$ does not belong to any predefined entity category. Considering each span $s_i$, SpIE uses a two-layer FFNN with ReLU activation to compute a vector of entity category scores in parallel. Thereafter, the softmax function was applied to the entity category score to obtain a distribution over the entity categories. A pair of overlapping entity belongs to different spans with different representations. These are difficult to define in the traditional token-level model.

Considering span $i$, the representation $\bf{g_i}$ is used to predict the entity category $e\in\varepsilon$:

The output sizes of $FFNN_{ner}$ and $p^{ner}$ are equal to the number of NER classes. Considering each span $s_i$, the predicted entity category corresponds to the span's highest entity category score. If a span is assigned to an entity category other than NONE, it will determine whether there is a relationship between it and other spans.

3.2.3.   Relation classification

The goal of the RC model is to predict the best relation category between the entity pair $(s_i, s_j)$, where $s_i$ and $s_j$ are gold entities ($y_e(\cdot)\in \varepsilon, \cdot$ can be $s_i$ or $s_j$) and are not overlapping entity. The study only considers binary relations, where $(s_i, s_j)$ and $(s_j, s_i)$ are considered to be different. Within a sentence, we consider each ordered pair of spans. The spans were obtained from the NER model. The output of the RC is $Y_R = {(s_i, s_j, r})$, where ${s_i, s_j \in S, r\in \mathcal{R} \cup NONE}$, and $\mathcal{R}$ are the sets of predefined relation categories. $r = NONE$ indicates that there is no relationship between the pair of candidate spans. A previous study ^[33] re-used span representation to predict their relations. However, other studies ^[34,35] have validated that the original span representation only captures the contextual information around each entity, which ignores the dependencies between each other. Furthermore, other studies ^[36,37] proved that extra information was vital to RC. Based on this idea, we introduce a pair of spans with an entity label. The experimental results demonstrate that it is easier to determine the relationship between a pair of spans.

Considering the relation model, we use category markers to replace entity mentions to obtain the representation of the label. We denote the adjusted $X$ by $X^*$, and Figure 4 provides an example.

$X^*$ is fed into the pre-trained language model to obtain the representation of entities' type $\bf{e_i}$, which incorporates the contextual information. Thereafter, we concatenate $\bf{e_i}$ with $\bf{g_i}$ (generated from the NER model) to represent the entity mention with the type information. $\bf{\hat{g}}_i = [\bf{e_i}; \bf{g_i}]$ is a new presentation of the span. Regarding a pair of spans $(s_i, s_j)$, we compute an ordered pair embedding $\bf{r}_{i, j} = [\bf{\hat{g}}_i; \bf{\hat{g}}_j]$. Considering $(s_i, s_j)$, the representation $\bf{r_{i, j}}$ is used to predict the relation category $r\in \mathcal{R}$:

The output sizes of $FFNN_{re}$ and $p^{re}_{i, j}$ are equal to the number of RE classes. Regarding each pair $s_i, s_j$, the predicted relation category corresponds to the pair's highest relation category score.

3.2.4.   Loss

In this study, we use supervised training on both the NER and RC models. When a set of sentences, well-labeled entities, and relations are given, we use the cross-entropy loss for these two models:

Considering Eq (3.6), $\mathcal{L}_{ner}$ represents the cross-entropy over the entity category including none, and $e^*_i$ represents the correct entity category for $s_i$. Regarding the relation model, we only consider the correct entities $S\subset {S}'$. Moreover, considering Eq (3.6), $\mathcal{L}_{re}$ represents the cross-entropy over the relation classes, including none, and $r^*_{i, j}$ represents the correct relation category for the ordered pair of spans $(s_i, s_j)$.

3.3.   Dataset preparetion and processing

This study aims to provide a high-performance model to automatically extract entities and relations from unstructured Chinese texts to construct and expand the knowledge graph. We construct a new dataset to describe recipe information. We build a web crawler in Python using two libraries (requests and Beautiful Soup) to collect the experimental data from Baidu Baike (https://baike.baidu.com/item/豉椒蒸腊肉). These data are complex samples of high-density knowledge for the Beijing 2022 Olympic and Paralympic Winter Games.

Sixteen annotators and two supervisors annotated the corpus using the corpus annotation system developed by the National Engineering Laboratory for Agri-product Quality Traceability. In order to ensure the correctness of the annotation, annotators were trained before annotating, and they annotated back-to-back in pairs. When the annotation data is inconsistent, the supervisor annotated again. Finally, the correct annotated corpus make up the Recipe-IE dataset and there are 2318 sample data obtained from 750 recipes that are semantically segmented for periods '.'. There are some distinctive features in the recipes corpus. 1) Information about the ingredients and cooking methods may be contained in the dish name, which will lead to overlapping entity. This feature is shown in Figure 1. An underlined span indicates an entity. The spans underlined in red and orange form a pair of overlapping entity. 2) Considering a sentence, a subject corresponds to multiple objects, and the relationship between any two spans belongs to different relation categories. Regarding Figure 1, the line with the arrow connects two spans, and there is a relation between them. Each triple describes the relevant dish's knowledge. The study developed seven types of dish entities:菜名(dish name), 菜系(cuisine), 原材料(ingredient), 工艺(cooking method), 营养物质(nutrient), 适合人群(people), 功效(efficacious) and seven relation categories:属于(belong_to), 原料是(ingredient_is), 烹饪方法(cook_by), 含有(contain), 适合(suitable_for), 功效(efficacy_is), 相克(overcome). The distribution of the Recipe-IE dataset is shown in Figure 5, and Table 1 displays the statistics for all entities and relations.

There are 683 instances of sentences with overlapping entity in the Recipe dataset, representing 29.45% of the total. Over 27% of the sentences contain over one triple, and the maximum length of the gold entity is 11.

4.   Results

4.1.   Experimental setup

We experimented on the Recipe-IE dataset and part of the publicly available DuIE dataset. We followed Luan's pre-processing steps and randomly divided the dataset into training, dev, and test sets. There was no duplication between the sets. The training set was used for model training in the training phase for the model training stage. The dev set validates the trained model and adjusts the hyper-parameters to optimize the model. The model was evaluated using the test set. Table 2 presents the information for the two datasets.

The BERT-wwm-ext1 ^[32] model was used as a sentence encoder, which was pre-trained in the Chinese language. We used Pytorch with a GPU to complete the module. The context window size was 300 with the two-layer FFNN with 150 hidden units. We trained the model with the Adam optimizer and a warmup ratio of 0.1. In addition, we trained the model for 10 epochs with a dropout of 0.2, a learning rate of 2e–5, and a batch size of 16.

4.2.   Evaluation metrics

This study follows the standard evaluation protocol and measures the F1-score, precision (P), and recall (R) as the evaluation metrics. The F1, P, and R are calculated as follows:

Considering the above equations, false negative (FN), false positive (FP), true positive (TP), and true negative (TN) indicate incorrect negative prediction, incorrect positive prediction, correct positive prediction, and correct negative prediction, respectively.

Bekoulis et al.^[38] proposed three types of evaluation, namely: 1) Strict: an entity is correct only if both the entity boundaries and type are correct, 2) Boundaries: only make sure the entity boundaries are correct and the type is not considered, 3) Relaxed: for a multi-token entity, only ensure at least one correct type is assigned to the tokens comprising the entity. We use Strict to evaluate the IE task. For named entity recognition, an entity is considered to be correctly predicted only if both span boundaries and the type match the truth. Considering the relation extraction, a predicted relation is considered as a correct prediction only if both the boundaries of the two spans and the predicted relation type are correct. In fact, the representation used for relationship prediction has fused the type information, and the correct span boundaries can ensure that the span type is also correct.

4.3.   Experimental results

The performance of the SpIE is benchmarked by comparing it with the latest models in the entity relation extraction task. There are two types of baseline models. 1) The basic Bi-LSTM-CRF model with different input features (both different embedding and tagging schemes) are defined as token-level models. Cao et al. ^[23] adopted an NN with an M-tag and an original tagging scheme for entities and relation extraction. Gao et al. ^[37] used a character-and word-attention-enhanced NN for information extraction. 2) Dixit et al. ^[33] used a span-based, which use Bi-LSTM and ELMo for raw token embedding. PURE ^[6] is a span-based model which uses bert-base-uncased as the base encoder and directly uses entity label as the input feature for the RC model. These two models are used as a baseline to verify whether different language models and input features affect the experimental results.

We conducted our experiment using SpIE and other baseline models on the self-constructed Recipe-IE dataset. The SpIE performed markedly better than the other models in both the NER and RC. Table 3 and Figure 6 show the experimental results in detail. Comparing SpIE to token-level models, we conclude that performing non-span-based models will struggle when dealing with complex Chinese samples, such as overlapping entity and multi-relations. These models take a token representation as the input. However, traditional token-level models have an inherent limitation because the representation is fixed and cannot model overlapping entity. Comparing SpIE to other span-based models, we observe that the performance of these two models is close to that of our model and higher than that of those token-lever models, in which PURE is closer to that of our model in NER task. We come to the conclusions:1) The span-based representation is more useful for overlapping entity recognition. 2) The key to achieving this result is that the encoder, using whole word masking as the core mechanism, captures the interactions between tokens. A suitable pre-trained language model can deliver further gains. 3) The SpIE model fully considers the impact of the entity type on relation extraction, and the inserted entity type information highlights the subject, object, and type. Considering Figures 6(a), (b) compared to the token-level models on the Recipe-IE dataset, our model achieves a F1 improvement of +21.05 and +18.83% for entity recognition and +8.09 and +14.20% for relation classification. Compared to other span-level models, our model achieves a F1 improvement of +2.51 and +1.05% for entity recognition and +3.15 and +2.02% for relation classification.

To future study the capability of the proposed SpIE in extracting general triple facts, we conduct extended experiments on DuIE dataset. Since Ge et al. ^[39] mixed the characters and words feature for Chinese IE and their study has demonstrated this method has reported a state-of-the-art performance on DuIE datasets. We evaluated the performance of our model and Ge et al.'s model on both Recipe-IE and DuIE datasets. After 10 epochs, due to the interaction between raw tokens being fully captured, our model achieved the same performance as Ge et al.'s model, while their model reported poor performance on Recipe-IE datasets. Table 3, Figures 6(c), (d) describe the specific performance of these two models on these two datasets, which demonstrates that our SpIE remains competitive in the normal IE task.

The SpIE model transforms both NER and RC into a typical multi-label classification task. Table 4 shows the results of each classification task. To further investigate the capabilities of the SpIE on IE, we statistically analyzed the details for each category, and the experimental results are shown in Table 4. Our model performs well on all entity and relation categories, especially on the dish name, ingredient, ingredient_is, and efficacy_is. These categories have a better performance because they have more structural features. Specifically, the position of the dish name is relatively fixed and consecutive ingredients are linked by ', '.

5.   Discussion

In this section, we demonstrate why this model performed well.

5.1.   Span pruning

A number of spans under consideration require significant computational overhead and cause memory issues. We set the maximum length of the span to seven and only consider spans that are entirely within a sentence. Our model creates a representation for the span instead of a token, and the maximum length we set can capture over 99.90% of the entities. Table 5 shows the statistics of the effects of different policies on span generation. When the span's max length changes from one to seven, we obtain a 3.3 $\times$ speedup, and the F1 score increases from 76.14 to 96.90%. However, when all the possible spans are considered, the time consumption reaches 7.4 $\times$ speedup, and F1 only increases by 0.21%. "Policy" indicates the max length of the span permitted. "Coverage" denotes the rate of coverage of entity, "F1" and "Speed" are the F1 score for NER and time consumption under different span generation policies, respectively. The trend of the fold in Figure 7 shows the phenomenon more intuitively. Considering L = 7 to all the spans, the time consumption exhibits an extraordinarily steep increase, whereas there is no significant difference in the F1-score. Considering entity coverage, F1-score, and time consumption, we set the maximum span length to seven.

We provide additional analysis of the performances on overlapping entities and flat entities separately. SpIE's performance on "overlapping entity only" and "flat entity only" are similar to it on the whole dataset, and the ability to recognize overlapping entity is slightly lower than that of flat entity. While in Table 5, when we select L = 1, we treat the overlapping entities as flat entities, which reports poor performance. The span-based method achieves significant performance for overlapping entity recognition at the cost of speed.

5.2.   Category marker

We argue that both entity mentions and types are important in determining the relationship between two entities. To train the relation model, a category marker (entity type information) was inserted into the original sentence. The presentation of $\bf{r}_{i, j} = [\bf{\hat{g}}_i; \bf{\hat{g}}_j]$ is the input of the relation model, and $\bf{\hat{g}}$ contains both entity mention and type information. To validate the importance of entity category markers, we present four options (illustrated in Figure 8) for the input of the RC model.

No Marker: The NO MARKER relation model references Luan ^[29] and Dixit's ^[33] relation model. The input of the relation model is $[\bf{{g}}_i; \bf{{g}}_j; \bf{{g}}_i \circ \bf{{g}}_j]$, where $\bf{{g}}_i$ and $\bf{{g}}_j$ are the span representations obtained from the entity model ($\bf{{g}}_i, \bf{{g}}_j\neq NONE$) and $\bf{{g}}_i \circ \bf{{g}}_j$ refers to the element-wise product to concatenate the hidden representation.

M Marker: We use [S], [/S], [O], [/O] (the marker without entity type information) at the input layer and use each entity's start marker to represent the entity.

T Mareker: The T markers $e_i$ and $e_j$ are used to replace the entity mentions $s_i$, $s_j$ at the input layer. The representations $\bf{e_i}$ and $\bf{e_j}$ have both context and type information.

M+T Marker: As described in Section 3.3, we use $\bf{\hat{g}}_i = [\bf{e_i}; \bf{g_i}]$ to describe the gold entity $s_i$, and $\bf{\hat{g}}_i$ contains context, entity mention and type information.

Furthermore, we conducted a comparison experiment on the Recipe-IE dataset. The relation of the F1-score with different input features is shown in Table 6. Type information is the most important feature to relation classification, and the M+T Marker achieves a F1 improvement of score for 1.72% than T Marker.

5.3.   Ablation test

To thoroughly analyze the effectiveness of the various components of SpIE, including the strategy of embedding, modeling with FFNN, and learning with correlation features, we added an LSTM layer above the BERT, and the performance improved slightly. However, this difference is small. This indicates that the in-domain pre-training language model can effectively capture full contextual representation. Thereafter, we skipped the FFNN, directly connected the BERT layer to the softmax layer, and observed that the performance dropped significantly. This indicates that BERT is not sufficient to classify spans into predefined entity categories. When we replaced $[\bf{\hat{g}}_i; \bf{\hat{g}}_j]$ with $[\bf{\hat{g}}_i; \bf{\hat{g}}_j; \bf{\hat{g}}_i \circ \bf{\hat{g}}_j]$ for the RC task, the performance increased slightly. However, the time consumption reaches 1.2 $\times$ speedup. In other word, using their element-wise multiplication is not enough to capture the interaction characteristics between this two spans. It comes to the same conclusion with Section 5.2 that No Marker reported the poorest performance. Table 7 lists the performance of the ablation test.

6.   Conclusions

This study proposes SpIE as a general information extraction framework for overlapping entity recognition and multi-relation classification from Chinese texts in the food domain. Moreover, SpIE tackles information extraction as two classification tasks through a span-based representation. The span-based model provides different representations for all the spans, which makes handling overlapping entity possible. The connection features of entity attributes fully considers the influence of entity mentions and types on the relationship between entity pairs. The comparative experiments and ablation tests indicate that i) span representation is effective for overlapping entity detection; ii) both entity mentions and type information are vital for relation classification; iii) SpIE outperforms previous neural approaches in complex IE tasks mentioned in this study and is still competitive in general IE tasks. In the future, we plan to extend SpIE to coarse-grained (document level) text extraction and demonstrate its effectiveness in more complex IE tasks.

Acknowledgments

This project was supported by Beijing Science and Technology Planning Project [Grant Numbers Z191100008619007].

Conflict of interest

The authors declare there is no conflict of interest.