Discourse-level temporal relation extraction using linguistic information and a temporal anchor

Abstract

In the discourse-level temporal relation extraction task, pre-train language models play an important role. However, in our experiments, we found that embeddings generated using pre-trained language models in recent studies do not encode contextual information well. This lack of contextual information embedding is fatal to this task. Besides, one import anchor, the document creation time of the document is recently found to be very helpful for this task. The input form of document creation time is essential for PLMs. Moreover, existing linguistic features within the corpus are ignored by recent research. We propose a new model that generates rich context embeddings containing linguistic information and sentence-style document creation time information. According to our experiment, our idea shows promising results.