Detecting syntactic differences automatically using the Minimum Description Length principle

Martin Kroon*, Sjef Barbiers, Jan Odijk, Stéphanie van der Pas

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

In this paper we present a systematic approach to detect and rank hypotheses about possible syntactic differences for further investigation by leveraging parallel data and using the Minimum Description Length (MDL) principle. We deploy the SQS-algorithm ('Summarising event seQuenceS'; Tatti and Vreeken 2012) - an MDL-based algorithm - to mine 'typical' sequences of Part of Speech (POS) tags for each language under investigation. We create a shortlist of potential syntactic differences based on the number of parallel sentences with a mismatch in pattern occurrence. We applied our method to parallel corpora of English, Dutch and Czech sentences from the Europarl v7 corpus (Koehn 2005). The approach proved useful in both retrieving POS building blocks of a language as well as pointing to meaningful syntactic differences between languages. Despite a clear sensitivity to tagging accuracy, our results and approach are promising.

Original languageEnglish
Pages (from-to)109-127
Number of pages19
JournalComputational Linguistics in the Netherlands Journal
Volume10
Publication statusPublished - 12 Dec 2020
Event30th Meeting of Computational Linguistics in the Netherlands, CLIN 2020 - Utrecht, Netherlands
Duration: 30 Jan 2020 → …

Cite this