Automatic test suite generation for PMCFG grammars

Authors: Inari Listenmaa and Koen Claessen

Paper Information

Title:Automatic test suite generation for PMCFG grammars
Authors:Inari Listenmaa and Koen Claessen
Proceedings:NLCS pre-workshop papers
Editors: Larry Moss, Valeria de Paiva and Ash Asudeh
Keywords:Grammar testing, Grammatical Framework, PMCFG

ABSTRACT. We present a method for finding errors in formalized natural language grammars, by automatically and systematically generating test cases that are intended to be judged by a human oracle. The method works on a per-construction basis; given a construction from the grammar, it generates a finite but complete set of test sentences (typically tens or hundreds), where that construction is used in all possible ways. Our method is an alternative to using a corpus or a treebank, where no such completeness guarantees can be made. The method is language-independent and is implemented for the grammar formalism \pmcfg{}, but also works for weaker grammar formalisms. We evaluate the method on a number of different grammars for different natural languages, with sizes ranging from toy examples to real-world grammars.

Talk:Jul 07 17:00 (Session 31K: Contributed talks)