Hybrid Filtering Approach for Question Answering

We describe a question answering system that took part in the bilingual CLEFQA task (German-English) where German is the source language and English the target language. We used the BableFish online translation system to translate the German questions into English. The system is targeted at Factoid and Definition questions. Our question answering system consists of the following core components: Question Analysis, Passage Retrieval, Sentence Analysis and Answer Selection. Our focus in designing the current system is on testing our online methods which are based on information extraction and linguistic filtering methods in which syntactic filtering constitutes the core component. For this, we used a Lexical Functional Grammar(LFG)-based parser developed at Dublin City University. The system takes the output of a syntactic parser (Charniak parser) and generates an F-Structure, a labelled bilexical dependency graph. The output is used to measure the syntactic similarity between the question and the answer.

The system returned 16 exact answers (8 Factoids and 8 Definitions), and 25 correct answers counting unsupported answers. The web reranking component contributed significantly. Error analysis shows different sources of errors: Translation, Questions classification, Named entity recognition. Post CLEF evaluation showed that minor adjustment to the question classifier and correcting some technical errors (coding errors) improved the result significantly: the system returned 19 exact answers for factoid questions. In the future, we would like to extend the application of the LFG-based parser output to other components. Initial application of the parser output to the ML based Question classifier showed significant improvement. We are also planning to extend dependency triple based scoring method to include the full LFG-based parse output. Furthermore, we are working to bring in logic-based reasoning methods in the system. The approach derives logic-based representations of questions and candidate sentences based on the output of the LFG Parser. This will allow us to make inferences which form the basis for finding implicit relations between questions and answers. Coreference resolution is an important component of a QA system that is not well developed in our system. We will explore the application of the LFG parser for problem of coreference resolution.