BiographTA

Processing modality and negation for machine reading

This is a pilot task of the Machine Reading Evaluation QA4MRE at CLEF 2011.

In this task we aim at evaluating whether systems understand extra-propositional aspects of meaning like modality and negation. Modality is a grammatical category that allows to express aspects related to the attitude of the speaker towards her statements. Modality understood in a broader sense is also related to the expression of certainty, factuality, and evidentiality. Negation is a grammatical category that allows to change the truth value of a proposition. Modality and negation interact to express extra-propositional aspects of meaning.

An event like "The earthquake adds further threats to the global economy" can be presented as a counterfact (1), and it can be presented with a broad range of meanings depending on features like modality, negation, tense and aspect (2).

• The earthquake does not add further threats to the global economy
The earthquake never added further threats to the global economy

• Does the earthquake add further threats to the global economy?
The earthquake will add further threats to the global economy
The earthquake will never add further threats to the global economy
The earthquake will probably add further threats to the global economy
The earthquake will certainly add further threats to the global economy
The earthquake might add further threats to the global economy
The earthquake might have added further threats to the global economy
According to some media sources, the earthquake adds further threats to the global economy, but this has not been confirmed with facts
The earthquake will add further threats to the global economy if the right measures are not applied
It is unclear whether the earthquake will add further threats to the global economy
It is expected that the earthquake will add further threats to the global economy
It has been denied that the earthquake adds further threats to the global economy
It is believed that the earthquake adds further threats to the global economy
Why would the earthquake not add further threats to the global economy?
Some experts are convinced that the earthquake adds further threats to the global economy, whereas other experts are more optimistic

We will provide texts where these phenomena occur and at most 10 questions per text in the form of multiple choice, where for each question a list of possible answers is given. The focus will be on testing the comprehension of aspects of meaning related to modality and negation:

• distinguishing events from negated, uncertain and conditional events
• distinguishing several event modalities
• the above distinctions in combination with finding whether an event is presented from the perspective of the author or not

Given a sentence like "Experts consider that it is unclear whether the earthquake will add further threats to the global economy", questions may be formulated as follows:

• Event --the earthquake ADD further threats to the global economy-- is presented in the text as:
• A negated event
• A condition for another event
• An event
• An uncertain event from the perspective of someone other than the author
• A purpose event

A more detailed description of the task can be found in the Guidelines to be downloaded here.

Two example tests can be downloaded here. The texts have been made available by The Economist journal.

Interested participants can register for the QA4MRE tasks via the CLEF registration website:
http://clef2011.org/index.php?page=pages/registrationForm.php

Registered participants will receive instructions by e-mail about how to download the test sets.

This pilot task will follow the same timeline as the main Machine Reading Evaluation QA4MRE
Roser Morante, roser.morante@ua.ac.be
Walter Daelemans, walter.daelemans@ua.ac.be