Annotating clinical conditions in reports is necessary for compiling reference standards against which automated indexing systems are compared. However, the task is vague and produces substantial variation among annotators. For example, the sentence "Patient has severe left-sided chest pain" could result in several different annotations, including "pain," "chest pain," "left-sided chest pain," and "severe left-sided chest pain." We created guidelines detailing medical and linguistic instructions about what text to include in annotations of clinical concepts and measured agreement between two annotators using the guidelines. I will present our results and describe future plans for the guidelines.
Beatriz will talk about her research and results of applying Cotraining and Self Training in Spoken Dialogue Data.