Evaluation of multi-terminology super-concepts for information retrieval.

Abstract : BACKGROUND: Following a recent change in the indexing policy for French quality controlled health gateway CISMeF, multiple terminologies are now being used for indexing in addition to MeSH®. OBJECTIVE: To evaluate precision and recall of super-concepts for information retrieval in a multi-terminology paradigm compared to MeSH-only. METHODS: We evaluate the relevance of resources retrieved by multi-terminology super-concepts and MeSH-only super-concepts queries. RESULTS: Recall was 8-14% higher for multi-terminology super-concepts compared to MeSH only super-concepts. Precision decreased from 0.66 for MeSH only super-concepts to 0.61 for multi-terminology super-concepts. Retrieval performance was found to vary significantly depending on the super-concepts (p<10^-4) and indexing methods (manual vs automatic; p<0.004). CONCLUSION: A multi-terminology paradigm contributes to increase recall but lowers precision. Automated tools for indexing are not accurate enough to allow a very precise information retrieval.