Julio Gonzalo - Böcker

This book constitutes the refereed proceedings of the 8th International Conference of the CLEF Initiative, CLEF 2017, held in Dublin, Ireland, in September 2017. The 7 full papers and 9 short papers presented together with 6 best of the labs papers were carefully reviewed and selected from 38 submissions.

This book constitutes the thoroughly refereed postproceedings of the 4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, held in Trondheim, Norway in August 2003.The 61 revised papers presented together with an introduction were carefully reviewed and selected for inclusion in the book. The papers are organized in topical sections on mainly cross-language experiments, mono lingual experiments, domain-specific document retrieval, interactive cross-language retrieval, cross-language question answering, cross-language image retrieval, and cross-language spoken document retrieval.

The ?fth campaign of the Cross-Language Evaluation Forum (CLEF) for Eu- pean languages was held from January to September 2004. Participation in the CLEF campaigns has increased each year and CLEF 2004 was no exception: 55 groups submitted results for one or more of the di?erent tracks compared with 42 groups in the previous year. CLEF 2004 also marked a breaking point with respect to previous campaigns. The focus was no longer mainly concentrated on multilingual document retrieval as in previous years but was diversi?ed to include di?erent kinds of text retrieval across languages (e. g. , exact answers in the question-answering track) and retrieval on di?erent kinds of media (i. e. , not just plain text but collections containing image and speech as well). In ad- tion, increasing attention was given to issues that regard system usability and user satisfaction with tasks to measure the e?ectiveness of interactive systems or system components being included in both the cross-language question - swering and image retrieval tasks with the collaboration of the coordinators of the interactive track. The campaign culminated in a two-and-a-half-day workshop held in Bath, UK, 15–17 September, immediately following the 8th European Conference on Digital Libraries. The workshop was attended by nearly 100 researchers and s- tem developers.

The third campaignof the Cross-LanguageEvaluation Forum (CLEF) for Eu- pean languages was held from January to September 2002. Participation in this campaignshowedaslightriseinthenumberofparticipantswith,37groupsfrom both academia and industry and a steep rise in the number of experiments they submitted for oneor moreof the ?ve o?cialtracks. The campaignculminated in atwo-dayworkshopheldinRome,Italy,19-20September,immediatelyfollowing the Sixth European Conference on Digital Libraries (ECDL 2002), attended by nearly 70 researchersand system developers. The objective of the workshop was to bring together the groups that had participated in CLEF 2002 so that they could report on the results of their experiments. Attendance at the workshop was thus limited to participants in the campaignplus severalinvited guests with recognized expertise in the multilingual information access ?eld. This volume contains thoroughly revised and expanded versions of the preliminary papers presented at the workshop accompanied by a complete run-down and detailed analysis of the results, and it thus provides an exhaustive record of the CLEF 2002 campaign.CLEF2002 wasconducted within the frameworkof a projectof the Infor- tion Society Technologies programme of the European Commission (IST-2000- 31002). The campaign was organized in collaboration with the US National - stitute ofStandardsandTechnology(NIST)andwiththe supportoftheDELOS Network of Excellence for Digital Libraries. The support of NIST and DELOS in the running of the evaluation campaign is gratefully acknowledged. We would also like to thank the other members of the Workshop Steering Committee for their assistance in the coordination of this event.

This book constitutes the thoroughly refereed post-proceedings of the Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, held in Darmstadt, Germany in September 2001. The 35 revised full papers presented together with two introductory survey articles and a comprehensive appendix were carefully improved during the round of reviewing and selections. The papers are organized in topical sections on systems evaluation experiments, mainly cross-language, monolingual experiments, interactive issues, and evaluation issues and results.

This book constitutes the refereed proceedings of the 10th European Conference on Research and Advanced Technology for Digital Libraries, ECDL 2006. The book presents 36 revised full papers together with the extended abstracts of 18 demo papers and 15 revised poster papers. The papers are organized in topical sections on architectures, preservation, retrieval, applications, methodology, metadata, evaluation, user studies, modeling, audiovisual content, and language technologies.

The sixth campaign of the Cross Language Evaluation Forum (CLEF) for European languages was held from January to September 2005. CLEF is by now an established international evaluation initiative and 74 groups from all over the world submitted results for one or more of the different evaluation tracks in 2005, compared with 54 groups in 2004. There were eight distinct evaluation tracks, designed to test the performance of a wide range of systems for multilingual information access. Full details regarding the design of the tracks, the methodologies used for evaluation, and the results obtained by the participants can be found in the different sections of these proceedings. As always the results of the campaign were reported and discussed at the annual workshop held in Vienna, Austria, September 21-23, immediately following the ninth European Conference on Digital Libraries. The workshop was attended by approximately 110 academic and industrial researchers and system developers. In addition to presentations by participants in the campaign, Noriko Kando from the National Institute of Informatics, Tokyo, gave an invited talk on the activities of the NTCIR evaluation initiative for Asian languages. Breakout sessions gave participants a chance to discuss ideas and results in detail. The final session was dedicated to proposals for activities for CLEF 2006. The presentations given at the workshop can be found on the CLEF Web site at: www. clef-campaign. org. We should like to thank the other members of the CLEF Steering Committee for their assistance in the coordination of this event.

The ninth campaign of the Cross-Language Evaluation Forum (CLEF) for European languages was held from January to September 2008. There were seven main eval- tion tracks in CLEF 2008 plus two pilot tasks. The aim, as usual, was to test the p- formance of a wide range of multilingual information access (MLIA) systems or s- tem components. This year, 100 groups, mainly but not only from academia, parti- pated in the campaign. Most of the groups were from Europe but there was also a good contingent from North America and Asia plus a few participants from South America and Africa. Full details regarding the design of the tracks, the methodologies used for evaluation, and the results obtained by the participants can be found in the different sections of these proceedings. The results of the CLEF 2008 campaign were presented at a two-and-a-half day workshop held in Aarhus, Denmark, September 17–19, and attended by 150 resear- ers and system developers. The annual workshop, held in conjunction with the European Conference on Digital Libraries, plays an important role by providing the opportunity for all the groups that have participated in the evaluation campaign to get together comparing approaches and exchanging ideas. The schedule of the workshop was divided between plenary track overviews, and parallel, poster and breakout sessions presenting this year’s experiments and discu- ing ideas for the future. There were several invited talks.

The tenth campaign of the Cross Language Evaluation Forum (CLEF) for European languages was held from January to September 2009. There were eight main eval- tion tracks in CLEF 2009 plus a pilot task. The aim, as usual, was to test the perfo- ance of a wide range of multilingual information access (MLIA) systems or system components. This year, about 150 groups, mainly but not only from academia, reg- tered to participate in the campaign. Most of the groups were from Europe but there was also a good contingent from North America and Asia. The results were presented at a two-and-a-half day workshop held in Corfu, Greece, September 30 to October 2, 2009, in conjunction with the European Conference on Digital Libraries. The workshop, attended by 160 researchers and system developers, provided the opportunity for all the groups that had participated in the evaluation campaign to get together, compare approaches and exchange ideas.