Project Manager

AIDAR(Adressage et Indexation de Documents Multimédias Assistés par des techniques de Reconnaissance Vocale): A project for multimedia document indexation Contact : Dr. Ir. Frédéric Beaugendre Project Manager Voice-InsightSummary The project intends to maximize the automation of the archiving process, through the use of speech recognition, automatic text segmentation and classification techniques. A broadcast news transcription system (Speeral) has been integrated and tailored to the users needs, using specific language models, segmentation and classification algorithms. An interface is being developed in order to provide a tool for archivists, that will help them creating semi-automatically meta-data for Radio and TV archives. Towards an automatic indexation process The great variety of means of communication such as internet, the rapid expansion of media diffusion though cable and satellite, access to quality digital programs through reliable and lowcost supports such as DVD have a common impact: professional as well as private users are confronted to an increasing quantity of multimedia documents originating from multiple sources. In order for them to be able to access in a fast and reliable way to specific information, there is an urgent need to generate consistent and accurate meta-data tailored to the user’s needs. Automatic segmentation and transcription methods based on statistical models are particularly well adapted to such tasks; they also have a drawback: in order to take into account the great variety of acoustic and linguistic speech segments encountered, there is an imperative need for big enough sets of data able to account for this variability and therefore train accurate statistical models. The number of projects the recent years also attest for the increasing interest on the topic: the EU projects THISL 15 and REVEAL THIS 16, the French RNRT project AGIR 16 to cite only a few. Finally, we shall mention the recent campaign ESTER 12 conducted to evaluate broadcast news transcription systems for French. The toolkit used to achieve the automatic segmentation and transcription tasks has been developed in order to participate to this campaign.General Description of the system The Broadcast News transcription toolkitFigure 1 present a global overview of the architecture of the AIDAR toolkit. It mainly relies on the LIA broadcast news transcription system and automatic annotation algorithms described below. The LIA system has been used for achieving the acoustic macro-class segmentation (music, wide band speech, narrow band speech), speaker segmentation and automatic transcription. Each step of the classification is based on statistical models (HMMs). In addition, the toolkit also performs speaker tracking 11 and adaptation 9 of acoustic models. The speech transcription is obtained using the continuous speech recognition system SpeeRAL 8. This engine uses a modified A* algorithm 10.Figure 1 Description of the AIDAR architecture Acoustic modeling As for the front-end, the speech signal is parametrized using 39 coefficients (the standard 12 mel warped cepstral coefficients plus energy and their first and second derivative). The models are gender dependant, with a 3 state left-to-right topology. Each contextdependant state is associated with a mixture of 32 Gaussians. Language modeling A model developed for the ESTER French evaluation campaign 12 was used to obtain a baseline transcription for our tests. This linguistic model was build using two trigram Language Models: i) a 300M word corpus containing text from the French newspaper Le Monde from 1987 to 2003; ii) a 960K word corpus containing manual transcriptions of radio broadcast collected for the ESTER campaign. The lexicon contains about 65K words. It was obtained by selecting the most frequent tokens from Le Monde and the all tokens of the ESTER corpus. The phonetic forms of the words were automatically obtained using the French grapheme-to-phoneme tool “LIA_PHON” with an additional manual checking of proper names.A comparable general model has been build for Belgium French using: i) a 200M word corpus containing text from the Belgium newspaper Le Soir from 1994 to 2003; ii) 90% of the manual transcriptions of the radio broadcast available for year 2004 (about 300K words). Automatic annotation In order to add metadata and semantic content to the audio material, we rely on text mining algorithms to annotate the speech transcriptions. The annotation procedure is made of two steps: i) an unsupervised linear text segmentation method to detect semantic boundaries inside the text 1, ii) an automatic text classification based on supervised machine learning 3. Initial results and observations OOV words lead to multi-word substitution errors. In addition, the nature of SLMs, that explicitely model inter-word relationships, imply that recognition errors often cascade. Misrecognized OOV words are often name entities, are of main importance for the which enhance the impact on classification results. Short function words tend to be misrecognized, and. Segmentation algorithms are very useful for the analysis of long text documents where several topics may appear. In particular, the AIDAR architecture contains an implementation of the Choi method 2 which partitions the text flow in a set of distinct units to be further processed by the classification system. The interchange format X-AXIS The AIDAR architecture is organized according to an open system approach including two components: the “Acquisition System” including the “AIDAR engine” and the “Autonomous Assets Entities” (Figure 2). Each AAE is a wrapped assembly of the audio contents, its associated metadata and its persistence control data. The AAE’s are represented according to a formalism using exclusively standards. The format of the AAE’s is defined under the name XAXIS. It is mainly based on XML, XSD, RDF, Dublin Core and SMIL.Figure 2 X-AXIS in the AIDAR project Status The AIDAR project will end in February 2007.Role in Project and Partner DescriptionULB (Belgium), Machine Learning Group The group brings its expertise for the development and implementation of the textual segmentation and classification algorithms, in order to create the meta-data associated to the multimedia files. www.ulb.be TITAN (Belgium), A.S.B.L TITAN has defined the data architecture (X-AXIS) for the project and is also responsible for dissemination. www.titan.be Voice-Insight (Belgium), Project leader, the main tasks of the company in the project are the integration of the broadcast news transcription system, the development of specific language models and the development of a user interface for archivists. LIA (France): d’Avignon » The « Laboratoire d’Informatique de l’UniversitéLogoThe laboratory has developed a broadcast news transcription system called “Speeral” used for this project. They also bring the expertise for developing additional functionalities within this plateform. www.lia.univ-avignon.fr/ RTBF (Belgium): The RTBF (The Belgian Radio and Television broadcasting organization) The RTBF helps in designing the user needs for an archivists interface and provides the database (samples and related meta-data, thesaurus) for developing the prototype of the application. www.rtbf.be SKEMA (France): company specialized in video self-production platforms for allowing easy production and management of audiovisual and multimedia content. Skema provides the specifications and development of X-AXIS. www.skema.frReferences 1. Choi F. Y. Y., “Advances in domain independent linear text segmentation”, Proceedings of NAACL’00, Seattle, USA, April 2000, ACL 2. Sitbon L., Bellot P., “Adapting and comparing linear segmentation methods for French”, proceedings of the 7th conference RIAO, Avignon, France, p.623 à 637, 2004 3. Sebastiani F., “A Tutorial on Automated Text Categorisation”, In Analia Amandi and Ricardo Zunino, editors, Proceedings of ASAI-99, 1st Argentinian Symposium on Artificial Intelligence, pages 7--35, Buenos Aires, AR, 1999 4. Yang Y., Liu X., “A re-examination of text categorization methods”, in SIGIR-99, 1999 5. Guyon I., Elisseeff A., An introduction to variable and feature selection, Journal of Machine Learning Research, 3:1157--1182, 2003 6. Yiming Y. An evaluation of statistical approaches to text categorization. Journal of Information Retrieval, Vol 1, No. 1/2, pp 67--88, 1999. 7. Salton G., Buckley C. Term weighting approaches in automatic text retrieval, Information Processing and Management, 24,5, 513-523,1988 8. Nocera P., Fredouille C., Linares G., Matrouf D., Meignier S., Bonastre J.-F., Massonié D., Béchet F., “The LIA’s French broadcast news transcription system”, SWIM: Lectures by Masters in Speech Processing, Maui, Hawaii, 2004 9. Matrouf D., Bellot P., Nocera P., Linares G., Bonastre J-F, “Structural Linear Model-Space Transformations for Speaker Adaptation”, Eurospeech 2003, Genève 10. Nocera P., Linares G., Massonié D., Lefort L., “Phoneme lattice based A* search algorithm for speech recognition”, 2002, TSD2002, Brno 11. Istrate D., Scheffer N., Fredouille C., Bonastre J.-F., “Broadcast News Speaker Tracking for ESTER 2005 Campaign”, Eurospeech’05, September 2005. Lisboa, Portugal 12. Galliano S., Geoffrois E., Mostefa D., Choukri K., Bonastre J.-F., Gravier G., “The ESTER Phase II Evaluation Campaign for the Rich Transcription of French Broadcast News”, Eurospeech’05, September 2005. Lisboa, Portugal 13. ‘The Open Archival Information /documents/650x0b1.pdf System’ (OAIS Blue book)14. Maréchal G., Flexible and User-Friendly Exploitation Based on Persistent Media Archives, in the “IASA journal” Volume n°26, December 2005 15. EU-Project THISL: Thematic Indexing /spandh/projects/thisl/ of Spoken Language,16. EU-Project REVEAL THIS FP6-IST-511689: Retrieval of Video and Language for The Home user in an Information Society http://sifnos.ilsp.gr/RevealThis 17. RNRT-Project AGIR (RNRT) Architecture Globale pour l’Indexation et la Recherche par le contenu de données multimédia, http://www.telecom.gouv.fr/rnrt/pagir.htm。