Stockinger - Introduction to Audiovisual Archives

This collective work deals with the analysis of audiovisual numerical texts or corpora, which may e.g. form part of an audiovisual library or archive.

The development of methods, tools and conceptual frameworks (or models) for the concrete analysis of audiovisual texts or corpora is one of the most important issues for multimedia (audiovisual) digital libraries, archives, collections, etc. and also for any project or program to compile and disseminate knowledge heritage (e.g. cultural, scientific etc.).

Analyzing audiovisual recordings, shoots, sound recordings, film or complex multimodal documents etc. obviously constitutes an essential step for any classification of the (digital) collection of an archive or library.

Above all, however, it is the most important activity by which an actor (an individual, group of individuals, institution, etc.) obtains and exploits numerical audiovisual data to transform them depending on their own skills, expectations and requirements, but also within the limitations imposed by the tools, methods and models available into genuine cognitive resources which they regard as useful, pleasant, interesting or simply relevant, i.e. which have a value for them.

Ten years ago now, along with a small nucleus of permanent collaborators from the ESCoM (Semiotics Cognitive and New Media Team), the research center at the Fondation Maison des Sciences de lHomme (FMSH House of the Human Sciences Foundation) in Paris, we set up the ARA (Audiovisual Research Archives) program. One of the objectives of this program, which will be described in more detail in the cultural heritage, notably through scientific events and field-work carried out in human and social sciences. Another objective of this program is to set up research and development projects aimed at:

a) collecting and producing audiovisual documentation (of field-work, for example);

b) compiling analysis corpora and effectively analyzing these corpora;

c) creating publishable corpora and publishing them;

d) defining and setting up (metalinguistic) models and essential procedures to successfully carry out the aforementioned three tasks.

In this book, and another collective work complementing this one (see [STO 12a]), we will present and discuss the results of our research and development relating to the analysis, description and indexing of audiovisual corpora. The question of analysis has been addressed from the start with regard to the following three issues:

1) a good understanding of the activity of analysis must take account of the internal structural organization of the audiovisual text and must have recourse to the semiotics of the audiovisual text or discourse;

2) a true analysis (going beyond, e.g. simply producing unstructured lists of keywords) of audiovisual corpora cannot be carried out without a metalanguage (an ontology), i.e. models of description representing the area of expertise covered by a corpus to be analyzed;

3) of course, no analysis can take place without an appropriate working environment.

Thanks to a series of French and European R&D projects project, financed by the French National Research Agency (Agence Nationale de la Recherche ANR), that gave us the time and means needed to develop:

a metalanguage for analyzing audiovisual corpora documenting a wide variety of areas of knowledge/expertise. This metalanguage is a generic ontology (called ASW

a working environment for segmenting and describing audiovisual corpora entirely based upon the ASW metalanguage of description. The name of this environment is ASW Studio; it is made up of several specialized workshops: the Segmentation Workshop, for (virtually) segmenting an audiovisual object; the Description Workshop, for describing an audiovisual object; the Publication Workshop, for publishing an audiovisual object; the Modeling Workshop, to model the metalinguistic resources needed to undertake an analysis/description of an audiovisual object. In this book we will present the two following workshops in particular: the Segmentation Workshop and the Description Workshop; the presentation of the Modeling Workshop will be the subject of [STO 12b]; as the Publication Workshop is still partially under development, it will be the object of a new publication in late 2012;

an as-yet relatively simple metalanguage for defining models for publishing/republishing audiovisual corpora in the form, e.g. of themed folders, bilingual folders, theme-limited video-glossaries, themed Websites, etc. These models are indeed used for publishing/republishing audiovisual corpora but the metalanguage enabling us to define them has not yet been made explicit. Clarifying the organization of this metalanguage and incorporating it into the ASW generic ontology will, conditions beyond the authors control permitting, constitute the main object of the ESCoMs research activities during the next few years.

This book is divided into two main parts. In , following an introductory chapter contextualizing our R&D activities since 2001, the different approaches to analyzing of an audiovisual corpus using ASW Studio will be presented:

strictly textual analysis, consisting of the identifying passages which are relevant to an analysis and to the (virtual) segmentation of an audiovisual object ();

metadescription, which clarifies the content and objectives of the analysis itself as well as the authors of the analysis, the rights associated with using the results, etc. ();

paratextual description, the aim of which is to formally identify the audiovisual object being analyzed (title, author, genre, summary of content, etc.) and the relative rights associated with its use ();

audiovisual description, which relates to analyzing visual, acoustic and audiovisual shots ();