Colette Faucher - Innovations in Intelligent Machines-4 Recent Advances in Knowledge Engineering

(1)

(2)

(3)

Real text mining systems have been developed for finding precise and specific information on large collections of texts from keyword-based or natural language questions. The development of Web searching and question-answering systems that aim to retrieve precise answers corresponds to the combination of methods from computational linguistics, graph analysis, machine learning and, of course, information extraction and information retrieval. The most recent systems tend to combine statistical machine learning methods along with symbolic and rule-based approaches. The challenge is to get robust efficient systems that can self-adapt to users needs and profiles, to dynamic collections composed of unstructured or weakly structured and interconnected documents.

Numerical methods based on a statistical study of corpus, have proven their ability to adapt quickly to different themes and languages and are very popular on Web search systems. This was done despite some approximation in the results that would be more accurate by using symbolic rule-based approaches as long as the rules would be fair, complete and unambiguous while natural language is absolutely ambiguous. On the other hand, Natural Language Processing (NLP) tasks such named entity recognition (automatic extraction of the names of things in texts: person or company names, locations) and part-of-speech tagging (automatic labeling of words according to their lexical classes: noun, verbs, adjectives) can be realized both by means of symbolic approach that rely on hand-coded rules, grammar and gazetteers and by following supervised machine learning-based approaches that need large quantities of manually annotated corpus. For Named Entity Recognition, symbolic systems tend to produce better results on well-formed texts [].

This chapter is composed of two main sections dedicated to text mining, information retrieval and information extraction. We begin by a general discussion about symbolic and numerical approaches for Natural Language Processing then we present the most popular tasks that have been evaluated through annual conferences for twenty years along with some resources one can employ to process texts. In the first section, we present some classical information retrieval models for document and Web searching and the principles of semantic information retrieval that can exploit specialized lexicon, thesaurus or ontologies. The second section introduces natural language processing for information extraction. We outline the most effective approaches for question-answering systems and semantic tagging of texts (named entities recognition, pattern extraction): rule and lexicon based approaches and machine learning approaches (Hidden Markov Models and Conditional Random Fields). Then, we present some approaches that aim to find information about entities and to populate knowledge bases. In this section, we describe the approaches we proposed and experimented in the last few years. The last section is dedicated to some industrial applications we work on and that respectively relate to digital libraries, marketing and dialog systems.

Symbolic and numerical approaches for natural language processing (NLP), have long been opposed, both linked to scientific communities distinguished by differing goals between limited but accurate prototypes and rough but functional systems and by attitudes more or less pragmatic. This opposition joined in some way the one that opposed and opposes always, some linguists and philosophers on the nature of language and its acquisition that is, in simplified terms, the pre-existence of a system (cognitive one) generating rules of possible sentences in a language (or at least the degree of pre-existence). This debate on the nature and role of grammar has its origins in the 17th century between proponents of empiricism (the human being is born empty and is fully shaped by experience) and rationalism (the man can not be reduced to experience). In the 1950s, the behaviorists, empiricist, attempted to define the acquisition of language learning as a form of chain reaction from positive reinforcement or negative one. In contrast, Chomsky proposed the pre-existence of mechanisms in the human brain that are specific to language and that could distinguish humans from other species. That suggests language is something really organic [] even in the very beginning of the life and that language learning does not only rely on association between stimuli and responses.

A direct consequence of the pre-existence of a minimalist program (generative grammar) in language acquisition has been to define both a universal grammar expressing linguistic universals and particular grammars for the specificities of given languages [].

The learning process can be viewed from two points of view, namely statistical learning or analytical (both unconscious and possibly combined in human mind). In the first case, the child has to observe which language productions lead to the goal (s)he fixed and, on a rolling basis, to accumulate a kind of accounting of what succeeds and what fails and allows him/her to achieve a selection of possibilities. Computational neural networks, involving different layers more or less explicit, linking lexicon, concepts and sounds, might simulate this kind of learning. In that sense, any communicational intention (any goal) and any linguistic production might be seen as specific paths within the neural network. The success or failure would result in strengthening or weakening connections and the issue of convergence during learning step arises as well as the reduction of combinatorial.

In the second case, learning is a progressive refinement of the value of the parameters of universal grammar allowing the production and the understanding, of the statements it can generate. Note that in the first case (statistical learning), a rule-based grammar can still be induced once the system is stabilized. It would allow producing explicit structural patterns, which would be necessary to improve consistency over time and mutual understanding between two dialoging people.