Sheetal S. Sonawane - Information Retrieval and Natural Language Processing : A Graph Theory Approach

Calmness, gentleness, silence, self-restraint, and purity: these are the disciplines of the mind

Bhagvad Gita

This book envisioned to present the precise and summarized contents that put more highlights on a mathematical view using graph theory for natural language processing and information retrieval methods and approaches and how graph theory can make better insights from these use cases. Since last decade, there is much advancement in utilizing graph properties in NLP and IR in preprocessing steps. In addition to this, the traditional BoW approach was found to be not sufficient for major text analytics task. The order and co-occurrence relation are achieved easily with graph. Also, major advanced graph types like hypergraph and semigraph are found to be applicable in various IR tasks. The state-of-the-art methods and techniques provide improvement in the performance of the system.

As the data increasing exponentially, unblocking true potential of this growing data is becoming a real challenge. Due to advancement in semiconductor industry, the space is getting cheaper day by day, and in the sequel storage, extraction, transformation and loading of the data are becoming easier. However, the major challenge faced by all IT leaders today is the information retrieval and making sense out of this big data by drawing meaningful insights. The main focus of this book is to present and discuss information retrieval using graph theory and different approaches. This book also presents the role of natural language processing in informational retrieval using appropriate case studies.

This book gives a comprehensive view of graph theory in IR and NLP. This book provides a number of techniques and examples from state of the art. The book provides understanding of graph theory basics, graph algorithms and networks using graph. The book is divided into three parts and contains nine chapters. The first part gives graph theory basics and graph networks, and the second part provides graph-based information retrieval. The third part covers IR recent applications using graph. This book provides a strong foundation to a beginner. All the technical details that include tools and technologies used for graph algorithms and implementation are explained in a clear and organized format.

Part I discusses graph theory basics, graph algorithms and network using graph. It lays the groundwork for the remainder of the book by introducing all necessary and important concepts in graph theory, including the notation, graph types, graph properties and graph representations. An overview of wide variety of graph methods, including shortest path algorithm, depth first search and breadth first search, maxflow network, graph clustering, graph coloring and more, is explained with examples and detail illustration. The term/concept network is a naturally occurring relations. This shows its adequacy to model power-law degree distribution. Zips and Heaps law are also explained with code. Further, social network measures like centrality, degree distributions, clustering coefficients are explained using examples.

Part II provides the basic of information retrieval, text document preprocessing using graph theory and graph theory in information retrieval. It gives insight of various steps included in information retrieval starting from preprocessing, indexing and then ranking. In text document, there are different components like words, phrases, sentences and paragraphs. These text components are connected to each other through various relationships which help to find meaning and relatedness in the document. The relationship among these components may be based on their occurrences in the document or similarity based on their context or concept. The graph representation of text along with graph operations for necessary preprocessing algorithms is explained. The graph characteristics, properties and operations useful to solve document preprocessing task is well explained in this part. The task of getting relevant document based on input query is important in information retrieval. The concept of term weighting in graph with generating extractive and abstractive summary by using graph operations is well explained with different graph types and properties. The applications like question answer system, sentiment analysis and recommendation system are also explored in this part. The connection of data elements is provided semantically using knowledge graph. The importance and its applications are discussed in this chapter. The detailing of its implementation using ontology, RDF, SPARQL and Neo4j is introduced in this part. The open source libraries, datasets and tools are provided in the last part of the book.