Probabilistic Indexing for Information Search and Retrieval in Large Collections of Handwritten Text Images

This book provides a comprehensive presentation of a recently introduced framework, named "probabilistic indexing" (PrIx), for searching text in large collections of document images and other related applications. It fosters the development of new search engines for effective information retrieval from manuscripts which, however, lack the electronic text (transcripts) that would typically be required for such search and retrieval tasks.

The book is structured into 11 chapters and three appendices. The first two chapters briefly outline the necessary fundamentals and state of the art in pattern recognition, statistical decision theory, and handwritten text recognition. Chapter 3 presents approaches for indexing (as opposed to "spotting") each region of a handwritten text image which is likely to contain a word. Next, Chapter 4 describes models adopted for handwritten text in images, namely hidden Markov models, convolutional and recurrent neural networks and language models, and provides full details of weighted finite-state transducer (WFST) concepts and methods, needed in further chapters of the book. Chapter 5 explains the set of techniques and algorithms developed to generate image probabilistic indexes which allow for fast search and retrieval of textual information in the indexed images. Chapter 6 then presents experimental evaluations of the proposed framework and algorithms on different traditional benchmark datasets and compares them with other approaches, while Chapter 7 reviews the most popular keyword-spotting approaches. Chapter 8 explains how PrIx can support classical free-text search tools, while Chapter 9 presents new methods that use PrIx not only for searching, but also to deal with text analytics and other related natural language processing and information extraction tasks. Chapter 10 shows how the proposed solutions can be used to effectively index very large collections of handwritten document images, before Chapter 11 eventually summarizes the book and suggests promising lines of future research. The appendices detail the necessary mathematical foundations for the work and presents details of the text image collections and datasets used in the experiments throughout the book.

This book is written for researchers and (post-)graduate students in pattern recognition and information retrieval. It will also be of interest to people in areas like history, criminology, or psychology who need technical support to evaluate, understand or decode historical or contemporary handwritten text.

Alejandro Héctor Toselli, is currently working as a PostDoc (María Zambrano grant) at the Universitat Politècnica de València. He obtained an Electrical Engineer degree from the University Nacional de Tucumán (Argentina, 1997) and a Phd in Computer Science from the Universitat Politècnica de València (UPV) (Spain, 2004). His research expertise focuses primarily on Document Analysis and Recognition, in which he has more than 20 years of experience, publishing on these topics and working on related projects funded by European and US institutions. He held a Post-Doctoral Fellow at Northeastern University (Boston, USA) in the the multi-institutional Open Islamicate Texts Initiative (OpenITI) and at the "Institut de Recherche en Informatique et Systèmes Aléatoires" (IRISA, Rennes France).

Joan Puigcerver received his MSc and PhD in Computer Science from the Universitat Politècnica de València, in 2014 and 2018, respectively, focusing on probabilistic indexing and handwritten text recognition. In 2018, he joined Google Research as a software engineer. His research focuses on deep learning architectures, transfer learning, and computer vision. Joan is a member of the Spanish Society for Pattern Recognition and Image Analysis (AERFAI), an affiliate organization of the International Association for Pattern Recognition (IAPR).

Enrique Vidal is an emeritus professor of the Universitat Politècnica de València (Spain) and former co-leader of the PRHLT research center there. He is co-author of hundreds of research papers in the fields of Pattern Recognition, Multimodal Interaction and applications to Language, Speech and Image Processing and has led many important projects in these fields. Enrique is a fellow of the International Association for Pattern Recognition (IAPR).