Syed Ejaz Ahmed - Post-Shrinkage Strategies in Statistical and Machine Learning for High Dimensional Data

This book presents some post-estimation and predictions strategies for the host of useful statistical models with applications in data science. It combines statistical learning and machine learning techniques in a unique and optimal way. It is well-known that machine learning methods are subject to many issues relating to bias, and consequently the mean squared error and prediction error may explode. For this reason, we suggest shrinkage strategies to control the bias by combining a submodel selected by a penalized method with a model with many features. Further, the suggested shrinkage methodology can be successfully implemented for high-dimensional data analysis. Many researchers in statistics and medical sciences work with big data. They need to analyze this data through statistical modeling. Estimating the model parameters accurately is an important part of the data analysis. This book may be a repository for developing improve estimation strategies for statisticians. This book will help researchers and practitioners for their teaching and advanced research and is an excellent textbook for advanced undergraduate and graduate courses involving shrinkage, statistical, and machine learning.

• The book succinctly reveals the bias inherited in machine learning method and successfully provides tools, tricks, and tips to deal with the bias issue.

• Expertly sheds light on the fundamental reasoning for model selection and post-estimation using shrinkage and related strategies.

• This presentation is fundamental because shrinkage and other methods appropriate for model selection and estimation problems, and there is a growing interest in this area to fill the gap between competitive strategies.

• Application of these strategies to real-life data set from many walks of life.

• Analytical results are fully corroborated by numerical work, and numerous worked examples are included in each chapter with numerous graphs for data visualization.

• The presentation and style of the book clearly makes it accessible to a broad audience. It offers rich, concise expositions of each strategy and clearly describes how to use each estimation strategy for the problem at hand.

• This book emphasizes that statistics/statisticians can play a dominant role in solving Big Data problems and will put them on the precipice of scientific discovery.

• The book contributes novel methodologies for HDDA and will open a door for continued research in this hot area.

• The practical impact of the proposed work stems from wide applications. The developed computational packages will aid in analyzing a broad range of applications in many walks of life.

Designed cover image: Askhat Gilyakhov

First edition published 2023

by CRC Press

6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742

and by CRC Press

4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

CRC Press is an imprint of Taylor & Francis Group, LLC

2023 Syed Ejaz Ahmed, Feryaal Ahmed and Bahadir Yzbai

Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged, please write and let us know so we may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, access

Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe.

ISBN: 978-0-367-76344-2 (hbk)

ISBN: 978-0-367-77205-5 (pbk)

ISBN: 978-1-003-17025-9 (ebk)

DOI: 10.1201/9781003170259

Typeset in CMR10

by KnowledgeWorks Global Ltd.

Publisher's note: This book has been prepared from camera-ready copy provided by the authors.

Dedicated in loving memory to Don Fraser and Kjell Doksum.

The discipline of statistical science is ever changing and evolving from the investigation of classical finite-dimensional data to high-dimensional data analysis. We are commonly encountering data sets containing huge numbers of predictors where in some cases the number of predictors exceeds the number of sample observations. Many modern scientific investigations require the analysis of enormous, complex, high-dimensional data far beyond the classical statistical methodologies developed decades ago. For example, data from genomic, proteomic, spatial-temporal, social network, and many other disciplines fall into this category. Modeling and making statistical sense of high-dimensional data is a challenging problem. A range of different models with increasing complexity can be considered, and a model that is optimal in some sense needs to be selected from a set of candidate models. Simultaneous variable selection and model parameter estimation play a central role in such investigations. There is a massive literature on variable selection and penalized regression methods that are currently available. A plethora of interesting and useful developments have been recently published in scientific and statistical journals. This area of research continues to grow in the foreseeable future.

The application of regression models for high-dimensional data analysis is challenging and rewarding task. Regularization/penalization methods have attracted much attention in this arena. Penalized regression is a technique for mitigating the difficulties that arise from collinearity and high dimensionality. This approach inherently incurs an estimation bias while reducing the variance of the estimator. A tuning parameter is needed to adjust the penalization effects so that a balance between model parsimony and goodness-of-fit can be achieved. Different forms of penalty functions have been studied intensively over the last three decades. However, development in this area is still in its infancy. For example, methods may require the assumption of sparsity in the model where most coefficients are exactly zero and the nonzero coefficients are big enough to be separated from the zero ones. There are situations where noise cannot easily be separated from the signal, especially in the presence of weak signals. Furthermore, penalty estimators are not efficient when the number of variables is extremely large compared to the sample size. To mitigate these problems, I suggested the shrinkage strategy, which combines a model containing strong signals with a model with weak signals. One of the goals of this book is to improve the understanding of high-dimensional modeling from an integrative perspective and to bridge the gap among statisticians, computer scientists, applied mathematicians and others in understanding each other's tools. This book highlights and expands the breadth of the existing methods in high-dimensional data analysis and their potential to advance both statistical learning and machine learning for future research in the theory of shrinkage strategies.