Minrui Zheng - Spatially Explicit Hyperparameter Optimization for Neural Networks

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd.

The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Neural networks as a commonly used machine learning algorithms (e.g., artificial neural networks and convolutional neural networks) have been extensively used in GIScience domain to explore the nonlinear or/and complex geographic phenomena. However, how to automatically adjust the parameters of neural networks is still an open question in GIScience. Moreover, the model performance of neural networks often depends on the parameter setting for a given dataset. Meanwhile, adjusting the parameter configuration of neural networks will increase the overall running time. In this book, the author proposed an automated spatially explicit hyperparameter optimization approach to identify optimal or near-optimal parameter settings for neural networks and accelerate the search process through both model and computing levels. The author used two spatial prediction models in this book to examine the utilities of spatially explicit hyperparameter optimization. The results demonstrate that the approach proposed in this book improves the computing performance at model and computing levels and addresses the challenge of finding optimal parameter settings for neural networks in the GIScience field.

In the remainder of this book, Chap. concludes this book.

My deepest gratitude goes to my committee members, Drs. Wenwu Tang, Elizebeth Delmelle, Minwoo Lee, and Akin Ogundiran for their support and guidance on this work.

I owe many thanks to former and current members of Center for Applied GIScience (Dr. Michael Desjardins, Dr. Alexander Hohl, Yu Lan, Dr. Jianxin Yang, Tianyang Chen, Zachery Slocum) and faculty (Drs. Heather Smith, Eric Delmelle, Craig Allan, Yu Wang, and Lisa Russell-Pinson) at the University of North Carolina at Charlotte who have helped and encouraged me. Also, I would like to express my appreciation to my friends (Greg Verret, Amanda Verret, Mark Verret, Nathan Verret, Yi Zhang, Li Liu, Qiang Li, and Jiayang Li).

In the past decades, with the increasing volume of spatial data and development of cutting-edge techniques, several spatial models have been created to investigate complex spatial phenomena and explore spatial process (Goodchild ).

Spatial modeling embraces a series of models and techniques that explore relationships, patterns, and phenomena across space and time. The steps of spatial modeling often proceed in a sequence from problem specification, model theory, data preparation, model verification, calibration, and evaluation to prediction (see Fig. ).

Fig. 1.1

Spatial modeling process

(Adapted from Shannon )

Although spatial modeling exists in a number of research areas, one of the vital parts of spatial modeling is algorithms. Figure illustrates the process of algorithms from input X to output Y. Algorithms are a sequence of computational steps that transform the input into the output. In each spatial modeling exercise, it includes one or multiple model units which have a single algorithm and related parameters. There are two types of parameters based on their contributions to spatial modeling: standard parameters and hyperparameters. Standard parameters are an internal component of spatial modeling, and their values usually are derived from models, such as coefficients of regression models and coefficients of objective functions of optimization models.

Fig. 1.2

Illustration of algorithms (Y=f(X); Y=[y1, y2,, yn]; X=[x1, x2,, xm]; f is the algorithm between X and Y), revised from Gahegan ()

Hyperparameters are an external component of spatial modeling, and the values are user-defined or pre-defined by other algorithms. Hyperparameters usually influence the algorithms themselves and the derivation of standard parameters. Some examples of hyperparameters are the learning rate of artificial neural networks (ANNs), the initial number of clusters of k-means clustering, and the number of trees in random forest algorithm. However, hyperparameters also exist in rule-based spatial modeling, such as the cellular automata (CA) model. Stochastic disturbance term from transition rules is a hyperparameter for CA, which allows the CA model generated patterns to be closed to reality (Yeh and Li ).