Sharad Saxena - Tree-Based Machine Learning Methods in SAS Viya

The correct bibliographic citation for this manual is as follows: Saxena, Sharad. 2022. Tree-Based Machine Learning Methods in SAS Viya . Cary, NC: SAS Institute Inc.

Tree-Based Machine Learning Methods in SAS Viya

Copyright 2022, SAS Institute Inc., Cary, NC, USA

ISBN 978-1-954846-71-5 (Hardcover)

ISBN 978-1-954846-63-0 (Paperback)

ISBN 978-1-954846-64-7 (Web PDF)
ISBN 978-1-954846-65-4 (EPUB)
ISBN 978-1-954846-66-1 (Kindle)

All Rights Reserved. Produced in the United States of America.

For a hard copy book: No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of the publisher, SAS Institute Inc.

For a web download or e-book: Your use of this publication shall be governed by the terms established by the vendor at the time you acquire this publication.

The scanning, uploading, and distribution of this book via the Internet or any other means without the permission of the publisher is illegal and punishable by law. Please purchase only authorized electronic editions and do not participate in or encourage electronic piracy of copyrighted materials. Your support of others rights is appreciated.

U.S. Government License Rights; Restricted Rights: The Software and its documentation is commercial computer software developed at private expense and is provided with RESTRICTED RIGHTS to the United States Government. Use, duplication, or disclosure of the Software by the United States Government is subject to the license terms of this Agreement pursuant to, as applicable, FAR 12.212, DFAR 227.7202-1(a), DFAR 227.7202-3(a), and DFAR 227.7202-4, and, to the extent required under U.S. federal law, the minimum restricted rights as set out in FAR 52.227-19 (DEC 2007). If FAR 52.227-19 is applicable, this provision serves as notice under clause (c) thereof and no other notice is required to be affixed to the Software or documentation. The Governments rights in Software and documentation shall be only those set forth in this Agreement.

SAS Institute Inc., SAS Campus Drive, Cary, NC 27513-2414

February 2022

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration.

Other brand and product names are trademarks of their respective companies.

SAS software may be provided with certain third-party software, including but not limited to open-source software, which is licensed under its applicable third-party software license agreement. For license information about third-party software distributed with SAS software, refer to http://support.sas.com/thirdpartylicenses .

Contents

About This Book

Preface

Decision trees are popular machine learning models. They are very intuitive and natural for us humans because in a way, they mimic the way we make decisions in our day-to-day lives. Several decision trees can be combined to produce ensemble models for better predictive performance than using a single decision tree. Decision trees and tree-based ensembles are supervised learning models used for problems involving classification and regression. They are largely used for prediction, but they also have several other uses in the modeling process and beyond.

Why you might want to use tree-based predictive modeling? Tree-based models are designed to handle categorical variables without you needing to create dummy variables. Tree-based models directly consider categorical levels when determining a split. Tree-based models can also handle observations with missing values instead of ignoring an observation with the missing value or being forced to impute. A tree can assign the missing value to one of those branches. This strategy can also be used to score an unknown level that occurs at scoring time but was not seen at training time. Tree-based models are also able to detect nonlinear relationships. Finally, a single decision tree can be interpreted as a set of rules or can easily be visualized in a tree plot.

SAS Visual Data Mining and Machine Learning software on SAS Viya has three tree-based predictive modeling techniques decision trees, forests, and gradient boosted trees. A decision tree is visually appealing and highly explainable. A forest model is a bagged ensemble of trees and has high predictive power but is harder to interpret than a single tree. A gradient boosting model is a boosted ensemble of trees and is very hard to interpret but has high predictive power. In SAS Visual Data Mining and Machine Learning, it is easy to start building one type of tree-based model and transition to building another. The output is similar so that you can understand the results more easily from one model to the next.

The analytics for all three tree-based predictive modeling techniques can be found in the Decision Tree Action Set in SAS Viya. These techniques can be accessed in the programming interface of SAS Visual Data Mining and Machine Learning when you use the TREESPLIT procedure, the FOREST procedure, or the GRADBOOST procedure. You can access these procedures through the SAS Studio HTML client. SAS Studio also has tasks that can help you generate code. SAS Visual Analytics has objects that can help you create these models interactively using a point-and-click interface. The actions in SAS Viya are also available to be called from other clients such as the SAS Visual Analytics client, Java, or Python, and more. For this book, we will primarily consider point-and-click visual interfaces that include mostly the pipelines in Model Studio, a bit of SAS Studio and SAS Visual Statistics. A little exposure of procedures in the SAS programming interface to the SAS Viya actions is also covered in the book.

The book includes discussions of tree-structured predictive models and the methodology for growing, pruning, and assessing decision trees, forests, and gradient boosted trees. You will acquire knowledge not only of how to use tree-based models for classification and regression, and some of their limitations, but also how the respective algorithms that shape them work. Each demonstration introduces a new data concern and then walks you through tweaking the modeling approach, modifying the properties, and changing the hyperparameters, thus building a right tree-based machine learning model. Along the way, you will gain experience making decision trees, forests, and gradient boosted trees that work for you.

The book also explains isolation forest (an unsupervised learning algorithm for anomaly detection), deep forest (an alternative for neural network deep learning), and Poisson and Tweedy gradient boosted regression trees. In addition, many of the auxiliary uses of trees, such as exploratory data analysis, dimension reduction, and missing value imputation are examined and running open source in SAS and SAS in open source is demonstrated.

What Does This Book Cover?

This book covers everything from using a single tree to more advanced bagging and boosting ensemble methods in SAS Viya.

provides an introduction to tree-structured models and the ones that are available in SAS Viya. Decision trees are powerful predictive and explanatory modeling tools. They are flexible in that they can model interval, ordinal, nominal, and binary targets, and they can accommodate nonlinearities and interactions. They are simple to understand and present. Model Studio is also introduced in this chapter.