• Complain

Elad Hazan - Introduction to Online Convex Optimization

Here you can read online Elad Hazan - Introduction to Online Convex Optimization full text of the book (entire story) in english for free. Download pdf and epub, get meaning, cover and reviews about this ebook. City: Cambridge, year: 2022, publisher: The MIT Press, genre: Computer. Description of the work, (preface) as well as reviews are available. Best literature library LitArk.com created for fans of good reading and offers a wide selection of genres:

Romance novel Science fiction Adventure Detective Science History Home and family Prose Art Politics Computer Non-fiction Religion Business Children Humor

Choose a favorite category and find really read worthwhile books. Enjoy immersion in the world of imagination, feel the emotions of the characters or learn something new for yourself, make an fascinating discovery.

No cover
  • Book:
    Introduction to Online Convex Optimization
  • Author:
  • Publisher:
    The MIT Press
  • Genre:
  • Year:
    2022
  • City:
    Cambridge
  • Rating:
    5 / 5
  • Favourites:
    Add to favourites
  • Your mark:
    • 100
    • 1
    • 2
    • 3
    • 4
    • 5

Introduction to Online Convex Optimization: summary, description and annotation

We offer to read an annotation, description, summary or preface (depends on what the author of the book "Introduction to Online Convex Optimization" wrote himself). If you haven't found the necessary information about the book — write in the comments, we will try to find it.

New edition of a graduate-level textbook on that focuses on online convex optimization, a machine learning framework that views optimization as a process.
In many practical applications, the environment is so complex that it is not feasible to lay out a comprehensive theoretical model and use classical algorithmic theory and/or mathematical optimization. Introduction to Online Convex Optimization presents a robust machine learning approach that contains elements of mathematical optimization, game theory, and learning theory: an optimization method that learns from experience as more aspects of the problem are observed. This view of optimization as a process has led to some spectacular successes in modeling and systems that have become part of our daily lives.
Based on the Theoretical Machine Learning course taught by the author at Princeton University, the second edition of this widely used graduate level text features:
  • Thoroughly updated material throughout
  • New chapters on boosting, adaptive regret, and approachability and expanded exposition on optimization
  • Examples of applications, including prediction from expert advice, portfolio selection, matrix completion and recommendation systems, SVM training, offered throughout
  • Exercises that guide students in completing parts of proofs

Elad Hazan: author's other books


Who wrote Introduction to Online Convex Optimization? Find out the surname, the name of the author of the book and a list of all author's works by series.

Introduction to Online Convex Optimization — read online for free the complete book (whole text) full work

Below is the text of the book, divided by pages. System saving the place of the last page read, allows you to conveniently read the book "Introduction to Online Convex Optimization" online for free, without having to search again every time where you left off. Put a bookmark, and you can go to the page where you finished reading at any time.

Light

Font size:

Reset

Interval:

Bookmark:

Make
Contents
List of Figures
List of Tables
Guide
Pagebreaks of the print version
Adaptive Computation and Machine Learning Francis Bach Editor Christopher - photo 1

Adaptive Computation and Machine Learning

Francis Bach, Editor

Christopher Bishop, David Heckerman, Michael Jordan, and Michael Kearns, Associate Editors

A complete list of books published in the Adaptive Computation and Machine Learning series appears at the back of this book.

Introduction to Online Convex Optimization

Second Edition

Elad Hazan

The MIT Press
Cambridge, Massachusetts
London, England

2022 Elad Hazan

This work is subject to a Creative Commons CC-BY-ND-NC license.

Subject to such license, all rights are reserved.

The MIT Press would like to thank the anonymous peer reviewers who provided - photo 2

The MIT Press would like to thank the anonymous peer reviewers who provided comments on drafts of this book. The generous work of academic experts is essential for establishing the authority and quality of our publications. We acknowledge with gratitude the contributions of these otherwise uncredited readers.

Library of Congress Cataloging-in-Publication Data

Names: Hazan, Elad, 1975 author.

Title: Introduction to online convex optimization / Elad Hazan.

Description: Second edition. | Cambridge: The MIT Press, 2022. | Series: Adaptive computation and machine learning series | Includes bibliographical references and index.

Identifiers: LCCN 2021050689 | ISBN 9780262046985 (hardcover)

Subjects: LCSH: Mathematical optimization. | Convex domains.

Classification: LCC QA402.5.H297 2022 | DDC 519.7/6dc23/eng/20211130

LC record available at https://lccn.loc.gov/2021050689

d_r0

To my family: Dana, Hadar, Yoav, Oded, and Deluca

you shall research and meditate therein day and night Yehoshua 18 Contents - photo 3

you shall research and meditate therein day and night

Yehoshua 1:8

Contents
  1. List of Figures
  2. List of Tables
Preface

This book serves as an introduction to the expanding theory of online convex optimization (OCO). It was written as an advanced textbook to serve as a basis for a graduate course, and/or as a reference to researchers diving into this fascinating world at the intersection of optimization and machine learning.

Such a course was given at the Technion in 20102014, with slight variations from year to year, and later at Princeton University in 20152020. The core material in these courses is fully covered in this book, along with exercises that allow students to complete parts of proofs, or that were found illuminating and thought-provoking by those taking the course. Most of the material is given with examples of applications, which are interlaced throughout various topics. These include prediction from expert advice, portfolio selection, matrix completion and recommendation systems, and support vector machine (SVM) training.

Our hope is that this compendium of material and exercises will be useful to you: the researcher and/or educator.

Placing This Book in the Machine Learning Library

The broad field of machine learning, as in the subdisciplines of online learning, boosting, regret minimization in games, universal prediction, and other related topics, have seen a plethora of introductory textbooks in recent years. With this note, we can hardly do justice to all of these, but perhaps point to the most related books on the topics of machine learning, learning in games, and optimization, whose intersection is our main focus.

The most closely related book, which served as an inspiration to the current book, and indeed an inspiration to the entire field of learning in games, is the wonderful text of Cesa-Bianchi and Lugosi (2006). From the literature on mathematical optimization theory, there are numerous introductory essays to convex optimization and convex analysis, to name only a few (Boyd and Vandenberghe 2004; Nesterov 2004; Nemirovski and Yudin 1983; Nemirovski 2004; Borwein and Lewis 2006; Rockafellar 1997). The author fondly recommends the text from which he has learned about mathematical optimization theory (Nemirovski 2004). The more broad texts on machine learning are too numerous to state here.

The primary purpose of this is to serve as an educational textbook for a dedicated course on OCO and the convex optimization approach to machine learning. OCO has already had enough impact to appear in several surveys and introductory texts (Hazan 2011; Shalev-Shwartz 2011; Rakhlin 2009; Rakhlin and Sridharan 2014). We hope this compilation of material and exercises will further enrich the literature.

Books Structure

This book is intended to serve as a reference for a self-contained course for graduate students in computer science/electrical engineering/operations research/statistics and related fields. As such, its organization follows the structure of the course Decision Analysis, taught at the Technion, and later Theoretical Machine Learning, taught at Princeton University.

Each chapter should take one or two weeks of classes, depending on the depth and breadth of the intended course. Chapter 1 is designed to be a teaser for the field, and thus it is less rigorous than the rest of the book.

Roughly speaking, the book can be conceived of as three units. The first, from chapters 2 through 4, contains the basic definitions, framework, and core algorithms for OCO. Chapters 5 to 7 contain more advanced algorithms and in-depth analysis of the framework and its extensions to other computational and information access models. The rest of the book deals with more advanced algorithms, more difficult settings, and relationships to well-known machine learning paradigms.

This book can assist educators in designing a complete course on the topic of OCO, or it can serve as a component in a comprehensive course on machine learning. An accompanying manual of solutions to selected exercises given in the book is available for educators only.

New in the Second Edition

The main additions to the second edition of this book include the following:

  • Expanded coverage of optimization in chapter 2, with a unified gradient descent analysis of the Polyak stepsize.
  • Expanded coverage of learning theory in chapter 9, with an introduction to compression and its use in generalization theory.
  • An expanded chapter 4, with discussion of the exponential weighted optimizer for exp-concave loss functions.
  • A revised chapter 5, with the addition of mirror descent analysis, as well as a revised section on adaptive gradient methods.
  • New chapter 10 on the notion of adaptive regret and algorithms for OCO with near-optimal adaptive regret bounds.
  • New chapter 11 on boosting and its relationship to OCO. Derivation of boosting algorithms from regret minimization.
  • New chapter 12 on online boosting.
  • New chapter 13 on Blackwell approachability and its strong connection to OCO.

In addition, numerous typos are fixed, exercises are corrected, and solutions to several questions have been made available in a separate manual for educators.

Acknowledgments
First Edition

First of all, I gratefully acknowledge the numerous contributions and insights from the students of the course Decision Analysis, given at the Technion during 20102014, as well as the students of Theoretical Machine Learning, a course taught at Princeton University during 20152016.

Next page
Light

Font size:

Reset

Interval:

Bookmark:

Make

Similar books «Introduction to Online Convex Optimization»

Look at similar books to Introduction to Online Convex Optimization. We have selected literature similar in name and meaning in the hope of providing readers with more options to find new, interesting, not yet read works.


Reviews about «Introduction to Online Convex Optimization»

Discussion, reviews of the book Introduction to Online Convex Optimization and just readers' own opinions. Leave your comments, write what you think about the work, its meaning or the main characters. Specify what exactly you liked and what you didn't like, and why you think so.