Peter D. Lax - Linear Algebra and Its Applications

Each generation has its unique needs and aspirations. When Charles Wiley first opened his small printing shop in lower Manhattan in 1807, it was a generation of boundless potential searching for an identity. And we were there, helping to define a new American literary tradition. Over half a century later, in the midst of the Second Industrial Revolution, it was a generation focused on building the future. Once again, we were there, supplying the critical scientific, technical, and engineering knowledge that helped frame the world. Throughout the 20th Century, and into the new millennium, nations began to reach out beyond their own borders and a new international community was born. Wiley was there, expanding its operations around the world to enable a global exchange of ideas, opinions, and know-how.

For 200 years, Wiley has been an integral part of each generations journey, enabling the flow of information and understanding necessary to meet their needs and fulfill their aspirations. Today, bold new technologies are changing the way we live and learn. Wiley will be there, providing you the must-have knowledge you need to imagine new worlds, new possibilities, and new opportunities.

Generations come and go, but you can always count on Wiley to provide you the knowledge you need, when and where you need it!

Copyright 2007 by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate percopy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212) 850-6008, E-Mail: .

For ordering and customer service, call 1-800-CALL-WILEY.

Wiley Bicentennial Logo: Richard J. Pacifico

Library of Congress Cataloging-in-Publication Data:

Lax, Peter D.

Linear algebra and its applications / Peter D. Lax. 2nd ed.

p. cm. (Pure and applied mathematics. A Wiley-Interscience of texts, monographs and tracts)

Previous ed.: Linear algebra. New York: Wiley, c1997.

Includes bibliographical references and index.

ISBN 978-0-471-75156-4 (cloth)

1. Algebras, Linear. I. Lax, Peter D. Linear algebra. II. Title.

QA184.2.L38 2008

512.5dc22

2007023226

10 9 8 7 6 5 4 3 2 1

Preface

The outlook of this second edition is the same as that of the original: to present linear algebra as the theory and practice of linear spaces and linear mappings. Where it aids understanding and calculations, I dont hesitate to describe vectors as arrays of numbers and to describe mappings as matrices. Render onto Caesar the things which are Caesars.

If you can reduce a mathematical problem to a problem in linear algebra, you can most likely solve it, provided that you know enough linear algebra. Therefore, a thorough grounding in linear algebra is highly desirable. A sound undergraduate education should offer a second course on the subject, at the senior level. I wrote this book as a suitable text for such a course. The changes made in this second edition are partly to make it more suitable as a text. Terse descriptions, especially in the early chapters, were expanded, more problems were added, and a list of solutions to selected problems has been provided.

In addition, quite a bit of new material has been added, such as the compactness of the unit ball as a criterion of finite dimensionality of a normed linear space. A new chapter discusses the QR algorithm for finding the eigenvalues of a self-adjoint matrix. The Householder algorithm for turning such matrices into tridiagonal form is presented. I describe in some detail the beautiful observation of Deift, Nanda, and Tomei of the analogy between the convergence of the QR algorithm and Mosers theorem on the asymptotic behavior of the Toda flow as time tends to infinity.

Eight new appendices have been added to the first editions original eight, including the Fast Fourier Transform, the spectral radius theorem, proved with the help of the Schur factorization of matrices, and an excursion into the theory of matrix-valued analytic functions. Appendix 11 describes the Lorentz group, 12 is an interesting application of the compactness criterion for finite dimensionality, 13 is a characterization of commutators, 14 presents a proof of Liapunovs stability criterion, 15 presents the construction of the Jordan Canonical form of matrices, and 16 describes Carl Pearcys elegant proof of Halmos conjecture about the numerical range of matrices.

I conclude with a plea to include the simplest aspects of linear algebra in high-school teaching: vectors with two and three components, the scalar product, the cross product, the description of rotations by matrices, and applications to geometry. Such modernization of the high-school curriculum is long overdue.

I acknowledge with pleasure much help I have received from Ray Michalek, as well as useful conversations with Albert Novikoff and Charlie Peskin. I also would like to thank Roger Horn, Beresford Parlett, and Jerry Kazdan for very useful comments, and Jeffrey Ryan for help in proofreading.

PETER D. LAX

New York, New York

Preface to the First Edition

This book is based on a lecture course designed for entering graduate students and given over a number of years at the Courant Institute of New York University. The course is open also to qualified undergraduates and on occasion was attended by talented high school students, among them Alan Edelman; I am proud to have been the first to teach him linear algebra. But, apart from special cases, the book, like the course, is for an audience that has somenot muchfamiliarity with linear algebra.

Fifty years ago, linear algebra was on its way out as a subject for research. Yet during the past five decades there has been an unprecedented outburst of new ideas about how to solve linear equations, carry out least square procedures, tackle systems of linear inequalities, and find eigenvalues of matrices. This outburst came in response to the opportunity created by the availability of ever faster computers with ever larger memories. Thus, linear algebra was thrust center stage in numerical mathematics. This had a profound effect, partly good, partly bad, on how the subject is taught today.

The presentation of new numerical methods brought fresh and exciting material, as well as realistic new applications, to the classroom. Many students, after all, are in a linear algebra class only for the applications. On the other hand, bringing applications and algorithms to the foreground has obscured the structure of linear algebraa trend I deplore; it does students a great disservice to exclude them from the paradise created by Emmy Noether and Emil Artin. One of the aims of this book is to redress this imbalance.