Ivan Idris - NumPy Beginners Guide

NumPy Beginner's Guide Third Edition Build efcient, high-speed programs using the high-performance NumPy mathematcal library Ivan Idris BIRMINGHAM - MUMBAI NumPy Beginner's Guide Third Edition Copyright 2015 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmited in any form or by any means, without the prior writen permission of the publisher, except in the case of brief quotatons embedded in critcal artcles or reviews. Every efort has been made in the preparaton of this book to ensure the accuracy of the informaton presented. However, the informaton contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark informaton about all of the companies and products mentoned in this book by the appropriate use of capitals.

However, Packt Publishing cannot guarantee the accuracy of this informaton. First published: November 2011 Second editon: April 2013 Third editon: June 2015 Producton reference: 1160615 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78528-196-9 www.packtpub.com Credits Author Project Coordinator Ivan Idris Shweta H. Birwatkar Reviewers Proofreader Alexandre Devert Safs Editng Davide Fiacconi Ardo Illaste Indexer Rekha Nair Commissioning Editor Amarabha Banerjee Graphics Sheetal Aute Acquisiton Editors Jason Monteiro Shaon Basu Usha Iyer Producton Coordinator Rebecca Youe Aparna Bhagat Content Development Editor Cover Work Aparna Bhagat Neeshma Ramakrishnan Technical Editor Rupali R. Shrawane Copy Editors Charlote Carneiro Vikrant Phadke Sameen Siddiqui About the Author Ivan Idris has an MSc in experimental physics.

His graduaton thesis had a strong emphasis on applied computer science. Afer graduatng, he worked for several companies as a Java developer, data warehouse developer, and QA Analyst. His main professional interests are business intelligence, big data, and cloud computng. Ivan enjoys writng clean, testable code and interestng technical artcles. He is the author of NumPy Beginner's Guide , NumPy Cookbook , Learning NumPy Array , and Python Data Analysis . You can fnd more informaton about him and a blog with a few examples of NumPy at http://ivanidris.net/ wordpress/ .

I would like to take this opportunity to thank the reviewers and the team at Packt Publishing for making this book possible. Also thanks go to my teachers, professors, colleagues, Wikipedia contributors, Stack Overfow contributors, and other authors who taught me science and programming. Last but not least, I would like to acknowledge my parents, family, and friends for their support. About the Reviewers Davide Fiacconi is completng his PhD in theoretcal astrophysics from the Insttute for Computatonal Science at the University of Zurich. He did his undergraduate and graduate studies at the University of Milan-Bicocca, studying the evoluton of collisional ring galaxies using hydrodynamic numerical simulatons. Davide's research now focuses on the formaton and coevoluton of supermassive black holes and galaxies, using both massively parallel simulatons and analytcal techniques.

In partcular, his interests include the formaton of the frst supermassive black hole seeds, the dynamics of binary black holes, and the evoluton of high-redshif galaxies. Ardo Illaste is a data scientst. He wants to provide everyone with easy access to data for making major life and career decisions. He completed his PhD in computatonal biophysics, prior to fully delving into data mining and machine learning. Ardo has worked and studied in Estonia, the USA, and Switzerland. www.PacktPub.com Support fles, eBooks, discount offers, and more For support fles and downloads related to your book, please visit www.PacktPub.com .

Did you know that Packt ofers eBook versions of every book published, with PDF and ePub fles available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are enttled to a discount on the eBook copy. Get in touch with us at service@packtpub.com for more details. At www.PacktPub.com , you can also read a collecton of free technical artcles, sign up for a range of free newsleters and receive exclusive discounts and ofers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib Do you need instant solutons to your IT questons? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entre library of books. Why subscribe? Fully searchable across every book published by Packt Copy and paste, print, and bookmark content On demand and accessible via a web browser Free access for Packt account holders If you have an account with Packt at www.PacktPub.com , you can use this to access PacktLib today and view 9 entrely free books.

Simply use your login credentals for immediate access. I dedicate this book to my aunt Lies who recently passed away. Rest in peace. Table of Contents Python 1 Time for acton installing Python on diferent operatng systems 2 The Python help system 3 Time for acton using the Python help system 3 Basic arithmetc and variable assignment 4 Time for acton using Python as a calculator 4 Time for acton assigning values to variables 5 The print() functon 6 Time for acton printng with the print() functon 6 Code comments 7 Time for acton commentng code 7 The if statement 8 Time for acton deciding with the if statement 8 The for loop 9 Time for acton repeatng instructons with loops 9 Python functons 11 Time for acton defning functons 11 Python modules 12 Time for acton importng modules 12 NumPy on Windows 13 Time for acton installing NumPy, matplotlib, SciPy, and IPython on Windows 13 NumPy on Linux 15 Time for acton installing NumPy, matplotlib, SciPy, and IPython on Linux 15 NumPy on Mac OS X 16 Time for acton installing NumPy, SciPy, matplotlib, and IPython with MacPorts or Fink 16 Building from source 16 Arrays 17 Time for acton adding vectors 17 IPython an interactve shell 21 Online resources and help 25 Summary 26 NumPy array object 28 Time for acton creatng a multdimensional array 29 Selectng elements 30 NumPy numerical types 31 Data type objects 33 Character codes 33 The dtype constructors 34 The dtype atributes 35 Time for acton creatng a record data type 35 One-dimensional slicing and indexing 36 Time for acton slicing and indexing multdimensional arrays 36 Time for acton manipulatng array shapes 39 Time for acton stacking arrays 41 Time for acton splitng arrays 46 Time for acton convertng arrays 51 Summary 51 File I/O 53 Time for acton reading and writng fles 54 Comma-seperated value fles 55 Time for acton loading from CSV fles 55 Volume Weighted Average Price 56 Time for acton calculatng Volume Weighted Average Price 56 The mean() functon 56 Time-weighted average price 57 Value range 58 Time for acton fnding highest and lowest values 58 Statstcs 59 Time for acton performing simple statstcs 59 Stock returns 62 Time for acton analyzing stock returns 63 Dates 65 Time for acton dealing with dates 65 Time for acton using the datetme64 data type 69 Weekly summary 70 Time for acton summarizing data 70 Average True Range 74 Time for acton calculatng Average True Range 75 Simple Moving Average 77 Time for acton computng the Simple Moving Average 77 Exponental Moving Average 80 Time for acton calculatng the Exponental Moving Average 80 Bollinger Bands 82 Time for acton enveloping with Bollinger Bands 83 Linear model 86 Time for acton predictng price with a linear model 86 Trend lines 89 Time for acton drawing trend lines 90 Methods of ndarray 94 Time for acton clipping and compressing arrays 94 Factorial 95 Time for acton calculatng the factorial 95 Missing values and Jackknife resampling 96 Time for acton handling NaNs with the nanmean(), nanvar(), and nanstd() functons 97 Summary 98 Correlaton 100 Time for acton trading correlated pairs 100 Polynomials 104 Time for acton ftng to polynomials 105 On-balance volume 108 Time for acton balancing volume 109 Simulaton 111 Time for acton avoiding loops with vectorize() 111 Smoothing 114 Time for acton smoothing with the hanning() functon 114 Initalizaton 118 Time for acton creatng value initalized arrays with the full() and full_like() functons 119 Summary 120 Matrices 122 Time for acton creatng matrices 122 Creatng a matrix from other matrices 123 Time for acton creatng a matrix from other matrices 123 Universal functons 125 Time for acton creatng universal functons 125 Universal functon methods 126 Time for acton applying the ufunc methods to the add functon 127 Arithmetc functons 129 Time for acton dividing arrays 129 Modulo operaton 131 Time for acton computng the modulo 131 Fibonacci numbers 132 Time for acton computng Fibonacci numbers 133 Lissajous curves 134 Time for acton drawing Lissajous curves 135 Square waves 136 Time for acton drawing a square wave 137 Sawtooth and triangle waves 138 Time for acton drawing sawtooth and triangle waves 139 Bitwise and comparison functons 140 Time for acton twiddling bits 141 Fancy indexing 143 Time for acton fancy indexing in-place for ufuncs with the at() method 144 Summary 144 Linear algebra 145 Time for acton invertng matrices 146 Solving linear systems 148 Time for acton solving a linear system 148 Finding eigenvalues and eigenvectors 149 Time for acton determining eigenvalues and eigenvectors 150 Singular value decompositon 151 Time for acton decomposing a matrix 152 Pseudo inverse 154 Time for acton computng the pseudo inverse of a matrix 154 Determinants 155 Time for acton calculatng the determinant of a matrix 155 Fast Fourier transform 156 Time for acton calculatng the Fourier transform 156 Shifing 158 Time for acton shifing frequencies 158 Random numbers 160 Time for acton gambling with the binomial 161 Hypergeometric distributon 163 Time for acton simulatng a game show 163 Contnuous distributons 165 Time for acton drawing a normal distributon 165 Lognormal distributon 167 Time for acton drawing the lognormal distributon 167 Bootstrapping in statstcs 169 Time for acton sampling with numpy.random.choice() 169 Summary 171 Sortng 173 Time for acton sortng lexically 174 Time for acton partal sortng via selecton for a fast median with the partton() functon 175 Complex numbers 176 Time for acton sortng complex numbers 177 Searching 178 Time for acton using searchsorted 178 Array elements extracton 179 Time for acton extractng elements from an array 179 Financial functons 180 Time for acton determining the future value 181 Present value 183 Time for acton getng the present value 183 Net present value 183 Time for acton calculatng the net present value 184 Internal rate of return 184 Time for acton determining the internal rate of return 185 Periodic payments 185 Time for acton calculatng the periodic payments 185 Number of payments 186 Time for acton determining the number of periodic payments 186 Interest rate 186 Time for acton fguring out the rate 186 Window functons 187 Time for acton plotng the Bartlet window 187 Blackman window 188 Time for acton smoothing stock prices with the Blackman window 189 Hamming window 190 Time for acton plotng the Hamming window 190 Kaiser window 191 Time for acton plotng the Kaiser window 192 Special mathematcal functons 192 Time for acton plotng the modifed Bessel functon 193 s inc 194 Time for acton plotng the sinc functon 194 Summary 196 Assert functons 198 Time for acton assertng almost equal 198 Approximately equal arrays 199 Time for acton assertng approximately equal 200 Almost equal arrays 200 Time for acton assertng arrays almost equal 201 Equal arrays 202 Time for acton comparing arrays 202 Ordering arrays 203 Time for acton checking the array order 203 Object comparison 204 Time for acton comparing objects 204 String comparison 204 Time for acton comparing strings 205 Floatng-point comparisons 205 Time for acton comparing with assert_array_almost_equal_nulp 206 Comparison of foats with more ULPs 207 Time for acton comparing using maxulp of 2 207 Unit tests 207 Time for acton writng a unit test 208 Nose test decorators 210 Time for acton decoratng tests 211 Docstrings 213 Time for acton executng doctests 214 Summary 215 Simple plots 217 Time for acton plotng a polynomial functon 218 Plot format string 219 Time for acton plotng a polynomial and its derivatves 219 Subplots 221 Time for acton plotng a polynomial and its derivatves 221 Finance 223 Time for acton plotng a year's worth of stock quotes 223 Histograms 226 Time for acton chartng stock price distributons 226 Logarithmic plots 228 Time for acton plotng stock volume 228 Scater plots 230 Time for acton plotng price and volume returns with a scater plot 230 Fill between 232 Time for acton shading plot regions based on a conditon 232 Legend and annotatons 234 Time for acton using a legend and annotatons 235 Three-dimensional plots 238 Time for acton plotng in three dimensions 238 Contour plots 240 Time for acton drawing a flled contour plot 240 Animaton 241 Time for acton animatng plots 241 Summary 243 MATLAB and Octave 245 Time for acton saving and loading a .mat fle 246 Statstcs 247 Time for acton analyzing random values 247 Sample comparison and SciKits 250 Time for acton comparing stock log returns 250 Signal processing 253 Time for acton detectng a trend in QQQ 253 Fourier analysis 256 Time for acton fltering a detrended signal 256 Mathematcal optmizaton 259 Time for acton ftng to a sine 259 Numerical integraton 263 Time for acton calculatng the Gaussian integral 263 Interpolaton 264 Time for acton interpolatng in one dimension 264 Image processing 266 Time for acton manipulatng Lena 266 Audio processing 268 Time for acton replaying audio clips 268 Summary 270 Pygame 271 Time for acton installing Pygame 272 Hello World 272 Time for acton creatng a simple game 272 Animaton 275 Time for acton animatng objects with NumPy and Pygame 275 matplotlib 278 Time for Acton using matplotlib in Pygame 278 Surface pixels 282 Time for Acton accessing surface pixel data with NumPy 282 Artfcial Intelligence 284 Time for Acton clustering points 284 OpenGL and Pygame 287 Time for Acton drawing the Sierpinski gasket 287 Simulaton game with Pygame 290 Time for Acton simulatng life 290 Summary 294 Python 299 Mathematcs and statstcs 300 Preface Scientsts, engineers, and quanttatve data analysts face many challenges nowadays. Data scientsts want to be able to perform numerical analysis on large datasets with minimal programming efort. They also want to write readable, efcient, and fast code that is as close as possible to the mathematcal language they are used to.