Juan A. Ael , Diego P. Montes and Javier Rodeiro Iglesias
Cloud and Serverless Computing for Scientists
A Primer
Juan A. Ael
Universidade de Vigo, Ourense, Spain
Diego P. Montes
Universidade de Vigo, Ourense, Spain
Javier Rodeiro Iglesias
Universidade de Vigo, Ourense, Spain
ISBN 978-3-030-41783-3 e-ISBN 978-3-030-41784-0
https://doi.org/10.1007/978-3-030-41784-0
Springer Nature Switzerland AG 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Foreword
As a researcher in a number of different domains over my career, all of which have benefited from the development in the areas of distributed computing, it is clear that we will continue on this journey of discovery and development of new tools and services for some time to come. We moved away from historic paradigms, such as the mainframe, with the rise of the personal computer and from then onto the increasingly powerful workstation, generic high-performance computing, grid computing, and now the cloud. One disadvantage of these changes though is that proponents of these new architectures often sell the newest paradigm as being better in nearly all ways than those that have proceeded it and therefore it should replace the systems following the older paradigms, moving users onto newer, bigger, better shinier systems. This is plainly not the case once hype has been ignored and as such the content of this book should be considered as aiming to educate the reader in terms of making sure that the right tool is always used for the right job.
I was struck when reading the book by the diversity of target audience within the research field, making sure that communities of practice, where the use of cloud will become ever more valuable are covered, with relevant examples is a great feature. I would though also make sure that one further cohort reads this book, the project PI or senior professor for whom this book should be required reading. Alongside more traditional responsibilities they are also now the ones who are under legal obligations to comply with requirements on open data and reproducibility, something that the migration of services and data storage to the cloud is extremely suitable for. They should also ensure that they understand the benefits and almost more importantly the challenges that a cloud-first strategy can bring.
When working with Juan and Diego prior to the development of this book, we spent a significant amount of time working through different models of the utilization of Infrastructure as a Service cloud computing for the environmental sciences, in particular climate science. Therefore, a large number of points made by this book have been developed through practical experience of the authors, not through desk-based research. Along with this with my background in the development of pan European Federated Cloud services for research, I can clearly see how lessons learnt, sometimes in very hard ways, in how researchers should be thinking about using the cloud could have been found quicker and more easily if such a text as this existed, or that we could have shared with members of the user community. As such I also see this book being of great use for those that will in time become users not only of large-scale public clouds to support their research but also in the usage of the European Open Science Cloud as it becomes a more and more useful tool to European Research.
I would finally highlight how the authors already say that in the fast-moving and dynamic world of cloud computing that things get out of date very quickly. Examples of this of course abound, though a key highlight is where the different deployment models of cloud computing mean that it is not always the same cloud that two different researchers talk about when they describe their method of use of cloud. Therefore, making sure that everyone understands the underlying model of cloud computing as is done early within the book is essential. So to conclude I would say that readers should, once they think they understand how they can use cloud computing within their research, stop, read, and then think about how they want to move forward with the cloud, making sure that they follow sound scientific and IT principles in how they deploy services within the cloud, testing all the time and recognize that cloud is not a magic bullet, but in the longer term it can make things much, much easier if their deployments are well designed.
Prof David Wallom
Preface
We have written this book for scientists, engineers, or anyone who wants to approach cloud computing, simply to know more about it or if they are considering or evaluating it as an alternative or complementary solution for their computing needs. Additionally, this book can be useful for IT professionals, such as Solutions Architects, who want to learn about the current and future needs of the scientific community and how they can be satisfied using cloud technologies, enabling them to offer more suitable proposals to their customers. Therefore, this text can provide a bridge between user and provider perspectives.
Moreover, this book attempts to provide a broad view of the current status of cloud computing and to serve as an introductory text for graduate, postgraduate, and Ph.D. students across every discipline, providing insight into potential uses and how it can help to develop projects and future work.
Juan A. Ael
Diego P. Montes
Javier Rodeiro Iglesias
Ourense, Spain Ourense, Spain Ourense, Spain
November 2019
Acknowledgements
We want to dedicate the book to our family and friends for their daily support and understanding, which makes it possible for us to devote our time to our research work, too often at the expense of our time with them.
We would also like to thank Springer for the opportunity to publish this book and their interest in it, especially Zachary Romano, our publishing editor here, and the AJE language editing service. Special thanks to Paulo Rodrguez from Dropbox for facilitating part of the work that has contributed to this book and useful discussions. Additionally, we would like to thank Google Enterprise, Microsoft Research,and Amazon Web Servicesfor granting us computational resources and facilitating technical support to perform experiments using their cloud computing platforms. We are deeply grateful to Tom Grey, Barak Regev, and Shailesh Rao from Googleand Kenji Takeda from Microsoft Research. We are also grateful to our colleagues at ClimatePrediction.net at the University of Oxford, in particular David Wallom, all the staff from the Supercomputing Centre of Galicia (CESGA), in particular Carlos Fernndez, and Toms Fernndez Pena of the Universidade de Santiago de Compostela.