Ronald Petty and Jan Newmarch - Network Programming with Go Language

Network Programming with Go Language Essential Skills for Programming, Using and Securing Networks with Open Source Google Golang Second Edition Dr. Jan Newmarch Ronald Petty Network Programming with Go Language: Essential Skills for Programming, Usingand Securing Networks with Open Source Google Golang Dr. Jan Newmarch Ronald Petty Oakleigh, VIC, Australia San Francisco, CA, USA ISBN-13 (pbk): 978-1-4842-8094-2 ISBN-13 (electronic): 978-1-4842-8095-9 https://doi.org/10.1007/978-1-4842-8095-9 Copyright 2022 by Jan Newmarch and Ronald Petty This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director, Apress Media LLC: Welmoed Spahr Acquisitions Editor: Steve Anglin Development Editor: James Markham Coordinating Editor: Mark Powers Cover designed by eStudioCalamar Cover image by Pat Kay on Unsplash (www.unsplash.com) Distributed to the book trade worldwide by Apress Media, LLC, 1 New York Plaza, New York, NY 10004, U.S.A. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc).

SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub.

For more detailed information, please visit https://github.com/Apress/network-prog-with-go-2e. Printed on acid-free paper I dedicate this to my family.Table of Contents About the Authors xvii About the Technical Reviewer xix Acknowledgments xxi Preface to the Second Edition xxiii Preface to the First Edition xxv Chapter 1: Architectural Layers 1 Protocol Layers 1 ISO OSI Protocol 1 OSI Layers 2 TCP/IP Protocol 2 Some Alternative Protocols 3 Networking 3 Gateways 4 Host-Level Networking 4 Packet Encapsulation 4 Connection Models 5 Connection Oriented 5 Connectionless 5 Communications Models 6 Message Passing 6 Remote Procedure Call 7 v Table of ConTenTs Distributed Computing Models 8 Client-Server System 9 Client-Server Application 9 Server Distribution 10 Communication Flows 11 Synchronous Communication 11 Asynchronous Communication 11 Streaming Communication 11 Publish/Subscribe 11 Component Distribution 12 Gartner Classification 12 Three-Tier Models 14 Fat vs Thin 15 Middleware Model 15 Middleware Examples 16 Middleware Functions 17 Continuum of Processing 17 Points of Failure 18 Acceptance Factors 18 Thoughts on Distributed Computing 19 Transparency 19 Access Transparency 19 Location Transparency 19 Migration Transparency 19 Replication Transparency 20 Concurrency Transparency 20 Scalability Transparency 20 Performance Transparency 20 Failure Transparency 20 vi Table of ConTenTs Eight Fallacies of Distributed Computing 20 Fallacy: The Network Is Reliable 21 Fallacy: Latency Is Zero 21 Fallacy: Bandwidth Is Infinite 21 Fallacy: The Network Is Secure 22 Fallacy: Topology Doesnt Change 22 Fallacy: There Is One Administrator 22 Fallacy: Transport Cost Is Zero 22 Fallacy: The Network Is Homogeneous 22 Conclusion 23 Chapter 2: Overview of the Go Language 25 Types 26 Slices and Arrays 26 Maps 28 Pointers 28 Functions 29 Structures 29 Methods 30 Multithreading 31 Packages 31 Modules 32 Type Conversion 32 Statements 33 GOPATH 33 Running Go Programs 33 Standard Libraries 33 Error Values 34 Conclusion 34 vii Table of ConTenTs Chapter 3: Socket-Level Programming 35 The TCP/IP Stack 35 IP Datagrams 36 UDP 36 TCP 37 Internet Addresses 37 IPv4 Addresses 37 IPv6 Addresses 38 IP Address Type 39 Using Available Documentation and Examples 40 The IPMask Type 43 Basic Routing 46 The IPAddr Type 48 Host Canonical Name and Addresses Lookup 49 Services 51 Ports 51 The TCPAddr Type 52 TCP Sockets 53 TCP Client 53 A Daytime Server 56 Multithreaded Server 57 Controlling TCP Connections 60 Timeout 60 Staying Alive 60 UDP Datagrams 60 Server Listening on Multiple Sockets 63 The Conn, PacketConn, and Listener Types 63 Raw Sockets and the IPConn Type 66 Conclusion 68 viii Table of ConTenTs Chapter 4: Data Serialization 69 Structured Data 69 Mutual Agreement 71 Self-Describing Data 71 Encoding Packages 72 ASN1 74 ASN1 Daytime Client and Server 84 JSON 86 A Client and A Server 90 The Gob Package 94 A Client and A Server 98 Encoding Binary Data As Strings 100 Protocol Buffers 103 Installing and Compiling Protocol Buffers 105 The Generated personv3pbgo File 105 Using the Generated Code 106 Conclusion 107 Chapter 5: Application-Level Protocols 109 Protocol Design 109 Why Should You Worry? 110 Version Control 110 The Web 111 Message Format 113 Data Format 113 Byte Format 113 Character Format 114 A Simple Example 115 A Stand-Alone Application 115 The Client-Server Application 116 ix Table of ConTenTs The Client Side 116 Alternative Presentation Aspects 117 The Server Side 117 Protocol: Informal 118 Text Protocol 118 Server Code 119 Client Code 121 Textproto Package 123 State Information 125 Application State Transition Diagram 127 Client-State Transition Diagrams 128 Server-State Transition Diagrams 128 Server Pseudocode 129 Conclusion 130 Chapter 6: Managing Character Sets and Encodings 131 Definitions 132 Character 132 Character Repertoire/Character Set 132 Character Code 132 Character Encoding 133 Transport Encoding 133 ASCII 133 ISO 8859 135 Unicode 135 UTF-8, Go, and Runes 136 UTF-8 Client and Server 137 ASCII Client and Server 137 UTF-16 and Go 137 Little-Endian and Big-Endian 138 UTF-16 Client and Server 138 Unicode Gotchas 141 x Table of ConTenTs ISO 8859 and Go 142 Other Character Sets and Go 145 Conclusion 145 Chapter 7: Security 147 ISO Security Architecture 147 Functions and Levels 148 Mechanisms 149 Data Integrity 150 Symmetric Key Encryption 153 Public Key Encryption 154 X509 Certificates 157 TLS 160 A Basic Client 160 Server Using a Self-Signed Certificate 162 Conclusion 165 Chapter 8: HTTP 167 URLs and Resources 167 i18n 167 HTTP Characteristics 169 Versions 169 HTTP/09 169 HTTP/10 170 HTTP 11 172 HTTP Major Upgrades 173 HTTP/2 173 HTTP/3 173 Simple User Agents 174 The Response Type 174 The HEAD Method 175 The GET Method 177 xi Table of ConTenTs Configuring HTTP Requests 181 The Client Object 182 Proxy Handling 184 Simple Proxy 185 Authenticating Proxy 187 HTTPS Connections by Clients 189 Servers 191 File Server 191 Handler Functions 193 Bypassing the Default Multiplexer 194 HTTPS 195 Conclusion 197 Chapter 9: Templates 199 Inserting Object Values 199 Using Templates 200 Pipelines 202 Defining Functions 203 Variables 205 Conditional Statements 206 The html/template Package 211 Conclusion 211 Chapter 10: A Complete Web Server 213 Browser Site Diagram 213 Browser Files 216 Basic Server 217 The listFlashCards Function 219 The manageFlashCards Function 222 The Chinese Dictionary 223 The Dictionary Type 224 xii Table of ConTenTs Flashcard Sets 224 Fixing Accents 226 The ListWords Function 229 The showFlashCards Function 231 Presentation on the Browser 234 Running the Server 234 Conclusion 234 Chapter 11: HTML 235 The html/template Package 236 Tokenizing HTML 237 XHTML/HTML 240 JSON 240 Conclusion 240 Chapter 12: XML 241 Unmarshalling XML 242 Marshalling XML 244 Parsing XML 245 The StartElement Type 246 The EndElement Type 246 The CharData Type 246 The Comment Type 246 The ProcInst Type 246 The Directive Type 247 XHTML 249 HTML 249 Conclusion 249 xiii Table of ConTenTs Chapter 13: Remote Procedure Call 251 Gos RPC 252 HTTP RPC Server 254 HTTP RPC Client 255 TCP RPC Server 256 TCP RPC Client 257 Matching Values 259 JSON 259 JSON RPC Server 259 JSON RPC Client 261 Conclusion 262 Chapter 14: REST 263 URIs and Resources 263 Representations 264 REST Verbs 265 The GET Verb 265 The PUT Verb 266 The DELETE Verb 266 The POST Verb 266 No Maintained State (That Is, Stateless) 267 HATEOAS 267 Representing Links 268 Transactions with REST 269 The Richardson Maturity Model 272 Flashcards Revisited 272 URLs 273 ServeMux (The Demultiplexer) 273 Content Negotiation 274 GET / 276 POST / 278 xiv Table of ConTenTs Handling Other URLs 278 The Complete Server 279 Client 285 Using REST or RPC 289 Conclusion 289 Chapter 15: WebSockets 291 WebSockets Server 292 The golangorg/x/net/websocket Package 292 The Message Object 292 The JSON Object 296 The Codec Type 299 WebSockets over TLS 302 WebSockets in an HTML Page 304 The githubcom/gorilla/websocket Package 308 Echo Server 308 Echo Client 310 Conclusion 311 Chapter 16: Gorilla 313 Middleware Pattern 313 Standard Library ServeMux Examples 316 Customizing Muxes 318 gorilla/mux 319 Why Should We Care 321 Gorilla Handlers 323 Additional Gorilla Examples 326 gorilla/rpc 327 gorilla/schema 328 gorilla/securecookie 329 Conclusion 331 xv Table of ConTenTs Chapter 17: Testing 333 Simple and Broken 333 httptest Package 336 Below HTTP 338 Leveraging the Standard Library 340 Conclusion 346 Appendix A: Fuzzing 347 Fuzzing in Go 347 Fuzzing Failures 350 Fuzzing Network-Related Artifacts 353 Conclusion 355 Appendix B: Generics 357 A Filtering Function Without Generics 358 Refactor Using Generics 359 Custom Constraints 360 Using Generics on Collections 362 How Not to Use Generics? 365 Conclusion 371 Index 373 xvi