Clinton L. Jeffery - Build Your Own Programming Language: A programmers guide to designing compilers, interpreters, and DSLs

Copyright 2021 Packt Publishing

This is an Early Access product. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the content and extracts of this book may evolve as it is being developed to ensure it is up-to-date.

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, ortransmitted in any form or by any means, without the prior written permission of the publisher,except in the case of brief quotations embedded in critical articles or reviews.

The information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing or its dealers and distributors will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Early Access Publication: Build Your Own Programming Language

Early Access Production Reference: B15963

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK

ISBN: 978-1-80020-480-5

Copyright 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Welcome to Packt Early Access. Were giving you an exclusive preview of this book before it goes on sale. It can take many months to write a book, but our authors have cutting-edge information to share with you today. Early Access gives you an insight into the latest developments by making chapter drafts available. The chapters may be a little rough around the edges right now, but our authors will update them over time. Youll be notified when a new version is ready.

This title is in development, with more chapters still to be written, which means you have the opportunity to have your say about the content. We want to publish books that provide useful information to you and other customers, so well send questionnaires out to you regularly. All feedback is helpful, so please be open about your thoughts and opinions. Our editors will work their magic on the text of the book, so wed like your input on the technical elements and your experience as a reader. Well also provide frequent updates on how our authors have changed their chapters based on your feedback.

You can dip in and out ofthis bookorfollow alongfrom start to finish; Early Access is designed to be flexible.We hope you enjoy getting to know more about the process of writing a Packt book. Join the exploration of new topics by contributing your ideas and see them come to life in print.

1. Why Build Another Programming Language?
2. Programming Language Design
3. Scanning Source Code
4. Parsing
5. Syntax Trees
6. Symbol Tables
7. Checking Base Types
8. Checking Types on Arrays, Method Calls, and Structure Accesses
9. Intermediate Code Generation
10. Syntax Coloring in an IDE
11. Bytecode Interpreters
12. Generating Bytecode
13. Native Code Generation
14. Implementing Operators and Built-in Functions
15. Domain Control Structures
16. Garbage Collection
17. Final Thoughts
18. Unicon Essentials

This book will show you how to build your own programming language, but first, you should ask yourself, why would I want to do this? For a few of you, the answer will be simple: because it is so much fun. However, for the rest of us, it is a lot of work to build a programming language, and we need to be sure about it before we make a start. Do you have the patience and persistence that it takes?

This chapter points out a few good reasons for building your own programming language, as well as some situations where you dont have to build your contemplated language after all; designing a class library for your application domain might be simpler and just as effective. However, libraries have their downsides, and sometimes only a new language will do.

After this chapter, the rest of this book will, having considered things carefully, take for granted that you have decided to build a language. In that case, you should determine some of the requirements for your language. But first, were going to cover the following main topics in this chapter:

• Motivations for writing your own programming language
• The difference between programming languages and libraries
• The applicability of programming language tools to other software projects
• Establishing the requirements for your language
• A case study that discusses the requirements for the Unicon language

Lets start by looking at motivations.

Sure, some programming language inventors are rock stars of computer science, such as Dennis Ritchie or Guido van Rossum! But becoming a rock star of computer science was easier back then. I heard the following report a long time ago from an attendee of the second History of Programming Languages Conference: The consensus was that the field of programming languages is dead. All the important languages have been invented already. This was proven wildly wrong a year or 2 later when Java hit the scene, and perhaps a dozen times since then when languages such as Go emerged. After a mere 6 decades, it would be unwise to claim our field is mature and that theres nothing new to invent that might make you famous.

Still, celebrity is a bad reason to build a programming language. The chances of acquiring fame or fortune from your programming language invention are slim. Curiosity and desire to know how things work are valid reasons, so long as youve got the time and inclination, but perhaps the best reasons for building your own programming language are need and necessity.

Some folks need to build a new language or a new implementation of an existing programming language to target a new processor or compete with a rival company. If thats not you, then perhaps youve looked at the best languages (and compilers or interpreters) available for some domain that you are developing programs for, and they are missing some key features for what you are doing, and those missing features are causing you pain. Every once in a blue moon, someone comes up with a whole new style of computing that a new programming paradigm requires a new language for.

While we are discussing your motivations for building a language, lets talk about the different kinds of languages, organization, and the examples this book will use to guide you. Each of these topics is worth looking at.

Whatever your reasons, before you build a programming language, you should pick the best tools and technologies you can find to do the job. In our case, this book will pick them for you. First, there is a question of the implementation language that you are building your language in. Programming language academics like to brag about writing their language in that language itself, but this is usually only a half-truth (or someone was being very impractical and showing off at the same time). There is also the question of just what kind of programming language implementation to build: