Flex and bison are tools designed for writers of compilers and interpreters, although they are also useful for many applications that will interest noncompiler writers. Any application that looks for patterns in its input or has an input or command language is a good candidate for flex and bison. Furthermore, they allow for rapid application prototyping, easy modification, and simple maintenance of programs. To stimulate your imagination, here are a few things people have used flex and bison, or their predecessors lex and yacc, to develop:
Scope of This Book
, gives an overview of how and why flex and bison are used to create compilers and interpreters and demonstrates some simple applications including a calculator built in flex and bison. It also introduces basic terms we use throughout the book.
, describes how to use flex. It develops flex applications that count words in files, handle multiple and nested input files, and compute statistics on C programs.
, gives a full example using flex and bison to develop a fully functional desktop calculator with variables, procedures, loops, and conditional expressions. It shows the use of abstract syntax trees (ASTs), powerful and easy-to-use data structures for representing parsed input.
, develops a parser for the MySQL dialect of the SQL relational database language. The parser checks the syntax of SQL statements and translates them into an internal form suitable for an interpreter. It shows the use of Reverse Polish Notation (RPN), another powerful form used to represent and interpret parsed input.
, provide detailed descriptions of the features and options available to flex and bison programmers. These chapters and the two that follow provide technical information for the now-experienced flex and bison programmer to use while developing flex and bison applications.
, explains bison ambiguities and conflicts, which are grammar problems that keep bison from creating a parser from a grammar. It then develops methods that can be used to locate and correct such problems.
, discusses techniques that compiler or interpreter designers can use to locate, recognize, and report errors in the compiler input.
, covers reentrant scanners and parsers, Generalized Left to Right (GLR) parsers that can handle grammars that regular bison parsers cant, and interfaces to C++.
The appendix provides the complete grammar and a cross-reference for the SQL parser discussed in .
The glossary lists technical terms from language and compiler theory.
We presume you are familiar with C, because most examples are in C, flex, or bison, with a few in C++ and the remainder in SQL or the special-purpose languages developed within the text.
Conventions Used in This Book
The following conventions are used in this book:
ItalicUsed for new terms and concepts when they are introduced.
Constant Width
Used for program listings, as well as within paragraphs to refer to program elements such as statements, classes, macros, states, rules, all code terms, and files and directories.
Constant Bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values determined by context.
$
is the shell prompt.
[]
surround optional elements in a description of program syntax. (Dont type the brackets themselves.)
Tip
This icon signifies a tip, suggestion, or general note.
Caution
This icon indicates a warning or caution.
Getting Flex and Bison
Flex and bison are modern replacements for the classic lex and yacc that were both developed at Bell Laboratories in the 1970s. Yacc was the first of the two, developed by Stephen C. Johnson. Lex was designed by Mike Lesk and Eric Schmidt (the same Eric Schmidt who now heads Google) to work with bison. Both lex and yacc have been standard Unix utilities since Seventh Edition Unix in the 1970s.
The GNU Project of the Free Software Foundation distributes bison, a foreward-compatible replacement for yacc. It was originally written by Robert Corbett and Richard Stallman. The bison manual is excellent, especially for referencing specific features. Bison is included with all common distributions of BSD and Linux, but if you want the most up-to-date version, its home page is:
http://www.gnu.org/software/bison/ |
BSD and the GNU Project also distribute flex ( F ast Lex ical Analyzer Generator), a rewrite of lex intended to fix some of that tools many bugs and deficiencies. Flex was originally written by Jef Poskanzer; Vern Paxson and Van Jacobson have considerably improved it. Common distributions of BSD and Linux include a copy of flex, but if you want the latest version, its now hosted at SourceForge:
http://flex.sourceforge.net/ |
This Books Example Files
The programs in this book are available online as:
They can be downloaded by any web browser or FTP client. The zip format file can be decoded by the popular freeware unzip utility on Unix-ish and Linux systems or opened as a compressed folder on Windows XP or newer.
The examples in the book were all tested with flex version 2.5.35 and bison 2.4.1.
Using Code Examples
This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless youre reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from OReilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your products documentation does require permission.