Peteris Krumins - Perl One-Liners: 130 Programs That Get Things Done

Peteris Krumins is a programmer, systems administrator, blogger, and all-around hacker. He is currently running his own company, Browserling, which focuses on cross-browser testing. He has self-published three books on essential UNIX tools, and he enjoys open-sourcing hundreds of small projects on GitHub.

Find his website and blog at http://www.catonmat.net/ , follow @pkrumins on Twitter, and see his open source projects at http://github.com/pkrumins/ .

Alastair McGowan-Douglas lives in Rugby in the UK. He has been a Perl developer since 2008 and is now stuck writing PHP for a living. His favorite pastime at work is writing Perl scripts for internal use to encourage others to embrace the language. Also a JavaScript developer and Git aficionado, his rantings and musings on these various subjects can be found at http://altreus.blogspot.com/ .

Id like to thank Eric Pement for inspiring me to write this book; Bill Pollock for giving me the opportunity to publish it at No Starch Press; Riley Hoffman and Laurel Chun for working with me to make it perfect; Alastair McGowan-Douglas for his technical review; and David Precious, Gabor Szabo, Jim Davis, Chris Fedde, Andy Lester, John D. Cook, Jonathan Scott Duff, and Jeremy Mates for reviewing the book and making great suggestions for improvements. Id also like to thank everyone who helped me on the #perl IRC channel on freenode. If I forgot anyone, Im sorry, but thanks for helping me to get this book written!

Perl one-liners are small and awesome Perl programs that fit in a single line of code. They do one thing really welllike changing line spacing, numbering lines, performing calculations, converting and substituting text, deleting and printing specific lines, parsing logs, editing files in-place, calculating statistics, carrying out system administration tasks, or updating a bunch of files at once. Perl one-liners will make you a shell warrior: what took you minutes (or even hours) to solve will now take you only seconds!

In this introductory chapter, Ill show you what one-liners look like and give you a taste of whats in the rest of the book. This book requires some Perl knowledge, but most of the one-liners can be tweaked and modified without knowing the language in depth.

Lets look at some examples. Heres one:

perl -pi -e 's/ you / me /g' file

This one-liner replaces all occurrences of the text you with me in the file file. Very useful if you ask me. Imagine youre on a remote server and you need to replace text in a file. You can either open the file in a text editor and execute find-replace or simply perform the replacement through the command line and, bam, be done with it.

This one-liner and others in this book work well in UNIX. Im using Perl 5.8 to run them, but they also work in newer Perl versions, such as Perl 5.10 and later. If youre on a Windows computer, youll need to change them a little. To make this one-liner work on Windows, swap the single quotes for double quotes. To learn more about using Perl one-liners on Windows, see .

Ill be using Perls -e command-line argument throughout the book. It allows you to use the command line to specify the Perl code to be executed. In the previous one-liner, the code says do the substitution (s/you/me/g command) and replace you with me globally (/g flag). The -p argument ensures that the code is executed on every line of input and that the line is printed after execution. The -i argument ensures that file is edited in-place. Editing in-place means that Perl performs all the substitutions right in the file, overwriting the content you want to replace. I recommend that you always make a backup of the file youre working with by specifying the backup extension to the -i argument, like this:

perl -pi.bak -e 's/ you / me /g' file

Now Perl creates a file.bak backup file first and only then changes the contents of file.

How about doing this same replacement in multiple files? Just specify the files on the command line:

perl -pi -e 's/ you / me /g' file1 file2 file3

Here, Perl first replaces you with me in file1 and then does the same in file2 and file3.

You can also perform the same replacement only on lines that match we, as simply as this:

perl -pi -e 's/ you / me /g if / we /' file

Here, you use the conditional if /we/ to ensure that s/you/me/g is executed only on lines that match the regular expression /we/.

The regular expression can be anything. Say you want to execute the substitution only on lines with digits in them. You could use the /\d/ regular expression to match numbers:

perl -pi -e 's/ you / me /g if /\d/' file

How about finding all lines in a file that appear more than once?

perl -ne 'print if $a{$_}++' file

This one-liner records the lines youve seen so far in the %a hash and counts the number of times it sees the lines. If it has already seen the line, the condition $a{$_}++ is true, so it prints the line. Otherwise it automagically creates an element that contains the current line in the %a hash and increments its value. The $_ special variable contains the current line. This one-liner also uses the -n command-line argument to loop over the input, but unlike -p, it doesnt print the lines automatically. (Dont worry about all the command-line arguments right now; youll learn about them as you work through this book!)

How about numbering lines? Super simple! Perls $. special variable maintains the current line number. Just print it together with the line:

perl -ne 'print "$. $_"' file

You can do the same thing by using the -p argument and modifying the $_ variable:

perl -pe '$_ = "$. $_"' file

Here, each line is replaced by the string "$. $_", which is equal to the current line number followed by the line itself. (See one-liner 3.1 on page 17 for a full explanation.)

If you omit the filename at the end of the one-liner, Perl reads data from standard input. From now on, Ill assume the data comes from the standard input and drop the filename at the end. You can always put it back if you want to run one-liners on whole files.

You can also combine the previous two one-liners to create one that numbers only the repeated lines:

perl -ne 'print "$. $_" if $a{$_}++'

Another thing you can do is sum the numbers in each line using the sum function from the List::Util CPAN module. CPAN (Comprehensive Perl Archive Network; http://www.cpan.org/ ) is an archive of over 100,000 reusable Perl modules. List::Util is one of the modules on CPAN, and it contains various list utility functions. You dont need to install this module because it comes with Perl. (Its in Perl core.)