Perl Power Tools (source)

Rob Hoelz

hoelz.ro

Booking.com

Introduction

A lot of programmers new to a language familiarize themselves with the syntax of the language as well as the libraries for accomplishing some common tasks, but when something unexpected happens or when something goes wrong, you need to learn about the unsung heroes that a programming environment offers: the tools.

Introduction

Introduction

A lot of programmers new to a language familiarize themselves with the syntax of the language as well as the libraries for accomplishing some common tasks, but when something unexpected happens or when something goes wrong, you need to learn about the unsung heroes that a programming environment offers: the tools.

There will be a number of demos over the course of this talk; the source for them is located under demos in the Git repository.

Module Installation

Over the course of this presentation, we'll be building up a developer's workbench. All of the tools I'm about to show you are available on the CPAN, so let's cover how to install them first.

Module Installation

Installing modules the old way

Module Installation

Installing modules the old way

Module Installation

Installing modules the old way

Module Installation

Installing modules the old way

Module Installation

Installing modules the old way

Note that all of these still have valid uses; some modules require a little more tender love and care and need to be installed manually; sometimes all you have on a machine is cpan, and sometimes you want to only work with a set of 'blessed' modules in your package repository.

Module Installation

So now that we've talked about the old way...

...what's the new way?

Module Installation

cpanminus (aka cpanm)

Module Installation

cpanminus (aka cpanm)

Unlike cpan, cpanm stays out of your way.

Module Installation

cpanminus (aka cpanm)

Unlike cpan, cpanm stays out of your way.

Module Installation

cpanminus (aka cpanm)

Unlike cpan, cpanm stays out of your way.

Module Installation

cpanminus (aka cpanm)

Unlike cpan, cpanm stays out of your way.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

cpanm is a zero-configuration tool, but it does support special options if you need them.

Module Installation

But what if you don't want to install modules globally? Maybe...

Module Installation

But what if you don't want to install modules globally? Maybe...

Module Installation

But what if you don't want to install modules globally? Maybe...

Module Installation

But what if you don't want to install modules globally? Maybe...

Module Installation

The solution: local::lib

local::lib allows you to maintain a project-specific (or user-specific, or whatever) directory for modules.

Module Installation

The solution: local::lib

local::lib allows you to maintain a project-specific (or user-specific, or whatever) directory for modules.

For example, I have a local::lib under $HOME/devlib at work so I can keep my tools isolated from production stuff.

Regular Expressions

As Perl programmers, we often leverage the power of regular expressions...

Regular Expressions

Regular Expressions

As Perl programmers, we often leverage the power of regular expressions...

...but their power and expressiveness can make them easy to abuse and tricky to debug. Have a look at this regex:

Regular Expressions

qr{

(?&VALUE) (?{ $_ = $^R->[1] })

(?(DEFINE)

(?<OBJECT>
  (?{ [$^R, {}] })
  \{
    (?: (?&KV) # [[$^R, {}], $k, $v]
      (?{ # warn Dumper { obj1 => $^R };
     [$^R->[0][0], {$^R->[1] => $^R->[2]}] })
      (?: , (?&KV) # [[$^R, {...}], $k, $v]
        (?{ # warn Dumper { obj2 => $^R };
       [$^R->[0][0], {
      )*
    )?
  \}
)

(?<KV>
  (?&STRING) # [$^R, "string"]
  : (?&VALUE) # [[$^R, "string"], $value]
  (?{ # warn Dumper { kv => $^R };
     [$^R->[0][0], $^R->[0][1], $^R->[1]] })
)

(?<ARRAY>
  (?{ [$^R, []] })
  \[
    (?: (?&VALUE) (?{ [$^R->[0][0], [$^R->[1]]] })
      (?: , (?&VALUE) (?{ # warn Dumper { atwo => $^R };
             [$^R->[0][0], [@{$^R->[0][1]}, $^R->[1]]] })
      )*
    )?
  \]
)

(?<VALUE>
  \s*
  (
      (?&STRING)
    |
      (?&NUMBER)
    |
      (?&OBJECT)
    |
      (?&ARRAY)
    |
    true (?{ [$^R, 1] })
  |
    false (?{ [$^R, 0] })
  |
    null (?{ [$^R, undef] })
  )
  \s*
)

(?<STRING>
  (
    "
    (?:
      [^\\"]+
    |
      \\ ["\\/bfnrt]
#    |
#      \\ u [0-9a-fA-f]{4}
    )*
    "
  )

  (?{ [$^R, eval $^N] })
)

(?<NUMBER>
  (
    -?
    (?: 0 | [1-9]\d* )
    (?: \. \d+ )?
    (?: [eE] [-+]? \d+ )?
  )

  (?{ [$^R, eval $^N] })
)

) }xms;

Regular Expressions

I didn't even include the whole thing; this is a regex that venerable Perl hacker Randal Schwartz wrote for parsing JSON. It's very clever, but if something like this breaks while you're writing it, you've got a problem.

Regular Expressions

I didn't even include the whole thing; this is a regex that venerable Perl hacker Randal Schwartz wrote for parsing JSON. It's very clever, but if something like this breaks while you're writing it, you've got a problem.

Perl Problems

Regular Expressions

So...how do we debug regexes?

Regular Expressions

So...how do we debug regexes?

Regular Expressions

So...how do we debug regexes?

Regular Expressions

So...how do we debug regexes?

Regular Expressions

Enter Regexp::Debugger.

Regular Expressions

DEMO (the screen refreshes are a bit annoying, but hopefully the point is illustrated)

http://showterm.io/74d851d162bf5c93b5bb1

Regular Expressions

Take note: Regexp::Debugger is lexical. That means that we can debug certain regular expression while leaving the rest of the program alone.

In addition to helping you debug hairy regexes, Regexp::Debugger can also help you learn how the Perl interpreter executes a regular expression. For example, did you know that $ doesn't really mean end of string? Run it through Regexp::Debugger (or consult perlreref) to find out more!

Inspecting Objects

At work, I do a lot with web services. Sometimes, I like to dump the result of a web service call to see exactly what I'm getting from the service. So how can we dump an arbitrarily-complicated structure to the console?

Inspecting Objects

Inspecting Objects

Inspecting Objects

Data::Printer

Inspecting Objects

Data::Printer Demo

Inspecting Objects

Note the fancy colors, the contextual information on where the value is being printed, the object internals, and the method listing for the blessed references. All of this can be tweaked when importing Data::Printer, or in a configuration file. If you look at the demo source, you can see that dumping something is done with a simple p function; Data::Printer makes sure it's quick and easy to see what you need. You may also tell Data::Printer to print certain objects specially; for example, if you don't care about the internals of a DateTime object. Check the documentation for all of the cool tricks you can do!

Inspecting Objects (We Need To Go Deeper)

Sometimes we need to go a level deeper into an object; by that, I mean looking at the actual in-memory structures that the Perl interpreter manages itself. Maybe you're writing an XS module and you want to make sure your reference counts are right; maybe you're fine tuning some code in a hot path and trying to make sure Perl isn't doing crazy stuff with your data. No matter what, the following module will come in handy.

Inspecting Objects (We Need To Go Deeper)

Devel::Peek

Inspecting Objects (We Need To Go Deeper)

Devel::Peek Demo

Inspecting Objects (We Need To Go Deeper)

The demo illustrates the fields in a scalar value (or SV), as well as an optimization that Perl performs called the offset hack. You can read more about this in the Perl documentation on its internals.

Devel::Peek isn't just a great debugging tool; playing around with it can give you some insight into how the Perl interpreter works as well.

Fun with Stack Traces

Fun with Stack Traces

Let's say you're stuck in a room with a moose.

Fun with Stack Traces

Let's say you're stuck in a room with a moose.

Room with a Moose

Fun with Stack Traces

...and down the line, part of your application throws an exception, leaving you with this wonderfully helpful error:

Bad value at accessor Room::width (defined at test.pl line 5) line 13.

Fun with Stack Traces

How can we make this better?

Fun with Stack Traces

How can we make this better?

Carp::Always

Fun with Stack Traces

New output after enabling Carp::Always:

Bad value at test.pl line 21.
  Room::_trigger_width('Room=HASH(0xde9878)', -10, 10) called at accessor Room::width (defined at test.pl line 5) line 13
  Room::width('Room=HASH(0xde9878)', -10) called at test.pl line 28
  main::bar('Room=HASH(0xde9878)') called at test.pl line 33
  main::foo('Room=HASH(0xde9878)') called at test.pl line 42
  main::main() called at test.pl line 21.
  Room::_trigger_width('Room=HASH(0xde9878)', -10, 10) called at accessor Room::width (defined at test.pl line 5) line 13
  Room::width('Room=HASH(0xde9878)', -10) called at test.pl line 28
  main::bar('Room=HASH(0xde9878)') called at test.pl line 33
  main::foo('Room=HASH(0xde9878)') called at test.pl line 42
  main::main() called at test.pl line 46

Fun with Stack Traces

Carp::Always is ridiculously easy to use; you need only use it at the beginning of your script. You don't even need to alter your program if you don't want to; you can add -MCarp::Always to your command line as well.

Getting Faster Feedback

Sometimes, when I want to test some Perl code, I create a file named test.pl, add some code, run it, and continue this cycle until I have what I need. However, sometimes you want to reduce the time in between this writing code/seeing output cycle. Maybe you're calling a web service and want to play with the results without exceeding the API limit. Maybe you've got some results from a database that took a while to get and you'd like to do something with them before finalizing your code. Other languages have the concept of a REPL (a read-eval-print loop); wouldn't it be nice if Perl had one?

Getting Faster Feedback

Getting Faster Feedback

Sometimes, when I want to test some Perl code, I create a file named test.pl, add some code, run it, and continue this cycle until I have what I need. However, sometimes you want to reduce the time in between this writing code/seeing output cycle. Maybe you're calling a web service and want to play with the results without exceeding the API limit. Maybe you've got some results from a database that took a while to get and you'd like to do something with them before finalizing your code. Other languages have the concept of a REPL (a read-eval-print loop); wouldn't it be nice if Perl had one?

It does!

Getting Faster Feedback

Devel::REPL (and its sibling Reply)

Getting Faster Feedback

Devel::REPL has been around a little longer than Reply, so there are a few more plugins for it. I recommend trying both and seeing which one you like better.

Getting Faster Feedback

Devel::REPL Demo

Getting Faster Feedback

You may have noticed values being printed using Data::Printer; that's because I use a plugin to enable that. Devel::REPL has a lot of useful plugins; I suggest browsing CPAN and seeing what's out there! Or better yet, write your own ;)

Honorable Mentions

Sadly, I couldn't fit every tool I find useful into this talk. So here's a list of others I considered. You may see some old friends here, but hopefully you'll make some new ones too!

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Honorable Mentions

Credits

Any Questions?