How Does Git Know What Functions Look Like?
While recently skimming the man page for
git-log, I came across a neat looking feature for the
... -L <start>,<end>:<file>, -L :<funcname>:<file> If ":<funcname>" is given in place of <start> and <end>, it is a regular expression that denotes the range from the first funcname line that matches <funcname>, up to the next funcname line.
After marveling at this functionality, I began to wonder: how does it know what
a function looks like? After all, Vim has some awareness of language features
like functions, but its assumptions are usually rooted in C and its kin. Does
-L :<funcname>:<file> feature only recognize functions in C-like
languages? Let's find out!
Exploring the Git Source
ctags and following the trail of breadcrumbs, I ended up finding the
logic that Git uses to determine function name boundaries. First, Git tries to
find what's called a userdiff driver; we'll get to the details of that in a
second. Then, regardless of whether or not it found one, it iterates over each
line of the file provided with
-L. If a line matches the function name
funcname is treated as a regular expression), Git's behavior
differs based on whether it was able to load a userdiff driver. If it was, it
uses a set of regular expressions provided by the driver; if any of those
regular expressions match the line that matched
funcname, that line is
considered the start of the function. For example, the Perl driver looks for
since Perl subroutines are introduced via the
If Git was not able to load a userdiff driver, it falls back to a very simple regular
expression: if the line matching
funcname begins with an alphanumeric character, an
underscore, or a dollar sign, that line is treated as the start of the function, which
works pretty well for a lot of programming languages…except for ones in which you
would indent your function definitions!
To find the end of the function, Git takes the set of regular expressions from the userdiff driver (or the fallback alphanumeric + underscore + dollar sign regular expression), and continues to iterate over of the lines of the file. Once it finds a line that matches its set of regular expressions, the preceding line is considered to be the end of the function. Now that Git has a range of lines, it uses its typical line range history algorithm to finish the job.
The fallback pattern is a pretty simple heuristic, and it works alright for languages like C. But what about languages that have different-looking functions? Let's get back to those user diff drivers!
User Diff Drivers
By default, files in a Git repository have no userdiff driver, but we can change that using
For example, to associate Perl files with the perl userdiff driver based on the
.pl file extension, you
can put the following into your
.gitattributes file at the top of your repository:
.pl files use the perl driver, where is that driver defined? Userdiff drivers can be defined
in two places: built into Git itself, or defined in
To see the drivers built into Git, we need only look at
userdiff.c; there are drivers for languages such
as Perl, Python, and CSS in here.
If Git doesn't have a driver for a language, or if you would like to override its default driver, you can
specify them in
gitconfig like so:
[diff.perl6] xfuncname = "^\\s*((my|our)\\s+)?(class|role|grammar|package|module|sub|method|multi)"
The flexibility of this system means that you can extend the definition of “funcname” beyond just functions - in my Perl 6 example above, it will pick up on classes, roles, modules, etc, in addition to functions. There are a few limits to this flexibility, however:
- You can't specify what the end of a function looks like. This normally isn't a problem, but if you allow indentation ahead of your functions, you can run into problems if you have nested definitions.
- If you work in a language that allows you to overload a function, such as Java, C++, Haskell, Elixir, or Perl 6, Git will only pick up on the first definition. You can provide a starting line to
-Lto cover other entries, but this is less than awesome.
Since you may specify
-L multiple times and it's basically sugar for a line range, you could get around both of these with a more syntax-aware tool that emits
-L options, though.
I learned a lot about Git and this variant of the
-L option; I hope you did
as well! One takeaway I have from this is what an untapped resource
gitattributes is; there seem to be a lot of goodies hiding out in its manpage.
Time for me to check them out!