entr(1)

Run arbitrary commands when files change

bitbucket/overview | download 3.2 | download 3.1

Example

$ ls *.c *.h | entr make

Screencast

Interactive SQL: OGG (47.0M, theora-vorbis) WEBM (14.2M, vp8-vorbis)

Theory and Operation

The Event Notify Test Runner is a general purpose Unix utility intended to make rapid feedback and automated testing natural and completely ordinary.

Some graphical applications such as the PostScript/PDF viewer gv ship with a -watch option with reloads the document whenever the source file is modified. This is useful, but it is even better for applications to provide a programmatic means of refreshing the display. The browser add-on LiveReload has attempted to solve this problem for web developers by injecting JavaScript that listens on a web socket. The should not be this complex, indeed this is all that is required: 

$ ls *.css *.html | entr reload-browser Firefox

reload-browser is a simple script which uses xdotool or AppleScript to send a refresh keystroke to the active tab in your browser. Some lesser known browsers, such as Midori can be controlled from the command line using arguments -e Reload.

It is not uncommon for modern web frameworks to continuously hunt for file system changes and auto-reloads when run in single threaded or standalone mode. This functionality is superfluous if the application can respond to signals. The following will instruct mupdf to reload a PDF whenever it is updated: 

$ ls *.pdf | entr pkill -HUP mupdf

entr is a zero-configuration tool with no external build or runtime dependencies. The interface to entr is not only minimal, it aims to be simple enough to create a new category of ad hoc automation. These micro-tests reduce keystrokes, but more importantly they emphasize the utility of automated checks.

Tightening the edit-test feedback loop requires a tool that is tuned for one task. inotifywait is lightweight, but it only works on Linux, and does not provide a direct means of saying \“run this command if any of these files change”. In practice scripting with inotify-tools is difficult because there are a number of significant conditions to contend with:

  1. Many applications attempt to make the file save operation atomic by writing a new file and then removing the original. entr deals with this by closing the old file descriptor and reopening it using the same pathname. Since there is a small delay while the new file is renamed, we must wait for the new file to appear before running the supplied command and attempting to watch the new file.
  2. File change events that occur while the utility is running need to be processed so as to ensure that files that have been replaced are monitored, but these events should not trigger an execution when the child process ends. entr allows you to safely edit files while tests are running without a repeated invocation of the utility.
  3. Typically version control software will often update a series of files in rapid succession. Ideally the build is launched when then entire operation is complete. To cope with this behavior entr repeatedly probes for subsequent events, and only executes the utility when the kernel returns with no results after a short timeout.
  4. The events reported when saving files on an NFS mount are different than those of a local file system. On Linux an inotify may report IN_MOVE_SELF|IN_DELETE_SELF instead of IN_MODIFY|IN_CLOSE_WRITE and on BSD kqueue may report NOTE_RENAME|NOTE_DELETE instead of NOTE_WRITE|NOTE_EXTEND.
  5. A race condition exists when executing a script that is also under watch since a script cannot be executed while another process has it open for write. Rather than allowing the enigmatic error “Text file busy”, entr retries the execution.
  6. On Linux editors that use the Gnome's GIO may write to a file and then subsequently delete it. To deal with this entr consolidates events over 50ms before responding.
  7. In some cases an editor will rename a file without removing it. This occurs the first time a file is saved in Vim if the backup option is set. To deal with this events must be explicitly unregistered to prevent the kernel from tracking changes to backup files.

Reducing Friction

entr adheres to the principle of separation of concerns, yet the reload (-r) option was added to solve a common use case that would otherwise require some careful scripting: 

$ ls *.rb | entr -r ruby main.rb

This will,

  1. immediately start the server
  2. block until any of the listed files change
  3. terminate the background process
  4. wait for the server to exit before restarting

The 3.1 release further tuned this behavior by setting a process group to ensure that all child processes receive a signal. This enables you to use a startup script without having to write custom signal in handlers.

Other special-purpose flags were added because they reduce highly repetitive actions or reduce friction. One of the most repetitive actions was to clear the screen before running tests; hence the -c flag: 

$ find src/ | entr -c ./test.sh

The special /_ argument (somewhat analogous to $_ in Perl) provides a quick way to refer to the first file that changed. When a single file is listed this is a handy way to avoid typing a pathname twice: 

$ echo /tmp/my.sql | entr psql -f /_

Watching for New Files

In the 2.9 release, a directory watch option (-d) was added to react to events when a new file is added to a directory. It was determined early on that entr would not implement it's own file search syntax, relying on standard Unix tools instead. The implication of this is that if a new file appears it must exit and allow an external shell loop to rescan the file system. One way to implement this feature would be to simply require the users to list directories: 

$ while sleep 1; do
> ls -d src/*.py src | entr ./setup.py
> done

This is “correct” from a POSIX point of view, but in addition to forcing the user to spend time thinking about how how options in ls interact with shell globing, using find to accomplish a similar task suddenly becomes difficult.

In this instance entr leans heavily on the principle of least surprise. The promise to the user is simply to “run the utility if a new file is added to the project”. The previous example works, and so does this: 

$ while sleep 1; do
> ls src/*.py | entr ./setup.py
> done

In directory watch mode the parent directory of each file is implicitly added to the watch list. This behavior is very unusual for a command line tool, but in practice it behaves as most users would expect.

Finally, the sharp edges of shell scripting were removed by providing a well-behaved example in the man page. Using sleep 1 as the test condition for the while loop prevents repeat failures from pegging the CPU and it allows the user to break out with Ctrl-C.

Feedback Panes

I have written as if incorporating automated responses can be accomplished without special demands on the Unix development environment, but in practice the ability to split a window into multiple panes is the key to making this workflow efficient. A window manager such as i3 or the terminal multiplexer tmux enables you to quickly spit the screen so that you can see the results as you work.

tmux steps automation up to the next level by enabling you to control applications in other panes via keystrokes. This combination can be wired up in any number of ways to create some very interesting auto-responders. Consider the following

watch http headers

With this mechanism in place we can fetch and compare headers using any tool capable of printing output or writing to a file—no plugins or specialized functionality required.

Last updated on April 09, 2015
Send questions or comments to ericshane@eradman.com