Deploying a Catalyst application with FastCGI and PAR

Securely deploying an application into a production environment can be tricky. This article will give you some tips on how to make this process as painless as possible. Instead of copying files around, messing with permissions, and restarting your web server, we'll take a different approach. Instead of a directory, our application will be stored in the form of a PAR -- a single executable file that contains your application. We will employ daemontools to launch a fastcgi server from the PAR and handle logging with multilog. We'll then configure your web server to connect to the fastcgi server.

The first step is to download daemontools and multilog from http://cr.yp.to/daemontools.html. There are a number of HOWTOs written to help you get started with daemontools. This tutorial assumes that you have svscan (daemontools) set up and working on your system.

If you have daemontools properly set up, you should have a /service directory. Subdirectories of this represent daemons that you want to be managed for you by daemontools. Each subdirectory contains a run script that tells daemontools how to start your server (and restart it if it fails). Optionally, a log subdirectory can contain commands to start the logging program as though it were itself a daemon.

To monitor the status of a daemon, use the svstat command:

    # svstat /service/*

You should see something like this - a line for each daemon that daemontools controls:

    /service/authdaemon: up (pid 21856) 2474300 seconds
    /service/imap-ssl: up (pid 9336) 2474300 seconds
    /service/qmail-send: up (pid 28877) 2474300 seconds
    /service/qmail-smtpd: up (pid 5262) 2474300 seconds
    /service/sshd: up (pid 13641) 2474300 seconds

Note that you need to be root for this to work. (We'll assume that you're root for the rest of this article and will use the # prompt to show this; the $ prompt will indicate commands that can be run as a non-root user.)

To manage individual daemons, use the svc utility:

    # svc -d /service/apache  # bring down apache
    # svc -t /service/myapp   # send myapp a SIGTERM (and restart it)
    # svc -u /service/qmail   # start qmail 

Finally, if something's not working, you can see if readproctitle has logged any error messages:

    # ps wwwaux | grep readproctitle

You should see something like:

    root 21943 0.0 0.0 168 288 ??  I 4Nov06 0:00.04 
    readproctitle service errors: .................

Everything after the "readproctitle service errors: " part are messages sent to STDERR from anything svscan tried to start. In this case, there are only periods, and that means everything's OK. (Or perhaps everything is just silently failing.)

We don't need these commands just yet, but we'll find them useful later.

The first step to deploying your application as a PAR is to build your application as a PAR. This is very simple -- just add a line that says catalyst_par() to your application's Makefile.PL, and type:

    # perl Makefile.PL
    # make catalyst_par

After you do that, you should have a "myapp.par" file in the root directory of your application - If this didn't work, make sure you've installed the PAR package from CPAN). For more details see Catalyst::Manual::Cookbook, under "Quick deployment: Building PAR Packages".

You can make sure the PAR works properly by running:

    # parl myapp.par myapp_server.pl -d

And then connecting to your server at the usual http://localhost:3000 address.

This myapp.par file is now the application that you'll deploy. One file contains your whole application. PAR can even include your application's dependencies, if you want. Check out the PAR documentation. In fact the pp (perl packager) can bundle the whole shebang into a binary executable as well, making your application deployable on systems where no perl, or the wrong version of perl, is installed, without having to tax your system administrator excessively.

Now that we have an application, we need a place for it to live, and a user for it to run as. Creating a user on your system varies according to your operating system. I'll assume that you have an adduser command and a addgroup command that add users and groups (respectively).

For security reasons, it's best to run each application as its own user. That way, if your application is compromised, it won't be able to do anything useful for the hacker. The hacker can of course circumvent any access controls that your application enforces, but he won't be able to get at other data on the system (or other web applications, or the web server itself).

So, let's create a user for MyApp:

    # adduser _myapp # add a _myapp user to the system

This should create a _myapp user and a _myapp group on your system. If you'd like to admister MyApp without becoming root, you can add yourself to this group:

    # adduser me _myapp # add me to the _myapp group

Now that we have some accounts, let's create a place for our application to live:

    # mkdir /var/www-apps/myapp
    # chown _myapp:_myapp /var/www-apps/myapp
    # chmod 770 /var/www-apps/myapp

This will create a directory that anyone in the _myapp group can write to, and is where myapp.par will live.

Now that myapp.par has a home, let's copy it over there:

    $ cp myapp.par /var/www-apps/myapp

Finally, let's create a directory for logfiles to be stored:

    # mkdir /var/www-apps/myapp/log
    # chown _myapp:_myapp /var/www-apps/myapp/log
    # chmod 750 /var/www-apps/myapp/log

Now that our application has a permanent home, we need to tell daemontools to manage it. We do this by creating a /service/myapp directory, and then populating it with some shell scripts.

Since daemontools will immediately start any application that it sees in the /service directory, we'll need to work outside of the service directory, and then symlink the myapp directory into /service when we're ready.

    # mkdir /var/www-apps/myapp/service

The first script we need to create is the service/myapp/run script, which runs our application. This is pretty simple:

    #!/bin/sh
    export MYAPP_DEBUG=1

    cd /var/www-apps/myapp

    exec 2>&1
    exec setuidgid _myapp parl myapp.par myapp_fastcgi.pl -e \ 
         -l 127.0.0.1:3010

The script starts by turning debugging on for MyApp. We then change to the /var/www-apps/myapp directory (so that core files, etc., will be created there). We then redirect STDERR to STDOUT, since STDERR is sent to readproctitle instead of multilog. Finally, we start myapp_fastcgi.pl. The setuidgid command comes with daemontools and sets the uid and gid for our process to the uid and gid of the _myapp user. Then parl is invoked on /var/www-apps/myapp/myapp.par, running the myapp_fastcgi.pl script that it contains. The arguments to myapp_fastcgi.pl are:

-e

Send all error message to STDOUT instead of to the web server over the fastcgi socket. (This allows multilog to handle the logging, instead of passing off the task to the webserver. If you like the way your webserver is setup to log, then omit this item.)

-l 127.0.0.1:3010

Listen for FastCGI connections on the loopback address on port 3010. You can specify a file on the filesystem instead, if you want to use UNIX sockets instead of TCP sockets. The advantage of TCP sockets is that you can run the frontend web server on a different machine, and that you don't have to worry about file permissions (for example, this is helpful when using a chroot httpd). The disadvantage is that any user on the system can connect to the FastCGI server and bypass any access restrictions that your webserver may provide. Feel free to use whatever suits your situation best.

Daemontools comes with other utilities such as setuidgid that you may find helpful. See the daemontools website for more information.

When you're finished editing this script, be sure to make it executable.

The next thing we need to do is create a service/log/run script, to run the logging daemon multilog:

    #!/bin/sh
    exec setuidgid _myapp /usr/local/bin/multilog t \
    /var/www-apps/myapp/log

This will start a multilog process that will log all messages to /var/www-apps/myapp/log. multilog will handle log rotation and timestamping for you. (See the multilog documentation for more details.)

Again, be sure this script is executable, and that the directory containing it is readable and executable.

Now that you have a complete service directory inside /var/www-apps/myapp, you can link that directory into the system's service directory:

    # ln -sf /var/www-apps/myapp/service /service/myapp

svscan will then start your application (and a logging process) within five seconds! If all is well, then:

    # svstat /service/myapp

Will print something like:

    /service/svnserve: up (pid 13579) 6 seconds

If it says 0 or 1 seconds (and the PID keeps changing), then something is wrong. You can check readproctitle for errors executing your scripts, and check the multilog log for any other errors (or messages):

    # tail -f /var/www-apps/myapp/log/current | tai64nlocal

Multilog stores the most recent log file in a file called current (actually, current is just a link to the current log file, which has a name based on the time when it was created). tail -f will follow the output of the file as long as it exists (beware of rotation, though). Finally, the tai64nlocal command will translate multilog's tai64 time format into local (human-readable) time.

Your app isn't very useful until you setup a webserver to pass requests to it over the FastCGI socket. The exact steps vary depending on your web server, but Apache 1.3x will look something like this:

    NameVirtualHost *

    LoadModule fastcgi_module /usr/lib/apache/modules/mod_fastcgi.so
    FastCgiExternalServer /var/www/htdocs/myapp.fcgi -host 127.0.0.1:3010

    <VirtualHost *>
        ServerName www.myapp.com
	ServerAdmin myapp-admin@myapp.com
	DocumentRoot /var/www/htdocs/myapp

	Alias /static /var/www/htdocs/myapp/static
	Alias / /var/www/htdocs/myapp.fcgi/

	<Location />
            Allow from all
	    Deny from none
	</Location>
    </VirtualHost>

First, we declare that we want every IP to be a name-based virtual host (your config might differ here). Then, we load the mod_fastcgi module into the server (usually, this line is added by your package manager when you install mod_fastcgi). Next, we tell FastCGI to bind the server at 127.0.0.1:3010 the Apache Location /fcgi-myapp. Finally, we configure our www.myapp.com virtual host to point to this application.

Note that, at least for my server, the virtual path to the fastcgi application has to be inside the DocumentRoot, and it has to end in .fcgi. If you're using lighttpd or Apache 2.0+ and mod_fcgid, you should be able to choose any name you like.

The only tricky bit here is the Alias /static line. Although your application's static content can be served by your application from within the PAR file, this isn't very efficient. As a compromise, we copy the static content from MyApp into the server's docroot and tell the server to serve it directly. A cleaner solution might be to configure mod_cache to cache responses for /static, and then let MyApp serve the static content. We'll leave mod_cache as an exercise for another day, and just copy the static content over when necessary.

Once you've setup your web server's configuration, restart the server and your app should be working!

With all of this in place, updating your application in the future will be very simple:

    $ make catalyst_par
    # cp myapp.par /var/www-apps/myapp
    # svc -t /service/myapp

That's it. You now have a deployment method that is very simple to maintain, and that is more secure than mod_perl or a fastcgi server running as the web server's user. And, since your web server doesn't need access to perl anymore, you can easily run it in a chroot jail for even more security.

Enjoy!

Jonathan Rockway <jrockway@cpan.org>

Copyright (c) 2006 Jonathan Rockway and the Catalyst Core Team. All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Distributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name 'Catalyst' nor the names of its contributors may be used to endorse or promote products derived from this documentation without specific prior written permission.

THIS DOCUMENTATION IS PROVIDED BY THE CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.