In this post, I'll outline the approaches I've tried, the benchmarking stategy I applied, and the results of benchmarking each approach.

Autoloading Strategies

I'm grouping strategies into two categories, PEAR/PSR-0 strategies, and classmap strategies.

I also started testing a third category. This included solutions that required PECL extensions, specifically the SplClassLoader and Automap extensions. However, I ultimately abandoned these solutions. In the case of SplClassLoader, I actually started testing it, but immediately ran into segfaults. This unfortunate event made me remember that I was looking for userland autoloaders that we could use within Zend Framework; both SplClassLoader and Automap can be dropped in by users at any point, but due to the requirement of compiling and installing for your platform, could never be an explicit requirement for using Zend Framework.

PEAR/PSR-0

Those of you familiar with Zend Framework are aware that we follow PEAR Coding Standards, and, specific to this exercise, their 1-class-1-file naming convention. The PEAR naming conventions dictate a 1:1 relation between the filesystem and the class. As an example, the class Foo_Bar_Baz would be found in the file "Foo/Bar/Baz.php" on your include_path.

This is a trivially easy convention to remember, and has been widely adopted in the PHP world. The one and only vote the PHP Framework Interoperability Group has held so far, http://groups.google.com/group/php-standards/web/psr-0-final-proposal, has simply ratified this standard going forward (with some additional verbiage regarding namespaces). Zend Framework's autoloader has been PSR-0 compliant since 1.10.0 (and was PEAR-compliant prior to that).

So, the very first approach has been simply to use the status quo.

That said, I also looked at other PSR-0-compliant approaches for some inspiration. The SplClassLoader proposed by Jon Wage, of the Doctrine project, as a GitHub gist takes a couple of departures from the ZF implementation:

• It allows specifying a specific directory under which to look for a specific namespace
• Instead of acting as a singleton, you create a single instance per namespace you want to load, and then call its register() method to register with the spl_autoload registry.

Additionally, I looked at the Symfony2 UniversalClassLoader. While it has its basis in the SplClassLoader implementation, it offers a feature we'd already added to the ZF2 autoloader: the ability to register both namespaces and vendor prefixes to autoload. I combined these ideas into a custom PSR-0 implementation.

(Note: the SplClassLoader extension is a literal port of the class from the GitHub gist to a C extension.)

Classmaps

The next category of autoloader solutions I looked at are best characterized by the term "classmaps." In this strategy, you create a map of classname/filename pairs, and feed it to your autoloader.

To my thinking, the key benefits of this strategy include:

• Ability to deviate from the PSR-0 standard if desired (as an example, application resources).
• Ability to drop in a classmap per component. For a library such as ZF, this opens up possibilities for distributing individual components with fewer dependencies, as you do not need to also ship artifacts such as your autoloader.
• Fail early. If the class does not exist in the map, the autoloader can exit early, allowing you to drop to another autoloader in the chain -- or simply have PHP raise its E_FATAL report of class-not-found.
• Drop in at will. You can use dynamic autoloaders during development, but then run a script during deployment or build time to generate a classmap autoloader. This allows you to have the benefits of a RAD cycle, while also reaping the benefits of a performance-optimized autoloader.

I will elaborate on the last point later, when I examine the benchmarks.

Despite the prevalence of PSR-0 style autoloaders, there are a number of classmap autoloaders in the wild. ez/zeta components has used one for years, in part due to using a non-PSR-0-compliant naming convention, but also in part due to performance considerations. Arne Blankerts also introduced me to one such solution in the form of his Autoload library on GitHub.

When building classmaps, you can either build them manually as you develop, or utilize a script. I like the script-based approach, as it ensures I don't forget to add items to the map, and because it's something I can run over existing libraries and then drop in.

When building such a script, the algorithm is quite simple:

• Recursively scan a filesystem tree
• For each PHP file found, scan for interfaces, classes, and abstract classes
• For each match, store the fully qualified class name and the file path

I blogged about my solution to scanning the filesystem tree earlier; an example of the script that consumes the ClassFileLocater class referenced in that blog and generates the actual classmap can be found in my GitHub account.

I took three different approaches to generating classmaps:

• Store file paths as relative to the include_path
• Store file paths using __DIR__ to prefix the path
• Store file paths using __DIR__ to prefix the path, and pass the map directly to a closure registered with spl_autoload.

In the first two cases, the map is stored in an array that is returned by the script. I then pass the map file's location to an autoloader that performs an include on that file, stores the map (optionally merging with one it already contains), and then uses that map for class lookups. You can find this autoloader on GitHub.

The third case was a trick borrowed from Arne Blankert's Autoload library. I deviated from his design in a couple of ways. First, Arne was defining the map as a static member of his closure. Theoretically, this should ensure the map is defined only once per request. However, in my tests, I discovered that the map was actually being constructed in memory each and every time the closure was invoked, and led to a serious degredation in performance. As a result, my version creates a variable in the local scope which is then passed in to the closure via a use statement:

namespace test;
$_map = array( /* ... */ );
spl_autoload_register(function($class) use ($_map) {
    if (array_key_exists($class, $_map)) {
        require_once __DIR__ . DIRECTORY_SEPARATOR . $_map[$class];
    }
});

Note the namespace declaration; this allows you to create multiple autoload class map files without trampling on previously loaded maps.

Benchmarking Strategy

Benchmarking autoloading is somewhat difficult; once you've autoloaded a class, you can't autoload it again. Additionally, libraries tend to have a finite number of classes in them, giving a limited set of data to benchmark against. Finally, no good autoloading benchmark should be done without also testing against an opcode cache -- and if you have too many class files, you can easily defeat the cache.

My solution was twofold:

• Create a script to generate finite, large numbers of class files
• Determine how many class files provide a reasonable set such that an opcode cache is not defeated, but also such that measurable differences may be observed.

The script for generating class files was easy to create. All I needed was a recursive function that would iterate through the letters of the alphabet and generate classes until a specific depth was reached; you can examine the script on GitHub.

I started off by looking at 26^3 class files, and this was excellent for getting some initial statistics. However, once I started benchmarking against an opcode cache, I discovered that I was defeating both the realpath cache as well as memory limits for my opcode cache. I then reduced my sampling size to 16^3 files. This sampling size proved to be sufficiently large to both produce reliable results on multiple runs while allowing caching to work normally.

When benchmarking, I ran the following strategies:

• Baseline: no autoloader (all class_exists() calls fail)
• ZF1 autoloader (PSR-0 compliant, uses include_path)
• PSR-0 autoloader (uses explicit namespace/path pairs, no include_path
• ClassMapAutoloader (using include_path)
• ClassMapAutoloader (using __DIR__-prefixed paths)
• Class map using __DIR__-prefixed paths, with closure registered directly to spl_autoload

My test algorithm was as follows:

• Load a list of all classes in the tree
• Perform any setup necessary to prepare the given autoloader
• Iterate over all classes in the list, and utilize class_exists to autoload

Timing was performed only over the loop. Benchmarks were run 10 times for each strategy, in order to determine an average, as well as to correct for outliers. Additionally, the benchmarks were performed both with and without bytecode caching, to see what differences might occur in both environments. A sample of such a script is as follows:

include 'benchenv.php';
require_once '/path/to/zf/library/Zend/Loader/ClassMapAutoloader.php';

$loader = new Zend\Loader\ClassMapAutoloader();
$loader->registerAutoloadMap(__DIR__ . '/test/map_abs_autoload.php');
$loader->register();

echo \"Starting benchmark of ClassFile map autoloader using absolute paths...\n\";
$start = microtime(true);
foreach ($classes as $class) {
    if (!class_exists($class)) {
        echo \"Aborting test; could not find class $class\n\";
        exit(2);
    }
}
$end = microtime(true);
echo \"total time: \" . ($end - $start) . \"s\n\";

I had one such script for each test case. To automate running all such scripts, and doing 10 iterations of each, I wrote the following scripts:

#!/usr/bin/zsh
# benchmark_noaccel.sh
# No opcode caching
for TYPE in baseline.php classmap_abs.php classmap_inc.php spl_autoload.php psr0_autoload.php zf1_autoload.php;do
    echo \"Starting $TYPE\"
    for i in {1..10};do
        curl http://autoloadbench/$TYPE
    done
done

#!/usr/bin/zsh
# benchmark_accel.sh
# With opcode caching
for TYPE in baseline.php classmap_abs.php classmap_inc.php spl_autoload.php psr0_autoload.php zf1_autoload.php;do
    echo \"Starting $TYPE\"
    # Clear Optimizer+ cache
    curl http://autoloadbench/optimizer_clear.php
    for i in {1..10};do
        curl http://autoloadbench/$TYPE
    done
done

I reran the tests a few times to ensure I wasn't seeing anomalies. While the actual numbers I received differed between iterations, statistically, they only varied within a few percentage points between runs.

The Results

No Opcode Cache

With Opcode Cache

Analysis

The three class map variants are the clear winners here, showing around a 25% improvement on the ZF1 autoloader when no acceleration is present, and 60-85% improvements when an opcode cache is in place. The three classmap approaches are roughly equivalent when no opcode caching is in place, but the variants that do not use the include_path are statistically faster when the opcode cache is present.

Perhaps the most interesting finding for myself was seeing how much the include_path affects performance, particularly when using an opcode cache. In each case, I had the directory with my test classes listed as the first item in the include_path -- which is the optimal location (previous benchmarking and profiling I've done shows that performance degrades quickly the deeper the matching entry is within the include_path). Even in the non-accelerated tests, the PSR-0 implementation, which does not use the include_path, is >10% faster, a difference that jumps to almost 40% with acceleration. The same differences are true also with the class map implementations.

While these changes are very much micro-optimizations (remember, the numbers above indicate 16^3 classes loaded -- a single class loads in matters of 1/10,000th of a second), if you are loading hundreds or thousands of unique classes over your application lifecycle, the evidence clearly shows some significant performance benefits from the usage of class maps and fully-qualified paths.

Conclusions

Each approach has its merits. During development, you don't want to necessarily run a script to generate the class map or manually update the class map every time you add a new class. That said, if you expect a lot of traffic to your site, it's trivially easy to run a script during deployment to build the class map for you, and thus let you eke out a little extra performance from your application.

One clear realization from this experiment is that the include_path is a poor way to resolve files in current incarnations of PHP. While the degradation is not huge when your initial path matches, it still introduces a performance hit. Additionally, it makes setup of the applications harder, as you then must document the proper usage of the include_path; using __DIR__ with either a PSR-0-style or class map autoloader is more easily portable and requires less education of end-users.

One interesting use case for a classmap based autoloader is to assist in optimizing existing ZF1 applications. The script could be run over your "application" directory to build a classmap of your application resource classes. This would allow you to bypass the various resource autoloaders and custom logic of the dispatcher for finding controller classes. It could also potentially serve as one part of a migration suite between ZF1 and ZF2.

I will be recommending that Zend Framework 2.0 use both PSR-0 and class map strategies, without reliance on the include_path, and provide tools and scripts to aid deployment.

Here's the short version:

• One trip to Belgium and The Netherlands.
• Two trips to Israel.
• Two trips to Atlanta, GA (not counting the return trip from Europe, when I was stranded for a day due to storms in the Northeast).
• Three different user groups attended, with three presentations.
• One major Zend Framework release
• One PEAR release.
• One podcast.
• One webinar.
• One book published.
• One conference attended.

What follows is my month-by-month breakdown:

January

I finished up the last of my three chapters for The PHP Anthology, 2nd Edition, and promptly used my advance to buy the family a Wii.

I was also introduced to Ivo Jansch through work, and had him wow me with one of the ibuildings products.

February

Mid-month, my boss at Zend, Boaz, got the go-ahead to use the ibuildings WDE platform to build our new website CMS; part of the plan would include training at the ibuildings home office in Vlissingen, NL... which meant I had to get my passport pronto.

Towards the end of the month, I was invited to BostonPHP to present building a simple Zend Framework CRUD application, along with Chuck Hagenbuch of the HORDE project. While there, Nate Abele heckled me, and then joined Chuck and myself for an impromptu framework panel; a good time was had by all.

March

I flew to Brussels, Belgium, where I met my supervisor, Boaz, so we could go to Vlissingen. We spent the day in Brussels, walking around and visiting such sites as the Cathedral of St. Michael, La Grand Place, and the Mannekin Pis.

Our visit to ibuildings was very productive, and I was very impressed by the team there; everybody was very knowledgeable and skilled. I presented a Zend Framework overview, as well as an abbreviated version of the Best Practices talk I'd given with Mike Naberezny at the 2006 ZendCon; the latter ignited a ton of questions and enthusiasm.

On returning home, I had a ton of work to do on the zend.com CMS, and this continued in spurts through November. The job was made much easier by the ibuildings WDE product.

I closed out a ton of MVC issues in the Zend Framework, and we released the first beta version late in the month.

April

At the beginning of the month, our landlord threw us for a loop and announced he was going to sell our apartment... meaning that we either had to step up our plans to purchase a home, or start looking for a new rental. Ultimately, we ended up looking for a rental, due to time constraints. The next two months would be highlighted with the look for a new place as well as countless showings of our apartment to potential buyers.

Mid-month, we packed up the family and flew down to Atlanta, GA, to visit my wife's family. While there, we were able to go to the Atlanta Zoo and see Mei Lan, their baby panda -- way cute!

May

Mid-month, we found a new place in Richmond, VT -- a small village about 10 minutes from Burlington, near where we originally lived when we first moved to Vermont.

During the first RC for Zend Framework, released at the end of the month, I introduced the ViewRenderer, a feature for auto-rendering views based on the current controller and action name -- a feature common to many frameworks. However, it ostensibly broke a ton of existing applications by being enabled by default -- not one of my more popular decisions. Since the 1.0.0 release, I've heard little grumbling about it, and it's now often cited as an ease-of-use feature -- go figure.

June

The first week of June, I flew to Tel Aviv, Israel, to start training people on the new CMS, as well as to work with our entire ebiz team to finalize the work plan for completing the CMS. It was, needless to say, my first time to Israel or the Middle East, and I was constantly confronted with culture shock. Europe was an easy transition to make, but Israel was completely foreign to me -- everything from the way people drove, to the architecture, to the food was different. Unfortunately, I arrived a day late due to a flight cancellation, and missed the tour of Jerusalem my supervisor had planned for all of us. However, he took me to the city of Yafa, an Arabic city where the Israeli's originally tried to settle before building Tel Aviv to the north. The architecture was amazing, as were the winding, narrow streets of the old city.

I was also told during this trip that Andi had requested transferring me full-time to the Zend Framework team. I would spend the next week or two weighing my options, and ultimately decided to do so.

A week after I returned, we moved into our new rental in Richmond. The kids love the new place, which has a bedroom for each of them, a yard, and porches on each entrance.

Somehow, I also found time to record my first (and so far only) PHP Abstract podcast.

July

We released Zend Framework 1.0.0 at the beginning of the month, marking our first stable release. While many still view it as incomplete, the overwhelming feedback has been positive, and we've had over 2 million downloads to date.

I accepted the transfer to the Zend Framework team, but the condition was made that I would stay part-time on the ebiz team until the new site was launched. This meant that the next 5 months were spent splitting my time between the two projects, often working late and on weekends to get work done.

Towards the end of the month, we took a long weekend camping in Vermont's Northeast Kingdom. The weather was unseasonably wet, but we persevered and had a great time. 5 days of offline time was definitely needed!

I also finally released the first stable version of File_Fortune on PEAR, over a year since I'd first proposed it. The package interfaces with mod_fortune files, allowing both the ability to read and write such files, with full binary compatability.

August

Not much to report in August, except work, work, and more work.

September

My ebiz supervisor, Boaz, flew me to Tel Aviv for a second time, this time to perform a "brain dump" for the rest of the team before I transitioned fully out of the team, and also to help setup our new data center and release procedures. This time, Boaz took me to Jerusalem himself during my last full day in the country. If you've never been to the city, you should definitely put it on your list of things to do before you die. With my degree in religion, the place was full of meaning for me, but it would be putting it lightly to say that religion is palpable in the air there. We visited the Wailing Wall, the Via Dolorosa, the Church of the Holy Sepulchre, and listened to the muezzins sing the call to prayer for the muslims. The tour was simply amazing.

A few days after I returned, I flew down to New York City for a special meeting of NYPHP, where Mark de Visser presented on various Zend products and initiatives, and I gave a Zend Framework overview.

A week after the NYPHP presentation, I did a zend.com webinar on the Zend Framework MVC layer.

October

October was the month of ZendCon. I presented a full-day tutorial on best practices and unit testing with Sebastian Bergmann and Mike Naberezny; despite the length and subject matter, we were SRO for most of the day.

I also did a main-stage presentation on Zend Framework's MVC components, directly following Terry Chay -- an intimidating situation at best. From the feedback I've seen, the presentation was well-received, and I had somewhere between 120 and 150 attendees -- phenomenal! (Even more amazing was how many people were familiar with MVC in general!)

One great thing about the conference was the fact that I got to network with a number of framework developers, both Zend Framework and otherwise, including Nate Abele of CakePHP as well as Paul M. Jones of the Solar framework. Many good conversations were had.

Late in the month, The PHP Anthology, 2nd Edition, my first published book as an author, was finally released!

November

I spent much of the month working on Zend_Layout, a much requested component that simplifies and automates Two Step Views in Zend Framework. I also started work implementing Zend_View Enhanced, a set of view helpers for making complex views with Zend_View possible.

I also started playing with Twitter a bit, and came up with a Zend_Service_Twitter proposal for interacting with the Twitter API via PHP.

And finally, the Sunday before Thanksgiving, we finally launched the new Zend.com site, which was well-received in the blogosphere.

December

A goal I've had for some time has been to form a PHP user group in the Burlington area. A friend of mine pointed out to me sometime this fall that there's actually already a Google Group formed; he and the original founder started planning a meeting for early December. I spoke at this inaugural meeting, presenting Zend Framework's MVC layer yet again; a good time was had by all, and a lot of enthusiasm for future meetings was generated.

I finished up Zend_Layout and Zend_View Enhanced with the help of Ralph Schindler, and got a new proposal up for Zend_Form, just in time for my holidays to begin -- 11 days with family and with little to no internet connectivity during a trip to Atlanta, GA for one of only a handful of Christmases I've spent without snow.

Summary

This year was incredibly busy -- three cross-seas trips, one cross-continent trip, a move, and several trips along the Eastern Seaboard; three user group presentations, and eight presentations over the course of the year; one conference; one move; one PEAR release; one podcast; one webinar; one book; and countless hours of programming.

My goals for the coming year? I'm too tired to even think about it ;-).

(Note: sure, File_Fortune may not be that big of a deal, but over 1000 downloads in the past two years indicates somebody is using it. Plus, it powers the random quotes on the family website. :-) )

As some examples:

require_once 'File/Fortune.php';

// Initialize and point it to a directory of fortunes
$fortunes = new File_Fortune('/path/to/fortunedir');

// Retrieve a random fortune 
// (works with either a directory or a single fortune file)
echo $fortunes->getRandom();

// Set to a specific fortune file:
$fortunes->setFile('myfortunes');

// Loop through and print all fortunes
foreach ($fortunes as $fortune) {
    echo str_repeat('-', 72), \"\n\", $fortune, \"\n\n\";
}

// Hmmm.. let's change one:
$fortunes[7] = \"I never really liked that fortune anyways.\";

// No need to explicitly save, as it's done during __destruct(), 
// but if you really want to:
$fortunes->save();

// Let's add a new fortune:
$fortunes->add('This is a shiny new fortune!');

// and now we'll verify it exists:
$index = count($fortunes) - 1;
echo $fortunes[$index];

All-in-all, it's a much better interface. Lesson learned: when porting code from other languages, it pays to take some time and determine if there might be a better API in your own.

In upcoming releases, I hope to modify the backend to use PHP's Streams API instead of direct file access, and also to allow providing a list of fortune files explicitly. After that, I should be ready for the initial stable release.

Update (2007-07-10): fixed parse error in examples