In this post, I'll be covering documenting your API -- techniques you can use to indicate what HTTP operations are allowed, as well as convey the full documentation on what endpoints are available, what they accept, and what you can expect them to return.

While I will continue covering general aspects of RESTful APIs in this post, I will also finally introduce several ZF2-specific techniques.

Why Document?

If you're asking this question, you've either never consumed software, or your software is perfect and self-documenting. I frankly don't believe either one.

In the case of APIs, those consuming the API need to know how to use it.

• What endpoints are available? Which operations are available for each endpoint?
• What does each endpoint expect as a payload during the request?
• What can you expect as a payload in return?
• How will errors be communicated?

While the promise of hypermedia APIs is that each response tells you the next steps available, you still, somewhere along the way, need more information - what payloads look like, which HTTP verbs should be used, and more. If you're not documenting your API, you're "doing it wrong."

Where Should Documentation Live?

This is the much bigger question.

Of the questions I raised above, detailing what should be documented, there are two specific types. When discussing what operations are available, we have a technical solution in the form of the OPTIONS method and its counterpart, the Allow header. Everything else falls under end-user documentation.

OPTIONS

The HTTP specification details the OPTIONS method as idempotent, non-cacheable, and for use in detailing what operations are available for the given resource specified by the request URI. It makes specific mention of the Allow header, but does not limit what is returned for requests made via this method.

The Allow header details the allowed HTTP methods for the given resource.

Used in combination, you make an OPTIONS request to a URI, and it should return a response containing an Allow header; from that header value, you then know what other HTTP methods can be made to that URI.

What this tells us is that our RESTful endpoint should do the following:

• When an OPTIONS request is made, return a response with an Allow header that has a list of the available HTTP methods allowed.
• For any HTTP method we do not allow, we should return a "405 Not Allowed" response.

These are fairly easy to accomplish in ZF2. (See? I promised I'd get to some ZF2 code in this post!)

When creating RESTful endpoints in ZF2, I recommend using Zend\Mvc\Controller\AbstractRestfulController. This controller contains an options() method which you can use to respond to an OPTIONS request. As with any ZF2 controller, returning a response object will prevent rendering and bubble out immediately so that the response is returned.

namespace My\Controller;
use Zend\Mvc\Controller\AbstractRestfulController;

class FooController extends AbstractRestfulController
{
    public function options()
    {
        $response = $this->getResponse();
        $headers  = $response->getHeaders();

        // If you want to vary based on whether this is a collection or an
        // individual item in that collection, check if an identifier from
        // the route is present
        if ($this->params()->fromRoute('id', false)) {
            // Allow viewing, partial updating, replacement, and deletion
            // on individual items
            $headers->addHeaderLine('Allow', implode(',', array(
                'GET',
                'PATCH',
                'PUT',
                'DELETE',
            )));
            return $response;
        }

        // Allow only retrieval and creation on collections
        $headers->addHeaderLine('Allow', implode(',', array(
            'GET',
            'POST',
        )));
        return $response;
    }
}

The next trick is returning the 405 response if an invalid option is used. For this, you can create a listener in your controller, and wire it to listen at higher-than-default priority. As an example:

namespace My\Controller;
use Zend\EventManager\EventManagerInterface;
use Zend\Mvc\Controller\AbstractRestfulController;

class FooController extends AbstractRestfulController
{
    protected $allowedCollectionMethods = array(
        'GET',
        'POST',
    );

    protected $allowedResourceMethods = array(
        'GET',
        'PATCH',
        'PUT',
        'DELETE',
    );

    public function setEventManager(EventManagerInterface $events)
    {
        parent::setEventManager($events);
        $events->attach('dispatch', array($this, 'checkOptions'), 10);
    }

    public function checkOptions($e)
    {
        $matches  = $e->getRouteMatch();
        $response = $e->getResponse();
        $request  = $e->getRequest();
        $method   = $request->getMethod();

        // test if we matched an individual resource, and then test
        // if we allow the particular request method
        if ($matches->getParam('id', false)) {
            if (!in_array($method, $this->allowedResourceMethods)) {
                $response->setStatusCode(405);
                return $response;
            }
            return;
        }

        // We matched a collection; test if we allow the particular request 
        // method
        if (!in_array($method, $this->allowedCollectionMethods)) {
            $response->setStatusCode(405);
            return $response;
        }
    }
}

Note that I moved the allowed methods into properties; if I did the above, I'd refactor the options() method to use those properties as well to ensure they are kept in sync.

Also note that in the case of an invalid method, I return a response object. This ensures that nothing else needs to execute in the controller; I discover the problem and return early.

End-User Documentation

Now that we have the technical solution out of the way, we're still left with the bulk of the work left to accomplish: providing end-user documentation detailing the various payloads, errors, etc.

I've seen two compelling approaches to this problem. The first builds on the OPTIONS method, and the other uses a hypermedia link in every response to point to documentation.

The OPTIONS solution is this: use the body of an OPTIONS response to provide documentation. (Keith Casey gave an excellent short presentation about this at REST Fest 2012).

The OPTIONS method allows for you to return a body in the response, and also allows for content negotiation. The theory, then, is that you return media-type-specific documentation that details the methods allowed, and what they specifically accept in the body. While there is no standard for this at this time, the first article I linked suggested including a description, the parameters expected, and one or more example request bodies for each HTTP method allowed; you'd likely also want to detail the responses that can be expected.

{
    "POST": {
        "description": "Create a new status",
        "parameters": {
            "type": {
                "type": "string",
                "description": "Status type -- text, image, or url; defaults to text",
                "required": false
            },
            "text": {
                "type": "string",
                "description": "Status text; required for text types, optional for others",
                "required": false
            },
            "image_url": {
                "type": "string",
                "description": "URL of image for image types; required for image types",
                "required": false
            },
            "link_url": {
                "type": "string",
                "description": "URL of image for link types; required for link types",
                "required": false
            }
        },
        "responses": [
            {
                "describedBy": "http://example.com/problems/invalid-status",
                "title": "Submitted status was invalid",
                "detail": "Missing text field required for text type"
            },
            {
                "id": "abcdef123456",
                "type": "text",
                "text": "This is a status update",
                "timestamp": "2013-02-22T10:06:05+0:00"
            }
        ],
        "examples": [
            {
                "text": "This is a status update"
            },
            {
                "type": "image",
                "text": "This is the image caption",
                "image_url": "http://example.com/favicon.ico"
            },
            {
                "type": "link",
                "text": "This is a description of the link",
                "link_url": "http://example.com/"
            },
        ]
    }
}

If you were to use this methodology, you would alter the options() method such that it does not return a response object, but instead return a view model with the documentation.

namespace My\Controller;
use Zend\Mvc\Controller\AbstractRestfulController;

class FooController extends AbstractRestfulController
{
    protected $viewModelMap = array(/* ... */);

    public function options()
    {
        $response = $this->getResponse();
        $headers  = $response->getHeaders();

        // Get a view model based on Accept types
        $model    = $this->acceptableViewModelSelector($this->viewModelMap);

        // If you want to vary based on whether this is a collection or an
        // individual item in that collection, check if an identifier from
        // the route is present
        if ($this->params()->fromRoute('id', false)) {
            // Still set the Allow header
            $headers->addHeaderLine('Allow', implode(
                ',', 
                $this->allowedResourceMethods
            ));

            // Set documentation specification as variables
            $model->setVariables($this->getResourceDocumentationSpec());
            return $model;
        }

        // Allow only retrieval and creation on collections
        $headers->addHeaderLine('Allow', implode(
            ',',
            $this->allowedCollectionMethods
        ));
        $model->setVariables($this->getCollectionDocumentationSpec());
        return $model;
    }
}

I purposely didn't provide the implementations of the getResourceDocumentationSpec() and getCollectionDocumentationSpec() methods, as that will likely be highly specific to your application. Another possibility is to use your view engine for this, and specify a template file that has the fully-populated information. This would require a custom renderer when using JSON or XML, but is a pretty easy solution.

However, there's one cautionary tale to tell, something I already mentioned: OPTIONS, per the specification, is non-cacheable. What this means is that everytime somebody makes an OPTIONS request, any cache control headers you provide will be ignored, which means hitting the server for each and every request to the documentation. Considering documentation is static, this is problematic; it has even prompted blog posts urging you not to use OPTIONS for documentation.

Which brings us to the second solution for end-user documentation: a static page referenced via a hypermedia link.

This solution is insanely easy: you simply provide a Link header in your response, and provide a describedby reference pointing to the documentation page:

Link: <http://example.com/api/documentation.md>; rel="describedby"

With ZF2, this is trivially easy to accomplish: create a route and endpoint for your documentation, and then a listener on your controller that adds the Link header to your response.

The latter, adding the link header, might look like this:

namespace My\Controller;
use Zend\EventManager\EventManagerInterface;
use Zend\Mvc\Controller\AbstractRestfulController;

class FooController extends AbstractRestfulController
{
    public function setEventManager(EventManagerInterface $events)
    {
        parent::setEventManager($events);
        $events->attach('dispatch', array($this, 'injectLinkHeader'), 20);
    }

    public function injectLinkHeader($e)
    {
        $response = $e->getResponse();
        $headers  = $response->getHeaders();
        $headers->addHeaderLine('Link', sprintf(
            '<%s>; rel="describedby"', 
            $this->url('documentation-route-name')
        ));
    }
}

If you want to ensure you get a fully qualified URL that includes the schema, hostname, and port, there are a number of ways to do that as well; the above gives you the basic idea.

Now, for the route and endpoint, there are tools that will help you simplify that task as well, in the form of a couple of ZF2 modules: PhlySimplePage and Soflomo\Prototype. (Disclosure: I'm the author of PhlySimplePage.)

Both essentially allow you to specify a route and the corresponding template name to use, which means all you need to do is provide a little configuration, and a view template. Soflomo\Prototype has slightly simpler configuration, so I'll demonstrate it here:

return array(
    'soflomo_prototype' => array(
        'documentation-route-name' => array(
            'route'    => '/api/documentation',
            'template' => 'api/documentation',
        ),
    ),
    'view_manager' => array(
        'template_map' => array(
            'api/documentation' => __DIR__ . '/../view/api/documentation.phtml',
        ),
    ),
);

I personally have been using the Link header solution, as it's so simple to implement. It does not write the documentation for you, but thinking about it early and implementing it helps ensure you at least start writing the documentation, and, if you open source your project, you may find you have users who will write the documentation for you if they know where it lives.

Conclusions

Document your API, or either nobody will use it, or all you're hear are complaints from your users about having to guess constantly about how to use it. Include the following information:

• What endpoint(s) is (are) available.
• Which operations are available for each endpoint.
  • What payloads are expected by the endpoint.
  • What payloads can a user expect in return.
  • What media types may be used for requests.
  • What media types may be expected in responses.

Additionally, make sure that you do the OPTIONS/Allow dance; don't just accept any request method, and report the standard 405 response for methods that you will not allow. Make sure you differentiate these for collections versus individual resources, as you likely may allow replacing or updating an individual resource, but likely will not want to do the same for a whole collection!

Next time

So far, I've covered the basics of RESTful JSON APIS, specifically recommending Hypermedia Application Language (HAL) for providing hypermedia linking and relations. I've covered error reporting, and provided two potential formats (API-Problem and vnd.error) for use with your APIs. Now, in this article, I've shown a bit about documenting your API both for machine consumption as well as end-users. What's left?

In upcoming parts, I'll talk about ZF2's AbstractRestfulController in more detail, as well as how to perform some basic content negotiation. I've also had requests about how one might deal with API versioning, and will attempt to demonstrate some techniques for doing that as well. Finally, expect to see a post showing how I've tied all of this together in a general-purpose ZF2 module so that you can ignore all of these posts and simply start writing APIs.

Updates

Note: I'll update this post with links to the other posts in the series as I publish them.

• Part 1
• Part 2

In this post, I cover an aspect of RESTful APIs that's often overlooked: reporting problems.

Background

APIs are useful when they're working. But when they fail, they're only useful if they provide us with meaningful information; if all I get is a status code, and no indication of what caused the issue, or where I might look for more information, I get frustrated.

In consuming APIs, I've come to the following conclusions:

• Error conditions need to provide detailed information as to what went wrong, and what steps I may be able to take next. An error code with no context gives me nothing to go on.
• Errors need to be reported consistently. Don't report the error one way one time, and another way the next.
• DO use HTTP status codes to indicate an error happened. Nothing is more irksome than getting back a 200 status with an error payload.
• Errors should be reported in a format I have indicated I will Accept (as in the HTTP header). Perhaps the only think more irksome than a 200 status code for an error is getting back an HTML page when I expect JSON.

Why Status Codes Aren't Enough

Since REST leverages and builds on HTTP, an expedient solution for reporting problems is to simply use HTTP status codes. These are well understood by web developers, right?

4xx error codes are errors made by the requestor, and are actually fairly reasonable to use for reporting things such as lack of authorization tokens, incomplete requests, unsupportable operations, or non-supported media types.

But what happens when the error is on the server - because something has gone wrong such as inability to reach your persistence layer or credential storage? The 5xx series of status codes is sparse and wholly unsuited to reporting errors of these types -- though you'll likely still want to use a 500 status to report the failure. But what do you present to the consumer so that they know whether or not to try again, or what to report to you so that you can fix the issue?

A status code simply isn't enough information most of the time. Yes, you want to define standard status codes so that your clients can perform reasonable branching, but you also need a way to communicate details to the end-user, so that they can log the information for themselves, display information to their own end-users, and/or report it back to you so you can do something to resolve the situation.

Custom Media Types

The first step is to use a custom media type. Media types are typically both a name as well as a structure -- and the latter is what we're after when it comes to error reporting.

If we return a response using this media type, the client then knows how to parse it, and can then process it, log it, whatever.

Sure, you can make up your own format -- as long as you are consistent in using it, and you document it. But personally, I don't like inventing new formats when standard formats exist already. Custom formats mean that custom clients are required for working with the services; using a standard format can save effort and time.

In the world of JSON, I've come across two error media types that appear to be gaining traction: application/api-problem+json and application/vnd.error+json

API-Problem

This particular media type is via the IETF. Like HAL, it provides formats in both JSON and XML, making it a nice cross-platform choice.

As noted already, the media type is application/api-problem+json. The representation is a single resource, with the following properties:

• describedBy: a URL to a document describing the error condition (required)
• title: a brief title for the error condition (required)
• httpStatus: the HTTP status code for the current request (optional)
• detail: error details specific to this request (optional)
• supportId: a URL to the specific problem occurrence (e.g., to a log message) (optional)

As an example:

HTTP/1.1 500 Internal Error
Content-Type: application/api-problem+json

{
    "describedBy": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html",
    "detail": "Status failed validation",
    "httpStatus": 500,
    "title": "Internal Server Error"
}

The specification allows a large amount of flexibility -- you can have your own custom error types, so long as you have a description of them to link to. You can provide as little or as much detail as you want, and even decide what information to expose based on environment.

I personally like to point to the HTTP status code definitions, and then provide request-specific detail; I find this gives quick and simple results that I can later shape as I add more detail to my API. However, the specification definitely encourages you to have unique error types with discrete URIs that describe them -- never a bad thing when creating APIs.

vnd.error

This is a proposed media type within the HAL community. Like HAL, it provides formats in both JSON and XML, making it a nice cross-platform choice.

It differentiates from API-Problem in a few ways. First, it allows, and even encourages, reporting collections of errors. If you consider PHP exceptions and the fact that they support "previous" exceptions, this is a powerful concept; you can report the entire chain of errors that led to the response. Second, it encourages pushing detail out of the web service; errors include a "logRef" property that points to where the error detail lives. This is probably better illustrated than explained.

The response payload is an array of objects. Each object has the following members:

• logRef: a unique identifier for the specific error which can then be used to identify the error within server-side logs (required)
• message: the error message itself (required)
• _links: HAL-compatible links. Typically, "help", "describes", and/or "describedBy" relations will be defined here.

As an example, let's consider the API-Problem example I had earlier, and provide a vnd.error equivalent:

HTTP/1.1 500 Internal Error
Content-Type: application/vnd.error+json

[
    {
        "logRef": "someSha1HashMostLikely",
        "message": "Status failed validation",
        "_links": {
            "describedBy": {"href": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html"}
        }
    }
]

vnd.error basically begs you to create custom error types, with documentation end-points that detail the source of the error and what you can do about it (this is true of API-Problem as well).

The requirement to include a log reference ("logRef") and have it be unique can be a stumbling block to implementation, however, as it requires effort for uniquely identifying requests, and logging. However, both the identification and logging can be automated.

Summary

Error reporting in APIs is as important as the normal resource payloads themselves. Without good error reporting, when an API raises errors, clients have difficulty understanding what they can do next, and cannot provide you, the API provider, with information that will allow you to debug on the server side.

As noted at the beginning of the article, if you follow the rules below, you'll make consumers of your API happier and more productive.

• DO use appropriate HTTP status codes to indicate an error happened.
• Report errors in a format I have indicated I will Accept (as in the HTTP header).
• Report errors consistently. Don't report the error one way one time, and another way the next. Standardize on a specific error-reporting media type . While you can create your own error structure, I recommend using documented, accepted standards. This will make clients more re-usable, and make many of your decisions for you.
• Provide detailed information as to what went wrong, and what steps I may be able to take next. Provide documentation for each type of error, and link to that documentation from your error payloads.

Which brings me to...

Next time

I realize I still haven't covered anything specific to ZF2, but I'll start next time, when I cover the next topic: documenting your API. An undocumented API is a useless API, so it's good to start baking documentation in immediately. I'll survey some of the possibilities and how they can be implemented in ZF2 in the next installment, and then we can get our hands dirty with actual API development.

Updates

Note: I'll update this post with links to the other posts in the series as I publish them.

• Part 1
• Part 3

Rob Allen and I proposed a workshop for PHP Benelux 2013 covering RESTful APIs with ZF2. When it was accepted, it gave me the perfect opportunity to dive in and start putting the various pieces together.

Background

I've attended any number of conference sessions on API design, read countless articles, and engaged in quite a number of conversations. Three facts keep cropping up:

1. JSON is fast becoming the preferred exchange format due to the ease with which it de/serializes in almost every language.
2. The "holy grail" is Richardson Maturity Model Level 3.
3. It's really hard to achieve RMM level 3 with JSON.

Richardson Maturity Model

As a quick review, the Richardson Maturity Model has the following 4 levels:

• Level 0: "The swamp of POX." Basically, a service that uses TCP for transport, primarily as a form of remote procedure call (RPC). Typically, these are not really leveraging HTTP in any meaningful fashion; most systems will use HTTP POST for all interactions. Also, you will often have a single endpoint for all interactions, regardless of whether or not they are strictly related. XML-RPC, SOAP, and JSON-RPC fall under this category.
• Level 1: "Resources." In these services, you start breaking the service into multiple services, one per "resource," or, in object oriented terms, per object. This means a distinct URL per object, which means each has its own distinct identity on the web; this often extends not only to the collection of objects, but to individual objects under the collection as well (e.g., "/books" as well as "/books/life-of-pi"). The service may still be RPC in nature, however, and, at this level, often is still using a single HTTP method for all interactions with the resource.
• Level 2: "HTTP Verbs." At this level, we start using HTTP verbs with our services in the way the HTTP specification intends. GET is for safe operations, and should be cacheable; POST is used for creation and/or updating; DELETE can be used to delete a resource; etc. Rather than doing RPC style methods, we leverage HTTP, occasionally passing additional parameters via the query string or request body. Considerations such as HTTP caching and idempotence are taken into account.
• Level 3: "Hypermedia Controls." Building on the previous level, our resource representations now also include links, which indicate what we can do next. At this level, our API becomes practically self-describing; given a single end-point, we should be able to start crawling it, using the links in a representation to lead us to the next actions.

When I first started playing with web services around a decade ago, everything was stuck at Level 0 or Level 1 -- usually with Level 1 users downgrading to Level 0 because Level 0 offerred consistency and predictability if you chose to use a service type that had a defined envelope format (such as XML-RPC or SOAP). (I even wrote the XML-RPC server implementation for Zend Framework because I got sick of writing one-off parsers/serializers for custom XML web service implementations. When you're implementing many services, predictability is a huge win.)

A few years ago, I started seeing a trend towards Level 2. Web developers like the simplicity of using HTTP verbs, as they map very well to CRUD operations -- the bread and butter of web development. Couple this concept with JSON, and it becomes trivially simple to both create a web service, as well as consume it.

I'd argue that the majority of web developers are quite happy to be at Level 2 -- and have no problem staying there. They're productive, and the concepts are easy -- both to understand and to implement.

Level 3, though, is where it becomes really interesting. The idea that I can examine the represention alone in order to understand what I can do next is very intriguing and empowering.

JSON and Hypermedia

With XML, hypermedia basically comes for free. Add some <link> elements to your representation, and you're done -- and don't forget the link relations!

JSON, however, is another story.

Where do the links go? There is no single, defined way to represent a hyperlink in JSON.

Fortunately, there are some emerging standards.

First is use of the "Link" HTTP header. While the page I linked shows only a single link in the header, you can have multiple links separated by commas. GitHub uses this when providing pagination links in their API. Critics will point out that the HTTP headers are not technically part of the representation, however; strict interpetations of REST and RMM indicate that the hypermedia links should be part of the resource representation. Regardless, having the links in the HTTP headers is useful for pre-traversal of a service, as you can perform HEAD requests only to discover possible actions and workflows.

Collection+JSON is interesting, as it describes the entire JSON envelope. My one criticism is that it details too much; whenever I see a format that dictates how to describe types, I think of XML-RPC or SOAP, and get a little twitchy. It's definitely worth a look, though.

What's captured my attention of late, however, is Hypertext Application Language, or HAL for short. HAL has very few rules, but succinctly describes both how to provide hypermedia in JSON as well as how to represent embedded resources - the two things that most need standardized structure in JSON. It does this while still providing a generic media type, and also describing a mirror image XML format!

HAL Media Types

HAL defines two generic media types: application/hal+xml and application/hal+json. You will use these as the response Content-Type, as they describe the response representation; the client can simply request application/json, and the response format remains compatible.

HAL and Links

HAL provides a very simple structure for JSON hypermedia links. First, all resource representations must contain hypermedia links, and all links are provided in a "_links" object:

{
    "_links": {
    }
}

Second, links are properties of this object. The property name is the link relation, and the value is an object containing minimally an "href" property.

{
    "_links": {
        "self": {"href": "http://example.com/api/status/1234"}
    }
}

If a given relation can have multiple links, you provide instead an array of objects:

{
    "_links": {
        "self": {"href": "http://example.com/api/status/1234"},
        "conversation": [
            {"href": "http://example.com/api/status/1237"},
            {"href": "http://example.com/api/status/1241"}
        ]
    }
}

Individual links can contain other attributes as desired -- I've seen people include the relation again so that it's self-contained in the link object, and it's not uncommon to include a title or name.

HAL and Resources

HAL imposes no structure over resources other than requiring the hypermedia links; even then, you typically do not include the hypermedia links when making a request of the web service; the hypermedia links are included only in the representations returned by the service.

So, as an example, you would POST the following:

POST /api/status
Host: example.com
Accept: application/json
Content-Type: application/json

{
    "status": "This is my awesome status update!",
    "user": "mwop"
}

And from that request, you'd receive the following:

201 Created
Location: http://example.com/api/status/1347
Content-Type: application/hal+json

{
    "_links": {
        "self": {"href": "http://example.com/api/status/1347"}
    },
    "id": "1347",
    "timestamp": "2013-02-11 23:33:47",
    "status": "This is my awesome status update!",
    "user": "mwop"
}

HAL and Embedded Resources

The other important thing that HAL defines is how to embed resources. Why is this important? If the resource references other resources, you will want to be able to link to them so you can perform operations on them, too.

Embedded resources are represented inside an "_embedded" object of the representation, and, as resources, contain their own "_links" object as well. Each resource you embed is assigned to a property of that object, and if multiple objects of the same type are returned, an array of resources is assigned. In fact, this latter is how you represent collections in HAL.

Let's consider a simple example first. In previous code samples, I have a "user" that's a string; let's make that an embedded resource instead.

{
    "_links": {
        "self": {"href": "http://example.com/api/status/1347"}
    },
    "id": "1347",
    "timestamp": "2013-02-11 23:33:47",
    "status": "This is my awesome status update!",
    "_embedded": {
        "user": {
            "_links": {
                "self": {"href": "http://example.com/api/user/mwop"}
            }
            "id": "mwop",
            "name": "Matthew Weier O'Phinney",
            "url": "http://mwop.net"
        }
    }
}

I've moved the "user" out of the representation, and into the "_embedded" object -- because this is where you define embedded resources. Note that the "user" is a standard HAL resource itself -- containing hypermedia links.

Now let's look at a collection:

{
    "_links": {
        "self": {"href": "http://example.com/api/status"},
        "next": {"href": "http://example.com/api/status?page=2"},
        "last": {"href": "http://example.com/api/status?page=100"}
    },
    "count": 2973,
    "per_page": 30,
    "page": 1,
    "_embedded": {
        "status": [
            {
                "_links": {
                    "self": {"href": "http://example.com/api/status/1347"}
                },
                "id": "1347",
                "timestamp": "2013-02-11 23:33:47",
                "status": "This is my awesome status update!",
                "_embedded": {
                    "user": {
                        "_links": {
                            "self": {"href": "http://example.com/api/user/mwop"}
                        }
                        "id": "mwop",
                        "name": "Matthew Weier O'Phinney",
                        "url": "http://mwop.net"
                    }
                }
            }
            /* ... */
        ]
    }
}

Note that the "status" property is an array; semantically, all resources under this key are of the same type. Also note that the parent resource has some additional link relations -- these are related to pagination, and allow a client to determine what the next and last pages are (and, if we were midway into the collection, previous and first pages). Since the collection is also a resource, it has some interesting metadata -- how many resources are in the collection, how many we represent per page, and what the current page is.

Also note that you can nest resources -- simply include an "_embedded" object inside an embedded resource, with additional resources, as I've done with the "user" resource inside the status resource shown here. It's turtles all the way down.

Next Time

The title of this post indicates I'll be talking about building RESTful APIs with ZF2 -- but so far, I've not said anything about ZF2.

I'll get there. But there's another detour to take: reporting errors.

Updates

Note: I'll update this post with links to the other posts in the series as I publish them.

• Part 2
• Part 3

Why not support multiple formats?

There's no reason you can't re-use your RESTful web service to support multiple formats. Zend Framework and PHP have plenty of tools to assist you in responding to different format requests, so don't limit yourself. With a small amount of work, you can make your controllers format agnostic, and ensure that you respond appropriately to different requests.

Content-Type Detection

The first problem to solve is going to be how to retrieve passed parameters. When using XML or JSON as your serialization format, you aren't getting your standard POST variables -- you're getting a raw post instead, and you'll need to deserialize the payload. In fact, if you're getting a PUT request, you also have some work to do, as PHP doesn't do anything with PUT requests.

I do this via an action helper. The basic algorithm is:

• Do we have a raw body in the request? If not, nothing more need be done.
• Determine the Content-Type passed in the request headers, and decode appropriately:
  • If it was JSON, pass the raw request body to json_decode or Zend_Json::decode.
  • If it was XML, I pass the raw request body to the Zend_Config_XML constructor, and then serialize to an arrya using the toArray() method. Yes, it's a hack, but it's effective.
  • Otherwise, I assume I've got a regular PUT-style request, and I pass the data to parse_str().

I keep the values within the action helper, and then retrieve them on demand within my action controller. The helper looks like the following:

class Scrummer_Controller_Helper_Params 
    extends Zend_Controller_Action_Helper_Abstract
{
    /**
     * @var array Parameters detected in raw content body
     */
    protected $_bodyParams = array();

    /**
     * Do detection of content type, and retrieve parameters from raw body if 
     * present
     * 
     * @return void
     */
    public function init()
    {
        $request     = $this->getRequest();
        $contentType = $request->getHeader('Content-Type');
        $rawBody     = $request->getRawBody();
        if (!$rawBody) {
            return;
        }
        switch (true) {
            case (strstr($contentType, 'application/json')):
                $this->setBodyParams(Zend_Json::decode($rawBody));
                break;
            case (strstr($contentType, 'application/xml')):
                $config = new Zend_Config_Xml($rawBody);
                $this->setBodyParams($config->toArray());
                break;
            default:
                if ($request->isPut()) {
                    parse_str($rawBody, $params);
                    $this->setBodyParams($params);
                }
                break;
        }
    }

    /**
     * Set body params
     * 
     * @param  array $params 
     * @return Scrummer_Controller_Action
     */
    public function setBodyParams(array $params)
    {
        $this->_bodyParams = $params;
        return $this;
    }

    /**
     * Retrieve body parameters
     * 
     * @return array
     */
    public function getBodyParams()
    {
        return $this->_bodyParams;
    }

    /**
     * Get body parameter
     * 
     * @param  string $name 
     * @return mixed
     */
    public function getBodyParam($name)
    {
        if ($this->hasBodyParam($name)) {
            return $this->_bodyParams[$name];
        }
        return null;
    }

    /**
     * Is the given body parameter set?
     * 
     * @param  string $name 
     * @return bool
     */
    public function hasBodyParam($name)
    {
        if (isset($this->_bodyParams[$name])) {
            return true;
        }
        return false;
    }

    /**
     * Do we have any body parameters?
     * 
     * @return bool
     */
    public function hasBodyParams()
    {
        if (!empty($this->_bodyParams)) {
            return true;
        }
        return false;
    }

    /**
     * Get submit parameters
     * 
     * @return array
     */
    public function getSubmitParams()
    {
        if ($this->hasBodyParams()) {
            return $this->getBodyParams();
        }
        return $this->getRequest()->getPost();
    }

    public function direct()
    {
        return $this->getSubmitParams();
    }
}

This helper is intended to be run on each request, so I register it in my bootstrap:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    // ...
    protected function _initActionHelpers()
    {
        // ...
        $params = new Scrummer_Controller_Helper_Params();
        Zend_Controller_Action_HelperBroker::addHelper($params);
        // ...
    }
    // ...
}

Within your action controller, all you need to do is call the helper:

$data = $this->params();

In a RESTful controller, you'll only need to use this with your postAction and putAction. The beauty is that your controller can remain ignorant of the Content-Type -- you write the same logic to retrieve your parameters regardless.

Responding to the client: Context Switching

So, the first half of the problem is taken care of: how to handle the request. The second half is responding appropriately.

Zend Framework has some built in tooling to help with this. The ContextSwitch and AjaxContext action helpers look for a particular parameter -- "format" by default -- and, if detected, will render an alternate view script named after the context. As an example, if an "XML" context is detected, it will render "<controller>/<action>.xml.phtml" -- note the ".xml" segment of the script name.

Both helpers work in the same basic way (the latter, AjaxContext, will only activate if the request is determined to originate from an XMLHttpRequest): you define which actions in the controller are context sensitive, and then if the context is detected, a new view script will be used.

So, the first trick is ensuring that the context is passed. As mentioned before, the helpers look for a "format" parameter in the request object. You can pass this using a query parameter -- "?format=xml" -- but I find that ugly. There's an HTTP header defined for this purpose already: "Accept".

Detecting the header and injecting the context into the request is absurdly simple, and can be done in a dispatchLoopStartup plugin:

class Scrummer_Controller_Plugin_AcceptHandler
    extends Zend_Controller_Plugin_Abstract
{
    public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
    {
        if (!$request instanceof Zend_Controller_Request_Http) {
            return;
        }

        $header = $request->getHeader('Accept');
        switch (true) {
            case (strstr($header, 'application/json')):
                $request->setParam('format', 'json');
                break;
            case (strstr($header, 'application/xml') 
                  && (!strstr($header, 'html'))):
                $request->setParam('format', 'xml');
                break;
            default:
                break;
        }
    }
}

The above can be registered in your application configuration:

resources.frontController.plugins[] = \"Scrummer_Controller_Plugin_AcceptHandler\"

I like my RESTful controllers to automatically expose their methods as context-aware. To make this happen, I defined a marker interface, "Scrummer_Rest_Controller", and created an action helper that checks if the current controller implements it; if it does, I then automatically add contexts for the RESTful actions.

class Scrummer_Controller_Helper_RestContexts
    extends Zend_Controller_Action_Helper_Abstract
{
    protected $_contexts = array(
        'xml', 
        'json',
    );

    public function preDispatch()
    {
        $controller = $this->getActionController();
        if (!$controller instanceof Scrummer_Rest_Controller) {
            return;
        }

        $this->_initContexts();

        // Set a Vary response header based on the Accept header
        $this->getResponse()->setHeader('Vary', 'Accept');
    }

    protected function _initContexts()
    {
        $cs = $this->getActionController()->contextSwitch;
        $cs->setAutoJsonSerialization(false);
        foreach ($this->_contexts as $context) {
            foreach (array('index', 'post', 'get', 'put', 'delete') as $action) {
                $cs->addActionContext($action, $context);
            }
        }
        $cs->initContext();
    }
}

Register this via the bootstrap as well:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    // ...
    protected function _initActionHelpers()
    {
        // ...
        $params = new Scrummer_Controller_Helper_Params();
        Zend_Controller_Action_HelperBroker::addHelper($params);

        $contexts = new Scrummer_Controller_Helper_RestContexts();
        Zend_Controller_Action_HelperBroker::addHelper($contexts);
        // ...
    }
    // ...
}

There are two things to note about this helper. First, you'll see that I specify a "Vary" header. This is to ensure that if the client chooses to cache responses, it will cache separate responses based on the value sent in the "Accept" header.

Second, note that I turn off automatic JSON serialization in the ContextSwitch helper. I do this so that I can keep my controller context agnostic; this will require additional view scripts, but the ability to keep my controller logic simple will be worth it. More on that in a moment.

We now have the infrastructure in place to respond to different contexts based on the "Accept" header, and can retrieve parameters appropriately based on the "Content-Type" provided us. Now comes the actual response.

Responding to the client: Views

Recall that ContextSwitch will attach an additional prefix to the specified view script -- "<controller>/<action>.phtml" will become "<controller>/<action>.xml.phtml" or "<controller>/<action>.json.phtml". Basically, for each context we will respond to, we have an additional view script per action.

views/
|-- scripts/
|   `-- foo/
|      |-- delete.phtml
|      |-- delete.json.phtml
|      |-- delete.xml.phtml
|      |-- get.phtml
|      |-- get.json.phtml
|      |-- get.xml.phtml
|      |-- index.phtml
|      |-- index.json.phtml
|      |-- index.xml.phtml
|      |-- post.phtml
|      |-- post.json.phtml
|      |-- post.xml.phtml
|      |-- put.phtml
|      |-- put.json.phtml
|      `-- put.xml.phtml

This may seem like overkill, but consider the following representative method from my controller:

    public function postAction()
    {
        $data    = $this->params();
        $service = $this->getService();
        $result  = $service->add($data);  
        if (!$result) {
            $this->view->form = $service->getBacklogForm();
            return;
        }

        $this->view->success = true;
        $this->view->backlog = $result;
    }

You don't see anything in there about headers, redirects, or XHR requests. Just slinging data to services and views. Real simple.

The view scripts then take care of the appropriate display logic. Let's look at two view scripts for the above action, one for plain old HTML, the other for a JSON response:

<?php // backlog/post.phtml ?>
<?php 
if ($this->success):
    $this->response->setRedirect($this->url(array(
        'controller' => 'backlog',
        'id'         => $this->backlog->id,
    ), 'rest', true));
else: ?>
<h2>Create new backlog</h2>
<?php
    $this->form->setAction($this->url())
               ->setMethod('post');
    echo $this->form;
endif ?>

<?php // backlog/post.json.phtml ?>
<?php
if ($this->success) {
    $url = $this->url(array(
        'controller' => 'backlog',
        'id'         => $this->backlog->id,
    ), 'rest', true);
    $this->response->setHeader('Location', $url)
                   ->setHttpResponseCode(201);
    echo $this->json($this->backlog->toArray());
    return;
}

$form = $this->form;
$form->setAction($this->url())
     ->setMethod('post');
echo $this->jsonFormErrors($form);

A few things to note: I inject my response object into the view. I feel HTTP headers are part of the view, and thus I deal with them there. That also serves the purpose of keeping my controllers thin and agnostic. Additionally, you'll note that I use different response codes for HTML versus JSON -- this allows my JSON-REST support to be RESTful, by returning a 201 status code indicating the resource was created; I also return a JSON representation of the object. Finally, you'll note that I have a special view helper for creating JSON representations of validation errors.

Closing points

This post is far from exhaustive, and I expect it will likely raise at least as many questions as it tries to answer.

My main point in this article is to get you, the reader and developer, thinking creatively about how to expose RESTful web services. Hopefully, you're taking the following away:

1. Architect in such a way as to minimize the code in your controllers; keep that code as agnostic as possible in regards to where input comes from and what type of response is required.
2. Use front controller plugins and action helpers to create scaffolding for your services; these are incredibly flexible and re-usable, and help make point 1 that much easier.
3. Offload as much as possible to your views. This will allow you to isolate logic specific to given formats.

What are you waiting for? Don't you have an API to expose?

At its heart, REST simply dictates that a given resource have a unique address, and that you interact with that resource using HTTP verbs. The standard verbs utilized are:

• GET: retrieve a list of resources, or, if an identifier is present, view a single resource
• POST: create a new resource with the data provided in the POST
• PUT: update an existing resource as specified by an identifier, using the PUT data
• DELETE: delete an existing resource as specified by an identifier

The standard URL structure used is as follows:

• "/resource" - GET (list) and POST operations
• "/resource/{identifier}" - GET (view), PUT, and DELETE operations

What the REST paradigm provides you is a simple, standard way to structure your CRUD (Create-Read-Update-Delete) applications. Due to the large number of REST clients available, it also means that if you follow the rules, you get a ton of interoperability with those clients.

As of Zend Framework 1.9.0, it's trivially easy to create RESTful routes for your MVC application, as well as to handle the various REST actions via action controllers.

Zend_Rest_Route allows you to define RESTful controllers at several levels:

• You can make it the default route, meaning that unless you have additional routes, all controllers will be considered REST controllers.
• You can specify modules that contain RESTful controllers.
• You can specify specific controllers per module that are RESTful

As examples:

$front = Zend_Controller_Front::getInstance();
$router = $front->getRouter();

// Specifying all controllers as RESTful:
$restRoute = new Zend_Rest_Route($front);
$router->addRoute('default', $restRoute);

// Specifying the \"api\" module only as RESTful:
$restRoute = new Zend_Rest_Route($front, array(), array(
    'api',
));
$router->addRoute('rest', $restRoute);

// Specifying the \"api\" module as RESTful, and the \"task\" controller of the
// \"backlog\" module as RESTful:
$restRoute = new Zend_Rest_Route($front, array(), array(
    'api',
    'backlog' => array('task'),
));
$router->addRoute('rest', $restRoute);

To define a RESTful action controller, you can either extend Zend_Rest_Controller, or simply define the following methods in a standard controller extending Zend_Controller_Action (you'll need to define them regardless):

// Or extend Zend_Rest_Controller
class RestController extends Zend_Controller_Action
{
    // Handle GET and return a list of resources
    public function indexAction() {}

    // Handle GET and return a specific resource item
    public function getAction() {}

    // Handle POST requests to create a new resource item
    public function postAction() {}

    // Handle PUT requests to update a specific resource item
    public function putAction() {}

    // Handle DELETE requests to delete a specific item
    public function deleteAction() {}
}

For those methods that operate on individual resources (getAction(), putAction(), and deleteAction()), you can test for the identifier using the following:

if (!$id = $this->_getParam('id', false)) {
    // report error, redirect, etc.
}

Responding is an art

Many developers are either unaware of or ignore the part of the specification that dictates what the response should look like.

For instance, in classic REST, after performing a POST to create a new item, you should do the following:

• Set the HTTP response code to 201, indicating "Created"
• Set the Location header to point to the canonical URI for the newly created item: "/team/31"
• Provide a representation of the newly created item

Note that there's no redirect, which flies in the face of standard web development (where GET-POST-Redirect is the typical format). This is a common "gotcha" moment.

Similarly, with PUT requests, you simply indicate an HTTP 200 status when successful, and show a representation of the updated item. DELETE requests should return an HTTP 204 status (indicating success - no content), with no body content.

Note: when building RESTful HTML applications, you may want to still do GET-POST-Redirect to prevent caching issues. The above applies to RESTful web services, which typically use XML or JSON for transactions, and have smart clients for interacting with the service.

I'll be writing another article soon showing some tips and tricks for interacting with HTTP headers, both from the request and for the response, as it's a subject lengthy enough for a post of its own. In the meantime, start playing with Zend_Rest_Route and standardizing on it for your CRUD operations!