Blog entries tagged zf2 :: mwop.net

Apigility: Using RPC with HAL

2014-03-26T15:30:00-05:00

A few days ago, we released our first beta of Apigility. We've started our documentation effort now, and one question has arisen a few times that I want to address: How can you use Hypermedia Application Language (HAL) in RPC services? HAL? Hypermedia Application Language is an IETF proposal for how to represent resources and their relations within APIs. Technically, it provides two mediatypes, application/hal+json and application/hal+xml; however, Apigility only provides the JSON variant. The important things to know about HAL are: It provides a standard way of describing relational links. All relational links are under a _links property of the resource. That property is an object. Each property of that object is a link relation; the value of each link relation is an object (or array of such objects) describing the link that must minimally contain an href proerty. The link object itself can contain some additional metadata, such as a mediatype, a name (useful for differentiating between multiple link objects assigned to the same relation). While not required, the specification recommends resources contain a "self" relational link, indicating the canonical location for the resource. This is particularly useful when we consider embedding (the next topic). Sound hard? It's not: { "_links": { "self": { "href": "/blog/2014-03-26-apigility-rpc-with-hal" } } } Besides link relations, HAL also provides a standard way of describing embedded resources. An embedded resource is any other resource you can address via your API, and, as such, would be structured as a HAL resource — in other words, it would have a _links property with relational links. Essentially, any property of the resource you're returning that can itself be addressed via the URI must be embedded in the resource. This is done via the property _embedded. Like _links, _embedded is an object. Each key in the object is the local name by which the resource refers to the embedded resource. The value of such keys can either be HAL resources or arrays of HAL resources; in fact, this is how collections are represented in HAL! As examples: { "_links": { "self": { "href": "/blog/2014-03-26-apigility-rpc-with-hal" } }, "_embedded": { "author": { "_links": { "self": { "href": "/blog/author/matthew" } }, "id": "matthew", "name": "Matthew Weier O'Phinney", "url": "http://mwop.net" }, "tags": [ { "_links": { "self": { "href": "/blog/tag/php" } }, "id": "php" }, { "_links": { "self": { "href": "/blog/tag/rest" } }, "id": "rest" } ] } } The example above shows two embedded resources. The first is the author; the second, a collection of tags. Note that every object under _embedded is a HAL object! You can go quite far with this — you can also have embedded resources inside your embedded resources, arbitrarily deep. RPC? RPC stands for Remote Procedure Call, and, when describing a web API, is usually used to describe a web service that publishes multiple method calls at a single URI using only POST; XML-RPC and SOAP are the usual suspects. In Apigility, we use the term RPC in a much looser sense; we use it to describe one-off services: actions like "authenticate," or "notify," or "register" would all make sense here. They are actions that usually only need to respond to a single HTTP method, and which may or may not describe a "thing", which is what we usually consider a "resource" when discussing REST terminology. That said: what if what we want to return from the RPC call are REST resources? Returning HAL from RPC Services In order to return HAL from RPC services, we need to understand (a) how Content Negotiation works, and (b) what needs to be returned in order for the HAL renderer to be able to create a representation. For purposes of this example, I'm positing a RegisterController as an RPC service that, on success, is returning a User object that I want rendered as a HAL resource. The zf-content-negotiation module takes care of content negotiation for Apigility. It introspects the Accept header in order to determine if we can return a representation, and then, if it can, will cast any ZF\ContentNegotiation\ViewModel returned from a controller to the appropriate view model for the representation. From there, a renderer will pick up the view model and do what needs to be done. So, the first thing we have to do is return ZF\ContentNegotiation\ViewModel instances from our controller. use Zend\Mvc\Controller\AbstractActionController; use ZF\ContentNegotiation\ViewModel; class RegisterController extends AbstractActionController { public function registerAction() { /* ... do some work ... get a user ... */ return new ViewModel(array('user' => $user)); } } The zf-hal module in Apigility creates the actual HAL representations. zf-hal looks for a "payload" variable in the view model, and expects that value to be either a ZF\Hal\Entity (single item) or ZF\Hal\Collection. When creating an Entity object, you need the object being represented, as well as the identifier. So, let's update our return value. use Zend\Mvc\Controller\AbstractActionController; use ZF\ContentNegotiation\ViewModel; use ZF\Hal\Entity; class RegisterController extends AbstractActionController { public function registerAction() { /* ... do some work * ... get a $user * ... assume we have also now have an $id */ return new ViewModel(array('payload' => array( 'user' => new Entity($user, $id), ))); } } zf-hal contains what's called a "metadata map". This is a map of classes to information on how zf-hal should render them: what route to use, what additional relational links to inject, how to serialize the object, what field represents the identifier, etc. In most cases, you will have likely already defined a REST service for the resource you want to return from the RPC service, in which case you will be done. However, if you want, you can go in and manually configure the metadata map in your API module's config/module.config.php file: return array( /* ... */ 'zf-hal' => array( 'metadata_map' => array( 'User' => array( 'route_name' => 'api.rest.user', 'entity_identifier_name' => 'username', 'route_identifier_name' => 'user_id', 'hydrator' => 'Zend\Stdlib\Hydrator\ObjectProperty', ), ), ), ); Finally, we need to make sure that the service is configured to actually return HAL. We can do this in the admin if we want. Find the "Content Negotiation" section of the admin, and the "Content Negotiation Selector" item, and set that to "HalJson"; don't forget to save! Alternately, you can do this manually in the API module's config/module.config.php file, under the zf-content-negotiation section: return array( /* ... */ 'zf-content-negotiation' => array( 'controllers' => array( /* ... */ 'RegisterController' => 'HalJson', ), /* ... */ ), ); Once your changes are complete, when you make a successful request to the URI for your "register" RPC service, you'll receive a HAL response pointing to the canonical URI for the user resource created! Apigility: Using RPC with HAL was originally published 26 March 2014 on https://mwop.net by Matthew Weier O'Phinney.

RESTful APIs with ZF2, Part 3

2013-02-25T06:29:00-06:00

In my previous posts, I covered basics of JSON hypermedia APIs using Hypermedia Application Language (HAL), and methods for reporting errors, including API-Problem and vnd.error. In this post, I'll be covering documenting your API — techniques you can use to indicate what HTTP operations are allowed, as well as convey the full documentation on what endpoints are available, what they accept, and what you can expect them to return. While I will continue covering general aspects of RESTful APIs in this post, I will also finally introduce several ZF2-specific techniques. Why Document? If you're asking this question, you've either never consumed software, or your software is perfect and self-documenting. I frankly don't believe either one. In the case of APIs, those consuming the API need to know how to use it. What endpoints are available? Which operations are available for each endpoint? What does each endpoint expect as a payload during the request? What can you expect as a payload in return? How will errors be communicated? While the promise of hypermedia APIs is that each response tells you the next steps available, you still, somewhere along the way, need more information — what payloads look like, which HTTP verbs should be used, and more. If you're not documenting your API, you're "doing it wrong." Where Should Documentation Live? This is the much bigger question. Of the questions I raised above, detailing what should be documented, there are two specific types. When discussing what operations are available, we have a technical solution in the form of the OPTIONS method and its counterpart, the Allow header. Everything else falls under end-user documentation. OPTIONS The HTTP specification details the OPTIONS method as idempotent, non-cacheable, and for use in detailing what operations are available for the given resource specified by the request URI. It makes specific mention of the Allow header, but does not limit what is returned for requests made via this method. The Allow header details the allowed HTTP methods for the given resource. Used in combination, you make an OPTIONS request to a URI, and it should return a response containing an Allow header; from that header value, you then know what other HTTP methods can be made to that URI. What this tells us is that our RESTful endpoint should do the following: When an OPTIONS request is made, return a response with an Allow header that has a list of the available HTTP methods allowed. For any HTTP method we do not allow, we should return a "405 Not Allowed" response. These are fairly easy to accomplish in ZF2. (See? I promised I'd get to some ZF2 code in this post!) When creating RESTful endpoints in ZF2, I recommend using Zend\Mvc\Controller\AbstractRestfulController. This controller contains an options() method which you can use to respond to an OPTIONS request. As with any ZF2 controller, returning a response object will prevent rendering and bubble out immediately so that the response is returned. namespace My\Controller; use Zend\Mvc\Controller\AbstractRestfulController; class FooController extends AbstractRestfulController { public function options() { $response = $this->getResponse(); $headers = $response->getHeaders(); // If you want to vary based on whether this is a collection or an // individual item in that collection, check if an identifier from // the route is present if ($this->params()->fromRoute('id', false)) { // Allow viewing, partial updating, replacement, and deletion // on individual items $headers->addHeaderLine('Allow', implode(',', array( 'GET', 'PATCH', 'PUT', 'DELETE', ))); return $response; } // Allow only retrieval and creation on collections $headers->addHeaderLine('Allow', implode(',', array( 'GET', 'POST', ))); return $response; } } The next trick is returning the 405 response if an invalid option is used. For this, you can create a listener in your controller, and wire it to listen at higher-than-default priority. As an example: namespace My\Controller; use Zend\EventManager\EventManagerInterface; use Zend\Mvc\Controller\AbstractRestfulController; class FooController extends AbstractRestfulController { protected $allowedCollectionMethods = array( 'GET', 'POST', ); protected $allowedResourceMethods = array( 'GET', 'PATCH', 'PUT', 'DELETE', ); public function setEventManager(EventManagerInterface $events) { parent::setEventManager($events); $events->attach('dispatch', array($this, 'checkOptions'), 10); } public function checkOptions($e) { $matches = $e->getRouteMatch(); $response = $e->getResponse(); $request = $e->getRequest(); $method = $request->getMethod(); // test if we matched an individual resource, and then test // if we allow the particular request method if ($matches->getParam('id', false)) { if (!in_array($method, $this->allowedResourceMethods)) { $response->setStatusCode(405); return $response; } return; } // We matched a collection; test if we allow the particular request // method if (!in_array($method, $this->allowedCollectionMethods)) { $response->setStatusCode(405); return $response; } } } Note that I moved the allowed methods into properties; if I did the above, I'd refactor the options() method to use those properties as well to ensure they are kept in sync. Also note that in the case of an invalid method, I return a response object. This ensures that nothing else needs to execute in the controller; I discover the problem and return early. End-User Documentation Now that we have the technical solution out of the way, we're still left with the bulk of the work left to accomplish: providing end-user documentation detailing the various payloads, errors, etc. I've seen two compelling approaches to this problem. The first builds on the OPTIONS method, and the other uses a hypermedia link in every response to point to documentation. The OPTIONS solution is this: use the body of an OPTIONS response to provide documentation. (Keith Casey gave an excellent short presentation about this at REST Fest 2012). The OPTIONS method allows for you to return a body in the response, and also allows for content negotiation. The theory, then, is that you return media-type-specific documentation that details the methods allowed, and what they specifically accept in the body. While there is no standard for this at this time, the first article I linked suggested including a description, the parameters expected, and one or more example request bodies for each HTTP method allowed; you'd likely also want to detail the responses that can be expected. { "POST": { "description": "Create a new status", "parameters": { "type": { "type": "string", "description": "Status type -- text, image, or url; defaults to text", "required": false }, "text": { "type": "string", "description": "Status text; required for text types, optional for others", "required": false }, "image_url": { "type": "string", "description": "URL of image for image types; required for image types", "required": false }, "link_url": { "type": "string", "description": "URL of image for link types; required for link types", "required": false } }, "responses": [ { "describedBy": "http://example.com/problems/invalid-status", "title": "Submitted status was invalid", "detail": "Missing text field required for text type" }, { "id": "abcdef123456", "type": "text", "text": "This is a status update", "timestamp": "2013-02-22T10:06:05+0:00" } ], "examples": [ { "text": "This is a status update" }, { "type": "image", "text": "This is the image caption", "image_url": "http://example.com/favicon.ico" }, { "type": "link", "text": "This is a description of the link", "link_url": "http://example.com/" }, ] } } If you were to use this methodology, you would alter the options() method such that it does not return a response object, but instead return a view model with the documentation. namespace My\Controller; use Zend\Mvc\Controller\AbstractRestfulController; class FooController extends AbstractRestfulController { protected $viewModelMap = array(/* ... */); public function options() { $response = $this->getResponse(); $headers = $response->getHeaders(); // Get a view model based on Accept types $model = $this->acceptableViewModelSelector($this->viewModelMap); // If you want to vary based on whether this is a collection or an // individual item in that collection, check if an identifier from // the route is present if ($this->params()->fromRoute('id', false)) { // Still set the Allow header $headers->addHeaderLine('Allow', implode( ',', $this->allowedResourceMethods )); // Set documentation specification as variables $model->setVariables($this->getResourceDocumentationSpec()); return $model; } // Allow only retrieval and creation on collections $headers->addHeaderLine('Allow', implode( ',', $this->allowedCollectionMethods )); $model->setVariables($this->getCollectionDocumentationSpec()); return $model; } } I purposely didn't provide the implementations of the getResourceDocumentationSpec() and getCollectionDocumentationSpec() methods, as that will likely be highly specific to your application. Another possibility is to use your view engine for this, and specify a template file that has the fully-populated information. This would require a custom renderer when using JSON or XML, but is a pretty easy solution. However, there's one cautionary tale to tell, something I already mentioned: OPTIONS, per the specification, is non-cacheable. What this means is that everytime somebody makes an OPTIONS request, any cache control headers you provide will be ignored, which means hitting the server for each and every request to the documentation. Considering documentation is static, this is problematic; it has even prompted blog posts urging you not to use OPTIONS for documentation. Which brings us to the second solution for end-user documentation: a static page referenced via a hypermedia link. This solution is insanely easy: you simply provide a Link header in your response, and provide a describedby reference pointing to the documentation page: Link: ; rel="describedby" With ZF2, this is trivially easy to accomplish: create a route and endpoint for your documentation, and then a listener on your controller that adds the Link header to your response. The latter, adding the link header, might look like this: namespace My\Controller; use Zend\EventManager\EventManagerInterface; use Zend\Mvc\Controller\AbstractRestfulController; class FooController extends AbstractRestfulController { public function setEventManager(EventManagerInterface $events) { parent::setEventManager($events); $events->attach('dispatch', array($this, 'injectLinkHeader'), 20); } public function injectLinkHeader($e) { $response = $e->getResponse(); $headers = $response->getHeaders(); $headers->addHeaderLine('Link', sprintf( '<%s>; rel="describedby"', $this->url('documentation-route-name') )); } } If you want to ensure you get a fully qualified URL that includes the schema, hostname, and port, there are a number of ways to do that as well; the above gives you the basic idea. Now, for the route and endpoint, there are tools that will help you simplify that task as well, in the form of a couple of ZF2 modules: PhlySimplePage and Soflomo\Prototype. (Disclosure: I'm the author of PhlySimplePage.) Both essentially allow you to specify a route and the corresponding template name to use, which means all you need to do is provide a little configuration, and a view template. Soflomo\Prototype has slightly simpler configuration, so I'll demonstrate it here: return array( 'soflomo_prototype' => array( 'documentation-route-name' => array( 'route' => '/api/documentation', 'template' => 'api/documentation', ), ), 'view_manager' => array( 'template_map' => array( 'api/documentation' => __DIR__ . '/../view/api/documentation.phtml', ), ), ); I personally have been using the Link header solution, as it's so simple to implement. It does not write the documentation for you, but thinking about it early and implementing it helps ensure you at least start writing the documentation, and, if you open source your project, you may find you have users who will write the documentation for you if they know where it lives. Conclusions Document your API, or either nobody will use it, or all you're hear are complaints from your users about having to guess constantly about how to use it. Include the following information: What endpoint(s) is (are) available. Which operations are available for each endpoint. What payloads are expected by the endpoint. What payloads can a user expect in return. What media types may be used for requests. What media types may be expected in responses. Additionally, make sure that you do the OPTIONS/Allow dance; don't just accept any request method, and report the standard 405 response for methods that you will not allow. Make sure you differentiate these for collections versus individual resources, as you likely may allow replacing or updating an individual resource, but likely will not want to do the same for a whole collection! Next time So far, I've covered the basics of RESTful JSON APIS, specifically recommending Hypermedia Application Language (HAL) for providing hypermedia linking and relations. I've covered error reporting, and provided two potential formats (API-Problem and vnd.error) for use with your APIs. Now, in this article, I've shown a bit about documenting your API both for machine consumption as well as end-users. What's left? In upcoming parts, I'll talk about ZF2's AbstractRestfulController in more detail, as well as how to perform some basic content negotiation. I've also had requests about how one might deal with API versioning, and will attempt to demonstrate some techniques for doing that as well. Finally, expect to see a post showing how I've tied all of this together in a general-purpose ZF2 module so that you can ignore all of these posts and simply start writing APIs. Updates Note: I'll update this post with links to the other posts in the series as I publish them. Part 1 Part 2 RESTful APIs with ZF2, Part 3 was originally published 25 February 2013 on https://mwop.net by Matthew Weier O'Phinney.

RESTful APIs with ZF2, Part 2

2013-02-13T07:40:00-06:00

In my last post, I covered some background on REST and the Richardson Maturity Model, and some emerging standards around hypermedia APIs in JSON; in particular, I outlined aspects of Hypermedia Application Language (HAL), and how it can be used to define a generic structure for JSON resources. In this post, I cover an aspect of RESTful APIs that's often overlooked: reporting problems. Background APIs are useful when they're working. But when they fail, they're only useful if they provide us with meaningful information; if all I get is a status code, and no indication of what caused the issue, or where I might look for more information, I get frustrated. In consuming APIs, I've come to the following conclusions: Error conditions need to provide detailed information as to what went wrong, and what steps I may be able to take next. An error code with no context gives me nothing to go on. Errors need to be reported consistently. Don't report the error one way one time, and another way the next. DO use HTTP status codes to indicate an error happened. Nothing is more irksome than getting back a 200 status with an error payload. Errors should be reported in a format I have indicated I will Accept (as in the HTTP header). Perhaps the only think more irksome than a 200 status code for an error is getting back an HTML page when I expect JSON. Why Status Codes Aren't Enough Since REST leverages and builds on HTTP, an expedient solution for reporting problems is to simply use HTTP status codes. These are well understood by web developers, right? 4xx error codes are errors made by the requestor, and are actually fairly reasonable to use for reporting things such as lack of authorization tokens, incomplete requests, unsupportable operations, or non-supported media types. But what happens when the error is on the server — because something has gone wrong such as inability to reach your persistence layer or credential storage? The 5xx series of status codes is sparse and wholly unsuited to reporting errors of these types — though you'll likely still want to use a 500 status to report the failure. But what do you present to the consumer so that they know whether or not to try again, or what to report to you so that you can fix the issue? A status code simply isn't enough information most of the time. Yes, you want to define standard status codes so that your clients can perform reasonable branching, but you also need a way to communicate details to the end-user, so that they can log the information for themselves, display information to their own end-users, and/or report it back to you so you can do something to resolve the situation. Custom Media Types The first step is to use a custom media type. Media types are typically both a name as well as a structure — and the latter is what we're after when it comes to error reporting. If we return a response using this media type, the client then knows how to parse it, and can then process it, log it, whatever. Sure, you can make up your own format — as long as you are consistent in using it, and you document it. But personally, I don't like inventing new formats when standard formats exist already. Custom formats mean that custom clients are required for working with the services; using a standard format can save effort and time. In the world of JSON, I've come across two error media types that appear to be gaining traction: application/api-problem+json and application/vnd.error+json API-Problem This particular media type is via the IETF. Like HAL, it provides formats in both JSON and XML, making it a nice cross-platform choice. As noted already, the media type is application/api-problem+json. The representation is a single resource, with the following properties: describedBy: a URL to a document describing the error condition (required) title: a brief title for the error condition (required) httpStatus: the HTTP status code for the current request (optional) detail: error details specific to this request (optional) supportId: a URL to the specific problem occurrence (e.g., to a log message) (optional) As an example: HTTP/1.1 500 Internal Error Content-Type: application/api-problem+json { "describedBy": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html", "detail": "Status failed validation", "httpStatus": 500, "title": "Internal Server Error" } The specification allows a large amount of flexibility — you can have your own custom error types, so long as you have a description of them to link to. You can provide as little or as much detail as you want, and even decide what information to expose based on environment. I personally like to point to the HTTP status code definitions, and then provide request-specific detail; I find this gives quick and simple results that I can later shape as I add more detail to my API. However, the specification definitely encourages you to have unique error types with discrete URIs that describe them — never a bad thing when creating APIs. vnd.error This is a proposed media type within the HAL community. Like HAL, it provides formats in both JSON and XML, making it a nice cross-platform choice. It differentiates from API-Problem in a few ways. First, it allows, and even encourages, reporting collections of errors. If you consider PHP exceptions and the fact that they support "previous" exceptions, this is a powerful concept; you can report the entire chain of errors that led to the response. Second, it encourages pushing detail out of the web service; errors include a "logRef" property that points to where the error detail lives. This is probably better illustrated than explained. The response payload is an array of objects. Each object has the following members: logRef: a unique identifier for the specific error which can then be used to identify the error within server-side logs (required) message: the error message itself (required) _links: HAL-compatible links. Typically, "help", "describes", and/or "describedBy" relations will be defined here. As an example, let's consider the API-Problem example I had earlier, and provide a vnd.error equivalent: HTTP/1.1 500 Internal Error Content-Type: application/vnd.error+json [ { "logRef": "someSha1HashMostLikely", "message": "Status failed validation", "_links": { "describedBy": {"href": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html"} } } ] vnd.error basically begs you to create custom error types, with documentation end-points that detail the source of the error and what you can do about it (this is true of API-Problem as well). The requirement to include a log reference ("logRef") and have it be unique can be a stumbling block to implementation, however, as it requires effort for uniquely identifying requests, and logging. However, both the identification and logging can be automated. Summary Error reporting in APIs is as important as the normal resource payloads themselves. Without good error reporting, when an API raises errors, clients have difficulty understanding what they can do next, and cannot provide you, the API provider, with information that will allow you to debug on the server side. As noted at the beginning of the article, if you follow the rules below, you'll make consumers of your API happier and more productive. DO use appropriate HTTP status codes to indicate an error happened. Report errors in a format I have indicated I will Accept (as in the HTTP header). Report errors consistently. Don't report the error one way one time, and another way the next. Standardize on a specific error-reporting media type . While you can create your own error structure, I recommend using documented, accepted standards. This will make clients more re-usable, and make many of your decisions for you. Provide detailed information as to what went wrong, and what steps I may be able to take next. Provide documentation for each type of error, and link to that documentation from your error payloads. Which brings me to… Next time I realize I still haven't covered anything specific to ZF2, but I'll start next time, when I cover the next topic: documenting your API. An undocumented API is a useless API, so it's good to start baking documentation in immediately. I'll survey some of the possibilities and how they can be implemented in ZF2 in the next installment, and then we can get our hands dirty with actual API development. Updates Note: I'll update this post with links to the other posts in the series as I publish them. Part 1 Part 3 RESTful APIs with ZF2, Part 2 was originally published 13 February 2013 on https://mwop.net by Matthew Weier O'Phinney.

RESTful APIs with ZF2, Part 1

2013-02-12T05:42:00-06:00

RESTful APIs have been an interest of mine for a couple of years, but due to circumstances, I've not had much chance to work with them in any meaningful fashion until recently. Rob Allen and I proposed a workshop for PHP Benelux 2013 covering RESTful APIs with ZF2. When it was accepted, it gave me the perfect opportunity to dive in and start putting the various pieces together. Background I've attended any number of conference sessions on API design, read countless articles, and engaged in quite a number of conversations. Three facts keep cropping up: JSON is fast becoming the preferred exchange format due to the ease with which it de/serializes in almost every language. The "holy grail" is Richardson Maturity Model Level 3. It's really hard to achieve RMM level 3 with JSON. Richardson Maturity Model As a quick review, the Richardson Maturity Model has the following 4 levels: Level 0: "The swamp of POX." Basically, a service that uses TCP for transport, primarily as a form of remote procedure call (RPC). Typically, these are not really leveraging HTTP in any meaningful fashion; most systems will use HTTP POST for all interactions. Also, you will often have a single endpoint for all interactions, regardless of whether or not they are strictly related. XML-RPC, SOAP, and JSON-RPC fall under this category. Level 1: "Resources." In these services, you start breaking the service into multiple services, one per "resource," or, in object oriented terms, per object. This means a distinct URL per object, which means each has its own distinct identity on the web; this often extends not only to the collection of objects, but to individual objects under the collection as well (e.g., /books as well as /books/life-of-pi). The service may still be RPC in nature, however, and, at this level, often is still using a single HTTP method for all interactions with the resource. Level 2: "HTTP Verbs." At this level, we start using HTTP verbs with our services in the way the HTTP specification intends. GET is for safe operations, and should be cacheable; POST is used for creation and/or updating; DELETE can be used to delete a resource; etc. Rather than doing RPC style methods, we leverage HTTP, occasionally passing additional parameters via the query string or request body. Considerations such as HTTP caching and idempotence are taken into account. Level 3: "Hypermedia Controls." Building on the previous level, our resource representations now also include links, which indicate what we can do next. At this level, our API becomes practically self-describing; given a single end-point, we should be able to start crawling it, using the links in a representation to lead us to the next actions. When I first started playing with web services around a decade ago, everything was stuck at Level 0 or Level 1 — usually with Level 1 users downgrading to Level 0 because Level 0 offerred consistency and predictability if you chose to use a service type that had a defined envelope format (such as XML-RPC or SOAP). (I even wrote the XML-RPC server implementation for Zend Framework because I got sick of writing one-off parsers/serializers for custom XML web service implementations. When you're implementing many services, predictability is a huge win.) A few years ago, I started seeing a trend towards Level 2. Web developers like the simplicity of using HTTP verbs, as they map very well to CRUD operations — the bread and butter of web development. Couple this concept with JSON, and it becomes trivially simple to both create a web service, as well as consume it. I'd argue that the majority of web developers are quite happy to be at Level 2 — and have no problem staying there. They're productive, and the concepts are easy — both to understand and to implement. Level 3, though, is where it becomes really interesting. The idea that I can examine the represention alone in order to understand what I can do next is very intriguing and empowering. JSON and Hypermedia With XML, hypermedia basically comes for free. Add some elements to your representation, and you're done — and don't forget the link relations! JSON, however, is another story. Where do the links go? There is no single, defined way to represent a hyperlink in JSON. Fortunately, there are some emerging standards. First is use of the "Link" HTTP header. While the page I linked shows only a single link in the header, you can have multiple links separated by commas. GitHub uses this when providing pagination links in their API. Critics will point out that the HTTP headers are not technically part of the representation, however; strict interpetations of REST and RMM indicate that the hypermedia links should be part of the resource representation. Regardless, having the links in the HTTP headers is useful for pre-traversal of a service, as you can perform HEAD requests only to discover possible actions and workflows. Collection+JSON is interesting, as it describes the entire JSON envelope. My one criticism is that it details too much; whenever I see a format that dictates how to describe types, I think of XML-RPC or SOAP, and get a little twitchy. It's definitely worth a look, though. What's captured my attention of late, however, is Hypertext Application Language, or HAL for short. HAL has very few rules, but succinctly describes both how to provide hypermedia in JSON as well as how to represent embedded resources — the two things that most need standardized structure in JSON. It does this while still providing a generic media type, and also describing a mirror image XML format! HAL Media Types HAL defines two generic media types: application/hal+xml and application/hal+json. You will use these as the response Content-Type, as they describe the response representation; the client can simply request application/json, and the response format remains compatible. HAL and Links HAL provides a very simple structure for JSON hypermedia links. First, all resource representations must contain hypermedia links, and all links are provided in a _links object: { "_links": { } } Second, links are properties of this object. The property name is the link relation, and the value is an object containing minimally an "href" property. { "_links": { "self": {"href": "http://example.com/api/status/1234"} } } If a given relation can have multiple links, you provide instead an array of objects: { "_links": { "self": {"href": "http://example.com/api/status/1234"}, "conversation": [ {"href": "http://example.com/api/status/1237"}, {"href": "http://example.com/api/status/1241"} ] } } Individual links can contain other attributes as desired — I've seen people include the relation again so that it's self-contained in the link object, and it's not uncommon to include a title or name. HAL and Resources HAL imposes no structure over resources other than requiring the hypermedia links; even then, you typically do not include the hypermedia links when making a request of the web service; the hypermedia links are included only in the representations returned by the service. So, as an example, you would POST the following: POST /api/status Host: example.com Accept: application/json Content-Type: application/json { "status": "This is my awesome status update!", "user": "mwop" } And from that request, you'd receive the following: 201 Created Location: http://example.com/api/status/1347 Content-Type: application/hal+json { "_links": { "self": {"href": "http://example.com/api/status/1347"} }, "id": "1347", "timestamp": "2013-02-11 23:33:47", "status": "This is my awesome status update!", "user": "mwop" } HAL and Embedded Resources The other important thing that HAL defines is how to embed resources. Why is this important? If the resource references other resources, you will want to be able to link to them so you can perform operations on them, too. Embedded resources are represented inside an _embedded object of the representation, and, as resources, contain their own _links object as well. Each resource you embed is assigned to a property of that object, and if multiple objects of the same type are returned, an array of resources is assigned. In fact, this latter is how you represent collections in HAL. Let's consider a simple example first. In previous code samples, I have a "user" that's a string; let's make that an embedded resource instead. { "_links": { "self": {"href": "http://example.com/api/status/1347"} }, "id": "1347", "timestamp": "2013-02-11 23:33:47", "status": "This is my awesome status update!", "_embedded": { "user": { "_links": { "self": {"href": "http://example.com/api/user/mwop"} } "id": "mwop", "name": "Matthew Weier O'Phinney", "url": "http://mwop.net" } } } I've moved the "user" out of the representation, and into the _embedded object — because this is where you define embedded resources. Note that the "user" is a standard HAL resource itself — containing hypermedia links. Now let's look at a collection: { "_links": { "self": {"href": "http://example.com/api/status"}, "next": {"href": "http://example.com/api/status?page=2"}, "last": {"href": "http://example.com/api/status?page=100"} }, "count": 2973, "per_page": 30, "page": 1, "_embedded": { "status": [ { "_links": { "self": {"href": "http://example.com/api/status/1347"} }, "id": "1347", "timestamp": "2013-02-11 23:33:47", "status": "This is my awesome status update!", "_embedded": { "user": { "_links": { "self": {"href": "http://example.com/api/user/mwop"} } "id": "mwop", "name": "Matthew Weier O'Phinney", "url": "http://mwop.net" } } } /* ... */ ] } } Note that the "status" property is an array; semantically, all resources under this key are of the same type. Also note that the parent resource has some additional link relations — these are related to pagination, and allow a client to determine what the next and last pages are (and, if we were midway into the collection, previous and first pages). Since the collection is also a resource, it has some interesting metadata — how many resources are in the collection, how many we represent per page, and what the current page is. Also note that you can nest resources — simply include an _embedded object inside an embedded resource, with additional resources, as I've done with the "user" resource inside the status resource shown here. It's turtles all the way down. Next Time The title of this post indicates I'll be talking about building RESTful APIs with ZF2 — but so far, I've not said anything about ZF2. I'll get there. But there's another detour to take: reporting errors. Updates Note: I'll update this post with links to the other posts in the series as I publish them. Part 2 Part 3 RESTful APIs with ZF2, Part 1 was originally published 12 February 2013 on https://mwop.net by Matthew Weier O'Phinney.

Zend Server, ZF2, and Page Caching

2012-11-05T15:25:00-06:00

Zend Server has a very cool Page Caching feature. Basically, you can provide URLs or URL regular expressions, and tell Zend Server to provide full-page caching of those pages. This can provide a tremendous performance boost, without needing to change anything in your application structure; simply enable it for a set of pages, and sit back and relax. However, this feature is not entirely straight-forward when using a framework that provides its own routing, such as ZF2. The reason is because it assumes by default that each match maps to a specific file on the filesystem, and prepares the caching based on the actual file it hits. What this means for ZF2 and other similar frameworks is that any page that matches will return the cached version for the first match that also matches the same file — i.e., index.php in ZF2. That's every page the framework handles. As an example, if I match on /article/\d+, it matches this to the file index.php, and then any other match that resolves to index.php gets served that same page. Not handy. The good part is that there's a way around this. When creating or modifying a caching rule, simply look for the text, "Create a separate cached page for each value of:" and click the "Add Parameter" button. Select _SERVER from the dropdown, and type [REQUEST_URI] for the value. Once saved, each page that matches the pattern will be cached separately. Note: the _SERVER key may vary based on what environment/OS you're deployed in. Additionally, it may differ based on how you define rewrite rules — some frameworks and CMS systems will append to the query string, for instance, in which case you may want to select the "entire query string" parameter instead of _SERVER; the point is, there's likely a way for you to configure it. Zend Server, ZF2, and Page Caching was originally published 5 November 2012 on https://mwop.net by Matthew Weier O'Phinney.

OpenShift, ZF2, and Composer

2012-11-01T15:25:00-05:00

I was recently shopping around for inexpensive cloud hosting; I want to try out a couple of ideas that may or may not have much traffic, but which aren't suited for my VPS setup (the excellent ServerGrove); additionally, I'm unsure how long I will maintain these projects. My budget for this is quite small as a result; I'm already paying for hosting, and am quite happy with it, so this is really for experimental stuff. I considered Amazon, Orchestra.io, and a few others, but was concerned about the idea of a ~$50/month cost for something I'm uncertain about. When I asked in #zftalk.dev, someone suggested OpenShift as an idea, and coincidentally, the very next day Zend announced a partnership with RedHat surrounding OpenShift. The stars were in alignment. In the past month, in the few spare moments I've had (which included an excellent OpenShift hackathon at ZendCon), I've created a quick application that I've deployed and tested in OpenShift. These are my findings. ZF2 I didn't really have to do anything different to have zf2 work; the standard .htaccess provided in the skeleton application worked flawlessly the first time (I've worked with some cloud environments where this is not the case). The only frustration I had was the default directory structure OpenShift foists upon us: data/ libs/ misc/ php/ This is not terrible, by any stretch. However, it's attempting to dictate the application structure, which I'm not terribly happy with — particularly as my structure may vary based on the framework I'm using (or not!), and because I may already have a project written that I simply want to deploy. In particular, the php directory is galling — it's simply the document root. Most frameworks I've used or seen call the equivalent directory public, or web, or html — but never php (in large part because the only PHP file under the document root in most frameworks is the index.php that acts as the front controller). It would be nice if this were configurable. This conflicts a bit with how a ZF2 app is structured. I ended up doing the following: Removed php and symlinked my public directory to it. Removed libs and symlinked my vendor directory to it. Removed misc as I had no need to it. Nothing too big, thankfully — but problematic from the perspective of, "I've already developed this app, but now I have to make changes for it to work on a specific cloud vendor." Composer My next question was how to use Composer during my deployment process, and some some googling found some answers for me. Basically, I needed to create a deploy task that does two things: Unset the GIT_DIR environment variable. Evidently, the build process operates as part of a git hook, and since Composer often uses git repositories, this can lead to problems. Change directory to OPENSHIFT_REPO_DIR, which is where the application root (not document root!) lives. Once I did those, I could run my normal composer installation. The deploy task looks like this: #!/bin/bash # .openshift/action_hooks/deploy ( unset GIT_DIR ; cd $OPENSHIFT_REPO_DIR ; /usr/local/zend/bin/php composer.phar install ) This leads into my next topic. Deployment First off, as you probably guessed from that last secton, there are hooks for deployment — it doesn't have to be simply git. I like this, as I may have additional things I want to do during deployment, such as retrieving and installing site-specific configuration files, installing Composer-defined dependencies (as already noted), etc. Over all, this is pretty seamless, but it's not without issues. I've been told that some of my issues are being worked on, so those I won't bring up here. The ones that were a bit strange, and which caught me by surprise, though, were: Though the build process creates the site build from git, your submodules are not updated recursively. This tripped me up, as I was using EdpMarkdown, and had installed it as a submodule. I ended up having to import it, and its own submodule, directly into my project so that it would work. I installed the MongoDB cartridge. Ironically, it was not then enabled in Zend Server, and I had to go do this. This should be turnkey. /usr/bin/php is not the same as /usr/local/zend/bin/php. This makes no sense to me if I've installed Zend Server as my base gear. Considering they're different versions, this can be hugely misleading and lead to errors. I understand there are reasons to have both — so simply be aware that if you use the Zend Server gear, your tasks likely should use /usr/local/zend/bin/php. The good parts? You can alias an application to a DNS CNAME — meaning you can point your domain name to your OpenShift applications. Awesome! Simplicity of adding capabilities, such as Mongo, MySQL, Cron, and others. In most cases, this is simply a "click on the button" and it's installed and available. Zend Server. For most PHP extensions, I can turn them on or off with a few mouse clicks. If I want page-level caching, I don't have to do anything to my application; I can simply setup some rules in the Zend Server interface and get on with it, and enjoy tremendous boosts to performance. I used to enjoy taming and tuning servers; most days anymore, I just want them to work. SSH access to the server, with a number of commands to which I've been given sudoer access. If you're going to sandbox somebody, this is a fantastic way to do it. Oh, also: SSH tunnels to services like Mongo and MySQL just work (via the rhc-port-forward command). Summary Over all, I'm quite pleased. While it took me a bit to find the various incantations I needed, the service is quite flexible. For my needs, considering I'm doing experimental stuff, the price can't be beat (the current developer preview is free). Considering most stuff I do will fall into this or the basic tier, and that most cartridges do not end up counting against your alotment of gears, the pricing ($0.05/hour) is extremely competitive. OpenShift, ZF2, and Composer was originally published 1 November 2012 on https://mwop.net by Matthew Weier O'Phinney.

ZF2 Modules Quickstart (Screencast)

2012-09-19T13:10:00-05:00

One of the exciting features of the newly released Zend Framework 2 is the new module system. While ZF1 had modules, they were difficult to manage. All resources for all modules were initialized on each request, and bootstrapping modules was an onerous task. Due to the difficulties, modules were never truly "plug-and-play", and thus no ecosystem ever evolved for sharing modules. In Zend Framework 2, we've architected the MVC from the ground up to make modular applications as easy as possible. Within ZF2, the MVC simply cares about events and services — and controllers are simply one kind of service. As such, modules are primarily about telling the MVC about services and wiring event listeners. To give you an example, in this tutorial, I'll show you how to install the Zend Framework 2 skeleton application, and we'll then install a module and see how easy it is to add it to the application and then configure it. To keep things simple, I'm using a unix-like environment. As such, if you are on Windows, you may not have the same command-line tools available. If you are in such a situation, perhaps try this inside a Linux virtual machine. Zend Framework 2 Module Quickstart Let's start by creating a new project. We'll execute a few commands to download a skeleton application archive and extract it. $ mkdir newproject $ cd newproject $ wget https://github.com/zendframework/ZendSkeletonApplication/tarball/master \ > -O ZendSkeletonApplication.tgz $ tar xzf ZendSkeletonApplication.tgz --strip-components=1 The Zend Framework skeleton application can be downloaded directly off of GitHub. I'm showing using the download from master, but you can also download a tarball or zipball for individual tags as well. Because the download URL does not include an extension, I use the -O switch to tell wget what filename to save to. tar has a nice option, --strip-components, which allows you to tell it to descend a certain number of levels deep into the archive when deflating. Since I know the tarball has a top-level directory named after the repository and a sha1, I'm simply telling tar to skip that and give me the contents of its child directory. At this point you have the skeleton application, but it has no dependencies — not even Zend Framework itself! Let's rectify that situation. We'll use the dependency management tool Composer to do this. We include the Composer phar file within the skeleton application to make this fairly easy. Simply execute the following: $ php composer.phar install You may get a notice indicating that the composer version is older, and to run self-update; you can ignore that for now. If all goes well, you should now have Zend Framework installed. Let's test it out. I'm going to use the built-in web server in PHP 5.4 to demonstrate. $ cd public $ php -S localhost:8080 If I browse to http://localhost:8080 I should now see the landing page for the skeleton application. Let's add a module to the application. Many sites require a contact form. I've written one as a module some time ago, and called it PhlyContact. To install it, I'll edit my project's composer.json and tell it about that dependency: { "require": { "php": ">=5.3.3", "zendframework/zendframework": "dev-master", "phly/phly-contact": "dev-master" } } I know the name of the component from http://packagist.org/, and I'm telling Composer that I want to use whatever the latest version is on its master branch on GitHub. I happen to also know that PhlyContact requires a dev-master version of Zend Framework, so I'll alter that dependency for now. Now, we need to tell composer to update our dependencies. $ php composer.phar update After executing the command, we should now see that it has installed; this may take a little while. You need to inform the application about the module. This is so that we don't have to perform expensive file-system scanning operations, but also to make it explicit in your code what modules you're actually using. Enabling a module is usually as easy as adding an entry to config/application.config.php: 'modules' => array( 'Application', 'PhlyContact', ), This particular module provides some reasonable defaults. In particular, it uses a CAPTCHA adapter that doesn't require additional configuration, and assumes that you will want to use the default Sendmail mail transport. As such, we can simply browse to it now. I happen to know that the module defines a /contact end point. Let's fire up our PHP web server again, and browse to that URL. $ cd public $ php -S localhost:8080 It just works! One philosophy we have for distributable modules in Zend Framework 2 is that you should not need to touch the code in modules you install in your application. Instead, you should be able to configure and override behavior within the application configuration or in your application's site-specific modules. Let's alter the contact module to: first, change the URL it responds to, and second, use the "file" mail transport. Let's look at the default configuration. I'll browse to vendor/phly/phly-contact/config/ and look at the module.config.php file. return array( 'phly_contact' => array( 'captcha' => array( 'class' => 'dumb', ), 'form' => array( 'name' => 'contact', ), 'mail_transport' => array( 'class' => 'Zend\Mail\Transport\Sendmail', 'options' => array( ) ), 'message' => array( /* 'to' => array( 'EMAIL HERE' => 'NAME HERE', ), 'sender' => array( 'address' => 'EMAIL HERE', 'name' => 'NAME HERE', ), 'from' => array( 'EMAIL HERE' => 'NAME HERE', ), */ ), ), /* ... */ 'router' => array( 'routes' => array( 'contact' => array( 'type' => 'Literal', 'options' => array( 'route' => '/contact', 'defaults' => array( '__NAMESPACE__' => 'PhlyContact\Controller', 'controller' => 'Contact', 'action' => 'index', ), ), 'may_terminate' => true, 'child_routes' => array( 'process' => array( 'type' => 'Literal', 'options' => array( 'route' => '/process', 'defaults' => array( 'action' => 'process', ), ), ), 'thank-you' => array( 'type' => 'Literal', 'options' => array( 'route' => '/thank-you', 'defaults' => array( 'action' => 'thank-you', ), ), ), ), ), ), ), /* ... */ ); Okay, that's interesting. I can define the captcha and options to use, the name of the contact form, the mail transport I want to use, and even who the email is sent from and who it goes to. In addition, it defines some routes. I'll create a new file, config/autoload/phly-contact.local.php. This is a local configuration file that will not be checked into my version control system. Now, let's add some configuration. First, I'll configure my mail transport. return array( 'phly_contact' => array( 'mail_transport' => array( 'class' => 'Zend\Mail\Transport\File', 'options' => array( 'path' => 'data/mail/', ), ), ), ); I'm telling the module to use the File mail transport, and telling the transport where I want messages written. By default, Zend Framework calls chdir() to change directory to the project root, so I can reference a directory relative to that. I'm simply going to write to a data/mail/ directory. Let's create that, and make it world-writable for now to ensure the web server can write to it. (In production, you'd only want it writable by the web server user.) $ mkdir -p data/mail $ chmod a+rwX data/mail Now, let's change the base URL the contact form responds to; I want it to respond to /contact-us. Another principle of re-usable modules in ZF2 is that we recommend creating tree routes for each module, with the root of the tree being a literal route. This makes it easy to alter the base for routing, without needing to redefine all the routes in the module. I'll add the following to my local configuration, then. I'll simply override the parent route for my module, named "contact", and point it at a different URL. 'router' => array( 'routes' => array( 'contact' => array( 'options' => array( 'route' => '/contact-us', ), ), ), ), Let's see if all this worked! Once again, I'll fire up PHP's built-in web server. $ cd public $ php -S localhost:8080 Now, let's browse to http://localhost:8080/contact-us — looks good! Just as an experiment, let's try the previously configured URL, http://localhost:8080/contact. We get a 404 now! Now, let's submit the form. I'll fill in some information; it's asking for my email address, a subject line, and a message, as well as for me to solve a simple CAPTCHA. Once I've done all that, I can send it. If all is well, we should now have a mail file in our data directory. Let's check. $ ls -l data/mail/ And now let's look at it. $ cat data/mail/ZendMail_1347989389_1009740165.tmp Date: Tue, 18 Sep 2012 12:29:49 -0500 From: me@mwop.net Reply-To: me@mwop.net Subject: [Contact Form] Suspense! Suspenseful, isn't it? Looks good! Zend Framework 2 provides a wonderful modular architecture that will enable an ecosystem of 3rd party modules that should save you time and energy when developing your applications. I've demonstrated a simple one, a contact form, but many, many more already exist, and with a stable release now available, you should see that number grow. This is truly a wonderful step forward for developers, and I hope you find it as exciting as I do. ZF2 Modules Quickstart (Screencast) was originally published 19 September 2012 on https://mwop.net by Matthew Weier O'Phinney.

On Microframeworks

2012-08-17T11:00:00-05:00

A number of months ago, Ed Finkler started a discussion in the PHP community about "MicroPHP"; to summarize, the movement is about: Building small, single-purpose libraries. Using small things that work together to solve larger problems. I think there are some really good ideas that have come out of this, and also a number of questionable practices1. One piece in particular I've focussed on is the concept of so-called “microframeworks”. What is a microframework? PHP has had microframeworks for quite some time2, though I only really first saw the term being used around 3 years ago. The “grand-daddy” of modern-day microframeworks can actually be traced to Ruby, however, and specifically Sinatra. Sinatra is not so much a framework as it is a domain-specific language (DSL). The language and structure it created, however, have been re-created in the vast majority of microframeworks you see currently in the PHP arena. Specifically, it describes how to map HTTP request methods and paths to the code that will handle them. It borrowed route matching ideas from Ruby on Rails, and relied on the fact that Ruby uses the last value of a block as the return value. As some simple examples: get '/hello/:name' do |n| "Hello #{n}!" end post '/address' # create address end put '/address/:id' |i| # update address end get '/feed.?:format?', :provides => ['rss', 'atom', 'xml'] do builder :feed end The language is expressive, and allows the developer to focus on two things: What are the specific entry points (URIs) for the application? What needs to be done for each specific entry point? I'd argue that the above two points are the defining characteristics of modern microframeworks. Typically, the entry points are given the term "routing", and the second corresponds to "controllers". PHP implementations I'd argue one of the earliest microframework implementations, though it wasn't termed as such, was Horde Routes3 (which was itself inspired by Python Routes, in turn inspired by the Rails routing system, like Sinatra). It follows the two principles I outlined above: it allows defining routes (entry points), and mapping them to controllers. Controllers for Routes are simply classes, and a route must provide both a controller and an action in the match, with the latter corresponding to a method on the controller class. Since around 2009, I've seen an increasing number of new PHP microframeworks4 that follow in the steps of Sinatra and Horde. In the various implementations I've looked at, instead of using a DSL, the authors have all opted for either a procedural or OOP interface. Starting with PHP 5.3, most authors have also primarily targetted any PHP callable as a controller, favoring callbacks specifically. The fundamental ideas remain the same as Sinatra, however: /* Procedural */ get('/hello/:name', function ($n) { return "Hello {$n}!"; }); post('/address', function () { // create address }); put('/address/:id' function ($i) { // update address }); get('/feed.?:format?', function($feed, $format) { return builder($feed, $format); }); /* OOP */ $app->get('/hello/:name', function ($n) { return "Hello {$n}!"; }); $app->post('/address', function () { // create address }); end $app->put('/address/:id', function ($i) { // update address }); $app->get('/feed.?:format?', function ($feed, $format) use ($app) { return $app->builder($feed, $format); })->constraints(['format' => '/^(rss|atom|xml)$/']); One key difference I've witnessed in the implementations is surrounding how route matches are passed to the callback. In the examples above, they are passed as individual arguments to the handler. Some, however, opt for an approach more like Sinatra, which passes a single "params" argument into the scope of the handler. This approach tends to be more expedient both from an implementation standpoint as well as a performance standpoint, as it does not require reflection to determine name and position of arguments, and makes handling wildcard arguments simpler. I've seen this latter approach handled several ways: // Pass in route match parameters as an argument. $app->get('/feed.:format', function ($params) { $format = $params['format']; }); // Pass in the $app instance, and retrieve route // match parameters from it. $app->get('/feed.:format', function ($app) { $format = $app->params('format'); }); // Curry in the $app instance when desired, and // retrieve route match parameters from it. $app->get('/feed.:format', function () use ($app) { $format = $app->params('format'); }); Another difference I've seen is in how route constraints, defaults, and names are handled. The most elegant solutions usually allow chaining method calls in order to alter this data: $app->get('/feed.:format', function ($app) { // ... })->constraints(['format' => '/^(atom|xml|json)$/']) ->name('feed'); One common feature I've seen is the ability to generate URLs based on the defined routes. Most commonly, this is a function or method urlTo(), which takes a route name, and an associative array of replacements. echo $app->urlTo('feed', ['format' => 'atom']); That's it in a nutshell: the ability to match HTTP request methods and path information, and map it to controllers/handlers, and to generate URLs based on those present in the application. What are they good for? In my research and experience, microframeworks have three typical use cases: Prototyping. Because of their simplicity, microframeworks are fantastic for prototyping a basic website. Very often, in the early stages of a site, you have a limited number of pages, and most often simply need to render a template with limited variable substitutions. Microframeworks are a perfect fit for this. APIs. API needs are usually quite well-defined, and often involve a small, finite number of URLs. The logic required is usually already encapsulated in business objects, so the application layer is simply for filtering and returning a representation. Microframeworks again offer a nice fit. Small, mostly static sites. Similar to the first point, if you know the site will be relatively small and mostly static, then the minimal overhead of a microframework is often a good fit. Where do microframeworks fail? Because of the rather declarative nature of microframeworks, and the typically 1:1 mapping of a route to a controller, microframeworks do not tend to promote code re-use. Additionally, this extends to how microframework applications are organized: usually, there are no clear guidelines on how to organize routes and controllers, much less separate them into multiple files. This can lead to maintenance issues as the application grows, as well as logistical issues whenever you need to add new routes and controllers (do they go at the top, or bottom? are there other routes that could potentially match as well? etc.). Additionally, though many frameworks offer ways to alter the workflow of the application either via hooks, events, or “middleware”5, most of these are limited in scope, often non-reusable, and often non-stackable. As such, comprehensive manipulation of the application workflow is out of reach. One other area that is overlooked, however, is one I find curious, particularly in light of the MicroPHP movement: so much of the underlying plumbing is basically the same, yet every microframework re-implements it. Specifically: Routing is basically the same across most implementations, following the same basic specifications outlined in Rails. There are very few differences in the public APIs. Request and Response object abstraction is largely the same as well, providing access to query/post/cookie/session/etc. parameters through roughly equivalent APIs. Many implement their own view layers.6 Most of this code should be considered commodity code at this point. There are several outstanding view layers and templating engines available (Smarty, Twig, Savant, Zend\View). Standalone routing libraries exist such as Horde Routes, and even those bundled with frameworks are often available separately via Composer or Pyrus; the same is true with Request and Response object abstraction. It seems to me that a few microframework authors should be working on abstracting these concerns, and then focussing their efforts on differentiators in their own microframeworks. An experiment Building on my last point, I looked at the APIs of Limonade and Slim Framework, and built up a specification for a microframework. I then matched as many pieces of it as possible to existing components in ZF2, and started building. In a matter of a few hours, I had written up a complete test suite7 and all code for a microframework, featuring the following (this is basically the testdox output from the unit test suite): Lazy loads request Lazy loads response Request is injectible Response is injectible Halt should raise halt exception Response should contain status provided to halt Response should contain message provided to halt Stop should raise halt exception Response should remain unaltered after stop Redirect should raise halt exception Redirect should set 302 response status by default Redirect should set response status based on provided status code Redirect should set location header Map creates a segment route when provided with a string route Map can receive a route object Passing invalid route raises exception Map can receive a callable Passing invalid controller to route does not immediately raise exception Accessing invalid controller raises exception Passing invalid method to route via method raises exception Can set methods route responds to singly Can set methods route responds to as array Can set methods route responds to as multiple arguments Can specify additional method types to respond to Can specify route name Adding route using method type creates route that responds to that method type Running with no matching routes raises page not found exception Routing sets list of named routes Routing sets lists of routes by method Successful routing dispatches controller Unsuccessful routing triggers 404 event Calling halt triggers halt event Invalid controller triggers 501 event Exception raised in controller triggers 500 event Can pass to next matching route Url for helper assembles url based on name provided Url for helper assembles url based on name and params provided Url for helper assembles url based on current route match when no name provided Composes logger instance by default Can inject specific logger instance Mustache view is used by default Can inject alternate view instance Render renders a template to the response View model returns mustache view model by default Subsequent calls to view model return separate instances Can provide view model prototype I utilized ZF2's routing library from its MVC component, the request and response objects from its HTTP component, its Log component, and the Session component. These had a few other dependencies, but nothing terribly onerous. For the view, I used my own phly_mustache, and provided a basic "view model" implementation that receives the application instance, thus allowing the ability to call application helpers (such as url generation). To make installation simple, I used Composer to manage my dependencies on specific ZF2 components and for phly_mustache. The microframework contains only the code it needs to get its work done, leveraging the work of others whenever possible. This post is not meant as a way to announce a new microframework, however.8 The point of the experiment was to prove something: microframeworks are trivially easy to write, particularly if you follow the principals of MicroPHP, and re-use existing code. Just because code comes from a framework or a third-party library does not make it suspect or inferior; in fact, whenever possible, you should leverage such code so you can focus on writing awesome applications. Lessons learned I really like microframeworks for specific problems: prototyping, APIs, and small, simple sites. I think they are ideally suited for these tasks. That said, I'd love to see some solid libraries targetting the fundamental, shared aspects of these efforts: routing, request and response abstraction, etc. With dependency management tools such as Composer and Pyrus, having required dependencies is not a big deal anymore, and re-use should be encouraged. Also, writing a microframework is an excellent coding exercise. It helps a developer appreciate the complexities of abstraction while limiting the number of moving parts. I highly recommend it as an exercise — but do it using available components, and be prepared to throw it away and instead collaborate with others, or adopt something which better solves both the problems you have and the problems you anticipate. In sum: Use the right tool for the job. If you foresee expanding requirements in your project's future, you may want to evaluate a full-stack framework,9 or consider building something robust that suits your specific project's needs. Use microframeworks where and when they make sense. Afterword I'm well aware that Fabien Potencier has written a comprehensive series of posts on creating a microframework using Symfony 2 components. I deliberately chose not to read them until (a) ZF2 was almost ready to release, and (b) I'd had a chance to formulate my own opinions on microframeworks. They're an excellent read, however, and show a nice progression of development from flat PHP to a fully functional microframework; click the link and see for yourself. Footnotes 1 In particular, I feel that the movement (a) disparages components from larger libraries simply because they originate from a larger library, and (b) distrust any code that has additional dependencies. This latter I find truly puzzling, as I'd think it fits the idea of “use small things that work together to solve larger problems.” If the code solves a particular problem and allows you to focus on a larger problem, where it originates and the number of dependencies should not be an issue. 2 In fact, my first foray into MVC in PHP was writing a clone of Perl's CGI::Application, which in many ways is also a microframework. 3 Trivia: Both authors of Horde Routes worked at Zend when I first started at the company, and Mike Naberezny wrote the very first lines of code for Zend Framework. 4 I swear, you see new ones on Github daily, and on PHP Developer at least once a week. 5 Slim has this concept. Basically, any callables placed between the route string and the last callable when defining a route — i.e., the “middle” arguments, and thus middleware — will be executed in order prior to attempting to execute the controller. 6 Slim is an outlier here, as it utilizes Twig by default. 7 I'm sure that my TDD experiment will warm the soul of the Grumpy Programmer. 8 That said, if you want to look at the results, you can find Phlyty on Github. 9 As you may guess, I'm biased towards Zend Framework. However, you should always carefully evaluate a framework against your project's needs. On Microframeworks was originally published 17 August 2012 on https://mwop.net by Matthew Weier O'Phinney.

ZF2's New Controller::init()

2012-07-30T15:40:00-05:00

In Zend Framework 1, controller's had an init() method, which was called after the controller was instantiated. The reason for it was to encourage developers not to override the constructor, and thus potentially break some of the functionality (as a number of objects were injected via the constructor). init() was useful for doing additional object initialization. class MyController extends Zend_Controller_Action { public function init() { // do some stuff! } } But this feature is missing from ZF2; how can we accomplish this sort of pattern? Constructor Overriding Why didn't we add the init() method in Zend Framework 2? Because we don't have a constructor by default, and also to reduce overall method calls; if a controller won't be overriding the method, there's no reason to make the call in the first place. So, that means, in ZF2, unlike ZF1, to do the same thing, you can simply define a constructor in your controller: use Zend\Mvc\Controller\AbstractActionController; class MyController extends AbstractActionController { public function __construct() { // do some stuff! } } Except there's one specific and often-needed use case where this fails: if you want to wire listeners to the controller's event manager. Events Why does this fail with the event manager? Because when we're in the constructor, we don't yet have an EventManager instance! The event manager instance is injected after instantiation. As such, we need to attach to it once we know we have an event manager. Which is… when it's set. This can be done very simply by overriding the setEventManager() method. In the next example, we'll define a listener for the "dispatch" event that redirects if certain criteria is not met. use Zend\EventManager\EventManagerInterface; use Zend\Mvc\Controller\AbstractActionController; class MyController extends AbstractActionController { public function setEventManager(EventManagerInterface $events) { parent::setEventManager($events); $controller = $this; $events->attach('dispatch', function ($e) use ($controller) { $request = $e->getRequest(); $method = $request->getMethod(); if (!in_array($method, array('PUT', 'DELETE', 'PATCH'))) { // nothing to do return; } if ($controller->params()->fromRoute('id', false)) { // nothing to do return; } // Missing identifier! Redirect. return $controller->redirect()->toRoute(/* ... */); }, 100); // execute before executing action logic } } The beauty of this is that we only override when we need to, and we override in the specific context we're trying to influence. Additionally, we're only registering the event listener if this particular controller is instantiated — which helps reduce our overall number of listeners, and thus shapes our call graph based on the controller invoked. Other dependencies As you'll likely note, the event manager is not the only case where you may need to follow a similar pattern. Any time your initialization logic may be based on a dependency, you'll want to override the setter in which that dependency is injected. Got any ZF2 tips of your own to share? Blog them! Update: ServiceManager A few folks in the comments were remarking that the felt that omitting the init() method makes it harder for developers to identify when and where to do initialization logic, particularly when you may be working with multiple dependencies. Which made me realize there's another place I missed, one that's potentially even better suited to initialization: the ServiceManager. Basically, if you find yourself having complex initialization needs, or many dependencies, you should be building a factory for your controller, and wiring it to the ServiceManager. This can happen in one of several places, but my preference is in my module's Module class, in the getControllerConfig() method. This method returns configuration for the controller manager that instantiates, validate, and injects controllers; it's basically a type of ServiceManager, and, in fact, has access to the main application's instance. I'll take the previous example, and wire it in the context of a factory: namespace My use Zend\EventManager\EventManagerInterface; class Module { /* * Assume some other methods, such as getConfig(), etc. * Also assume that a route will return a controller named * "My\Controller\My" which we assume will reference a controller * within our current namespace. */ public function getControllerConfig() { return array('factories' => array( 'My\Controller\My' => function ($controllers) { $services = $controllers->getServiceLocator(); $controller = new Controller\MyController(); $events = $services->get('EventManager') $events->attach('dispatch', function ($e) use ($controller) { $request = $e->getRequest(); $method = $request->getMethod(); if (!in_array($method, array('PUT', 'DELETE', 'PATCH'))) { // nothing to do return; } if ($controller->params()->fromRoute('id', false)) { // nothing to do return; } // Missing identifier! Redirect. return $controller->redirect()->toRoute(/* ... */); }, 100); // execute before executing action logic $controller->setEventManager($events); return $controller; }; )); } } The above will create the controller, grab an event manager instance, attach the listener, and then inject the event manager into the controller. If you wanted to do more complex work, you definitely could — and this would be the place to do it. ZF2's New Controller::init() was originally published 30 July 2012 on https://mwop.net by Matthew Weier O'Phinney.

ZF2 Forms in Beta5

2012-07-05T15:40:00-05:00

Forms are a nightmare for web development. They break the concept of separation of concerns: They have a display aspect (the actual HTML form) They have a validation aspect And the two mix, as you need to display validation error messages. On top of that, the submitted data is often directly related to your domain models, causing more issues: Not all elements will have a 1:1 mapping to the domain model — buttons, CSRF protection, CAPTCHAs, etc. usually are application-level concerns, but not domain issues. Names valid for your domain model may not be valid names for HTML entities. Add to this that the validation logic may be re-usable outside of a forms context, and you've got a rather complex problem. Forms in ZF2 Starting in 2.0.0beta4, we offerred a completely rewritten Form component. In fact, it's not just a Form component — a new component, InputFilter, was also added. InputFilter is a component that provides re-usable validation and normalization logic, and can be used with forms or your domain model. The Form component is basically a bridge between domain models/validation and the view layer. However, this means a bit more complexity for the end-user. You now must: Create your form, which consists of elements and fieldsets. Create an input filter, consisting of inputs. Inform the form of the input filter. It's a bit of work. And there's more: we wanted to simplify the process of getting your validated values into your domain objects. For this, we added a concept of hydrators, which map the validated form values to an object you bind to the form. Now you have three pieces to keep track of — form (and its elements), input filter (and its inputs), and a hydrator. So, a few developers had an idea: use annotations on the domain model objects to define these items, letting you keep it all in one place. While I'm not normally a fan of annotations, I immediately saw the appeal in this particular situation. An Example Let's consider a very simple example. The following domain object represents data for a user, and includes a variety of elements we'd represent in a form. namespace MyVendor\Model; use Zend\Form\Annotation; /** * @Annotation\Hydrator("Zend\Stdlib\Hydrator\ObjectProperty") * @Annotation\Name("user") */ class User { /** * @Annotation\Attributes({"type":"text" }) * @Annotation\Validator({"type":"Regex","options":{"regex":"/^[a-zA-Z][a-zA-Z0-9_-]{1,19}/"}}) * @Annotation\Options({"label":"Username:"}) */ public $username; /** * @Annotation\Required(false) * @Annotation\Attributes({"type":"text" }) * @Annotation\Options({"label":"Your full name:"}) */ public $fullname; /** * @Annotation\Type("Zend\Form\Element\Email") * @Annotation\Options({"label":"Your email address:"}) */ public $email; /** * @Annotation\Type("Zend\Form\Element\Url") * @Annotation\Options({"label":"Your home page:"}) */ public $uri; } So, what does the above do? The "name" annotation gives a form or element a specific name. The "attributes" annotation indicates what attributes to compose into the form or element. Similarly, the "options" annotation specifies options to compose into an element. These typically include the label, but may include other configuration that doesn't have an exact analog in the HTML attributes. The "validator" annotation indicates a validator to compose for the input for a given element. We also ship a "filter" annotation. The "type" annotation indicates a class to use for that particular form or element. In the specific cases used above, the elements actually provide default filters and validators, simplifying setup further! Last, but not least, the "hydrator" annotation indicates a Zend\Stdlib\Hydrator implementation to use to relay data between the form and the object. I'll cover this more shortly. So, let's now turn to creating a form and consuming it. use MyVendor\Model\User; use Zend\Form\Annotation\AnnotationBuilder; $user = new User(); $builder = new AnnotationBuilder(); $form = $builder->createForm($user); $form->bind($user); $form->setData($dataFromSomewhere); if ($form->isValid()) { // $user is now populated! echo $form->username; return; } else { // probably need to render the form now. } You're not quite done, really — most likely, you'll need to include a submit button of some sort, and it's always good practice to include a token to prevent CSRF injections. But with the above, you've accomplished the major headaches of setting up a form — and using the data — with minimal fuss. Much more! The form support in ZF2 offers a ton of other features, some of which are not specific to forms even. ZF2 supports a variety of hydration strategies, which allow you to pass data to and from objects. The example above uses one that suggests a 1:1 mapping between the inputs and the object properties; other strategies include using ArrayObject, using class mutator methods, and more. At this point, you can hydrate an entire form, as well as individual fieldsets! You can provide custom annotations. While this feature is not documented yet, you can tell the AnnotationBuilder about additional annotation classes, as well as provide listeners for those annotations so that they can interact with the form construction process. As an example, one contributor has already used these features to utilize Doctrine annotations to inform the builder about the name of a property, as well as indicate validators. (Side note: ZF2 now uses Doctrine's annotation syntax and parser by default.) There are a number of features targetting collections, so that your client-side code can return arbitrary numbers of a specific fieldset type (e.g., collecting addresses for an applicant), and the form will be able to validate each. You can read more about those features from the author himself. These features are all now available starting with the newly released 2.0.0beta5 version, which you can grab from the ZF2 packages site. I'm really excited with the solutions we've created in ZF2, and even more excited to see people put them to use! ZF2 Forms in Beta5 was originally published 5 July 2012 on https://mwop.net by Matthew Weier O'Phinney.