2019-01-24T11:30:00-06:00 Laminas_Feed_Writer https://mwop.net/blog/tag/programming 2019-01-24T11:30:00-06:00 2019-01-24T11:30:00-06:00 https://mwop.net/blog/2019-01-24-expressive-routes.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net In Expressive, we have standardized on a file named config/routes.php to contain all your route registrations. A typical file might look something like this: declare(strict_types=1); use Zend\Expressive\Csrf\CsrfMiddleware; use Zend\Expressive\Session\SessionMiddleware; return function ( \Zend\Expressive\Application $app, \Zend\Expressive\MiddlewareFactory $factory, \Psr\Container\ContainerInterface $container ) : void { $app->get('/', App\HomePageHandler::class, 'home'); $app->get('/contact', [ SessionMiddleware::class, CsrfMiddleware::class, App\Contact\ContactPageHandler::class ], 'contact'); $app->post('/contact', [ SessionMiddleware::class, CsrfMiddleware::class, App\Contact\ProcessContactRequestHandler::class ]); $app->get( '/contact/thank-you', App\Contact\ThankYouHandler::class, 'contact.done' ); $app->get( '/blog[/]', App\Blog\Handler\LandingPageHandler::class, 'blog' ); $app->get('/blog/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, App\Blog\Handler\BlogPostHandler::class, ], 'blog.post'); $app->post('/blog/comment/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, App\Blog\Handler\ProcessBlogCommentHandler::class, ], 'blog.comment'); } and so on. These files can get really long, and organizing them becomes imperative. Using Delegator Factories One way we have recommended to make these files simpler is to use delegator factories registered with the Zend\Expressive\Application class to add routes. That looks something like this: namespace App\Blog; use Psr\Container\ContainerInterface; use Zend\Expressive\Application; use Zend\Expressive\Csrf\CsrfMiddleware; use Zend\Expressive\Session\SessionMiddleware; class RoutesDelegator { public function __invoke( ContainerInterface $container, string $serviceName, callable $callback ) : Application { /** @var Application $app */ $app = $callback(); $app->get( '/blog[/]', App\Blog\Handler\LandingPageHandler::class, 'blog' ); $app->get('/blog/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, Handler\BlogPostHandler::class, ], 'blog.post'); $app->post('/blog/comment/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, Handler\ProcessBlogCommentHandler::class, ], 'blog.comment'); return $app; } } You would then register this as a delegator factory somewhere in your configuration: use App\Blog\RoutesDelegator; use Zend\Expressive\Application; return [ 'dependencies' => [ 'delegators' => [ Application::class => [ RoutesDelegator::class, ], ], ], ]; Delegator factories run after the service has been created for the first time, but before it has been returned by the container. They allow you to interact with the service before it's returned; you can configure it futher, add listeners, use it to configure other services, or even use them to replace the instance with an alternative. In this example, we're opting to configure the Application class further by registering routes with it. We've even written this approach up in our documentation. So far, so good. But it means discovering where routes are registered becomes more difficult. You now have to look in each of: config/routes.php Each file in config/autoload/: looking for delegators attached to the Application class, and then checking those to see if they register routes. In config/config.php to identify ConfigProvider classes, and then: looking for delegators attached to the Application class, and then checking those to see if they register routes. The larger your application gets, the more work this becomes. Your config/routes.php becomes way more readable, but it becomes far harder to find all your routes. One-off Functions In examining this problem for the upteenth time this week, I stumbled upon a solution that is initially acceptable to me, finally. What I've done is as follows: I've created a function in my ConfigProvider that accepts the Application instance and any other arguments I want to pass to it, and which registers routes with the instance. I call that function within my config/routes.php. Building on the example above, the ConfigProvider for the App\Blog module now has the following method: namespace App\Blog; use Zend\Expressive\Application; use Zend\Expressive\Csrf\CsrfMiddleware; use Zend\Expressive\Session\SessionMiddleware; class ConfigProvider { public function __invoke() : array { /* ... */ } public function registerRoutes( Application $app, string $basePath = '/blog' ) : void { $app->get( $basePath . '[/]', App\Blog\Handler\LandingPageHandler::class, 'blog' ); $app->get($basePath . '/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, Handler\BlogPostHandler::class, ], 'blog.post'); $app->post($basePath . '/comment/{id:[^/]+\.html', [ SessionMiddleware::class, CsrfMiddleware::class, Handler\ProcessBlogCommentHandler::class, ], 'blog.comment'); } } Within my config/routes.php, I can create a temporary instance and call the method: declare(strict_types=1); return function ( \Zend\Expressive\Application $app, \Zend\Expressive\MiddlewareFactory $factory, \Psr\Container\ContainerInterface $container ) : void { (new \App\Blog\ConfigProvider())->registerRoutes($app); } This approach eliminates the problems of using delegator factories: There's a clear indication that a given class method registers routes. I can then look directly at that method to determine what they are. One thing I like about this approach is that it allows me to keep the routes close to the code that handles them (i.e., within each module), while still giving me control over their registration at the application level. What strategies have you tried? Registering Module-Specific Routes in Expressive was originally published 24 January 2019 on https://mwop.net by Matthew Weier O'Phinney. 2018-12-05T16:26:00-06:00 2018-12-05T16:26:00-06:00 https://mwop.net/blog/2018-12-05-on-the-fly-exceptions.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net We pioneered a pattern for exception handling for Zend Framework back as we initially began development on version 2 around seven years ago. The pattern looks like this: We would create a marker ExceptionInterface for each package. We would extend SPL exceptions and implement the package marker interface when doing so. What this gave users was the ability to catch in three ways: They could catch the most specific exception type by class name. They could catch all package-level exceptions using the marker interface. The could catch general exceptions using the associated SPL type. So, as an example: try { $do->something(); } catch (MostSpecificException $e) { } catch (PackageLevelExceptionInterface $e) { } catch (\RuntimeException $e) { } This kind of granularity is really nice to work with. So nice that some standards produced by PHP-FIG now ship them, such as PSR-11, which ships a ContainerExceptionInterface and a NotFoundExceptionInterface. One thing we've started doing recently as we make packages support only PHP 7 versions is to have the marker ExceptionInterface extend the Throwable interface; this ensures that implementations must be able to be thrown! So, what happens when you're writing a one-off implementation of something that is expected to throw an exception matching one of these interfaces? Why, use an anonymous class, of course! As an example, I was writing up some documentation that illustrated a custom ContainerInterface implementation today, and realized I needed to throw an exception at one point, specifically a Psr\Container\NotFoundExceptionInterface. I wrote up the following snippet: $message = sprintf(/* ... */); throw new class($message) extends RuntimeException implements NotFoundExceptionInterface { }; Done! This works because RuntimeException takes a message as the first constructor argument; by extending that class, I gain that behavior. Since NotFoundExceptionInterface is a marker interface, I did not need to add any additional behavior, so this inline example works out-of-the-box. What else are you using anonymous classes for? Creating Exception types on-the-fly in modern PHP was originally published 5 December 2018 on https://mwop.net by Matthew Weier O'Phinney. 2018-10-17T11:30:00-05:00 2018-10-18T13:20:00-05:00 https://mwop.net/blog/2018-10-17-long-live-zf.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net For the past thirteen years, I've been either consuming Zend Framework or directly contributing to it. Since 2009, I've operated as project lead, and, since then, shepherded the version 2 and 3 releases, added Apigility to the ZF ecosystem, and helped bring middleware paradigms to the mainstream by assisting with the creation of Stratigility and coordination of the Expressive project. As I write this, the various ZF packages have been downloaded over 300 MILLION times, with 200 million of those being in the past 18 months! In the last three years, I have performed this work under the umbrella of Rogue Wave Software, who acquired Zend in 2015. However, Rogue Wave has recently made a strategic decision to focus its efforts on the Zend Server product of the Zend portfolio. Consequently, this means both myself and Enrico Zimuel will be leaving the company and looking for new opportunities in the not-too-distant future, along with Zeev Suraski and Dmitry Stogov. We all care deeply about the Zend Framework ecosystem, and we are evaluating options to ensure its continuation and longevity. These include either finding a new corporate sponsor for the project, or forming a foundation. This is where YOU come in: if you work for a company that would be interested in supporting such efforts, I would love to hear from you. Feel free to reach out to me at matthew@weierophinney.net with questions or queries. The Future of Zend Framework was originally published 17 October 2018 on https://mwop.net by Matthew Weier O'Phinney. 2018-10-16T16:00:00-05:00 2018-10-17T08:30:00-05:00 https://mwop.net/blog/2018-10-16-swoole.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net Have you used Node.js? For those of my readers unfamiliar with Node.js, it's a server-side JavaScript framework that provides the ability to create, among other things, network services. To do so, it provides an event loop, which allows for such things as asynchronous processing. In the PHP ecosystem, a group of Chinese developers have been creating an extension that provides many of the same capabilities as Node.js. This extension, called Swoole, allows you to create web servers with asynchronous capabilities. In many cases, the asynchronous capabilities are handled via coroutines, allowing you to write normal, synchronous code that still benefits from the asynchronous nature of the system event loop, allowing your server to continue responding to new requests as they come in! We've been gradually adding and refining our Swoole support in Expressive, and recently issued a stable release that will work with any PSR-15 request handler. In this post, I'll enumerate what I feel are the reasons for considering Swoole when deploying your PHP middleware application. I feel there are three key advantages to Swoole, and, by extension, any async PHP runtime: Application-specific servers Performance Async processing Application-specific servers There are a few general architectures for applications: A single web server sitting in front of many web applications. A single web server sitting in front of a single web application. A load balancer sitting in front of many servers. Some servers might serve the same application, to provide redundancy. (Often, today, these may even be identical docker containers.) The first scenario is common in internal networks and development, and in many shared hosting scenarios. It's generally considered less secure, however, as a vulnerability in one application can potentially escalate to affect all applications hosted on the server. Additionally, it means that any updates to PHP versions must be tested on all applications, which often means updates are few and far between — which is also problematic from a security standpoint. When you want to isolate the environment, you'll move to a single web server, single PHP application model: And when you start scaling, this becomes a load balancer sitting in front of many of these web server/PHP application pairs: In each of these last two scenarios, there's one thing I want to point out: your application consists of at least two distinct services: the PHP processes, and a web server. You may have other services as well, such as an RDBMS or document database, cache, search, etc. But generally these are on separate servers and scaled separately. As such, they're outside of this discussion. In these scenarios, this means each "server" is actually a composite. And when you are adding redundancy to your architecture, this adds significant complexity. It's one more process on each and every node that can fail, and additional configuration you need when deploying. When we start thinking about microservices, this becomes more problematic. Microservices should be quick and easy to deploy; one service per container is both typical and desired. What Swoole lets us do is remove one layer of that complexity. We can have a service per container, and that container can be built with only PHP. We start the Swoole HTTP server, and it's ready to go. We then tell the reverse proxy or load balancer how to route to it, and we're done. This is useful in each of the scenarios, including the one web server/mulitiple applications scenario, as we can have different PHP runtimes per application. Our "web server" becomes a reverse proxy instead. Application-specific servers allow us to simplify our deployment, and ship microservices quickly. Performance Remember when PHP 7 came out, and it was like doubling the performance of your application? What if you could do that again? In our initial benchmarks of Expressive applications, we found that they performed four times better under Swoole than under a traditional nginx+php-fpm pair. More interesting: when benchmarking with a high number of concurrent requests, we also found that Swoole had fewer failed requests. This means you get both better performance and better resilience! And the hits keep rolling in: when we enabled Swoole's coroutine support and benchmarked endpoints that made use of functionality backed by that coroutine support, we observed up to a ten-fold increase! The coroutine support covers primarily network I/O operations. As such, operations that hit cache servers, use PDO, or make web requests benefit from it immediately, with no changes to your code. Swoole makes this possible in a couple of ways. First, because you are firing up a server exactly once, you lose the price of bootstrapping your application that you normally incur on each and every request; your application is bootstrapped from the moment you start accepting requests. Bootstrapping often accounts for the greatest single amount of resource usage in your application. Second, Swoole runs as an event loop, just like Node.js, allowing it to defer processing of long-running requests in order to respond to new, incoming requests. This leads into my last point. Async processing Swoole's event loop provides async functionality to PHP applications. While a number of userland libraries have popped up over the past five years or so that provide async capabilities for PHP, Swoole's is done as a native C extension, and works regardless of the operating system. When you have an event loop, you can defer processing, which allows the server to respond to additional requests. Commonly, deferment can be explicit: public function handle(ServerRequestInterface $request) : ResponseInterface { $ts = new DateTimeImmutable(); \Swoole\Event::defer($this->createCacheDeferment($ts)); return new EmptyResponse(202); } public function createCacheDeferment(DateTimeImmutable $ts) : callable { return function () use ($ts) { sleep(5); $now = new DateTimeImmutable(); $item = $this->cache->getItem('ts'); $item->set(sprintf( "Started: %s\nEnded: %s", $ts->format('r'), $now->format('r') )); $this->cache->save($item); }; } In this example, we calculate the content to return, defer caching, and return a response immediately. This means your user does not need to wait for you to finish caching content. Logging is another use case. In the Expressive Swoole bindings, we do access logging after we mark the response complete. This ensures that logging does not impact response times. Another use case is webhooks. Your application can accept a payload immediately, but finish processing of it after sending the response back to the client. Swoole also provides async-enabled versions of common filesystem operations, Mysql, Redis, and an HTTP client. In each of these, you provide a callback indicating what should be done once the operation is complete: use Swoole\Http\Client as HttpClient; $client = new HttpClient('https://example.com'); $client->setHeaders([ 'Accept' => 'application/json', 'Authorization' => sprintf('Bearer %s', $token), ]); // Make the request, telling it what code to execute once // it is complete: $client->get('/api/resource', function ($response) { // process the response }); // This code executes before the request completes: $counter++; Code like the above has led to the term "callback hell" when you have many such deferments that depend on each other. So, what do you do if you want your code to be "non-blocking", but don't want to write callbacks all the time? Well, recent versions of Swoole allow you to enable coroutine support for most I/O operations. What this means is that you can write your code just like you would in a synchronous environment, but whenever code that triggers a coroutine occurs, the server will advance the event loop, allowing it to answer additional requests before the current one completes its work, and then resume execution once it has. // This spawns a coroutine: $statement = $pdo->query($sql); Async functionality may not directly improve the performance of your application, but it will let your application answer more requests, allowing you to handle greater volumes of traffic! zend-expressive-swoole We released zendframework/zend-expressive-swoole 1.0.0 two weeks ago. This library acts as a zend-httphandlerrunner RequestHandlerRunner implementation, which means: It can be used with any PSR-15 application. It can be used with any PSR-7 implementation. In other words, if you want to use Swoole with the upcoming Slim 4 or with equip/dispatch or with northwoods/broker or any of the myriad PSR-15 dispatch systems out there, you can. The library provides some interesting features for users: Serving of static resources, with HTTP client-side caching headers. Configurable logging. Abiility to restart worker processes. I've been running applications on versions of it for the past two months, and have noted that it has been stable and reliable. I definitely think it's worth giving it a spin! Fin I'm really excited about the possibilities of Swoole and other async systems, as I think they afford us better performance, better reliability, and the ability to defer functionality that doesn't need to complete before we respond to clients. I'd love to hear YOUR experiences, though, particularly in the form of blog posts! Send me a link to a blog post via a comment, or by tweeting at me, and I'll add it to the ZF newsletter. Updates 2018-10-17: Fixed typo in first sentence. Async Expressive with Swoole was originally published 16 October 2018 on https://mwop.net by Matthew Weier O'Phinney. 2018-07-18T17:05:00-05:00 2018-09-19T08:10:00-05:00 https://mwop.net/blog/2018-07-18-graphql.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net The last week has been my first foray into GraphQL, using the GitHub GraphQL API endpoints. I now have OpinionsTM. The promise is fantastic: query for everything you need, but nothing more. Get it all in one go. But the reality is somewhat... different. What I found was that you end up with a lot of garbage data structures that you then, on the client side, need to decipher and massage, unpacking edges, nodes, and whatnot. I ended up having to do almost a dozen array_column(), array_map(), and array_reduce() operations on the returned data to get a structure I can actually use. The final data I needed looked like this: [ { "name": "zendframework/zend-expressive", "tags": [ { "name": "3.0.2", "date": "2018-04-10" } ] } ] To fetch it, I needed a query like the following: query showOrganizationInfo( $organization:String! $cursor:String! ) { organization(login:$organization) { repositories(first: 100, after: $cursor) { pageInfo { startCursor hasNextPage endCursor } nodes { nameWithOwner tags:refs(refPrefix: "refs/tags/", first: 100, orderBy:{field:TAG_COMMIT_DATE, direction:DESC}) { edges { tag: node { name target { ... on Commit { pushedDate } ... on Tag { tagger { date } } } } } } } } } } Which gave me data like the following: { "data": { "organization": { "repositories: { "pageInfo": { "startCursor": "...", "hasNextPage": true, "endCursor": "..." }, "nodes": [ { "nameWithOwner": "zendframework/zend-expressive", "tags": { "edges": [ "tag": { "name": "3.0.2", "target": { "tagger": { "date": "2018-04-10" } } } ] } } ] } } } } How did I discover how to create the query? I'd like to say it was by reading the docs. I really would. But these gave me almost zero useful examples, particularly when it came to pagination, ordering results sets, or what those various "nodes" and "edges" bits were, or why they were necessary. (I eventually found the information, but it's still rather opaque as an end-user.) Additionally, see that pageInfo bit? This brings me to my next point: pagination sucks, particularly if it's not at the top-level. You can only fetch 100 items at a time from any given node in the GitHub GraphQL API, which means pagination. And I have yet to find a client that will detect pagination data in results and auto-follow them. Additionally, the "after" property had to be something valid... but there were no examples of what a valid value would be. I had to resort to StackOverflow to find an example, and I still don't understand why it works. I get why clients cannot unfurl pagination, as pagination data could appear anywhere in the query. However, it hit me hard, as I thought I had a complete set of data, only to discover around half of it was missing once I finally got the processing correct. If any items further down the tree also require pagination, you're in for some real headaches, as you then have to fetch paginated sets depth-first. So, while GraphQL promises fewer round trips and exactly the data you need, my experience so far is: I end up having to be very careful about structuring my queries, paying huge attention to pagination potential, and often sending multiple queries ANYWAYS. A well-documented REST API is often far easier to understand and work with immediately. I end up doing MORE work client-side to make the data I receive back USEFUL. This is because the payload structure is based on the query structure and the various permutations you need in order to get at the data you need. Again, a REST API usually has a single, well-documented payload, making consumption far easier. I'm sure I'm probably mis-using GraphQL, or missing a number of features to make this stuff easier, but so far, I'm left wishing I could just have a number of useful REST endpoints that I can hit consistently in order to aggregate the data I need. Before anybody suggests it, yes, I am very aware that GitHub also offers a REST API, and the v3 API has endpoints for most of what I needed. However, I had to rely on tags, not releases, as not all of our tags have associated releases. However, the data returned for tags does not include the commit date; for that, you need to fetch the associated commit, and then the date may be under either the author or the committer. This approach would have meant literally thousands of calls to get the data I need, which would have had me hitting rate limits, and potentially taking hours to complete. My point: perhaps instead of GraphQL, aggregating a bit more data in REST resources (e.g., including commit data with tags), or providing endpoints that allow merging specific resource types could have solved the problem easily. This is where having a developer relations team that finds out what data consumers are needing comes in handy, instead of simply mandating graphql all the things to allow infinite flexibility (and the frustrations of such flexibility, both for the API developer and consumer). Updates 2018-09-19: syntax highlighting fixes. Notes on GraphQL was originally published 18 July 2018 on https://mwop.net by Matthew Weier O'Phinney. 2018-01-23T14:05:00-06:00 2018-01-23T14:05:00-06:00 https://mwop.net/blog/2018-01-23-psr-15.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net Yesterday, following a unanimous vote from its Core Committee, PHP-FIG formally accepted the proposed PSR-15, HTTP Server Handlers standard. This new standard defines interfaces for request handlers and middleware. These have enormous potential impact on the PHP ecosystem, as they provide standard mechanisms for writing HTTP-facing, server-side applications. Essentially, they pave the way for developers to create re-usable web components that will work in any application that works with PSR-15 middleware or request handlers! Caveat I acted as sponsor on PSR-15, and as final arbiter of changes during the review period. Background PSR-15 was started by Woody Gilk, who has acted in the role of Editor for its duration. The original intent was to ratify a middleware standard, and it was initially thought that it would be a quick ratification of a pattern that was already in wide use: function ( ServerRequestInterface $request, ResponseInterface $response, callable $next ) : ResponseInterface where $next should implement the following signature: function ( ServerRequestInterface $request, ResponseInterface $response ) : ResponseInterface "Double Pass" The above pattern has been dubbed "double pass" middleware, for the fact that it passes two instances to the collaborator to pass to the next layer. However, a number of critiques of this existing practice started to arise almost immediately, with one from Anthony Ferrara holding particular weight. The primary problems noted were: Passing the response from layer to layer can lead to issues where an outer layer makes a change to the response it passes to an inner layer, expecting it to propagate back out, but an inner layer returns a different response entirely. Essentially, the pattern promotes problematic practices. If middleware needs to operate on a response, it should operate on the response returned by another layer. Typehinting $next as callable means there's no way to ensure that the callable is actually capable of accepting the arguments passed to it. In other words, it's not type safe. After debate within the working group, the next iteration proposed the following (some details differ, but basic interactions are the same): interface DelegateInterface { public function process(ServerRequestInterface $request) : ResponseInterface; } interface MiddlewareInterface { public function process( ServerRequestInterface $request, DelegateInterface $delegate ) : ResponseInterface; } This largely solved the problems highlighted above. However, a few more details came up as different teams developed implementations. First, many noted that they felt defining the same method name prevented polymorphism. A common use case was to define a "request handler" that could be called and which would in turn process itself. So, we updated the interfaces as follows: interface RequestHandlerInterface { public function handle(ServerRequestInterface $request) : ResponseInterface; } interface MiddlewareInterface { public function process( ServerRequestInterface $request, RequestHandlerInterface $handler ) : ResponseInterface; } Second, once that change was done, a number of others noted that the request handler could be useful in and of itself. For example, when creating a simple site, you could marshal a server request, pass it to a handler, and emit the response returned; middleware might not be necessary in this case. Another use case is for the final, internal end points of a middleware application: instead of implementing these as middleware, one could implement them as request handlers instead, as they do not operate on the results of the handler. As a result, we made the change to ship the two interfaces as separate packages, with the package containing the MiddlewareInterface depending on the package defining the RequestHandlerInterface. Finally, over the close to two years that this specification was being developed, PHP 7 gained in maturity, with 7.1 and 7.2 releases. We decided to pin the specification to PHP 7 or greater, and formally adopted return type hints within it. While work on the specification was in progress, each iteration of the interfaces was published within the github organization http-interop, with packages matching whatever the current specification detailed (http-middleware, then http-server-middleware, and, eventually, adding http-server-handler). These packages also used Interop\Http as the top-level namespace. Members of the working group, as well as other interested parties, would pin their offerings to specific iterations. The final packages are now owned by the PHP-FIG group, however, and use the Psr top-level namespace. The Interfaces That brings us to the final standard: PSR-15 PSR-15 Meta Document (which covers the whys behind the specification) psr/http-server-handler provides the following interface: namespace Psr\Http\Server; use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; interface RequestHandlerInterface { public function handle(ServerRequestInterface $request) : ResponseInterface; } psr/http-server-middleware provides the following interface: namespace Psr\Http\Server; use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; use Psr\Http\Server\RequestHandlerInterface; interface MiddlewareInterface { public function process( ServerRequestInterface $request, RequestHandlerInterface $handler ) : ResponseInterface; } Both packages depend on PSR-7 as they typehint against the HTTP message interfaces that package defines. The http-server-middleware package depends on the http-server-handler package. How to write re-usable middleware Most middleware dispatchers available currently (and there are a LOT of them, as it turns out!) allow you to compose middleware in such a way that none of it needs to know how or what is composing it. This is A Good Thing™. It allows you to write middleware that is de-coupled from the context in which it is used. But how do you do that? In the meta document for the specification, we suggest the following: Test the request for required pre-conditions, if any. If it does not satisfy any, use a composed response prototype or response factory to generate and return a response. If pre-conditions are met, delegate creation of the response to the provided handler, optionally providing a "new" request (PSR-7 requests are immutable, so this means calling one of its with*() methods, which return new instances). Either pass the response back from the handler verbatim, or return a new response by manipulating the one returned (again, via one of the with*() methods). The first point is probably the most important here: do not directly instantiate a response in your middleware, but instead use a prototype or a factory that is provided during instantiation. This allows you to de-couple your middleware from the PSR-7 implementation used by the application. In practice, that might look something like this: class CheckOriginMiddleware implements MiddlewareInterface { private $acceptedOrigins; private $responsePrototype; public function __construct(array $acceptedOrigins, ResponseInterface $responsePrototype) { $this->acceptedOrigins = $acceptedOrigins; $this->responsePrototype = $responsePrototype; } public function process(ServerRequestInterface $request, RequestHandlerInterface $handler) : ResponseInterface { $origin = $request->getHeaderLine('origin'); if (! in_array($origin, $this->acceptedOrigins, true)) { return $this->responsePrototype ->withStatus(401) ->withHeader('X-Invalid-Origin', $origin); } $response = $handler->handle($request); return $response->withHeader('X-Origin', $origin); } } A few things to note about this middleware: It accepts dependencies via its constructor. This practice allows us to easily test the middleware, and defines what it needs in order to do its work. The response prototype ensures we are de-coupled from the PSR-7 implementation. I can pass a Diactoros response, a Guzzle response, a Slim response, or any other implementation. As a result, consumers of this middleware will not need to potentially install another PSR-7 implementation. The middleware has no idea where it will be used, or what stack it will be used in. It simply operates on the request and the handler provided when process() is called. How might I consume such middleware? In Expressive), I might do any one of the following: // Pipe it as a service to pull from the DI container: $app->pipe(CheckOriginMiddleware::class); // Use it within a route-specific pipeline: $app->post('/api/foo', [ CheckOriginMiddleware::class, FooMiddleware::class, ]); In northwoods/broker (maintained by Woody Gilk, the PSR-15 editor), it looks like this: $broker->always([CheckOriginMiddleware::class]); In middlewares/utils Dispatcher, you'd do this: $dispatcher = new Dispatcher([ /* ... */ new CheckOriginMiddleware($acceptedOrigins, $responsePrototype), /* ... */ ]); With any one of these solutions, if your middleware is executed, it will act exactly the same; how it is composed doesn't matter, as the way it operates is only dependent on the request and handler passed to the middleware during process(). What about request handlers? Most of the libraries I've looked at at this time define handlers in one of two ways: As a middleware dispatcher. In this particular case, each middleware is processed until one returns a response. If the last one processed calls on the handler again, then a canned response is returned, an exception is thrown, or the next scenario comes into play: As a "final" handler to pass to the middleware dispatcher. In other words, if the last middleware processed also calls on its handler, this "fallback" or "final" handler will be what's invoked. This will typically return a 404 response or a 500 response, depending on the implementation. One other possibility that's been floated by several is for use with routing middleware. In this case, when routing middleware matches a request, it would then call on a request handler mapped to that request. Notes for implementors PSR-15 is accepted; let's make all the things PSR-15! But have some patience! While a number of projects have been working with various iterations of the http-interop packages, and may need some time to update to the final PSR-15 specification. For example, we've been tracking various iterations of http-interop in both Stratigility and Expressive, but updating to the PSR-15 specification requires backwards-incompatible changes, necessitating a new 3.0 version — which need a few more weeks to drop. Slim also has a patch submitted with PSR-15 support, which would likely not drop until an upcoming 4.0 release. As such, have patience with library and framework maintainers, and help test releases for them. Additionally, consider tracking and testing the proposed PSR-17 specification. This proposal will standardize PSR-7 factories, which will provide a standard way for middleware to generate, in particular, responses to return. Instead of composing a response prototype, you would compose a factory. Why is this easier? Well, in cases where you may also want to address the response body, which is a Psr\Http\Message\StreamInterface instance, it allows you to create new instances of those as well. Since streams cannot be immutable (due to language limitations), any time you write to a stream, you could be appending existing content, which means middleware that writes to the response prototype body generally needs to also compose a stream prototype. What if you could compose a single factory instead? Closing thoughts When I started work on PSR-7 originally, it was because I wanted a standard middleware interface for PHP. I'd been playing with Node, and, more specifically, Sencha Connect and ExpressJS. The middleware ecosystem in Node was and continues to be tremendous. The reason it exists is because of two factors: Accepted, standard middleware signature. Even though JS doesn't provide interfaces, and there is no userland standards body, a consensus signature emerged, and everyone used it. These were possible because of: Built-in HTTP message abstractions in the Node core library. If I wanted standard middleware in PHP, we first needed standard HTTP messages, which PSR-7 accomplished. That could have been the end of it, as many libraries started using the same middleware signatures; however, we soon had at least two, and possibly as many as a half-dozen different approaches. Thankfully, Woody stepped up to the task and proposed what became PSR-15; further, he, and the other members of the working group, had the patience and stamina to see it to acceptance (though I know there were several times he and others almost threw the towel in!). With PSR-15 accepted, we are a step closer to something I have long envisioned: a possibility for PHP developers to no longer work within monolithic MVC frameworks, but instead compose applications out of commodity, reusable middleware. PSR-15 was originally published 23 January 2018 on https://mwop.net by Matthew Weier O'Phinney. 2017-03-30T13:20:00-05:00 2017-03-30T13:20:00-05:00 https://mwop.net/blog/2017-03-30-anonymous-class-middleware.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net I faced an interesting question recently with regards to middleware: What happens when we go from a convention-based to a contract-based approach when programming? Convention-based approaches usually allow for duck-typing; with middleware, it means you can write PHP callables — usually closures — and just expect them to work. Contract-based approaches use interfaces. I think you can see where this is going. PSR-7 Middleware When PSR-7 was introduced, a number of middleware microframeworks adopted a common signature for middleware: use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; function ( ServerRequestInterface $request, ResponseInterface $response, callable $next ) : ResponseInterface where $next had the following signature: use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; function ( ServerRequestInterface $request, ResponseInterface $response ) : ResponseInterface This approach meant that you could wire middleware using closures, which makes for a nice, succinct, programmatic interface: // Examples are using zend-stratigility use Zend\Diactoros\Response\TextResponse; use Zend\Stratigility\MiddlewarePipe; $pipeline = new MiddlewarePipe(); $pipeline->pipe(function ($request, $response, callable $next) { $response = $next($request, $response); return $response->withHeader('X-ClacksOverhead', 'GNU Terry Pratchett'); }); $pipeline->pipe(function ($request, $response, callable $next) { return new TextResponse('Hello world!'); }); Easy-peasey! This convention-based approach was easy to write for, because there was no need to create discrete classes. You could, but it wasn't strictly necessary. Just throw any PHP callable at it, and profit. (I'll note that some libraries, such as Stratigility, codified at least the middleware via an interface as well, though implementation of the interface was strictly optional.) The big problem, however, is that it can lead to subtle errors: what happens if you expect more arguments than the middleware dispatcher provides? what happens if you expect different arguments and/or argument types than the middleware dispatcher provides? what happens if your middleware returns something unexpected? Essentially, a convention-based approach has no type safety, which can lead to a lot of subtle, unexpected, runtime errors. PSR-15 Middleware The proposed PSR-15 (HTTP Server Middleware) is not convention-based, and instead proposes two interfaces: namespace Interop\Http\ServerMiddleware; use Psr\Http\Message\ResponseInterface; use Psr\Http\Message\ServerRequestInterface; interface MiddlewareInterface { /** * Docblock annotations, because PHP 5.6 compatibility * * @return ResponseInterface */ public function process(ServerRequestInterface $request, DelegateInterface $delegate); } interface DelegateInterface { /** * Docblock annotations, because PHP 5.6 compatibility * * @return ResponseInterface */ public function process(ServerRequestInterface $request); } This leads to type safety: if you typehint on these interfaces (and, typically, for middleware dispatchers, you're only concerned with the MiddlewareInterface), you know that PHP will have your back with regards to invalid middleware. However, this also means that for any given middleware, you must create a class! Well, that makes things more difficult, doesn't it! Or does it? Anonymous classes Starting in PHP 7, we now have the ability to declare anonymous classes. These are similar to closures, which can be thought of as anonymous functions (though with quite a lot more semantics and functionality!), applied at the class level. Interestingly, anonymous classes in PHP allow for: Extension Interface implementation Trait composition In other words, they behave just like any standard class declaration. Let's adapt our previous pipeline to use PSR-15 instead. (We'll continue using Stratigility, as, since version 2, it supports the proposed PSR-15 specification.) use Interop\Http\ServerMiddleware\DelegateInterface; use Interop\Http\ServerMiddleware\MiddlewareInterface; use Psr\Http\Message\ServerRequestInterface; use Zend\Diactoros\Response\TextResponse; use Zend\Stratigility\MiddlewarePipe; $pipeline = new MiddlewarePipe(); $pipeline->pipe(new class implements MiddlewareInterface { public function process (ServerRequestInterface $request, DelegateInterface $delegate) { $response = $delegate->process($request); return $response->withHeader('X-ClacksOverhead', 'GNU Terry Pratchett'); } }); $pipeline->pipe(new class implements MiddlewareInterface { public function process(ServerRequestInterface $request, DelegateInterface $delegate) { return new TextResponse('Hello world!'); } }); While there's slightly more verbiage — what were essentially our anonymous functions previously are now wrapped in a class, adding a couple lines to each — the result is not terribly onerous, and gives us important type-safety. Our middleware runner no longer has to assume that any middleware piped to it is correctly defined, but can instead know, as it can enforce a typehint. The approach is also useful to IDEs, which can now properly typehint arguments, and let us know when the contract is being violated. What about closures? A closure in PHP allows you to close over or bind variables in the current scope to the anonymous function. As an example, if I want to create logging middleware, I might do the following: // Where $log is a PSR-3 logger: use Zend\Diactoros\Response\EmptyResponse; $pipeline->pipe(function ($request, $response, callable $next) use ($log) { try { $response = $next($request, $response); return $response; } catch (Throwable $e) { } $log->error(sprintf( '[%d] (%s) %s', $e->getCode(), get_class($e), $e->getMessage() ), ['exception' => $e]); return new EmptyResponse(500); }); How would I accomplish this with an anonymous class? Anonymous classes let you pass arguments during declaration that are then passed to the constructor. As such, you bind variables from the current scope into the class typically as class properties: // Where $log is a PSR-3 logger: use Interop\Http\ServerMiddleware\DelegateInterface; use Interop\Http\ServerMiddleware\MiddlewareInterface; use Psr\Http\Message\ServerRequestInterface; use Psr\Log\LoggerInterface; use Zend\Diactoros\Response\EmptyResponse; $pipeline->pipe(new class($log) implements MiddlewareInterface { private $log; public function __construct(LoggerInterface $log) { $this->log = $log; } public function process(ServerRequestInterface $request, DelegateInterface $delegate) { try { $response = $delegate->process($request); return $response; } catch (Throwable $e) { } $this->log->error(sprintf( '[%d] (%s) %s', $e->getCode(), get_class($e), $e->getMessage() ), ['exception' => $e]); return new EmptyResponse(500); } }); This approach gives you added type-safety: if $log is of a different type, you'll know when that middleware is created, as PHP will raise a fatal error. Another thing I like about this approach is it allows me to prototype classes before I write them formally. I can start seeing what the re-use possibilities are, what arguments I might need, and more. Because the syntax for anonymous classes is identical to declared classes, I can later extract it to a named class by simply cutting the definition and pasting it into a file of its own. So, don't let the PSR-15 interfaces stop you! Start using anonymous classes for your own middleware prototypes! Using Anonymous Classes to Write Middleware was originally published 30 March 2017 on https://mwop.net by Matthew Weier O'Phinney. 2017-01-26T13:50:00-06:00 2017-01-26T13:50:00-06:00 https://mwop.net/blog/2017-01-26-http-message-util.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net We all know the standard HTTP request methods and status codes, right? Or do we? We definitely know whether or not they should be integers or strings, and/or how string values should be normalized, right? And our IDEs can totally autocomplete them, right? Oh, that's not the case? Some time ago, a few folks floated the idea of creating a utility repository related to the PSR-7 psr/http-message package, but containing some useful bits such as constants for HTTP request methods and status codes. Six months ago, we released it... but didn't publicize it. I remembered that fact today while writing some unit tests that were utilizing the package, and thought I'd finally write it up. The package is fig/http-message-util, and is available via Composer and Packagist: $ composer require fig/http-message-util It provides two interfaces: Fig\Http\Message\RequestMethodInterface, containing constants for HTTP request method values. Fig\Http\Message\StatusCodeInterface, containing constants for HTTP status code values. The constants are prefixed with METHOD_ and STATUS_, respectively, and use the standard names as presented in the various IETF specifications that originally define them. As an example, I could write middleware that looks like this: use Fig\Http\Message\RequestMethodInterface as RequestMethod; use Fig\Http\Message\StatusCodeInterface as StatusCode; use Interop\Http\ServerMiddleware\DelegateInterface; use Interop\Http\ServerMiddleware\MiddlewareInterface; use Psr\Http\Message\ServerRequestInterface; use Zend\Diactoros\EmptyResponse; class RequestMethodNegotiation implements MiddlewareInterface { private $alwaysAllowed = [ RequestMethod::METHOD_HEAD, RequestMethod::METHOD_OPTIONS, ]; private $map; public function __construct(array $map) { $this->map = $map; } public function process(ServerRequestInterface $request, DelegateInterface $delegate) { $path = $request->getUri()->getPath(); if (! isset($this->map[$path])) { return $delegate->process($request); } $method = $request->getMethod(); if (in_array($method, $this->alwaysAllowed, true)) { // Always allowed return $delegate->process($request); } if (in_array($method, $this->map[$path], true)) { // In map; proceed return $delegate->process($request); } // Not allowed! return new EmptyResponse(StatusCode::STATUS_METHOD_NOT_ALLOWED, [ 'Allow' => implode(',', $this->map[$path]); ]); } } The things to notice in the above are: $alwaysAllowed uses the RequestMethodInterface constants in order to provide a list of always allowed HTTP methods; it doesn't use strings, which are prone to typos. When a dis-allowed method is encountered, we use a StatusCodeInterface constant to provide the status. This allows us to use code completion, but also signify the intent of the code. Integer values are great, but unless you have all the status codes memorized, it's often easy to forget what they mean. The other thing to notice is that I alias the interfaces to shorter names. We require interfaces to have the Interface suffix in FIG, but in situations like these, I don't particularly care that the constants are defined in an interface; I just want to consume them. This is one of the reasons PHP supports aliasing. If you're not already using this package, and use PSR-7 middleware, I highly recommend checking the package out! PSR-7 Request and Method Utilities was originally published 26 January 2017 on https://mwop.net by Matthew Weier O'Phinney. 2016-10-24T15:25:00-05:00 2016-10-24T15:25:00-05:00 https://mwop.net/blog/2016-10-24-watch-phpunit-with-node.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net I've been trying to automate everything this year. When working on OSS, this is usually as simple as setting up Travis CI; in some cases, even that becomes a little more involved, but remains possible. But that's continuous integration. What about continuous development? Continuous development? With continous integration, every time I push to a branch associated with a pull request or on the origin repository, a build is triggered. Which is great, because I can verify and validate that my code runs fine on all the target platforms. But I have to wait for the build to trigger and then run. Ideally, I should also be testing locally; I likely don't want to push anything upstream that will fail! So, I look in the contributing guidelines, and determine how to run coding standards checks and unit tests, and do those manually. Manually? Ugh. Too easy to forget, and too easy to lose track and make a ton of changes between runs, making breakage easier. I'd like to automate running these as part of my development process. I want continuous development cycles. Preparing your project The first step is preparing your project. I like to run my tests and CS checks using Composer, as that allows me to change what I'm using later, but also allows me to standardize invocation of the tools. I define the following scripts in my composer.json: "scripts": { "check": [ "@cs-check", "@test" ], "cs-check": "phpcs --colors", "cs-fix": "phpcbf --colors", "test": "phpunit --colors=always" } You may, of course, need to alter these to use the tools specific to your own project. The main thing is that you have a "check" target, which runs all the various QA tools. You don't need to do this. But I definitely recommend it. If you can simplify invocation for your users, and for your tools, automation is far easier. Using gulp Node has some great tools for watching the filesystem and reacting to it. Two of these are considered "build" or "workflow" tools: Grunt and Gulp. I've opted for Gulp here, as the setup is far simpler; that said, it's not difficult to do in Grunt, either. First, you'll need npm, which usually comes packaged with node, or yarn, a more recent addition to the node ecosystem. Once you have these, you can continue. Second, I installed a few dependencies: gulp is the actual taskrunner. It needs to be installed both globally, and locally. It includes the functionality for watching the filesystem. gulp-shell provides the ability to execute arbitrary command line tools. gulp-notify ties into your system's notifications abilities. Navigate to your project directory, and install these as follows: $ npm install -g gulp # this may require sudo, depending on your system $ npm install --dev gulp gulp-shell gulp-notify If you are using yarn: $ yarn global add gulp # this may require sudo, depending on your system $ yarn add --dev gulp gulp-shell gulp-notify Third, create the following gulpfile.js in your project: /* jshint: node: true */ var gulp = require('gulp'); var notify = require('gulp-notify'); var shell = require('gulp-shell'); var project = require('path').posix.basename(__dirname); gulp.task('default', ['watch']); gulp.task('php_check', function () { gulp.src('') .pipe(shell('composer check')) .on('error', notify.onError({ title: project + ' failures', message: project + ' CS checks and/or tests failed' })); }); gulp.task('watch', function () { gulp.watch( ['phpunit.xml.dist', 'phpcs.xml', 'src/**/*.php', 'test/**/*.php'], ['php_check'] ); }); What the above does is: Watch the filesystem for changes to any of: phpunit.xml.dist, which would indicate a change to the test runner behavior. phpcs.xml, which would indicate a change to the coding standards. PHP files found in either the src/ or test/ directories. On changes, run composer check. On errors running composer check, create a system notification. Using the gulp automation Once you've done the above, run the following within your project directory: $ gulp This will spawn a process that watches the filesystem; any time you save a change to any of the files listed, it will run composer check, which in turn runs your CS checks and unit tests. If either of these processes fails, it spawns a system notification, which will draw your attention to the fact that you've just done something wrong. (If no errors occur, no notification is created.) What this means is that I can spawn the process in a terminal that I hide, and then start editing in my favorite editor or IDE, and get notifications immediately when I break something. Making it reusable with node The above is nice, but do you really want to add this to every project? While it's a useful utility, different projects run things differently, and some may or may not be amenable to adding tooling just to support specific development workflows. So, I decided to take this a step further, and see if I could automate for the more generic use case. The result is an npm package, phly-php-qa-watch, which ships with the binary php-qa-watch. Install it as follows: $ npm install phly-php-qa-watch -g # via npm; may require sudo $ yarn global add phly-php-qa-watch # via yarn; may require sudo Once installed, you can run it in your project: $ php-qa-watch If you need to specify an alternate checker, or a different list of files, flags will provide this: -c|--check-command allows you to specify an alternate check command to use; it defaults to composer-check. -w|--watch-files allows you to specify a comma-separated list of files, directories, and globs to watch. It defaults to phpunit.xml.dist,phpcs.xml,src/**/*.php,test/**/*.php. So, as an example, I could run: $ php-qa-watch \ > -c "./vendor/bin/php-cs-fixer fix --dry-run && ./vendor/bin/phpunit" \ > -w ".php_cs,phpunit.xml.dist,phpunit.xml,lib/**/*.php,tests/**/*.php" The above would run the locally installed php-cs-fixer in dry-run mode and phpunit. Further, it would re-run if the php-cs-fixer configuration changes, the local or project PHPUnit configuration changes, or if PHP files in either the lib/ or tests/ directories change. This finally hits the sweet-spot for me with automation: it's literally the least effort I can muster in any given repository in order to automate my testing tools. Automate all the things It's terribly easy to get complacent and lazy when developing, particularly on open source where you may be investing your off hours, and want to economize your activity. This can lead to having patches rejected, or frustration at only discovering a completely avoidable build error days later, as you left your console immediately after issuing a pull request. For these reasons, I try and automate whenever possible, not just for continuous integration, but for my own development workflow. Note This post is derived from a talk I prepared recently on quality PHP packages, and details one facet of the "automation" section. I plan to blog on the topics covered, and release related tools, in the coming weeks. Automating PHPUnit with Node was originally published 24 October 2016 on https://mwop.net by Matthew Weier O'Phinney. 2016-08-17T12:15:00-05:00 2016-08-17T12:15:00-05:00 https://mwop.net/blog/2016-08-17-zf-composer-autoloading.html Matthew Weier O'Phinney contact@mwop.net https://mwop.net One aspect of Zend Framework 3, we paid particular focus on was leveraging the Composer ecosystem. We now provide a number of Composer plugins for handling things such as initial project installation, registering installed modules with the application, and more. It's the "more" I particularly want to talk about. Registering modules on install With ZF2, we were able to realize the ability to install third-party modules into existing applications, enabling a module ecosystem. The standard mantra for install has been: Install module: composer require some/module Register module with application: edit config/(application|modules).config.php and add Some\Module to the list of modules. This second item has been problematic: Easy to forget Easy to introduce a typo For the v3 release, we wanted to solve this if we could. We were able to do so via a Composer plugin, zend-component-installer. Module authors may add some metadata to their package now, like the following: "extra": { "zf": { "module": "Some\\Module" } } and, if the plugin is present in the user's application, on installation, it will register the package as a module with the application! (Moreover, if you later remove the package, it will remove it!) We also added rules to allow specifying a package as a component; in this case, the module is added to the top of the module list, to ensure that userland modules can override its settings. This ability to make a common task turn-key via Composer makes me happy. Autoloading your own modules via Composer Recently, while working on Apigility, a collaborator made a suggestion: "We recommend using Composer for autoloading, and yet Apigility creates modules that use the default module autoloading capabilities; couldn't we create a utility for enabling Composer autoloading of a generated module?" This turned out to be really easy to accomplish, and we ended up creating a new package, zfcampus/zf-composer-autoloading, to make it re-usable. Let's say you've created a new module in your ZF or Apigility application, named Blog. Chances are, you put a Module.php file at the module's root, and it either contains a Blog\Module class, or requires a classfile from your source tree that will. Let's setup autoloading: $ composer require --dev zfcampus/zf-composer-autoloading $ ./vendor/bin/autoload-module-via-composer Blog Done. The package ships with only a vendor binary, and that does the following: Adds an entry in your composer.json to autoload the module. Regenerates the Composer autoloading rules. It will autodetect the module type (PSR-0 or PSR-4) based on the detected directory structure, but also allows you to specify the type via a CLI flag. You can also tell it where your Composer binary is, if it's not on your path. Once you're done, if you defined a getAutoloaderConfig() method in your module, you can now remove it, as it's redundant! This tool will work with existing ZF2 and Apigility installs, of any version. Composer all the things! Using Composer to Autoload ZF Modules was originally published 17 August 2016 on https://mwop.net by Matthew Weier O'Phinney.