To recreate the conditions, you can try the following:

$date     = '2016-06-16';
$fromForm = DateTimeImmutable::createFromFormat('Y-m-d', $date);
$today    = new DateTimeImmutable('today');
echo $fromForm == $today ? 'Equal' : 'Not equal'; // outputs "Not equal"

What's happening? Well, if you were to echo the results of each of $fromForm->format('c') and $today->format('c'), the difference is clear: the $fromForm value includes the time when the instance was created, while $today has the time set to midnight.

So, how do you zero out the time when using createFromFormat()?

It turns out that one of the format characters you can use is the | operator. When you include this at the end of your format string, any fields not included in the format are zero'ed out:

$fromForm = DateTimeImmutable::createFromFormat('Y-m-d|');

Reference

• DateTimeImmutable::createFromFormat() Parameters

PHP DateTimeImmutable::createFromFormat Reset Character was originally published 16 April 2026 on https://mwop.net by Matthew Weier O'Phinney.

What was happening

There's a fantastic StackOverflow answer describing what happens, and it's great reading.

The long and the short of it is that the php_fastcgi directive looks for matching files on the disk, and in the case of paths that look like directories, an index.php under that directory. In other words, it cannot match arbitrary paths (aka pretty URLs), just files. Which doesn't work with basically any fraṁework-based application; as the author of that SO answer posits, the directive was likely written to target Wordpress. Additionally, this causes issues if the PHP-FPM web pool is using a different root directory than you're using in your Caddy server, as Caddy will look for the files in the root defined for the server, and return a 404 if it can't find it, without ever passing it on to PHP-FPM.

The solution

The solution is to use the standard reverse_proxy directive, and have it use the fastcgi transport. You wrap this in a handle directive so that you can also specify a different filesystem root to pass to PHP-FPM.

As an example, let's assume:

• I'm running PHP-FPM on port 9000 of the host "app".
• The filesystem root for my PHP application is in /var/www/app.example.org/public
• I want to route all requests to index.php in that root so that they are handled by the application.

I can write the Caddyfile as follows:

app.example.org {
    # Note that this is the filesystem root for the _server_
    root * /static/app.example.org

    file_server

    handle {
        # This is the filesystem root for the FPM pool
        root * /var/www/app.example.org/public
        rewrite index.php
        reverse_proxy app:9000 {
            transport fastcgi {
                split .php
                capture_stderr
            }
        }
    }
}

If the file is not found in the filesystem, it will try to pass it on to PHP-FPM. When it does, it rewrites to index.php in the PHP-FPM root (which is a different root than Caddy uses for its own file server!); PHP-FPM will use the matched path as the PATH_INFO, and your framework remains happy.

Handling PHP-FPM using Caddy was originally published 21 March 2025 on https://mwop.net by Matthew Weier O'Phinney.

The problem

For my art gallery, the raw SQL looks something like the following:

SELECT
    p.filename,
    p.description,
    p.created,
    array_agg(DISTINCT t.tag) as tags
FROM
    photos p
LEFT JOIN tags t ON p.filename = t.content_id
WHERE
    p.filename = :filename
    AND t.content_type = :content_type
GROUP BY
    p.filename

I started noticing an odd issue where, post insertion of an image into my gallery, I'd get no error, but the form would not appear to have processed. On further inspection, using Z-Ray from ZendHQ, I realized that the insertion was successful, but that the redirect to view the inserted image was returning a 404. I grabbed the executed SQL from Z-Ray for retrieving the image, and it returned no rows.

I started thinking about why this image wasn't posting, when others were, and realized there was one trivial difference: I wasn't including any hashtags in my description, which meant no tags.

Hopefully you can see where this is leading.

A LEFT JOIN normally will not cause the entire query to fail if it finds no matching rows on the joined table; that's the behavior of INNER JOIN. However, if you put a condition that is based on a joined table outside the join itself, it essentially acts like an INNER JOIN, as this is now a condition of the SELECT query.

Sure enough, when I removed the LEFT JOIN and the array_agg column, I got a hit.

"Obvious" solution: move the condition

As a commenter on this post noted, the immediate solution is to move the AND t.content_type = :content_type clause to the JOIN:

SELECT
    p.filename,
    p.description,
    p.created,
    array_agg(DISTINCT t.tag) as tags
FROM
    photos p
LEFT JOIN tags t ON p.filename = t.content_id AND t.content_type = :content_type
WHERE
    p.filename = :filename
GROUP BY
    p.filename

This does work, and requires no huge changes to the DBAL query builder; I just move the condition into the joinLeft(), and carry on.

Preferred solution: nested query

The solution I chose was to do a nested query, and to aggregate those results as an array. This makes it more clear when reading the query as to the intent: I want to select all distinct tags for this image and assign them as an array to a column. I'm using PostgreSQL, so the query looks like this:

SELECT
    p.filename,
    p.description,
    p.created,
    (SELECT ARRAY(
        SELECT DISTINCT tag FROM tags WHERE p.filename = content_id AND content_type = :content_type
        )) as tags
FROM
    photos p
WHERE
    p.filename = :filename
GROUP BY
    p.filename

With this approach, if no rows are returned from the tags table, an empty array is created; otherwise an array of the tag values that match is returned.

However, I didn't know how to create this query using the Doctrine DBAL query builder.

DBAL solution

When creating a SELECT using the DBAL query builder, you do something like this:

$select = $dbal->createQueryBuilder();
$select
    ->select(
        // one argument per column to select
    )
    ->from('table_name', 't') // alias the table

The arguments to select() are expected to be strings, and any given string can be an arbitrary SQL expression.

Creating the subselect is easy; you do it like any other query:

$tags = $dbal->createQueryBuilder();
$tags
    ->select('tag')
    ->distinct()
    ->from('tags')
    ->where('content_id = p.filename')
    ->andWhere('content_type = :content_type');

Now, how do I get that into a column string for a select?

The getSQL() method of a query builder will spit out the SQL sent. Moreover, it does not replace placeholders, so even if you set a bound parameter, it won't be injected into the generated SQL.

Knowing all this, I did the following:

$tags = $dbal->createQueryBuilder();
$tags
    ->select('tag')
    ->distinct()
    ->from('tags')
    ->where('content_id = p.filename')
    ->andWhere('content_type = :content_type');

$select = $dbal->createQueryBuilder();
$select
    ->select(
        'p.filename',
        'p.description',
        'p.created',
        sprintf('(SELECT ARRAY(%s)) as tags', $tags->getSQL()),
    )
    ->from('photos', 'p')
    ->where('p.filename = :filename')
    ->groupBy('p.filename')
    ->setParameter('filename', $filename, ParameterType::STRING)
    ->setParameter('content_type', 'photo', ParameterType::STRING);

(Where ParameterType is imported from the namespace Doctrine\DBAL.)

This approach worked immediately, and generated exactly the same result as the raw SQL I had tested.

Changelog

• 2025-03-07: clarified that the solution was targeting the DBAL query builder. DBAL can consume raw SQL as well, and does not require usage of the query builder.
• 2025-03-10: noted that a LEFT JOIN will still work, as long as the t.content_type = :content_type condition is moved from the SELECT to the LEFT JOIN.

SQL Nested Queries or Sub Queries with Doctrine DBAL was originally published 6 March 2025 on https://mwop.net by Matthew Weier O'Phinney.

• What's New in PHP 8.4: Features, Changes, and Deprecations
• A Guide to PHP 8.4 Property Hooks
• Asymmetric Visibility in PHP 8.4: What It Means for PHP Teams
• HTTP Verbs Changes in PHP 8.4

Roundup of PHP 8.4 Posts was originally published 4 December 2024 on https://mwop.net by Matthew Weier O'Phinney.

What was it? quite simply, the php_admin_value struct can be used to configure php.ini settings for the pool. This is a great alternative to also adding PHP configuration settings via php.ini (or an include file for php.ini), as it allows you to keep the settings specific to that pool. That way, if you must have multiple pools (e.g., to serve multiple applications from the same machine and/or same PHP version), you can still have separate configuration for each.

How does it work? In your pool configuration, add values to that struct:

php_admin_value[memory_limit] = 32M
php_admin_flag[error_reporting] = E_ALL & ~E_NOTICE & ~E_DEPRECATED
php_admin_flat[track_errors] = Off
; etc

With ZendPHP, we just launched some Ansible tooling, which operates on the assumption that you are deploying PHP-FPM — and as part of its operation, it creates a template for the FPM pool configuration, but not for the PHP SAPI. And this is fine! Because you can use the php_admin_value settings to configure the pool for the application you're deploying!

Looking forward to simplifying a few of my deployments with this!

Configuring PHP.INI settings in a PHP-FPM pool was originally published 27 August 2024 on https://mwop.net by Matthew Weier O'Phinney.

Besides packaging long-term support versions of PHP, we also are publishing a product called ZendHQ. This is a combination of a PHP extension, and an independent service that PHP instances communicate with to do things like monitoring and queue management.

It's this latter I want to talk about a bit here, as (a) I think it's a really excellent tool, and (b) in using it, I've found some interesting patterns for prepping it during deployment.

What does it do?

ZendHQ's JobQueue feature provides the ability to defer work, schedule it to process at a future date and time, and to schedule recurring work. Jobs themselves can be either command-line processes, or webhooks that JobQueue will call when the job runs.

Why would you use this over, say, a custom queue runner managed by supervisord, or a tool like Beanstalk, or cronjobs?

There's a few reasons:

• Queue management and insight. Most of these tools do not provide any way to inspect what jobs are queued, running, or complete, or even if they failed. You can add those features, but they're not built in.
• If you are using monitoring tools with PHP... queue workers used with these tools generally cannot be monitored. If I run my jobs as web jobs, these can run within the same cluster and communicate to the same ZendHQ instance, giving me monitoring and code traces for free.
• Speaking of using web workers, this means I can also re-use technologies that are stable and provide worker management that I already know: php-fpm and mod_php. This is less to learn, and something I already have running.
• Retries. JobQueue allows you to configure the ability to retry a job, and how long to wait between retries. A lot of jobs, particularly if they rely on other web services, will have transient failures, and being able to retry can make them far more reliable.

So, what about queue warmup?

When using recurring jobs, you'll (a) want to ensure your queue is defined, and (b) define any recurring jobs at application deployment. You don't want to be checking on each and every request to see if the queues are present, or if the recurring jobs are present. Ideally, this should only happen on application deployment.

When deploying my applications, I generally have some startup scripts I fire off. Assuming that the PHP CLI is configured with the ZendHQ extension and can reach the ZendHQ instance, these scripts can (a) check for and create queues, and (b) check for and create recurring jobs.

As a quick example:

use ZendHQ\JobQueue\HTTPJob;
use ZendHQ\JobQueue\JobOptions;
use ZendHQ\JobQueue\JobQueue;
use ZendHQ\JobQueue\Queue;
use ZendHQ\JobQueue\QueueDefinition;
use ZendHQ\JobQueue\RecurringSchedule;

$jq = new ZendHQ\JobQueue();

// Lazily create the queue "mastodon"
$queue = $jq->hasQueue('mastodon')
    ? $jq->getQueue('mastodon')
    ? $jq->addQueue('mastodon', new QueueDefinition(
        QueueDefinition::PRIORITY_NORMAL,
        new JobOptions(
            JobOptions::PRIORITY_NORMAL,
            60, // timeout
            3, // allowed retries
            30, // retry wait time
            JobOptions::PERSIST_OUTPUT_ERROR,
            false, // validate SSL
    ));

// Look for jobs named "timeline"
$jobs = $queue->getJobsByName('timeline');
if (count($jobs) === 0) {
    // Job does not exist; create it
    $job = new HTTPJob('http://worker/mastodon/timeline', HTTPJob::HTTP_METHOD_POST);
    $job->setName('timeline');
    $job->addHeader('Content-Type', 'application/my-site-jq+json');
    $job->setRawBody(json_encode([
        'type' => MyApp\Mastodon\Timeline::class,
        'data' => [ /* ... */ ],
    ]);

    // Schedule to run every 15 minutes
    $queue->scheduleJob($job, new RecurringSchedule('* */15 * * * *'));
}

That's literally it.

The takeaway points:

• You can check for an existing queue and use it, and only define it if it's not there. You could also decide to suspend the queue and delete it before creating it, if you know that the existing jobs will not run with the current deployment.
• If you give a job a name (you don't actually have to, but it helps you identify related jobs far easier if you do), you can search for it. In the example above, if I find any jobs with that name, I know it's already setup, and I can skip the step of scheduling the job.

Running one-off jobs on deployment

Something else I also like to do is run one-off tasks at deployment. Often these are related to recurring tasks, and I might want to fetch the content at initialization rather than waiting for the schedule. In other cases, I might want to do things like reset caches.

Because these scripts run before deployment, which might mean restarting the web server, or, more often, waiting for the php-fpm container and/or web server to be healthy, I cannot run the jobs immediately, because there's nothing to answer them.

The answer to this is to queue a job in the future:

use DateTimeImmutable;
use ZendHQ\JobQueue\ScheduledTime;

$queue->scheduleJob($job, new ScheduledTime(new DateTimeImmutable('+1 minute')));

(I find it usually takes less than a minute for my FPM pool and/or web server to be online after running these scripts.)

The beauty of this approach is that my bootstrapping scripts now tend to be very fast, as I'm not trying to do all of this stuff before launching the site updates. The jobs then execute very soon after the site is up, and there's no noticeable differences in content or behavior.

Closing notes

I know I'm biased around ZendHQ. I'm also generally one of my own biggest critics. I had my team re-implement a lot of features that were present in Zend Server that I was never terribly keen on, and was hugely worried that we were going to make some of the same mistakes I felt we'd made with that product. However, the end result has been something that I am delighted to use, and which has opened up a ton of possibilities for how I build sites. The ability to warm my queues and manage them just like the rest of my PHP application is hugely powerful. I'm looking forward to seeing what others build with it!

Initializing ZendHQ JobQueue During Application Deployment was originally published 7 May 2024 on https://mwop.net by Matthew Weier O'Phinney.

Request

Please, please, please do not take this as an attack on Laravel or on Joel. I have nothing but respect for Joel, and while I'm not a fan of Laravel, I'm also not a hater. It's never a bad thing to have a popular framework that brings folks to a language; Laravel has done that in spades for PHP.

Summarize the article, already...

In the article, Joel notes the problem with providing return types in a Laravel controller is due to the fact that it could return a view, a JSON response, an array, a redirect, or more. If there are multiple types that could be returned, based on the request context, you would need to provide a union type. And if you refactor or make changes to the controller later that result in new types being returned, you now need to remember to change the return type declaration.

In other words, it introduces brittleness.

So what?

I've worked on multiple iterations of a major MVC framework, and I ran into these same issues. As PHP's type system got incrementally better, the cracks in how frameworks interact with controllers became more evident. Personally, I find the increasing number of type capabilities in PHP to be a huge boon in helping the correctness of applications, and preventing whole classes of errors. But if the framework prevents you from using the type system, or makes adding type declarations into a situation that can now introduce errors, it puts the developer and maintainer of an application into a problematic situation.

What are the alternatives?

I worked for quite some time on PSR-7 HTTP Message Interfaces, largely so that we could have a proper HTTP message abstraction in PHP on which to build a better foundation for applications and frameworks. From this emerged PSR-15 HTTP Server Request Handlers (which I sponsored and collaborated on, but was not primary author of).

What I love about PSR-15 is that there is no ambiguity about what you return from middleware or a handler. You return a response. That's all you can return.

This means there's no magic about different return values resulting in different behavior from the framework. You don't need to keep a mental map about what will happen, or do a deep dive into the framework internals to understand the ramifications of returning a view versus an array.

Instead, your handler will create a response, and provide the logic for how that is done. If you need HTML, you render a template, and feed it to the response. If you need JSON, you serialize data to JSON, and feed it to the response. If you need a redirect, you create a response with the appropriate status code and Location header. And so on and on.

Yes, this can lead to a little extra code at times, but:

• You can see exactly what you intend to return to the user, and why.
• If you try and return anything but a response, it'll result in a TypeError.
• You can test all of the different possible returns easily, by doing assertions on the returned response based on different requests provided to the handler or middleware.

But should you do everything in a handler? What about things that will happen for whole sections of the site, or will be repeated in many locations, like initializing a session, or checking for an authenticated user, or validating headers, or caching?

For those things, PSR-15 provides middleware. These are expected to be chained together, like a pipeline or a command bus, and the request is passed down through them, and a response returned on the way back up. They're a powerful way to provide re-usable pieces of functionality to your application.

What's more, using middleware is often far easier to understand than how and when various events will intercept a request. You can see the list of middleware for a given handler, and understand that they act either as filters on the incoming request (authentication, caching, etc.), or as decorators on the response (e.g. encoding or compressing the response, caching, etc.). Since each does exactly one thing (ideally), you can test how each works, and understand how and when to compose each, and how they might work in combination.

Building complex behavior via piping one thing to another is hugely powerful. There's a reason that the Unix Philosophy has existed as long as it has, and I can appreciate an approach to web development that builds on it.

Advent 2023: PSR-15 was originally published 14 December 2023 on https://mwop.net by Matthew Weier O'Phinney.

In the last couple of years, though, we came to the realization in the Laminas Project that we didn't really have anybody with the expertise or time to maintain it. We've marked it security-only twice now, and while we've managed to keep it updated to each new PHP version, it's becoming harder and harder, and whenever there's a CI issue, it's anybody's guess as to whether or not we'll be able to get it resolved.

My alternatives have been straight PDO, or Doctrine DBAL, with the latter being my preference.

Doctrine what?

When most folks who use PHP hear "Doctrine", they immediately think "ORM"; it's how most folks use it, and what it's best known for.

Underlying the ORM is its database abstraction layer (hence "DBAL"). This library exposes an API that will work across any database it supports; this is essentially what zend-db, and later laminas-db, were doing as well. What most folks don't realize is that you can use the DBAL by itself, without the ORM.

Why no ORM?

ORMs are fine. Really. But they add an additional layer of complexity to understanding what you are actually doing. Additionally, if you want to do something that doesn't quite fit how the ORM works, you'll need to drop down to the DBAL anyways. So my take has always been: why not just use the DBAL from the beginning?

So, how does Matthew write code that interacts with the database?

I start by writing value objects that represent discrete aspects of the application. Most of my work will be in consuming or creating these. From there, I write a repository class that I use for purposes of persisting and retrieving them. I can usually extract an interface from this, which aids in my testing, or if I decide I need a different approach to persistence later.

I push the work of mapping the data from the database to these objects, and vice versa, either in the repository, or in the value objects themselves (often via a named constructor). Using these approaches creates lean code that can be easily tested, and for which there's no real need to understand the underlying system; it's all right there in what I've written for the application.

Some gripes about the documentation, and some tips

The Doctrine DBAL docs are a bit sparse, particularly when it comes to its SQL abstraction. And there's no "getting started" or "basic usage" guide. In fact, it's not until the third page within the docs that you get any code examples; thankfully, at that point they give you information on how to get a database connection:

use Doctrine\DBAL\DriverManager;

$connectionParams = [
    'dbname'   => 'mydb',
    'user'     => 'user',
    'password' => 'secret',
    'host'     => 'localhost',
    'driver'   => 'pdo_mysql',
];
$conn = DriverManager::getConnection($connectionParams);

They also provide a number of other approaches, including using a DSN (an acronym they never explain, but based on using PDO, likely means "data source name").

Once you have a connection, what do you do? Well the DBAL connection allows you to prepare and execute queries, including via the use of prepared statements. It provides a variety of methods for fetching individual or multiple rows, with a variety of options for how the data is returned (indexed arrays, associative arrays, individual columns, individual values, etc.). These retrieval methods are mirrored in the result instances returned when executing prepared statements as well.

And that brings me to the SQL abstraction.

First, it's really, really good. It's minimal, but it covers just about anything you need to do. If you need to write something complex, you probably can; the beauty is that if you can't, you can always fall back to a SQL query, and using the connection's API for binding values.

But the documentation could be better.

It felt like it was written by a database admin who has forgotten more than most people ever learn about databases, and never considered that others might not know as much as them. The fact that it starts with architecture and not usage feels hugely antagonistic for somebody coming in just wanting to know how to connect to the database, build a query, and fetch some results. (The irony is not lost on me that this is almost exactly how Laminas and Mezzio docs are written, and, yes, I recognize we could all do better!)

Before folks start grousing, yes, I have on my TODO list an item for contributing to the DBAL docs. I'm trying to work up an outline of what I would have found useful, what acronyms need explanation, and some examples of common patterns before I make any suggestions, however.

First, they have a whole documentation page related to the SQL query builder, and a lot of examples. But not a single one details how to actually execute the query! So, for those wondering:

$sql = $conn->createQueryBuilder();

// ... build your query ...

// Execute a query that will retrieve results (generally SELECT queries):
$result = $sql->executeQuery();

// Execute a query that produces changes (INSERT, UPDATE, DELETE, etc.):
$count = $sql->executeStatement();

Query results have a variety of fetch*() operations on them, while executing a statement returns an integer indicating the number of rows affected (assuming the database supports this).

Second, when I started doing joins, the argument names were confusing, and made it harder to understand what was needed. I eventually figured it out, but it was really easy to flip the arguments for the different tables being joined. The usage below illustrates names that would better describe how to use it:

$sql->innerJoin(
    $primaryTableOrItsAliasIfYouSpecifiedOne, // e.g. "user" or "u"
    $newTableToJoin,                          // e.g. "address"
    $aliasForNewTableToJoin,                  // e.g. "a"
    $conditionToJoinOn                        // e.g. "u.id = a.uid"
);

Third, there's some odd differences in the API between INSERT and UPDATE operations., When setting a value, one takes setValue(), while the other takes set(), and only one of these is valid for a given operation (it's setValue() for INSERT operations, and set() for UPDATE operations, in case you were wondering). This is especially confusing when using bound parameters, because both can use the setParameter() method for binding positional placeholder values.

Speaking of plaeholders, the docs don't do a great job of detailing how to handle placeholders gracefully.

The documentation suggests patterns like this:

$queryBuilder
    ->select('id', 'name')
    ->from('users')
    ->where('email = ?')
    ->setParameter(0, $userInputEmail);

Which is fine when there's only one parameterized value, but what if you have several, or if you're dynamically building the query (e.g., looping through user-supplied sorting or criteria, etc.), and you don't know their exact position in the final query? And what if you want to use named parameters instead of positional parameters, but you're not sure if your database supports them?

The answer is in the docs, but the various examples don't use the pattern (other than in the discussion of the methods), which is infuriating. The above can also be written as follows:

$queryBuilder
    ->select('id', 'name')
    ->from('users')
    ->where('email = ' . $queryBuilder->createNamedParameter($userInputEmail));

There's also a createPositionalParameter() method. Both accept an optional second argument, where you can specify the value type, which can help ensure that values are quoted correctly for the SQL type they will map to. This also allows you to do IN() operations, and each value will be quoted correctly, with the appropriate list separator for the database.

Once you know this approach, it's easy to remember and use, but it took me a few times through the docs before I stumbled across it.

The SQL it generates, though, is great, and when I've used tools like ZendHQ's Z-Ray to introspect queries, I'm always impressed by what was actually sent over the wire.

2023-12-11 Update

Alexander Kim pointed out to me that you can use named parameters within the query builder, along with the setParameter() method. That usage looks like this:

use Doctrine\DBAL\ParameterType;

$queryBuilder
    ->select('id', 'name')
    ->from('users')
    ->where('email = :email')
    ->setParameter('email', $userInputEmail, ParameterType::STRING);

You can also specify named parameters when using set() and setValue(), though I'd argue that using createNamedParameter() is easier in those contexts.

But for all these issues, the fact is that the docs generally give you enough, and the API is so clean and reasonably documented that you can generally figure out how things work just from your IDE hints and autocompletion. Yes, I have gripes, but the library is very solid, very well written, and absolutely something I can depend on.

Final Thoughts

I've often used straight PDO for projects, and it works fine. However, having a tool available like Doctrine DBAL has been a huge boon in ensuring I can switch from SQLite while prototyping to MySQL for production, and know that things will "just work".

I also find the way it juggles types to be really useful. I know that if a value is typed in the database as a NULL or as text or as a float or integer, I'll actually get those types back when I query; the same is true for when I send data to the database. There's no magic involved, and I don't have to remember to do type conversions to and from the database. That's exactly the type of functionality I want from a DBAL.

Yes, writing database-centric code is cumbersome, and there's a reason folks use ORMs, ActiveRecord, and the like. However, it generally only needs to be written once, with occasional updates. Having a good DBAL available helps keep complexity of your application down and gives you the tools to communicate securely with your database.

Advent 2023: Doctrine DBAL was originally published 10 December 2023 on https://mwop.net by Matthew Weier O'Phinney.

• Ability to normalize values
• Ability to validate values
• Ability to get validation error messages
• Ability to render HTML forms, and have customizable markup
• Ability to do nested values
• Ability to handle optional values
• Ability to report missing values

and quite a lot more. But those are some of the things that stuck out that I can remember off the top of my head.

Zend_Form was considered a big enough new feature that we actually bumped the version from 1.0 to 1.5 to call it out.

And, honestly, in hindsight, it was a mistake.

A mistake?

Considering the timeframe when I was developing Zend_Form, it was actually a good effort, and it's still one of those features that folks tell me sold them on the framework. But within a year or two, I was able to see some of the drawbacks.

I first realized the issues when we started integrating the Dojo Toolkit with ZF. We ended up having to create first a suite of Dojo-specific form elements, and second a whole bnch of Dojo-specific decorators, which were what we used to render form elements. While the library gave us this flexibility, I saw a few issues:

• Duplication. We had multiple versions of the same form elements, and it was actually possible to get the wrong version for your form context. And with duplication comes increased maintenance: any time we fixed an issue in one element, we had to check to see if the same issue existed with the Dojo versions, and fix them there as well.
• Javascript. One of the reasons for integrating Dojo was to allow doing fun things like client-side validation; this allowed giving early feedback, without a round-trip to the server. But this also meant that we had validation logic duplicated between the server-side and client-side logic. And more interestingly: the form might be sent as a request by javascript, instead of a standard form request, which meant that we needed to validate it only, and then serialize validation status and messages. Basically, all the rendering aspects of the form were irrelevant in this scenario. Which brings me to...
• APIs. Around this time, APIs started trending. It would be a few years before REST became popular and commonly understood by developers, but folks were starting to see that we'd be needing them for the nascent mobile application markets, and that they were going to be a useful way to conduct business-to-business transactions. Once you start having APIs in the mix, a library centered on web forms becomes less interesting.

By the time we started planning for version 2 of ZF, we realized we'd need to reconsider how we did forms. The first step we took was splitting the validation aspect from the form aspect, and created Zend\InputFilter to address the first, and Zend\Form to address the second. Input filters encapsulated how to filter, normalize, and validate incoming data. Forms composed an input filter, and then provided hints for the view layer to allow rendering the elements. This separation helped a fair bit: you could re-use input filters for handling API or JS requests easily, while the form layer helped with rendering HTML forms.

But I still feel we didn't get it right:

• Our validation component and our input filter component were each stateful. When you performed validation, each would store the values, validation status, and validation messages as part of the state. This makes re-use within the same request more difficult (it was not uncommon to use the same validator with multiple elements, and this now required multiple instances), makes testing more difficult, and makes it harder to understand if the instance represents the definition, or the results of validation.
• The longer I've worked in web development, the more I've realized that while the HTML generation aspects of these form libraries are useful for prototyping, they inevitably cannot be used for the final production code. Designers, user experience experts, and accessibility developers will each want different features represented, and these will never fall into the defaults the framework provides. Even if the framework provides customization features, the end result is more programming effort. It's almost always better to code the HTML markup in your templates, and then feed state (e.g., element IDs/names, validation state, whether or not to display placeholders and/or error messages, etc.) from some object representing form or element state.

A few years back, I started an RFC for Laminas to create an idempotent validation library, one that would not even consider web form integration, but never quite hit on a good design. What with my work role changing, and having more and more varied interests outside work, I essentially abandoned it.

Uh oh, I did it again

Until recently.

I develop a number of internal tools for work to support some of the different functional teams with whom I work. These often require validation at some point, with varying amounts of complexity. As such, I've used these tools as a way for me to play with some of these ideas around validation and forms.

In developing the last couple of tools, I found a pattern that was working. I decided to extract it, and then iterated on it some more. Each iteration, I'd update one of these applications to see how it worked, what it enabled, and what was getting in the way.

I came up with a few goals:

• Provide an idempotent way to validate individual items and/or data sets.
• Provide an extensible framework for developing validation rules.
• Allow handling optional data, with default values.
• Allow reporting validation error messages.
• Ensure missing required values are reported as validation failures.
• Use as few dependencies as possible.

I also came up with some explicit non-goals:

• Creating an extensive set of validation rule classes.
• Providing extensive mechanisms for validating and returning nested data sets.
• Providing a configuration-driven mechanism for creating rule sets.
• Providing HTML form input representations or all metadata required to create HTML form input representations.

What I wanted was something that could validate an incoming data set, return a validation result, and then use that result to report back to the user. In the case of an API, for an invalid result, I'd be able to get the validation error messages, which could then be used to seed a Problem Details for HTTP APIs message. In the case of a web form, I'd be able to extract values, validation status, and validation error messages.

One thing I realized early on was that it was also useful to be able to represent a form's initial state. This would allow using the same template for both the initial form, as well as reporting form validation errors later.

Finally, I wanted a solution that reported types and would play nicely with static analysis. If I'm pulling a result out of a result set, I want to know that the value type is what I expect it to be. This helps with testing, provides IDE hinting, and helps ensure I'm using the features correctly. I think I ended up spending more time on this aspect than anything.

The result is my phly/phly-rule-validation library. I developed it for PHP 8.2 and up, as I wanted to use some specific features (though the ones specific to 8.2 and up... I ended up having to remove, so it would likely work on 8.0 or 8.1 as well). It's a little over 600 lines of code in total, and has no additional dependencies. It's also incredibly sparse; I only include 2 default validation rules.

The basic idea is:

• You create a rule set, consisting of rules.
• A rule defines:
  • The key it maps to in the data set being validated.
  • A method for validating a value, which produces a result.
  • A way to produce results for each of a default value, and when the value is missing.
• Rule validation produces a result, which composes:
  • The key associated with the result.
  • The value associated with the result. The validation routine can normalize the result if desired, so this value might not be 1:1 with what was submitted. This approach allowed me to not require splitting filtering/normalization from validation, as it becomes an implementation detail.
  • The validation state: is it valid, or not?
  • The validation message: this will generally only be populated for invalid values, and representes a validation error message.
• A rule set produces a result set, which is a collection of results.

In all cases, there are static analysis templates provided to allow defining the types. A validation result allows defining the value type, and a result set allows mapping keys to specific result types. Rules return result types. And so on.

A rule set can produce a valid result set, and this can be used to seed the initial state of a form. And I built support for nested results, which allows having forms that have groups of data.

The library provides usage examples, and I wrote quite a bit of documentation, if you want to see how it works.

Some thoughts

Is the result perfect? Probably not. I know that folks used to things like ZF, Laminas, Symfony, or Laravel forms will likely dislike the approach, as it does not allow for quick prototyping of web forms. I don't find that to be a detriment, however; as I noted earlier, the final production version of a form is likely going to be created by a designer, and won't work well with the HTML generation aspects of these systems anyways. For folks who only want to validate API payloads, while it will be a nice, lightweight approach, it doesn't provide a lot of defaults. Again, that's by design, as it allows developers to customize their validation logic and, more importantly, test it independently.

I've updated some of my applications to use this library. In some cases, I had a net reduction of code. In others, I ended up with more, but a far clearer understanding of what's in a form, how each item is validated, and what types are expected. And since the bulk of phly-rule-validation is around interfaces, it means that I'm not concerned about how the library works; it's pretty clear how it will work just from viewing the classes I've created.

One benefit of creating the library is that it helped me better understand Psalm and type templates. There are definitely limitations, and some things produce WTF moments, but when it all comes together, it's kind of magical. In some forms I built, it was amazing to be in a view template and get completion for everything, along with an understanding of what various types were, and warnings when I was doing an operation that couldn't use the type for a given element.

And these are the reasons I developed the library. I wanted something explicit, idempotent, and static analysis friendly, as these would make testing and IDE integration more straight-forward. I think I succeeded in that goal.

Advent 2023: Forms was originally published 9 December 2023 on https://mwop.net by Matthew Weier O'Phinney.

tl;dr: I'm leaving Twitter. You can find me in the Fediverse as @matthew@mwop.net.

In the beginning

I started using Twitter because of ZendCon 2007. Cal Evans had the idea that if folks attending the conference were to tweet about it, those who were unable to attend would get an idea of what the conference was about, get links to slides if speakers posted them, and more; it would both feed FOMO, and respond to it. (It also became an unofficial way for many of us to organize non-conference events during the evenings.)

Once the conference was done, I wasn't quite sure what to do with it. There was a bit of engagement, but not a ton. Hash tags, replies, retweets, quote tweets — none of these existed yet. Hell, even direct messages were just a specially formatted tweet, and heaven forbid you get the initial character sequence wrong! We started creating conventions, many of which later became codified into Twitter itself.

Over the next year or two, I found it became my "virtual watercooler." Being somebody who worked remotely, from home, I didn't have office conversations. A few of my colleagues and collaborators were on IRC, but back then, that was about it. If I wanted to talk to a larger group, or somebody not in my regular channels... Twitter became that place.

I made friends. I got job offers. I learned about places to visit on my travels. When abroad, I could coordinate meet-ups with friends.

When I realized folks couldn't spell my handle, I reached out on Twitter to see if I knew somebody at Twitter, or if somebody had a friend at Twitter, to see if I could change my handle, as somebody was squatting on "mwop". A friend of a friend made it happen — and I made a new friend in the process.

That was the honeymoon period, it seems.

The start of the fall

Sometime in the early 2010s, I began seeing the ugly side of Twitter. You know the folks, the ones who slide into your mentions or DMs when you post an opinion, the ones who ask for receipts and links or push whatabout-isms nonstop until you give in or stop replying (which they also take as victory). The ones who treat your lived experience as invalid, because it does not match theirs. The ones who cannot even imagine a valid experience outside their own. The ones who would not even allow another person's beliefs, body, heritage, circumstances to exist if they had their way.

Before muting and blocking existed on Twitter, the service was quickly becoming somewhere I did not want to engage. Somewhere I only felt comfortable posting non-revealing content about things like my open source projects, or retweeting work-related content. (I haven't posted anything about my family in years.) When Twitter allowed you to limit DMs to people you mutually followed, that helped a bit. But even then, I'd get folks in my mentions arguing or trolling; I cannot tell you how many times I was told the projects I worked on were crap, should die in a fire, that I should be embarrassed to even share them, that I should quit and get a different job, preferably in a different field. And this is only a fraction of what I see in the replies to women, people of color, LGBTQ+, people with accessibility issues — where the very act of existing as who they are is evidently an egregious offence. It's easy to see why so many leave the service, even though it can be hugely powerful at connecting you to others in your chosen community.

With muting and blocking, the service became more bearable, but only barely. I'd still get the tweets, replies, and quote tweets, but now the first time somebody spewed vitriol at me, it would be their last.

But I still had to see them at least once.

Crumbling

And then 2016 came along.

I am a liberal. My wife and I laugh at the assertion that you become more conservative with age. If anything, we've become more liberal.

And the run-up to the 2016 US elections broke us.

On Twitter, I was seeing either tons of right-wing hate spewed by folks, or reactions from others to that hate. The few times I addressed it were horrible; the amount of vitriol in my mentions shocked me. Some people have the energy and mental reserves to fight back. I'm not one of those; I internalize the attack, and it replays in my mind over and over. It tears me apart.

So following the election, I started pulling back.

I created a couple lists that I'd check daily, mostly those of authors or artists I like and admire. This created a little oasis for me, and made things somewhat manageable.

But here's the thing: we are all political. Living in a society means we engage with politics. And this meant that, even following creators, I was still seeing politics; the politics of the era affect us all. And I was seeing how people responded, reacted, attacked these people I love and admire, and that was somehow even worse than when it was directed at me.

I started checking Twitter less and less frequently.

I started using Instagram in 2019, primarily to share my Zentangle-inspired art. Oh, my, was that a breath of fresh air. Yes, there are ads, but I would open it, and be greeted by primarily screen after screen after screen of art. It was bliss.

I was also increasingly using Facebook, mostly in private groups for, you guessed it, sharing Zentangle-inspired art. When one of these moved to circle.so, I was amazed at how much better the experience was on a private platform. We could engage directly with each other, and the website facilitated meaningful interaction better. It showed me that social media doesn't have to be algorithmically determined, and that decent tooling could facilitate better quality interactions. And I re-discovered that smaller is better; I don't need the entire world at my fingertips all the time.

But I also kept checking my creator lists on Twitter; it was a habit I couldn't quite shake.

Chaos

And then Musk came along, and threw his ego and money around, announcing his intent to buy Twitter.

When he announced his intent in April of this year, I remembered Mastodon, and remembered that the PHP community had an instance. I joined, and started using that as my goto microblogging location, even setting up ways to send tweets when I posted certain keywords. The community then was small (around 200 folks), and I could even follow our local server timeline quickly each day, which would give me recommendations for new folks to follow.

But I still kept checking my creator lists on Twitter; I missed these creators, and wanted to follow their work.

And then the Twitter deal closed, and now Musk was "in charge". (The quotes are deliberate; his flailing hardly feels like somebody in charge.)

And I just can't.

Allowing Trump back on (even if he hasn't yet rejoined); sending dog whistles and outright overtures to white nationalists and antisemitism and outright fascists; unbanning people banned for those exact beliefs; playing roulette with the verification system until it becomes meaningless; firing the very people that made the service work for so many years, and who worked to improve its community safety (though these efforts still needed a lot of work); and and and and

I just can't.

I don't want to provide content for a billionaire to make money off of. I don't want to engage with the ads that will help pay for this new version of the service. As much as I cherish free speech, I'm not a free speech absolutist; hate speech should not be protected, and certainly not amplified. I don't want to be in the "world's town hall" if that means having to argue with people who do not bring arguments to the table in good faith.

Where I'll be

And... I don't have to.

Since Twitter transferred ownership to Musk, the Masto instance I'm on, phpc.social, has grown to over 1600 users. The majority of the creators I follow are at least trying Mastodon out, usually on one of the big instances. Many friends I've not communicated with in years are also moving, to many different hosts. Where I was able to follow my instance timeline easily each day, I've now already given up on even following my home timeline, and am, in fact, needing to segregate into lists again... but now not out of a desire to limit what I see, but instead to allow me to dive into conversations around topics when I have the time and interest.

Look, Mastodon isn't perfect. It could use some UX designers and experts. And there's a moderation problem, particularly if you're on a big instance; in particular, BIPOC users are reporting issues with moderation and discrimination that need to be addressed head on. And while the design of ActivityPub, the protocol underlying Mastodon, is such that while "spin up your own instance" is a valid answer, the fact is that if your audience grows, you can DDoS your instance really quickly any time you post.

But it's also important to note that Mastodon has been chugging along for years already, powering communities that are largely queer and neurodivergent, and you can quickly find communities that share your values, and which are small and inviting, and fiercely protective. These are the same communities that, when each of Gab and Truth Social came online (both of which are built on Mastodon), immediately defederated them, ensuring they would not show up in their timelines, or any of the servers they federate with.

It's good enough, at least for now.

A week ago, I decided I was done with Twitter. I requested my archive. I deleted my one remaining Twitter-related app from my phone (I loathed the "official" app). I modified my browser such that the words "twitter" or "tweetdeck" now suggest the site I use with my Mastodon accounts. I setup WebFinger on my website, such that "@matthew@mwop.net" will resolve to wherever I am microblogging. And I removed all references to Twitter on my website; no more "Tweet this" links, no more embedded tweet streams.

I'll continue occasionally posting to Twitter, via my Mastodon account. But I'm no longer going to visit it to read timelines or mentions. It'll be like sending the odd communique to the wilderness for now.

Goodbye, Twitter. You were fun, until you weren't.

Goodbye Twitter was originally published 28 November 2022 on https://mwop.net by Matthew Weier O'Phinney.