Upgrading to Drupal 8.4.0 and Drush 9 with Composer

Time to upgrade.

Drupal 8.4.0 was released last week. This means that Drupal 8.3 is officially unsupported, so it is time to upgrade existing sites.

This particular upgrade presents some challenges, especially for sites managed with composer. According to the release notes,

Drupal 8.4.0 includes major version updates for two dependencies: Symfony 3.2 and jQuery 3. Both updates may introduce backwards compatibility issues for some sites or modules, so test carefully.

I have not looked into the consequences of the jQuery update, but a major version update of Symfony means that a lot of other PHP components have to be upgraded or removed.

In particular, the current released version of drush is incompatible with Symfony 3.2. That means you have a few options:

• Install the latest version of drush (as I write, 8.1.14) globally, not as part of your Drupal project.
• Install the 8.x-dev version as part of your project.
• Install version 9 (currently 9.0.0-beta6) as part of your project.

I have only tried the last of these.

Upgrading to Drupal 8.4.0

My Drupal 8 projects all use the semi-official standard for managing Drupal with composer: Composer template for Drupal projects. Some of my difficulties may be due to problems with this starting point, and others are probably caused by mistakes I have made along the way. I will not apologize for the mistakes: think of them as learning opportunities.

I did not try upgrading everything at once with a simple composer update. This might have worked after some initial steps, but it is generally agreed that upgrading All The Things at once is too risky.

My general procedure for arriving at a compatible set of requirements:

1. Try composer update drupal/core — with-dependencies.
2. Find a conflicting package and composer why conflicting/package.
3. Find the source of the conflict and either composer update source/package — with-dependencies or composer remove source/package.
4. Repeat.
5. Restore the removed packages.

By the way, composer why is an alias of composer depends. To save a little typing, I did alias why=’composer why’.

Here is a slightly idealized git log recording the packages I had to remove and re-install before I was able to upgrade Drupal:

• Temporarily remove drupal/console
• Temporarily remove drush/drush
• Remove drupal/console-* from composer.lock
• Update Drupal core from 8.3.7 to 8.4.0

In other words, this sequence of composer commands:

composer remove drupal/console drush/drush
composer remove drupal/console-core drupal/console-en drupal/console-extend-plugin
composer update drupal/core --with-dependencies

I am not sure why the drupal/console-* packages were not removed when I removed drupal/console.

So far, I have not been able to restore Drupal console. See the next section for restoring drush.

Installing a compatible version of Drush

The release notes for Drupal 8.4.0 mention

Versions of Drush earlier than 8.1.12 will not work with Drupal 8.4.x. Update Drush to 8.1.12 or higher before using it to update to Drupal core 8.4.x or you will encounter fatal errors that prevent updates from running.

That assumes you install drush globally, not as a composer dependency of your project. If you follow that link, you will see, under “Current status”,

Drush 8.1.12 site-local Drush + Drupal 8.4.x: Does not work (See https://github.com/drush-ops/drush/pull/2800)

Drush 8.1.12 phar + Drupal 8.4.x: Works (Not officially supported) **

Drush 9.x site-local Drush + Drupal 8.4.x: Supported

Drush 9.x global install + Drupal 8.4.x: Works (Not officially supported, cgr install recommended) **

(Follow the link for notes on Drupal 8.3.x and for the explanation of “Not officially supported”.)

I see one commit on the 8.x branch aimed at making it compatible with Symfony 3, so there may soon be a release of drush 8 that is compatible.

Here is the log as I restored drush:

• Allow webflo/drupal-finder either ^0.2.1 or ^1.0
• Temporarily remove consolidation/robo
• Temporarily remove stecman/symfony-console-completion
• Temporarily remove phpunit/phpunit
• Require drush/drush version 9, currently 9.0.0-beta5
• Restore phpunit/phpunit (dev requirement)
• Temporarily remove alchemy/zippy

In other words, something like the following:

composer require "webflo/drupal-finder:^0.2.1|^1.0"
composer remove consolidation/robo stecman/symfony-console-completion phpunit/phpunit
composer require drush/drush:^9.0.0
composer require --dev "phpunit/phpunit:>=4.8.28 <5"
composer remove alchemy/zippy

I think that webflo/drupal-finder is one of the packages we got from drupal-composer/drupal-project. I am not sure why alchemy/zippy was still there (in composer.lock), nor why I decided to remove it.

Difficulties with Drush 9

Drush 9 is still in beta, and there are some things that do not seem to work yet. My initial tests were with beta5, and beta6 is already out, so it is possible that some of these problems have already been fixed.

I see these commit messages in my git log:

• Add a drush alias file for drush 9
• Update Probo commands for drush 9
• Temporarily disable Probo commands to download JS libraries

Drush 9 now uses YAML format for its site-alias files. If you have an older site-alias file, then drush will automatically generate the new format for you, and then you can commit that to version control.

We use Probo CI for continuous integration. Some of the drush commands we run on Probo need to be updated because of changes to drush, and some (especially those provided by contrib modules) just do not work.

In order to use drush instead of vendor/bin/drush with Drush 9, install Drush Launcher. If (like me) you have previously installed drush with Homebrew, then first brew unlink drush.

Maybe it is just me, but drush does not seem to be honoring the SSH options in the site-alias file. I need that because I run Drupal on a virtual machine (VM) using Vagrant. I found two ways to work around this problem. One is to append my public SSH key (~/.ssh/id_rsa.pub) to ~vagrant/.ssh/authorized_keys on the VM. The other way is to add these lines to ~/.ssh/config (on the host):

Host mysite.dev
  Hostname mysite.dev
  User vagrant
  IdentityFile ~/.vagrant.d/insecure_private_key

There are still some issues: I think that drush does not always pass the correct command when it delegates to drush on the VM. With either of the above work-arounds, I can use drush ssh to log in to the VM and then run vendor/bin/drush from there.

This article was originally published by Benji Fisher on our Isovera blog.