Creating a DDEV Addon
In my quest to create a test for a Drupal contrib module…as a way to study for the Acquia Drupal certification…as a way to further my professional development…but really to please my job…and as a way to not feel like a total loser…I ended up going down several rabbit holes.
I like to try and combine my goals into the activities I pursue to gain the most efficiency possible. If I can see that a particular activity furthers several goals, then I’m more likely to try and pursue that activity over the multitude of other things I could be doing.
In the dev world, that’s why I usually try to start a blog post any time I’m doing something new as it might have the potential to help others understand what I did not. Even if I fail at my main goal, at least I can post something and call that part a success.
Step One: Try to write a test
I go over this step in another blog post where I was trying to help contribute to a Drupal contrib module and ended up needing to write a functional JS test.
[I’ll link to this post when it is published…]
In that process, I had some trouble getting the Drupal WebDriverTestBase class to connect to a web driver and run my test. Turns out I needed to configure a Docker container to run the web driver and point PHPUnit to that container to run my test.
Step Two: Ask for help
At this point, I did some Googling to see what I could find specifically about using DDEV to run the Selenium server I needed to connect to.
Turns out the Drupal Slack instance has a “#ddev” channel, and to my amazement, someone was asking how to run PHPUnit tests in the recent message history.
And then that thread led me to a blog post that goes over configuring PHPUnit to run Drupal tests amongst some other things.
But wait…there’s more! And a second post is linked from that blog post to specifically talk about running functional JS tests…cause running functional JS tests is for some reason so complicated.
And that post has a Docker Compose file that looked like what I needed to run my test. However, it only led me to an error message that had me stumped.
So I posted my error into the #ddev channel and said a prayer 🙏.
Step Three: Receive Help
I was thrilled to get a response to my query from the DDEV maintainer in no less than a minute flat. How cool!
Turns out I had yet another issue with my fancy new laptop and Docker images. The Selenium Docker image being used in the blog post I found isn’t compatible with my system.
Luckily, another #ddev user, Moshe, pointed me to their Docker Compose file where an “arm64” image was used.
All I had to do was plop that YAML into my failing Docker Compose file and voila, it worked!
Step Four: Get Encouraged To Contribute
After a long Slack thread, I was encouraged by the DDEV maintainer to contribute something back to the DDEV community since so many people have trouble running Drupal tests locally and hence contribute less to the project than they might if running tests were easier.
If you so choose, you too can create a DDEV addon with the help of a handy template.
I ended up following the advice of that repo to create an addon for the Selenium Chrome Docker container that worked for my functional JS test. Since there were several steps mentioned in the readme, I simply made them into GitHub issues so I could modify the readme template to suit my project.
Step Five: Learn About Bats
Any time I think about bats, images of caves, guano, and Michael Keaton dance around my mind. But not anymore. Now I know about the testing framework called “bats”, which stands for “Bash Automated Testing System.”
Bats is a TAP-compliant testing framework for Bash. It provides a simple way to verify that the UNIX programs you write behave as expected. A Bats test file is a Bash script with special syntax for defining test cases. Under the hood, each test case is just a function with a description.
Since you can use bash commands to run the DDEV CLI, I think it makes sense to promote bats as the testing strategy for DDEV addons. I have no idea if the internals of DDEV is tested using bats, but all I have to do is write one or two tests to confirm the Selenium server is running at the proper port used in the Docker Compose YAML file.
But how do I write a test to know that my addon is working? Well, I thought I would try and take some inspiration from the addon template:
# Do something useful here that verifies the add-on
ddev exec "curl -s elasticsearch:9200" | grep "${PROJNAME}-elasticsearch"If I can call the service and search for a status message successfully, then I think I can confirm the addon works. Initially, I tried calling the service just to see what was returned.
# Get a list of all the services.
ddev describe
# No response recorded.
ddev exec "curl http://selenium-chrome:4444"
# Some kind of noVNC HTML returned.
ddev exec "curl https://d9-study.ddev.site:7900"If I used the public URL with port 7900 exposed, I got some weird “noVNC” web page that looked like it was trying to connect to something…and rather than going down a rabbit hole to see what this noVNC thing was, I decided to look up the Docker image to see if it had clues in its own tests.
After a little Googling, I found a test where they called an endpoint to determine the status of the “Selenium Grid”.
try:
response = urlopen('http://%s:%s/status' % (SELENIUM_GRID_HOST, port))
status_json = json.loads(response.read())
self.assertTrue(status_json['value']['ready'], "Container is not ready on port %s" % port)
status_fetched = TrueSo, why not try calling the status endpoint and see what happens, I thought. Sure enough, I got back some JSON that I could grep through and search for a message of success.
# Search the status JSON for a message of success.
ddev exec "curl http://selenium-chrome:4444/status" | grep "\"message\": \"Selenium Grid ready.\""And that’s it. Boom! I’m sure more tests can be added, but now on to the next step. To note, I took some inspiration from the Varnish DDEV addon tests as well.
Step Six: Make a Release
This part is possibly the simplest step since GitHub makes it so nice to package up a release.
If you’re not familiar with making releases on GitHub, all you need to do is go to your project page and click the “Create a new release” link in the right-hand sidebar. It will take you to a screen with a form to help with your release.
For all releases, you create a Git tag to a release commit and publish that to the world. I almost let GitHub create my tags, but since I’ve done this before I just created a tag locally and pushed it up.
# Create the tag with semnantic versioning and as an alpha.
git tag v0.1.0-alpha
# Push the tag to GitHub.
git push --tagsStep Seven: Fumble Around Trying to Over-engineer Your Addon
At this point, I could have gone back to the original Slack thread and told people what I’ve accomplished so far and gotten some advice on how to make the package better.
However, I thought: “why not try to make this addon work for all OSes even if not using arm64-based architecture?” For some reason, I was convinced that the arm64-based Docker image would not work for amd64-based machines, but without another laptop to test, I was flying blind and hoping the GitHub arm64-based Workflows would be all I needed to test out my theory.
My plan was to have two Docker files: one for arm64 and one for amd64. The install script would swap files based on what architecture it detected. After grabbing the addon, you would have the right Docker Compose file for your machine and pull down the correct Selenium Docker image upon DDEV booting up.
post_install_actions:
- |
#ddev-nodisplay
if [ "$(arch)" = "arm64" -o "$(arch)" = "aarch64" ]; then
echo "Using arm64 image";
cp -f selenium-chrome-arm64.yaml docker-compose.selenium-chrome.yaml;
rm -f selenium-chrome-amd64.yaml;
else
echo "Using amd64 image";
cp -f selenium-chrome-amd64.yaml docker-compose.selenium-chrome.yaml;
rm -f selenium-chrome-arm64.yaml;
fiWhile I thought I had a clever plan, I couldn’t get things working locally or on GitHub so I paused my addon work for a bit.
Step Eight: Find Out Someone Already Made Your Addon
One day, as I was looking at the Drupal Slack’s #testing channel and a thread about setting up functional testing for Drupal, I found the experienced developer who lent me their YAML, Moshe, had already created an addon for Selenium and the Chrome driver.
Waaahh 😢. But rather than being sad, I decided to look into the addon and see what I could have done better. For example, I only tested that the Selenium Server responded with a successful status message, but when I looked at Moshe’s tests, I saw a much more convincing set of assertions.
# All I tested.
ddev exec "curl -v selenium-chrome:4444/wd/hub/status"
# Extra things he tested..
echo "Run a FunctionalJavascript test." >&3
ddev exec -d /var/www/html/web "../vendor/bin/phpunit -v -c ./core/phpunit.xml.dist ./core/modules/system/tests/src/FunctionalJavascript/FrameworkTest.php"
echo "Run a Nightwatch test." >&3
ddev exec -d /var/www/html/web/core yarn install
ddev exec -d /var/www/html/web/core touch .env
ddev exec -d /var/www/html/web/core yarn test:nightwatch tests/Drupal/Nightwatch/Tests/exampleTest.js
echo "Install Drupal and run a DTT test." >&3
ddev exec -d /var/www/html/web "../vendor/bin/drush si -y --account-name=admin --account-pass=password standard"
ddev exec -d /var/www/html/web "../vendor/bin/phpunit --log-junit dtt.junit.xml --bootstrap=../vendor/weitzman/drupal-test-traits/src/bootstrap-fast.php --printer '\Drupal\Tests\Listeners\HtmlOutputPrinter' ../vendor/weitzman/drupal-test-traits/tests/ExampleSelenium2DriverTest.php"Not only does he make sure that Selenium’s status is okay, but he also runs an actual functional test, goes further by running a Nightwatch test, and then tops it off with installing Drupal and running more tests…and that’s way more useful than what I had going.
Oh well, this was a good learning experience for me, and hopefully, I can contribute something back to that DDEV Selenium Chrome addon as I start to use it.
Even if I fail at my main goal, at least I can post something and call that part a success.
I also find it funny that I presciently predicted the value of my post as I was writing it but before I failed at my main goal. Time to hit “post”!
