How to test CSV in Drupal without losing your mind

Published in

Soulweb Academy

6 min readMar 4, 2021

There are moments in a developer’s life when the only way you have to understand how to do your task is to get your hands dirty and look under the hood too see what’s going on. This time it was tests.

Today we want to share our experience on writing a test, on Drupal 8, for a CSV file generated from the system.

Our goal is not only to test if the file is generated, but also its content.

We couldn’t find any relevant information around, fitting this exact purpose, so we thought that could be nice to share it.

Summary

Scenario
Write the test
Run the test
Further notices
Conclusion

Scenario

In this scenario we are going to use, as example, the contrib module Tablefield.

In our company we develop many data solutions and systems with Drupal and data for the PA. We recently discovered the tablefield module, that it makes easy to load and export small groups of CSV data in a tabular view.

We wrote the test example of this article while working with it, for teaching reason.

The `module` itself can generate a CSV file from the data inserted in the field, that is going to be the focus of our test!

Write the test

To test it, we need to write and run a new Browser Test for Drupal.

As a start, we can copy/paste the file TableValueFieldTest.php under tests/src/Functional to create a new file for our test, named TableFieldExportCsvExportFileTest.php.

The final version of this file is going to look like this:

Let’s have a look at it.

As usual, the first thing we have to do is to set up what we need in order to run this test. In our case we need a content type, with a field of type tablefield, for the test, making sure to set export CSV to on for this field. So, our setup() method should be edited like the following:

 protected function setUp() {
 parent::setUp();$this->drupalLogin($this->rootUser);$this->drupalCreateContentType([‘type’ => ‘article’, ‘name’ => ‘Article’]);
 $this->createTableField(‘field_table’, ‘article’, [], [‘export’ => 1]);
 }

createTablefield() is a helper method, already present on Tablefield, to create the proper field setup for our test. With ['export'=>1] we can enable the export CSV.

Next, we need to edit the method four our test, testTableField(). Within this method, we are going to use the block of code already on it to create a new node with a field_table:

 $this->drupalGet(‘node/add/article’);// Create a node.
 $edit = [];
 $edit[‘title[0][value]’] = ‘Llamas are cool’;
 $edit[‘body[0][value]’] = ‘Llamas are very cool’;
 $edit[‘field_table[0][caption]’] = ‘Table caption’;
 $edit[‘field_table[0][tablefield][table][0][0]’] = ‘Header 1’;
 $edit[‘field_table[0][tablefield][table][0][1]’] = ‘Header 2’;
 $edit[‘field_table[0][tablefield][table][0][2]’] = ‘Header 3’;
 $edit[‘field_table[0][tablefield][table][0][3]’] = ‘Header 4’;
 $edit[‘field_table[0][tablefield][table][0][4]’] = ‘Header 5’;
 $edit[‘field_table[0][tablefield][table][1][0]’] = ‘Row 1–1’;
 $edit[‘field_table[0][tablefield][table][1][1]’] = ‘Row 1–2’;
 $edit[‘field_table[0][tablefield][table][1][2]’] = ‘Row 1–3’;
 $edit[‘field_table[0][tablefield][table][1][3]’] = ‘Row 1–4’;
 $edit[‘field_table[0][tablefield][table][1][4]’] = ‘Row 1–5’;
 $edit[‘field_table[0][tablefield][table][2][0]’] = ‘Row 2–1’;
 $edit[‘field_table[0][tablefield][table][2][1]’] = ‘Row 2–2’;
 $edit[‘field_table[0][tablefield][table][2][2]’] = ‘Row 2–3’;
 $edit[‘field_table[0][tablefield][table][2][3]’] = ‘Row 2–4’;
 $edit[‘field_table[0][tablefield][table][2][4]’] = ‘Row 2–5’;
 $edit[‘field_table[0][tablefield][table][3][0]’] = ‘Row 3–1’;
 $edit[‘field_table[0][tablefield][table][3][1]’] = ‘Row 3–2’;
 $edit[‘field_table[0][tablefield][table][3][2]’] = ‘Row 3–3’;
 $edit[‘field_table[0][tablefield][table][3][3]’] = ‘Row 3–4’;
 $edit[‘field_table[0][tablefield][table][3][4]’] = ‘Row 3–5’;
 $edit[‘field_table[0][tablefield][table][4][0]’] = ‘Row 4–1’;
 $edit[‘field_table[0][tablefield][table][4][1]’] = ‘Row 4–2’;
 $edit[‘field_table[0][tablefield][table][4][2]’] = ‘Row 4–3’;
 $edit[‘field_table[0][tablefield][table][4][3]’] = ‘Row 4–4’;
 $edit[‘field_table[0][tablefield][table][4][4]’] = ‘Row 4–5’;$this->drupalPostForm(NULL, $edit, t(‘Save’));

Now, we are ready to make our assertions to perform the real test.

$assert_session = $this->assertSession();

The first assertion is to make sure our node has been successfully created:

 $assert_session->pageTextContains(‘Article Llamas are cool has been created.’);

By default, when the export is on, tablefield renders a Export Table Data link:

We need to click the link to initiate the CSV export:

 $this->clickLink(‘Export Table Data’);

Once clicked, there is a redirection to a new route where the CSV is generated and made available to download. The nice thing is that PHPUnit is able to catch the whole flow and test the CSV file content itself.

So, let’s test it! First step, we want to make sure that the file has been successfully generated:

 $assert_session->statusCodeEquals(200);
 $assert_session->responseHeaderContains(‘Content-Type’, ‘text/csv; charset=utf-8’);

Finally, it’s time to test the CSV content:

 $assert_session->responseContains(
 “\”Header 1\”,\”Header 2\”,\”Header 3\”,\”Header 4\”,\”Header 5\”\n”.
 “\”Row 1–1\”,\”Row 1–2\”,\”Row 1–3\”,\”Row 1–4\”,\”Row 1–5\”\n”.
 “\”Row 2–1\”,\”Row 2–2\”,\”Row 2–3\”,\”Row 2–4\”,\”Row 2–5\”\n”.
 “\”Row 3–1\”,\”Row 3–2\”,\”Row 3–3\”,\”Row 3–4\”,\”Row 3–5\”\n”.
 “\”Row 4–1\”,\”Row 4–2\”,\”Row 4–3\”,\”Row 4–4\”,\”Row 4–5\””
 );

We’ll explain later why the CSV content is formatted like that. For now, we are ready to run our test!

Run the test

To run the test, from your Drupal root, execute:

 $ ./vendor/bin/phpunit -c web/phpunit.xml ./web/modules
/contrib/tablefield/tests/src/Functional/TableFieldExportCsvExportFileExistsTest.php

making sure to replace -c web/phpunit.xml with your phpunit config file.

If the final output looks similar to this:

…then we are done!!! 🥳🥳🥳

Further notices

If you are curious about the process we followed to write a successfully completing test, go ahead with this section! Otherwise, we are pretty much done, you can go back to your work if you like! 😉

Diving closer to the test output, we can see the list of the HTML output generated.

It’s always interesting to give a look at them and analyze what actually happens behind the scene during our test.

When we first started to write this test, we didn’t really have a clue on how to test the content of the CSV, plus we couldn’t find anything from our researches around.

So, we proceeded writing the most obvious part of the test, where we generate our content with a field table and perform the click on ‘Export Table Data’.

That’s when we decided to give a look at the test output — the last on the list — on this last action.

The output looked similar to:

Having a chance to get a proper look at it, we soon noticed the useful elements to make the right assertions.

First we focused on the Headers.
Our intent is to make sure the file is correctly generated as CSV. As you can easily spot, in the Header we have:

‘Content-Type’ => ‘text/csv; charset=utf-8’,

So we can write an assertion for it:

 $assert_session->statusCodeEquals(200);
 $assert_session->responseHeaderContains(‘Content-Type’, ‘text/csv; charset=utf-8’);

And now it’s the tricky part, to test the CSV content!
Of course, we can see how phpunit sees it:

“Header 1”,”Header 2",”Header 3",”Header 4",”Header 5" “Row 1–1”,”Row 1–2",”Row 1–3",”Row 1–4",”Row 1–5" “Row 2–1”,”Row 2–2",”Row 2–3",”Row 2–4",”Row 2–5" “Row 3–1”,”Row 3–2",”Row 3–3",”Row 3–4",”Row 3–5" “Row 4–1”,”Row 4–2",”Row 4–3",”Row 4–4",”Row 4–5"

Looking at it, we were aware that some weird character could be hidden somewhere — just because it didn’t look like the real CSV file. Anyway, the first attempt was just to copy/paste this content in an assertion:

 $assert_session->responseContains(‘“Header 1”,”Header 2",”Header 3",”Header 4",”Header 5" “Row 1–1”,”Row 1–2",”Row 1–3",”Row 1–4",”Row 1–5" “Row 2–1”,”Row 2–2",”Row 2–3",”Row 2–4",”Row 2–5" “Row 3–1”,”Row 3–2",”Row 3–3",”Row 3–4",”Row 3–5" “Row 4–1”,”Row 4–2",”Row 4–3",”Row 4–4",”Row 4–5"’);

After running the test we realized that it was failing. So, as second attempt, we tested just small portions of the output, like only a cell of our table:

 $assert_session->responseContains(‘“Header 1”’);

or a row:

 $assert_session->responseContains(‘“Header 1”,”Header 2",”Header 3",”Header 4",”Header 5"’);

or two rows:

 $assert_session->responseContains(‘“Header 1”,”Header 2",”Header 3",”Header 4",”Header 5" “Row 1–1”,”Row 1–2",”Row 1–3",”Row 1–4",”Row 1–5"’);

The first two were fine, the test passed, but the last one failed!

At this point was quite clear that the new line character, between two lines, was in some way using a different coding.

Finally, we decided to try to write a proper output for our assertion, with the help of php escaping characters, to try to match the new line character:

 $assert_session->responseContains(
 “\”Header 1\”,\”Header 2\”,\”Header 3\”,\”Header 4\”,\”Header 5\”\n”.
 “\”Row 1–1\”,\”Row 1–2\”,\”Row 1–3\”,\”Row 1–4\”,\”Row 1–5\”\n”.
 “\”Row 2–1\”,\”Row 2–2\”,\”Row 2–3\”,\”Row 2–4\”,\”Row 2–5\”\n”.
 “\”Row 3–1\”,\”Row 3–2\”,\”Row 3–3\”,\”Row 3–4\”,\”Row 3–5\”\n”.
 “\”Row 4–1\”,\”Row 4–2\”,\”Row 4–3\”,\”Row 4–4\”,\”Row 4–5\””
 );

Conclusion

Nobody loves tests (😈) but we all know they are very useful to be sure that our code is working flawlessly as expected.
This was a good exercise to have a deeper look on how tools that we work with everyday are behaving under the hood and, while sharing the technical process, try too share a ‘real life’ situation and all the kind of questions and answers that we went through to finally get to a solution.

We really hope you enjoyed it and found it useful!
Cheers! 😉