Testing invalid form validation in Laravel using PHPUnit
Introduction
One thing I have noticed when reviewing code relating to form validation is that the focus is predominantly on testing the true path, or best-case scenario. By this, I mean that we have a set of business rules that the request must meet, and the payload provided is made up of values which are intended to pass said rules.
However, I believe that when testing our form validation rules, we should really be focusing more on making sure that we are writing tests that actually "fail" these rules, and asserting that the errors in question are accurate based on the provided values. By doing it this way, we can be far more confident that anything an end user submits will be handled in the way we intend and is validated properly before being passed into our code to be handled later.
It's important to note that, while making sure that all the specified rules are being validated correctly, it's only one part of helping make your application more secure from malicious threats or invalid data, and should be treated as a first step rather than the only step.
Throughout this post, we will be looking at a simple ruleset, and iterating over the test cases; from not asserting anything at all, to asserting exactly what we expect to fail, in, hopefully, an easy-to-follow manner.
I will also preface this by stating that this example will be done in the Laravel framework, as per the title, however, the same basic idea should be possible regardless of the language used, albeit with different syntax etc.
All the examples shown in this post can be seen in the following repo, https://github.com/flukedit/laravel-form-validation-example, and can be pulled down and played around with.
An iterative process
The way this will be laid out is by iterating over a test on a single endpoint, improving it a bit at a time to show how we can do a number of small changes to get to our end goal rather than jumping straight to the end, and potentially causing unnecessary confusion.
The following examples will all use the same ruleset defined below:
use Illuminate\Foundation\Http\FormRequest;
class ExampleRequest extends FormRequest
{
public function rules(): array
{
return [
'userId' => ['required', 'integer', 'digits:6'],
'description' => ['required', 'string', 'max:20'],
];
}
}
Attempt #1
Let's say we have the following test where we are not providing any form data at all to validate:
/** @test */
public function it_returns_422_status_when_form_data_not_provided(): void
{
$this->postJson(route('example'), [])
->assertStatus(422);
}
Now, this test passes, but what is it that caused it to fail?
If all we do is assert `422` (which we will get to later), then it could be failing for any random reason, and we wouldn't know.
The end user would be provided with a list of errors, but there's no guarantee that the ones we want the user to get are being provided.
Just in case you are unaware, status code 422 is the default response code for a validation error in Laravel when using the FormRequest class.
Attempt #2
This time, let's submit the form while providing the `userId`
/** @test */
public function it_returns_422_status_when_userId_valid_but_description_is_missing(): void
{
$this->postJson(route('example'), [
'userId' => 9999999,
])->assertStatus(422);
}
This test passes as well, as it should because we still haven't included `description`, which is a required field.
However, if we look carefully, this test would also pass, even if the description was provided with a valid string.
If we take a look at the response that comes back, you'll see that there was a second problem with that form data, which would be completely obfuscated due to the test name and the payload clearly not having the `description` field included.
{
"message":"The given data was invalid.",
"errors":{
"userId":[
"The user id must be 6 digits."
],
"description":[
"The description field is required."
]
}
}
/** @test */
public function it_returns_422_status_when_userId_valid_but_description_is_missing(): void
{
$this->postJson(route('example'), [
'userId' => 9999999, // <- The userId provided is seven digits which is why that rule did not pass
])->assertStatus(422);
}
Attempt #3
Now that we can see the potential issues with just asserting against the status, let's see what we can do to make it more defined. Using the first example again, we will post an empty payload.
We know from the request class that both fields are `required`, so we should be asserting that the returned JSON response contains those particular error messages.
Laravel gives us a very handy function, `assertJsonValidationErrors`, which makes this a breeze.
/** @test */
public function it_returns_422_status_when_form_data_not_provided_as_all_fields_are_required(): void
{
$this->postJson(route('example'), [])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The user id field is required.',
'description' => 'The description field is required.',
]);
}
Laravel also provides another handy function, `assertInvalid`, which allows you to check that the correct rule failed.
/** @test */
public function it_returns_422_status_when_form_data_not_provided_as_all_fields_are_required(): void
{
$this->postJson(route('example'), [])
->assertStatus(422)
->assertInvalid([
'userId' => 'required',
'description' => 'required',
]);
}
The full message will be used for the rest of this post due to personal preference.
Looking at this, you may realise that you are going to write, potentially, 10s (such a large number I know...) of tests like this depending on the scope and size of your form.
Using this process, as we want to loosely group the same errors together, we would end up with something like the following:
/** @test */
public function it_returns_422_status_when_form_data_not_provided_as_all_fields_are_required(): void
{
$this->postJson(route('example'), [])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The user id field is required.',
'description' => 'The description field is required.',
]);
}
/** @test */
public function it_returns_422_status_when_fields_are_provided_as_invalid_types(): void
{
$this->postJson(route('example'), [
'userId' => 'string',
'description' => ['array'],
])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The user id must be an integer.',
'description' => 'The description must be a string.',
]);
}
/** @test */
public function it_returns_422_status_when_userId_is_too_few_digits(): void
{
$this->postJson(route('example'), [
'userId' => 55555,
])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The user id must be 6 digits.',
]);
}
/** @test */
public function it_returns_422_status_when_userId_is_too_many_digits(): void
{
$this->postJson(route('example'), [
'userId' => 7777777,
])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The user id must be 6 digits.',
]);
}
/** @test */
public function it_returns_422_status_when_description_is_too_many_characters(): void
{
$this->postJson(route('example'), [
'description' => 'string-more-than-20-chars',
])
->assertStatus(422)
->assertJsonValidationErrors([
'userId' => 'The description must not be greater than 20 characters.',
]);
}
Imagine if there were more than just two fields that needed to be validated!!!
Attempt #4
For this attempt, let's write the same tests as above, but instead of writing them as individual tests, we accomplish the same thing using PHPUnit's Data Provider feature. You can read more about this in the documentation about [`data-providers](https://phpunit.readthedocs.io/en/9.5/writing-tests-for-phpunit.html#data-providers)`\.
/**
* @test
* @dataProvider invalidFormValidationProvider
*/
public function it_returns_422_status_when_invalid_form_data_is_provided(
array $formData,
array $errorMessages
): void {
$this->postJson(route('example'), $formData)
->assertStatus(422)
->assertJsonValidationErrors($errorMessages);
}
public function invalidFormValidationProvider(): array
{
return [
[
[],
[
'userId' => 'The user id field is required.',
'description' => 'The description field is required.',
],
],
[
[
'userId' => 'string',
'description' => ['array'],
],
[
'userId' => 'The user id must be an integer.',
'description' => 'The description must be a string.',
],
],
[
['userId' => 55555],
['userId' => 'The user id must be 6 digits.'],
],
[
['userId' => 7777777],
['userId' => 'The user id must be 6 digits.'],
],
[
['description' => 'string-more-than-20-chars'],
['description' => 'The description must not be greater than 20 characters.'],
],
];
}
Now we have just one test which we use the `dataProvider` to pass in a payload and the expected errors that the payload should generate.
There is a small problem with this current iteration though, which is related to the output that the test case provides when an assertion fails.
As an example, if we modify the third test case to provide the correct number of digits:
There was 1 failure:
1) Tests\Feature\ExampleControllerTest::it_returns_422_status_when_invalid_form_data_is_provided with data set #2 (array(555555), array('The user id must be 6 digits.'))
Failed to find a validation error in the response for key: 'userId'
Response has the following JSON validation errors:
{
"description": [
"The description field is required."
]
}
Failed asserting that an array has the key 'userId'.
It's not immediately obvious which of the test case is the one with the problem, even with the note specifying which data set was the one with the issue.
with data set #2 (array(555555), array('The user id must be 6 digits.'))
As we only have five datasets, it's not too bad, but if we had a large number of them, it gets increasingly more difficult to work it out. Luckily, we can give each of the datasets a name to help identify the test.
Attempt #5
In this iteration, we have renamed the name of the test to remove the specific response code, and also modified the status assertion to use the `assertUnprocessable()` function, which does the same thing as `assertStatus(422)`, but makes it human-readable, or at least removes the need to remember what the code means.
There are a number of these functions which can be used to replace a lot of the more commonly used codes, as well as a neat way to write status assertions which don't have the function, however, that will not be covered further here.
/**
* @test
* @dataProvider invalidFormValidationProvider
*/
public function it_returns_validation_error_status_when_invalid_form_data_is_provided(
array $formData,
array $errorMessages
): void {
$this->postJson(route('example'), $formData)
->assertUnprocessable()
->assertJsonValidationErrors($errorMessages);
}
public function invalidFormValidationProvider(): array
{
return [
'All fields must be required' => [
[],
[
'userId' => 'The user id field is required.',
'description' => 'The description field is required.',
],
],
'All fields must be the correct type' => [
[
'userId' => 'string',
'description' => ['array'],
],
[
'userId' => 'The user id must be an integer.',
'description' => 'The description must be a string.',
],
],
'user id must not be less than 6 digits' => [
['userId' => 55555],
['userId' => 'The user id must be 6 digits.'],
],
'user id must not be more than 6 digits' => [
['userId' => 7777777],
['userId' => 'The user id must be 6 digits.'],
],
'The description may not be greater than 20 characters' => [
['description' => 'string-more-than-20-chars'],
['description' => 'The description must not be greater than 20 characters.'],
],
];
}
And now if we make the same change as before with the `userId`, we get the following:
There was 1 failure:
1) Tests\Feature\ExampleControllerTest::it_returns_422_status_when_invalid_form_data_is_provided with data set "user id must not be less than 6 digits" (array(555555), array('The user id must be 6 digits.'))
Failed to find a validation error in the response for key: 'userId'
Response has the following JSON validation errors:
{
"description": [
"The description field is required."
]
}
Failed asserting that an array has the key 'userId'.
Which we can now see, provides us with the specific `dataset` which caused the issue.
Conclusion
Hopefully, this has shown the benefits that the testing of *fail* paths can provide, as well as showing how it can be attempted in an iterative way so as to not be too overwhelming before starting.
Obviously, this is only one way of doing something like this, and the way that this can be done may differ based on your team's way of working etc. The main thing I am hoping for people to take from this is that form validation is not difficult or scary.