ALEMIL

Welcome to a blog about application and game development

Laravel TDD workflow

Laravel TDD workflow 2018-05-28 21:40:14
Est. read time: 12 minutes, 32 seconds

Laravel is the most popular PHP framework for building web applications. It provides simple solutions for common problems, like secure database querying, routing, console commands, ORM or input validation. It is a very solid choice if you wish to follow test-driven development process.

In this article, I will explain step by step the workflow that worked for me when building web applications, specifically focusing on web APIs. Before we get started, you might want to set up a clean example project to follow and test these steps. The code will be available on github if you will want to tinker with it.

What we are making


I like to use real-world examples. They are usually a bit more complex than regular hello worlds but give the opportunity to explain everything in more depth. To demonstrate the workflow we will implement fully functional import of coordinates list from CSV file into the SQL database and assign them to an existing user.

For reference here are links to each step that we will follow:
Step 1: Write a test
Step 2: Define the route
Step 3: Create the controller
Step 4: Set up a database table
Step 5: Import coordinates to the database
Step 6: Validate the input
Conclusion

Step 1: Write a test


If you start by writing an integration test, you have to imagine the input and desired output of your REST API. This can be very helpful if you don't know yet how to solve a specific problem. Let's start with the input. We need an example file that will be uploaded to our application. And we need to figure out the format that we accept.
latitude,longitude
52.2297,21.0122
41.8781,87.6298
51.5074, 0.1278
The first line defines the field name in our database and all the others are values that go there. Create a coordinates.csv file in tests/data folder with the above content. We now have an example input, so let's write a test that will check the output.
php artisan make:test CoordinatesControllerTest

The above command will create CoordinatesControllerTest class in our tests/Feature folder. CoordinatesController will be a class that will process our files and will upload the content to the database. Let's replace the testExample function that was created with what we want.
/**
* Check if file with coordinates is being uploaded correctly into the database
*/
public function testUploadFile()
{
// First we prepare our example file for upload.
$fileName = 'coordinates.csv';
$filePath = base_path('tests/data/' + $fileName);
$file = new UploadedFile($filePath, $fileName, 'text/csv', filesize($filePath));

// We create a fake user to assign the coordinates to.
$user = factory(User::class)->create();

// Now we imitate the request, that we could make manually by Postman for example.
$response = $this->post('/api/coordinates/upload', ['file' => $file, 'user_id' => $user->id]);

// We make sure everything went fine and there was no error thrown.
$response->assertExactJson(['success' => true]);

// Last thing, we check if database has our coordinates uploaded and assigned to proper user.
$this->assertDatabaseHas('coordinates', ['user_id' => $user->id, 'latitude' => 52.2297, 'longitude' => 21.0122]);
$this->assertDatabaseHas('coordinates', ['user_id' => $user->id, 'latitude' => 41.8781, 'longitude' => 87.6298]);
$this->assertDatabaseHas('coordinates', ['user_id' => $user->id, 'latitude' => 51.5074, 'longitude' => 0.1278]);
}
We have planned out our application before we started to implement it and figured out how we want to accomplish our goal. By making the test we have decided that we will use HTTP POST method for our upload, that we will name our route api/coordinates/upload and that our database table will be named coordinates. We also defined that if everything goes as planned, our API will return success: true message in JSON format. We could do this part iteratively, by adding the database checks after we finished step 3, but if we know what we want we can go ahead and complete the test in one go.

Step 2: Define the route


If we run the test by typing in the console in the main folder of our application:
phpunit
We should get this result:
1) Tests\Feature\CoordinatesControllerTest::testUploadFile
Symfony\Component\HttpKernel\Exception\NotFoundHttpException:
The Exceptions tells us that our API doesn't have the route defined. We will create it next.

Laravel has split it's routing into two files. Because we are making an API we are interested in routes/api.php file.
Route::group(['middleware' => ['throttle']], function () {
Route::post('coordinates/upload', 'CoordinatesController@uploadFile');
});

We wrap our route definition within a group. Its sole purpose in our case is to assign middlewares to every route inside. Middlewares are pieces of code that will run before or after your Request hits the Controller.

Throttle middleware will block multiple requests in short amount of time, helping with spam attempts. You might also add an authorization middleware here if you want to prevent unknown users to upload files to your application. Every route in api.php file will start with "api/" so it doesn't have to be defined.
Route::post('coordinates/upload', 'CoordinatesController@uploadFile');
This line points the URL domain/api/coordinates/upload to uploadFile function inside CoordinatesController. If we run our test we would see an exception that the class and function doesn't exist yet, so let's create them.

Step 3: Create the controller


We can create our controller by running below command in the main folder of our application:
php artisan make:controller CoordinatesController

Then we add our missing function into app/Http/Controllers/CoordinatesController.php file:
/**
* Upload coordinates file for a user into the database
* @param Request $request
* @return \Illuminate\Http\JsonResponse
*/
public function uploadFile(Request $request)
{
return response()->json(['success' => true]);
}
If we run the test now, we should see result similar to the one below:

1) Tests\Feature\CoordinatesControllerTest::testUploadFile
Illuminate\Database\QueryException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'homestead.coordinates' doesn't exist (SQL: select count(*) as aggregate from `coordinates` where (`user_id` = 11 and `latitude` = 52.2297 and `longitude` = 21.0122))

First part of our test passed. The new error indicates that we don't have a database table set up yet. That is going to be our next step.

Step 4: Set up a database table


In Laravel, we create database tables in migrations. Migrations allow us to apply and rollback changes to databases easily. You can think of this as versioning your database. You can run a single command to rollback to the previous version or reset the database if you make a mistake while developing.
php artisan make:migration create_table_coordinates --create=coordinates
Our migration will be created in database/migrations folder and its name will start with the current timestamp. Migrations consist of two functions. The up function is run when you want to change your database and the down function will rollback the migration to the previous version. In our example the down function will consist of a generated table drop command, so we only need to add some fields in the up function.
public function up()
{
Schema::create('coordinates', function (Blueprint $table) {
// Create PRIMARY KEY id field
$table->increments('id');
// Create a foreign key pointing our coordinates to user record
$table->unsignedInteger('user_id');
$table->foreign('user_id')->references('id')->on('users');
// Create the fields that we need to store the coordinates from csv file.
$table->decimal('latitude', 8, 5);
$table->decimal('longitude', 8, 5);
});
}
After we changed the migration file, we can run it with a command:
php artisan migrate
We have the table created, so we run our tests:
There was 1 failure:

1) Tests\Feature\CoordinatesControllerTest::testUploadFile
Failed asserting that a row in the table [coordinates] matches the attributes {
"user_id": 13,
"latitude": 52.2297,
"longitude": 21.0122
}.

The table is empty.
The table is set up, let's fill it with data from our file!

Step 5: Import coordinates to the database


To keep our code clean, we need a place where we could implement CSV parsing and extracting the coordinates that we need. Our controller already has a purpose - it controls the flow of data in our application, so let's not make it do more. We will create a service that will do what we need at app/Services/Import/Coordinates.php.
<?php namespace App\Services\Import;

class Coordinates
{
/**
* Extract coordinates in CSV file
* @param string $filePath
* @return array
*/
public function extractCSV(string $filePath): array
{
// We use clean PHP class to deal with file in OOP way
$file = new \SplFileObject($filePath);

$result = [];
// Extract first line as a fields array
$fields = $this->extractCSVLine($file->fgets());
while (!$file->eof()) {
$line = $this->extractCSVLine($file->fgets());
// Map values from the line into keys extracted before loop.
$result[] = array_combine($fields, $line);
}

return $result;
}

/**
* Extract fields from a line in CSV file
* @param string $line
* @param string $separator
* @return array
*/
protected function extractCSVLine(string $line, string $separator = ','): array
{
$line = explode($separator, $line);
// It is often a good idea to trim spaces when extracting data from text files
$line = array_map('trim', $line);

return $line;
}
}
The above class extracts data from CSV file and returns PHP array ready to be persisted in the database. We will use it in our CoordinatesController.
/**
* Upload coordinates file for a user into the database
* @param Request $request
* @return \Illuminate\Http\JsonResponse
*/
public function uploadFile(Request $request)
{
$import = new Coordinates();
$records = $import->extractCSV($request->file('file')->getPathname());
foreach ($records as &$record) {
$record['user_id'] = $request->input('user_id');
}

$success = DB::table('coordinates')->insert($records);

return response()->json(compact('success'));
}
If we run the test, it should be green. We finished the functionality. However, this is a bit naive implementation. In an ideal world, we would receive that CSV file with properly named fields every time. Unfortunately at some point, someone will try to upload an image or make a typo in the field name and our application will break.

What we need is input validation. But before we implement it, let's write a quick unit test. It is a good idea to add some extra tests to cover our code.
php artisan make:test Services/Import/CoordinatesTest --unit
Our test case will be created at tests/Unit/Services/Import/CoordinatesTest.php. Let's add another CSV file at tests/data/coordinates2.csv. We will intentionally swap the fields here and add some extra spaces around values to make sure out service can still properly extract the data.
  longitude ,  latitude
72.8777, 19.0760
79.3832, 43.6532
43.1729 , 22.9068
Our unit test is just a simple version of integration test, as we only need to test a single function - it's input and output.
/**
* Make sure space trimming works properly.
* Make sure fields in the file can be swapped.
*/
public function testExtractCsv()
{
$filePath = base_path('tests/data/coordinates2.csv');

$coordinates = new \App\Services\Import\Coordinates();
$result = $coordinates->extractCSV($filePath);

$this->assertEquals([
['latitude' => 19.0760, 'longitude' => 72.8777],
['latitude' => 43.6532, 'longitude' => 79.3832],
['latitude' => 22.9068, 'longitude' => 43.1729]
], $result);
}
If we run phpunit command in the console now we should see:
PHPUnit 7.1.5 by Sebastian Bergmann and contributors.

.. 2 / 2 (100%)


Time: 779 ms, Memory: 16.00MB

OK (2 tests, 5 assertions)
We can move on to the last step.

Step 6. Validate the input


It's the last step we need to finish our coordinates importing service and the most important one. You can never trust any input from an external client that sends anything to your backend application. It does not matter if you are using a framework or not. You have to make sure that you allow only the expected input.

Laravel has a dedicated place for validation. It's in the Request classes, where we define rules that provided input must obey to be processed. To create a new Request class that will validate our input we can run below comman:
php artisan make:request Coordinates/ImportCsvRequest
If we open app/Http/Requests/Coordinates/ImportCsvRequest.php class and consider what rules we need to apply. If we get something else than we need, we must indicate to our client that we expect something else.

First, we deal with the file. We don't want to process this route if a file is not uploaded, so we add a required rule. We expect it to be sent in the field name file as per our decisions we made in step 1. We also expect it to be a CSV file or properly formatted TXT. We definitely don't want PDF or some image, right? Luckily most common validation rules are provided by Laravel out of the box. We add rules in the array returned by the rules function:
public function authorize()
{
return true;
}

public function rules()
{
return [
'file' => ['required', 'file', 'mimes:csv,txt']
];
}
We set up our authorize method to return true since we don't need to authorize access to this route right now, but if you do need to allow only specific types of users to be able to upload files, you can check for permissions here.

The 3 predefined rules for file field will make sure that our uploaded file is provided and it is of mime type CSV or TXT. Since we want to assign the content of the file to a user we should also make sure the his id number is provided and that it exists in our database.
public function rules()
{
return [
'file' => ['required', 'file', 'mimes:csv,txt'],
'user_id' => ['required', 'integer', 'exists:users,id']
];
}
The exists rule will check the database if the user is available. If you are interested in available rules you can see them all explained in the official documentation.

To add our ImportCsvRequest to our CoordinatesController we have to replace the default Request instance.
public function uploadFile(Request $request) { }
public function uploadFile(ImportCsvRequest $request) { }
Laravel will automatically use the rules and if the request will not fit the requirements, a default message that you can customize will be returned from the API.

We also need to make sure that fields latitude and longitude are provided in the CSV file. We could write a custom validation rule for that, but it is much simpler to just throw an Exception in our service and catch it in the controller to return a proper message.

Let's first generate the exception with Laravel console command:
php artisan make:exception Import/Coordinates/InvalidFieldsException
Add a message that will be returned from your API to the newly generated Exception at app/Exceptions/Import/Coordinates/InvalidFieldsException.php
class InvalidFieldsException extends Exception
{
protected $message = 'Invalid fields provided. Allowed fields are: latitude, longitude';
}
Add another csv file named coordinates3.csv in tests/data folder:
lon    ,lat
72.8777,19.0760
79.3832,43.6532
43.1729,22.9068
And let's create another unit test. In our CoordinatesTest class let's add a function:
/**
* Make sure class throws proper Exception on invalid fields provided
*/
public function testInvalidFieldsException()
{
$this->expectException(InvalidFieldsException::class);
$filePath = base_path('tests/data/coordinates3.csv');

$coordinates = new \App\Services\Import\Coordinates();
$coordinates->extractCSV($filePath);
}
Our new test will fail if we try to run it. Let's add the exception to our Coordinates class:
// ...
// Put first line into field names array
$fields = $this->extractCSVLine($file->fgets());
// Check if proper fields are provided
if (count($fields) != 2 || array_diff($fields, ['latitude', 'longitude']) !== []) {
throw new InvalidFieldsException();
}
while (!$file->eof()) {
// ...

Conclusion


And that's all folks. Quite a bit of code, but we have a fully functional and tested coordinates import available. I hope it will provide you with some insight into how you can use TDD when creating your applications in Laravel.

If you want to tinker with full code, you can clone it from Github repository. If you enjoyed the article or want to know more about Laravel, don't hesitate to leave a comment.



Comments +