Writing tests is an integral part of the developer’s work sooner or later. Most of them start with unit tests, although some people start their adventure with high-level tests, e.g. written in Behat. Unfortunately, there are times when programmers get stuck, sticking to what they learned in the beginning, thus ignoring the range of advantages of a full scope of testing.
Today we will discuss the topic of development tests – their types and principles – complementing it with a subjective opinion.
Let’s start from the beginning
Tests are called micro-programs which, using the application’s code (or the application itself), check whether the code meets the requirements specified in these tests. The leading specialized libraries in PHP to create tests are PHPSpec, PHPUnit, and Behat.
There are many theories about how to write tests well. Some programmers believe that we should test every application element, even the simplest one. Another group will say that the best method of writing tests is to use the Test Driven Development methodology, which consists of three stages in a loop:
- Creating comprehensive test scenarios
- Implementation of the functionalities until all tests turn green
- Code refactoring, performance fixes, etc.
Another group of programmers believes that the tests should be written by a specially trained person, i.e., the Automatic Tester.
All these groups have one thing in common: they are all right.
After a few years of writing tests, we can say that depending on the situation, one solution may be better than the other or vice versa. Most often, the method of creating tests consists of such factors as:
- Developers’ knowledge of writing tests
- Goals set by the leader of the development team
- The business conditions
- Mistakes of decision-makers resulting from the lack of appropriate competences
Decisions about how to write tests should be made at the start of the project. Then, unless the team finds a better methodology, each developer should write tests based on the rules lasting in the project.
Tests characteristics
You can write many different types of tests to verify that your application is working as intended. Therefore, the most important thing is to verify that business requirements have been met. For this purpose, there may be used four types of tests.
Unit testing
The most basic tests are unit tests. They are tightly integrated with the code. Their task is to verify the correct operation of the code unit. What may be this ‘unit’? Most often, it is a method, although there is nothing to prevent it from being one class or even a set of classes.
The most important part of unit tests is that they run code completely isolated from the outside world. It means you shouldn’t allow the code to communicate with the file system, databases, and other external systems. As a result, unit tests are ultra-fast, based only on memory and CPU.
You don’t need to run the application (or any part of it) to run unit tests. The only thing that needs to be taken care of is properly selecting values transferred to the tested methods.
The most popular libraries for creating unit tests are PHPSpec and PHPUnit.
Integration testing
Integration testing, similar to unit testing, works across the code space. These tests don’t need a running application instance to run correctly.
The main difference between unit and integration testing is integrating your code with the outside world. So there is a possibility of communication with the file system, database services, etc. Most often, before running a single test, you will need to generate the so-called fixtures, i.e. a set of initial data on the systems you will reach during the test execution.
Due to the need to communicate with external resources, integration testing is slower than unit testing.
The library to create integration tests is the PHPUnit.
Functional testing
From this level, we stop talking about code testing – instead, we talk about application testing. It means that while the test is running, you need to run an instance of your application (sometimes, it is done by starting the application kernel while a test is running).
A functional test verifies the operation of a single and as simple as possible scenario (in terms of functionality). Good examples may be testing the registration form, logging in, adding a product to the basket, or entering the resource page to which you don’t have access. Remember that you can also test the API in addition to classic application testing.
There is a rule saying that the verification of the application’s operation should also be functional, i.e., you add a new resource via API, then, to verify that this resource has been added correctly, you should ask the endpoint returning information about this resource. However, it can be very restrictive, as you only sometimes need to have an endpoint that returns information about resources.
As functional tests need a working part of the application, at the very beginning of the test, you are obliged to create fixtures that will make this part of the application work properly.
Due to the running application, functional tests are usually much slower in operation than integration tests.
To create functional tests, there are used PHPUnit and Behat libraries.
End-to-end testing
End-to-end tests are basically very similar to functional tests – they test a selected application fragment. End-to-end tests can cover testing both the application and the API.
The scope of the test scenario makes end-to-end testing different from functional ones. For example, while functional testing tests one simple scenario, end-to-end tests may test a specific process flow inside the application. Thus, the end-to-end test can cover the whole process of user registration, verification, and log in. Another example is testing the purchasing process from the moment the product is placed in the basket until the thank you page appears. You can also test the process on the API side by calling specific endpoints one by one.
Since you operate in a broader scope within the application, preparing much more extensive fixtures may be necessary. Additionally, before running the test, you may need to initiate specific processes inside the application, e.g. index the data prepared in fixtures to the ElasticSearch index. All this makes end-to-end tests undoubtedly one of the slowest tests.
To create end-to-end tests, there are used PHPUnit and Behat libraries.
Testing pyramid
As you may already see, each of the above characteristics includes information about the test’s speed. Unfortunately, the higher the testing level, the longer it takes to complete a single test. While several thousand unit tests should be performed within one or two seconds, one end-to-end test can be performed in even several seconds.
It brings to mind that it will be a relatively poor idea to accurately cover the application with end-to-end tests because the test execution time will be so long that the developer will lose interest in running them. As common sense, you should distribute the tests inspiring yourself with the Testing Pyramid.
The Testing Pyramid shows the optimal (in terms of execution time) distribution of tests you should maintain in the business application. Maintaining this distribution is a challenging task. However, there is a principle that should help you – if you could test a specific requirement on several different levels (unit, integration, etc.), you should always choose a lower level of tests.
The AAA pattern
Each of the tests mentioned above is different from the others. However, there is something that each of them has in common – it is the 3xA principle, i.e. Arrange, Act and Assert.
This rule says that each test should consist of three consecutive sections:
- Arrange – This is the part of the test for all preparation that should be done before the test. For example, here, you clean the redundant data generated by the previous run or generate the fixtures needed for the next run.
- Act – In this part, the test part of the code or application is run. It is often associated with calling a single method. However, nothing prevents it from calling several methods or HTTP requests.
- Assert – is the part used to verify the data returned by the Act section. Since we like writing assertions very much, this is usually the most comprehensive part of the test 🙂
It is a good practice that by looking at the test, you can easily define the boundaries between each section mentioned above.
Final thoughts by the author
A few years of writing tests allowed me to form my opinion on some very popular issues that appear in discussions from time to time. If you have a different opinion on the following points – no problem. However, if you are at the stage of searching – I encourage you to read my thoughts.
Automated testing will never replace acceptance testing
Acceptance tests are manual tests that are carried out on the side of the entity receiving the application from the development team. They have a different purpose than development tests. That is why you should not give up on them. Remember that the trained eye of the tester (or the demanding eye of the customer) can pay attention to things that you will not catch in development tests.
Less is faster?
When you create software, you take responsibility for its correct operation, ensuring that everything is in perfect condition. You are writing tests for this purpose, which, unfortunately, is reflected in task estimations. However, you should not succumb to any pressure from the client to give up creating tests, as this will “shorten the time it takes to complete the task.” The client will see a smaller bill at the beginning of the work. However, looking at this issue in the long term, the tests will allow you to eliminate errors that are difficult to reproduce or find inconsistent business requirements, and most importantly – we will find out the bugs from tests, not from the client.
You decide what to test harder and less
Tests should not be a sad duty! You should treat them more as a tool that can bring you value. During development, you know what will give you real benefits. Succumbing to any trend related to what and how to test – usually does not pay off.
Why don’t I like 100% coverage?
Code coverage is a cool stat that can show us whether (or not) we are on a good course over time. Misusing this statistic, i.e. trying to achieve 100% coverage with e.g. unit tests, may result in specs written for Doctrine Query Builder.
Well-written scripts are essential
One of the most important things for me when writing tests is to prepare an appropriate set of test scenarios. If you can’t name any of the scenarios you have noticed, it means that the tested code still needs some work. It may most often mean that you need to isolate the code here and there into separate classes.