One of the core parts of your Continuous Integration Pipeline (CI) is often to generate reports that detail the status of unit tests, their coverage and reports covering the static analysis of the codebase.
Usually, CI/CD Pipeline tools such as Jenkins or CircleCI prefer the reports in a specific format, so that they can easily display the contents within their UI. This format is usually the "JUnit" format, which originates from the Java community.
This causes some issues for us in the Python community. Tools such as
which can generate a .txt report are not generated in the JUnit format, and the same goes for the normal test-runner used by
. So how can we then generate these reports, or transform the existing reports into a format that CI/CD tools such as Jenkins or CircleCI will be compatible with?
Generate a Flake8 JUnit Report
Flake8 is a great tool that provides static analysis of your codebase and reports any issues when it comes to formatting, styling, syntax, docstrings or type hints. It's a great way to catch any obvious errors before the code gets pushed into production.
Normally when using the tool in the command line using
, the tool will simply output the report within the terminal. This is not very helpful for us when we want to read the output and display it in the UI of our CI Pipeline.
You could use the
flag to redirect the flake8 output to a file, but that will not be in the JUnit format.
The solution? Use the pip package
you can easily convert the output generated from
into a .xml file in the JUnit format.
Run the commands in the following order:
mkdir -p test-reports/flake8 flake8 . --output-file=test-reports/flake8/flake8.txt flake8_junit test-reports/flake8/flake8.txt test-reports/flake8/flake8_junit.xml
Let's recap what these commands actually do:
- We create a new folder to store our test-reports in case it does not exist yet.
We run the standard
flake8tool but we redirect output to a
We convert our
flake8.txtfile into a
flake8_junit.xmlfile with the correct JUnit format.
Voila, your static analysis is now saved to a format that your Continuous Integration tool will love to work with.
Generate a Python Unittest JUnit Report
Next up is the normal unit test report that you might want to generate to be able to display the results of your test run within your CI/CD tool of choice. Normally this information is only output to the terminal, but you might want to save it in a more structured format.
You could route the output of the
python -m unittest
command by piping it with
| tee unittest.log
but that is still just the text output and it's not in a structured JUnit format that can be displayed within the interface of the tool of choice for your project.
Luckily for us, within this great python community there is always a solution! Install the
package consists of a
test-runner that generates the output as a JUnit (xUnit) .xml file, it's super easy to use and it even comes with special test runners to use with frameworks such as the
Django Web Framework
In the case of a normal python application you would use it in the following manner:
Either simply by calling it in the command line:
python -m xmlrunner discover -t ./tests -o ./test-reports/junit
Or you could also make it part of a python script where you can have some additional flexibility:
if __name__ == '__main__': unittest.main( testRunner=xmlrunner.XMLTestRunner(output='test-reports'), # these make sure that some options that are not applicable # remain hidden from the help menu. failfast=False, buffer=False, catchbreak=False )
Generate JUnit Test Report with Django
Unlike normal python projects, unit tests written in a Django project is normally executed with the
python manage.py test
command which is an internal Django management command.
package comes with a Django TestRunner class out of the box and it's easy to just plug-and-play.
Simply add the following lines to your
file within your Django project:
TEST_RUNNER = 'xmlrunner.extra.djangotestrunner.XMLTestRunner' TEST_OUTPUT_DIR = './test-reports/unittest' TEST_OUTPUT_FILE_NAME = 'unittest.xml'
Each time you run the
python manage.py test
command, it will now use the
and generate a JUnit report at
By following the steps in this article, you should now be able to see both your unit test output, and your flake8 static analysis output within your CI/CD tool of choice.