Parametrizing Test Functions with Pytest

I will first create a class that is used for representing the Zabbix protocol flags (zabbix_flags.py):

from enum import IntFlag, auto


class ZabbixFlags(IntFlag):
    ZABBIX_PROTOCOL = auto()
    COMPRESSION = auto()
    LARGE_PACKET = auto()

Subclassing enum.IntFlag is very suitable for this purpose, and enum.auto() can be used to represent the flag values.

I have four test functions to check that the flag values work as expected (test_zabbix_flags.py):

from zabbix_flags import ZabbixFlags


def test_flag_zabbix_protocol():
    assert ZabbixFlags(0x01) == ZabbixFlags.ZABBIX_PROTOCOL


def test_flag_compression():
    assert ZabbixFlags(0x02) == ZabbixFlags.COMPRESSION


def test_flag_large_packet():
    assert ZabbixFlags(0x04) == ZabbixFlags.LARGE_PACKET


def test_all_flags():
    assert ZabbixFlags(0x07) == ZabbixFlags.ZABBIX_PROTOCOL | ZabbixFlags.COMPRESSION | ZabbixFlags.LARGE_PACKET

Let’s create a virtual environment and install pytest there:

markku@devel:~/devel/intflag_pytest$ python3 -m venv venv
markku@devel:~/devel/intflag_pytest$ . venv/bin/activate
(venv) markku@devel:~/devel/intflag_pytest$ pip install -U pip wheel
...
Successfully installed pip-23.1.2 wheel-0.40.0
(venv) markku@devel:~/devel/intflag_pytest$ pip install pytest
...
Successfully installed exceptiongroup-1.1.1 iniconfig-2.0.0 packaging-23.1 pluggy-1.0.0 pytest-7.3.1 tomli-2.0.1
(venv) markku@devel:~/devel/intflag_pytest$

And run the tests:

(venv) markku@devel:~/devel/intflag_pytest$ pytest
========================== test session starts ===========================
platform linux -- Python 3.9.2, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/markku/devel/intflag_pytest
collected 4 items

test_zabbix_flags.py ....                                          [100%]

=========================== 4 passed in 0.01s ============================
(venv) markku@devel:~/devel/intflag_pytest$

Great success: all four tests passed.

Since the test functions are very similar to each other, I can create a new test function to replace the other tests and parametrize it (test_zabbix_flags_param.py):

import pytest

from zabbix_flags import ZabbixFlags


@pytest.mark.parametrize(
    ["int_flags", "zabbix_flags"],
    [
        (0x01, ZabbixFlags.ZABBIX_PROTOCOL),
        (0x02, ZabbixFlags.COMPRESSION),
        (0x04, ZabbixFlags.LARGE_PACKET),
        (0x07, ZabbixFlags.ZABBIX_PROTOCOL | ZabbixFlags.COMPRESSION | ZabbixFlags.LARGE_PACKET),
    ],
)
def test_zabbix_flags(int_flags, zabbix_flags):
    assert ZabbixFlags(int_flags) == zabbix_flags

Let’s run all the tests again, this time with increased verbosity:

(venv) markku@devel:~/devel/intflag_pytest$ pytest -v
============================ test session starts =============================
platform linux -- Python 3.9.2, pytest-7.3.1, pluggy-1.0.0 -- /home/markku/devel/intflag_pytest/venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/markku/devel/intflag_pytest
collected 8 items

test_zabbix_flags.py::test_flag_zabbix_protocol PASSED                 [ 12%]
test_zabbix_flags.py::test_flag_compression PASSED                     [ 25%]
test_zabbix_flags.py::test_flag_large_packet PASSED                    [ 37%]
test_zabbix_flags.py::test_all_flags PASSED                            [ 50%]
test_zabbix_flags_param.py::test_zabbix_flags[1-ZabbixFlags.ZABBIX_PROTOCOL]
PASSED                                                                 [ 62%]
test_zabbix_flags_param.py::test_zabbix_flags[2-ZabbixFlags.COMPRESSION]
PASSED                                                                 [ 75%]
test_zabbix_flags_param.py::test_zabbix_flags[4-ZabbixFlags.LARGE_PACKET]
PASSED                                                                 [ 87%]
test_zabbix_flags_param.py::test_zabbix_flags[7-
ZabbixFlags.LARGE_PACKET|COMPRESSION|ZABBIX_PROTOCOL] PASSED           [100%]

============================= 8 passed in 0.02s ==============================
(venv) markku@devel:~/devel/intflag_pytest$

The -v option in pytest causes it to show all the test cases, and we can see that pytest run the parametrized test function with the listed parameters.

Since the parametrized test was successful I can delete test_zabbix_flags.py.

Key points when using parametrized test functions:

  • The first argument to the @pytest.mark.parametrize() decorator is list of parameter names (here “int_flags” and “zabbix_flags”)
  • The second argument is list of tuples, where each tuple consists of as many items (parameter values) as there were parameter names in the first argument
    • (To be exact, it doesn’t have to be a list of tuples, it can be list of lists or however you like it, but using list in the outer level and tuples in the inner level makes it more readable to me)
  • The parameter names are used in the test function definition
  • The parameter values are then passed to the test function, and the test function is run as many times as there were parameter value tuples provided.

Bonus:

As shown above pytest will use the combinations of the parameter values in the test names by default (“1-ZabbixFlags.ZABBIX_PROTOCOL” and so on). If desired, they can be replaced with custom names using the ids argument (test_zabbix_flags_param.py):

import pytest

from zabbix_flags import ZabbixFlags


@pytest.mark.parametrize(
    ["int_flags", "zabbix_flags"],
    [
        (0x01, ZabbixFlags.ZABBIX_PROTOCOL),
        (0x02, ZabbixFlags.COMPRESSION),
        (0x04, ZabbixFlags.LARGE_PACKET),
        (0x07, ZabbixFlags.ZABBIX_PROTOCOL | ZabbixFlags.COMPRESSION | ZabbixFlags.LARGE_PACKET),
    ],
    ids=[
        "Zabbix protocol",
        "Compression",
        "Large packet",
        "All flags combined",
    ],
)
def test_zabbix_flags(int_flags, zabbix_flags):
    assert ZabbixFlags(int_flags) == zabbix_flags
(venv) markku@devel:~/devel/intflag_pytest$ pytest -v
============================ test session starts =============================
platform linux -- Python 3.9.2, pytest-7.3.1, pluggy-1.0.0 -- /home/markku/devel/intflag_pytest/venv/bin/python3
cachedir: .pytest_cache
rootdir: /home/markku/devel/intflag_pytest
collected 4 items

test_zabbix_flags_param.py::test_zabbix_flags[Zabbix protocol] PASSED    [ 25%]
test_zabbix_flags_param.py::test_zabbix_flags[Compression] PASSED        [ 50%]
test_zabbix_flags_param.py::test_zabbix_flags[Large packet] PASSED       [ 75%]
test_zabbix_flags_param.py::test_zabbix_flags[All flags combined] PASSED [100%]

============================= 4 passed in 0.01s ==============================
(venv) markku@devel:~/devel/intflag_pytest$

Leave a Reply