blender-addon-tester 0.10.2

Add-On Tester for Blender using Pytest

blender-addon-tester

Background

The 3D tool, Blender, allows its functionality to be extended, using python. These are called addons.

Typically an addon will be released and it will be working with the version of blender that is on release at that time. However as time pass the, blender releases new versions, adding to and deprecrating parts of the exposed API. After a while it is not uncommon for an addon to stop working.

This was seen a lot when blender made a upgrade move from 2.79 to 2.80. Some parts of the exposed API were radically different and it broke a lot of addons.

A major effort was put into rework thes addons to get them working again, but the exercise merely highlighted that there is no native testing environment that:

• Allows an addon to be tested against multiple versions of blender and multiple OSs (ubuntu, windows and macosx)
• Used an industry standard of testing, i.e. pytest
• Plugs into a continuous integration tool, i.e. Github Actions, Travis CI
• Ability to test a passing addon against the nightly builds, to catch API breaks as close to when they happen
• Code coverage of the addon. Used to assess comprehensivness of the tests.

What does it do

This python module allows command line pytesting to be performed on different versions of blender.

It will download and maintain different versions of blender locally.

It will install the addon under test into a location accessible by the respective version of blender.

It will execute a series of tests, by default, located in the tests directory. These tests are written in the pytest format.

Once the addon has been completed and the tests have been written, they are checked in to github. They can be run against a continous integration tool. There is currently support for both Github Actions and Travis CI.

Usage

It can be confusing with blender as it has an internal version of python that is different from the system python. blender-addon-tester is install to the system python and is used to call different versions of blender. It is inside this instance of blender that the addon under test gets installed.

blender-addon-tester can be install from pypi:

pip install blender-addon-tester

Then it can be called from any script:

    import blender_addon_tester as BAT
    BAT.test_blender_addon(addon_path=addon, blender_revision=blender_rev)

Once called, it will

• Check to see if the version of blender is presnet in the cache location, if not it will download that version of blender and install it to the cache location, installing all the extra python modules required to enable pytest-ing and coverage.
• It will install the addon to the version of blender
• It will run all the tests, default tests directory, but this can be explictly set. \it will report a pass or failure. This result is written to be capturable by CI tools.
• It will also report to coverage of the test. This coverage can be almalgamated across all versions to get a full idea of testing.

Online continuous integration and code coverage need to be setup explictly.

pytest

Example output of a successful pytest.

============================= test session starts =============================
platform win32 -- Python 3.7.4, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- C:\blender\blender-2.93\blender.exe
cachedir: .pytest_cache
rootdir: C:\blender\blender-fake-addon
plugins: cov-2.8.1
collected 2 items

tests/test_version.py::test_versionID_pass PASSED                         [ 50%]
tests/test_version.py::test_versionID_fail PASSED                         [100%]

========================== 2 passed in 0.20 seconds ===========================

Example of a failing pytest.

_____________________________ test_versionID_pass ______________________________
bpy_module = 'fake_addon'
    def test_versionID_pass(bpy_module):
        expect_version = (1, 0, 1)
        return_version = get_version(bpy_module)
>       assert  expect_version == return_version
E       assert (1, 0, 1) == (0, 0, 1)
E         At index 0 diff: 1 != 0
E         Use -v to get the full diff
tests/test_pytest.py:11: AssertionError
====================== 1 failed, 1 passed in 0.08 seconds ======================

Operational

To see a working addon environment checkout this repo. In the sub directory examples\testing-fake-addon, it contains a dummy addon that that can be sued to verify that the whole enviroment is setup correctly.

cd examples\testing-fake-addon
test_addon_blender.py fake_addon 3.2

However it is better to use this modile with an addon in a different repo. Check out this repo for that example, fake_addon

Projects using blender-addon-tester

fake_addon

blender-import-lwo

ba_io_scene_obj

gmic-blender

Releases

v0.10

Add support for 3.4 Add support for 3.5 Bumped all the guthub actions revs Added support for github actions release

v0.9

Add support for 3.1 Add support for 3.2 Add support for 3.3 Move to python3.10 on CI

v0.8

Needed to updated how the blender packages got fetch from the server (again) Fixed the non default addon directory option, #26 Break out the addon directory setup from the addon install Deprecated 2.79, which removes any dependancy on python3.5 Deprecated 2.80

v0.7

Re do how most recent version of blender is fetched as the format used on the server has changed. Deprecated 2.78.

v0.6

Added PYTEST_ARGS on the config option Need a separate PYTHONPATH for the scripts directory that the addon uses for helper scripts: addon_helper = os.environ.get("ADDON_TEST_HELPER", None)

v0.5

Fix blender rev edit bug

v0.4

Introduce per blender rev addons support

v0.3

New release required to to repo migration

v0.2

Added default cache location if not specified Deleted built in addon from reference blender, if testing new version of addon Updated setup.py to remove unnecessary modules needed for blender-addon-tester to work. Updated documentation

v0.1

Initial checkin to pypi Working against github actions Working against Travis CI