User’s Guide¶
Compatibility¶
Right now python-tia supports development machines and CI environments running an UNIX operating system only.
How does it work?¶
python-tia support two main use cases. The usage in a local development environment and the usage in a CI environment.
The typical scenario/processing logic in the local development environment use case is as follows:
- The developer installs and configures python-tia.
- The developer changes one or several files.
- The developer calls tia.
- tia detects the runtime environment based on environment variables and sets the file scope change detection strategy to mime.
- The file scope changes are detected using the mime module.
- Changed files are input for semantic module which determines affected pipelines based on pipeline definitions in semantic map.
- For every affected test tool pipeline it is checked if a full-scope or
partial-scope execution has to be triggered or not (based on files
configured in the semantic map).
- Full-scope execution: The corresponding command is executed and the pipeline specific coverage map recreated. The impact map is recreated based on the recreated coverage map.
- Partial-scope execution: Tests which have to be executed are determined based on the changed and configured test code files, the changed and configured production code files and the impact map. Every test is executed separatelly and the coverage map updated on a per test basis (test -> executed production code). The impact map is recreated based on the updated coverage map (updated production code -> tests).
- For every affected analysis tool pipeline the corresponding command is executed considering the changed production code and configuration files only.
The typical scenario/processing logic in the CI environment use case is as follows:
- The developer installs and configures python-tia.
- The developer pushes commits to a git repository branch.
- The triggered CI job calls tia.
- tia detects the runtime environment based on environment variables and sets the file scope change detection strategy to git.
- The file scope changes are detected using the git module.
- Same as above for the “local development environment” use case…
Installation¶
User-level installation:
$ pip3 install --user
Project-level installation (here: with pipenv):
$ cd <project-root>
$ pipenv install python-tia
Configuration¶
Semantic mapping of test tools¶
tia depends on the definition of semantic mappings between tools and files in its configuration file tia.yaml. This section shows an exemplary semantic mapping for a Python package which uses the test runner pytest to execute tests.
Consider a Python package test file structure like follows:
package
tests
conftest.py
fixtures.py
utils.py
test_group1
test_file1.py
test_file2.py
test_group2
test_file1.py
test_file2.py
pytest.ini
A full-scope execution (of all tests) needs to be triggered whenever one of the files pytest.ini, tests/conftest.py, tests/fixtures.py or tests/utils.py has changed.
A partial-scope execution (of the minimal possible subset of tests) is sufficient whenever one of the files in the directories tests/test_group1, tests/test_group2 or any corresponding production code files in the impact map has changed.
A suitable pipeline configuration which executes pytest for the given project structure could be defined with the following semantic mapping.
[pipeline:pytest]
dirs =
- tests/test_group1/
- tests/test_group2/
files =
- pytest.ini
full-scope: True
- tests/utils.py
full-scope: True
- tests/conftest.py
full-scope: True
full-scope-command: <command one would use w.o. python-tia>
partial-scope-command: <command with file and placeholder for test dirs/files>
The default value for the execution scope is partial-scope. Full-scope execution is indicated on a (recursive) directory/file basis with the full-scope option.
Semantic mapping of static analysis tools¶
This section shows exemplary semantic mappings for common static analysis tools in Python application development.
Trigger the static analysis pipeline for hadolint in case either the config file hadolint.yaml or the docker file Dockerfile changes.
[pipeline:hadolint]
files =
- hadolint.yaml
- Dockerfile
command = hadolint --config hadolint.yaml Dockerfile
Trigger the static analysis of the manifest file with check-manifest in case the MANIFEST.in changes.
[pipeline: check-manifest]
file = MANIFEST.in
command = check-manifest
Impact mapping¶
TODO
Coverage mapping¶
TODO
A coverage map descibes which production code is executed by which test code. Independent of the programming language this information is gathered using dynamic analysis. For coverage analysis the tests are executed which executes specific parts of the production code. The coverage information gathered from a tool can vary. In the best case one gets some mapping of single tests vs. language agnostic, granular production code entities like the following:
In case of the Python programming language (the language this package is primarily made for) the best granularity one can think of would be something like the following: