========== Developing ========== Developer Environment --------------------- To develop, we suggest using Python `virtual environments `__ together with ``pip`` and steered by `nox `__. Once the virtual environment is activated and you have `SSH keys setup with GitHub `__, clone the repo from GitHub .. code-block:: console git clone git@github.com:scikit-hep/pyhf and install all necessary packages for development .. code-block:: console python -m pip install --upgrade --editable . --group dev Then setup the Git pre-commit hooks with `prek `__ by running .. code-block:: console prek install inside of the virtual environment. `pre-commit.ci `__ keeps the pre-commit hooks updated through time, so the hooks will automatically be updated when you run ``prek`` locally after the hook revisions were updated. It is then suggested that you use ``nox`` to actually run all development operations in "sessions" defined in ``noxfile.py``. To list all of the available sessions run .. code-block:: console nox --list Linting ------- Linting and code formatting is handled by ``prek``. To run the linting either run ``prek`` .. code-block:: console prek run --all-files or use ``nox`` .. code-block:: console nox --session lint Testing ------- Writing tests ~~~~~~~~~~~~~ Data Files ^^^^^^^^^^ A function-scoped fixture called ``datadir`` exists for a given test module which will automatically copy files from the associated test modules data directory into a temporary directory for the given test execution. That is, for example, if a test was defined in ``test_schema.py``, then data files located in ``test_schema/`` will be copied to a temporary directory whose path is made available by the ``datadir`` fixture. Therefore, one can do: .. code-block:: python def test_patchset(datadir): data_file = open(datadir.join("test.txt"), encoding="utf-8") ... which will load the copy of ``text.txt`` in the temporary directory. This also works for parameterizations as this will effectively sandbox the file modifications made. Running with pytest ~~~~~~~~~~~~~~~~~~~ To run the test suite in full, from the top level of the repository run .. code-block:: console pytest More practically for most local testing you will not want to test the benchmarks, contrib module, or notebooks, and so instead to test the core codebase a developer can run .. code-block:: console nox --session tests --python 3.14 Contrib module matplotlib image tests ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To run the visualization tests for the ``contrib`` module with the ``pytest-mpl`` ``pytest`` plugin run .. code-block:: console nox --session tests --python 3.14 -- contrib If the image files need to be regenerated, run the tests with the ``--mpl-generate-path=tests/contrib/baseline`` option or just run .. code-block:: console nox --session regenerate Doctest ^^^^^^^ ``pyhf``'s configuration of ``pytest`` will automatically run ``doctest`` on all the modules when the full test suite is run. To run ``doctest`` on an individual module or file just run ``pytest`` on its path. For example, to run ``doctest`` on the JAX backend run .. code-block:: console pytest src/pyhf/tensor/jax_backend.py Coverage ~~~~~~~~ To measure coverage for the codebase run the tests under ``coverage`` with .. code-block:: console coverage run --module pytest or pass ``coverage`` as a positional argument to the ``nox`` ``tests`` session .. code-block:: console nox --session tests --python 3.14 -- coverage Coverage Report ^^^^^^^^^^^^^^^ To generate a coverage report after running the tests under ``coverage`` run .. code-block:: console coverage or to also generate XML and HTML versions of the report run the coverage ``nox`` session .. code-block:: console nox --session coverage Documentation ------------- To build the docs run .. code-block:: console nox --session docs To view the built docs locally, open the resulting ``docs/_build/html/index.html`` file in a web browser or run .. code-block:: console nox --session docs -- serve Publishing ---------- Publishing to TestPyPI_ and PyPI_ is automated through the `PyPA's PyPI publish GitHub Action `__ and the ``pyhf`` `Prepare release`_ and `Tag release`_ GitHub Actions workflows. Release Checklist ~~~~~~~~~~~~~~~~~ As part of the release process a checklist is required to be completed to make sure steps aren't missed. There is a GitHub Issue template for this that the maintainer in charge of the release should step through and update if needed. Preparing a Release ~~~~~~~~~~~~~~~~~~~ A release is prepared by a maintainer running the `Prepare release`_ GitHub Actions workflow through workflow dispatch. The maintainer needs to: * Select the branch to release from (``main`` or a ``release/vX.Y.x`` release branch). * Input the version of the release (e.g. ``1.2.3`` or ``1.2.3rc1``). The workflow validates that the version is newer than the current version on the selected branch and opens a release preparation pull request that bumps the version of all files defined in `tbump.toml `__ to it. The pull request serves as the release dry run. The maintainer should verify the new version and the diff of the bumped files and let the CI validate the changes before merging. .. note:: The version validation is relative to the version recorded in ``tbump.toml``, which merging a release preparation pull request updates. If a release is abandoned after its release preparation pull request has been merged, but before the release tag has been created, revert the release preparation pull request to be able to prepare a release with a lower version (e.g. a release candidate of the abandoned release). Tagging a Release ~~~~~~~~~~~~~~~~~ After the release preparation pull request has been merged, a maintainer runs the `Tag release`_ GitHub Actions workflow through workflow dispatch on the release branch. The workflow requires approval through the ``release-tag`` GitHub Actions environment, and then creates an annotated tag for the version defined in ``tbump.toml`` and pushes the tag to the release branch. The ``release-prepare`` and ``release-tag`` `GitHub Actions environments `__ must be configured in the repository settings with the maintainers as required reviewers and with the deployment branches restricted to ``main`` and ``release/v*``, as GitHub creates a referenced environment without any protection rules. The ``ACCESS_TOKEN`` secret used to push the release preparation pull request and the release tag is stored as an environment secret in both environments, not as a repository level secret, so that only workflow runs approved by the required reviewers can access it. Additionally, every deployment workflow approval is now recorded in the environment's deployment history which gives an audit history. If the release workflows are not available on the release branch (e.g. historic release branches) a maintainer can perform the same steps locally by bumping the version of the files .. code-block:: console tbump --non-interactive --only-patch X.Y.Z and, after the pull request with these changes has been merged into the release branch, creating and pushing the release tag .. code-block:: console git tag --annotate vX.Y.Z --message "pyhf vX.Y.Z" git push origin vX.Y.Z Release Branches ~~~~~~~~~~~~~~~~ Each minor release series has a corresponding release branch, named ``release/vX.Y.x`` (e.g. ``release/v1.2.x``), so that patch releases for the series can be made after development on ``main`` has moved on to the next release series. After a minor or major release has been tagged, a maintainer can create the release branch from the release tag and push it to the repository .. code-block:: console git fetch origin git branch release/vX.Y.x vX.Y.0 git push origin release/vX.Y.x Creating the release branch from the release tag makes the tag reachable from the branch, which ``hatch-vcs`` requires to correctly derive the release series versions for the distributions built from the branch (e.g. dev versions for untagged commits). As the ``tbump.toml`` on the release branch records the latest release of its release series, the version validation of a patch release prepared from the branch is automatically scoped to the release series. Patch releases follow the same release procedure as all other releases, with the ``release/vX.Y.x`` branch selected when running the `Prepare release`_ and `Tag release`_ workflows, and with the changes for the patch release landing on the release branch as backports of pull requests merged into ``main``. Deployment ~~~~~~~~~~ The push of a tag to the repository will trigger a build of a sdist and wheel, and then the deployment of them to TestPyPI_. TestPyPI ^^^^^^^^ ``pyhf`` tests packaging and distribution by publishing to TestPyPI_ in advance of releases. Installation of the latest test release from TestPyPI can be tested by first installing ``pyhf`` normally, to ensure all dependencies are installed from PyPI, and then upgrading ``pyhf`` to a test release from TestPyPI .. code-block:: console python -m pip install pyhf python -m pip install --upgrade --extra-index-url https://test.pypi.org/simple/ --pre pyhf .. note:: This adds TestPyPI as `an additional package index to search `__ when installing. PyPI will still be the default package index ``pip`` will attempt to install from for all dependencies, but if a package has a release on TestPyPI that is a more recent release then the package will be installed from TestPyPI instead. Note that dev releases are considered pre-releases, so ``0.1.2`` is a "newer" release than ``0.1.2.dev3``. PyPI ^^^^ Once the TestPyPI deployment has been examined, installed, and tested locally by the maintainers final deployment to PyPI_ can be done by creating a GitHub Release: #. From the ``pyhf`` `GitHub releases page `__ select the `"Draft a new release" `__ button. #. Select the release tag that was just pushed, and set the release title to be the tag (e.g. ``v1.2.3``). #. Use the "Auto-generate release notes" button to generate a skeleton of the release notes and then augment them with the prepared release notes the release maintainer has written. #. Select "This is a pre-release" if the release is a release candidate. #. Select "Create a discussion for this release" if the release is a stable release. #. Select "Publish release". Once the release has been published to GitHub, the publishing workflow will build a sdist and wheel, and then deploy them to PyPI_. Context Files and Archive Metadata ---------------------------------- The ``.zenodo.json`` file has the version number automatically updated through ``tbump``, though its additional metadata should be checked periodically by the dev team (probably every release). .. _Prepare release: https://github.com/scikit-hep/pyhf/actions/workflows/release-prepare.yml .. _Tag release: https://github.com/scikit-hep/pyhf/actions/workflows/release-tag.yml .. _PyPI: https://pypi.org/project/pyhf/ .. _TestPyPI: https://test.pypi.org/project/pyhf/