feat: add support for poetry
New python dependencies are introduced by the memory mapping script.
Rather than add another `requirements.txt` utilise poetry. This is a
proper dependency management framework for Python. The two main upsides
of using poetry instead of the traditional requirements.txt are
maintainability and reproducibility.
Poetry provides a proper lock file for pinning dependencies, similar to
npm for JavaScript. This allows for separate environments (i.e. docs,
tools) to be created efficiently, and in a reproducible manner, wherever
the project is deployed. Having dependencies pinned in this manner is a
boon as a security focused project. An additional upside is that we will
receive security updates for dependencies via GitHub's Dependabot.
Change-Id: I5a3c2003769b878a464c8feac0f789e5ecf8d56c
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
diff --git a/docs/getting_started/docs-build.rst b/docs/getting_started/docs-build.rst
index 4a48059..99cba1e 100644
--- a/docs/getting_started/docs-build.rst
+++ b/docs/getting_started/docs-build.rst
@@ -19,36 +19,26 @@
For building a local copy of the |TF-A| documentation you will need:
-- Python 3 (3.5 or later)
+- Python 3 (3.8 or later)
- PlantUML (1.2017.15 or later)
-- Python modules specified in ``docs/requirements.txt``
-
- You can install these with ``pip3`` (the Python Package Installer) by
- passing it the requirements file above (with ``-r``). An optional ``--user``
- argument will install them locally, but you have to add their location to
- $PATH (pip will emit a warning). Alternatively, they can be installed
- globally (but will probably require root privileges).
-
- .. note::
- Although not necessary, it is recommended you use a virtual environment.
- More advanced usage instructions for *pip* are beyond the scope of this
- document but you can refer to the `pip homepage`_ for detailed guides.
-
+- Poetry Python dependency and package manager
+- Python modules specified in ``pyproject.toml``
- Optionally, the `Dia`_ application can be installed if you need to edit
existing ``.dia`` diagram files, or create new ones.
+
+Poetry will handle the creation of a virtual build environment, either creating
+a new environment or re-using one created by the user, and installing all
+dependencies herein. This ensures that the Python environment is isolated from
+your system environment.
+
-An example set of installation commands for Ubuntu follows, assuming that the
-working directory is ``docs``:
+An example set of installation commands for Ubuntu follows:
.. code:: shell
sudo apt install python3 python3-pip plantuml [dia]
- pip3 install [--user] -r requirements.txt
-
-.. note::
- Several other modules will be installed as dependencies. Please review
- the list to ensure that there will be no conflicts with other modules already
- installed in your environment.
+ curl -sSL https://install.python-poetry.org | python3 -
+ poetry install
Building rendered documentation
-------------------------------
@@ -58,7 +48,7 @@
.. code:: shell
- make doc
+ poetry run make doc
Output from the build process will be placed in:
@@ -74,7 +64,14 @@
.. code:: shell
+ poetry run make help
+
- make help
+.. note::
+
+ The ``run`` command used above executes ``make`` in the projects virtual
+ environment. To spawn a shell in this environment, use ``poetry
+ shell``. For other use cases, please see the official `Poetry`_
+ documentation.
Building rendered documentation from a container
------------------------------------------------
@@ -90,7 +87,7 @@
docker run --rm -v $PWD:/TF sphinxdoc/sphinx \
bash -c 'cd /TF && \
- pip3 install plantuml -r ./docs/requirements.txt && make doc'
+ poetry install && poetry run make doc'
The above command fetches the ``sphinxdoc/sphinx`` container from `docker
hub`_, launches the container, installs documentation requirements and finally
@@ -103,9 +100,10 @@
--------------
-*Copyright (c) 2019, Arm Limited. All rights reserved.*
+*Copyright (c) 2019-2023, Arm Limited. All rights reserved.*
.. _Sphinx: http://www.sphinx-doc.org/en/master/
+.. _Poetry: https://python-poetry.org/docs/cli/
.. _pip homepage: https://pip.pypa.io/en/stable/
.. _Dia: https://wiki.gnome.org/Apps/Dia
.. _docker: https://www.docker.com/
diff --git a/docs/getting_started/prerequisites.rst b/docs/getting_started/prerequisites.rst
index 5b49d2e..f30216a 100644
--- a/docs/getting_started/prerequisites.rst
+++ b/docs/getting_started/prerequisites.rst
@@ -98,6 +98,13 @@
Git hooks and helper tools. Without these tools you will need to rely on the
CI for feedback on commit message conformance.
+- Poetry >= 1.3.2
+
+ Required for managing Python dependencies, this will allow you to reliably
+ reproduce a Python environment to build documentation and run analysis tools.
+ Most importantly, it ensures your system environment will not be affected by
+ dependencies in the Python scripts.
+
Package Installation (Linux)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -177,7 +184,7 @@
--------------
-*Copyright (c) 2021-2022, Arm Limited. All rights reserved.*
+*Copyright (c) 2021-2023, Arm Limited. All rights reserved.*
.. _Arm Developer website: https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/downloads
.. _Gerrit Code Review: https://www.gerritcodereview.com/
diff --git a/docs/requirements.in b/docs/requirements.in
deleted file mode 100644
index ae20b7d..0000000
--- a/docs/requirements.in
+++ /dev/null
@@ -1,5 +0,0 @@
-myst-parser==0.18.1
-pip-tools==6.4.0
-sphinx==5.3.0
-sphinx-rtd-theme==1.1.1
-sphinxcontrib-plantuml==0.24.1
diff --git a/docs/requirements.txt b/docs/requirements.txt
deleted file mode 100644
index 1ed78d0..0000000
--- a/docs/requirements.txt
+++ /dev/null
@@ -1,95 +0,0 @@
-#
-# This file is autogenerated by pip-compile with Python 3.8
-# by the following command:
-#
-# pip-compile docs/requirements.in
-#
-alabaster==0.7.12
- # via sphinx
-babel==2.11.0
- # via sphinx
-certifi==2022.12.7
- # via requests
-charset-normalizer==2.1.1
- # via requests
-click==8.1.3
- # via pip-tools
-docutils==0.17.1
- # via
- # myst-parser
- # sphinx
- # sphinx-rtd-theme
-idna==3.4
- # via requests
-imagesize==1.4.1
- # via sphinx
-importlib-metadata==6.0.0
- # via sphinx
-jinja2==3.1.2
- # via
- # myst-parser
- # sphinx
-markdown-it-py==2.1.0
- # via
- # mdit-py-plugins
- # myst-parser
-markupsafe==2.1.1
- # via jinja2
-mdit-py-plugins==0.3.3
- # via myst-parser
-mdurl==0.1.2
- # via markdown-it-py
-myst-parser==0.18.1
- # via -r docs/requirements.in
-packaging==23.0
- # via sphinx
-pep517==0.13.0
- # via pip-tools
-pip-tools==6.4.0
- # via -r docs/requirements.in
-pygments==2.14.0
- # via sphinx
-pytz==2022.7
- # via babel
-pyyaml==6.0
- # via myst-parser
-requests==2.28.1
- # via sphinx
-snowballstemmer==2.2.0
- # via sphinx
-sphinx==5.3.0
- # via
- # -r docs/requirements.in
- # myst-parser
- # sphinx-rtd-theme
- # sphinxcontrib-plantuml
-sphinx-rtd-theme==1.1.1
- # via -r docs/requirements.in
-sphinxcontrib-applehelp==1.0.3
- # via sphinx
-sphinxcontrib-devhelp==1.0.2
- # via sphinx
-sphinxcontrib-htmlhelp==2.0.0
- # via sphinx
-sphinxcontrib-jsmath==1.0.1
- # via sphinx
-sphinxcontrib-plantuml==0.24.1
- # via -r docs/requirements.in
-sphinxcontrib-qthelp==1.0.3
- # via sphinx
-sphinxcontrib-serializinghtml==1.1.5
- # via sphinx
-tomli==2.0.1
- # via pep517
-typing-extensions==4.4.0
- # via myst-parser
-urllib3==1.26.13
- # via requests
-wheel==0.38.4
- # via pip-tools
-zipp==3.11.0
- # via importlib-metadata
-
-# The following packages are considered to be unsafe in a requirements file:
-# pip
-# setuptools