PyPI PyPI PyPI PyPI

Tom's Pants backends

This is the documentation for my Pants backend, which I use for my own projects - especially managing my own cloud, but also building open-source software.

Please see the documentation for more information!

Kustomize backend for Pants

PyPI

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This backends implements targets for kustomize templates.

kustomize for overlaying state ontop of raw kubernetes files

Planned and missing features

Key/secret/... generation from built artifacts

Targets

There's currently one target.

`kustomize`

A code-generation target for converting a bundle of kubernetes files into a single multi-docuent YAML file with state injected from other Pants targets.

kustomize(
    name="kustomize",
    sources=[
        "deployment.yaml",
        "server.py",
        "service.yaml",
        "namespace.yaml",
		"kustomization.yaml",
    ],
    dependencies=[":bin"],
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`sources`	Resources used by this target	Required
`dependencies`	Targets to package and pass to the build context, as well as bases	`[]`
`decsription`	A description of the target
`tags`	List of tags	`[]`

For dependencies, the builder will replace labels in the kustomization.yaml with the path of the built package.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.4.0 - 2024-09-19

Now targets pants version 2.22.
Add tailor support
Import git to sandbox during expansion to support github downloads
- submodule check seems to fail so it might help to add ?submodules=false to the link to disable it

[0.2.0] - 2023-06-18

Target pants version is now 2.16.0, with support for 2.15.0.

[0.1.2] - 2022-12-06

Add support for specializing how other packages are injected into Kustomize files.

Kubernetes backend for Pants

PyPI

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This backends implements targets for working with Kubernetes clusters using raw YAML.

kubectl for cluster operations

Targets

There's currently three targets for pants-backend-k8s:

`k8s_source`

A file that can be passed into other k8s fields that are not source fields. For example, k8s_object.template.

k8s_source(
    name="namespace.yaml",
    source="namespace.yaml",
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`source`	The raw file	Required
`decsription`	A description of the target
`tags`	List of tags	`[]`

This'll eventually be automated like other rules once a suitable heuristic for generation with tailor is found. PRs welcome!

`k8s_object`

Input for a kubernetes command, either generated via kustomize or via k8s_source.

k8s_object(
    name="k8s",
    description="the chat backend"
    template=[":kustomize"],
    namespace="chat-app",
    cluster="prod",
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`template`	The target to act on	Required
`namespace`	Namespace to target	Optional, will use default kubectl namespace
`cluster`	cluster to target	Optional, will use default kubectl cluster
`decsription`	A description of the target
`tags`	List of tags	`[]`

k8s_object is a generator for kubernetes target parametrized by the potential commands that are available: apply, create, get, describe, replace, and delete.

`k8s_objects`

A collection of kubernetes objects that should be managed together.

k8s_objects(
    name="my-service",
    description="all components of service-x"
    objects=[":namespace", ":deployment"],
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`objects`	k8s_object targets that should be managed	Required
`decsription`	A description of the target
`tags`	List of tags	`[]`

Like k8s_object, k8s_objects is a generator for parametrized targets for the commands that are available: apply, create, get, describe, replace, and delete.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.4.0 - 2024-09-19

Now targets pants version 2.22.
Kubeconfig files are no longer automatically picked up from the host. In order to support scripted provisioning and local configuration files, all targets now take a kubeconfig field. This can point to either kubeconfig target which uses a straight source or a generated, or a host_kubeconfig target which will attempt to load from ~/.kube/config. Both of these also allow you to specify default namespaces, contexts, clusters and users. All these fields can now also be specified on the object, whereas only cluster/context could before.

[0.2.0] - 2023-06-18

Target pants version is now 2.16.0, with support for 2.15.0.

OCI backend for Pants

PyPI

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This is a backend implementing support for building OCI images in pants; running them, and publishing them to container registries. To do this, this plugin uses three different tools:

umoci for manipulating OCI images
runc for exeuction
skopeo for pulling and pushing images

Planned and missing features

Currently there's no support for pulling tags, as that would break determinism
Multi-platform SHA/.sig is untested/unsupported
skopeo doesn't support MacOS, preventing pulling and pushing images.
No "in-container" build steps

Targets

There's six targets currently implemented, of which five are generic:

oci_pull_image
oci_pull_images
oci_image_build
oci_image_empty
oci_build_layer

And one with some special language semantics:

oci_python_image - this is the same as oci_image_build, but will prefer to set the entrypoint to .pex files.

`oci_pull_image`

Pull an image from a repository with a specific digest.

oci_pull_image(
    name="base-python",
    repository="docker.io/library/python",
    sha="b78b777208be08edd8f297035cdfbacddb45170ad778fd643c792ee045187e39"
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`repository`	Fully qualified repository name	Required
`sha`	The digest of the image, minus the @sha: prefix.	Required
`anonymous`	Whether to pull the image anonymously.	`false`
`decsription`	A description of the target
`tags`	List of tags	`[]`

`oci_pull_images`

Pull multiple shas for an image, generating a target for each. In the below example, we'd get the targets :python#slim and :python#buster.

oci_pull_image(
    name="python",
    repository="docker.io/library/python",
    variants={
       "slim": "f8fbb2370c6314c806b2ddbec8d94375987e16bc122379bef979c6fc5e962920",
       "buster": "97c123c899c8c9ca46248f4002ec4173322e0a1086b386efefac163c64967ba2"
    }
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`repository`	Fully qualified repository name	Required
`variants`	Dictionary with local tags to the remote sha	Required
`anonymous`	Whether to pull the image anonymously	`false`
`decsription`	A description of the target
`tags`	List of tags	`[]`

`oci_build_image`

Build an image with the provided packages embedded.

oci_image_build(
    name="my-server",
    base=":python#slim",
    repository="my-registry.example.com/a-namespace/an-image",
    tag="latest",
    packages=[":my_pex"]
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`base`	The base image to use. Matches the `FROM` directive in a Dockerfile	Required
`packages`	Packaged targets to include. The first element will be used as the entrypoint.	`[]`
`repository`	Fully qualified repository name	Required when publishing
`tag`	Remote tag to use	Required when publishing
`decsription`	A description of the target
`tags`	List of tags	`[]`

`oci_python_image`

Build a Python image with the provided packages embedded.

oci_python_image(
    name="my-server",
    base=":python#slim",
    repository="my-registry.example.com/a-namespace/an-image",
	main="/app/server/start.py",
    tag="latest",
    packages=[":my_pex"]
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`base`	The base image to use. Matches the `FROM` directive in a Dockerfile	Required
`packages`	Packaged targets to include. The first element will be used as the entrypoint.	`[]`
`python_main`	The main file to run	The last `.pex` in the dependency list
`repository`	Fully qualified repository name	Required when publishing
`tag`	Remote tag to use	Required when publishing
`decsription`	A description of the target
`tags`	List of tags	`[]`

`oci_image_empty`

An empty base image with no contents at all. This is declared as //:empty automatically, but you can use this to create new targets.

oci_image_empty(
    name="empty",
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`decsription`	A description of the target
`tags`	List of tags	`[]`

`oci_build_layer`

Run an image command, and capture the configured output into a layer artifact, that can be injected into other images. This matches the COPY --from workflows.

oci_build_layer( name="layer" base=[":rust-1-70"], packages=[":files"], env=['RUSTC_OPTS=...'], command=['cd /my-package && cargo build --release'], outputs=['/my-package/target/release/my-package'], )


| Argument      | Meaning                                                                        | Default value                                          |
|---------------|--------------------------------------------------------------------------------|--------------------------------------------------------|
| `name`        | The target name                                                                | Same as any other target, which is the directory name  |
| `packages`    | Packaged targets to include. The first element will be used as the entrypoint. | `[]`                                                   |
| `env`         | Environment variables to set. Does not support interpolation.                  | `[]`                                                   |
| `outputs`     | Paths to capture into the built layer.                                         | `[]`                                                   |
| `exclude`     | Globs to not include in the output.                                            | `[]`                                                   |
| `decsription` | A description of the target                                                    |                                                        |
| `output_path` | The output path during `pants package`                                         | A variant generated from the target name and directory |
| `tags`        | List of tags                                                                   | `[]`                                                   |

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.7.0 - 2024-09-19

Now targets pants version 2.22.

0.6.2 - 2024-09-19

Adds [oci].unsafe_tar_ignore_file_changed flag that will set --warning=no-file-changed when invoking tar

0.6.1 - 2024-03-28

Fix a bug where the plugin would try to publish OCI images without a repository.
Will now default to "latest" as a tag if none is specified.

0.6.0 - 2024-03-21

Adding layers to an image is changing. As a preparatory step, there's now an oci_layer target. Add these to the layers field of an oci_image_build, instead of dependencies. These will process a bit better, and gives more control over what goes into each layer.
Mac support has improved. In order to provide determinism, the gtar (GNU tar) binary has to be available.
Both umoci and skopeo are now supported on Mac M1 and x86_64. This plugin does not support image run steps on Mac still.
When pulling multi-arch images you can now specify os and architecture on the oci_pull_image target
Fix a bug where OCI layer building would fail if no dependencies were specified

0.5.0 - 2023-11-19

Fix a crash when image pulling fails
Add nightly version for umoci with experimental support for Mac
Add v1.13.3 version for skopeo with experimental support for Mac

0.4.0 - 2023-06-18

Improved support for very large layers > 2GB. A lot of layers will now be compressed in transit. This adds some overhead later when injecting them into image, but fixes some crashes inside Pants.
Improved support for multi-stage builds (COPY --from)

There is now support for building artifacts in one container and copying them into a new container. To do this, use oci_build_layer, and configure the output files and directories. The files and data will be inserted verbatim into the downstream container.
Adds support for empty base images (FROM scratch)

This change enables you to use base=["//:empty"] to start from a completely empty container. This can be useful to produce containers with statically linked binaries that require no runtime environment at all.
- To change the target name, set [oci].empty_image_target in pants.toml.

0.3.1 - 2023-03-16

Handle files when building layers

0.3.0 - 2023-03-14

[breaking] Change output format when publishing

0.2.0 - 2023-02-10

[bugfix] Ensure layer tars has no metadata or user information
[bugfix] Properly pass args to runc
[bugfix] Fix log output in package_oci_image
[feature] Add proper metadata where possible
[feature] Add support for anonymous image pulling (anonymous=true)
[feature] Add support for passing image SHA along to Kustomize
[feature] Add python_image_target which will set entrypoint appropriately
[feature] Forward PATH, HOME, XDG_RUNTIME_DIR env variables to publish step to make credHelpers work

[0.1.1] - 2022-11-17

Move to new repo
Compatibility fixes for 2.15.0a0

[0.1.0] - 2022-11-13

Initial release.

MDBook backend for Pants

PyPI

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This provides a tool for building mdbook targets with pants. There is currently a single very simple rule:

md_book(
    name="my-docs",
    sources=["book.toml", "src/*"],
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`sources`	Files included when building the book	`book.toml` and the `src` directory
`decsription`	A description of the target	`""`
`tags`	List of tags	`[]`

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.5.0 - 2024-09-19

Now targets pants version 2.22.

[0.4.1] - 2024-04-12

Will now use x86_64 mdbook binary on arm64, relying on Rosetta.

[0.4.0] - 2024-04-05

Will now also include codegen sources, allowing integration with adhoc_tool and other generators. Note that only file and direct mdbook sources are included.

[0.3.0] - 2023-11-21

Target pants version is now 2.18.0, with support for 2.17.0.
Updated mdbook version to 0.4.35
Now includes support for arm64 on Linux

[0.2.0] - 2023-06-18

Target pants version is now 2.16.0, with support for 2.15.0.

[0.1.3] - 2022-11-18

Raise error if book.toml is not found
Handle all sources in dependencies

[0.1.2] - 2022-11-17

Merge with all other backends
Fix README links

[0.1.1] - 2022-11-12

Fix README.

[0.1.0] - 2022-11-12

Initial release.

Secrets backend for Pants

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This backends implements utilities for handling secrets.

Planned and missing features

Setting and creating secrets via Pants

Targets

`env_secret`

A secret to be read from the environment.

env_secret(
    name="bw_session_key",
    key="BW_SESSION",
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`key`	The environment variable to read.	Required
`decsription`	A description of the target
`tags`	List of tags	`[]`

Goals

`decrypt`

Decrypts and prints a secret.

BW_SESSION="..." pants decrypt //examples/bitwarden:pypi_token
22:29:42.17 [INFO] Completed: Decrypting examples/bitwarden:pypi_token
Secret examples/bitwarden:pypi_token from BitWarden: pypi-...

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.4.0 - 2024-09-19

Now targets pants version 2.22.

[0.3.0] 2023-11-21

Target pants version is now 2.18.0, with support for 2.17.0.

[0.2.0] - 2023-06-18

Target pants version is now 2.16.0, with support for 2.15.0.

0.1.0 - 2023-02-10

Initial release

BitWarden backend for Pants

Warning This plugin is in development. No stability is guaranteed! Contributions welcome.

This backends implements targets for reading BitWarden secrets.

bw - the BitWarden CLI client

Planned and missing features

Setting and creating secrets via Pants

Targets

`bw_item`

Matches one entry in your vault.

bw_item(
    name="pypi",
    id="386c6037-cbdd-4aa3-ba80-9ed6661f751b",
    session_secret=":bw_session_key",
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`id`	Item id used by this target as seen in the address bar	Required
`item_name`	The name in the vault. If ambiguous this will fail. Prefer IDs.
`session_secret`	The secret to use for the BW_SESSION variable.	`env["BW_SESSION"]`
`decsription`	A description of the target
`tags`	List of tags	`[]`

`bw_password`

The password of an item in your vault.

bw_password(
    name="pypi_password",
    item=[":pypi"],
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`item`	The item containing the password.	Required
`decsription`	A description of the target
`tags`	List of tags	`[]`

`bw_field`

A field from an item in your vault. These are the "Custom Fields" at the bottom of an item, not to be confused with attachments.

bw_field(
    name="pypi_token",
	field_name="api_token"
    item=[":pypi"],
)

Argument	Meaning	Default value
`name`	The target name	Same as any other target, which is the directory name
`item`	The item containing the password.	Required
`field_name`	The item containing the password.	Required
`decsription`	A description of the target
`tags`	List of tags	`[]`

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.4.0 - 2024-09-19

Now targets pants version 2.22.

[0.3.0] - 2023-11-21

Target pants version is now 2.18.0, with support for 2.17.0.
Default bitwarden version is now 2023.10.0

[0.2.0] - 2023-06-18

Target pants version is now 2.16.0, with support for 2.15.0.

0.1.0 - 2023-02-10

Initial release

Custom Pants Backends