Publishing a LitePolis package to PyPI using Github Actions

The Python Package Index (PyPI) is the standard repository for distributing Python packages, enabling other developers to install your work with a simple pip install. This tutorial walks you through publishing your LitePolis package to the PyPI using pyproject.toml and Github Actions.

Why Automate Package Publishing

Automating the PyPI packaging process with GitHub Actions and PyPI's Trusted Publishers brings about the following benefits:

1. Consistency and Reliability
Automation ensures that your package builds follow the same steps every time, creating a reliable process that produces identical distribution files regardless of who triggers it.

2. Time Efficiency
Once set up, publishing becomes as simple as creating a new release in your repository. The workflow will handle building the source distributions and wheels, running pre-publish tests, and uploading the files to PyPI.

3. Enhanced Security
With PyPI's Trusted Publisher feature, you can authorize GitHub Actions to publish packages directly to PyPI without storing API tokens or passwords as secrets.

4. Version Control Integration
With GitHub Actions, your publishing workflow lives alongside your code, meaning changes to the build process are versioned, publishing history is visible in your repository's actions logs, and the entire process is transparent to contributors.

5. Enhanced Collaboration
Team members can trigger releases without needing PyPI credentials on their machines. With Trusted Publishers, maintainers can authorize specific repositories and workflows to publish, making it easier to share responsibilities without sharing credentials.

Naming Convention

To keep the LitePolis ecosystem easy-to-navigate, ensure that the name of your package adheres to the naming convention followed by LitePolis:

LitePolis-<moduleType>-<module>

• moduleType is the category of the package. Must be one of:
  • ui for user interface packages
  • middleware for business logic or orchestration layers
  • database for database integrations
• module is the name of your specific extention

For example, if you are creating a package that adds support for the StarRocks database, name it

LitePolis-database-StarRocks

This convention helps developers immediately understand what the package does and how it fits into the LitePolis architecture.

Configuring your build backend and project metadata

The pyproject.toml serves as a central place for your project's configuration. It tells the build frontend tool which backend is needed to build your package. Some available options include Hatchling, Setuptools, Flit, or PDM. Below is an example of a common build backend.

[build-system]
requires = [
    "setuptools >= 61.2",
]
build-backend = "setuptools.build_meta"

Configuring metadata

Ensure that your pyproject.toml file has the following, modifying as needed.

[project]
name = "litepolis-database-StarRocks"
version = "0.0.3"
authors = [
    { name = "Your name" },
]
description = "The StarRocks database module for LitePolis"
dependencies = [
    "fastapi",
    "sqlmodel",
]

[project.license]
text = "file: LICENSE"

[project.readme]
file = "README.md"
content-type = "text/markdown"

[project.urls]
Homepage = "https://github.com/myGithubAccount/LitePolis-database-StarRocks"

[tool.setuptools]
include-package-data = false

[tool.setuptools.packages.find]
exclude = [
    "tests",
    "docs",
]
namespaces = false

[project]
The core metadata of your package. PyPI and other tools use this information when building, installing, and displaying your package.

• name is the distribution name of your package. Follow the LitePolis naming convention but use lowercase + dashes.
• version is the current version of you package. Update this before each release.
• authors is the list of author(s). You can choose to include your email.
• description is a short summary of your package that will appear on PyPI.
• dependencies are the runtime packages that your module needs to work. They will be installed automatically when someone installs your package via pip.

[project.license]
Helps users understand how your code can be used or reused.

• text = "file: LICENSE" means the license is stored in a separate file named LICENSE.

[project.readme]
Tells PyPI to use README.md as the long description for your project.

[project.urls]
Helpful links that appear on your PyPI project page.

• Homepage typically points to your GitHub repository, but you can also add Docs, Issues, Changelog, etc.

[tool.setuptools]
Customize how setuptools builds your package

• include-package-data = false means non-code files won't be included in your package.

[tool.setuptools.packages.find]
Controls how setuptools finds your code to include in the package.

• exclude tells setuptools not to include the specified folders.
• namespace = false signals that you are not using namespace packages.

Using PyPI Trusted Publishing

PyPI Trusted Publishing lets you link your GitHub repository to your PyPI project. PyPI will then trust GithHub Actions to publish your package without the need of tokens when you create a new release from GitHub.

Create the Workflow File

GitHub provides a convenient workflow template for publishing packages to PyPI. This template requires minimal configuration and handles most of the heavy-lifting for you.

To get started, navigate to your repository on GitHub and follow these steps:

1. Click on the Actions tab at the top of your repository
2. If you have not created any workflows yet, you will see a page with suggested workflow templates
3. Otherwise, click on "New workflow" to see the available templates

You will now see a page titled "Choose a workflow" with various template options. Choose the workflow template titled "Publish Python Package"

After clicking the "Configure" button, GitHub will generate a workflow YAML file in your repository under the .github/workflows directory, typically named python-publish.yml (or whatever you want to call it). Here's what the default workflow file looks like:

# This workflow will upload a Python Package to PyPI when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

name: Upload Python Package

on:
  release:
    types: [published]

permissions:
  contents: read

jobs:
  release-build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.x"

      - name: Build release distributions
        run: |
          # NOTE: put your own distribution build steps here.
          python -m pip install build
          python -m build

      - name: Upload distributions
        uses: actions/upload-artifact@v4
        with:
          name: release-dists
          path: dist/

  pypi-publish:
    runs-on: ubuntu-latest
    needs:
      - release-build
    permissions:
      # IMPORTANT: this permission is mandatory for trusted publishing
      id-token: write

    # Dedicated environments with protections for publishing are strongly recommended.
    # For more information, see: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#deployment-protection-rules
    environment:
      name: pypi
      # OPTIONAL: uncomment and update to include your PyPI project URL in the deployment status:
      # url: https://pypi.org/p/YOURPROJECT
      #
      # ALTERNATIVE: if your GitHub Release name is the PyPI project version string
      # ALTERNATIVE: exactly, uncomment the following line instead:
      # url: https://pypi.org/project/YOURPROJECT/${{ github.event.release.name }}

    steps:
      - name: Retrieve release distributions
        uses: actions/download-artifact@v4
        with:
          name: release-dists
          path: dist/

      - name: Publish release distributions to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          packages-dir: dist/

Remember to commit your change (adding this YAML workflow file) to the repository.

Configure PyPI

If you have not already, register an account on PyPI at https://pypi.org/manage/account/publishing/ and complete the steps on the page. To create a PyPI project, go to https://pypi.org/manage/account/publishing/, scroll to "Add a new pending publisher" and fill in the following project details.

• PyPI Project Name: Make sure this matches exactly the one set in [project.name] within your pyproject.toml.
• Owner: This should be the GitHub username or organization that owns the repository. It must match the owner shown on GitHub.
• Repository Name: Enter the exact name of your GitHub repository (case-sensitive).
• Workflow Name: This is the name of the .yml you file created for your GitHub Actions workflow (e.g. python-publish.yml)
• Environment Name: Set this to pypi (recommended but optional). Helps isolate and manage the publishing environment within GitHub Actions.

After you have clicked Add, GitHub Actions (on your behalf) will now be a trusted publisher, automatically building your package, authenticating with PyPI via OpenID Connect (OIDC), and publishing your package when you create a new release on GitHub.

Triggering a PyPI Release

Once your workflow is set up, publishing your LitePolis package to PyPI is as simple as creating a release on your GitHub repository. To create a new release from your repository:

1. Click on "Releases" in your repository
2. Click "Draft a new release"
3. Add a tag (e.g. "v1.0.0")
4. Fill in the title and description
5. Click "Publish release"

That's it! Your GitHub Actions workflow will take over, building your LitePolis package and publishing it to PyPI using Trusted Publishing.

🎉 Congrats! You're officially a LitePolis publisher on PyPI!