Packaging and Distributing Python Projects

The Distinction That Shapes Every Decision in This Module

Before touching any packaging tool, it is worth being precise about what you are actually building, because the answer changes almost every decision that follows. An application is something you run — a script, a web service, a CLI tool someone executes directly. A library is something other code imports and calls — requests, pandas, or a small internal utility your team shares across several projects. You have been building applications throughout most of this track. This module is fundamentally about the second case: turning code into something installable and reusable, not just runnable.

APPLICATION                         LIBRARY
- Has a specific entry point         - Exposes an importable API
  (e.g. "python app.py")               (e.g. "import mylib; mylib.do_thing()")
- Dependencies are usually pinned    - Dependencies are usually version RANGES
  to exact versions for reproducibility  (so it doesn't conflict with whatever
                                          else is installed alongside it)
- Rarely published to PyPI            - Often published to PyPI, or at least
                                         installed via a private index/Git URL

The reason this matters here: everything from Part 02 onward — pyproject.toml, building wheels, publishing to PyPI — exists specifically to support the library case, or an application specifically meant to be installed as a reusable command (which Module 44, immediately after this one, builds end to end). If you are only ever going to run python app.py yourself, you genuinely do not need most of this module. The moment someone else needs to pip install your code, or you need to reuse it across multiple projects without copy-pasting files, packaging becomes the right tool.

The src/ Layout vs the Flat Layout

There are two common ways to lay out a Python package's files on disk, and the debate between them shows up in nearly every real-world project's README or contributing guide.

my-project/
├── pyproject.toml
├── README.md
├── mypackage/
│   ├── __init__.py
│   ├── core.py
│   └── utils.py
└── tests/
    └── test_core.py

my-project/
├── pyproject.toml
├── README.md
├── src/
│   └── mypackage/
│       ├── __init__.py
│       ├── core.py
│       └── utils.py
└── tests/
    └── test_core.py

The flat layout is simpler to look at and slightly more common in small scripts and tutorials. The src/ layout is what most modern, professionally maintained Python packages actually use, for one specific, genuinely important reason: it prevents your tests from accidentally importing the local, uninstalled source code instead of the actually-installed package. With a flat layout, running tests from the project root can silently succeed by importing mypackage directly off disk — even if the package was never correctly installed at all, or has a broken pyproject.toml. With the src/ layout, that accidental import is impossible, because mypackage is not on Python's import path unless it was actually installed properly — forcing your tests to exercise the real, installed package the same way a user would encounter it.

The Modern Standard — Superseding setup.py

For most of Python's history, packaging a project meant writing a setup.py file — a genuine Python script, executed at build and install time, that called a function named setup() with a large pile of keyword arguments describing the package. It worked, but it had a real design flaw: because it was executable code, tools could not safely inspect a package's metadata (its name, version, dependencies) without actually running arbitrary Python — a real security and reliability concern.

pyproject.toml is the modern replacement, standardized across the Python packaging ecosystem (formalized in PEP 518 and later PEP 621). It is a plain, static TOML configuration file — no code execution required to read it — that every modern Python packaging tool (pip, build, Poetry, Hatch, and others) understands out of the box.

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "orderkit"
version = "0.3.1"
description = "Utilities for parsing and validating e-commerce order data"
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
    { name = "Your Name", email = "you@example.com" }
]
dependencies = [
    "requests>=2.31,<3.0",
    "python-dateutil>=2.8",
]

[project.optional-dependencies]
dev = ["pytest>=7.0", "black>=24.0", "ruff>=0.4"]

[project.urls]
Homepage = "https://github.com/yourname/orderkit"

Three distinct concerns live in this one file, each worth naming explicitly. The [build-system] table tells installers which tool actually knows how to build this project into a distributable package (here, setuptools) — this section is read before anything else, even before your project's own code is touched. The [project] table is metadata: name, version, description, and crucially, dependencies — every third-party package this project needs to run, with version constraints, replacing what used to live in a separate requirements.txt for a library's own declared dependencies (Module 18 covered requirements.txt for pinning an application's exact environment — this is the equivalent concept for a publishable library's stated, flexible requirements). [project.optional-dependencies] declares extra dependency groups, like packages only needed for running tests, that are not installed by default.

python -m build — What Actually Gets Produced

With a valid pyproject.toml in place, building your project into a distributable package is a single command, using the standard build tool (installed with pip install build).

pip install build
python -m build

# Produces a new dist/ directory:
dist/
├── orderkit-0.3.1-py3-none-any.whl
└── orderkit-0.3.1.tar.gz

Two files come out, and they serve genuinely different purposes.

WHEEL (.whl)
  A pre-built, ready-to-install package. Files are already laid out
  exactly how they'll be installed — pip just unpacks it directly.
  Fast to install. The format most users end up installing.

SDIST (.tar.gz)
  A "source distribution" — the raw source files plus enough metadata
  to build a wheel from them. Slower to install (pip has to build a
  wheel from it first), but necessary as a fallback for platforms or
  package configurations a pre-built wheel wasn't produced for.

In practice, when you run pip install orderkit, pip prefers to download and install the wheel directly if one compatible with your system is available — it is faster and requires no build step on the installing machine at all. The sdist exists as the complete, buildable source, and as the fallback when no matching pre-built wheel exists (common for packages that include compiled code for a specific operating system and Python version).

Semantic Versioning — MAJOR.MINOR.PATCH

The version number you put in pyproject.toml is not just a label — for a published package, other projects' dependency constraints (like requests>=2.31,<3.0 from Part 03) rely on it meaning something consistent. The near-universal convention across the Python ecosystem, and most of the software industry, is semantic versioning (semver): three numbers, MAJOR.MINOR.PATCH, each incremented for a specific reason.

MAJOR — incremented for a BREAKING change.
        Code written against the old major version may stop working.
        1.4.2 -> 2.0.0

MINOR — incremented for a new, backward-COMPATIBLE feature.
        Existing code keeps working; new functionality is added.
        1.4.2 -> 1.5.0

PATCH — incremented for a backward-compatible BUG FIX.
        No new features, no breaking changes — just a fix.
        1.4.2 -> 1.4.3

This is what makes a constraint like requests>=2.31,<3.0 meaningful rather than arbitrary: it is trusting that requests' maintainers will only increment the major version (to 3.0) when they genuinely break backward compatibility, so anything within the 2.x range is presumed safe to use without re-testing everything. A package that bumps its major version for a trivial change, or ships a breaking change as a minor version, violates the entire premise other projects are relying on when they set version constraints — which is exactly why teams treat semver discipline as a real commitment, not a formality.

twine upload — And Why You Test on TestPyPI First

PyPI (the Python Package Index) is the official public registry that pip install downloads from by default. Publishing your own package there makes it installable by anyone, anywhere, with a single pip install yourpackage. The standard tool for uploading a built package is twine.

pip install twine
twine upload dist/*

# Prompts for a PyPI username (or __token__) and an API token,
# then uploads both the wheel and sdist from Part 04.

TestPyPI is a completely separate, parallel instance of PyPI that exists specifically for practicing the publish process without consequences — same tooling, same commands, a throwaway environment.

twine upload --repository testpypi dist/*

# Then install FROM TestPyPI to confirm it actually installs and imports correctly:
pip install --index-url https://test.pypi.org/simple/ orderkit

The professional habit worth internalizing: build, upload to TestPyPI, install from TestPyPI into a clean virtual environment, and confirm the package actually imports and works as expected — only then repeat the same twine upload command against the real PyPI. This single extra round-trip catches an enormous fraction of packaging mistakes (a missing dependency, a file that didn't get included, a broken import path) before they become a permanent, undeletable entry on the real index.

Turning a Package Into a Real, Installed Command

Everything so far makes a package importable — import orderkit. A different, extremely useful capability is making part of a package runnable directly from the terminal as its own command, the way pip, black, or pytest themselves work — you do not write python -m black every time; you just type black. This is configured with an entry point in pyproject.toml.

[project.scripts]
orderkit = "orderkit.cli:main"

# This says: after installing this package, create a command called
# "orderkit" that, when run, calls the function "main" inside
# the module "orderkit.cli".

def main():
    print("orderkit CLI running")
    # real argument parsing goes here — covered in full in Module 44

After installing this package (pip install . during development, or pip install orderkit once published), a genuine new command called orderkit becomes available directly in the shell — no python prefix, no remembering which file to run. This is exactly the mechanism Module 44, immediately after this one, relies on to turn a complete CLI tool built with argparse into something installed and runnable as a real, first-class command rather than a script someone has to locate and invoke manually.

Four Copy-Pasted Files at a Raleigh Insurance-Tech Company

Four Misconceptions About Packaging

5 Interview Questions — With Complete Answers

Packaging Mistakes Beginners Make Constantly

Packaging Errors You Will Hit — And Exactly Why