Python Engineering Practices: From "A Script That Runs" to "Maintainable Software"

Python lets you turn ideas into code quickly. But between "it runs" and "it remains maintainable in production over the long term" lies an entire engineering discipline. Many teams run into trouble not because Python is inadequate, but because they carry a scripting mindset directly into engineering projects: no dependency lockfile, no type checking, no tests, an arbitrary directory structure, and releases performed with scp.

This article explains the key elements of production-grade Python engineering: dependency management, type annotations, project structure, testing, packaging and releases, and code quality. Each is work you will have to do sooner or later, so it is better to do it early.

1. Scripting Mindset vs. Engineering Mindset

First, define the boundary. Applying full engineering rigor to a disposable 200-line data-cleaning script is overengineering. But you should adopt engineering practices as soon as any of the following applies:

• Multiple people will collaborate on and modify the code;
• The code will run for more than three months while requirements continue to change;
• The code must be deployed across testing, staging, and production environments;
• Other services or packages will import the code.

A scripting mindset focuses on "making it work today." An engineering mindset focuses on "making sure it still works six months from now, a new team member can maintain it, and dependency upgrades do not break it." The core difference spans four dimensions: structure, type constraints, test coverage, and dependency locking. Each section below addresses one of these dimensions.

2. Dependency Management: Stop Installing Packages Globally with pip

requirements.txt is the most basic approach, but it has two critical problems: it does not lock transitive dependencies or isolate environments. If your requirements.txt contains fastapi, a new team member installing it three months later may get the latest release, whose API could already be incompatible.

Dependency management has evolved roughly as follows:

1. requirements.txt (basic): It pins only the packages you declare directly, while pip freeze > requirements.txt creates a one-time snapshot of transitive dependencies. It works, but it is difficult to maintain.
2. pip-tools (version locking): Declare direct dependencies in requirements.in, use pip-compile to generate a fully locked requirements.txt, and run pip-sync to align the local environment with the lockfile. It is lightweight and minimally invasive, making it suitable for legacy projects that do not want to change toolchains.
3. poetry / uv (modern): A unified pyproject.toml, lockfile, and virtual environment management in one toolchain.

uv is currently the preferred choice. Written in Rust, it installs and resolves dependencies one to two orders of magnitude faster than poetry. It combines Python version management, virtual environments, package installation, and publishing. A few commands handle everything:

uv init my-project
uv add fastapi pydantic
uv add --dev pytest mypy ruff
uv lock
uv sync

Commit the generated uv.lock to the repository, and anyone can run uv sync to reproduce exactly the same environment, including the Python version. This is the baseline requirement for engineering discipline: reproducible environments.

If your team already uses poetry, there is no need to force a migration. Poetry's poetry.lock provides the same reproducibility. But you should stop using global pip install with the system Python. That workflow belongs to the scripting era.

3. Type Annotations + mypy/pyright: Essential for Large Projects

Python is dynamically typed. That flexibility is one of its strengths, but it can also become a source of failure in large projects. Without typed function signatures, you cannot tell which keys a dictionary contains during refactoring, IDE navigation becomes unreliable, and changing one line can break ten other places.

Type annotations, introduced through the typing module and PEP 484, bring static type checking to Python. The basic syntax is:

from typing import Optional


def get_user(user_id: int) -> Optional[User]:
    if user_id <= 0:
        return None
    return User(id=user_id)

Gradual adoption is essential. Do not expect to annotate an entire legacy project at once. The right approach is:

1. Require type annotations for all new code and enforce the rule in PR reviews;
2. Annotate public APIs and core data models first;
3. Enable mypy's --strict checks incrementally: start with --disallow-untyped-defs, then add --disallow-incomplete-defs, and finally enable strict mode for the entire project.

For tool selection, pyright, including Pylance, is better suited to real-time editor feedback because it is fast and provides clear error messages. mypy is better suited to CI gates because its ecosystem is mature and its configuration is flexible. You can use both: pyright locally for immediate feedback and mypy in CI for strict enforcement. Put the configuration in pyproject.toml:

[tool.mypy]
python_version = "3.12"
strict = true
warn_unused_ignores = true
ignore_missing_imports = true

Type annotations are not decoration. They are a form of "executable documentation": the type system verifies that the documentation remains consistent with the code.

4. Project Structure: src Layout + pyproject.toml

A beginner's Python project often looks like this:

my_project/
├── main.py
├── utils.py
├── config.py
└── data.db

This "flat layout" is manageable for fewer than 100 lines, but files begin to conflict once the project grows beyond a few hundred lines. Production projects should use a src layout:

my-project/
├── pyproject.toml
├── README.md
├── src/
│   └── my_project/
│       ├── __init__.py
│       ├── api/
│       │   ├── __init__.py
│       │   └── routes.py
│       ├── models/
│       │   └── user.py
│       ├── services/
│       │   └── auth.py
│       └── config.py
├── tests/
│   ├── conftest.py
│   ├── test_api.py
│   └── test_services.py
└── .github/workflows/ci.yml

The benefit of a src layout is that it forces you to import the project as an installed package, rather than importing directly from the current directory. This exposes many implicit dependency problems and prevents tests from accidentally using the source directory instead of the installed package.

Centralize all project metadata in pyproject.toml as defined by PEP 517, 518, and 621. Do not scatter it across setup.py, setup.cfg, requirements.txt, MANIFEST.in, and several other files. A single pyproject.toml handles everything:

[project]
name = "my-project"
version = "0.1.0"
description = "An example project"
requires-python = ">=3.12"
dependencies = [
    "fastapi>=0.110",
    "pydantic>=2.0",
]

[project.optional-dependencies]
dev = ["pytest", "mypy", "ruff", "pytest-cov"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.ruff]
line-length = 100
target-version = "py312"

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--strict-markers --cov=my_project"

5. Testing: pytest Is the De Facto Standard

The Python ecosystem reached a consensus on testing long ago: use pytest. It is far more concise than unittest; assertions use plain assert, so there is no need to memorize methods such as self.assertEqual.

def test_create_user():
    user = create_user(name="alice")
    assert user.name == "alice"
    assert user.id > 0

There are three key engineering practices:

First, use fixtures to manage test dependencies. Database connections, test clients, and fake data should all be provided through fixtures in conftest.py. Their scopes should be explicit (scope="session" / "function"), and they should be reusable and composable:

import pytest


@pytest.fixture
def db_session():
    session = SessionLocal()
    yield session
    session.rollback()
    session.close()


def test_create_user(db_session):
    user = create_user(db_session, name="alice")
    assert user.id is not None

Second, monitor coverage, but do not worship it. Run pytest --cov in CI. A target above 80% for core modules is reasonable, but do not write meaningless tests merely to inflate the score. Coverage can tell you "which code was not executed," but it cannot tell you "whether the executed code was correct."

Third, use mocks sparingly. Use real objects whenever practical. For example, test an ORM with an in-memory SQLite database instead of mocking numerous Session methods. Excessive mocking tightly couples tests to implementation details, causing every test to fail after a refactor.

6. Packaging and Releases: pyproject.toml + build

If other projects need to import your project, or if you need to publish it to PyPI or an internal index, you must package it. The modern workflow is:

uv build      # Generate the wheel and sdist under dist/
uv publish    # Publish to PyPI (or a configured internal index)

Under the hood, python -m build generates the artifacts and twine uploads them. The [build-system] section of pyproject.toml declares the build backend, such as hatchling, setuptools, or flit, standardizing the entire build process.

Publishing to an internal index, such as a company's private PyPI, requires only configuring ~/.pypirc or setting the UV_PUBLISH_URL / TWINE_REPOSITORY_URL environment variable in CI. Version numbers should follow SemVer. You can automate versioning with uv version or bump-my-version.

One frequently overlooked detail: before publishing, ensure that the lockfile matches pyproject.toml and that dependencies in pyproject.toml uses flexible version constraints, such as >=0.110 instead of ==0.110.2. Otherwise, your lock will constrain downstream installations unnecessarily.

7. Code Quality: ruff + pre-commit

ruff is one of the biggest advances in the latest generation of Python tooling. Written in Rust, it replaces more than a dozen tools, including flake8 for linting, black for formatting, isort for import sorting, pyupgrade, isort, and pydocstyle, while running so quickly that it is almost unnoticeable. Keep all configuration in pyproject.toml:

[tool.ruff]
line-length = 100
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "SIM", "RET"]
ignore = ["E501"]

[tool.ruff.format]
quote-style = "double"

ruff check .        # Lint
ruff check --fix .  # Automatically fix issues
ruff format .       # Format

A pre-commit hook runs these checks before each commit, preventing basic problems from entering the repository:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.6.0
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.11.0
    hooks:
      - id: mypy

pre-commit install   # Install into the Git hook

Run the complete ruff check + mypy + pytest sequence again in CI, providing another line of defense if local hooks are bypassed.

Summary

Here is an actionable checklist distilled from the seven sections above:

1. Dependencies: Adopt uv, or poetry, with pyproject.toml and a lockfile to make environments reproducible; stop using global pip install.
2. Types: Require type annotations in new code, enable mypy/pyright strict checks incrementally, and start with public APIs.
3. Structure: Use a src layout and centralized pyproject.toml; do not scatter metadata across files.
4. Testing: Use pytest, fixtures, and coverage; limit mocking and enforce 80% coverage for core modules.
5. Quality: Standardize on ruff, with both pre-commit hooks and CI gates.

Engineering discipline is not about showing off technical sophistication. It is insurance for your future self and your team. Code that runs only proves it is correct today; maintainable code can remain correct tomorrow. The distance from a script to an engineered project is the distance between informal conventions and tool-enforced rules. Every successful ruff check, every silent mypy run, and every clean environment restored by uv sync extends the project's useful life.