Common Modules

jyablonski Common Modules is an internal Python library hosted on PyPI that is used throughout various services & applications in the project

---

How It Works

The library provides a collection of helper and utility functions that can be reused across scripts and services throughout the project.

It can be installed as a Python package in any environment using the following command:

pip install jyablonski-common-modules

The library currently includes functions for SQL-related connections and upsert operations, general Python logging, and AWS utilities. I plan to expand its functionality and scope as needed in the future.

Code Layout

The project is broken down like below:

• Directoryjyablonski_common_modules

  • Directoryaws/

    • …
  • Directorygeneral/

    • …
  • Directorylogging/

    • …
  • Directorysql/

    • …
• Directorytests

  • confest.py
  • Directoryunit/

    • …
  • Directoryintegration/

    • …

This source code structure allows for clean and concise import statements, such as:

from jyablonski_common_modules.logging import create_logger
from jyablonski_common_modules.sql import create_sql_engine, write_to_sql_upsert

To enable importing any function from subdirectories like sql or logging, the __init__.py for each subdirectory is configured like so:

from .connection import create_sql_engine
from .writers import write_to_sql_upsert


__all__ = ["create_sql_engine", "write_to_sql_upsert"]

CI / CD

Continuous Integration

The CI workflow runs on every pull request:

• Build & test - A Dockerized test environment runs the full test suite against Postgres to validate the shared SQL and AWS helpers used by downstream services.

Deployment

Once a PR is merged, the deploy pipeline runs:

1. Re-run CI to confirm the merged code is valid on the main branch.
2. Package build - Builds the library distribution and generates platform-specific wheels.
3. PyPI publish - Publishes the new version and wheels to PyPI.

Downstream services can pick up the published package version in their next dependency update.