[FDE-360] Add stdlib JSON formatter and configure_logging to pylogtracer

.github/workflows/ci-python.yml

@@ -24,3 +24,7 @@ jobs:
         run: |
           . .venv/bin/activate
           make lint-python
+      - name: Run tests
+        run: |
+          . .venv/bin/activate
+          make test-python

Makefile

@@ -31,6 +31,9 @@ lint-go:
 	fi
 	golangci-lint run ./...
 
+test-python:
+	pytest pylogtracer/tests/ -v
+
 lint-python:
 	ruff format pylogtracer --check
 	ruff check pylogtracer

pylogtracer/README.md

@@ -1,65 +1,97 @@
-# PyLogTracer - Structured Logging and Tracing for Python
+# PyLogTracer
 
-## Overview
-PyLogTracer is a Python logging and tracing library that wraps `loguru` for structured logging and integrates with OpenTelemetry to send traces to Grafana Tempo.
+Structured JSON logging and optional OpenTelemetry tracing for Python services.
 
-## Features
-- JSON structured logging via `loguru`
-- OpenTelemetry tracing with automatic span context propagation
-- Supports external tracing backends like Grafana Tempo and Jaeger
-- Configurable via environment variables
+## Core Features (zero dependencies)
 
-## Installation
+- **`JSONLogFormatter`** — stdlib `logging.Formatter` that outputs one JSON object per line, matching the Go zap logger schema
+- **`configure_logging()`** — one-call root logger setup with dev/prod dual mode
 
-### Using uv (recommended)
-```bash
-# Install uv if you haven't already
-curl -LsSf https://astral.sh/uv/install.sh | sh
+## Optional Features
 
-# Clone the repository
-git clone https://github.com/nullify/pylogtracer.git
-cd pylogtracer
+Install extras for additional capabilities:
 
-# Install dependencies
-uv sync
+| Extra | Provides | Dependencies |
+|-------|----------|-------------|
+| `[tracing]` | OpenTelemetry tracer setup | opentelemetry-api, -sdk, -exporter-otlp |
+| `[loguru]` | `StructuredLogger` (loguru-based) | loguru |
+| `[all]` | Everything above | all of the above |
+
+## Installation
 
-# Install in development mode
-uv pip install -e .
+### Core only (recommended for most services)
+
+```bash
+pip install pylogtracer
 ```
 
-### Using pip
+### With tracing support
+
 ```bash
-pip install git+ssh://git@github.com/nullify/pylogtracer.git
+pip install "pylogtracer[tracing]"
 ```
 
-## Development
+### With all extras
 
-### Setup with uv
 ```bash
-# Install development dependencies
-uv sync --extra dev
+pip install "pylogtracer[all]"
+```
 
-# Run tests
-uv run pytest
+## Usage
 
-# Run linting
-uv run ruff check .
+### JSON Structured Logging
 
-# Format code
-uv run ruff format .
+```python
+import logging
+from pylogtracer import configure_logging
+
+# Set up once at service startup
+configure_logging()
+
+logger = logging.getLogger(__name__)
+logger.info("Processing request", extra={"request_id": "abc-123", "tool": "search"})
 ```
 
-### Using traditional pip
+**Production output** (single JSON line):
+```json
+{"timestamp":"2025-06-01T12:34:56.789Z","level":"info","msg":"Processing request","caller":"app.py:8","logger":"__main__","request_id":"abc-123","tool":"search"}
+```
+
+**Development output** (set `DEVELOPMENT_MODE=true`):
+```
+2025-06-01 12:34:56,789 - __main__ - INFO - Processing request
+```
+
+### JSON Field Schema
+
+| Field | Description | Go zap equivalent |
+|-------|-------------|-------------------|
+| `timestamp` | ISO 8601 UTC with `Z` suffix | `zapcore.ISO8601TimeEncoder` |
+| `level` | `debug`, `info`, `warn`, `error`, `fatal` | `zapcore.LowercaseLevelEncoder` |
+| `msg` | Log message | `MessageKey: "msg"` |
+| `caller` | `filename:lineno` | `zapcore.ShortCallerEncoder` |
+| `logger` | Logger name | `NameKey: "logger"` |
+| `stacktrace` | Present on exceptions only | `StacktraceKey: "stacktrace"` |
+
+Any `extra={}` dict entries are promoted to top-level JSON keys.
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `DEVELOPMENT_MODE` | Set to `true` or `1` for human-readable text output | _(JSON mode)_ |
+| `PYTHON_LOG_LEVEL` | Log level (`DEBUG`, `INFO`, `WARNING`, `ERROR`) | `INFO` |
+
+## Development
+
 ```bash
-# Install development dependencies
-pip install -e ".[dev]"
+# Install dev dependencies
+uv sync --all-groups
 
 # Run tests
-pytest
-
-# Run linting
-ruff check .
+make test-python
 
-# Format code
-ruff format .
+# Lint and format
+make lint-python
+make fix-python
 ```

pylogtracer/pylogtracer/__init__.py

@@ -1,93 +1,47 @@
-import os
+"""
+PyLogTracer -- Structured logging and optional tracing for Python services.
 
-import boto3
-from opentelemetry import trace
-from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
-from opentelemetry.sdk.resources import Resource
-from opentelemetry.sdk.trace import TracerProvider
-from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
+Core (zero external dependencies):
+    - ``JSONLogFormatter``: stdlib ``logging.Formatter`` producing Go zap-compatible JSON
+    - ``configure_logging``: one-call root logger setup (JSON in prod, text in dev)
 
-from .logger import structured_logger
-from .tracer import track
+Optional extras (install via ``pip install pylogtracer[tracing]``, etc.):
+    - ``get_structured_logger()``: loguru-based structured logger (requires ``[loguru]``)
+    - ``get_tracer()``: OpenTelemetry tracer (requires ``[tracing]``)
+    - ``initialize_tracer()``: OpenTelemetry tracer setup (requires ``[tracing]``)
+"""
 
-__all__ = ["structured_logger", "track"]
+# Always available -- stdlib only
+from .config import configure_logging
+from .formatter import JSONLogFormatter
 
+__all__ = [
+    "JSONLogFormatter",
+    "configure_logging",
+    "get_structured_logger",
+    "get_tracer",
+]
 
-def get_secret_from_param_store(param_name_env):
-    """Fetch secret from AWS Parameter Store"""
-    param_name = os.getenv(param_name_env)
-    if not param_name:
-        return None
 
-    try:
-        ssm = boto3.client("ssm")
-        response = ssm.get_parameter(Name=param_name, WithDecryption=True)
-        return response["Parameter"]["Value"]
-    except Exception as e:
-        print(f"Failed to fetch parameter {param_name}: {e}")
-        return None
+def get_structured_logger():
+    """Return the loguru-based structured logger.
 
+    Requires the ``[loguru]`` extra::
 
-def create_exporter():
-    """Create appropriate span exporter based on environment configuration"""
-    if endpoint := os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT"):
-        # Check for headers in Parameter Store
-        headers = {}
-        if headers_param := get_secret_from_param_store(
-            "OTEL_EXPORTER_OTLP_HEADERS_NAME"
-        ):
-            try:
-                # Parse headers string "key1=value1,key2=value2" format
-                headers = dict(
-                    pair.split("=", 1)
-                    for pair in headers_param.split(",")
-                    if "=" in pair
-                )
-            except Exception as e:
-                print(f"Failed to parse headers: {e}")
-        try:
-            traces_endpoint = endpoint + "/v1/traces"
+        pip install pylogtracer[loguru]
+    """
+    from .logger import structured_logger
 
-            return OTLPSpanExporter(endpoint=traces_endpoint, headers=headers)
-        except Exception as e:
-            print(f"Failed to create OTLP exporter: {e}")
+    return structured_logger
 
-    # Fall back to console exporter if TRACE_OUTPUT_DEBUG is set
-    if os.getenv("TRACE_OUTPUT_DEBUG"):
-        try:
-            return ConsoleSpanExporter()
-        except Exception as e:
-            print(f"Failed to create console exporter: {e}")
 
-    return None
+def get_tracer():
+    """Return an initialised OpenTelemetry tracer.
 
+    Requires the ``[tracing]`` extra::
 
-def initialize_tracer():
-    """Initialize the OpenTelemetry tracer"""
-    # Create resource with service information
-    resource = Resource.create(
-        {
-            "service.name": os.getenv("OTEL_SERVICE_NAME", "pylogtrace-service"),
-            "service.namespace": os.getenv("OTEL_SERVICE_NAMESPACE", "default"),
-            "deployment.environment": os.getenv(
-                "DEPLOYMENT_ENVIRONMENT", "development"
-            ),
-            "service.version": os.getenv("SERVICE_VERSION", "0.0.0"),
-        }
-    )
+        pip install pylogtracer[tracing]
+    """
+    from .tracing_setup import initialize_tracer
 
-    # Set up tracer provider with resource
-    provider = TracerProvider(resource=resource)
-    trace.set_tracer_provider(provider)
-
-    # # Configure exporter
-    if exporter := create_exporter():
-        provider.add_span_processor(BatchSpanProcessor(exporter))
-    else:
-        structured_logger.warn("No exporter created")
-
-    return trace.get_tracer(__name__)
-
-
-# Initialize the tracer
-tracer = initialize_tracer()
+    return initialize_tracer()

pylogtracer/pylogtracer/config.py

@@ -0,0 +1,61 @@
+"""
+Logging configuration for Python services.
+
+Provides ``configure_logging()`` which sets up the root logger with either
+structured JSON output (production) or human-readable text (development).
+"""
+
+import logging
+import os
+import sys
+
+from .formatter import JSONLogFormatter
+
+
+def configure_logging(
+    log_level: str = "INFO",
+    enable_http_debug: bool = False,
+    suppress_langfuse: bool = True,
+) -> None:
+    """Configure the root logger for service execution.
+
+    In production (default) logs are emitted as structured JSON matching
+    the Go zap schema.  Set the ``DEVELOPMENT_MODE`` env var to ``true``
+    or ``1`` to fall back to human-readable text output.
+
+    All output goes to stderr so that processes can write structured data
+    to stdout without corruption.
+
+    Args:
+        log_level: Default log level (overridden by ``PYTHON_LOG_LEVEL`` env var).
+        enable_http_debug: Enable debug logging for urllib3/requests.
+        suppress_langfuse: Suppress verbose Langfuse logging.
+    """
+    level_str = os.getenv("PYTHON_LOG_LEVEL", log_level).upper()
+    level = logging.getLevelNamesMapping().get(level_str, logging.INFO)
+
+    dev_mode = os.getenv("DEVELOPMENT_MODE", "").lower() in ("true", "1")
+
+    # Clear any previously-configured handlers so behaviour is deterministic.
+    root = logging.getLogger()
+    root.handlers.clear()
+    root.setLevel(level)
+
+    handler = logging.StreamHandler(sys.stderr)
+    handler.setLevel(level)
+
+    if dev_mode:
+        handler.setFormatter(
+            logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
+        )
+    else:
+        handler.setFormatter(JSONLogFormatter())
+
+    root.addHandler(handler)
+
+    if suppress_langfuse:
+        logging.getLogger("langfuse").setLevel(logging.ERROR)
+
+    if enable_http_debug:
+        logging.getLogger("urllib3").setLevel(logging.DEBUG)
+        logging.getLogger("requests").setLevel(logging.DEBUG)