Environment Variables: Doing It Right

2025-08-03 10 min read python
python configuration pydantic

Environment Variables: Doing It Right

Stop hardcoding config. Use environment variables. It's a simple practice that makes your code more secure, more flexible, and easier to deploy.

The idea is simple: anything that might change between environments (development, staging, production) should come from environment variables, not from your code.

I learned this the hard way when I accidentally pushed an AWS secret key to a public GitHub repo. Within 20 minutes, my account was compromised and mining Bitcoin. GitHub caught it and warned me, but the damage was done. That's when I got serious about environment variables.

Why environment variables?

Security: API keys, database passwords, and secrets don't belong in code. If your repo is public (or gets leaked), those secrets are exposed. With env vars, the secrets stay on the server.

Think about it: your codebase might be accessed by dozens of people - contractors, new hires, that intern from last summer. Your production secrets should only be accessible to people who actually need them. Environment variables let you enforce this separation.

Flexibility: Same code runs in development and production with different config. No if ENVIRONMENT == 'production' scattered through your codebase.

I've seen projects where developers have three different database URLs hardcoded with environment checks everywhere. It's a nightmare to maintain and incredibly error-prone. One wrong condition and you're testing against production data. Not fun.

12-Factor App: This is industry standard practice. If you're deploying to any modern platform (Heroku, Vercel, AWS, etc.), they all expect you to configure via environment variables.

The Twelve-Factor App methodology (https://12factor.net/) established this as best practice over a decade ago, and for good reason. It makes your app portable, scalable, and cloud-ready. If you ignore this, you'll fight your deployment platform every step of the way.

The naive way (and its problems)

import os

api_key = os.environ['API_KEY']

This works, but has issues:

  • Crashes if missing: KeyError with no helpful message
  • No type conversion: Everything is a string
  • No defaults: Can't say 'use 8000 if PORT isn't set'
  • No validation: Invalid values aren't caught early

Slightly better:

api_key = os.environ.get('API_KEY')
if not api_key:
    raise ValueError('API_KEY is required')

debug = os.environ.get('DEBUG', 'false').lower() == 'true'
port = int(os.environ.get('PORT', '8000'))

But now you're writing a lot of boilerplate for every variable.

The better way: pydantic-settings

pydantic-settings handles all of this automatically:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # Required - will error if not set
    api_key: str
    database_url: str

    # Optional with defaults
    debug: bool = False
    port: int = 8000
    log_level: str = 'INFO'

    # Complex types work too
    allowed_hosts: list[str] = ['localhost']

settings = Settings()

When you instantiate Settings(), it:

  1. Reads from environment variables (API_KEY, DATABASE_URL, etc.)
  2. Converts types automatically (DEBUG=true becomes the boolean True)
  3. Uses defaults for missing optional values
  4. Raises a clear error if required values are missing
pydantic_settings.ValidationError: 1 validation error for Settings
api_key
  Field required [type=missing, input_value={}, input_type=dict]

Clear errors at startup, not mysterious failures later.

The .env file for local development

In development, you don't want to set env vars manually. Use a .env file:

# .env
API_KEY=dev-secret-key
DATABASE_URL=postgresql://localhost/myapp_dev
DEBUG=true

pydantic-settings reads .env automatically (with python-dotenv installed). Add it to .gitignore so you don't commit secrets:

.env
*.env
.env.*

Create a .env.example with dummy values for documentation:

# .env.example - copy to .env and fill in real values
API_KEY=your-api-key-here
DATABASE_URL=postgresql://user:pass@localhost/dbname
DEBUG=false

This tells new developers what variables they need to set.

Organizing settings

I usually create a single settings.py file:

# app/settings.py
from functools import lru_cache
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    # App
    app_name: str = 'MyApp'
    debug: bool = False

    # Server
    host: str = '0.0.0.0'
    port: int = 8000

    # Database
    database_url: str

    # External APIs
    stripe_api_key: str
    sendgrid_api_key: str | None = None

    class Config:
        env_file = '.env'

@lru_cache
def get_settings() -> Settings:
    return Settings()

settings = get_settings()

The lru_cache ensures settings are only parsed once. Import and use anywhere:

from app.settings import settings

if settings.debug:
    print('Debug mode enabled')

Variable naming conventions

Environment variables are traditionally uppercase:

DATABASE_URL=...
API_KEY=...
DEBUG=...

pydantic-settings handles the conversion. In your code, use lowercase:

class Settings(BaseSettings):
    database_url: str  # Reads from DATABASE_URL
    api_key: str       # Reads from API_KEY
    debug: bool        # Reads from DEBUG

Prefixes for namespacing

If you have many apps on the same server, prefix your variables:

class Settings(BaseSettings):
    api_key: str
    debug: bool = False

    class Config:
        env_prefix = 'MYAPP_'

Now it reads from MYAPP_API_KEY, MYAPP_DEBUG, etc.

Don't put secrets in code, ever

Not even 'temporary' ones. Not even for testing. I've seen:

# TODO: move to env var
API_KEY = 'sk-live-abc123...'

That TODO never gets done, and the key ends up on GitHub.

If you need test credentials, create a separate test .env file or use fake values that won't work in production.

Advanced patterns and gotchas

Handling sensitive data in logs

One common mistake: logging your settings object to debug configuration issues.

# DON'T DO THIS
logger.info(f"Starting app with settings: {settings}")

This will dump all your secrets to your logs. Instead, use Pydantic's Field to mark sensitive values:

from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    api_key: SecretStr = Field(repr=False)
    database_url: SecretStr = Field(repr=False)
    debug: bool = False

settings = Settings()
print(settings)  # Won't show api_key or database_url values

SecretStr also prevents accidentally serializing secrets to JSON or printing them in tracebacks.

Multiple environment files

Different developers might need different local setups. You can support multiple .env files:

class Settings(BaseSettings):
    # ... your fields ...

    class Config:
        env_file = '.env'
        env_file_encoding = 'utf-8'

        # Load from multiple files - later files override earlier ones
        @classmethod
        def customise_sources(cls, init_settings, env_settings, file_secret_settings):
            return (
                init_settings,
                env_settings,
                file_secret_settings,
            )

I usually have:
- .env.example - Template with dummy values (committed to git)
- .env - Local development settings (gitignored)
- .env.local - Personal overrides (gitignored)
- .env.test - Test environment settings (committed to git)

Environment-specific defaults

Sometimes you want different defaults based on environment:

from functools import cached_property

class Settings(BaseSettings):
    environment: str = 'development'
    debug: bool | None = None
    database_pool_size: int | None = None

    @cached_property
    def is_production(self) -> bool:
        return self.environment == 'production'

    @cached_property
    def effective_debug(self) -> bool:
        # Default debug based on environment if not explicitly set
        if self.debug is not None:
            return self.debug
        return self.environment == 'development'

    @cached_property
    def effective_pool_size(self) -> int:
        # Production needs more connections
        if self.database_pool_size is not None:
            return self.database_pool_size
        return 20 if self.is_production else 5

This gives you smart defaults while still allowing explicit overrides.

Complex types and nested configs

pydantic-settings handles complex types beautifully:

from pydantic import HttpUrl, PostgresDsn, validator

class DatabaseSettings(BaseSettings):
    host: str = 'localhost'
    port: int = 5432
    user: str
    password: SecretStr
    database: str

    @property
    def url(self) -> str:
        return f"postgresql://{self.user}:{self.password.get_secret_value()}@{self.host}:{self.port}/{self.database}"

class Settings(BaseSettings):
    # URLs are validated
    api_base_url: HttpUrl

    # JSON strings are parsed
    allowed_origins: list[str] = []

    # Nested settings
    database: DatabaseSettings

    @validator('allowed_origins', pre=True)
    def parse_cors_origins(cls, v):
        if isinstance(v, str):
            return [origin.strip() for origin in v.split(',')]
        return v

# Usage in .env:
# API_BASE_URL=https://api.example.com
# ALLOWED_ORIGINS=https://app.com,https://www.app.com
# DATABASE_HOST=db.example.com
# DATABASE_USER=myuser
# DATABASE_PASSWORD=secret
# DATABASE_DATABASE=mydb

The nested DatabaseSettings automatically reads from DATABASE_* prefixed variables.

Testing with environment variables

In tests, you often want to override settings. There are a few approaches:

import pytest
from app.settings import Settings

# Approach 1: Use monkeypatch (pytest built-in)
def test_with_different_config(monkeypatch):
    monkeypatch.setenv('DATABASE_URL', 'postgresql://localhost/test_db')
    monkeypatch.setenv('DEBUG', 'true')

    settings = Settings()
    assert settings.debug is True

# Approach 2: Create settings directly
def test_with_direct_config():
    settings = Settings(
        database_url='postgresql://localhost/test_db',
        api_key='test-key',
        debug=True
    )
    assert settings.debug is True

# Approach 3: Use a test .env file
@pytest.fixture
def test_settings():
    return Settings(_env_file='.env.test')

def test_with_env_file(test_settings):
    assert test_settings.environment == 'test'

I prefer approach 2 for unit tests (explicit is better) and approach 3 for integration tests.

Dynamic configuration reloading

Most apps don't need this, but if you want settings to reload without restarting:

import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

class SettingsReloader(FileSystemEventHandler):
    def __init__(self, settings_factory):
        self.settings_factory = settings_factory

    def on_modified(self, event):
        if event.src_path.endswith('.env'):
            # Clear the cache and reload
            self.settings_factory.cache_clear()
            new_settings = self.settings_factory()
            print(f"Settings reloaded: debug={new_settings.debug}")

# Usage
@lru_cache
def get_settings() -> Settings:
    return Settings()

# Set up file watching
event_handler = SettingsReloader(get_settings)
observer = Observer()
observer.schedule(event_handler, path='.', recursive=False)
observer.start()

This is useful for development but don't use it in production - config changes should go through proper deployment.

Common mistakes to avoid

Mistake 1: Not failing fast

# BAD: Silent fallback to empty string
api_key = os.environ.get('API_KEY', '')

# Later, mysterious failures when making API calls...

If a required config is missing, fail immediately at startup. pydantic-settings does this automatically.

Mistake 2: Boolean strings

# BAD: This is always True!
debug = bool(os.environ.get('DEBUG', 'false'))
# bool('false') is True because it's a non-empty string

# GOOD: pydantic-settings handles this correctly
# DEBUG=false becomes the boolean False
# DEBUG=0 becomes False
# DEBUG=no becomes False

Mistake 3: Committing .env files

I've seen this dozens of times. Someone commits .env with production secrets. Even if you delete it in the next commit, it's still in git history.

Add to .gitignore immediately:

.env
.env.local
.env.*.local

If you accidentally commit secrets, you need to:
1. Rotate the secrets immediately
2. Use git-filter-branch or BFG Repo-Cleaner to remove from history
3. Force push (coordinate with team)
4. Consider the secrets compromised forever

Mistake 4: Environment variables in Docker builds

# BAD: Build-time ARG becomes baked into image
ARG API_KEY
RUN echo $API_KEY > /app/config

This means your secrets are in the Docker image layers and can be extracted. Use runtime ENV instead:

# GOOD: Runtime environment variable
ENV API_KEY=""

# Set when running container
docker run -e API_KEY=secret myapp

Mistake 5: Not documenting required variables

Create a comprehensive .env.example:

# .env.example
# Copy this to .env and fill in real values

# Required - no defaults
DATABASE_URL=postgresql://user:password@localhost:5432/dbname
API_KEY=your-api-key-here

# Optional - have defaults
DEBUG=false
PORT=8000
LOG_LEVEL=INFO

# External services (optional)
SENDGRID_API_KEY=  # Only needed for email features
STRIPE_API_KEY=    # Only needed for payment features

Platform-specific tips

Heroku

heroku config:set API_KEY=abc123
heroku config:get DATABASE_URL

Heroku automatically sets DATABASE_URL when you add Postgres.

Vercel

vercel env add API_KEY
vercel env pull .env.local  # Pull remote env vars for local dev

Vercel has separate env vars for production, preview, and development.

AWS (Elastic Beanstalk)

eb setenv API_KEY=abc123 DEBUG=false

Or use AWS Systems Manager Parameter Store for better secret management.

Docker Compose

version: '3.8'
services:
  app:
    build: .
    env_file:
      - .env
    environment:
      - DATABASE_URL=postgresql://db:5432/myapp

Kubernetes

Use Secrets and ConfigMaps:

apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
type: Opaque
data:
  api-key: YWJjMTIz  # base64 encoded

The bottom line

Use pydantic-settings. It takes 5 minutes to set up and gives you:

  • Automatic .env file loading
  • Type conversion and validation
  • Clear errors for missing config
  • A single source of truth for settings
  • Protection against common mistakes

No more scattered os.environ calls. No more 'works on my machine' because of different config. Just clean, validated settings that make your app more secure and easier to deploy.

Start with a simple Settings class and evolve it as your app grows. Your future self (and your team) will thank you.

And please, never commit secrets to git. Just don't do it.

Send a Message