If you’ve ever accidentally opened 500 database connections when you only needed one, this post is for you.
The Singleton pattern is one of those ideas that sounds fancier than it is. The concept: make sure a class can only ever have one instance, and give everything in your app a single, shared way to access it. That’s it. One object. Everyone shares it.
Let’s dig into when that actually matters — and when ignoring it will quietly wreck your app.
The Problem: Your Database Doesn’t Want 500 Friends
Imagine you’re building a web API. Every time a request comes in, you need to talk to a database. The naive approach looks something like this:
# BEFORE: The "just make a new engine" approach
from sqlalchemy import create_engine, text
DATABASE_URL = "postgresql://user:password@localhost/mydb"
def get_user(user_id: int):
engine = create_engine(DATABASE_URL) # 🚨 new engine every single call
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM users WHERE id = :id"), {"id": user_id})
return result.fetchone()
This looks harmless. It works in development. Then you push to production.
Under any real load, this becomes a disaster:
- Connection exhaustion: Every
create_engine()call opens a new connection pool. PostgreSQL has a default limit of around 100 concurrent connections. Under moderate traffic, you’ll hit that ceiling and start seeingtoo many connectionserrors. - Memory waste: Each SQLAlchemy engine carries overhead — connection pool bookkeeping, dialect state, compiled query caches. Spin up hundreds of engines and you’re bleeding memory for no reason.
- Performance hit: Establishing a TCP connection and authenticating with the database takes time — typically 5–50ms. Do that on every API call instead of reusing an existing connection, and your latency climbs fast.
The fix isn’t complicated. You need one engine, created once, shared forever.
The Fix: One Engine to Rule Them All
# AFTER: Singleton database engine
from sqlalchemy import create_engine, text
DATABASE_URL = "postgresql://user:password@localhost/mydb"
class DatabaseEngine:
_instance = None
def __new__(cls):
if cls._instance is None:
print("Creating the database engine (you should see this ONCE)")
cls._instance = create_engine(
DATABASE_URL,
pool_size=10, # maintain up to 10 connections
max_overflow=20, # allow 20 extra under load
pool_pre_ping=True, # verify connections before using them
)
return cls._instance
def get_user(user_id: int):
engine = DatabaseEngine() # same instance every time
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM users WHERE id = :id"), {"id": user_id})
return result.fetchone()
Now you can call DatabaseEngine() from anywhere in your codebase — a FastAPI route, a background task, a utility function — and you’ll always get back the exact same engine object. One pool of connections, shared intelligently across your whole app.
The magic is in __new__. Python calls this before __init__ when creating an object. By checking _instance first, we intercept the instantiation and hand back the pre-existing object instead of building a new one.
Prefer Functions? There’s a Closure for That.
Classes not your thing? You can get the exact same singleton guarantee with a plain function and a closure — no __new__, no class attributes:
# Function-based Singleton using a closure
from sqlalchemy import create_engine, text
DATABASE_URL = "postgresql://user:password@localhost/mydb"
def make_engine_factory():
_engine = None # lives inside the closure, persists between calls
def get_engine():
nonlocal _engine
if _engine is None:
print("Creating engine — you'll see this exactly once")
_engine = create_engine(
DATABASE_URL,
pool_size=10,
max_overflow=20,
pool_pre_ping=True,
)
return _engine
return get_engine
# Call once at module level to create the accessor
get_engine = make_engine_factory()
# Usage — call from anywhere, always returns the same engine
def get_user(user_id: int):
engine = get_engine() # same engine every time
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM users WHERE id = :id"), {"id": user_id})
return result.fetchone()The _engine variable is captured inside make_engine_factory‘s scope. It’s not a global, not a class attribute — just a value that the inner get_engine function holds onto permanently. Call get_engine() a thousand times and _engine is only ever created once.
Class vs. function — which should you use?
- Use the class if you need inheritance, want to attach extra methods, or your team is more comfortable with OOP patterns.
- Use the closure if you want something lightweight and self-contained with no boilerplate. It’s also slightly harder to accidentally subclass and break.
Both are valid. Pick whichever fits your codebase’s style.
What About Thread Safety?
Fair question. In a multi-threaded server, two requests could theoretically hit if cls._instance is None at the exact same moment — both see None, both create an engine, and now you’ve got two. A classic race condition.
In Python, this is less dangerous than it sounds, thanks to the GIL (Global Interpreter Lock). The GIL ensures that only one thread runs Python bytecode at a time, which means truly simultaneous execution of that if check is unlikely in practice.
That said, “unlikely” isn’t “impossible.” If you want airtight thread safety, add a lock:
import threading
from sqlalchemy import create_engine
DATABASE_URL = "postgresql://user:password@localhost/mydb"
class DatabaseEngine:
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
# Check again inside the lock (double-checked locking)
if cls._instance is None:
cls._instance = create_engine(DATABASE_URL, pool_size=10)
return cls._instance
The double-check inside the lock prevents multiple threads from piling up waiting to create the engine. The first thread creates it; by the time the others acquire the lock, _instance is already set.
For most Python web apps, the GIL makes the simple version safe enough. If you’re using asyncio with thread pool executors or concurrent.futures, the locked version gives you stronger guarantees.
Real-World Example: App Configuration
App configuration is one of the most satisfying uses of the Singleton pattern because the problem is so obvious once you see it — if two modules each load your .env file independently, you have two config objects that could theoretically diverge. Worse, loading files and parsing environment variables on every import adds startup noise you don’t need.
Here’s a clean singleton AppConfig that reads your .env once, validates that critical values exist, and then hands the same object to every corner of your app:
import os
from dotenv import load_dotenv
class AppConfig:
_instance = None
def __new__(cls):
if cls._instance is None:
print("Loading config — runs once")
load_dotenv() # reads your .env file
instance = super().__new__(cls)
# All config values loaded and validated in one place
instance.database_url = os.getenv("DATABASE_URL")
instance.secret_key = os.getenv("SECRET_KEY")
instance.debug = os.getenv("DEBUG", "false").lower() == "true"
instance.port = int(os.getenv("PORT", 8000))
instance.redis_url = os.getenv("REDIS_URL", "redis://localhost:6379")
# Fail loud at startup if critical values are missing
if not instance.database_url:
raise ValueError("DATABASE_URL is not set")
if not instance.secret_key:
raise ValueError("SECRET_KEY is not set")
cls._instance = instance
return cls._instanceYour .env file to go with it:
DATABASE_URL=postgresql://user:password@localhost/mydb
SECRET_KEY=your-secret-key-here
DEBUG=true
PORT=8000
REDIS_URL=redis://localhost:6379And usage across your app — all three calls return the exact same object:
# In your database module
config = AppConfig()
engine = create_engine(config.database_url)
# In your FastAPI app setup
config = AppConfig()
app = FastAPI(debug=config.debug)
# In a background worker
config = AppConfig()
redis_client = Redis.from_url(config.redis_url)Two things worth noting here. First, missing critical values raise immediately at startup — not mid-request when a real user is waiting. That’s a deliberate choice. Silent config failures are some of the nastiest bugs to track down in production. Second, because it’s a singleton, you can call AppConfig() freely without worrying about whether some other module already loaded it. The answer is always: yes, it did, and you’re getting the same object.
Other Places Singletons Earn Their Keep
The database engine is the clearest example, but you’ll see this pattern pop up all over production codebases:
Logging
Python’s built-in logging.getLogger("myapp") is already a singleton under the hood — calling it multiple times with the same name returns the same logger object. Wrapping your own logger in a singleton gives you consistent log formatting and handlers across every module without passing a logger object around everywhere.
Cache Clients
Redis and Memcached clients maintain their own connection pools. Creating a new redis.Redis() client per request has the same problems as the database engine example — connection churn, overhead, potential pool exhaustion. A singleton cache client connects once and stays connected.
The Takeaway
Singletons aren’t magic — they’re just shared state with guardrails. Use them when:
- Creating the object is expensive (network connections, file I/O, heavy initialization)
- Having multiple instances would cause bugs or resource exhaustion
- You want a single point of control for something global
Don’t use them when every caller genuinely needs their own isolated copy of something, or when you’re fighting to write testable code — singletons can make mocking harder. But for database engines, loggers, configs, and cache clients? They’re exactly the right tool.
One instance. Shared everywhere. Created once. That’s the whole idea.
Pingback: The Factory Pattern in Python: Let Someone Else Decide What to Build