Learn Python Series (#45) - Metaclasses & Class Design Patterns

Repository

https://github.com/realScipio/learn-python-series

What will I learn

You will learn what metaclasses actually do and why they're called "the 99% don't need this" feature;
how Python's class creation process works under the hood;
when metaclasses solve real problems vs. when they're overkill;
practical class design patterns that don't require metaclasses;
how frameworks like Django and SQLAlchemy use metaclasses behind the scenes.

Requirements

A working modern computer running macOS, Windows or Ubuntu;
An installed Python 3(.11+) distribution, such as (for example) the Anaconda Distribution;
The ambition to learn Python programming.

Difficulty

Advanced

Curriculum (of the `Learn Python Series`):

GitHub Account

https://github.com/realScipio

Learn Python Series (#45) - Metaclasses & Class Design Patterns

Tim Peters, one of Python's core developers, once wrote: "Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't."

He's absolutely right. And yet — here we are ;-) Because understanding metaclasses is like understanding how a combustion engine works: you don't need it to drive your car, but if something breaks under the hood (or if you want to build something unusual), that knowledge becomes invaluable. More importantly, metaclasses reveal how Python itself works at a fundamental level. And that understanding makes you a better Python programmer even if you never write a single metaclass in production code.

Nota bene: I'm going to teach you metaclasses AND simultaneously teach you why you probably shouldn't use them. Both lessons are equally important. The real skill — and this applies to software engineering in general — is knowing which tool to reach for. And metaclasses are almost never the right answer.

Having said that, frameworks like Django, SQLAlchemy, and even Python's own Enum are built on metaclasses. So if you want to understand why those frameworks work the way they do (and if you ever need to debug something deep inside them), this episode is for you.

The mental model: classes are objects too

We all know this by now from previous episodes: in Python, everything is an object. Functions are objects. Modules are objects. Integers, strings, booleans — all objects. And classes? Also objects.

When you write:

class MyClass:
    pass

print(type(MyClass))   # 
print(type(42))        # 
print(type("hello"))   #

Python creates an object called MyClass. That class object lives in memory, has its own id(), has attributes and methods, and can be passed around, assigned to variables, stored in lists — just like any other object.

But if classes are objects.. what creates them? What is the "class of a class"? The answer: a metaclass.

A metaclass is to a class what a class is to an instance:

metaclass  →  creates  →  class  →  creates  →  instance
   type    →  creates  →  int    →  creates  →  42
   type    →  creates  →  str    →  creates  →  "hello"
   type    →  creates  →  MyClass → creates  →  MyClass()

The built-in type is Python's default metaclass. Every class you've ever written was secretly created by type.

How class creation actually works under the hood

When the Python interpreter encounters a class statement, it doesn't just magically produce a class. There's a well-defined sequence of steps. Understanding this is the key to understanding metaclasses:

Step 1: Python determines the metaclass. It looks for metaclass= in the class definition, then checks the base classes, and falls back to type.

Step 2: Python calls __prepare__() on the metaclass (if it exists) to get the namespace dictionary that will collect the class body's definitions.

Step 3: Python executes the class body as code inside that namespace dict. Every def, every assignment, every nested class — all of it populates this dict.

Step 4: Python calls the metaclass with three arguments: the class name (a string), the tuple of base classes, and the populated namespace dict. The metaclass returns the class object.

You can see this explicitly — these two approaches produce the exact same result:

# Approach 1: normal class statement
class Dog:
    sound = "woof"
    
    def speak(self):
        return self.sound

# Approach 2: calling type() directly with 3 arguments
Dog = type('Dog', (), {
    'sound': 'woof',
    'speak': lambda self: self.sound,
})

# Both produce the same result
d = Dog()
print(d.speak())  # woof

type() called with three arguments — type(name, bases, namespace) — is the metaclass constructor. It's how Python creates every class you've ever defined. The class statement is really just syntactic sugar for this call.

Let me prove that with a more complete example:

# Let's watch the process happen
class Verbose(type):
    def __new__(mcs, name, bases, namespace):
        print(f"Metaclass __new__ called!")
        print(f"  name: {name}")
        print(f"  bases: {bases}")
        print(f"  namespace keys: {list(namespace.keys())}")
        return super().__new__(mcs, name, bases, namespace)

class Animal(metaclass=Verbose):
    species = "unknown"
    
    def identify(self):
        return f"I am a {self.species}"

Running that code (not even creating an instance, just defining the class!) prints:

Metaclass __new__ called!
  name: Animal
  bases: ()
  namespace keys: ['__module__', '__qualname__', 'species', 'identify']

See? The class body was already executed and collected into the namespace dict. The metaclass receives everything — all your methods, all your class variables — and gets to inspect, modify, or reject them before the class is actually created.

Writing a custom metaclass (step by step)

A metaclass is a class that inherits from type and overrides __new__, __init__, or both. Here's the full pattern:

class Meta(type):
    def __new__(mcs, name, bases, namespace):
        # Called BEFORE the class object is created
        # 'mcs' is the metaclass itself (like 'cls' in classmethod)
        # You can modify 'namespace' here
        
        namespace['created_by'] = 'Meta'
        namespace['_registry_name'] = name.lower()
        
        # Actually create the class
        cls = super().__new__(mcs, name, bases, namespace)
        return cls
    
    def __init__(cls, name, bases, namespace):
        # Called AFTER the class object is created
        # 'cls' is the newly created class (not an instance!)
        
        super().__init__(name, bases, namespace)
        print(f"Class {name} has been initialized")

class MyClass(metaclass=Meta):
    x = 5

print(MyClass.created_by)      # 'Meta'
print(MyClass._registry_name)  # 'myclass'
print(MyClass.x)               # 5

The difference between __new__ and __init__ in a metaclass is subtle but important: __new__ runs before the class exists (so you work with the raw namespace dict), and __init__ runs after the class is created (so you get the actual class object). Most of the time you want __new__, because that's where you can modify what goes into the class.

And here's the thing that trips people up: the __call__ method on a metaclass controls what happens when you instantiate the class (not when you define it):

class Tracked(type):
    def __call__(cls, *args, **kwargs):
        print(f"Creating instance of {cls.__name__}")
        instance = super().__call__(*args, **kwargs)
        print(f"Instance created: {instance}")
        return instance

class User(metaclass=Tracked):
    def __init__(self, name):
        self.name = name

u = User("Scipio")
# Creating instance of User
# Instance created:

So a metaclass can intercept both class creation AND instance creation. That's a lot of power — which is exactly why you should be careful with it ;-)

Practical example: automatic registration

Suppose you're building a plugin system. You want every plugin to auto-register itself when the class is defined. No manual register_plugin(MyPlugin) calls, no import-time setup functions — just define a class and it registers:

class PluginRegistry(type):
    _plugins = {}
    
    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        
        # Don't register the base class itself
        if bases:  # Only register subclasses, not the root
            mcs._plugins[name] = cls
            print(f"  Registered plugin: {name}")
        
        return cls
    
    @classmethod
    def get_plugin(mcs, name):
        return mcs._plugins.get(name)
    
    @classmethod
    def list_plugins(mcs):
        return list(mcs._plugins.keys())

class Plugin(metaclass=PluginRegistry):
    """Base class for all plugins."""
    def process(self, data):
        raise NotImplementedError

class CSVPlugin(Plugin):
    def process(self, data):
        return f"Parsing CSV: {len(data)} bytes"

class JSONPlugin(Plugin):
    def process(self, data):
        return f"Parsing JSON: {len(data)} bytes"

class XMLPlugin(Plugin):
    def process(self, data):
        return f"Parsing XML: {len(data)} bytes"

# All plugins registered automatically:
print(PluginRegistry.list_plugins())
# ['CSVPlugin', 'JSONPlugin', 'XMLPlugin']

# Look up and use a plugin by name:
plugin_cls = PluginRegistry.get_plugin("JSONPlugin")
plugin = plugin_cls()
print(plugin.process(b'{"key": "value"}'))
# Parsing JSON: 16 bytes

This pattern is exactly how frameworks like Django's ORM track model classes. When you write class Article(models.Model):, Django's ModelBase metaclass registers Article into the app registry, creates field objects from your class attributes, adds the DoesNotExist exception, and sets up the objects manager. All of that happens at class definition time — before you create a single instance.

`prepare`: controlling the class namespace

Here's a lesser-known metaclass hook. __prepare__ lets you control what kind of dict is used as the class namespace. By default it's a regular dict, but you can substitute an OrderedDict (historically useful before Python 3.7 when dicts became insertion-ordered) or a custom mapping:

from collections import OrderedDict

class OrderedMeta(type):
    @classmethod
    def __prepare__(mcs, name, bases):
        return OrderedDict()
    
    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        cls._field_order = [
            key for key in namespace
            if not key.startswith('_')
        ]
        return cls

class Config(metaclass=OrderedMeta):
    host = "localhost"
    port = 8080
    debug = True
    timeout = 30

print(Config._field_order)
# ['host', 'port', 'debug', 'timeout']

A more practical use case for __prepare__ is returning a namespace that raises an error on duplicate definitions:

class NoDuplicateDict(dict):
    def __setitem__(self, key, value):
        if key in self and key != '__module__' and key != '__qualname__':
            raise TypeError(f"Duplicate definition of '{key}'")
        super().__setitem__(key, value)

class StrictMeta(type):
    @classmethod
    def __prepare__(mcs, name, bases):
        return NoDuplicateDict()
    
    def __new__(mcs, name, bases, namespace):
        return super().__new__(mcs, name, bases, dict(namespace))

class MyConfig(metaclass=StrictMeta):
    timeout = 30
    # timeout = 60  # Uncommenting this raises TypeError!

This kind of validation at class-definition time is hard to achieve without metaclasses (well, until __init_subclass__ came along, but more on that later).

When metaclasses are actually useful (and when they're overkill)

After years of writing Python, here is my honest assessment of when metaclasses solve real problems:

Automatic registration — plugins, ORM models, serializers. This is the classic case.
Interface enforcement at definition time — catching missing methods when the class is defined, not when it's instantiated (earlier than abc.ABC catches it).
Namespace manipulation — controlling what goes into the class dict (using __prepare__).
Framework-level magic — when you're building a library that others will subclass, and the subclass behavior needs to be modified transparently.

But — and this is the important bit — for cases 1 and 2, simpler alternatives exist today. Let me show you.

The simpler alternatives (and why you should prefer them)

Alternative 1: class decorators for registration

_plugins = {}

def register(cls):
    """Register a plugin class by name."""
    _plugins[cls.__name__] = cls
    return cls

def get_plugin(name):
    return _plugins[name]

@register
class CSVPlugin:
    def process(self, data):
        return f"CSV: {len(data)} bytes"

@register 
class JSONPlugin:
    def process(self, data):
        return f"JSON: {len(data)} bytes"

print(list(_plugins.keys()))  # ['CSVPlugin', 'JSONPlugin']

This is explicit (you can see the @register decorator), simple to understand, and doesn't require anyone reading the code to know what a metaclass is. For most plugin systems, this is the right choice.

Alternative 2: `abc.ABC` for interface validation

from abc import ABC, abstractmethod

class Plugin(ABC):
    @abstractmethod
    def process(self, data):
        """Process the input data and return result."""
        pass
    
    @abstractmethod
    def name(self):
        """Return the plugin's display name."""
        pass

class CSVPlugin(Plugin):
    def process(self, data):
        return f"CSV: {len(data)} bytes"
    
    # Forgot to implement name()!

csv = CSVPlugin()  # TypeError: Can't instantiate abstract class 
                    # CSVPlugin with abstract method name

Python's abc module uses ABCMeta (a metaclass) internally — so you're getting metaclass power without writing metaclass code. The error message is clear, it's well-understood by any Python developer, and it's in the standard library.

Alternative 3: `__init_subclass__` (Python 3.6+)

This is the big one. __init_subclass__ was added specifically to eliminate most metaclass use cases:

class Plugin:
    _plugins = {}
    
    def __init_subclass__(cls, *, plugin_name=None, **kwargs):
        super().__init_subclass__(**kwargs)
        name = plugin_name or cls.__name__
        cls._plugins[name] = cls
        cls.registered_name = name

class CSVPlugin(Plugin, plugin_name="csv"):
    def process(self, data):
        return f"CSV: {len(data)} bytes"

class JSONPlugin(Plugin, plugin_name="json"):
    def process(self, data):
        return f"JSON: {len(data)} bytes"

class XMLPlugin(Plugin):  # Uses class name as default
    def process(self, data):
        return f"XML: {len(data)} bytes"

print(Plugin._plugins)
# {'csv': , 'json': , 
#  'XMLPlugin': }

print(CSVPlugin.registered_name)  # 'csv'

Notice how __init_subclass__ even accepts keyword arguments via the class definition syntax (class CSVPlugin(Plugin, plugin_name="csv")). That's elegant. And it's way easier to understand than a metaclass.

Alternative 4: class decorators for method modification

If you want to add logging, timing, or validation to all methods in a class, a decorator is cleaner:

import functools
import time

def timed_methods(cls):
    """Add timing to all public methods."""
    for name, method in list(cls.__dict__.items()):
        if callable(method) and not name.startswith('_'):
            @functools.wraps(method)
            def timed(self, *args, _orig=method, _name=name, **kwargs):
                start = time.perf_counter()
                result = _orig(self, *args, **kwargs)
                elapsed = time.perf_counter() - start
                print(f"  {_name}() took {elapsed:.4f}s")
                return result
            setattr(cls, name, timed)
    return cls

@timed_methods
class DataProcessor:
    def fetch(self):
        time.sleep(0.1)
        return [1, 2, 3]
    
    def transform(self, data):
        time.sleep(0.05)
        return [x * 2 for x in data]

proc = DataProcessor()
data = proc.fetch()       # fetch() took 0.1003s
proc.transform(data)      # transform() took 0.0501s

Class decorators run after the class is created, which means they work on the fully-formed class object. You get the same power as a metaclass __new__, with a much simpler mental model.

The singleton pattern: three approaches

The singleton pattern ensures only one instance of a class exists. It's a common interview question ("implement singleton with a metaclass!"), so let me show you all three approaches — from most complex to simplest:

Approach 1: metaclass (don't do this)

class SingletonMeta(type):
    _instances = {}
    
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]

class Database(metaclass=SingletonMeta):
    def __init__(self):
        self.connected = False

db1 = Database()
db2 = Database()
assert db1 is db2  # Same object

Approach 2: __new__ override (simpler)

class Database:
    _instance = None
    
    def __new__(cls):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance

db1 = Database()
db2 = Database()
assert db1 is db2  # Same object

Approach 3: module-level instance (simplest — do this)

# database.py
class _Database:
    def __init__(self):
        self.connected = False

db = _Database()  # The singleton

# other_module.py
from database import db  # Always the same instance

Python modules are natural singletons — they're imported once and cached. Module-level instances are the Pythonic way to implement singletons. No metaclass needed, no __new__ hack needed.. just a module with an instance.

Real-world metaclass: how Enum works

Python's Enum is one of the few standard library classes that legitimately needs a metaclass. Here's why:

from enum import Enum

class Color(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

print(Color.RED)         # Color.RED (not 1!)
print(Color.RED.value)   # 1
print(Color.RED.name)    # 'RED'
print(Color(2))          # Color.GREEN (reverse lookup)
print(list(Color))       # [Color.RED, Color.GREEN, Color.BLUE]

# Enums are singletons:
assert Color.RED is Color(1)

# Enums aren't regular ints:
# Color.RED + 1  # TypeError!

The EnumMeta metaclass intercepts class creation and transforms your simple RED = 1 assignments into full Enum member objects with .name, .value, iteration support, reverse lookups, and singleton identity. This transformation happens in __new__ and __prepare__ — it fundamentally changes how the class body is interpreted.

You cannot achieve this with a class decorator or __init_subclass__, because by the time those run, the class attributes are already plain integers. The metaclass intervenes during class body execution itself. That's the real power of metaclasses — and the real reason they're rarely needed. Most classes don't need their body to be reinterpreted.

The descriptor protocol (a better tool for attribute control)

If what you actually want is custom behavior on attribute access — validation, type checking, computed properties — descriptors are the right tool. They're simpler than metaclasses and more targeted:

class TypeChecked:
    """Descriptor that enforces type at assignment time."""
    def __init__(self, expected_type):
        self.expected_type = expected_type
    
    def __set_name__(self, owner, name):
        self.name = name
        self.storage_name = f'_typechecked_{name}'
    
    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return getattr(obj, self.storage_name, None)
    
    def __set__(self, obj, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(
                f"{self.name} must be {self.expected_type.__name__}, "
                f"got {type(value).__name__}"
            )
        setattr(obj, self.storage_name, value)

class User:
    name = TypeChecked(str)
    age = TypeChecked(int)
    email = TypeChecked(str)
    
    def __init__(self, name, age, email):
        self.name = name
        self.age = age
        self.email = email

user = User("Scipio", 99, "test@example.com")  # OK
print(user.name)  # Scipio
print(user.age)   # 99

try:
    user.age = "not a number"
except TypeError as e:
    print(e)  # age must be int, got str

try:
    User("Scipio", "old", "test@example.com")
except TypeError as e:
    print(e)  # age must be int, got str

The __set_name__ hook (Python 3.6+) is what makes this particularly clean — the descriptor automatically knows its own attribute name. No metaclass needed. We'll do a full deep-dive into descriptors in a future episode (because there's quit some more to cover there).

Composition over inheritance

And finally — since this episode is about class design patterns, not just metaclasses — let's talk about the single most important pattern: composition over inheritance.

Deep inheritance hierarchies are fragile. The moment you change something in a base class, every subclass is potentially affected. Composition gives you the same code reuse without the coupling:

# Instead of this (fragile inheritance):
class Animal:
    def speak(self): ...

class Dog(Animal):
    def speak(self):
        return "Woof"

class TrainedDog(Dog):
    def do_trick(self):
        return "Rolls over"

# Prefer this (flexible composition):
class Speaker:
    def __init__(self, sound):
        self.sound = sound
    def speak(self):
        return self.sound

class TrickPerformer:
    def __init__(self, tricks):
        self.tricks = tricks
    def do_trick(self, name):
        return self.tricks.get(name, "I don't know that trick")

class Dog:
    def __init__(self):
        self.voice = Speaker("Woof")
        self.performer = TrickPerformer({
            "roll": "Rolls over",
            "shake": "Offers paw",
        })
    
    def speak(self):
        return self.voice.speak()
    
    def do_trick(self, name):
        return self.performer.do_trick(name)

rex = Dog()
print(rex.speak())              # Woof
print(rex.do_trick("roll"))     # Rolls over
print(rex.do_trick("fly"))      # I don't know that trick

With composition, you can swap the Speaker for a different sound, add new TrickPerformer behaviors, or test each component independently. Try doing that with a five-level inheritance tree...

Dussssss, samengevat

In this episode, we explored metaclasses and class design patterns:

Metaclasses are the classes that create classes — Python's default metaclass is type
The class creation process follows specific steps: __prepare__ → execute body → __new__ → __init__
Custom metaclasses intercept class creation via __new__ and can modify the class namespace
Automatic registration is the classic metaclass use case (Django's ORM, plugin systems)
__prepare__ controls the namespace dict (useful for ordering or duplicate detection)
But — most metaclass use cases have simpler alternatives:
- Class decorators for registration and method modification
- abc.ABC for interface enforcement
- __init_subclass__ for subclass hooks (Python 3.6+)
- Descriptors for attribute-level behavior
Singletons are best implemented as module-level instances, not metaclass magic
Enum is a legitimate metaclass use because it reinterprets the class body itself
Composition over inheritance is usually the right class design choice
The golden rule: ask yourself "Could this be a class decorator instead?" and the answer is usually yes

Metaclasses are the deep end of Python's object model. Understanding them teaches you how the language actually works. But writing them in production code? That's almost always a sign you're reaching for the wrong tool. Use the simpler alternatives — they're simpler for a reason ;-)

De groeten!

Hive account@scipio

Learn Python Series (#45) - Metaclasses & Class Design Patterns

Learn Python Series (#45) - Metaclasses & Class Design Patterns

Repository

What will I learn

Requirements

Difficulty

Curriculum (of the Learn Python Series):

GitHub Account

Learn Python Series (#45) - Metaclasses & Class Design Patterns

The mental model: classes are objects too

How class creation actually works under the hood

Writing a custom metaclass (step by step)

Practical example: automatic registration

__prepare__: controlling the class namespace

When metaclasses are actually useful (and when they're overkill)

The simpler alternatives (and why you should prefer them)

Alternative 1: class decorators for registration

Alternative 2: abc.ABC for interface validation

Alternative 3: __init_subclass__ (Python 3.6+)

Alternative 4: class decorators for method modification

The singleton pattern: three approaches

Real-world metaclass: how Enum works

The descriptor protocol (a better tool for attribute control)

Composition over inheritance

Dussssss, samengevat

De groeten!

Curriculum (of the `Learn Python Series`):

`prepare`: controlling the class namespace

Alternative 2: `abc.ABC` for interface validation

Alternative 3: `__init_subclass__` (Python 3.6+)