logging.Handler

The Handler class is the base class for all logging handlers. Handlers determine where log records are sent - to the console, files, network destinations, or other outputs. Python provides many built-in handler types for different use cases.

Basic Usage

Simple Example

import logging

# Create logger
logger = logging.getLogger('example')
logger.setLevel(logging.DEBUG)

# Create and configure console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)

# Create formatter
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)

# Add handler to logger
logger.addHandler(console_handler)

# Log messages
logger.debug("Debug message")    # Won't appear (below handler level)
logger.info("Info message")      # Will appear
logger.error("Error message")    # Will appear

Multiple Handlers

import logging
import logging.handlers

logger = logging.getLogger('multi_handler')
logger.setLevel(logging.DEBUG)

# Console handler for INFO and above
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
console_formatter = logging.Formatter('%(levelname)s: %(message)s')
console_handler.setFormatter(console_formatter)

# File handler for DEBUG and above
file_handler = logging.FileHandler('debug.log')
file_handler.setLevel(logging.DEBUG)
file_formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
file_handler.setFormatter(file_formatter)

# Add handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Test logging
logger.debug("Debug info")      # Only in file
logger.info("General info")     # Console and file
logger.error("Error occurred")  # Console and file

Handler API Reference

Base Handler Class

Method	Description	Parameters	Return Type
`setLevel(level)`	Set handler threshold level	`level`: Log level (int/str)	`None`
`setFormatter(formatter)`	Set formatter for handler	`formatter`: Formatter instance	`None`
`addFilter(filter)`	Add filter to handler	`filter`: Filter instance/callable	`None`
`removeFilter(filter)`	Remove filter from handler	`filter`: Filter instance/callable	`None`
`filter(record)`	Apply filters to record	`record`: LogRecord	`bool`
`emit(record)`	Output log record	`record`: LogRecord	`None`
`handle(record)`	Process log record	`record`: LogRecord	`None`
`flush()`	Flush any buffered output	None	`None`
`close()`	Close handler and release resources	None	`None`

Handler Properties

Property	Description	Type	Access
`level`	Handler threshold level	`int`	Read/Write
`formatter`	Associated formatter	`Formatter`	Read/Write
`filters`	List of filters	`list`	Read-only

Built-in Handler Types

StreamHandler

Sends log output to streams like sys.stdout or sys.stderr.

import logging
import sys

# Console output (default: stderr)
console_handler = logging.StreamHandler()

# Specific stream
stdout_handler = logging.StreamHandler(sys.stdout)
stderr_handler = logging.StreamHandler(sys.stderr)

# Usage
logger = logging.getLogger('stream_example')
logger.addHandler(console_handler)
logger.info("This goes to stderr by default")

Parameters:

stream: Output stream (default: sys.stderr)

FileHandler

Sends log output to disk files.

import logging

# Basic file handler
file_handler = logging.FileHandler('application.log')

# Append mode (default)
append_handler = logging.FileHandler('app.log', mode='a')

# Write mode (overwrites)
write_handler = logging.FileHandler('app.log', mode='w')

# Specific encoding
utf8_handler = logging.FileHandler('app.log', encoding='utf-8')

# Usage
logger = logging.getLogger('file_example')
logger.addHandler(file_handler)
logger.info("This message goes to the file")

Parameters:

filename: Path to log file
mode: File open mode ('a' append, 'w' write)
encoding: Text encoding (default: platform default)
delay: Delay file opening until first emit

RotatingFileHandler

Rotates log files based on size limits.

import logging.handlers

# Rotate when file reaches 1MB, keep 5 backup files
rotating_handler = logging.handlers.RotatingFileHandler(
    'app.log',
    maxBytes=1024*1024,  # 1MB
    backupCount=5
)

# When app.log reaches 1MB:
# app.log -> app.log.1
# New app.log is created
# When it reaches 1MB again:
# app.log.1 -> app.log.2
# app.log -> app.log.1
# New app.log is created

Parameters:

filename: Base filename
mode: File open mode
maxBytes: Maximum file size in bytes
backupCount: Number of backup files to keep
encoding: Text encoding
delay: Delay file opening

TimedRotatingFileHandler

Rotates log files at specific time intervals.

import logging.handlers

# Rotate daily at midnight
daily_handler = logging.handlers.TimedRotatingFileHandler(
    'daily.log',
    when='midnight',
    interval=1,
    backupCount=30  # Keep 30 days
)

# Rotate every hour
hourly_handler = logging.handlers.TimedRotatingFileHandler(
    'hourly.log',
    when='H',
    interval=1,
    backupCount=24  # Keep 24 hours
)

# Rotate weekly on Sunday
weekly_handler = logging.handlers.TimedRotatingFileHandler(
    'weekly.log',
    when='W0',  # W0=Monday, W6=Sunday
    interval=1,
    backupCount=4  # Keep 4 weeks
)

Time Units:

'S': Seconds
'M': Minutes
'H': Hours
'D': Days
'W0'-'W6': Weekly (0=Monday, 6=Sunday)
'midnight': Daily at midnight

SocketHandler

Sends log records to network sockets.

import logging.handlers

# TCP socket handler
tcp_handler = logging.handlers.SocketHandler('localhost', 9999)

# Log server example (separate process)
import pickle
import socket
import struct

def log_server():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.bind(('localhost', 9999))
    sock.listen(1)
    
    while True:
        conn, addr = sock.accept()
        while True:
            chunk = conn.recv(4)
            if len(chunk) < 4:
                break
            slen = struct.unpack('>L', chunk)[0]
            chunk = conn.recv(slen)
            while len(chunk) < slen:
                chunk += conn.recv(slen - len(chunk))
            obj = pickle.loads(chunk)
            print(f"Received: {obj}")

SysLogHandler

Sends log records to Unix syslog daemon.

import logging.handlers

# Local syslog
syslog_handler = logging.handlers.SysLogHandler(address='/dev/log')

# Remote syslog
remote_syslog = logging.handlers.SysLogHandler(
    address=('logserver.example.com', 514)
)

# Facility and priority
syslog_handler = logging.handlers.SysLogHandler(
    facility=logging.handlers.SysLogHandler.LOG_LOCAL0
)

Facilities:

LOG_KERN, LOG_USER, LOG_MAIL, LOG_DAEMON
LOG_AUTH, LOG_SYSLOG, LOG_LPR, LOG_NEWS
LOG_UUCP, LOG_CRON, LOG_AUTHPRIV, LOG_FTP
LOG_LOCAL0 through LOG_LOCAL7

SMTPHandler

Sends log records via email.

import logging.handlers

# Basic SMTP handler
smtp_handler = logging.handlers.SMTPHandler(
    mailhost='smtp.example.com',
    fromaddr='app@example.com',
    toaddrs=['admin@example.com'],
    subject='Application Error'
)

# With authentication
smtp_handler = logging.handlers.SMTPHandler(
    mailhost=('smtp.gmail.com', 587),
    fromaddr='app@gmail.com',
    toaddrs=['admin@gmail.com'],
    subject='Critical Error',
    credentials=('username', 'password'),
    secure=()  # Use TLS
)

# Only send on critical errors
smtp_handler.setLevel(logging.CRITICAL)

HTTPHandler

Sends log records to web servers via HTTP.

import logging.handlers

# HTTP POST handler
http_handler = logging.handlers.HTTPHandler(
    host='logserver.example.com',
    url='/logs',
    method='POST'
)

# HTTPS with credentials
https_handler = logging.handlers.HTTPHandler(
    host='secure-logs.example.com',
    url='/api/logs',
    method='POST',
    secure=True,
    credentials={'username': 'user', 'password': 'pass'}
)

MemoryHandler

Buffers log records in memory until threshold reached.

import logging.handlers

# Buffer up to 100 records, flush on ERROR or higher
memory_handler = logging.handlers.MemoryHandler(
    capacity=100,
    flushLevel=logging.ERROR,
    target=file_handler  # Where to flush records
)

# Manual flush
memory_handler.flush()

# Example: Buffer debug info, flush on errors
logger = logging.getLogger('buffered')
logger.addHandler(memory_handler)

# These get buffered
logger.debug("Debug info 1")
logger.debug("Debug info 2") 
logger.info("Info message")

# This triggers flush of all buffered records
logger.error("Error occurred!")

WatchedFileHandler

Monitors log files for external changes (useful with log rotation tools).

import logging.handlers

# Watches file for external rotation (logrotate, etc.)
watched_handler = logging.handlers.WatchedFileHandler('app.log')

# If external tool rotates the file, handler detects it
# and reopens the new file automatically

Common Errors and Troubleshooting

Typical Error Messages

import logging
import logging.handlers

# Error 1: File permission denied
try:
    handler = logging.FileHandler('/var/log/app.log')
except PermissionError:
    # Fallback to user directory or temp
    handler = logging.FileHandler('app.log')

# Error 2: Network connection failed
try:
    handler = logging.handlers.SocketHandler('logserver.com', 514)
    # Test connection
    handler.emit(logging.LogRecord(
        name='test', level=logging.INFO, pathname='', lineno=0,
        msg='Test message', args=(), exc_info=None
    ))
except ConnectionRefusedError:
    print("Log server unavailable, using local file")
    handler = logging.FileHandler('fallback.log')

# Error 3: SMTP authentication failed
try:
    handler = logging.handlers.SMTPHandler(
        mailhost=('smtp.gmail.com', 587),
        fromaddr='app@gmail.com',
        toaddrs=['admin@gmail.com'],
        subject='App Error',
        credentials=('user', 'wrong_password')
    )
except Exception as e:
    print(f"Email handler failed: {e}")
    handler = logging.FileHandler('email_errors.log')

Handler Configuration Debugging

import logging

logger = logging.getLogger('debug_handlers')

# List all handlers
print("Current handlers:")
for i, handler in enumerate(logger.handlers):
    print(f"  {i}: {handler.__class__.__name__}")
    print(f"     Level: {logging.getLevelName(handler.level)}")
    print(f"     Formatter: {handler.formatter}")
    print(f"     Filters: {len(handler.filters)}")

# Test handler levels
test_levels = [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR]
for level in test_levels:
    level_name = logging.getLevelName(level)
    record = logging.LogRecord(
        name='test', level=level, pathname='test.py', lineno=1,
        msg=f'Test {level_name} message', args=(), exc_info=None
    )
    
    for handler in logger.handlers:
        if handler.filter(record):
            print(f"Handler {handler.__class__.__name__} would process {level_name}")

Primary Use Cases

1. Multi-Destination Logging

Use Case: Send different log levels to different destinations Why Handlers: Flexible routing of log messages

import logging
import logging.handlers

def setup_multi_destination_logging():
    logger = logging.getLogger('multi_dest')
    logger.setLevel(logging.DEBUG)
    
    # Console: INFO and above
    console = logging.StreamHandler()
    console.setLevel(logging.INFO)
    console.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
    
    # File: DEBUG and above with detailed format
    file_handler = logging.handlers.RotatingFileHandler(
        'app.log', maxBytes=10*1024*1024, backupCount=5
    )
    file_handler.setLevel(logging.DEBUG)
    file_handler.setFormatter(logging.Formatter(
        '%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s'
    ))
    
    # Email: CRITICAL only
    email_handler = logging.handlers.SMTPHandler(
        mailhost='smtp.company.com',
        fromaddr='app@company.com',
        toaddrs=['admin@company.com'],
        subject='CRITICAL Application Error'
    )
    email_handler.setLevel(logging.CRITICAL)
    
    logger.addHandler(console)
    logger.addHandler(file_handler)
    logger.addHandler(email_handler)
    
    return logger

# Usage
logger = setup_multi_destination_logging()
logger.debug("Debug info")        # File only
logger.info("User action")        # Console + File
logger.critical("System down!")   # Console + File + Email

2. Log Rotation Management

Use Case: Manage log file growth and retention Why Handlers: Automatic log rotation prevents disk space issues

import logging.handlers
import os

def setup_rotating_logs(app_name, log_dir='logs'):
    """Setup comprehensive log rotation."""
    
    # Create log directory
    os.makedirs(log_dir, exist_ok=True)
    
    logger = logging.getLogger(app_name)
    logger.setLevel(logging.DEBUG)
    
    # Daily rotation for application logs
    daily_handler = logging.handlers.TimedRotatingFileHandler(
        filename=os.path.join(log_dir, f'{app_name}.log'),
        when='midnight',
        interval=1,
        backupCount=30,  # Keep 30 days
        encoding='utf-8'
    )
    daily_handler.setLevel(logging.INFO)
    
    # Size-based rotation for debug logs
    debug_handler = logging.handlers.RotatingFileHandler(
        filename=os.path.join(log_dir, f'{app_name}-debug.log'),
        maxBytes=50*1024*1024,  # 50MB
        backupCount=3,
        encoding='utf-8'
    )
    debug_handler.setLevel(logging.DEBUG)
    
    # Error-only logs with longer retention
    error_handler = logging.handlers.TimedRotatingFileHandler(
        filename=os.path.join(log_dir, f'{app_name}-errors.log'),
        when='midnight',
        interval=1,
        backupCount=90,  # Keep 90 days of errors
        encoding='utf-8'
    )
    error_handler.setLevel(logging.ERROR)
    
    # Add formatters
    detailed_formatter = logging.Formatter(
        '%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s'
    )
    for handler in [daily_handler, debug_handler, error_handler]:
        handler.setFormatter(detailed_formatter)
        logger.addHandler(handler)
    
    return logger

# Usage
logger = setup_rotating_logs('myapp')
logger.info("Application started")

3. Remote Logging Setup

Use Case: Centralize logs from distributed applications Why Handlers: Network handlers enable centralized log collection

import logging.handlers
import json
import socket

class JSONSocketHandler(logging.handlers.SocketHandler):
    """Custom handler that sends JSON-formatted logs."""
    
    def makePickle(self, record):
        """Override to send JSON instead of pickle."""
        log_data = {
            'timestamp': record.created,
            'level': record.levelname,
            'logger': record.name,
            'message': record.getMessage(),
            'module': record.module,
            'function': record.funcName,
            'line': record.lineno,
            'host': socket.gethostname()
        }
        
        if record.exc_info:
            log_data['exception'] = self.format(record)
        
        json_data = json.dumps(log_data) + '\n'
        return json_data.encode('utf-8')

def setup_remote_logging(service_name, log_server_host, log_server_port):
    """Setup logging to remote server."""
    
    logger = logging.getLogger(service_name)
    logger.setLevel(logging.INFO)
    
    try:
        # Try remote logging first
        remote_handler = JSONSocketHandler(log_server_host, log_server_port)
        remote_handler.setLevel(logging.INFO)
        logger.addHandler(remote_handler)
        
        # Test connection
        logger.info("Connected to remote log server")
        
    except (ConnectionRefusedError, socket.gaierror) as e:
        # Fallback to local file if remote unavailable
        print(f"Remote logging unavailable: {e}")
        fallback_handler = logging.handlers.RotatingFileHandler(
            f'{service_name}.log', maxBytes=10*1024*1024, backupCount=5
        )
        fallback_handler.setLevel(logging.INFO)
        logger.addHandler(fallback_handler)
        logger.warning("Using local logging fallback")
    
    return logger

# Usage
logger = setup_remote_logging('microservice-1', 'logserver.local', 5140)
logger.info("Service initialized")

4. Buffered Error Reporting

Use Case: Collect context around errors for better debugging Why Handlers: Memory handler buffers context, flushes on errors

import logging.handlers

def setup_buffered_error_logging():
    """Setup logging that captures context around errors."""
    
    logger = logging.getLogger('buffered_app')
    logger.setLevel(logging.DEBUG)
    
    # File handler for immediate ERROR+ messages
    error_file = logging.FileHandler('errors.log')
    error_file.setLevel(logging.ERROR)
    error_formatter = logging.Formatter(
        '%(asctime)s - %(levelname)s - %(message)s'
    )
    error_file.setFormatter(error_formatter)
    
    # Memory handler: buffers DEBUG/INFO, flushes on ERROR
    memory_handler = logging.handlers.MemoryHandler(
        capacity=50,  # Buffer last 50 debug/info messages
        flushLevel=logging.ERROR,
        target=error_file
    )
    memory_handler.setLevel(logging.DEBUG)
    
    # Console for immediate feedback
    console = logging.StreamHandler()
    console.setLevel(logging.WARNING)
    console.setFormatter(logging.Formatter('%(levelname)s: %(message)s'))
    
    logger.addHandler(memory_handler)
    logger.addHandler(console)
    
    return logger

# Usage example
logger = setup_buffered_error_logging()

# These messages are buffered in memory
logger.debug("Processing user request")
logger.debug("Validating input data")
logger.info("Data validation successful")
logger.debug("Querying database")
logger.info("Database query completed")

# This error causes all buffered messages to be written to file
try:
    result = 1 / 0
except ZeroDivisionError:
    logger.error("Division by zero error")
    # Now error.log contains all the buffered context messages

Performance Considerations

Handler Performance Comparison

Handler Type	Performance	Buffering	Network	Notes
StreamHandler	Fast	None	No	Direct to console
FileHandler	Medium	OS-level	No	Disk I/O overhead
RotatingFileHandler	Medium	OS-level	No	Periodic rotation overhead
SocketHandler	Variable	Optional	Yes	Network latency
SMTPHandler	Slow	None	Yes	Email delivery overhead
MemoryHandler	Fast	Memory	Depends on target	Good for batching

Optimization Tips

import logging.handlers

# Use appropriate buffering for file handlers
buffered_handler = logging.FileHandler('app.log')
# OS handles buffering automatically

# For high-volume logging, use MemoryHandler
high_volume_handler = logging.handlers.MemoryHandler(
    capacity=1000,  # Larger buffer for high volume
    flushLevel=logging.ERROR
)

# Minimize formatter complexity for high-frequency logs
simple_formatter = logging.Formatter('%(levelname)s:%(message)s')
complex_formatter = logging.Formatter(
    '%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(funcName)s - %(message)s'
)

# Use simple formatter for high-frequency, complex for errors
debug_handler.setFormatter(simple_formatter)
error_handler.setFormatter(complex_formatter)

When to Use Different Handlers

Handler Selection Guide

StreamHandler:

Development and debugging
Container environments (stdout/stderr)
Simple applications
Real-time log monitoring

FileHandler:

Production applications
Log persistence requirements
Audit trails
Offline analysis

RotatingFileHandler:

Long-running applications
Disk space constraints
Automated log management
Size-based rotation needs

TimedRotatingFileHandler:

Time-based log analysis
Daily/weekly reports
Compliance requirements
Scheduled log processing

SocketHandler:

Distributed systems
Centralized logging
Real-time log aggregation
Microservices architecture

SMTPHandler:

Critical error alerts
System administrator notifications
Low-frequency, high-importance events
Automated incident reporting

MemoryHandler:

Context preservation around errors
High-frequency debug logging
Batch processing scenarios
Performance optimization

When NOT to Use Specific Handlers

Avoid FileHandler when:

Running in containers without persistent volumes
High-frequency logging (>1000 msgs/sec)
Temporary or ephemeral applications

Avoid SMTPHandler when:

High-frequency events (spam risk)
Network unreliability
Security restrictions on email

Avoid SocketHandler when:

Network connectivity is unreliable
Security policies block network logging
Local-only applications

Additional Learning Resources

Official Python Resources

Handler Documentation: logging.handlers — Logging handlers
Logging HOWTO: Logging HOWTO
Cookbook: Logging Cookbook

logging.Logger - Core logging interface
logging.Formatter - Message formatting
logging.Filter - Record filtering
logging.config - Configuration utilities

Basic Usage​

Simple Example​

Multiple Handlers​

Handler API Reference​

Base Handler Class​

Handler Properties​

Built-in Handler Types​

StreamHandler​

FileHandler​

RotatingFileHandler​

TimedRotatingFileHandler​

SocketHandler​

SysLogHandler​

SMTPHandler​

HTTPHandler​

MemoryHandler​

WatchedFileHandler​

Common Errors and Troubleshooting​

Typical Error Messages​

Handler Configuration Debugging​

Primary Use Cases​

1. Multi-Destination Logging​

2. Log Rotation Management​

3. Remote Logging Setup​

4. Buffered Error Reporting​

Performance Considerations​

Handler Performance Comparison​

Optimization Tips​

When to Use Different Handlers​

Handler Selection Guide​

When NOT to Use Specific Handlers​

Additional Learning Resources​

Official Python Resources​

Related Documentation​

Basic Usage

Simple Example

Multiple Handlers

Handler API Reference

Base Handler Class

Handler Properties

Built-in Handler Types

StreamHandler

FileHandler

RotatingFileHandler

TimedRotatingFileHandler

SocketHandler

SysLogHandler

SMTPHandler

HTTPHandler

MemoryHandler

WatchedFileHandler

Common Errors and Troubleshooting

Typical Error Messages

Handler Configuration Debugging

Primary Use Cases

1. Multi-Destination Logging

2. Log Rotation Management

3. Remote Logging Setup

4. Buffered Error Reporting

Performance Considerations

Handler Performance Comparison

Optimization Tips

When to Use Different Handlers

Handler Selection Guide

When NOT to Use Specific Handlers

Additional Learning Resources

Official Python Resources

Related Documentation