Performance

Optimize Git operations for large repositories, improve clone times, and maintain efficient workflows.

Repository Performance Analysis

Checking Repository Size

# Show repository statistics
git count-objects -vH

# Example output:
# count 1234           # Loose objects
# size 5.67 MiB        # Loose objects size
# in-pack 5678         # Packed objects
# packs 2              # Number of pack files
# size-pack 45.67 MiB  # Pack files size
# prune-packable 0     # Objects that can be pruned
# garbage 0            # Corrupted objects
# size-garbage 0 bytes # Corrupted objects size

# Show disk usage
du -sh .git/

# Show largest files
git ls-tree -r -t -l --full-name HEAD | sort -n -k 4 | tail -20

Finding Large Objects

# Find largest objects in repository
git rev-list --objects --all | \
  git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \
  awk '/^blob/ {print substr($0,6)}' | \
  sort --numeric-sort --key=2 | \
  tail -20

# Find large files by path
git ls-tree -r -t -l --full-name HEAD | \
  sort -n -k 4 | \
  tail -10 | \
  awk '{print $4 " " $5}'

# Show object count by type
git cat-file --batch-check --batch-all-objects | \
  cut -d' ' -f1 | sort | uniq -c | sort -nr

Repository Maintenance

Garbage Collection

# Standard garbage collection
git gc

# Aggressive garbage collection (slower but more thorough)
git gc --aggressive

# Garbage collection with immediate pruning
git gc --prune=now

# Dry run to see what would be cleaned
git gc --dry-run

# Configure auto garbage collection
git config gc.auto 6700          # Run gc after 6700 loose objects
git config gc.autoPackLimit 50   # Run gc after 50 pack files

Manual Cleanup

# Repack repository for optimal storage
git repack -ad

# Aggressive repack
git repack -adf --window=250 --depth=250

# Prune unreachable objects
git prune

# Prune with expire time
git prune --expire=2.weeks.ago

# Clean up reflog
git reflog expire --expire=90.days --all
git reflog expire --expire-unreachable=30.days --all

Automated Maintenance

# Enable automatic maintenance (Git 2.31+)
git maintenance start

# Run maintenance manually
git maintenance run

# Configure maintenance schedule
git config maintenance.gc.schedule daily
git config maintenance.prefetch.schedule hourly
git config maintenance.loose-objects.schedule daily
git config maintenance.incremental-repack.schedule weekly

# Stop automatic maintenance
git maintenance stop

Large Repository Optimization

Shallow Clones

# Clone with limited history
git clone --depth 1 https://github.com/user/repo.git

# Clone with specific depth
git clone --depth 50 https://github.com/user/repo.git

# Clone specific branch with depth
git clone --depth 1 --branch main https://github.com/user/repo.git

# Deepen shallow clone
git fetch --depth=100

# Convert shallow to full clone
git fetch --unshallow

# Check if repository is shallow
git rev-parse --is-shallow-repository

Partial Clone

# Clone without blobs (large files)
git clone --filter=blob:none https://github.com/user/repo.git

# Clone without large blobs
git clone --filter=blob:limit=1m https://github.com/user/repo.git

# Clone without trees (directory structure)
git clone --filter=tree:0 https://github.com/user/repo.git

# Combine filters
git clone --filter=combine:blob:none+tree:0 https://github.com/user/repo.git

# Fetch missing objects when needed
git fetch --filter=blob:none origin main

Sparse Checkout

# Enable sparse checkout
git config core.sparseCheckout true

# Create sparse-checkout file
echo "src/" >> .git/info/sparse-checkout
echo "docs/" >> .git/info/sparse-checkout
echo "!docs/temp/" >> .git/info/sparse-checkout

# Apply sparse checkout
git read-tree -m -u HEAD

# Cone mode sparse checkout (Git 2.25+)
git sparse-checkout init --cone
git sparse-checkout set src docs tests

# Show sparse-checkout patterns
git sparse-checkout list

# Disable sparse checkout
git sparse-checkout disable

Git LFS for Large Files

Setup Git LFS

# Install Git LFS
git lfs install

# Install for specific repository
git lfs install --local

# Track large files
git lfs track "*.zip"
git lfs track "*.mp4"
git lfs track "*.psd"
git lfs track "assets/images/*"

# View tracked patterns
git lfs track

# Add .gitattributes to repository
git add .gitattributes
git commit -m "Add Git LFS tracking"

Working with LFS

# Add large file
git add large-file.zip
git commit -m "Add large file"

# Show LFS files
git lfs ls-files

# Show LFS status
git lfs status

# Fetch LFS objects
git lfs fetch
git lfs fetch --all

# Pull LFS objects
git lfs pull

# Push LFS objects
git lfs push origin main

LFS Maintenance

# Clean LFS cache
git lfs clean

# Prune old LFS objects
git lfs prune

# Prune with dry run
git lfs prune --dry-run

# Prune with verification
git lfs prune --verify-remote

# Show LFS environment
git lfs env

Network Performance

Protocol Optimization

# Use SSH instead of HTTPS (faster)
git remote set-url origin git@github.com:user/repo.git

# Enable HTTP/2
git config --global http.version HTTP/2

# Configure connection settings
git config --global http.maxRequests 12
git config --global http.postBuffer 524288000

# Set low speed limits
git config --global http.lowSpeedLimit 1000
git config --global http.lowSpeedTime 30

Transfer Optimization

# Enable compression
git config --global core.compression 6

# Configure pack compression
git config --global pack.compression 9

# Enable delta compression
git config --global pack.deltaCacheSize 256m

# Configure fetch settings
git config --global fetch.parallel 4
git config --global fetch.writeCommitGraph true

Index Performance

Index Optimization

# Use index version 4 (faster)
git config --global index.version 4

# Enable split index
git config --global core.splitIndex true

# Enable file system cache
git config --global core.fscache true

# Enable preload index
git config --global core.preloadindex true

# Refresh index
git update-index --refresh

File System Monitoring

# Enable file system monitor
git config --global core.fsmonitor true

# Configure watchman (if installed)
git config --global core.fsmonitor watchman

# Check fsmonitor status
git config --get core.fsmonitor

Pack File Optimization

Pack Configuration

# Configure pack settings
git config --global pack.threads 0           # Use all available CPUs
git config --global pack.windowMemory 512m   # Memory for pack window
git config --global pack.packSizeLimit 2g    # Maximum pack size
git config --global pack.deltaCacheSize 256m # Delta cache size

# Repack with custom settings
git repack -adf --window=250 --depth=250

Multi-Pack Index

# Create multi-pack index
git multi-pack-index write

# Verify multi-pack index
git multi-pack-index verify

# Repack with multi-pack index
git multi-pack-index repack --batch-size=1g

# Expire multi-pack index
git multi-pack-index expire

Bitmap Indexes

# Create bitmap index
git repack -adb

# Verify bitmap index
git show-index < .git/objects/pack/pack-*.bitmap

# Use bitmap for rev-list
git rev-list --use-bitmap-index --count --all

Performance Monitoring

Tracing Git Operations

# General tracing
GIT_TRACE=1 git status

# Performance tracing
GIT_TRACE_PERFORMANCE=1 git clone https://github.com/user/repo.git

# Pack tracing
GIT_TRACE_PACK_ACCESS=1 git log --oneline

# Setup tracing
GIT_TRACE_SETUP=1 git status

Timing Commands

# Time git operations
time git clone https://github.com/user/repo.git
time git status
time git log --oneline

# Measure specific operations
time git repack -adf
time git gc --aggressive

Troubleshooting Performance Issues

Common Performance Problems

Slow Status/Diff Operations

# Enable file system cache
git config core.fscache true

# Use sparse checkout
git sparse-checkout init --cone
git sparse-checkout set important-dirs/

# Refresh index
git update-index --refresh

Large Clone Times

# Use shallow clone
git clone --depth 1 repo-url

# Use partial clone
git clone --filter=blob:none repo-url

# Use SSH protocol
git clone git@github.com:user/repo.git

Slow Push/Pull

# Enable compression
git config core.compression 6

# Use SSH
git remote set-url origin git@github.com:user/repo.git

# Increase post buffer
git config http.postBuffer 524288000

Diagnostic Commands

# Repository health check
git fsck --full

# Verify pack files
git verify-pack -v .git/objects/pack/pack-*.idx

# Check connectivity
git fsck --connectivity-only

# Show repository statistics
git count-objects -vH

Best Practices

Repository Structure

Avoid large files - Use Git LFS for binaries
Use .gitignore - Exclude build artifacts and temporary files
Limit file count - Don't track every generated file
Regular maintenance - Schedule garbage collection
Monitor size - Track repository growth

Workflow Optimization

Use shallow clones for CI - Faster build times
Enable sparse checkout - Work with subset of files
Clean up branches - Remove merged branches regularly
Optimize commit size - Make focused commits
Use appropriate protocols - SSH for frequent operations

Configuration for Performance

# Recommended performance settings
git config --global core.preloadindex true
git config --global core.fscache true
git config --global index.version 4
git config --global pack.threads 0
git config --global gc.auto 256
git config --global feature.manyFiles true
git config --global fetch.writeCommitGraph true

Advanced Performance Techniques

Commit Graph

# Write commit graph
git commit-graph write --reachable

# Verify commit graph
git commit-graph verify

# Use commit graph for operations
git log --graph --oneline  # Faster with commit graph

Reference Packing

# Pack references
git pack-refs --all

# Pack only branches and tags
git pack-refs --all --prune

# Check packed references
cat .git/packed-refs

Worktree Performance

# Create worktree for parallel work
git worktree add ../feature-branch feature-branch

# Benefits:
# - No branch switching overhead
# - Parallel builds in different directories
# - Separate working directories per feature

Monitoring and Alerts

Performance Metrics

# Script to monitor repository performance
#!/bin/bash

# Repository size
REPO_SIZE=$(du -sh .git | cut -f1)
echo "Repository size: $REPO_SIZE"

# Object count
OBJECT_COUNT=$(git count-objects -v | grep "count" | cut -d' ' -f2)
echo "Object count: $OBJECT_COUNT"

# Pack file count
PACK_COUNT=$(git count-objects -v | grep "packs" | cut -d' ' -f2)
echo "Pack files: $PACK_COUNT"

# Check for large files
echo "Largest files:"
git ls-tree -r -t -l --full-name HEAD | \
  sort -n -k 4 | \
  tail -5 | \
  awk '{print $4 " " $5}'

Automation

# Automated maintenance script
#!/bin/bash

# Daily maintenance
if [ $(date +%u) -eq 1 ]; then  # Monday
    git gc --aggressive
else
    git gc
fi

# Weekly cleanup
if [ $(date +%u) -eq 7 ]; then  # Sunday
    git reflog expire --expire=90.days --all
    git prune --expire=2.weeks.ago
fi

# Alert if repository is too large
REPO_SIZE_KB=$(du -s .git | cut -f1)
if [ $REPO_SIZE_KB -gt 1000000 ]; then  # 1GB
    echo "Warning: Repository size exceeds 1GB"
fi

Integration with CI/CD

CI Performance Optimization

# Optimized CI clone
git clone --depth 1 --single-branch https://github.com/user/repo.git

# Fetch only needed files
git sparse-checkout init --cone
git sparse-checkout set src tests

# Cache Git objects
# Use CI caching for .git directory

Build Performance

# Parallel operations
git config pack.threads 0
git config checkout.workers 4

# Optimize for build servers
git config core.preloadindex true
git config core.fscache true
git config index.version 4

See Configuration for performance-related settings and Advanced Git for sophisticated optimization techniques.

Repository Performance Analysis​

Checking Repository Size​

Finding Large Objects​

Repository Maintenance​

Garbage Collection​

Manual Cleanup​

Automated Maintenance​

Large Repository Optimization​

Shallow Clones​

Partial Clone​

Sparse Checkout​

Git LFS for Large Files​

Setup Git LFS​

Working with LFS​

LFS Maintenance​

Network Performance​

Protocol Optimization​

Transfer Optimization​

Index Performance​

Index Optimization​

File System Monitoring​

Pack File Optimization​

Pack Configuration​

Multi-Pack Index​

Bitmap Indexes​

Performance Monitoring​

Tracing Git Operations​

Timing Commands​

Troubleshooting Performance Issues​

Common Performance Problems​

Diagnostic Commands​

Best Practices​

Repository Structure​

Workflow Optimization​

Configuration for Performance​

Advanced Performance Techniques​

Commit Graph​

Reference Packing​

Worktree Performance​

Monitoring and Alerts​

Performance Metrics​

Automation​

Integration with CI/CD​

CI Performance Optimization​

Build Performance​