Git Internals Deep Dive — Objects, Packfiles, GC
AdvancedUnder the hood: how Git stores objects in loose and packed format, how the index works, how garbage collection reclaims space, and how remote protocols transfer data efficiently.
Overview
Most developers never need to know this — until they do. A repository growing to gigabytes, an inexplicable "not a valid SHA-1" error, or an interview at a top company asking "how does git push work internally" — these are the moments where internals knowledge pays off. Git's design is elegant: the object store is an append-only content-addressable database, packfiles enable efficient transfer, and the index is a binary cache of the working directory state. Understanding these makes advanced Git operations (gc, repack, filter-repo, partial clone) approachable.
Loose Objects and Packfiles
Every object initially lives as a zlib-compressed file in .git/objects/xx/yyy... (first 2 chars = directory, remaining 38 = filename). Git periodically runs auto-gc to pack loose objects into binary packfiles, which are much smaller and faster to transfer.
# Loose objects live here (one file per object)
ls .git/objects/
# 4a/ 9c/ f2/ pack/ info/
ls .git/objects/4a/
# 7f3b2c... (rest of SHA-1 hash)
# Pack loose objects manually
git gc
git gc --aggressive # deeper compression, slower
# Inspect a packfile
ls .git/objects/pack/
# pack-abc123.idx ↠index for fast lookup
# pack-abc123.pack ↠compressed objects
git verify-pack -v .git/objects/pack/pack-abc123.idx
# Outputs: hash, type, size, compressed-size, offset
# Helps find large objects bloating the repo
# Git auto-gc triggers around 6700 loose objects
# Manual: git count-objects -v
# count: 150
# size: 600 (KB, loose objects)
# in-pack: 45000 (objects in packfiles)The Index (Staging Area) Internals
The staging area is a binary file at .git/index that caches the state of the working directory. Each entry stores file path, mode, size, mtime, and blob SHA-1. git status compares this cache to the working directory and HEAD commit — this is why git status is fast.
# The index is a binary cache of working directory state
# git ls-files shows what's in the index
git ls-files --stage
# 100644 4a7f3b2c 0 src/auth.js
# 100644 9c2d8e1f 0 src/index.js
# Mode SHA-1 Stage Path
# During a merge, conflicting files have multiple stages:
# 100644 abc123 1 conflicted.js ↠common ancestor
# 100644 def456 2 conflicted.js ↠HEAD version
# 100644 789abc 3 conflicted.js ↠merging branch version
# git add updates the index (computes SHA-1, writes blob, updates index)
# git commit creates a tree from the index, then a commit pointing to it
# Diagnose index issues
git ls-files --unmerged # see unresolved conflicts
git ls-files --others # see untracked filesHow git push/pull Work Over the Wire
Git uses the "smart HTTP" or "SSH" protocol. On push, Git negotiates which objects the remote is missing and sends only those, compressed in a packfile. This is why initial clone is large but subsequent pushes are small.
// git push protocol (simplified)
//
// 1. Reference Discovery
// Client → Server: "What branches and commit hashes do you have?"
// Server → Client: "main → a3f8c12, v2.0 → 9d2e441"
//
// 2. Negotiation
// Client computes: what objects does server NOT have?
// = git rev-list HEAD ^<server's heads>
//
// 3. Pack sending
// Client packs those objects → sends as packfile
// Server receives, unpacks, updates refs
//
// 4. Result
// Server → Client: "updated refs/heads/main"
# You can trace the protocol:
GIT_TRACE_PACKET=1 git push origin main 2>&1 | head -30
# Bandwidth diagnosis
git count-objects -v # see loose vs packed
git gc --prune=now # clean up unreachable objects
# filter large blobs from history (DESTRUCTIVE — rewrites all commits)
git filter-repo --strip-blobs-bigger-than 10MKey Points to Remember
- 1Loose objects: one zlib-compressed file per object. Packfiles: many objects in one binary, with delta compression
- 2git gc packs loose objects into packfiles — reduces disk usage and speeds up network transfer
- 3.git/index is a binary cache mapping file paths to blob hashes — git status reads this, not the disk
- 4git push only sends objects the remote doesn't have — negotiated before transfer begins
- 5git filter-repo (not git filter-branch) is the modern tool to permanently remove large files from history
Interview Questions
Sign in to ask AriaHow does Git store data internally? What is a packfile?
Why is git push efficient — how does Git avoid sending objects the remote already has?
What is the Git index? Why does git status check it instead of scanning the filesystem each time?
Ask Aria about Git Internals Deep Dive — Objects, Packfiles, GC
Your personal AI tutor — ask anything about this concept
Revision Status
Personal Notes
Sign in to save personal notes for this topic.
Discussion
Sign in to join the discussion.