Hash Functions

A cryptographic hash function is a one-way fingerprint: it maps any input to a short, fixed-size digest such that the output is deterministic, fast to compute, and practically impossible to reverse or to collide on purpose. This one primitive shows up everywhere, from verifying a download, to deduplicating storage, to anchoring digital signatures. Knowing which functions still hold their guarantees (and which are broken) is essential security hygiene.

Preimage, second-preimage, and collision resistance

The security of a hash is defined by three resistance properties. They sound similar but protect against different attacks, and they fail at different difficulty levels.

Property	The attacker is given	And must find	Generic difficulty (n-bit hash)
Preimage resistance	a digest h	any m with hash(m) = h	2^n
Second-preimage resistance	a message m1	a different m2 with hash(m2) = hash(m1)	2^n
Collision resistance	nothing	any pair m1 != m2 with equal hashes	2^(n/2)

The last row is the one that surprises people. Because of the birthday paradox, finding any collision takes only about 2^(n/2) work, not 2^n. For a 256-bit hash that is 2^128, still far out of reach, which is why SHA-256 is comfortable. But it explains why 128-bit-ish digests (MD5, and effectively SHA-1) fell to collision attacks long before anyone could reverse them.

The avalanche effect ties these together in practice: changing one input bit flips about half the output bits, so the digest reveals nothing about the input and near-identical inputs look completely unrelated.

The MD5/SHA-1 graveyard and what killed them

Two once-ubiquitous hashes are now cryptographically dead. What killed both was collision resistance, exploited in real attacks:

• MD5 (128-bit): collisions were demonstrated in 2004 and became trivial. In 2008 researchers used chosen-prefix MD5 collisions to forge a rogue certificate authority certificate, and the Flame malware (2012) abused MD5 to forge a Microsoft code-signing certificate.
• SHA-1 (160-bit): weaknesses were theorized for years, and in 2017 Google’s SHATTERED produced two different PDFs with the same SHA-1 hash. By 2020 chosen-prefix SHA-1 collisions were cheap enough to break real protocols.

The lesson is that “no known attack yet” is a countdown, not a guarantee. Both are still fine as non-security checksums (detecting accidental corruption), but must never be used where an adversary could benefit from a collision: certificates, signatures, integrity of downloads, or commitments.

Use the SHA-2 family (SHA-256, SHA-384, SHA-512) or SHA-3 (Keccak) instead. SHA-3 uses a completely different internal design (a sponge construction) from SHA-2, so it is a valuable structural backup should SHA-2 ever weaken. BLAKE2 and BLAKE3 are also strong, modern, and very fast.

mkdir -p ~/crypto-lab && cd ~/crypto-lab
echo "release-v1.4.2.tar.gz contents" > artifact.txt

# Modern, safe digests
openssl dgst -sha256 artifact.txt
openssl dgst -sha512 artifact.txt
openssl dgst -sha3-256 artifact.txt

# Legacy digests: fine only as non-security checksums
openssl dgst -md5 artifact.txt
openssl dgst -sha1 artifact.txt

Expected output:

SHA2-256(artifact.txt)= 3f8a1c... (64 hex chars)
SHA2-512(artifact.txt)= b90e77... (128 hex chars)
SHA3-256(artifact.txt)= 5d21ab... (64 hex chars)
MD5(artifact.txt)= 9e107d9d... (32 hex chars, do not trust for security)
SHA1(artifact.txt)= 2fd4e1c6... (40 hex chars, do not trust for security)

Uses: integrity, dedupe, commitments, content addressing

Cryptographic hashes are load-bearing in far more than “checksum a download”:

• Integrity verification: publish the SHA-256 of a release so downloaders can detect corruption or tampering. Verify by recomputing and comparing.
• Deduplication: storage systems key blocks by their hash, so identical content is stored once. Collision resistance is what makes this safe.
• Commitments: publish hash(value) now to lock in a choice, then reveal value later. Preimage resistance hides it; collision resistance stops you from changing your mind.
• Content addressing: Git names every object by its hash, and IPFS addresses content by hash. The name is derived from the data, so any change produces a new name and tampering is self-evident. (This is also why Git’s historical use of SHA-1 prompted a move toward SHA-256.)

A quick integrity-check workflow:

cd ~/crypto-lab
openssl dgst -sha256 artifact.txt | awk '{print $2"  artifact.txt"}' > artifact.sha256
# ... later, or on another machine ...
shasum -a 256 -c artifact.sha256

Expected output:

artifact.txt: OK

Remember that a plain hash proves integrity, not authenticity. Anyone can recompute a hash after altering the file, so publish the digest over a trusted channel, or use a keyed MAC or signature. See HMAC & MACs for adding authenticity, and note that password storage needs deliberately slow hashes instead, covered in Password Hashing.

Practical Guidance

1. Default to SHA-256 for general integrity and fingerprinting; use SHA-512 or SHA-3-256 where you want extra margin or a different internal design.
2. Never use MD5 or SHA-1 for any security purpose. They are broken against collisions and only acceptable as accidental-corruption checksums.
3. Remember collisions cost about 2^(n/2) work, so pick a digest at least twice the security level you need.
4. Do not rely on a bare hash for authenticity. Distribute the digest over a trusted channel, or use a MAC or signature.
5. For content addressing and dedupe, rely on collision resistance and choose a modern hash so the naming scheme stays safe long-term.
6. Use audited implementations (OpenSSL, libsodium, language standard libraries). Never write your own hash function.