# Consistent Hashing Implementation

A comprehensive implementation of consistent hashing with extensive test coverage and demo applications.

## Overview

Consistent hashing is a distributed hashing technique that minimizes the number of keys that need to be redistributed when the hash table is resized. It's widely used in:

- **Distributed Cache Systems** (Redis, Memcached)
- **Load Balancing** (Nginx, HAProxy)
- **Key-Value Stores** (DynamoDB, Cassandra)
- **Content Delivery Networks** (CDN routing)
- **Peer-to-Peer Systems** (BitTorrent)

## Key Properties

✓ **Minimal Key Redistribution**: Only ~1/n keys need to be redistributed when a node is added/removed (n = number of nodes)

✓ **Load Balancing**: Keys are distributed roughly evenly across nodes

✓ **Virtual Nodes**: Supports virtual nodes for better distribution uniformity

✓ **Fast Lookup**: O(log n) time complexity for key lookup

✓ **Scalability**: Handles dynamic node addition/removal efficiently

## Project Structure

```
ConsistentHashing/
├── src/
│   ├── main/java/org/ds/
│   │   ├── ConsistentHash.java       # Core implementation
│   │   ├── ConsistentHashDemo.java   # 8 comprehensive demos
│   │   └── TestRunner.java           # 27 test cases
│   └── test/java/org/ds/
│       └── ConsistentHashTest.java   # JUnit 5 test suite (optional)
├── pom.xml                           # Maven configuration
└── README.md                          # This file
```

## Core Components

### 1. ConsistentHash<T> Class

Main implementation with the following features:

```java
// Create with default virtual nodes (150)
ConsistentHash<String> hash = new ConsistentHash<>();

// Create with custom virtual nodes count
ConsistentHash<String> hash = new ConsistentHash<>(500);

// Add nodes
hash.addNode("server1");
hash.addNode("server2");

// Get responsible node for a key
String node = hash.getNode("user:123");

// Remove nodes
hash.removeNode("server1");

// Get statistics
Map<String, Integer> distribution = hash.getDistribution(10000);
String stats = hash.getStatistics(10000);
```

### 2. Hash Functions

**MD5HashFunction** (Default)
- Provides uniform distribution
- Based on MD5 cryptographic hash
- Recommended for production use

**SimpleHashFunction**
- Fast alternative using String.hashCode()
- Less uniform but suitable for some use cases

**Custom Hash Functions**
- Implement `ConsistentHash.HashFunction` interface
- Override `hash(String key)` method

## Usage Examples

### Example 1: Basic Usage

```java
ConsistentHash<String> hash = new ConsistentHash<>(150);

// Add servers
hash.addNode("cache-01");
hash.addNode("cache-02");
hash.addNode("cache-03");

// Route keys to servers
String server = hash.getNode("user:12345");
System.out.println(server);  // Consistent routing
```

### Example 2: Cache Cluster

```java
ConsistentHash<String> cache = new ConsistentHash<>(150);

// Initial setup
cache.addNode("cache-01");
cache.addNode("cache-02");
cache.addNode("cache-03");

// Server failure
cache.removeNode("cache-02");  // Only cache-02 data is lost

// Server recovery
cache.addNode("cache-02");     // Automatically rebalances
```

### Example 3: Load Balancer

```java
ConsistentHash<String> lb = new ConsistentHash<>(150);

lb.addNode("web-server-1");
lb.addNode("web-server-2");
lb.addNode("web-server-3");

// Route user session to consistent server
String sessionKey = "session:" + userId;
String server = lb.getNode(sessionKey);
// Same user always routes to same server (sticky sessions)
```

### Example 4: Custom Hash Function

```java
ConsistentHash<String> hash = new ConsistentHash<>(150, 
    new ConsistentHash.SimpleHashFunction());

// Or with lambda
ConsistentHash<String> hash = new ConsistentHash<>(150,
    key -> Math.abs((long) key.hashCode() * 31));
```

## Virtual Nodes

Virtual nodes improve load distribution uniformity by replicating each physical node multiple times on the ring.

### Impact Analysis

| Virtual Nodes | Distribution Uniformity | Ring Size |
|---|---|---|
| 1 | Poor (20-30% imbalance) | n |
| 10 | Good (10-15% imbalance) | 10n |
| 150 | Excellent (3-5% imbalance) | 150n |
| 500 | Near-perfect (1-2% imbalance) | 500n |

## Test Coverage

### Test Suite: 27 Comprehensive Tests

**Basic Functionality (5 tests)**
- Empty ring handling
- Single/multiple node management
- Consistent key routing
- Key consistency validation

**Node Operations (3 tests)**
- Node removal
- Routing after node changes
- Complete node removal

**Load Distribution (3 tests)**
- Key distribution analysis
- Virtual nodes impact
- Ring size vs node count relationship

**Key Redistribution (4 tests)**
- Minimal key movement on node addition
- Minimal key movement on node removal
- Rapid node additions
- Rapid node removals

**Edge Cases (5 tests)**
- Duplicate node additions
- Removing non-existent nodes
- Empty string keys
- Very long keys (10,000+ characters)
- Special characters in keys/nodes

**Virtual Nodes (2 tests)**
- Virtual node count validation
- Distribution improvement verification

**Hash Functions (2 tests)**
- MD5 hash consistency
- Hash diversity verification

**Complex Scenarios (3 tests)**
- Complete lifecycle testing
- Cache cluster failure recovery
- Load balancer simulation

## Demo Applications

### 8 Comprehensive Demos

**Demo 1: Basic Functionality**
- Node addition and key routing
- Virtual node statistics

**Demo 2: Load Distribution**
- Key distribution across nodes
- Distribution metrics (avg, max, min, imbalance)

**Demo 3: Virtual Nodes Impact**
- Comparison of 1, 10, 50, 150, 500 virtual nodes
- Shows improvement in distribution uniformity

**Demo 4: Key Redistribution**
- Minimal key movement on node addition
- Minimal key movement on node removal

**Demo 5: Cache Cluster Failure**
- Initial distribution
- Node failure and recovery
- Automatic load rebalancing

**Demo 6: Load Balancer**
- User session routing
- Sticky sessions (same user → same server)
- Load distribution

**Demo 7: Edge Cases**
- Empty ring handling
- Single/multiple nodes
- Special characters in keys and node names
- Large node counts (100 nodes)

**Demo 8: Performance Analysis**
- 1 million key lookups
- Throughput measurement
- Average lookup time

## Running the Project

### Compile

```bash
# Compile all source files
javac -d target/classes src/main/java/org/ds/*.java
```

### Run Demo

```bash
# Run all 8 demos
java -cp target/classes org.ds.ConsistentHashDemo
```

### Run Tests

```bash
# Run 27 comprehensive tests
java -cp target/classes org.ds.TestRunner

# Or with JUnit 5 (requires Maven)
mvn test
```

## Performance Characteristics

### Time Complexity
- **Node Addition**: O(k) where k = virtual nodes per physical node
- **Node Removal**: O(k) where k = virtual nodes per physical node
- **Key Lookup**: O(log n) where n = number of virtual nodes
- **Distribution Query**: O(sample size)

### Space Complexity
- **Ring Storage**: O(v × p) where v = virtual nodes per physical node, p = physical nodes
- **Node Set**: O(p) where p = physical nodes

### Key Redistribution
- **Node Addition**: ~1/n of keys need redistribution (n = number of nodes before)
- **Node Removal**: ~1/(n-1) of keys need redistribution

## Example Output

### Demo Output
```
╔════════════════════════════════════════════════════════════════╗
║          CONSISTENT HASHING COMPREHENSIVE DEMO                 ║
╚════════════════════════════════════════════════════════════════╝

┌─ DEMO 2: LOAD DISTRIBUTION ───────────────────────────────────────┐
│ Key distribution across multiple nodes                          │

Distribution of 10000 keys across 4 servers:

  server-A    :  2692 [█████████████] 26.92%
  server-B    :  2565 [████████████] 25.65%
  server-C    :  2541 [████████████] 25.41%
  server-D    :  2202 [███████████] 22.02%

Distribution Metrics:
  Average:   2500
  Max:       2692
  Min:       2202
  Imbalance: 19.60%
```

### Test Output
```
╔════════════════════════════════════════════════════════════════╗
║       CONSISTENT HASHING - COMPREHENSIVE TEST SUITE            ║
╚════════════════════════════════════════════════════════════════╝

✓ testEmptyRing
✓ testAddSingleNode
✓ testAddMultipleNodes
... (24 more tests)

╔════════════════════════════════════════════════════════════════╗
║                        TEST SUMMARY                             ║
╚════════════════════════════════════════════════════════════════╝
Total Tests:   27
Passed:        27 ✓
Failed:        0 ✗

Success Rate:  100.0%

🎉 ALL TESTS PASSED!
```

## Real-World Applications

### 1. Memcached/Redis Cluster
```
Client → Consistent Hash → cache-node-1, cache-node-2, cache-node-3
Node failure → Only cache data from failed node is lost
Minimal rehashing of existing keys
```

### 2. Load Balancing
```
HTTP Request → Consistent Hash(session_id) → web-server
Same session always routes to same server (sticky sessions)
Easy server addition/removal without user disruption
```

### 3. Database Sharding
```
Database Partition Key → Consistent Hash → Shard-1, Shard-2, ..., Shard-n
Minimal data migration on shard addition
Uniform distribution of data across shards
```

### 4. CDN Content Routing
```
Content ID → Consistent Hash → Edge-Server-A, Edge-Server-B, ...
Consistent content routing
Cache affinity maintained
```

## Advantages vs Naive Approaches

### Naive Hash (hash(key) % n)
- ❌ k-1 out of n keys need rehashing when nodes change
- ❌ Causes thundering herd/cache stampede
- ❌ Difficult to manage dynamic scaling

### Consistent Hashing
- ✅ Only k/n keys need rehashing (k = keys affected)
- ✅ Graceful degradation on node failure
- ✅ Supports seamless scaling
- ✅ Better cache locality

## Mathematical Properties

### Ring Balance
With v virtual nodes per physical node and n physical nodes:
- **Ring Size**: v × n
- **Expected Keys per Node**: Total Keys / n
- **Standard Deviation**: ≈ √(v × n) / n
- **Distribution Uniformity Improves**: ∝ √(v)

### Probability of Key Migration
When adding one node:
- **Probability a key migrates**: 1/n
- **Expected keys migrated**: Total Keys / n
- **Keys staying on original node**: Total Keys × (n-1)/n

## Limitations and Considerations

1. **Virtual Node Count**: Higher = better distribution but more memory
2. **Hash Function Quality**: Poor hash function leads to uneven distribution
3. **Node Metadata**: May need to track node capabilities/capacity
4. **Read Repair**: In replicated systems, need mechanism to sync after changes
5. **Replication**: Often combined with replication for fault tolerance

## Future Enhancements

- [ ] Weighted nodes (node capacity varies)
- [ ] Replica placement (backup nodes)
- [ ] Ring visualization
- [ ] Partition key suggestions
- [ ] Automatic virtual node tuning

## References

1. **Original Paper**: "Consistent Hashing and Random Trees" by Karger, Lehman, et al.
2. **Applications**: 
   - Memcached by Brad Fitzpatrick
   - Amazon's DynamoDB
   - Cassandra's partition algorithm
   - Nginx consistent hash module

## License

Educational implementation for teaching purposes.

## Author

Created for Advanced Data Structures course - Consistent Hashing Lab

---

**Last Updated**: March 2026

For questions or improvements, refer to the comprehensive test suite and demo applications.

