How Discord Migrated Trillions of Messages From Cassandra to ScyllaDB

ScyllaDB = Cassandra reimagined in C++

𝐊𝐞𝐲 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞

» Shard-per-Core ➞ Each CPU core gets dedicated data partition + memory

» Seastar Framework ➞ C++ async framework, zero-copy networking

» Storage ➞ In-memory memtables → immutable SSTables on disk

» Consensus ➞ Optimized Paxos for consistency across replicas

» Communication ➞ Gossip protocol for cluster coordination

𝐃𝐢𝐬𝐜𝐨𝐫𝐝'𝐬 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞

» 177 Cassandra nodes storing trillions of messages

» Hot partitions causing 40-125ms p99 read latency

» Unpredictable performance during traffic spikes

𝐌𝐢𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐬𝐮𝐥𝐭𝐬

» Nodes ➞ 177 to 72 (60% reduction)

» Read Latency ➞ 40-125ms to 15ms p99

» Write Latency ➞ 5-70ms to 5ms p99

» Storage ➞ 9TB per node (2x capacity)

» Migration Time ➞ 9 days (custom Rust tool)

 𝐊𝐞𝐲 𝐁𝐞𝐧𝐞𝐟𝐢𝐭𝐬

» No garbage collection issues (vs Java Cassandra)

» Linear scalability with core count

» Full CQL compatibility for easy migration

» Eliminated operational firefights

ScyllaDB's C++ foundation + shard-per-core design = predictable performance at scale