Gibbons_bigtable-updated

Google, Inc.
Appeared in OSDI’06
Slides adapted from a reading group presentation by Erik Paulson, U. Washington October 2006
Google Scale
• Lots of data
• Column family:qualifier. Family is heavyweight, qualifier lightweight
• Column-oriented physical store- rows are sparse!
• Lookup, insert, delete API • Each read or write of data under a single row key is atomic
1000-byte values per server per second
16
Aggregate rate
17
Application at Google
18
Lessons learned
• Only implement some of the requirements, since the last is probably not needed • Many types of failure possible • Big systems need proper systems-level monitoring • Value simple designs
4
Chubby
• {lock/file/name} service • Coarse-grained locks, can store small amount of data in a lock • 5 replicas, need a majority vote to be active • Also an OSDI ’06 Paper
5
Data model: a big map
• <Row,
Column, Timestamp> triple for key
• Each value is an uninterpreted array of bytes
• Arbitrary “columns” on a row-by-row basis
Tablet Insert Insert Memtable (sorted) apple_two_E boat
Delete
Insert Delete Insert
SSTable SSTable (sorted) (sorted)
13
Compactions
• Minor compaction – convert a full memtable into an SSTable, and start a new memtable
19Biblioteka Google Bigtable
Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber
– Popular compression scheme: (1) long common strings across a large window, (2) standard compression across small 16KB windows
• Bloom Filters on locality groups – avoid searching SSTable
6
BigTable vs. Relational DB
• • • • No table-wide integrity constraints No multi-row transactions Uninterpreted values: No aggregation over data Immutable data ala Versioning DBs
Tablet aardvark
Tablet
apple
apple_two_E
boat
SSTable SSTable
SSTable SSTable
10
Finding a tablet
• Client library caches tablet locations • Metadata table includes log of all events pertaining to each tablet
– Can specify: keep last N versions or last N days
• C++ functions, not SQL (no complex queries) • Clients indicate what data to cache in memory • Data stored lexicographically sorted: Client control locality by the naming of rows & columns
7
SSTable
• Immutable, sorted file of key-value pairs • Chunks of data plus an index
– Index is of block ranges, not values – Index loaded into memory when SSTable is opened – Lookup is a single disk seek
– Avoid mingling data, i.e. page contents and page metadata – Can keep some locality groups all in memory
• Can compress locality groups (10:1 typical)
12
Editing/Reading a table
• Mutations are committed to a commit log (in GFS) • Then applied to an in-memory version (memtable) • For concurrency, each memtable row is copy-on-write • Reads applied to merged view of SSTables & memtable • Reads & writes continue during tablet split or merge
• Firm believers in the End-to-End argument • 450,000 machines (NYTimes estimate, June’06)
2
Building Blocks
• • • • Scheduler (Google WorkQueue) Google File System Chubby Lock service Two other pieces helpful but not required
• Alternatively, client can load SSTable into mem
64K block 64K block 64K block SSTable
Index
8
Tablet
• Contains some range of rows of the table • Unit of distribution & load balance • Built out of multiple SSTables
11
Servers
• Tablet servers manage tablets, multiple tablets per server. Each tablet is 100-200 MBs
– Each tablet lives at only one server – Tablet server splits tablets that get too big
• Master responsible for load balancing and fault tolerance
– Use Chubby to monitor health of tablet servers, restart failed servers – GFS replicates data. Prefer to start tablet server on same machine that the data is already at
– Reduce memory usage – Reduce log traffic on restart
• Merging compaction
– Reduce number of SSTables – Good place to apply policy “keep only N versions”
• Major compaction
– Merging compaction that results in only one SSTable – No deletion records, only live data
14
Locality Groups
• Group column families together into an SSTable
– Sawzall – MapReduce
• BigTable: Build a more application-friendly storage service using these parts