e0fac66223
## Proposed changes Hadoop snappycodec source : https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/codec/SnappyCodec.cc Example: OriginData(The original data will be divided into several large data block.) : large data block1 | large data block2 | large data block3 | .... The large data block will be divided into several small data block. Suppose a large data block is divided into three small blocks: large data block1: | small block1 | small block2 | small block3 | CompressData: <A [B1 compress(small block1) ] [B2 compress(small block1) ] [B3 compress(small block1)]> A : original length of the current block of large data block. sizeof(A) = 4 bytes. A = length(small block1) + length(small block2) + length(small block3) Bx : length of small data block bx. sizeof(Bx) = 4 bytes. Bx = length(compress(small blockx))