diff --git a/README.rst b/README.rst
new file mode 100644
index 0000000..34073b6
--- /dev/null
+++ b/README.rst
@@ -0,0 +1,567 @@
+TurboPFor: Fastest Integer Compression [](https://travis-ci.org/powturbo/TurboPFor)
+======================================
+* **TurboPFor: The new synonym for "integer compression"**
+ * :new: (2019.7) all TurboPFor functions now available under 64 bits ARMv8 including NEON SIMD.
+ * 100% C (C++ headers), as simple as memcpy
+ * :+1: **Java** Critical Natives/JNI. Access TurboPFor **incl. SIMD/AVX2!** from Java as fast as calling from C
+ * :sparkles: **FULL** range 8/16/32/64 bits scalar + 16/32/64 bits SIMD functions
+ * No other "Integer Compression" compress/decompress faster
+ * :sparkles: Direct Access, **integrated** (SIMD/AVX2) FOR/delta/Delta of Delta/Zigzag for sorted/unsorted arrays
+ * :new: **16 bits** + **64 bits** SIMD integrated functions
+* **For/PFor/PForDelta**
+ * **Novel TurboPFor** (PFor/PForDelta) scheme w./ **direct access** + **SIMD/AVX2**. :new:**+RLE**
+ * Outstanding compression/speed. More efficient than **ANY** other fast "integer compression" scheme.
+ * Compress 70 times faster and decompress up to 4 times faster than OptPFD
+* **Bit Packing**
+ * Fastest and most efficient **"SIMD Bit Packing"** **10 Billions integers/sec (40Gb/s!)**
+ * Scalar **"Bit Packing"** decoding nearly as fast as SIMD-Packing in realistic (No "pure cache") scenarios
+ * **Direct/Random Access** : Access any single bit packed entry with **zero decompression**
+* **Variable byte**
+ * Scalar **"Variable Byte"** faster than **ANY** other (incl. SIMD) implementation
+* **Simple family**
+ * **Novel** **"Variable Simple"** (incl. **RLE**) faster and more efficient than simple16, simple-8b
+* **Elias fano**
+ * Fastest **"Elias Fano"** implementation w/ or w/o SIMD/AVX2
++ **Transform**
+ * Scalar & SIMD Transform: Delta, Zigzag, Zigzag of delta, XOR, Transpose/Shuffle,
+ * :new: **lossy** floating point compression with *TurboPFor* or [TurboTranspose](https://github.com/powturbo/TurboTranspose)+lz77
+* **Floating Point Compression**
+ * Delta/Zigzag + improved gorilla style + (Differential) Finite Context Method FCM/DFCM floating point compression
+ * Using **TurboPFor**, unsurpassed compression and more than 5 GB/s throughput
+ * :new: Error bound **lossy** floating point compression
+* :new: **Time Series Compression**
+ * **Fastest Gorilla** 16/32/64 bits style compression (:new: **zigzag of delta** + **RLE**).
+ * can compress times series to only 0.01%. Speed > 10 GB/s compression and > 13 GB/s decompress.
+* **Inverted Index ...do less, go fast!**
+ * Direct Access to compressed *frequency* and *position* data w/ zero decompression
+ * **Novel** **"Intersection w/ skip intervals"**, decompress the minimum necessary blocks (**~10-15%)!**.
+ * **Novel** Implicit skips with zero extra overhead
+ * **Novel** Efficient **Bidirectional** Inverted Index Architecture (forward/backwards traversal) incl. "integer compression".
+ * more than **2000! queries per second** on GOV2 dataset (25 millions documents) on a **SINGLE** core
+ * :sparkles: Revolutionary Parallel Query Processing on Multicores **> 7000!!! queries/sec** on a simple quad core PC.
+ **...forget** ~~Map Reduce, Hadoop, multi-node clusters,~~ ...
+
+### Integer Compression Benchmark:
+- :new: Download [IcApp](https://sites.google.com/site/powturbo/downloads) a new benchmark for TurboPFor
+ for testing allmost all integer and floating point file types.
+- Practical (No **PURE** cache) "integer compression" benchmark w/ **large** arrays.
+- CPU: Skylake i7-6700 3.4GHz gcc 7.2 **single** thread
+
+##### - Synthetic data:
+ - Generate and test (zipfian) skewed distribution (100.000.000 integers, Block size=128/256)
+ Note: Unlike general purpose compression, a small fixed size (ex. 128 integers) is in general used in "integer compression".
+ Large blocks involved, while processing queries (inverted index, search engines, databases, graphs, in memory computing,...) need to be entirely decoded.
+
+ ./icbench -a1.5 -m0 -M255 -n100M ZIPF
+
+|C Size|ratio%|Bits/Integer|C MB/s|D MB/s|Name|
+|--------:|-----:|--------:|----------:|----------:|--------------|
+|62,939,886| 15.7| 5.04|**1588**|**9400**|**TurboPFor256**|
+|63,392,759| 15.8| 5.07|1320|6432|**TurboPFor**|
+|63,392,801| 15.8| 5.07|1328|924|**TurboPForDA**|
+|65,060,504| 16.3| 5.20|60|2748|[FP_SIMDOptPFor](#FastPFor)|
+|65,359,916|16.3| 5.23| 32|2436|PC_OptPFD|
+|73,477,088|18.4| 5.88|408|2484|PC_Simple16|
+|73,481,096| 18.4| 5.88|624|8748|[FP_SimdFastPFor](#FastPFor) 64Ki *|
+|76,345,136| 19.1| 6.11|980|2612|**VSimple**|
+|91,947,533| 23.0| 7.36|284|11737|[QMX](#QMX) 64k *|
+|93,285,864| 23.3| 7.46|1568|10232|[FP_GroupSimple](#FastPFor) 64Ki *|
+|95,915,096|24.0| 7.67| 848|3832|Simple-8b|
+|99,910,930| 25.0| 7.99|**13976**|**11872**|**TurboPackV**|
+|99,910,930| 25.0| 7.99|9468|9404|**TurboPack**|
+|99,910,930| 25.0| 7.99|8420|8876|**TurboFor**|
+|100,332,929| 25.1| 8.03|**14320**|**12124**|**TurboPack256V**|
+|101,015,650| 25.3| 8.08|9520|9484|**TurboVByte**|
+|102,074,663| 25.5| 8.17|5712|7916|[MaskedVByte](#MaskedVByte)|
+|102,074,663| 25.5| 8.17|2260|4208|[PC_Vbyte](#PolyCom)|
+|102,083,036| 25.5| 8.17|5200|4268|[FP_VByte](#FastPFor)|
+|112,500,000| 28.1| 9.00|1528|**12140**|[VarintG8IU](#VarintG8IU)|
+|125,000,000| 31.2|10.00|4788|11288|[StreamVbyte](#StreamVByte)|
+|400,000,000| 100.00| 32.00| 8960|8948|Copy|
+| | | | N/A | N/A |EliasFano|
+
+(*) codecs inefficient for small block sizes are tested with 64Ki integers/block.
+
+- MB/s: 1.000.000 bytes/second. **1000 MB/s = 1 GB/s**
+- **#BOLD** = pareto frontier.
+- FP=FastPFor SC:simdcomp PC:Polycom
+- TurboPForDA,TurboForDA: Direct Access is normally used when accessing few individual values.
+- Eliasfano can be directly used only for increasing sequences
+------------------------------------------------------------------------
+##### - Data files:
+ - gov2.sorted from [DocId data set](#DocId) Block size=128/Delta coding
+
+ ./icbench -fS -r gov2.sorted
+
+
+
+|Size |Ratio %|Bits/Integer|C Time MB/s|D Time MB/s|Function |
+|-----------:|------:|-----:|-------:|-------:|---------------------|
+| 3,321,663,893| 13.9| 4.44|**1320**|**6088**|**TurboPFor**|
+| 3,339,730,557| 14.0| 4.47| 32| 2144|PC.OptPFD|
+| 3,350,717,959| 14.0| 4.48|**1536**|**7128**|**TurboPFor256**|
+| 3,501,671,314| 14.6| 4.68| 56| 2840|**VSimple**|
+| 3,768,146,467| 15.8| 5.04|**3228**| 3652|**EliasFanoV**|
+| 3,822,161,885| 16.0| 5.11| 572| 2444|PC_Simple16|
+| 4,521,326,518| 18.9| 6.05| 836| 3296|Simple-8b|
+| 4,649,671,427| 19.4| 6.22|3084|3848|**TurboVbyte**|
+| 4,955,740,045| 20.7| 6.63|**7064**|**10268**|**TurboPackV**|
+| 4,955,740,045| 20.7| 6.63|5724|8020|**TurboPack**|
+| 5,205,324,760|21.8| 6.96|6952|9488|SC_SIMDPack128|
+| 5,393,769,503| 22.5| 7.21|**9912**|**11588**|**TurboPackV256**|
+| 6,221,886,390| 26.0| 8.32|6668|6952|**TurboFor**|
+| 6,221,886,390| 26.0| 8.32|6644| 2260|**TurboForDA**|
+| 6,699,519,000| 28.0| 8.96| 1888| 1980|FP_Vbyte|
+| 6,700,989,563| 28.0| 8.96| 2740| 3384|MaskedVByte|
+| 7,622,896,878| 31.9|10.20| 836|4792|VarintG8IU|
+| 8,060,125,035| 33.7|11.50| 3536|8684|Streamvbyte|
+| 8,594,342,216| 35.9|11.50|5228|6376|libfor|
+|23,918,861,764|100.0|32.00|5824|5924|Copy|
+
+Block size: 64Ki = 256k bytes. Ki=1024 Integers
+
+|Size |Ratio %|Bits/Integer|C Time MB/s|D Time MB/s|Function |
+|----------:|-----:|----:|------:|------:|---------------------|
+| 3,164,940,562| 13.2|**4.23**|**1344**|**6004**|**TurboPFor 64Ki**|
+| 3,273,213,464| 13.7| 4.38|**1496**|**7008**|**TurboPFor256 64Ki**|
+| 3,965,982,954| 16.6| 5.30|**1520**| 2452|[lz4](#lz4)+DT 64Ki|
+| 4,234,154,427| 17.7| 5.66| 436| 5672|qmx 64Ki|
+| 6,074,995,117| 25.4| 8.13| 1976| 2916|[blosc_lz4](#blosc) 64Ki|
+| 8,773,150,644| 36.7|11.74| 2548|5204|blosc_lz 64Ki|
+
+"lz4+DT 64Ki" = Delta+Transpose from TurboPFor + lz4
+"blosc_lz4" internal lz4 compressor+vectorized shuffle
+
+##### - Time Series:
+- Test file [Timestamps: ts.txt(sorted)](https://github.com/zhenjl/encoding/tree/master/benchmark/data)
+
+ ./icapp -Ft ts.txt -I15 -J15
+
+|Function |C MB/s| size |ratio%| D MB/s|Text
+|----------------|-----:|--------:|------:|------:|--------------------|
+|bvzenc32 |**10632**|45,909|0.008|**12823**|ZigZag|
+|bvzzenc32 |**8914**|56,713|0.010|**13499**|ZigZag Delta of delta|
+|vsenc32 |**12294**|140,400| 0.024 |12877 |Variable Simple|
+|p4nzenc256v32 | 1932| 596,018| 0.10 |13326 |TurboPFor256 ZigZag|
+|p4ndenc256v32 | 1961| 596,018| 0.10 |13339 |TurboPFor256 Delta|
+|bitndpack256v32 |**12564**|909,189| 0.16 |13505 |TurboPackV256 Delta|
+|p4nzenc32 | 1810| 1,159,633| 0.20 | 8502 |TurboPFor ZigZag|
+|p4nzenc128v32 | 1795| 1,159,633| 0.20 |13338 |TurboPFor ZigZag|
+|bitnzpack256v32 | 9651| 1,254,757| 0.22 |**13503**|TurboPackV256 ZigZag|
+|bitnzpack128v32 |10155| 1,472,804| 0.26 |13380 |TurboPackV ZigZag|
+|vbddenc32 | 6198| 18,057,296| 3.13 |10982 |TurboVByte Delta of delta|
+|memcpy |13397|577,141,992|100.00||
+
+##### - Transpose/Shuffle (no compression)
+ ./icbench -eTRANSFORM ZIPF
+
+|Size |C Time MB/s|D Time MB/s|Function|
+|----------:|------:|------:|-----------------------------------|
+|100,000,000|**9400**|**9132**|**TPbyte 4** TurboPFor Byte Transpose/shuffle AVX2|
+|100,000,000|8784|8860|**TPbyte 4** TurboPFor Byte Transpose/shuffle SSE|
+|100,000,000|7688|7656|Blosc_Shuffle AVX2|
+|100,000,000|**5204**|**7460**|**TPnibble 4** TurboPFor Nibble Transpose/shuffle SSE|
+|100,000,000|6620|6284|Blosc shuffle SSE|
+|100,000,000|3156|3372|Bitshuffle AVX2|
+|100,000,000|2100|2176|Bitshuffle SSE|
+
+##### - (Lossy) Floating point compression:
+ ./icapp -Fd file " 64 bits floating point raw file
+ ./icapp -Ff file " 32 bits floating point raw file
+ ./icapp -Fcf file " text file with miltiple entries (ex. 8.657,56.8,4.5 ...)
+ ./icapp -Ftf file " text file (1 entry per line)
+ ./icapp -Ftf file -v5 " + display the first entries read
+ ./icapp -Ftf file.csv -K3 " but 3th column in a csv file (ex. number,Text,456.5 -> 456.5
+ ./icapp -Ftf file -g.001 " lossy compression with allowed error 0.001
+
+- see also [TurboTranspose](https://github.com/powturbo/TurboTranspose)
+
+##### - Compressed Inverted Index Intersections with GOV2
+ GOV2: 426GB, 25 Millions documents, average doc. size=18k.
+
+ + Aol query log: 18.000 queries
+ **~1300** queries per second (single core)
+ **~5000** queries per second (quad core)
+ Ratio = 14.37% Decoded/Total Integers.
+
+ + TREC Million Query Track (1MQT):
+ **~1100** queries per second (Single core)
+ **~4500** queries per second (Quad core CPU)
+ Ratio = 11.59% Decoded/Total Integers.
+
+- Benchmarking intersections (Single core, AOL query log)
+
+| max.docid/q|Time s| q/s | ms/q | % docid found|
+|-----------------:|---:|----:|-----:|-------:|
+|1.000|7.88|2283.1|0.438|81|
+|10.000|10.54|1708.5|0.585|84|
+| ALL |13.96|1289.0|0.776|100|
+q/s: queries/second, ms/q:milliseconds/query
+
+- Benchmarking Parallel Query Processing (Quad core, AOL query log)
+
+| max.docid/q|Time s| q/s | ms/q | % docids found|
+|-----------------:|----:|----:|-----:|-------:|
+|1.000|2.66|6772.6|0.148|81|
+|10.000|3.39|5307.5|0.188|84|
+|ALL|3.57|5036.5|0.199|100|
+
+###### Notes:
+- Search engines are spending 90% of the time in intersections when processing queries.
+- Most search engines are using pruning strategies, caching popular queries,... to reduce the time for intersections and query processing.
+- As indication, google is processing [40.000 Queries per seconds](http://www.internetlivestats.com/google-search-statistics/),
+using [900.000 multicore servers](https://www.cloudyn.com/blog/10-facts-didnt-know-server-farms/) for searching [8 billions web pages](http://searchenginewatch.com/sew/study/2063479/coincidentally-googles-index-size-jumps) (320 X size of GOV2).
+- Recent "integer compression" GOV2 experiments (best paper at ECIR 2014) [On Inverted Index Compression for Search Engine Efficiency](http://www.dcs.gla.ac.uk/~craigm/publications/catena14compression.pdf) using 8-core Xeon PC are reporting 1.2 seconds per query (for 1.000 Top-k docids).
+
+### Compile:
+ Download or clone TurboPFor
+ git clone git://github.com/powturbo/TurboPFor.git
+ cd TurboPFor
+
+ To benchmark external libraries:
+ git clone --recursive git://github.com/powturbo/TurboPFor.git
+ cd TurboPFor
+
+###### Linux, Windows (MingW), Clang,... (see also makefile)
+ make
+ or
+ make AVX2=1
+
+ Include external libs
+ make CODEC1=1 CODEC2=1
+
+ Disable SIMD
+ make NSIMD=1
+
+###### Windows visual c++
+ nmake /f makefile.vs
+
+### Testing:
+##### - Synthetic data (use ZIPF parameter):
+ + benchmark groups of "integer compression" functions
+
+ ./icbench -eBENCH -a1.2 -m0 -M255 -n100M ZIPF
+ ./icbench -eBITPACK/VBYTE -a1.2 -m0 -M255 -n100M ZIPF
+
+ >*Type "icbench -l1" for a list*
+
+ >*-zipfian distribution alpha = 1.2 (Ex. -a1.0=uniform -a1.5=skewed distribution)
+ -number of integers = 100.000.000
+ -integer range from 0 to 255
*
+
+ + Unsorted lists: individual function test (ex. Copy TurboPack TurboPFor)
+
+ ./icbench -a1.5 -m0 -M255 -ecopy/turbopack/turbopfor/turbopack256v ZIPF
+
+ + Unsorted lists: Zigzag encoding w/ option **-fz** or FOR encoding
+
+ ./icbench -fz -eturbovbyte/turbopfor/turbopackv ZIPF
+ ./icbench -eturboforv ZIPF
+
+ + Sorted lists: differential coding w/ option **-fs** (increasing) or **-fS** (strictly increasing)
+
+ ./icbench -fs -eturbopack/turbopfor/turbopfor256v ZIPF
+
+ + Generate interactive "file.html" plot for browsing
+
+ ./icbench -p2 -S2 -Q3 file.tbb
+
+ + Unit test: test function from bit size 0 to 32
+
+ ./icbench -m0 -M32 -eturbpfor -fu
+ ./icbench -m0 -M8 -eturbopack -fs -n1M
+
+##### - Data files:
+ - Raw 32 bits binary data file [Test data](https://github.com/ot/partitioned_elias_fano/tree/master/test/test_data)
+
+ ./icbench file
+ ./icapp file
+ ./icapp -Fs file "16 bits raw binary file
+ ./icapp -Fu file "32 bits raw binary file
+ ./icapp -Fl file "64 bits raw binary file
+ ./icapp -Ff file "32 bits raw floating point binary file
+ ./icapp -Fd file "64 bits raw floating point binary file
+
+ - Text file: 1 entry per line. [Test data: ts.txt(sorted) and lat.txt(unsorted)](https://github.com/zhenjl/encoding/tree/master/benchmark/data))
+
+ ./icbench -eBENCH -fts ts.txt
+ ./icbench -eBENCH -ft lat.txt
+
+ ./icapp -Fts data.txt "text file, one 16 bits integer per line
+ ./icapp -Ftu ts.txt "text file, one 32 bits integer per line
+ ./icapp -Ftl ts.txt "text file, one 64 bits integer per line
+ ./icapp -Ftf file "text file, one 32 bits floating point (ex. 8.32456) per line
+ ./icapp -Ftd file "text file, one 64 bits floating point (ex. 8.324567789) per line
+ ./icapp -Ftd file -v5 "like prev., display the first 100 values read
+ ./icapp -Ftd file -v5 -g.00001 "like prev., error bound lossy floating point compression
+ ./icapp -Ftt file "text file, timestamp in seconds iso-8601 -> 32 bits integer (ex. 2018-03-12T04:31:06)
+ ./icapp -FtT file "text file, timestamp in milliseconds iso-8601 -> 64 bits integer (ex. 2018-03-12T04:31:06.345)
+ ./icapp -Ftl -D2 -H file "skip 1th line, convert numbers with 2 decimal digits to 64 bits integers (ex. 456.23 -> 45623)
+ ./icapp -Ftl -D2 -H -K3 file.csv "like prev., use the 3th number in the line (ex. label=3245, text=99 usage=456.23 -> 456.23 )
+ ./icapp -Ftl -D2 -H -K3 -k| file.csv "like prev., use '|' as separator
+
+ - Text file: multiple numbers separated by non-digits (0..9,-,.) characters (ex. 134534,-45678,98788,4345, )
+
+ ./icapp -Fc data.txt "text file, 32 bits integers (ex. 56789,3245,23,678 )
+ ./icapp -Fcd data.txt "text file, 64 bits floting-point numbers (ex. 34.7689,5.20,45.789 )
+
+ - Multiblocks of 32 bits binary file. (Example gov2 from [DocId data set](#DocId))
+ Block format: [n1: #of Ids][Id1] [Id2]...[IdN] [n2: #of Ids][Id1][Id2]...[IdN]...
+
+ ./icbench -fS -r gov2.sorted
+
+
+##### - Intersections:
+ 1 - Download Gov2 (or ClueWeb09) + query files (Ex. "1mq.txt") from [DocId data set](#DocId)
+ 8GB RAM required (16GB recommended for benchmarking "clueweb09" files).
+
+ 2 - Create index file
+
+
+ ./idxcr gov2.sorted .
+
+
+ >*create inverted index file "gov2.sorted.i" in the current directory*
+
+ 3 - Test intersections
+
+
+ ./idxqry gov2.sorted.i 1mq.txt
+
+
+ >*run queries in file "1mq.txt" over the index of gov2 file*
+
+##### - Parallel Query Processing:
+ 1 - Create partitions
+
+
+ ./idxseg gov2.sorted . -26m -s8
+
+
+ >*create 8 (CPU hardware threads) partitions for a total of ~26 millions document ids*
+
+ 2 - Create index file for each partition
+
+
+ ./idxcr gov2.sorted.s*
+
+
+ >*create inverted index file for all partitions "gov2.sorted.s00 - gov2.sorted.s07" in the current directory*
+
+ 3 - Intersections:
+
+ delete "idxqry.o" file and then type "make para" to compile "idxqry" w. multithreading
+
+
+ ./idxqry gov2.sorted.s*.i 1mq.txt
+
+ >*run queries in file "1mq.txt" over the index of all gov2 partitions "gov2.sorted.s00.i - gov2.sorted.s07.i".*
+
+### Function usage:
+See benchmark "icbench" program for "integer compression" usage examples.
+In general encoding/decoding functions are of the form:
+
+ >**char *endptr = encode( unsigned *in, unsigned n, char *out, [unsigned start], [int b])**
+ endptr : set by encode to the next character in "out" after the encoded buffer
+ in : input integer array
+ n : number of elements
+ out : pointer to output buffer
+ b : number of bits. Only for bit packing functions
+ start : previous value. Only for integrated delta encoding functions
+
+
+ >**char *endptr = decode( char *in, unsigned n, unsigned *out, [unsigned start], [int b])**
+ endptr : set by decode to the next character in "in" after the decoded buffer
+ in : pointer to input buffer
+ n : number of elements
+ out : output integer array
+ b : number of bits. Only for bit unpacking functions
+ start : previous value. Only for integrated delta decoding functions
+
+ **Simple high level functions:**
+ >**size_t compressed_size = encode( unsigned *in, size_t n, char *out)**
+ compressed_size : number of bytes written into compressed output buffer out
+
+ >**size_t compressed_size = decode( char *in, size_t n, unsigned *out)**
+ compressed_size : number of bytes read from compressed input buffer in
+
+### Function syntax:
+ - {vb | p4 | bit | vs}[n][d | d1 | f | fm | z ]{enc/dec | pack/unpack}[| 128V | 256V][8 | 16 | 32 | 64]:
+ vb: variable byte
+ p4: turbopfor
+ vs: variable simple
+ bit: bit packing
+ n : high level array functions for large arrays.
+
+ '' : encoding for unsorted integer lists
+ 'd' : delta encoding for increasing integer lists (sorted w/ duplicate)
+ 'd1': delta encoding for strictly increasing integer lists (sorted unique)
+ 'f' : FOR encoding for sorted integer lists
+ 'z' : ZigZag encoding for unsorted integer lists
+
+ 'enc' or 'pack' : encode or bitpack
+ 'dec' or 'unpack': decode or bitunpack
+ 'NN' : integer size (8/16/32/64)
+
+header files to use with documentation:
+
+| c/c++ header file|Integer Compression functions| examples |
+|------------|-----------------------------|-----------------|
+|vint.h|variable byte| vbenc32/vbdec32 vbdenc32/vbddec32 vbzenc32/vbzdec32 |
+|vsimple.h|variable simple| vsenc64/vsdec64 |
+|vp4.h|TurboPFor| p4enc32/p4dec32 p4denc32/p4ddec32 p4zenc32/p4zdec32 |
+|bitpack.h|Bit Packing, For, +Direct Access| bitpack256v32/bitunpack256v32 bitforenc64/bitfordec64|
+|eliasfano.h|Elias Fano| efanoenc256v32/efanoc256v32 |
+
+Note: Some low level functions (like p4enc32) are limited to 128/256 (SSE/AVX2) integers per call.
+
+### Environment:
+###### OS/Compiler (64 bits):
+- Linux: GNU GCC (>=4.6)
+- clang (>=3.2)
+- Windows: MinGW-w64 (no parallel query processing demo app)
+- Visual c++ (VS2008-VS2017)
+- Linux aarch64 for 64 bits ARM CPU : gcc
+
+###### Multithreading:
+- All TurboPFor integer compression functions are thread safe
+
+### References:
+
+* **Benchmark references:**
+ * [FastPFor](https://github.com/lemire/FastPFor) + [Simdcomp](https://github.com/lemire/simdcomp): SIMDPack FPF, Vbyte FPF, VarintG8IU, StreamVbyte, GroupSimple
+ * [Optimized Pfor-delta compression code](http://jinruhe.com): OptPFD/OptP4, Simple16 (limited to 28 bits integers)
+ * [MaskedVByte](http://maskedvbyte.org/). See also: [Vectorized VByte Decoding](http://engineering.indeed.com/blog/2015/03/vectorized-vbyte-decoding-high-performance-vector-instructions/)
+ * [Streamvbyte](https://github.com/lemire/streamvbyte).
+ * [Index Compression Using 64-Bit Words](http://people.eng.unimelb.edu.au/ammoffat/abstracts/am10spe.html): Simple-8b (speed optimized version tested)
+ * [libfor](https://github.com/cruppstahl/for)
+ * [Compression, SIMD, and Postings Lists](http://www.cs.otago.ac.nz/homepages/andrew/papers/) QMX integer compression from the "simple family"
+ * [lz4](https://github.com/Cyan4973/lz4). included w. block size 64K as indication. Tested after preprocessing w. delta+transpose
+ * [blosc](https://github.com/Blosc/c-blosc). blosc is like transpose/shuffle+lz77. Tested blosc+lz4 and blosclz incl. vectorizeed shuffle.
+ * [Document identifier data set](http://lemire.me/data/integercompression2014.html)
+
+* **Integer compression publications:**
+ * :green_book:[In Vacuo and In Situ Evaluation of SIMD Codecs (TurboPackV,TurboPFor/QMX)](http://dl.acm.org/citation.cfm?id=3015023) + [paper](http://www.cs.otago.ac.nz/homepages/andrew/papers/)
+ * :green_book:[SIMD Compression and the Intersection of Sorted Integers](http://arxiv.org/abs/1401.6399)
+ * :green_book:[Partitioned Elias-Fano Indexes](http://www.di.unipi.it/~ottavian/files/elias_fano_sigir14.pdf)
+ * :green_book:[On Inverted Index Compression for Search Engine Efficiency](http://www.dcs.gla.ac.uk/~craigm/publications/catena14compression.pdf)
+ * :green_book:[Google's Group Varint Encoding](http://static.googleusercontent.com/media/research.google.com/de//people/jeff/WSDM09-keynote.pdf)
+ * :green_book:[Integer Compression tweets](https://twitter.com/search?q=%23integercompression&src=typd)
+ * :green_book:[Efficient Compression of Scientific Floating-Point Data and An Application in Structural Analysis](https://www.jstage.jst.go.jp/article/jsces/2017/0/2017_20170002/_article)
+ * :green_book:[SPDP is a compression/decompression algorithm for binary IEEE 754 32/64 bits floating-point data](http://cs.txstate.edu/~burtscher/research/SPDPcompressor/)
+ :green_book:[ SPDP - An Automatically Synthesized Lossless Compression Algorithm for Floating-Point Data](http://cs.txstate.edu/~mb92/papers/dcc18.pdf) + [DCC 2018](http://www.cs.brandeis.edu//~dcc/Programs/Program2018.pdf)
+
+* **Applications:**
+ * [Graph500](https://github.com/julianromera/graph500)
+ * [Small Polygon Compression](https://arxiv.org/abs/1509.05505) + [Poster](http://abhinavjauhri.me/publications/dcc_poster_2016.pdf) + [code](https://github.com/ajauhri/bignum_compression)
+ * [Parallel Graph Analysis (Lecture 18)](http://www.cs.rpi.edu/~slotag/classes/FA16/) + [code](http://www.cs.rpi.edu/~slotag/classes/FA16/handson/lec18-comp2.cpp)
+
+Last update: 15 Jul 2019
+
+## APPENDIX: icbench Integer Compression Benchmark
+
+##### TurboPFor + external libraries
+
+TurboPFor https://github.com/powturbo/TurboPFor +FastPFor (FP) https://github.com/lemire/FastPFor +lz4 https://github.com/Cyan4973/lz4 +LittleIntPacker (LI) https://github.com/lemire/LittleIntPacker +MaskedVbyte http://maskedvbyte.org +Polycom (PC) https://github.com/encode84/bcm +simdcomp (SC) https://github.com/lemire/simdcomp +Simple-8b optimized https://github.com/powturbo/TurboPFor +Streamvbyte https://github.com/lemire/streamvbyte +VarintG8IU https://github.com/lemire/FastPFor ++ +##### Functions integrated into 'icbench' for benchmarking +
+Codec group: +TURBOPFOR TurboPFor library TurboPFor256V/TurboPack256V/TurboPFor256N/TurboPFor/TurboPackV/TurboVByte/TurboPack/TurboForDA/EliasFano/VSimple/TurboPForN/TurboPackN/TurboPForDI +DEFAULT Default TurboPFor/TurboPackV/TurboVByte/TurboPack/TurboFor/TurboPForN/TurboPackN/TurboPForDI/TurboPFor256V/TurboPack256V/TurboPFor256N +BENCH Benchmark TurboPFor/TurboPackV/TurboVByte/TurboPack/QMX/FP.SimdFastPfor/FP.SimdOptPFor/MaskedVbyte/StreamVbyte +EFFICIENT Efficient TurboPFor/vsimple/turbovbyte +TRANSFORM transpose/shufle,delta,zigzag tpbyte4s/tpbyte,4/tpnibble,4/ZigZag_32/Delta_32/BitShuffle,4 +BITPACK Bit Packing TurboPack256V/TurboPackV/TurboPackH/TurboPack/SC.SimdPack128/SC.SimdPack256 +VBYTE Variable byte TurboVByte/FP.VByte/PC.Vbyte/VarintG8IU/MaskedVbyte/StreamVbyte +SIMPLE Simple Family simple8b/simple16/vsimple/qmx +LZ4 lz4+bitshufle/transpose 4,8 lz4_bitshufle/lz4_tp4/lz4_tp8 +LI Little Integer LI_Pack/LI_TurboPack/LI_SuperPack/LI_HorPack + + +Function Description level + +-------- ----------- ----- +TurboPFor PFor (SSE2) +TurboPForN PFor (SSE2) large blocks +TurboPFor256 PFor (AVX2) +TurboPFor256N PFor (AVX2) large blocks +TurboPForDA PFor direct access +TurboPForDI PFord min +TurboPForZZ PFor zigzag of delta +TurboFor FOR +TurboForV FOR (SIMD) +TurboFor256V FOR (AVX2) +TurboForDA FOR direct access +TurboPackDA Bit packing direct access +TurboPack Bit packing (scalar) +TurboPackN Bit packing (scalar) large blocks +TurboPackV Bit packing (SSE2 Vertical) +TurboPackH Bit packing (SSE2 Horizontal) +TurboPackVN Bit packing (SSE2 large block) +TurboPack256V Bit packing (AVX2 Vertical) +TurboPack256N Bit packing (AVX2 large block) +TurboVByte Variable byte (scalar) +VSimple Variable simple (scalar) +EliasFano Elias fano (scalar) +EliasFanoV Eliasfano (SSE2) +EliasFano256V Elias fano (AVX2 +memcpy memcpy +copy Integer copy +tpbyte4s Byte Transpose (scalar) +tpbyte Byte transpose (simd) 2,4,8 +tpnibble Nibble transpose (simd) 2,4,8 +ZigZag32 ZigZag encoding (sse2) +Delta32 Delta encoding (sse2) +DDelta32 Delta of delta encoding (sse2) +Xor32 Xor encoding (sse2) +FP_PREV64 Floating point PFOR +FP_FCM64 Floating point PFOR (FCM) +FP_DFCM64 Floating point PFOR (DFCM) +TurboPFor64 PFOR 64 +TurboPFor64V PFOR 64 +Simple8b 64 bits Simple family (instable) +PC_Simple16 Simple 16. limited to 28 bits +PC_OptPFD OptPFD. limited to 28 bits +PC_Vbyte Variable byte +PC_Rice Rice coding (instable) +VarintG8IU Variable byte SIMD +MaskedVbyte Variable byte SIMD +StreamVbyte Variable byte SIMD +FP_FastPFor PFor scalar (inefficient for small blocks) +FP_SimdFastPFor PFor SIMD (inefficient for small blocks) +FP_OptPFor OptPFor scalar +FP_SIMDOptPFor OptPFor SIMD +FP_VByte Variable byte +FP_Simple8bRLE Simple-8b + rle +FP_GROUPSIMPLE Group Simple +SC_SIMDPack128 Bit packing (SSE4.1) +SC_SIMDPack256 Bit packing (SSE4.1) +SC_For For (SSE4.1) +SC_ForDA For direct access (SSE4.1) +LibFor_For For +LibFor_ForDA For direct access +LI_Pack Bit packing (scalar) +LI_TurboPack Bit packing (scalar) +LI_SuperPack Bit packing (scalar) +LI_HorPack Bit packing (sse4.1 horizontal) +LI_BMIPack256 Bit packing (avx2) +lz4 lz4 +lz4_bit Bitshuffle + [delta]+lz4 2,4,8 +lz4_nibble TurboPFor's [delta]+nibble transpose + lz4 2,4,8 +lz4_bitxor Bitshuffle + [xor]+lz4 2,4,8 +lz4_nibblexor TurboPFor's [xor]+nibble transpose + lz4 2,4,8 +lz4_byte TurboPFor's [delta]+byte transpose + lz4 2,4,8 +BitShuffle Bit shuffle (simd) 2,4,8 ++