I wrote a post on Data Compression back in October, http://fluentreports.com/blog/?p=18, discussing how I sped up a Data Compression Library that we have been using internally for all web socket traffic and how by combing techniques from different comparable libraries LZJBn.js was born.
Well fast forward several months --- I ran across another library that well professional curiosity compels me to to bench mark to see how well my cool LZJBn will trounce it.
Using 526 different sized files from real packets that we send:
| Compression | Decompression | Compressed Size | Original Size | ||
| LZJBn.js | 0.503752017 | 0.1777535 | 15,890,401 | 37,345,189 | |
| node-lz4* | 0.363441773 | 0.1109069 | 11,364,620 | 37,345,189 | |
* - Node-lz4 does not compress files under a really small size; so there was 8 files comprising of a total of 182 bytes of data that was not compressed in this test on the lz4 side. So because of this; when sending any data packets you will have to tag your packets as compressed or uncompressed.
And on even larger size using the ENWIKI file I used in the prior blog post:
| Compression | Decompression | Compressed Size | Original Size | ||
| LZJBn.js | 1.87325003 | 0.279163174 | 34,332,875 | 50,000,896 | |
| node-lz4 | 0.713367233 | 0.207553456 | 27,591,715 | 50,000,896 | |
Now if you look at the numbers it not only compressed and decompressed faster; but it also had a even better compression ratio.
After many tests and a very timely bug fix from the author of node-lz4 on a bug I reported; I have to shamefully say node-lz4 totally skunks my LZJBn.js module in the tests. In addition Node-lz4 also has a native module for the node side, but those numbers aren't relevant to this test as this was purely testing the JavaScript library speeds.
So those who are wanting to implement as close to real time compression as possible using JavaScript; there is a now a new King of the Hill and sadly (for me) it is node-lz4.
Congrats Pierre; for a Job well done porting LZ4 to JavaScript -- and I know which library I will be using in the future!
For those who are interested the primary LZ4 site is here and the original author (Yann Collet) who created the lz4 compression format has a blog here: http://fastcompression.blogspot.fr/