Monthly Archives: May 2014

Node & Browser Javascript Compression Update

I wrote a post on Data Compression back in October,, discussing how I sped up a Data Compression Library that we have been using internally for all web socket traffic and how by combing techniques from different comparable libraries LZJBn.js was born.

Well fast forward several months --- I ran across another library that well professional curiosity compels me to to bench mark to see how well my cool LZJBn will trounce it.

Using 526 different sized files from real packets that we send:

Compression Decompression Compressed Size Original Size
LZJBn.js 0.503752017 0.1777535 15,890,401 37,345,189
node-lz4* 0.363441773 0.1109069 11,364,620 37,345,189

* - Node-lz4 does not compress files under a really small size; so there was 8 files comprising of a total of 182 bytes of data that was not compressed in this test on the lz4 side.  So because of this; when sending any data packets you will have to tag your packets as compressed or uncompressed.

And on even larger size using the ENWIKI file I used in the prior blog post:

Compression Decompression Compressed Size Original Size
LZJBn.js  1.87325003 0.279163174 34,332,875 50,000,896
node-lz4 0.713367233 0.207553456 27,591,715 50,000,896

Now if you look at the numbers it not only compressed and decompressed faster; but it also had a even better compression ratio.

After many tests and a very timely bug fix from the author of node-lz4 on a bug I reported; I have to shamefully say node-lz4 totally skunks my LZJBn.js module in the tests.   In addition Node-lz4 also has a native module for the node side, but those numbers aren't relevant to this test as this was purely testing the JavaScript library speeds.

So those who are wanting to implement as close to real time compression as possible using JavaScript; there is a now a new King of the Hill and sadly (for me) it is node-lz4.

Congrats Pierre; for a Job well done porting LZ4 to JavaScript -- and I know which library I will be using in the future!

For those who are interested the primary LZ4 site is here and the original author (Yann Collet) who created the lz4 compression format has a blog here:


FCC and the new Net Neutrality Rules in English

Internet without Neutrality

Internet without Neutrality

By this time you might have seen a story or two about the FCC allowing your internet provider to charge companies like Amazon or Netflix to use faster connections to your house; or you might have read a story or two about how Netflix has actually inked a deal with Comcast for faster connections to all the Comcast customers.   And so here you are; wondering what is the big deal?  Why is there any fuss at all?   Taken at face value; that seems reasonable, and that capitalism is working.

The reason this is a big deal, in a nutshell, is it is going to raise your rates for something you are already paying for.   And unless the FCC retracts these new rules; and actually brings back network neutrality you will end up paying considerably more for any and all the internet services you want in the future.   And finally, it will stifle innovation on the web.

Now, let me explain in details why this is bad for the internet.   The issue is your phone company and cable companies controls what we in the computer industry call the “last mile”.  The “last mile” is the actual physical wire connection from your house to their facility which then connects you to the internet.   In most cities your cable and phone company are a Government mandated monopoly.    As such; you do not have much of a choice for where to get your internet beyond these few places.     So now for you to use any of the sites on the internet; you are stuck having to go through that “last mile” connection.      Now because of this “last mile” and the mandated monopolies; they have no reason to compete for your business since you are effectively stuck with so few providers.

Now who pays for the use of that “last mile” connection to communicate on the internet?   (You do.)      And who pays for your monthly Netflix account?   (You do).       So now, Comcast can also charge Netflix for the same use of the “last mile” that they continue to charge you for.       What do you expect Netflix is going to have to do with those new costs they are having to pay to deliver the content you already pay for to your computer over that “last mile”?   (Raise your monthly Rates).      And who will be paying the new rates?  (You.)     Does this sound at all reasonable now?     So the end result is you will now end up paying Comcast more for the exact same connection.    So who won in the deal?   (Comcast)    And who Lost? (You and Netflix).    And since this is a Government sponsored monopoly the cable and phone company have no reason to compete on price and so instead of rates going down, Comcast has now figured out another way to get more money out of your pocket and there is nothing you can do about it unless we can convince the FCC to revert to real network neutrality.

Now in general; how is this bad for the overall internet?    The next “big” thing; Comcast (& others) can now actually price out of the market (especially if they don’t like them) causing sites like Google, Facebook, Youtube, Netflix, Amazon and others to be cost prohibitive to even get started because they have to be able to get the data down your “last mile” connection to you.   And if those existing and/or new sites don't pay the "extortion fees"  they will be REALLY slow for you to use.

At this point probably the best thing that could happen is if the “last mile” internet connection was classified as a “Common Carrier” so that other FCC rules would be in place to keep any of the Internet service providers from being able to charge internet companies for access to you.

What can you do about it:

  1. Fill in a FCC comment on  the “proceeding number”: 14-28 at
    1. You will need to put in the number “14-28” in the “Proceeding number”
    2. Fill out your information
    3. Write you own personalized note why you think this is bad and that you want ISP (Internet Service Providers) classified as a “Common Carrier” in some text editor like WordPad, Notepad, and then attach your note with the Documents -> Browse Button.


  2.  Fill out the “We The People” petition at