{"id":11,"date":"2013-08-15T06:54:41","date_gmt":"2013-08-15T06:54:41","guid":{"rendered":"http:\/\/fluentreports.com\/blog\/?p=11"},"modified":"2013-08-15T13:04:21","modified_gmt":"2013-08-15T13:04:21","slug":"transforming-javascript-json","status":"publish","type":"post","link":"https:\/\/fluentreports.com\/blog\/?p=11","title":{"rendered":"Transforming JavaScript JSON"},"content":{"rendered":"

Colt McAnlis posted a very interesting blog post (http:\/\/mainroach.blogspot.com\/2013\/08\/json-compression-transpose-binary.html)<\/a> this evening on using Transposing to reduce the JSON data size; his post was right on the money.<\/p>\n

We have been using a similar technique for a couple years now.\u00a0 (Although, we use a different compression method over websocket as gzip is too expensive in pure JavaScript).<\/p>\n

However, one thing that I commented on is that he went to step one, and step two gives him better results -- it actually improves the compression.<\/p>\n

I created my own \"original dataset\" to show this example.\u00a0\u00a0 The Dataset has Spaces here in the blog and show it for formatting purposes to make it easier to read; but all my numbers are excluding spaces and returns as a raw json wouldn't have those in it.<\/p>\n

The original Data (265 Characters):
\n

[{Id: 1, Name: 'Nathan', Address: 'Somewhere', Country: 'USA', City:'Here', State:'OK',Zip:'55555'},\n {Id: 2, Name: 'Colt', Address: 'Elsewhere', Country: 'USA', City: 'There', State: 'CA',Zip:'44444'}\n {Id: 3, Name: 'You', Address: 'Not Sure', Country: 'USA', City: 'Where', State: 'AZ', Zip:'33333'}]<\/pre>
\nColt's Transposing (211 Characters):
\n
{'id':[1,2,3],\n'Name':['Nathan','Colt','You'],\n'Address':['Somewhere','Elsewhere','Not Sure'],\n'Country':['USA','USA','USA'],\n'City':['Here','There','Where'],\n'State':['OK','CA','AZ'],\n'Zip': ['55555','44444','33333']}<\/pre>
\nWe transpose it into basically a JSON CSV (206 Characters):
\n
[['Id','Name','Address','Country','City','State','Zip'],\n [1,'Nathan','Somewhere','USA','Here','OK','55555'],\n [2,'Colt','Elsewhere','USA','There','CA','44444'],\n [3,'You','Note Sure','USA','Where','AZ','33333']]<\/pre>
\nNow for every additional row of data we add with this dataset you add:<\/p>\n
Original: 48 Characters of Static unchanging field definitions. (Ouch!)
\nColt's: 7 Characters
\nOurs: 9 Characters<\/p>\n
So how do we end up with better compression when after a dozen or so records our raw size is actually larger than Colt's?\u00a0\u00a0\u00a0 Well; we only use [] and comma's.\u00a0\u00a0 He has added additional data to his data stream in addition to the [] and commas, he has\u00a0 {}, and the colons.\u00a0\u00a0\u00a0 By having more redundancy in our stream we compress better.<\/p>\n
Wait; there is another easy savings if you think about the data... \u00a0\u00a0 Why send the header row?\u00a0 If you already know the layout of what you are requesting; you can entirely eliminate the header row; which would then shrink your \"raw\" data down another 55 characters.\u00a0 Meaning we start out at a small 151 characters.<\/p>\n
So if you are dealing with straight raw characters; Colt's method actually is smaller (after about 30 rows) .\u00a0 However, If you are going to compress the stream; the additional redundancy in our transformation appears to be better suited to make smaller compressed files.<\/p>\n
Measure everything and think about how you actually use your data might be the difference in how you send your data making all the difference in how fast your app actually responds to requests because Performance Matters.<\/p>\n","protected":false},"excerpt":{"rendered":"
Colt McAnlis posted a very interesting blog post (http:\/\/mainroach.blogspot.com\/2013\/08\/json-compression-transpose-binary.html) this evening on using Transposing to reduce the JSON data size; his post was right on the money. We have been using a similar technique for a couple years now.\u00a0 (Although, we use a different compression method over websocket as gzip is too expensive in pure… Continue reading Transforming JavaScript JSON<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","footnotes":""},"categories":[3,4,7,1],"tags":[],"class_list":["post-11","post","type-post","status-publish","format-standard","hentry","category-javascript","category-node-js","category-performance","category-uncategorized","entry"],"_links":{"self":[{"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/11","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=11"}],"version-history":[{"count":5,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/11\/revisions"}],"predecessor-version":[{"id":16,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/11\/revisions\/16"}],"wp:attachment":[{"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=11"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=11"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fluentreports.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=11"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}