Category Archives: Performance

NativeScript 4.1.0 Released

I took a bit of a break from reporting on NativeScript things; but guess what, I'm BACK!!!

Well, looks like I was on the ball again this time.   😉    4.1.0 has been released to NPM, and it is HOT!

Version 4.0 was a great release and all; had a lot of cool changes.   But 4.1 is an awesome release building upon that great release and if you haven't upgrade to the NativeScript 4 yet; this is what you have been waiting for!   Thank you Progress.

First of all the item that we have been all waiting for if you do any Android development; they have updated the Android V8 engine to pretty much the latest stable released version.  So we jumped from v8 v5.5 to v6.6.   Now that might not seem like a large change but the number of changes in the v8 engine from v5 to v6 are incredibly large.   We now have the brand new JavaScript optimizing engine (Turbo Fan); the new optimized Garbage Collection engine, pretty much full ES6 support ; and so many other countless optimized features -- I could dedicate a post to all the cool things we now have access to in Android.   In a nutshell this v8 is now considerably faster.  In fact one of the tweets you might have seen is how much faster this version even starts the application.     Always awesome for our customers to have the app startup now almost twice as fast.   For all you iOS fans; v4.0.0 of NativeScript upgraded the JavaScript core to pretty close to the latest, so now both sides are running the state of the art JavaScript engines..

In addition to that awesome change that is totally worth the upgrade by itself - we can now use Angular 6 and Webpack 4!   I'm sure many of you will be very happy to see these.

Now that I'm done drooling over all this new found speed and these two other awesome feature updates; lets run down some of the other new features and fixes that you get for free when you upgrade:

  • Android: "in" names space is now fixed.  Prefix it with a $ so that it is $in.rest.of.namespace.
  • Android: Java ByteArray to ArrayBuffer support was improved to support more types of ByteArray's.
  • Android: Latest Gradle now used (and a bug fixed against it).
  • Android: Several Console fixes; including release mode no longer logs.
  • Android: Fixed JNI crash
  • Android: Debug tools are and LiveSync fixes.
  • Android: Android P fixes!
  • Android: ReturnKey no longer fires twice
  • Android: Transition no longer crashes
  • iOS: Console fixes
  • iOS: Inspector port no longer times out quickly
  • iOS: Flexbox fixes
  • iOS: getImageAsync() added
  • iOS: searchbar and listview sizing fixes
  • All: Modal dialog fixes
  • All: Navigation fix
  • All: LayoutChanged event added
  • All: css; linear-gradient support
  • All: File System; get file size
  • All: TabView font size
  • All: Read XML from Bundles
  • Angular: Angular 6 Support!
  • Angular: router state should no longer crash the app in an invalid state
  • Webpack: Webpack 4 Support!
  • Webpack: XML Loading support
  • Webpack: Android Compression
  • CLI: Supports driving more than one iOS simulator
  • CLI: Support Java 10
  • CLI: Full application templates (Beta)
  • CLI: Allow native Objective C source as part of plugins.
  • CLI: iOS wifi driving should work again.

What do you think; I think this is a Rock'n awesome release, thank you to the different development teams (and community members) who contributed to this release!

First of all to upgrade is done is a couple steps:
> npm install -g nativescript@latest
> npm install tns-core-modules@latest --save

Next try the new update command or you can manually run the commands below
> tns update

For Android:
> tns platform remove android
> tns platform add android@latest

For iOS
> tns platform remove ios
> tns platform add ios@latest

Then you can type tns info and verify that everything says 4.1.x

Common Issues:

  1. None so far.  🙂

NativeScript: Performance from the Trenches

Well my video for Performance from the Trenches, was released today!   This is the talk I gave at the NativeScript Developer Days 2017.

 

There is a lot of good useful information in this Talk, so enjoy.

Slide Deck: Performance

NativeScript App: https://github.com/NathanaelA/PerformanceFromTheTrenches

V8-Natives Repo: https://github.com/NathanaelA/v8-natives

 

 

 

MySQL SSL required connection Ubuntu solutions

I went to implement MySQL replication for a client this evening and ran into some interesting issues that I haven't ran into before. Guess it has been a while since I had to set it up from a client.   So this post is for notes for me or someone else who might need to do this in the future. The normal installation replication installation works great but if you are going to enable ssl connections this is where the things can get a bit more complex.

The first thing to find out is if you have your SSL setup correct, try doing:
And verify the SSL is enabled and build in, in my case everything looked good:

mysql> SHOW VARIABLES LIKE '%ssl%';

have_openssl  = YES
have_ssl      = YES
ssl_ca        = /etc/mysql/certs/ca-cert.pem
ssl_capath    =
ssl_cert      = /etc/mysql/certs/server-cert.pem
ssl_cipher    =
ssl_crl       =
ssl_crlpath   =
ssl_key       = /etc/mysql/certs/server-key.pem\";

This looks correct, so the next thing to figure out is where the error log file is located;
mysql> SHOW VARIABLES LIKE '%error_log%';

log_error     =  ./mysql-bin.err
or something like
log_error     = /var/log/mysql/error.log

Now that you know where the error log is at you can see why it is failing.

In my case the error was this:

2016-12-06 23:21:33 32695 [Warning] Failed to setup SSL
2016-12-06 23:21:33 32695 [Warning] SSL error: SSL_CTX_set_default_verify_paths failed

I love that it is a "Warning".   It is totally broken, but we will list it as a Warning...

Well, this can be caused by several things:

  1. No permissions to the files in the folder, use chmod/chown to give perms.
  2. SELinux blocking it, disable selinux or grant permissions via SELinux
  3. AppArmor blocking it.  (this was my case)

Edit the /etc/apparmor.d/usr.sbin.mysqld file.

You'll see something like this in the file:

/etc/mysql/*.pem r,
/etc/mysql/conf.d/ r,
/etc/mysql/conf.d/* r,
/etc/mysql/*.cnf r,
---> /etc/mysql/certs/*.pem r,  <---
/usr/lib/mysql/plugin/ r,

Add the ---> line <---, make sure it matches your path to where you are storing the certs.  Then restart mysql. After restarting the server, I then got this error: SSL error: Unable to get private key from '/etc/mysql/certs/server-key.pem' 2016-12-06 23:53:32 21728 [Warning] Failed to setup SSL 2016-12-06 23:53:32 21728 [Warning] SSL error: Unable to get private key Ok, this one threw me for a while.  The files are fully readable by MySQL.  The issue ends up being incompatibilities between SSL libraries in use.  OpenSSL 1.0x vs yaSSL The key file will start like this:
-----BEGIN PRIVATE KEY-----
If you used OpenSSL to generate the keys;  OpenSSL creates keys in PKCS#8 with a SHA256 digest.  Of course yaSSL which is (normally) used by MySQL doesn't support either, and want PKCS#1.  So despite having the files fully readable, MySQL is telling you it can't figure out how to "get the private key" out of the file.  Once you know the issue, it has a simple solution:
openssl rsa -in server-key.pem -out server-key.pem
when you are done with this command the beginning of the file should look like this:
-----BEGIN RSA PRIVATE KEY-----
Again, the internal format is different, so don't try and just change the text and insert the "RSA" into it -- it will look like it works until something try's to connect using SSL. Once you have this done, restart mysql again and you should be good to go.

NativeScript - Workers

ns-workersOne of the best new features in the brand new 2.4.0 release of NativeScript is WebWorkers.    For those who have seen me around in the community; you will probably all know how long I have been harassing the NativeScript Core Teams to get this done (Since Aug of 2015).  I even went so far as to create a plugin (NativeScript-WebWorkers) that allowed you to spin up more JS threads but with the major limitation that they didn't have access to the native OS side of things, they were only pure JS.    So getting real 100% NativeScript webworkers this release means I am a VERY happy camper!

The feature is fully cross platform (i.e. works on iOS and Android) and allows you to spin up additional JavaScript engines to do all your heavy lifting needs in the background.    Now obviously, the more you spin up the more memory and cpu you will use; so you want to treat them as a precious resource and only spin up those you need.   Let me re-iterate; use these for only heavy duty processes; each worker is another FULL JS engine, which takes a chunk of memory and cpu to just start and maintain.

They still have full access to the iOS and/or Android runtime just like normal.   The only difference between them and the main thread is that you do not have any valid access to the GUI or GUI elements.   You can attempt to modify the GUI, but you will crash your app as you are not allowed in even in a native app to modify the GUI outside the main thread.  Same rules apply to a NativeScript app.

The NativeScript Core Teams modeled the background threads after the web workers model.  They are created, developed and destroyed the same way as a browser web worker would be.   So lets dig in.

Everything is passed via messages between the workers and the main thread; so lets look at a sample demo:

main file

"use strict";
exports.onNavigatingTo = function(args) {
   let myWorker = new Worker('./myWorker.js');
   myWorker.onmessage = function(msg) {
     console.log("Hi I'm a message from the worker: ", msg.data);
     myWorker.terminate();  // We no longer need the worker around, so kill it.
   };
   myWorker.onerror = function(err); {
      console.log("Opps, something went wrong in the worker", err.message);
   };
   setTimeout(function() { worker.postMessage("a Cool Message"); }, 500);
}

myWorker.js
"use strict";
require("globals");
global.onmessage = function(msg) {
  console.log("Got a message form the main thread!", msg.data);
  postMessage("Worker's cool Message");
  // global.close();  // If ran, this would close the worker from inside.
};

global.onerror = function(err) {
  console.log("We can handle our own errors too", err.message);
};

Now as you can see we have two files; the first file is from the main thread it starts the new worker by doing let myWorker = new Worker([path to worker script]); this is how you start a brand new worker.   The new worker will load that JavaScript file and start it up.  Now there are some gotcha's we are going to cover on the worker side that you will want to know about.

  1. You want to require('globals'); as your first or second line.    If you do NOT require the global module, you will not have access to console, setTimeout , setInterval, and any other function you are used to using globally.   So requiring this function is pretty important for most workers.
  2. When you assign .onmessage (or .onerror if you are using it) you must assign them to the global variable.  The new version of the Android engine is enforces "use strict"; properly and having implied "this" variables is NOT allowed.  So as a habit when assigning something to the global scope; implicitly use global.
  3. All messages have a .data parameter that contains the data you sent from the other side.  When you do a postMessage({cool: "wow", I: "am"}); this will be in msg.data.cool and msg.data.I in the onmessage message.  This might catch you, but is easily fixed.  Please make sure that any objects you send across to the other side is fully serializable (i.e. no recursion, no native gui elements) ; if not it will fail unless you use some third party lib to serialize the recursive structure.
  4. Terminate() or Close() the webworker if you are no longer going to use it.  If you are planning on continuing to use it; then leave it running it is cheaper to leave it running (& not doing anything) than to terminate and restart.
  5. If you get an error message like this: Worker Error: Uncaught TypeError: Cannot read property 'prototype' of undefined this can mean it can't find the worker file that you wanted to load.  Using the tilde to say main app folder '~/path/to/worker' is the easiest way to fix it.  OR it can mean that the file that is required is doing something that is causing the worker to crash.
  6. If you see the error: Uncaught TypeError: global.moduleMerge is not a function The solution is to do a require('globals'); at the top of your worker file.

Once you understand these items, you are ready to rock and create cool background threads to do all your busy work so that your main thread never freezes again...

NativeScript: WebPacking

The NativeScript team released a cool plugin using the "WebPack" code to packup the entire code base and since it does things like tree shaking; your end result is a much smaller app.

Well I had an opportunity to test it over the last couple weeks for a friend of mine (Nathan Walker) as he is prepping to release a very very cool app.

However, the NS-Webpack plugin is unfortunately very (very) buggy; and odds are very (very) likely if you are doing anything beyond a very simple application; it will not work.   You can see a stack of bug reports in the repo for all of these issues I mentioned and some others; so hopefully these issues will soon be just a distant memory, and it will work as intended.

So many days later; and after much debugging, we can declare that we have smacked and cajoled webpack into shape and we were able to get it to actually webpack an application properly!

So here is the things you need to do if you want to webpack something while it is in its current state...

Fonts

The first thing you need to be aware of; is that on iOS if you are using any custom fonts; the automatic font registration is totally broken when webpack'ed.  The solution is to fallback to what we used to have to do on iOS; which is manually register the font.   The code I wrote for this project looks like this:

if (global.NSObject &amp;&amp; global.NSString) {
  var font = require('ui/styling/font');
  font.ios.registerFont('fontawesome-webfont.ttf');
}

Basically the if statement tests for iOS, then I manually require the font class; and finally the last line; I registered the font I needed.  You can repeat the last line for each font you are using.

Pathing Issues

The second issue is only if you use your screens/components in a subfolder type system the url pathing to find them breaks under webpack code.  So when you have something like this:

/app/components/blah/
-blah.js
-blah.html
-blah.css

and inside your blah.js you have code like this:
Blah = _decorate([
...
templateUrl: 'blah.html'
...

This will fail; ALL the JS is actually bundled into the webpack'd bundle file; so when it goes to try to find this file, the JS code is running from the root directory of your app.  Where your actual html file is not in the root.  So you have to change the path to be:
templateUrl: './components/blah/blah.html'

Pretty simple fix if you know about it; but man if you don't it starts crashing...

Crashing on Startup

The final issue is simple and complex at the same time.  NativeScript iOS runtime has some built in short-circuits for __extends and ALL the Native objects (i.e. like NSObject, NSString...) -- well Webpacking unfortunately changes the signature of the __extends function and so the runtime doesn't detect that the function is actually a __extends...  This causes the runtime to run code that isn't actually supported and so it, well, crashes...   This can be solved in one of two ways;

  1. You edit each of the plugins that are installed and manually strip out the __extends function.  (and then let the plugin author's know to disable creating the __extends in their plugin as this code normally is just pointless and a waste of space and memory, and in the webpack case it is now deadly -- So just add "NoEmitHelpers: true" to the tsconfig file).
  2. You add a simple plugin that I wrote to your main project directory; this plugin will automatically strip out all __extends while it is webpacks so that it doesn't matter if the plugin author forgot to set that flag; the simple plugin eliminate it.

/**********************************************************************************
 * (c) 2016, Master Technology
 * Licensed under the MIT license or contact me for a Support or Commercial License
 *
 * I do contract work in most languages, so let me solve your problems!
 *
 * Any questions please feel free to email me
 * Version 1.0.2                                      Nathan@master-technology.com
 *********************************************************************************/
"use strict";

function eliminateExtends(pre) {
    var i = pre.indexOf('var __extends = ');
    if (i === -1) {
      return pre;
    }
    var x = pre.indexOf('};',i);
    return pre.substring(0,i) + pre.substring(x+2);
}
module.exports = eliminateExtends;

You copy this code into an eliminateextends.js file in your root folder of the project; and modify your apps webpack's config file to use this as a loader plugin.

Please note this is not all the bugs in webpack; there are a couple others on the https://github.com/NativeScript/NativeScript-Dev-WebPack/issues repo that we didn't run into; but these were the only ones that we ran into for a iOS NG2 project.   So make sure you check out the bug list if you are still having issues...

Happy NativeScript-WebPacking!

NativeScript: Patreon Posts & Plugins

For those who are unaware; I've started doing paid content; see my post on "Why Patreon" to see the reasons.   I'm going to attempt to keep this post updated with plugins and posts that are available to those who are my patreon supporters.

Posts:

 

Plugins:

 

NativeScript v1.2.0 Built in LiveSync vs the NativeScript-LiveSync Plugin

Pros of Telerik's LiveSync:

  • Works from the NativeScript Command Line
  • No extra code added to your application!
  • Possibly works on Real IOS Devices (Untested on real device, but does not currently appear to work on a IOS Simulator)

Cons of Telerik's LiveSync:

  • Not really Live. It syncs the files; but then has to restart the application from scratch, no matter what file is changed.
  • Delays while it detects any changes and then deploys the changes.
  • Delays while it is re-launching Application.
  • Loss of all application state since it reloads the app on every change.
  • If you navigated three screens deep, and make a CSS file change; you will need to re-navigate to that screen again to see it.
  • Incredibly slow LiveSync startup time. (What in the world is it doing for about a minute?)
  • Can crash the LiveSync watcher code easily (don't change any files in the tns_modules!).
  • Does not apparently detect any new files...
  • Reset of the Application even if you change a file that isn't even being used.
  • Easy to crash your application as the JavaScript and XML are not checked before being sent to the application.

Con's of Master Technology's LiveSync:

  • Until Telerik accepts the patch; you have to use the included patched runtime. (Please vote up the issue!)
  • Added coded to your project.
  • Only works on the Android platform, no IOS support.

Pro's of Master Technology's LiveSync:

  • Live, You see the app change almost exactly when your editor saves the files.
  • New files are detected and synced instantly.
  • Application state is almost always fully maintained.
  • The screen you are working on only reloads ONLY if it is the code you just changed.
  • Built in ability to detect errors in XML and JS before pushing to device to eliminate crashing the app on the device.
  • Ability to only reload application on files that are singletons or other files that you would rather have the app reloaded for.
  • Ability to restart application by touching or creating a "restart.livesync" file.

 

The new LiveSync code has been updated to be a seemless installation on your Box.    It now includes the modified runtimes for v1.20 of the Android runtimes.     All you have to do to install it is: tns plugin add nativescript-livesync

 

 

NativeScript - Real time development on Android


Photo (CCA): https://www.flickr.com/photos/sleepyjeanie/5738474150

Now if you haven't guessed recently I've really taken to NativeScript.   It is awesome tool set for development applications for your mobile phone.

However, one of its failings compared to some of the other tools is the speed of iteration.     On my machine; when I want to test a change;  I switch to a command prompt (or hit a hotkey) which then compiles the app and deploys it.     This whole process is between 20-30 seconds each time on my beefy machine.  If I made any code mistakes; then I watch it crash on the emulator, have to fix it and then wait another 20-30 seconds while it re-compiles & deploys it again.    Another, slow issue testing area is if the screen I'm changing is a little ways into the application then I have to re-navigate back to the screen and test my changes each time I redeploy the app.    So all in all, it is a slow iteration cycle.

Well, I decided to do something about it; and so if you watch my youtube video, Real Time NativeScript Development , you will see how simply including the "nativescript-updater" the screen/code updates on the emulator INSTANTLY.    And this works for any screen I'm on, no matter how far I have navigated into the program.

In addition the watcher utility will also run any changed JS or XML through a linter before transferring it to the Emulator.  This allows me to keep the majority of my stupidity from even getting to the device in the first case which will end up in a crashed callstack.

You can still intermix the normal NativeScript deploys for any reason we want.   This project has been released at http://www.github.com/nathanaela/nativescript-livesync/ and is completely trivial to include into your project.

I'm waiting on a pull request that I added in the Android-runtime (https://github.com/NativeScript/android-runtime/pull/92) -- until it is accepted; the neat magic only partially work.

Announcing a v8-Natives v0.0.1

What are v8-natives, you might ask?    

Well, they are the mostly undocumented javascript commands that control the v8 engine in Google Chrome, Opera and Joyent Node.js.      Some of the commands are %CollectGarbage(), %GetV8Version(), %GetOptimizationStatus() which ties with my other favorite of %OptimizeFunctionOnNextCall()

 

What can I do with them?

You can tell the engine to Optimize a routine, un-optimize a routine, never optimize a routine, ask it about internal data structures of an variable/object, and one of the most important items is ask if a routine is optimizable.

 

Why is this important?

Well, the v8 engine has several compilers built in; the lowest compiler is just a full featured javascript interpreter -- it is fast; but compared to one of the actual compilers it is so slow that molasses moves faster.     Do you want to figure out which of your code can be promoted to the faster compilers?   Do you want to see what code is a bottleneck even though at a glance it actually looks good?

 

So, which of these routines optimizable?

function sum1(a,b) {
try {
var c=a+b;
} catch (err) {
return -1;
}
return (c);
}

function sum2(a,b) {
return sum.call(this, arguments);
}

function sum3(a,b) {  return  sum(arguments);
}

 

Look no further:

Available on isle 15, at the deep discount of totally free; we now have all the tools you need to answer the above questions.    A fully working support library that wraps over 20 of the internal v8 native commands in a simple to use library that will not crash your script no matter if you have the v8 native support turned on or off.  Can be left in your app and deployed; and finally   supports both Node and Browsers.

Simple things like "v8.helpers.testOptimization(sum1);"  would tell you right away if the sum1 can be optimized....   Or v8.collectGarbage() will do a full GC before you run some timings on a performance critical code...   Lots of things to help your inner-performance surface.

You can get it at your local npm repository:  npm install v8-natives or check out the github page @ https://github.com/Nathanaela/v8-Natives

 

 

Node & Browser Javascript Compression Update

I wrote a post on Data Compression back in October, http://fluentreports.com/blog/?p=18, discussing how I sped up a Data Compression Library that we have been using internally for all web socket traffic and how by combing techniques from different comparable libraries LZJBn.js was born.

Well fast forward several months --- I ran across another library that well professional curiosity compels me to to bench mark to see how well my cool LZJBn will trounce it.

Using 526 different sized files from real packets that we send:

Compression Decompression Compressed Size Original Size
LZJBn.js 0.503752017 0.1777535 15,890,401 37,345,189
node-lz4* 0.363441773 0.1109069 11,364,620 37,345,189

* - Node-lz4 does not compress files under a really small size; so there was 8 files comprising of a total of 182 bytes of data that was not compressed in this test on the lz4 side.  So because of this; when sending any data packets you will have to tag your packets as compressed or uncompressed.

And on even larger size using the ENWIKI file I used in the prior blog post:

Compression Decompression Compressed Size Original Size
LZJBn.js  1.87325003 0.279163174 34,332,875 50,000,896
node-lz4 0.713367233 0.207553456 27,591,715 50,000,896

Now if you look at the numbers it not only compressed and decompressed faster; but it also had a even better compression ratio.

After many tests and a very timely bug fix from the author of node-lz4 on a bug I reported; I have to shamefully say node-lz4 totally skunks my LZJBn.js module in the tests.   In addition Node-lz4 also has a native module for the node side, but those numbers aren't relevant to this test as this was purely testing the JavaScript library speeds.

So those who are wanting to implement as close to real time compression as possible using JavaScript; there is a now a new King of the Hill and sadly (for me) it is node-lz4.

Congrats Pierre; for a Job well done porting LZ4 to JavaScript -- and I know which library I will be using in the future!

For those who are interested the primary LZ4 site is here and the original author (Yann Collet) who created the lz4 compression format has a blog here: http://fastcompression.blogspot.fr/