Tag Archives: NativeScript

Another pointless NativeScript Security Plugin....

So, I saw this blog (https://blog.nativescript.org/nativescript-ios-security/index.html) article today, and was thinking awesome someone is actually talking about security in NativeScript apps... My elation was very short lived. Shortly, after starting to read it, I'm like, oh my G*D! Bad security advice is way worse than no security advice! At least the other insecure plugin NativeScript/Secure-Storage has a point, this one....

Well, if you read any of my prior posts on the in-security of React Native, NativeScript, Ionic here, and here. You might have guessed what my response to this is...

Lets show how trivial this is to bypass...

Creating the Demo Applications

First, lets create a throw away folder, and two simple test apps.

mkdir bypass
chdir bypass
ns create demoapp --js
ns create shellbypass --js
cd shellbypass
ns plugin add @nativescript/ios-security
cd ../demoapp
ns plugin add @nativescript/ios-security

Creating a Demo to test it...

The demoapp is our awesome multi-year released app that we have poured hundreds of hours and millions of dollars in. The shellbypass is the hackers app.

Open your favorite editor... Mine is phpStorm and it should look like this:

Next you need to open the demoapp/app/main-page.js file.

Then at the top we want to add this awesome plugin so add:

const IOSSecurity = require("@nativescript/ios-security").IOSSecurity;

Then in right below the page.bindingContext = ... we need to add our security check...

let amInEmulator = false;
if (!global.android) {
  amInEmulator = IOSSecurity.amIRunInEmulator();
}

if (amInEmulator) {
    console.log("Hey I'm in a emulator");
    page.bindingContext.set('message', "I'm in a Emulator");
  } else {
    console.log("Hey, we are on a real device!");
    page.bindingContext.set('message', "I'm on a real device!");
}

So the main-page.js should look like this (with all comments deleted):

Ok, lets run this on a simulator.

> ns run ios

And look our security check worked perfectly!

You can see on the top it is built for a simulator; and the console.log says: "Hey I'm in a emulator". So we have deployed everything correctly. Now switching roles to the attacker, this app won't run on an emulator, or will it?

Lets just pretend we uploaded it the app store and then used one of the techniques I outlines in the other blog posts to download your application IPA directly from Apple to my mac (or you can just use a jail broken phone). The IPA will look something like this:

Lets go into the "app" sub-folder, as it contains our original NativeScript app code.

So what do we have here... I outlined them in the other blog posts but we will dip briefly into it.. First lets copy the bundle.js, runtime.js and vendor.js to our "shellbypass" app into a new folder called "other". Now normally I would run some my custom tooling to re-extract the code back out of bundle.js/vendor.js into all their separate files, so I can easily rebuild the app from virtually the same code that you had. But in this case I'm lazy and I want you to be able to easily follow along so we are just going to use the raw files from the "original" app..

If we are on an actual jail broken phone, we actually don't have to do the "Creating a Shell" step, you can skip all the way down to "Removing Security" as you can edit the files directly on the device...

Creating a Shell

  1. Delete everything in app/ folder except the "app.js" file.

2. make app.js just have the following line of code to make sure that the IOSSecurity native code is built into the new NS app:

const IOSSecurity = require("@nativescript/ios-security").IOSSecurity;

3. Edit the webpack.config.js and make it look like this:

const webpack = require("@nativescript/webpack");
const fs = require('fs');
const platformInfo = require("@nativescript/webpack/dist/helpers/platform");

module.exports = (env) => {
webpack.init(env);

let output = platformInfo.getAbsoluteDistPath() + "/";
if (fs.existsSync(output)) {
fs.rmdirSync(output, {recursive: true});
}
setWait(output);

return webpack.resolveConfig();
};

function setWait(output) {
if (!fs.existsSync(output+"vendor.js") || !fs.existsSync(output+"runtime.js")) {
setTimeout(() => { setWait(output); }, 100)
return;
}
fs.copyFileSync("./other/bundle.js", output+"bundle.js");
fs.copyFileSync("./other/runtime.js", output+"runtime.js");
fs.copyFileSync("./other/vendor.js", output+"vendor.js");
console.log("Copied");
}

Again I'm being very lazy, basically we are letting webpack work normally, then copying the original demoapp files into our hackers shellbypass app.

In all reality splitting the code back out of the bundle into its separate files makes it a lot easier, but my tooling is not very clean so I'm not ready to release it yet. But it is trivial if you read my prior blogs to split the files back out and de-minimize them...

So now we have a shell app that should run the original demoapp code as is. Lets test it.

> ns run ios --no-hmr

As you can see this app is called "shellbypass" and it is printing the same "Hey I'm in a emulator".

WooHoo, we have successfully cloned your million dollar app into our own free NativeScript shell.

Removing Security

All of that just to show you have this plugin doesn't matter... So first there is two ways to do this... I'll show you both ways, just to show you how trivial it is to remove these types of check's.

Bundle.js changes:

Search for "amIRunInEmulator" -- Look we found it on line 205... Lets just comment that right out...

Now what happens when I run my app...

Now the app prints "Hey, we are on a real device!" The awesome plugin's security sure did stop me from running on an emulator, didn't it? The app you so carefully programmed, and may have even spent years on -- I just simply removed the check and ran it and now I'm playing with it on a simulator or a jail broken device.

However, the vastly better method is to just change the vendor.js file, so that I don't have to find every place you put the checks in your code. (So first undo the comment we just added in the bundle.js, so it is back to "stock" demoapp)

Vendor.js changes:

This is the vastly superior solution, lets look about line 46849, and just return false....

All I would do is return false on basically every single one of these functions in the IOSSecurity class, and the plugin doesn't do a single thing no matter how many times, nor where you called it inside your application (the bundle.js).... Defeated in less that 5 minutes...

Wrapping it all up...

The problem with security measures like this, is that NativeScript, React Native, Cordova/PhoneGap, Ionic is they all ship the source code ships inside the app. It is trivial to extract it out, create a new shell and make the changes to make it my own "clone" project or to steal/corrupt your data by using your endpoints. If you plan on using something like this plugin you need to use a product like JSScrambler or https://AppProtection.net (my own product which encrypts the NativeScript source code). If you just ship the app using minimization (pointless for security, but useful for optimization). Please read the prior blog posts on how easy it is to pull everything out.

If you are interested in more info on my current NativeScript solution AppProtection, or if you want to help fund the vastly superior solution that eventually can work for both React Native and NativeScript with no coding changes in your app; please contact me at nathan@master.technology.

PSA: Your Android and iOS Application is not secure...

I was talking to a community member about some of the sites I run (and I mentioned a site called https://AppProtection.net) as they didn't realize I also do a lot of server side work. However, she was shocked that the source code for her app was easily accessible by everyone.

If you write an application with NativeScript, React Native, Cordova/PhoneGap/Ionic (and many other platforms) are you aware you ship the entire JavaScript source code, styles, layout code in your app.

For simplicity we are going to look at a NativeScript application that we will generate quickly so you can easily follow along. These steps apply equally for React Native and the Cordova based eco-systems.

You can uses tools like apkeep to download APK's directly from Google play store, or you can use something like Apple Configurator 2 on Macintosh to download the IPA from the Apple iOS store to your computer. Anything published to either store can be retrieved to your computer directly, and that is when the fun begins...

How easy is it?

 ns create demo --js
 cd demo

This command creates a demo application, for simplicity I'm just going to use pure JS, but again this applies to all the NativeScript flavors, Vue, Angular, TypeScript, React, etc.

Now we are going to make some minor changes to the main-page.js file to look like this, as we want emphasize that any secrets are NOT secret! This includes all Tokens, your http(s) URL endpoints, API keys, passwords, usernames, etc....

import { createViewModel } from './main-view-model';

const secret_HTTP_APIKey = "Hi_IM_A_SECRET_TOKEN_USED_ACCESS_SOME_API";
const private_Firebase_Token = "Hi_IM_your_Secret_Firebase_token";

export function onNavigatingTo(args) {
  const page = args.object
  doSomethingWith(secret_HTTP_APIKey);
  doSomethingWith(private_Firebase_Token);

  page.bindingContext = createViewModel()
}

function doSomethingWith(value) {
  <strong><em>console</em></strong>.log("Pretend HTTP Call with token", value);
}

Now we will build it:

ns build android --copy-to .

I'm using the <strong>--copy-to .</strong> so that we copy the final apk to this folder so we don't have to go looking for it in the platforms/android... folder, you should have something like this now...

The app-debug.apk (or .ipa for ios) is basically what is uploaded to Google Play or the Apple iOS store. Lets look at this file. If you unaware a .apk is just a .zip file with a different file extension. In my case, I just renamed it to .zip so that Midnight Commander (my favorite terminal shell) can navigate inside and view files directly....

Opening up the .zip/.apk file, and for NativeScript navigating to assets/app you will see the files that matter. The bundle.js is your application code, the vendor.js is all the plugins and NativeScript core code, so 99.9% of the time this is a pointless file for reversing your app. However, if you have custom plugins this would have that code. You can also see any worker javascript files if you are using any workers, and angular has some code splitting that can create some other separate javascript files. The bundle is typically the most important file... Lets just copy it out of the zip file so we can open it into a real editor....

How secure is my JavaScript, lets see the main-page.js?

Opening bundle.js up in PHPStorm, this is what we see:

Opening up bundle.js

Interesting it tells us we have 6 files that were compiled into this app.

Lets look for the main-page.js inside the bundle! Simply do a search for main-page.js and this what we find around line 100 in the bundle.js file (that is because the prior 80 lines or so is all the CSS that was bundled from app.css and the default imported theme css files)

main-page.js entry

Wow, 100% full source code including all your awesome secret keys and tokens (which I selected)... If you compare this to our file above, the only difference in this code is that we original did <meta http-equiv="content-type" content="text/html; charset=utf-8">export function onNavigatingTo(args) and this removed the <strong>export</strong> from the function on 114 and added it on line 105 - 107 as a __webpack_exports__. Otherwise this code is identical.

What about the layout, Main-Page.xml?

Lets search for main-page.xml, oh it shows up on line 193...

main-page.xml entry

It is just one long string... What happens if we copy & paste it to a newly created test.xml file. How different is this from the original main-page.xml file, not a single change. It is the exact same file.

Uh oh...

Can anyone tell me how quickly I can take your fully released app, that you spent months or years on creating and make my own 100% full clone? Or perhaps even do things more nefarious and add, edit, or steal any data from any of your private endpoints?

What can you do about this?

There is three things that I'm aware of

  • Minification (Built in to NativeScript & React Native)
  • Obsfucation (Lots of different tools)
  • Encryption

Minification

Well, the first thing people think of the minimization using Terser (or uglify). NativeScript uses Terser by default.

Lets see what happens when we minimize the code using NativeScript 8. Here is the bundle.js file using minification.

Minimized output

This looks hard to parse... But lets see what happens if we use the awesome "Reformat code" option in phpStorm/webStorm...

Code automatically reformatted.

Well, it is different, but still totally readable. How about the main-page.js and our secret keys?

Well that id disappointing, the code is perfectly readable and usable. All your secrets api keys are well, not at all secret. The only thing missing is all the comments were stripped and my "console.log" was removed from the doSomethingWith routine. But the code is very readable and can be dropped virtually as-is in a new main-file.js javascript file. If I wanted to make this file to work in a new app, I would just need to replace "r" with "require" and add the "export" to the onNavigatingTo function and it would be 100% working. Add a couple refactors like "t" to "page" and extract the API keys to variables and it would be virtually identically to the original you wrote. Lets see, hacker 5 minutes, you 5 months.

As you can see minification does nothing really to hinder anyone. But it is a good step for optimization but adds zero security.

Obfuscation

What about obfuscation, this really depends on the tool. I have a blog post about this security of this. However, rather than re-iterate it, it is almost as trivial to reverse as we just did with minification. The only tool that took me longer than a few minutes to reverse was JSScrambler. However for JSScrambler to work you have to set this at its highest level and enable code protections so that a hacker can't easily modify the code. Unfortunately, the last time I tried this is slowed all of the code down by 30-50% the entire life time of the application.

Compiled Code

This is probably the next best thing, if you create a function that has your api keys encrypted in it and you have to call a couple functions to return them before you use them they won't be visible in the source which seems like an awesome win. However, if your JavaScript code has:

const api = getAPI(); 
const key = getKey; 
https.get(api+key);

Do you have any idea how quickly I'll get your api & key, I'll just change the code to be:

const api = getAPI();  
const key = getKey;  
console.log(api+key);

Did you slow me down, yes, by how much -- maybe 15 seconds to modify the apk and upload it to by emulator (or a jail broken android/ios device) . So if you use this method, please do NOT telegraph how to get your Key & api endpoints. At least be more discrete as when I do a search for "https" or "firebase" in your code base you don't want to point me to where I need to go next.

React Native offers a engine called Hermes where it partially compiles the code, this is more secure than the pure JS code mode, but obviously not as secure as pure compiled. However, even in pure compiled code, if you do nothing to obfuscate your keys and api end points, just running strings against it will give you everything a hacker needs to attack your end points.

Encrypted Code

First a major disclaimer here -- I am the author of the AppProtection system, of course I recommend it. Second disclaimer, this is currently NativeScript only, but if I get enough interest for a React Native version, I'll port it as I have all the code all done for both v8 and JavaScriptCore engines. But I have to spend the time adding it to the React Native tooling. Basically this encrypts the source code for your release apps. This is what a release file looks like:

Encrypted Code

The source code AES encrypted, and the only unencrypted copy is in memory (never saved to disk) while the app is running. The actual v8/JSC runtime engine itself has the modifications for understanding AES encrypted JS code, no extra code is shipped in your app. Is it perfect? Nope, but it adds another very difficult wall to climb...

The 100% Solution

Click bait! Sorry there is no 100% solutions. I haven't heard of any of the AppProtection applications being cracked, but that doesn't mean it can't be (or hasn't been). I will tell you upfront everything can be cracked, including AppProtection. Even Denovo which is used in high profile games and has millions of dollars in revenue. Eventually, each new release gets cracked. Everything can be hacked with enough time, patience and knowledge. However, the whole idea of security is you put enough walls in front of your attacker to make him want to go to a much easier target. If they leave, then you have thwarted an attack. Do you really think most attackers or application cloners want to spend weeks pulling your data or cloning your app, or switch to something that only takes them a few minutes?

More information

  • I have another blog post I wrote in 2019, everything is still valid and I discuss other things you can do in your app for security and other security gotcha's.
  • I am available to do consulting work to help you beef up your security both server and mobile. I also do core reviews, and training if your developers need to learn some better practices.
  • You can email me at nathan@master.technology

Fond memories, and onto new adventures....

For those who may or may not know me, I was one of the very earliest NativeScript developers. Started the Facebook and Google groups NativeScript forums. Created some of the very first plugins, helped train a wide number of early NativeScript developers, wrote books, and have some of the most widely used plugins in the entire eco-system. I also run a whole slew of different NativeScript related sites like PNR & SearchCode all under the Master Technology brand. Sufficed to say, I have been a advocate and cornerstone of NativeScript eco-system since it first started.

I also co-founded the nStudio company several years back with a couple other co-owners to focus solely on NativeScript work, and leave Master Technology to do everything else I do (i.e. servers, security, desktop, etc) . nStudio became the largest NativeScript development houses. Then awesome fortune came our way and we were able to take ownership of NativeScript. Finally over the last several months, nStudio was able to join and transfer NativeScript to the OpenJS Foundation (the foundation that supports projects like NodeJs, webpack, electron). So it now has a awesome open source home and foundation to support it for the foreseeable future.

However, unfortunately, during this season of time we had some large disagreements in nStudio leadership, on a myriad of issues. Do to some of these disagreements, I was abruptly terminated from the NativeScript TSC (a couple months back) and finally I officially left nStudio this week. So my work as an NativeScript Core/TSC member ended. I might still push a PR here and there for any client issues I run into, but that is probably the extent of it.

NativeScript...

Before the fallout, I had several ideas for NativeScript that I really wanted to pursue, and never could get any internal nStudio support. Since I am now 100% funded again by my clients work. I'm curious to know if anyone in the eco-system is interested in helping fund any of the following, if we have enough interest I'm willing to pursue any/all of them. I have the experience to make them work.

  • More NativeScript targets:
    • Web (A web front end, so you can use your NS skills and create web sites or apps that run inside of electron/tauri)
    • Desktop (actual native desktop front end, not using html)
    • Sailfish
    • PureOS
  • NativeScript build servers (i.e. easy ability to build everything from the cloud)
  • NativeScript front end, basically like the old SideKick, with lots of helpful tooling.
  • Compiled NativeScript (ios & android), your JS compiled to 100% native code.

Of course you can still hire me for any other NativeScript work you might have, if you have no interest in any of the above ideas.

New Adventures

Interestingly enough towards the end of this period of time, shortly after I finally decided to leave nStudio. Another technology stack was opened up basically exclusively for me to use externally. It is not a stack directly for mobile development, but a stack for rapid application database development. Virtually all applications use a database for the data, this takes the concept to the next logical leap. I've been doing database development for 30 years, and this technology stack excites me. You might compare it to Clarion, Filemaker or even Access (but way better). Designed from the ground up to have per field/row/screen security & auditing, mobile first, charts, reporting, etc. If you company needs any type of desktop/mobile application built let me know, this stack allows us to build it a lot faster meaning we can save you a lot in development costs. It doesn't matter if what you need is as simple as a contact database to something as complex as something that can fully run every part of your company. It scales and allows tying of different data sources very seamlessly.

In addition, if you have existing Access databases you want to convert to be modernized and intranet facing. I'm already pursuing working on a Access DB conversion to allow your app to hopefully seamlessly transition into this platform with little or no changes. I'm not opposed to writing converters from other platforms also, feel free to hit me up if you have some internal company apps you want modernized.

NativeScript, Master Technology, ProPlugins, and nStudio.

So since this question has come up multiple times; I decided to write a brief history of my involvement with NativeScript, Master Technology, ProPlugins, LLC and nStudio, LLC.

So this story begins a little over 5 years ago; I had been doing some Cordova/Phonegap development on a simple application for my wife; and its performance ended up being sub-par, ran into screen sizing issues, and a whole myriad of the common standard issues you run into with programming for a webview. Most these hybrid issues are much better now. At that point -- frustration was enough to make me decide to see if their was anything better.

I downloaded a huge number of platforms, React Native, Titanium, Fuse, Xamarin, you name it; I found a link to it I downloaded it and tried it out. After spending multiple days installing and testing things, I finally decided to using React Native -- it was the best of the bunch. A few days later I saw a article on a news site; saying checkout something called NativeScript. I figured, might as well -- I wasn't totally sold on React-Native and I hadn't started my project.

So I downloaded NativeScript v0.90 or v0.91, which was the first official "open" release in early March of 2015. I totally fell in love with it. It has a good template syntax, data binding and best of all full easy access to the entire native platform all using JavaScript. It really was the "perfect" mobile development system.


Now as to Master Technology, it was started a long long time ago, in a galaxy far far away... Almost 30 years now... So basically, in a nutshell this was the company I did all of my contracting work under. As such, when I started doing NativeScript work I continued to do all of my work under it. This is why you see a large number of plugins and NativeScript related sites with the Master Technology logo. One of the first, https://plugins.nativescript.rocks, started as a simple MD file on github...

However, that soon became unworkable when the community started adding multiple new plugins a week. So I created the first version of https://plugins.nativescript.rocks... From there images, searchcode, compatibility chart and many other community resources have been added...


After a couple years of working in the NativeScript community, I realized that as a sole contractor it was very hard to handle larger jobs by myself. In early 2017, I had the brilliant idea to start a company that would be more like a law firm, with senior and junior partners but geared for developers. I enlisted my co-owners Brad Martin, and Nathan Walker and we created a brand new company called nStudio in early 2017. From there we grew a bit, to the point we even added a new co-owner in 2019, Dave Coffin. We continued to grow adding several partners known highly in the community to our team like: Igor Randjelovic (NativeScript-Vue), Osei Fortune (author of many awesome plugins), Alex Ziskind (the tutorial master of NativeScripting.com). In addition we have developed strong ties with many other companies in the greater developer eco-system.

When any new NativeScript clients would contact me, I move them to nStudio. Any other clients (Servers, PHP, Clarion, Node, etc) stayed on Master Technology as that company is now more oriented to server side stuff, and nStudio was for all the mobile development.

One strange thing about having two companies is dealing with IP, so any IP that I had developed at Master Technology, remained with Master Technology. You will still see Master Technology on a LOT of sites and plugins in the NativeScript eco-system even though Master Technology only does a small amount of work in the NativeScript eco-system now.


In early 2019; as still one of the major open source plugin NativeScript developers; I realized that NativeScript community needed some way to fund plugin maintenance. We were seeing a great number of plugins that were not maintained and no longer working. The maintenance time on my own plugins, that were free to the community, became far too excessive for me to be able to keep them to a standard I'd be happy to have my name on, and still be able to feed my family. I got together with several other large plugin developers and we started ProPlugins, LLC. This has helped at least with a chunk of the plugins under our control; and so ProPlugins, was founded and is owned by Master Technology.


Now comes the most interesting turn of events in this entire story, one that I would NEVER have expected in a million years.

Because of NativeScript, I helped found two additional successful companies. Which just by itself is totally awesome! However, now nStudio, LLC has been able to purchase NativeScript and its assets. We now have control over not only our companies future, but the framework that was directly responsible for spawning nStudio, LLC.

As such, you should now see my name (and the other partners from nStudio, LLC) on the all the new pull request approvals and reviews for NativeScript. I do want to sincerely thank all the developers over the last 5 years who actually built NativeScript, and if any of you want to continue to work on the framework, please feel free to drop us a line at team (@) nativescript org.

(c) Adam Carter, https://www.flickr.com/photos/44811338@N05/7505286308

Optimization Gotcha's for for/i and forEach

So I mentioned something on my interview with Alex of NativeScripting.com that he did with me. And someone asked about this in the comments, so I decided to create a blog article on this specific optimization tip.

I am going to code this to a browser rather than in NativeScript because well JS in a browser is just plain easier to show the memory issues because you can plop the code in to a new browser tab and step through the code instantly. 🙂

&lt;html>
&lt;body>&lt;div id="stacklayout">&lt;/div>&lt;/body>
&lt;script type="application/javascript">

// Pretend Color class to be similar to NativeScript
class Color {
    constructor(val) { this._color = val; }
    toString() { return this._color; }
}

const <strong><em>colors </em></strong>= ["#ff0000", "#00ff00", "#0000ff"];
for (let i=0;i&lt;<strong><em>colors</em></strong>.length;i++) {
    const labels=["Wow","Awesome","Interview", "Alex", "Created!"];
    for (let j=0;j&lt;labels.length;j++) {
        const clr = new Color(<strong><em>colors</em></strong>[i]);
        const txt = <strong><em>document</em></strong>.createElement("div");
        txt.innerText = labels[j];
        txt.style.backgroundColor = clr.toString();
        const parent = <strong><em>document</em></strong>.getElementById("stacklayout");
        parent.appendChild(txt);
    }
}
&lt;/script>
&lt;/html>

And you should see something like this:

We are repeating the labels for each of the primary colors...

Or you might be used to doing something like this instead:

&lt;html>
&lt;body>&lt;div id="stacklayout">&lt;/div>&lt;/body>
&lt;script type="application/javascript">

class Color {
  constructor(val) { this._color = val; }
  toString() { return this._color; }
}

["#ff0000", "#00ff00", "#0000ff"].forEach((clr) => {
  ["Wow","Awesome","Interview", "Alex", "Created!"].forEach((word) => {
     const txtColor = new Color(clr);
     const txt = <strong><em>document</em></strong>.createElement("div");
     txt.innerText = word;
     txt.style.backgroundColor = txtColor.toString();
     const parent = <strong><em>document</em></strong>.getElementById("stacklayout");
     parent.appendChild(txt);
  });
});

&lt;/script>
&lt;/html>

Both have several issues, however the second one can be vastly worse than the first one depending on how many iterations. So lets walk through the issues.

So the Color class is fine, we are using it to show creating objects... So just ignore it for now.

const <strong><em>colors </em></strong>= ["#ff0000", "#00ff00", "#0000ff"];
for (let i=0;i&lt;<strong><em>colors</em></strong>.length;i++) {

These lines are fine, we allocated colors, then looped through them. So far so good. Lets proceed, what about:

const labels=["Wow","Awesome","Interview", "Alex", "Created!"];

Well here is the first issue; Fortunately for us, the v8 JS engine actually pre-allocates the 5 different string "Wow" through "Created!" when it parses the JavaScript however, each loop through it does create a brand new Labels array and then links those strings into it. So in this case it is small overhead; however in the 3 loops that still is 2 more sets of allocations and all new GC enties that have to be cleaned up after the loop is completed. If you were to put this single line of code outside the loop; your code runs the same, but no less memory usage and less processing.

 const clr = new Color(<strong><em>colors</em></strong>[i]);

What about this line, well in this case all 15 times it creates a brand new Color object. You can easily make this only be ran 3 times just by moving it outside the inner for (j) loop into the for (i) loop. In this example the Color class is very light weight. But many classes you create inside loops will be very heavy with lots of allocations going on when you allocate the class. So pay attention, 3 allocations vs 15 allocations, all by putting the line in the wrong spot...

Continuing on, most the rest of the lines has to be inside the loop; but I tossed another item into the loop, that wasn't needed... This line is very easy to accidentally end up in the loop because you needed it at that point.

     const parent = <strong><em>document</em></strong>.getElementById("stacklayout");

Yep, this has to do an expensive lookup in the dom to find this element, it never changes. It also has to allocate it to that "parent" variable 15 times. If you dropped it before/outside of both loops, you would have a single lookup and allocation...

---

Finally lets look at the last example; it suffers from the same issues. However, this one I used a Array.forEach rather than the simple for (i) loop. Why do I say it can be worse then? Well in small numbers, the difference is actually pretty tiny. The extra amount of memory it uses over a for (i) loop is also reasonable. But in larger loops (especially loops inside of loops inside of loops); the difference can end up being exponentially larger.

Your creating and calling functions, and all those functions now has added scopes and the engine has to setup all the callstacks for each and every iteration, additional error checking and just plain running a lot more code behind the scenes. So what is a simple for (i) loop in the first example, now became 15 EXTRA functions calls with all the extra overhead of cpu and memory that entails. Again in small chunks they are perfectly fine, but if you are wanting performance, every ms and gc adds up and sometimes they can add up very quickly when dealing with the Array helper functions that use callbacks...

As a better way to phrase this; imagine you have a for (i) loop that takes 5ms and a forEach loop takes 25ms to finish. Both are still dang fast, hard to even benchmark. But now you embed it in another forEach that also by itself takes 25ms. 25ms * 25ms = 625ms. More than half of a second. But, if you were to switch this to a for (i) loop each of them is 5ms. So, 5ms * 5ms = 25ms. Both are exponentially larger, but 25ms vs 625ms and the difference starts becoming much clearer. Now add a third loop; 5ms*5ms*5ms = 125ms. 25ms*25ms*25ms=15,625ms (or > 15 seconds!) In small chunks the time and memory used from a forEach is minuscule. But depending on how many times it is called; it can add up very quickly to be a major difference.

Please note this goes for all the nice Array helper functions, .map, .reduce, .filter, etc. Everyone of them that uses a callback, has somewhere between 2 times and 50 times as much overhead as a simple for (i) loop. Remember, despite this overhead; these functions are extremely fast. So don't be afraid to use them, but be more wary of using them in deeply nested code that can easily be called in loops when the datasets they are working against are also large.

As many items that you can actually move outside of your loop, the better off you are. This Includes any expensive calculations! You might be able to do 99% of the expensive calculation outside of the loop; and then finish off the last part of the calculation inside the loop...

I hope this answers the question about loops and performance. Every loop is a golden opportunity to greatly enhance your app,

NativeScript Fonter revisited again (using DuoTone FontAwesome)

In a prior post I discussed using fonts in NativeScript. This morning I saw an issue in the NativeScript github repo about how to use the Font Awesome Duotone fonts.

Since I love puzzles, and I know the NativeScript font system fairly well, and I also own a subscription to the commercial Font Awesome -- I figured I could easily see if I can make it work. Immediately, I had a idea on how to solve the issue.

So I downloaded both the WebFonts and the Desktop fonts. Interestingly, enough the Desktop fonts actually have a warning that links to a post about Duotone fonts and the issues with the desktop... So after I read that post; it basically confirmed my understanding on how this font works and exactly how to make this work inside NativeScript.

So first things first, you need to update your app.css file; you want to add something like this to it:

.fad {
   font-family: fa-duotone-900, Font Awesome 5 Duotone;
   font-weight: 900;
}

.fad-op {
   opacity: .4;
}

Why TWO names for font-family?

If you recall from my prior fonter post; you want to create the rule that has the name of the actual font file without the .TTF or .OTF extension. So since the font folder looks like this:

The last file in the example fonts folder, is fa-duotone-900.ttf; straight from the web download zip file that I downloaded this morning.

So you see the rule is:

   font-family: fa-duotone-900, Font Awesome 5 Duotone;

This allows NativeScript to find the actual file name, in a lot of cases it can use just the filename to work its magic. However on iOS specifically, we want to include the actual internal font name. So double clicking on the font gives us this image under linux.

Under macintosh the title bar also shows the exact same information. So that is why I then typed "Font Awesome 5 Duotone" as the second part of the rule -- this must EXACTLY match. The two names covers all the cases we need, font file name to load it and actual internal font name to reference it later.

The second css rule we just added was fad-op, it is so that we can have some opacity for which ever section of the font we want or need opacity on.

We can do no opacity, solid/solid, and it would look like this:
Notice both parts are solid.

Now if we add opacity to section two; you can see the beak, legs on the bird are slightly transparent; and the acorn body is partially transparent.

Finally if we reverse the opacity to make section one have the opacity it looks like this:

You can also apply any opacity you want like 0.10 looks like this

And you can apply opacity to both sections and it looks like this:

All of them are using the EXACT same colors; just a difference in opacity. The sky is the limit; you are free to color and apply opacity on whatever amounts to each of the two sections.

So how do we do this magic?

Well technically the hard part is already done, that awesome css rule and dropping the font file into the app/fonts folder has 99% of the work done for us. The rest is simple stuff.

So the next thing you need to know is; which duotone icons do you want to use?

So first we need to know what is available for duotone; so we go to the font awesome icon page; and select the duotone filter. Once we have that; we can choose something; so lets look at the crow.

The crow at the top of the page lists this information:

Now at this point we can't really use "fa-crow" just because NativeScript doesn't know what that is; but the two pieces of information we do need to know to make it work is the f520 and the 10f520 these are the unicode values for the two sections of the icon we want.

So our xml needs to look like this:

 
  

This gives us the two characters in the same location on the screen. The first character is using the "fad" css rule; along with the text color blue. The second character is also using the "fad" css rule, and it adds the red text color and the "fad-op" rule for opacity.

Which gives us the duotone bird:

The key item is that you want the characters to print over each other; Gridlayout by default allows you to put multiple items in the same row and column. Since the characters are the same size; they by default will automatically line up and work.

Angular Flavor Issue

Angular's HTML parser has an issue which characters in the upper unicode -- it is great with any unicode value 65535 and below. Which of course if you have seen the first character of each of these duotone values is < 0xffff (65535). However the second character starts in the 0x100000 (1048576) range which is a lot larger than 0xffff. Unfortunately this means that in Angular even though you pass in 0x10f520 you actually get 0xf520 assigned. If you are paying really close attention, this is the same value as the very first code. So in angular you get two labels with the exact same unicode value. So how do we fix this?

Well ultimately; the underlying Angular parser should be fixed -- so if one of you wants to tackle this, track down where in the parser it breaks and push a PR; that would make a lot of people in the Angular community happy! However, since this is not something I am going to tackle and I still want you to be able to use Duotone fonts, I will present an alternate method to how to fix this on Angular.

In Angular; we can do it one of two ways; automagically; or direct assign. I prefer the automagically; so this is the HTML that we have:

<Label col="1" text="&#xf6ae;" class="larger fad "></Label>
<Label col="1" text="&#x10f6ae;" class="larger fad blue fad-op" (loaded)="fixAngularParsingIssue($event)"></Label>

As you can see it is identical to the above NS-Core XML, except I added the (loaded) event. This event fires once in the lifecycle of the Label. And runs my handy dandy "fixAngularParsingIssue" function.

fixAngularParsingIssue(args: any): void {
  let field = args.object;
  let val = field.text.codePointAt(0);

  // Check to see if this is already a valid unicode string above 0xffff.
  if (val > 65535) { return; }

  // We found a character we need to change...
  val = val | 0x100000;
  field.text = String.fromCodePoint(val);
}

The function is straight forward; we grab the current text value; verify it is actually still less than the unicode value we want as the parser could get fixed in a future version. Then we take that value and add our missing 0x100000; and then re-assign the text. Now this function could instead be written to take the actual value you want to assign rather than computing it, but I prefer simplicity as then I can just add this function to all labels that need it and just not think about it.

Angular Directive

I suggested in the issue that someone create a directive; and Pandishpan spent the time and converted my simple code into a directive for Angular making it even easier to use in angular! Thank you for your awesome work! So now, all you have to do is apply the "labelDuoTone" directive to the html, and it handles the rest of the work for you.

You can grab the code for the angular directive in the github issue here.

NativeScript Core demo application

You can download the NativeScript-Core fonter application from the github repo, https://github.com/NathanaelA/fonter

Please note the fa-duotone-900.ttf font is NOT shipped with the demo; you must download it yourself and copy it into the app/fonts folder.

Android (on left) and iOS (on right) looks like this:

Android
iOS


NativeScript 6.4 released

NativeScript 6.40 has a couple cool new things; but better yet it fixes several issues that can will affect your users if you are using 6.3.x lets look at them...

Some of the fixes include instability during android's fragment handling occurred in 6.32, which is worth while. A couple really awesome community PR's, and of course one of my favorite items is the new CSS parser by default and the v8 upgrades...

[UPDATE] The new 6.4.1 Android engine that was just released a day or so after 6.4.0 adds v8 version 8! Which add a lot of cool features (see v8 section)

Core Modules

  • NativeScript is got a new much faster CSS parser in v6.3 -- it is now default in 6.4!
  • The community (Ryan & Hamdi) and the NativeScript team worked on 3d rotation for a while now; this has now shipped! .rotate-x, .rotate-y, and .rotate-z
  • .Fixed background color reset on Tabs

We had even more awesome community PR's this month, a big thank you to the community!

CLI

The CLI team has fixed several outstanding bugs; including several webpack hmr issues. This also includes Node 13 and Android API 29 compatibility.

  • CLI now cleans up after itself in a corner case where CTRL-C'ing when first starting the build.
  • Doctor now will throw error if using Java 13 (NativeScript doesn't support it yet)
  • CLI Will now detect and let you know about the breaking issue of including multiple versions of the same plugins or multiple NativeScript core versions.
  • Native Metadata filtering, allow Application/Plugins to declare what Metadata they need; to allow smaller metadata and faster startup.

Webpack Changes

  • Snapshot bug fixes
  • Ability to set a Webpack config location so you can add custom application rules
  • Ability to allow custom platforms building (--env.platform <platform>)

V8

If you didn't read my notice on the last update; I'm creating this new v8 Section, because Android and soon iOS will both be using the same v8 engine. v8 was upgraded to v7.9 (in 6.4.0) and then the awesome v8.0 was slipped into (6.4.1) which adds:

  • IC Handlers are now cached properly; giving a ~12% for all IC calls into the C++ runtime.
  • OSR Replacement - 5% - 18% speed up.
  • Pointer Compression (Saves up to 40% of heap memory) which increased speed on large JS by a few percentage points!
  • Optional Chaining
  • Null coalescing

Android

  • Async WebAssembly issue resolved
  • Memory freed on some objects that were being retained
  • Upgraded to V8 8.0 (See v8 section)
  • Fix for Elevation usage on some API versions
  • "FlipRight/flipLeft" transition fixed

IOS

  • The CLI will warn you if you forget a module.modulemap file in your included .M files
  • Project Template is FastLane compatible
  • Fixed discarded exceptions
  • Safari Inspector now shows startup errors
  • A lot more changes to the new v8 version of the iOS runtime that is now in BETA testing to eventually replace the JSCore version... Overall now is the time to test it!

Updating NativeScript

To get updated; you first need to do:
npm i -g nativescript@latest

To update a project; you need to do the following:

Latest Runtimes:
tns platform remove android && tns platform add android@latest
tns platform remove ios && tns platform add ios@latest

Latest Core modules:
npm r tns-core-modules --save
npm i tns-core-modules@latest --save

To install Webpack support:
npm i nativescript-dev-webpack@latest --save-dev

To install latest NativeScript Angular plugin
npm i nativescript-angular@latest --save
You will then need to install the actual angular bits; which as of this post v6 is currently supported.

The addition of t additional analytics/tracking to the CLI reminded me; you can disable it permanently; if you value your privacy by doing:
tns usage-reporting disable && tns error-reporting disable

NativeScript 6.3.0 Released

NativeScript 6.30 has a couple cool new things; but better yet it fixes several issues that can will affect your users if you are using 6.2.x lets look at them...

Some of the fixes include instability during android's switching of applications and resuming it. An issue with unloading views on Android. Out of memory situation on Android, and dialog fixes for iOS. In addition to several important build fixes that appear to solve a lot of build HMR related issues. Overall this release is worth upgrading to for just the bug fixes alone.

Core Modules

NativeScript is getting a new CSS parser; unfortunately it is not enabled by default in 6.3; but based on a PR should be enabled by default in 6.4. If you would like to enable this in your app to see how much faster it can parse your css; you do so by editing the packages.json file that ships with the application and adding a new key called "cssParser" and set it to "css-tree". The valid options are "rework" (current parser) and also "nativescript" which was the original css parser.

We had a couple awesome community PR's this month the first by Eduardo Speroni adds requestAnimationFrame as a supported function. And the second by Shailesh Lolam  add dialog sizing (width & height properties) to the modal dialogs on iOS.

And finally to round up our new core module features; Modal dialogs now have all the same CSS properties applied to them to allow better styling. In addition to the normal properties, a new .ns-modal class is applied.

CLI

The CLI team has fixed several outstanding bugs; including several webpack hmr issues. This also includes Node 13 and Android API 29 compatibility.

V8

I'm going to create this new v8 Section, because Android and soon iOS will both be using the same v8 engine. v8 was upgraded to v7.8 which adds:

  • Faster object destructuringconst {x, y} = object; This is now as fast as the normal ES5 const x = object.x, y = object.y;
  • Lazy source positioning; which can save 1 - 2.5% of memory; basically instead of storing all source code lines for when errors occur; it recalculates it when it needs it.
  • Faster RegEx
  • Several Wasm improvements

Android

The Android team kept busy this month;

  • Fixed some Kotlin issues
  • Android Signal 1 & 11 are now captured
  • Fixed a worker issue
  • Upgraded to V8 7.8 (See v8 section)

IOS

  • Upgraded to WebKit 13.2
  • Several metadata fixes
  • Fixed a worker issue
  • Fixed debugger not working issue
  • Fixed discardUncaughtJsExceptions issue

    A lot of changes to the new v8 version of the iOS runtime that is still in alpha testing to eventually replace the JSCore version... It appears the iOS v8 version is shaping up to be an awesome release.

Updating NativeScript

To get updated; you first need to do:
npm i -g nativescript@latest

To update a project; you need to do the following:

Latest Runtimes:
tns platform remove android && tns platform add android@latest
tns platform remove ios && tns platform add ios@latest

Latest Core modules:
npm r tns-core-modules --save
npm i tns-core-modules@latest --save

To install Webpack support:
npm i nativescript-dev-webpack@latest --save-dev

To install latest NativeScript Angular plugin
npm i nativescript-angular@latest --save
You will then need to install the actual angular bits; which as of this post v6 is currently supported.

The addition of t additional analytics/tracking to the CLI reminded me; you can disable it permanently; if you value your privacy by doing:
tns usage-reporting disable && tns error-reporting disable

ProPlugins - end of year update

We started the ProPlugins project pretty close to the launch of NS (NativeScript) 6.0. So it has only been active for about five months now. So how is it doing???

Well, first let cover the reason why we created it. This plays in strongly into how well ProPlugins is doing.

Why ProPlugins?

For most of us plugin developers who have worked in the NativeScript eco-system for a while. We all knew that NS 6.0 had a range of breaking changes; which meant it was the fifth set of breaking changes in less than 4 years for plugins. Each one of the NS major releases, had a range of breaking changes that required certain types of plugins to make fixes. Sometimes the changes were minor, other times the changes actually requires some major rewrites and/or major time debugging why something was no longer working properly. Of course the vast majority of this work was done by the original author; with very little or no help from the rest of the community. I recommend you read all three of my prior blog posts on the plugin costs and the plugin problem. And then why we believe ProPlugins is the solution. With those posts you will have a pretty good idea that the actual costs to an author, the community involvement, and how the open source seems pretty much unsustainable.

In addition to our experience, their are lots of blog posts by many other authors, on how for the most part, the majority of projects that are open source by independent developers has a serious funding and/or burnout issues. We have seen the exact same thing in the NativeScript plugin community way too many times. in fact, the other day I responded to a issue saying that I believe a specific plugin was still maintained as this author has been around for a while -- and then several people pointed me to the issue where the author has mentioned it is no longer maintained. Sadly another excellent plugin author appears to have left the community.

Community Involvement

When we FORKED all these plugins into ProPlugins, the original version was left to see how much the community would be actually willing to maintain them. My stance has been that I didn't believe the community would change any; but I have been 100% committed to allowing the community to prove me wrong and send in any PR's and I would publish the new version fairly quickly.

Sadly, my prior blogs posts on the problem, which tells the sad tale of how many PR's per plugin is received from the community before ProPlugins was created. The communities involvement still has not changed, and as such we have still only seen less than a handful of PR's for all 40 of the plugins we forked into ProPlugins. Meaning the vast majority of all these plugins probably don't work with NativeScript 6.x. This doesn't surprise me, since I had seen this lack of involvement for the last 4 years of plugin development...

The most interesting thing, is that this experiment really could have gone a couple ways. I could have been proven wrong and the community would have finally stepped up and worked on fixing all the plugins for NS 6 support. Or even the worst possible outcome, nobody could have joined, and also nobody did anything to fix any plugins. However, it seems the community has decided that joining ProPlugins as the best option to move forward. It eliminates them from having any additional time commitments, and helps us create sustainable source code.

So how has has ProPlugins done?

We grew to have 40 fully maintained plugins, which have had around 30 PR's for issues introduced in the NS 6.0 upgrade. We have done several PR's for issues people found in the last couple months to several of the plugins. We have also added several new features to multiple plugins. Finally, recently about 10 PR's for different plugins to fix the breaking change that was accidentally introduced in NS 6.2. So adding and maintenance of existing plugins seems to be off to a very successful start.

In addition, this new year we have several new plugins that we have plans to add. Even better, we have more authors lined up for joining the ProPlugins program. So by the end on 2020; we hope we will have around a hundred fully maintained top quality plugins in the program.

Despite the lack of any recent advertising and really my total lack of my focus on it for the last couple months; ProPlugins has each month done considerably better than the prior month. So our subscription revenue has been increasing faster than I had planned.

Since 80% of all revenue goes to the authors, the remainder goes to the actual expenses like paypal fees, advertising, servers, etc. We believe, the ProPlugins experiment has successfully started the process to having sustainable source code in our community, completely from the community. So a big congrats to the community for helping fund this!

We hope you all have a merry Christmas, and a happy New Year. We look forward to even more of you becoming part of the ProPlugins project and helping us fund sustainable open source for the NativeScript community.

NativeScript Offline Plugins

There are cases where you might want to package plugins with your app in your version control; like for example you are using a commercial plugin or maybe a proplugin and need to use cloud building.

The simplest setup that I have personally used is in the very root of your application.


You create a folder called plugins. Inside this folder you put any commercial and plugins you need to keep with the apps source. For example one of my projects looks like the above picture, I have 5 plugins that are actually in version control with the application. So anyone checking out the source code from git will get all these plugins with the application.

To use these plugins in your NativeScript app, in the root directory you type "tns plugin add ./plugins/nativescript-compress-0.0.1.tgz" you need to type the ENTIRE name including the extension.

How do I download them

So say I want to have the proplugins/nativescript-dialog plugin.

npm pack @proplugins/nativescript-dialog
will download the latest .tgz file into the current directory for you.