How do I know if a node.js library will work in my add-on?

Hi Mozillans,

I’m trying to convert a “jwk” formatted public-key into a PEM formatted public-key. I found a node.js library that does it(jwk-to-pem), installed it, but some of its many dependencies don’t work. And I suspect they will never work in jpm.

Specifically I get this error message after ‘jpm run’.

Message: Module util is not found at resource://gre/modules/commonjs/util.js
Stack:
@resource://hoba/node_modules/inherits/inherits.js:1:18
@resource://hoba/node_modules/asn1.js/lib/asn1/api.js:2:16
@resource://hoba/node_modules/asn1.js/lib/asn1.js:5:15
@resource://hoba/node_modules/jwk-to-pem/src/ec.js:3:12
@resource://hoba/node_modules/jwk-to-pem/src/jwk-to-pem.js:3:10
@resource://hoba/index.js:8:16
run@resource://gre/modules/commonjs/sdk/addon/runner.js:147:19
startup/</<@resource://gre/modules/commonjs/sdk/addon/runner.js:87:9
Handler.prototype.process@resource://gre/modules/Promise-backend.js:934:23
this.PromiseWalker.walkerLoop@resource://gre/modules/Promise-backend.js:813:7
this.PromiseWalker.scheduleWalkerLoop/<@resource://gre/modules/Promise-backend.js:747:1

I tried “npm install util --save” but that didn’t help. I don’t know what it wants.

Can I use any node-js libraries when developing in jpm? If so, which ones?

This is a recurring problem I have with doing add-on development. I just don’t know which libraries I can use. Is there an exhaustive list of libraries that I can look at that I know will work?

Or maybe I’m just an idiot and I can easily solve this error message :wink: Any help is appreciated.

Thanks,
Andrew

If you can successfully browserify it, then you can use it.

http://browserify.org/

The stuff that fails browserify can be be used but you have to get into the internals and patch up the stuff that doesnt work. Like port the c code to js-ctypes etc.

Sorry, but I’m confused again.

If I understand Browserify correctly, it will convert node.js packages so that I can use them in web pages, and have the JS code evaluate in a browser’s interpreter. So it converts the module.exports list into objects which will enter into the global namespace of a program when included with an HTML statement().

That’s all well and good, but how does that help me with Add-on development? If I’m developing my add-on using jpm I use the node.js syntax like var foo = require(“foo”) to include shared libraries in my Add-on. I can’t use an HTML script statement in my index.js.

What am I missing from my understanding?

Thanks,
Andrew

It just lets you know if everything could be converted. During the conversion if it hits anything that can’t be used in a browser it lets you know/errors.

Once it passes then you know you can use that module in as you would any other npm module.

For example this a npm module made for jpm addons - https://www.npmjs.com/package/jetpack-panelview

Gtihub is here - https://github.com/freaktechnik/jetpack-panelview and a guide on how to use npm module is here - https://github.com/freaktechnik/jetpack-panelview/blob/master/docs/panelview.md

You can’t use any module that relies on the native node modules (or one of its deps relies on them). You could in theory mock those modules with some loader magic.

1 Like

Thanks for your answers. But if I can be frank this doesn’t really establish anything concrete. I could have a module that worked when I first imported it, but then didn’t work after an update. Since node.js maintainers have no way of knowing when they’re updating their code that I’m using it with jpm.

The only answer I’m hearing is “test it”. If it works now, it might work in the future, or maybe it won’t. Who knows? Given the dependency-hell that is node.js, there are very few interesting libraries that even work now.

Testing via Browserify is more work than just installing via npm and testing to see if it works. So I don’t see the point of spending time on understanding Browserify.

For example, I wanted a library that supported SHA-256, but I can’t use the node.js sha package because it has a dependency on graceful-fs. Why a hashing library has a dependency on a filesystem library is beyond me, but it is what it is.

Given all this it’s a wonder why Mozilla even chose this crippled node.js method of development for their add-ons. I’m forced to use JS, but I can’t rely on “native JS” libraries because we use a node.js syntax for requiring libraries. I also can’t use node.js libraries because most of them don’t work, and even for the ones that do work we can’t guarantee that they’ll work in the future.

Am I accurately describing the situation?

What mozilla did was creating a different CommonJS (and not up to date with the latest spec) environment than node. So no, it’s not node.js and it doesn’t claim to be, it just happens to use the same base API. It doesn’t do any hard checks, yes, but it doesn’t claim to be compatible with the “engine” “nodejs”, which all packages on npm are assumed to be compatible with by default (which is not true, afaik).

It means that most npm packages aren’t usuable out of the box. However most packages under the “jetpack” keyword are usable (and built for Firefox even).

You can use things like page-worker to use “Web” libraries instead, which might be the future proof choice either way.

And because some people don’t like the “crippled” state of the dependency system they’re trying to solve it better in WebExtensions, which throw the whole CommonJS thing out of the window again.

1 Like

Hi Martin,

Thanks for your reply. I’m really language and platform agnostic. I don’t care what language or what libraries I have to use in order to get my work done, I just want something. The problem I have with the direction Mozilla has taken is that they’ve given me no option but to reinvent the wheel. There is no vast library of functionality I can draw upon to develop my application.

In 2016 this is simply inexcusable.

I can’t even draw upon a library that does basic/ancient MD5 hashing. I found this MD5 Hash Native JS Code. But I can’t use it unless I rewrite it to use module.exports{} notation.

Which I will do because I don’t have any choice, but it introduces a range of problems with maintainability. It also takes up my time, and might introduce errors.

This is why we have libraries. So that people like me don’t have to reinvent wheels.

Thanks,
Andrew

Depending on which Firefox versions you target (34+), you could use the Web Crypto API: https://github.com/diafygi/webcrypto-examples

It’s available under the ‘window’ object, but you can import it in your add-on (assuming Addon SDK here) using:
var {Cu} = require("chrome"); Cu.importGlobalProperties(['crypto']); var import = crypto.subtle.importKey({....});

You can export your keys in the PEM format, see the section “Converting the Results to PEM Format” in the following page: http://blog.engelke.com/2015/03/03/creating-x-509-certificates-with-web-crypto-and-pkijs/
where the author provides the function convertBinaryToPem
Or check this GIST which provides a similar function: https://gist.github.com/jonleighton/958841

2 Likes

Thanks for your response. I am using WebCrypto, and it’s great to have access to that.

The code for convertBinaryToPem is useful, and I’ll copy that out and use it. However, you must admit it’s a shame that I have to copy-paste someone else’s code, and not just reference a library.

Unfortunately, I cannot really use the Asn1.js, or PKI.js referenced on that same page[engelke.com]. And this is exactly what I’m talking about in my previous post. I cannot take advantage of lots of JS libraries because there is no standard way to include them in my application’s name space. I can look at how they’re implemented and re-implement them. But that’s kinda why we have libraries, so that developers don’t have to do that.

Thanks,
Andrew