Review process for a big application?

Add-ons addons.mozilla.org
#1

Hey all,

Wanted to discuss an idea; we’ve been trying to get our firefox addon into the store for a pretty long time now (our first attempt was in april of 2015.) We just resubmitted again, and in order to make the process go more easily, we’ve uploaded a version with all of the sources un-minified by our build process and have included a spreadsheet with references to all of the sources used, so fingers crossed.

However, it seems to me like larger addons like ours (which are extensions of larger products) are pretty prone to these sorts of issues, and the feedback cycle takes a really long time to work through. Has there been any consideration for a different review process for larger addons, especially ones that are connected with a business?

Some of the requirements aren’t practical for us; we want to be able to uglify all of the sources in our build pipeline, but that’s an insta-fail because the sources would then be obfuscated and not match your checksums. We’re also a security company, so we take extra strides to make sure data is being handled properly—if we were truly a risk we’d not need to have a modified 3rd party library to gain access to sensitive data.

I can imagine the work needed to verify our gigantic addon is a lot more than the usual addon, and it’s also valuable for us to have this live, so I’d even be willing to pay for a ‘addon buddy’ to help us work through the submission and get it ready to go live. I could also see something of an interview type process to quickly get the review team acclimated with the addon to make review easier.

To be clear I’m not suggesting that this is purchasing an approval, but something to justify the extra cost we’re putting on the system to review a super large addon.

Would be very curious to hear if that option has been discussed before—it seems totally fair from my perspective, and I’d love to get our addon up before the 1 year mark rolls around :smiley:.

#2

One of our goals is to ensure that the add-ons we list are safe to use for our users. There’s a broad audience that trusts us and any security or stability problem that comes up from an add-on on our site erodes that trust.

Large and complex add-ons like yours are nearly impossible to completely code-review, which puts us in an awkward situation where we either need to reject them or relax our requirements and do a “good enough” review. Either way, it can take a long time because only admin reviewers handle such cases. We are constrained by the amount of time we dedicate to admin reviews, and I don’t think that paying us would help in that regard. We’re already scaling up our paid reviewer time, and what sets us back in reviewing has more to do with the amount of time given by volunteers, and other add-on tasks we have to work on.

It’s usually easier in such circumstances to go unlisted and just distribute the add-on yourselves, because that gives you much more control over the process.

#3

Thanks for the response Jorge—I definitely understand that puts the review team in a tough situation.

Isn’t it the case that unapproved add-ons can’t be distributed ourselves? We are hosting the XPI for download, but newer versions of firefox throw an error and don’t allow people to install unless they jump through some hoops.

Is there a way to get approved / signed so we can distribute ourselves? Is that a different type of review?

#4

For details about the different distribution channels, see https://developer.mozilla.org/en-US/Add-ons/Distribution

#5

For self distribution, add-ons are automatically signed. There are some hoops users have to jump through in order to install from sites other than AMO, but It’s probably a smaller effort than the one required to get through review given the situation.

#6

Ok, got it. It looks like in order to get jpm sign to work I’ll need to bail 100% on getting into the AMO, correct? Looks like being a listed extension and signing ourselves are incompatible.

If so, I think it make sense to give this last round of submission our best try before giving up. Definitely think getting in AMO is best for users, so willing to spend the extra effort on our end.

#7

Our add-on products are all Free Software (open source), but we sometimes find it very helpful to minify the codes to optimize certain use cases of an add-on.
Especially, when we have to store thousands of add-on copies on a server to run certain tasks, or when we have to deliver/install/launch/uninstall an add-on very quickly behind user interfaces / APIs (e.g. online services that use Firefox on back ends, or a batch test cases runner). In those cases, doing aggressive minification on JavaScript codes can cut down latency by up to 70% and/or improve throughput by at most 80%. And we as a transparent developer want to use the same builds of an add-on everywhere (also to reduce development cost).
In contrast, although we use no 3rd-party code libraries not included in Mozilla products natively, all of the add-ons’ source codes are/should be available on public online code repositories. So we want no secrecy. Is there any plans to improve situations for a case like this?

Additionally, we are planning to introduce code generation to implement complex extension logic in a secure and reliable way. In this case most of source code files will be in languages other than JavaScript (with reasons). Will AMO reviewers handle non-JavaScript reviews? Imagine for example Haskell or Rust.

#8

Since add-ons are usually distributed via XPI files, minification doesn’t make a significant difference when moving them around. It’s also less error-prone than moving individual code files.

Add-ons that have special needs will continue to be reviewed to our best abilities, but you shouldn’t expect them to be reviewed expediently.