Optimizing the review queue?

Add-ons addons.mozilla.org
#1

I have read some of the posts here and have seen the same problems again and again.
Many extensions must wait very long to release.
I have the same problem. In all other Browsers the review was after a maximum of 3 Days over.

Perhaps this could be accelerated for some extensions.

For a better understanding first the situation as it is momentarily

  • I have uploaded a very simple extension 9 days ago
  • Currently it is in the queue on place 237 of 273
  • The extension is so simple they could be checked in 5 minutes but is not noticed and must wait until other complex extensions are first checked

My suggestion is that you make two queues.
1 queue for more complex extensions and 1 for simple.

To make the difference easier to recognize some things should be examined and the procedure changed.

The first thing you could do is make a small checklist when the developer submitting the extension.
For example:

  • Use your extension external scripts or images?
  • Use your extension libraries?
  • Use your extension minified files?
  • ….And so on.

If something is wrong you can inform the developer immediately. So it does not have to be checked.

When the extension is uploaded then you can check how big the files are.
This can be done fully automatically.
How much kb has the whole expansion minus all image files.

Now it only has to be determined from which size an expansion belongs to the fast queue.
Because less code is faster to check.

If certain colleagues only take care of fast queues the throughput of the reviewed extensions should be increased. If they have checked everything they can help with the slow queue until a new extension for the fast queue is submitted.

This is now only a concept from what I have read here.
You can do this better because you know the internal processes.

What do you think of the suggestion?

1 Like
#2

Thank you for the suggestions. We have considered similar possibilities in the past, and there are a few signals in the queue that reviewers can use to determine if an add-on is easy or hard to review. In fact, most volunteer reviewers spend time looking at the bottom of the queues for add-ons that can be easily reviewed.

We’re making some big changes in the review process in the coming months. We’re meeting next week to talk about the specifics, so we should have a more concrete plan to share soon.

#3

Greetings. I’d like to expand upon this idea. We submitted an extension and it was rejected (not a wide enough audience???). We responded with supporting information and included notes to the reviewer, but after we requested a review request, the extension was placed back at the bottom of the queue. It would make sense that a same extensions that is responded to (and code not modified) and resubmitted for review be higher in the queue since there was already the wait period for the initial review.

#4

Perhaps our biggest request would be that you don’t lose your position in the queue if you publish a new version. We are making updates faster than the review process completes and can’t submit updates until our old version is reviewed even though we have a new version ready to go.

3 Likes
#5

In my experience, I didn’t loose my position with new versions upload.

Published an new extension on 6 Feb (still waiting for the review… grrr… even it’s a simple webextension one), and upload 4 new versions since without any position loose (now 84 on 300).

#6

I wish that were the case with me c2covh. I submitted an extension on 1/18. It was rejected based on not having a wide enough audience (??). The Add-on passed all validations upon initial submission. I generated supporting evidence to warrant a public listing. There was no code change. I submitted the evidence and requested review. My exact same add-on with no code change got pushed to the end of the queue. I’m in the low 200s out of 300+. I would think that add-ons that have already sat through the queue would get a higher position upon re-submission especially if there was no change to the code. Not the people, but the process. I know the reviewers have their hands full and do a great job.

#7

I was in position 64-ish of 200+, uploaded new version today and got moved to the end of the queue again.

1 Like