The metrics arenât really just for rewards - Theyâre for measuring the effectiveness of the program. We donât have any information about current buddies and newbies but we know there are some out there, how can we tell if the newbies are completing the program enough without any data to work from?
3 Likes
Hi Yousef! 
and thank you for your reply!
Being new to this, Iâm not sure what the âprogramâ is, what is considered to be effective, successful, failure, completion etc. Probably Iâm wrong but in my mind once a ânewbieâ decides where and how he/she is going to contribute to SUMO/Mozilla OR that SUMO is not for him/her, thatâs about where the âprogramâ ends. The newbieâs final decision whether to contribute or not and where he/she feels most comfortable to do it does not depend (IMHO) only on the Buddy assigned or on the program itself. It depends on a million other parameters totally unrelated to SUMO or the Buddies program. So, what metrics show and how these metrics are assessed or evaluated maybe very subjective.
From my experience: When I first joined SUMO I felt totally lost since everything was unknown to me. I dared PM a contributor that I considered to be active on the forum (and guessing that he wouldnât be too angry with a newbie asking stupid questions) asking for some general direction. I am still grateful to that person for guiding me on my first baby steps. What I would appreciate as a newbie back then, would be a person patient enough to tolerate me asking any (stupid or not) question regarding contribution to SUMO (and/or any other for that matter) AND FRIENDLY enough to make me feel welcome (and not a nuisance). If I had that, THAT would be success of a âBuddy programâ (Luckily and thankfully I DID have that :))
In my ⌠oversimplified version⌠the program is generally successful if the number of newbies remaining as active contributors is greater than 0 
2 Likes
Itâs similar to what you say but if a newbie decides he wants to contribute, we have a buddy who mentors them for a little bit at the start. After the newbie has completed the milestones for an area of SUMO (see the Google doc), they can then be considered completed.
Rosana and myself came up with this structure, not sure if we should use it.
4 Likes
Beautiful!!!
Question: What is the participation of the various âareasâ of expertise. For example (and since AoA is my âbackgroundâ): If a newbie decided they would want to try AoA shouldnât the AoA âdepartmentâ take over the newbiesâ training? Or is a Buddy required to have knowledge/expertise of ALL SUMO areas?
As a Buddy: If a newbie wants to work on coding (that Iâm not familiar with) I would like to have a âcontactâ in the âCoding Departmentâ where I could refer the newbie to for further guidance/training. (Does it make sense? I can try to explain better if Iâm not very clear).
1 Like
Yeah, the areas of participation are the Forums, AoA, KB and l10n. Ideally, we would be able to pair up a newbie who wants to try AoA with someone who contributes to AoA but from our experience, it wasnât always possible.- How much knowledge a buddy needs to have is probably a different discussion.
Yup, if they are interested in a different area than your expertise then you can ask a buddy in that area to take that newbie. If they want to work on something not related to SUMO (it happens) then you can link them to the contribute page for that project.
2 Likes
Thank you for your patience! You have covered a great part of my ignorance on how things work (or how they should work :))
My only concern is to avoid generating more work for ourselves as volunteers because we run the risk of either not following up on that or cheating hours away from our non-volunteer life which may not be available.
Thanks again!
1 Like
Is there any study/research/literature available on the quality of work of newbies who have graduated from the Buddies program Vs those you have gone directly to contributing?
The target of the question is this: Other than the smoother joining of a newbie willing to contribute, do we have evidence that the program is actually beneficial to SUMO? (with regards to quality of contribution?)
Follow up question: If the benefit to SUMO from newbies completing the program is real and evident would it make sense that ALL new contributors should go through the Buddies program before starting to contribute?
(You didnât think I would go away, did you? :P)
1 Like
Hi CAKCy,
We got some awesome contributors in KB,l10n and Forum via Buddy program.
SUMO Buddy program is like polishing the diamond. we are providing the right path.
When they are SIGNUP for contributor, they are redirected to Buddy Program to join and get benefits. Up to now its an optional.
we are contributors, we can join anytime and go away anytime.
2 Likes
Thank you for kicking the discussion off, everyone!
I started summarising all the great points you make in the top post, for easier navigation.
I also started a separate thread on the Newbie progress tracking: https://discourse.mozilla-community.org/t/discuss-documenting-newbie-progress/726
Take a look and let me know what you think
2 Likes
Hi vesper!
I really, really donât want to start an argument with you (especially after knowing that you are a âbig guyâ and all :P) but:
Are we ready to accept that poor contributions are OK just because they are on a volunteer basis? My experience with AoA is that new contributors, no matter how well intended they are, need some guidance. I believe that only this way we can improve the quality of what SUMO has to offer.
I do recognize the danger of losing contributors that are unwilling to undergo training before contributing, but I daresay that we are better off without them.
1 Like
Hahah, youâre funny. I donât see you (or anyone else) voicing their opinion as starting an argument with me, especially when weâre brainstorming
All good points. How are new contributors learn in ways other than doing? (I know, answering questions with questions is bad manners⌠but hey, Iâm a big guy, so I can do it, right? ;-))
Keep the feedback coming!
2 Likes
FYI you canât scare me! I know Judo, Karate, Jujutsu (and other Japanese words :P)
Agreed with one ⌠twist: âDoingâ should be part of the training process and not out âin the fieldâ. What I do in AoA is give new contributors mock up tweets to respond to, so they can learn without disturbing any User or exposing SUMO.
I may be too much of a purist but⌠hey⌠we only live once!
1 Like
In order to save everyoneâs time at the forthcoming meeting, here is a synopsis of my thoughts on the subject:
The Buddies programâs main goals may be:
- To help new Contributors find their way around SUMO and how they can best contribute
- The train and guide the new Contributors so their contribution is of as good quality as possible.
If ONLY (1) above is the goal
The only thing that we need to do is to make sure that no new Contributor is left without a Buddy. There is no need to do anything else since the Newbie will (hopefully) keep asking questions, and the Buddy will (hopefully) keep answering them until the Newbie finds what he/she wants to do.
Furthermore:
- the program does not have to be obligatory for all new Contributors,
since there may be some who do not need a guide.
- we do not need to keep track of anything since the program will depend basically on the
willingness of any new Contributor to participate or not. Keeping track of progress etc. could only serve statistics IF there is a serious follow up of the quality of contribution of trained Vs untrained contributors.
If (2) above is ALSO a goal
If this is the case, then I donât see how we will choose NOT to make the program obligatory for ALL new Contributors. How can we decide to train only those Newbies that decide to register with the program and allow the rest to contribute irrespective of their contribution quality?
So, IMHO, the meeting should first decide what the actual goals are. Any further action should be based on those goals. A âmixâ of the above (we only train SOME of the new contributors; therefore, we need to keep track of their training etc.) does not make sense. âChasingâ untrained newbies in the fields and putting out fires, instead of developing a fireproof system, is counterproductive and more time-consuming in the long run.
Thank you for your time!
1 Like
Very well formulated post, CAKCy. Iâm adding the possible goals to the top post and letâs keep discussing what they should be.
I think that even if only (1) is the goal, tracking progress of new contributors is important to see whether the program is actually making a difference. Additionally, it could help us learn whatâs a realistic and comfortable level of contribution for the average SUMO member (even though maybe that last point is not really that relevant).
I hope @madalina chimes in and we can hear more from her
For the sake of making the meeting as productive as possible:
I wonder if answers to both of your questions:
- Does the program make a difference?
- What is a realistic and comfortable level of contribution for the average SUMO member?
can be extracted from the previous run of Buddies. From what Iâve seen there was data recorded for each Newbie registration.
Thinking out loud:
I think that to get the answer to the first question we need to compare the performance of trained with that of untrained contributors. Are we in a position to do that with the past data available?
To get the answer to the second question we need to do a follow up of Newbies trained already in the Buddies program and see what is their contribution level today (after they have graduated from Buddies). Is this data available?
Iâm not opposing collecting fresh data but it wouldnât make much sense to start collecting new data if we are not in a position to utilize the data collected in the past.
Hi folks,
Here are some thoughts:
- I think one important thing to keep in mind is that the goal of the Buddy program is contributor retention
We have a lot of people who contact us because they want to get involved, more than we can actually handle at the moment, but the biggest problem is that more than 90% of these people drop out after a making a contribution or two.
The main reason for that is that it is hard in the beginning to get around so people get discouraged and leave.
So when we think of tracking the impact of the Buddy program we need to look at the number of contributors that are still contributing after âxâ amount of time and basically calculate the retention rate. If we can also look at contributors who make quality contributions after x amount of time, thatâs even better.
2 I agree with CACKy that we need to look at previous data, unfortunately we do not have a lot of previous data as the program only ran for a very short amount of time and it was never really official. So we might need to run a bit in the dark at the beginning until we have real data we can count on. We did a bit of tracking in the past and I know at least two contributors who really stepped up after being onboarded through the Buddy program but we have very little data.
It might be useful to go through what we have and do a rough estimation though.
2 Likes
I agree with you @madasan on the retention part of the program. Once we get the buddy program off the ground, we need to make sure we have some type of retention program in place. I hate to see people start dropping of the face of the earth, because we didnât have some type of retention program in place. Thats my take of things, but I like everyoneâs ideas including @vesper and @CAKCy. You guys rock!
Hello Madalina,
Question: Where/how these people contact you/us? Are these contacts on a private basis? Are those interested redirected to the SUMO fora?
With all due respect I disagree with your statement that the goal of the Buddy Program is contributor retention. If this is actually the goal as defined by SUMO Management then Iâm in the wrong program! 
The problem of retaining contributors cannot be limited to, and cannot be solved by the Buddy Program (BP) alone. BP has a role to play (to make the entry of a new contributor easier) but that is only the first step. I think that the current ârewardsâ and ârecognitionâ system within SUMO has to be re-evaluated and re-built from scratch. It is a different discussion altogether but itâs one closely related to the evolution of a âNewbieâ to a âfully grown contributorâ to a âBuddyâ and the longer we postpone it the longer SUMO will suffer unnecessary losses of contributors. One cannot expect the Buddy Program to be effective as a stand alone program loosely connected with the rest of SUMO reality.
Understood!
What I can share is the experience I gained running the AoA (much smaller) version of the Buddy Program. This, of course, is based on my own observations and there is no statistical data recorded. 
This experience may have an answer to some questions like:
- What are the reasons that drive a person to joining SUMO (or any other help community)?
- What do new contributors feel they have to offer when they join SUMO?
- What do new contributors expect to gain when they join SUMO?
- How is involvement with SUMO going to affect new contributors personally (short term/long term)
- What are the current standards within SUMO (as those are demonstrated by current/older contributors)
- How is the contribution of a new contributor evaluated and what are the rewards if any?
- How is a new contributor treated by the current contributors/paid personnel?
etc.
1 Like
This is the âwhole SUMOâ approach (and IMHO the best approach towards raising retention and quality):
1 Like
CAKCy, awesomeâŚ