Regional Ambassadors Evaluation

Regional Ambassadors Evaluation #RegionalAmbassadors

This evaluation plan will help us determine whether the NetSquared Regional Ambassadors pilot is successful and whether it should be continued and expanded to all regions.

Primary Goal

Building the organizer’s ‘community’ and creating strong meetups and events. We care less about the size of the program and more about the quality of the programming and interactions.

However, we acknowledge that measuring and evaluating “quality” is difficult. Therefore we will use quantitive proxy measures that will indicate whether we are succeeding at our qualitative goal:

  1. # of active organizers (held at least one event in last six months)
    1. Logic being that engaged organizers keep holding events on a regular basis.
  2. Average number of event attendees at beginning and end of pilot
    1. Logic being that the more attendees, the more successful the local group.

Our primary activities

  1. Increase the number of interactions between organizers by email, phone and chat with monthly outreach to Ambassadors to regional organizers
  2. Get groups of organizers face-to-face with the help of Net2 Camps and travel bursaries for nonprofit tech events
  3. Offer support to Ambassadors with quarterly calls and group training meeting with a focus on mentorship skills

Theory of Change

Net2 organizers increase their commitment to the network and hold more events when they receive more one-on-one contact from other organizers and TSG staff.

Furthermore, we believe that face-to-face contact is more valuable than online contact. So while email, phone, video chat, etc. will be the primary outreach method (it’s cheap and scales!) we will strive to provide opportunities for Net2 organizers to have in-person meetings.

Finally, we believe that regional ambassadors, who may share a language, culture, time zone and have knowledge about local events and contacts will be more effective offering support than a single NetSquared Community Manager in Canada. 🙂

Connecting the dots

  • We want more event frequency from our organizers, therefore we will reach out to them personally more often because we believe this will increase engagement. We will know it’s working by measuring # of regional events at the beginning and end of the pilot.
  • We want have more face-to-face interactions between our organizers, therefore we will offer travel bursaries which will help bring organizers to regional camps. We will know this by counting the number of in-person meetings with Net2 organizers during pilot.

What do we want to learn?

  1. How do we help organizers hold better (no definition), more frequent events?
  2. Will Regional Ambassadors increase organizer retention and network growth?
  3. How can we get Net2 organizers face-to-face in a cost-effective manner?
  4. What do Ambassadors need to succeed in mentoring organizers in their region?

What will success look like?

  • Organizers are holding events more frequently
  • More touch points between organizers, both face-to-face and online
  • New organizers have been recruited and are holding events
  • New meetup participants are recruited (joined meetup groups)
  • Our Net2 organizer support process is better documented and provides both frequent and meaningful support and interactions
  • Our orientation process is better documented and supports new organizers

How we’ll evaluate

We will collect qualitative and quantitive data before, during and after the pilot.

Beginning

  • Count number of events held and number of active organizers in the previous 10 months for each region.
  • Survey all Net2 organizers to determine their level of engagement and perceived support from Net2

Ongoing

  • Survey Net2 Ambassadors monthly to learn if and how they are reaching out to their regions, how they are doing in the role, and what additional support is necessary.

End

  • Survey all Net2 organizers to determine their level of engagement and perceived support from Net2
  • Count number of events held and number of active organizers during 10 months of the pilot in each region and compare against previous year
  • Interview all Ambassadors individually to get their feedback on the pilot

Evaluation Challenges

  • the data set is quite small
  • there is organizer turn-over, so the regional set will change from the beginning to end of the pilot
  • The “regions” are very large during the pilot. “Africa” as a region is a bit ridiculous… it’s HUGE!

Evaluation sketch from Brian Hoessler
Screen Shot 2013-09-10 at 10.36.42 AM.png