Measuring Online Community Engagement

logo
Having healthy levels of engagement is a hard nut to crack for any online community. Take note of the much-discussed 90-9-1 rule. Also called the 1% rule, or “Participation Inequality”, a term coined by Jakob Nielson, which refers to the imbalance of media creation in any online group: 90% are lurkers, 9% are editors, 1% are contributors. This concept/theory has also been similarly validated in real-world scenarios, ala the Pareto Principle. In relative terms the Pareto Principle claims an 80/20 split: 20% of the group members will contribute 80% of the content. Most community managers will say that the holy grail of engagement within a community is UGC, or User-Generated Content. If these paradigms are true, then it would seem driving higher and higher levels of engagement may be analogous to the futile uphill efforts of Sisyphus.

So if you’re kicking off a new online community, you may be asking these sorts of questions: What metrics define engagement? How can we measure those metrics? What is a healthy level of engagement and what are the ways to get there? Let’s take a look at a systematic approach in understanding the variables of engagement and developing strategies to reach your engagement goals.


How do users engage with your site?


The first order of business is to list out all the ways in which people can engage with your site. Look at your platform (in my case, it’s Microsoft SharePoint 2010, with lots of customization), and enumerate all the so-called site actions. What are the things you can click on or do on the site that would constitute something of value to the community?
In my case we first looked at ways people can create content:

  1. Write an article
  2. Promote an event
  3. Ask a question
  4. Create a group
  5. Share a link
  6. Upload a file
  7. Posting a comment on any of the above

These seven things constitute all the ways a member can contribute content.

But members may also do things that could be considered more passive levels of engagement. In my online community site, members can get involved with these additional actions:

  1. Flagging a resource
  2. Sharing a resource
  3. Rating a resource
  4. Following a resource, person, or group
  5. Inviting others to join

When you inventory your various site activities, you may break them down in a similar way, i.e. active vs. passive engagement, or you may want something more fitting to your particular situation. Maybe your actions have more than two levels and so giving each a value is more appropriate. More on that later (see Scoring the Activities).

Now that you’ve listed out the activities, it’s time to ask “can those things be tracked and how?”. In aggregate? Or per member? You need to be sure the things you can track can seamlessly tie back to your engagement goal (see Setting a Goal).

Scoring the Activities

Once the site activities have been enumerated, and you’ve figured out what activities can be tracked, it’s time think about scoring. In terms of community engagement, does one activity carry the same weight as all the rest? Or are there tiers to engagement? That’s up to you to decide based on your site’s activities and your goals. In our case, we went with two tiers: active, and passive. Active engagement activities are creating new resources or posting a comment (the 1% folks). Passive engagement activities are the less-visible but still important ones, what I call the gateway drug activities: Following, Sharing, Flagging, Rating, Tagging. As a member becomes comfortable with these types of contributory actions, they should start to feel more comfortable with the community and seek deeper levels of contribution.

Setting a goal

Now that you know the ways users can engage on your site, what’s your goal? What do you deem a healthy level of engagement? In other words, how may actions and in what combination constitutes an engaged member? 1 site action? 2? 10? And what percentage of members ought to be healthfully engaged? 1%? 10%? 30%? At this point your initial goal will simply be your best guess based on what you know at that moment.

Establishing the Baseline

Once you’ve spelled out the inventory of activities, scored them, and figured out what you can measure and monitor effectively, it’s time to establish your baseline levels of engagement. But what’s the right frequency to measure this? Daily? Weekly? Monthly? When you’re first starting up the community, it probably makes sense to elongate the period of measure, i.e. weekly or monthly. When the community is new there are likely going to be high degrees of fluctuation with engagement, so to establish a more accurate baseline, you want to get as close to an average as possible, and that means a longer measure period. Another reason to go with a longer time frame on your measure is that of statistical significance. The more data you have in that time frame, the higher the statistical merit.

Resetting your goals

That first measure of engagement will be a tell-tale sign if you’re first guess was a good one or not, and whether or not you may need to adjust your goals.

For example, I initially considered healthy engagement to be 2 contributions per month (new resource, or a new comment), but then after seeing our initial results I brought that down to one contribution per month, while recognizing a new type of engaged user who performed one of the passive actions in any given month.

As for percent of users engaged, I initially went with a goal of 30% actively engaged members (high for many communities, but in my case, this group is highly collaborative in the real world so 30% seemed reasonable).  After our initial month of tracking, I realized we were way off: only 7% of users were actively engaged. So we reset our goal to 10%. This is still a pretty high level of engagement for any community, considering the 90-9-1 rule.

Honing strategies to improve engagement

Now that you know the baseline, how do you improve it to reach your goal? Hopefully when you developed your goal, you put together some strategies to help you get there. Hone in on your strategies and develop tactics to support them. Some strategies could revolve around education/training, outreach and support, making members feel safe/secure, starting with the easy activities (passive engagement), or showcasing success stories/case studies. In my case one of the things I knew was lacking was making new users feel safe and comfortable with their new community. So we developed a nice welcome kit (a series of emails) to string them along a new member’s journey. The key messages were about safety, comfort, and things you can do to get your feet wet. Whatever your strategies and supporting tactics are, you’ll do a bit of trial and error. For example, when you started that welcome series campaign, did it drive up engagement? At this point it’s a game of rinse and repeat or rinse, choose a new shampoo, and repeat.

Monitor

To be effective in reaching your goals you have to be diligent about monitoring your progress towards them. Once you’ve set up a frequency for monitoring, hold yourself and your team accountable by creating a report that can be distributed to key management and other community members. You might wince a little at the notion of sending out this type of report to other community members, but it can be very effective. One of the tenets of good community management is transparency, public discourse, and an open feedback process. By making this report visible to community members, you make them personally aware of their part in achieving the goal. You also may get some really good feedback about how the goal, strategies, or tactics can be improved.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s