Incentives in an online Q&A community

Imagine some student from around the world contacts you to get information about TSE programs. She is looking to apply for a Master in Economics, but the website is not fully informative (they have clearly no incentives to provide the cons). Are you going to reply? If so, how long will your message be? Would your choice be affected if you shared the same nationality and language?

Many Question & Answering (Q&A) online communities have appeared on the Internet, exploiting three efficiency arguments rising from the above story: first of all, if you put some effort into answering the prospective master student, then maybe others could also benefit from the answer (making your effort more valuable). Second, while you probably do not have any incentives to answer other than for pure altruism, the Q&A system can put in place incentives not only to push for contributions, but also for quality. Here the third advantage comes into play: while in bilateral communication, messages are private, but in a community everything is observable, allowing for the creation of mutual monitoring devices.

The Q&A websites collected under the Stack Exchange label are probably the best of their kind in exploiting these features and they are growing very fast. Stack Overflow, the Q&A website focused on programming language related topics, receives 9.6 million visits per day[1], 74K new questions per week and currently, counts 15 million question and 24 million answers since its launch in 2008. The other websites of the family are less popular, but still not negligible. For instance, Ask Ubuntu, which focuses on the Ubuntu (Linux) OS, receives 587K visits per day, while the Q&A on English Language receives 467K daily visits.

The incentive structure in place is very sophisticated:  they exploit two incentive devices, badges and reputation points. The first are sort of medals, rewards that are delivered to community members when they accomplish a predetermined task, such as publishing a given number of answers. There are 94 of those, each specifying different “missions” that are more or less difficult to achieve. Reputation points are instead of the principal mechanism, and are awarded, for instance, through up-votes that community members allocate to your questions and answers. If users care about accumulating them, the system succeeds in incentivizing both posting content and making more effort for higher quality: the rest of the community has to find your post useful to up-vote it and they can also punish you with down-votes.

What has all of this got to do with Economics? A wide literature develops models of communication and information transmission. Motivated by the problems caused by asymmetric information, researchers investigated how different incentive structures can lead to a variety of outcomes. Communication may occur or break down, the message may be informative (i.e. internalized in the action of the receiver), or leave the receiver indifferent.

PhD Article - Source - SMBC by Zach WeinersmithThese questions are of interest for many applications. My interest is to study them within the context of organizations, since information sharing between employees and between collaborative production units is often essential[2].

In Stack Exchange, information sharing is at the heart of the organisation. . Even if it looks very different from our usual idea of a firm, we do observe a group of people producing a service, i.e. providing answers. This product not only benefits the community that produces it, but also a far larger group of Internet users that have the same questions asked by community members. Stack Overflow has 8.4M registered users which are likely not sustaining 9.6M visits per day. A significant share of these visits come from unregistered users. The latter group differs from the registered users, as they are completely extraneous to the system, and are often unaware of using Stack Overflow, by getting to the answers directly from the search engine[3].

As all organisations, Stack Exchange has a power structure (i.e. a specific distribution of authority), but it is very different  compared to others. To give an example, Facebook is also an online community that creates content, sharing many features with Stack Exchange such as self-monitoring mechanisms. Nevertheless, in Facebook the authority is in the hands of the CEO or his delegates, who have the power to make the final decision. Users can propose the elimination of posts but can’t directly delete them.

Stack Exchange is rather a community-managed system. Even if it is formally a company and has a CEO (Jeal Spolsky, also co-founder), authority is released to community members based on the amount of reputation points accumulated with their activity.

More specifically, there are 26 “privileges” awarded sequentially once the users reach a certain level of reputation. They span from allowing more flexibility on posting behaviour to access to moderation tools and so on.

It took me some time, but finally here is the initial research question of this project. Do community members care about achieving these privileges? If so, do they adjust their amount of activity to obtain the privileges faster?

There are two possible hypotheses that would justify positive answers. The first refers to what is denoted “gamification”[4] of the system. In other words, community members see the higher privilege as a target and they “get fun” by obtaining a high enough reputation level to get it.[5] An alternative possibility is that gaining authority within the organisation incentivizes users: if I am approaching the “moderator” level of authority and that role gives me utility for the powers that comes with it, I will then try to get it as soon as possible.

1_5dOUhQl1ati7FpCfQehhrQ

I used data from the Stack Exchange website denoted “Ask Different” which concerns questions and answers over Apple products. The carried analysis is for now very preliminary. Identification is based on the assumption that, given their average rate of posting, each user may or may not change her activity level (posting of questions or answers) when approaching the achievement of the privilege.

In practice it consists of a linear regression with discontinuities. It relates the accumulated amount of reputation to the amount of activity and checks for significant changes of activity for reputation points levels close enough to the privilege thresholds.

In the preliminary results, out of 7748 users, 2330 are significantly sensitive to the privilege thresholds, which means that, for some reason, they modified their usual pattern of posting by increasing or decreasing the amount of activity. Sensitive users are heterogeneous, in the sense that some appeared to have positive change in activity while others have negative changes. The projects will then aim to measure the impact of incentives on the quality of the content.

To conclude, incentive mechanisms that push people for more and more voluntary online contributions are becoming increasingly popular. If they allow the existence of open-source projects, they also allow many for-profit firms to boost their revenues, using review systems, for instance. And we start now to see the first “fun-facts”, as soldiers using the Strava running app disclosed the secret locations of military bases. Was it the search of prestige or just negligence?

By Jacopo Bregolin

 

[1]     Median number of visit between January 21 and February 4, 2018, source: https://stackexchange.com/sites?view=list#traffic . The comparable statistics reported afterward have the same source.

[2]     An example is provided by a report of the RAND Project Air Force (“Facilitating Information Sharing Across the International Space Community”, K. M. Keller et al. 2013) that pushes for more information sharing providing suggestion on how to improve it.

[3]     96.9% of unregistered users visit only the question the are interested in, without going through the home page or ather sections of the website. Source: “Does Anyone Actually Visit Stack Overflow’s Home Page?”, D. Robinson, 2017, Stack Overflow Blog.

[4]     Here the word “game” does not refer to Game Theory, but rather to computer games or other games in the more general sense.

[5]     “What Motivates Wikipedians?”, O. Nov, 2007, Vol. 50, N.11, Communications of the ACM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s