The Thinking Behind Codidact’s Abilities System
Recently, the Codidact team announced that an entirely new Abilities system is in development; this is the system that will allow people to earn new privileges and perform trusted actions within the system.
Now that that’s finally been officially announced — instead of just cryptic Tweets — I thought I’d get a bit more into the reasoning and thoughts behind this. Buckle up…
When dealing with any sort of online community, you’re going to come across the question of barriers to entry. At a certain level, you generally want to open up your site to as many people as possible; on the other hand, you need to prevent spammers and trolls from being able to wreck things.
It’s not only spammers and trolls you need to think about, either. It’s not a good idea to place hammers into the hands of people who don’t know how to use them, even if those people are well-intentioned; they’ll just wind up smacking themselves and the people around them. You need to first teach them how to use the hammer, and then let them swing it.
Different platforms have different ways of addressing this issue. Some work purely via rate limits (only allowing a certain number of actions within a set amount of time); others via age of the account; others via post scores.
The problem with Stack’s approach
Stack Exchange — which Codidact largely grew out of — awards various privileges based on your reputation score, which is calculated largely by votes on your posts.
This… is a bit problematic, in a couple different ways.
First off, reputation itself is wearing way too many hats. It’s simultaneously supposed to be acting as:
…a general indication of a user’s proven expertise on the site’s subject matter. (More simply put: It’s supposed to give you a rough idea of how much that user knows about whatever that specific site is about. A high reputation score on a site such as Math shows that you probably know what you’re talking about with math-related stuff.)
…a general indication of a user’s familiarity with the site mechanics. This is why reputation unlocks privileges, and why users who have 200 reputation on another site are automatically granted 100 reputation when they join another site on the network.
…a gamification incentive. Also known as “Pointz”. People love earning points for their contribution; reputation serves as a gamification incentive by awarding people points when they get votes. (See also: Reddit karma; Imgur Internet Points.)
As a result of trying to do all these things at once, it winds up doing all of them… less than ideally.
Anyone who’s paid any attention to Stack Overflow Meta will notice that there are users with thousands of reputation points who have absolutely no idea how site mechanics or policies work. Sure, they’re probably good coders, but they don’t know how to edit a post or what the heck a “review queue” might be.
You can also earn up to 1,000 reputation by suggesting edits on a site. I’ve personally earned a bunch of reputation across various sites that I know nothing about, just by suggesting edits… but if I decide to post, my reputation score might lead people to think I’m more familiar with the subject than I actually am.
It works for gamification pretty well, I suppose. Not perfectly, but good enough… and getting into that is really off-topic for this article.
Circling back up to what I just said, I want to emphasize the important part: “…there are users with thousands of reputation points who have absolutely no idea how site mechanics or policies work.”
That is the problem… because reputation points give privileges.
As users with thousands of reputation points, they have access to moderation tools such as voting to close posts, voting to delete posts, unilateral edits, and more… because they proved they know how to answer questions.
They don’t have those privileges because they’ve showed they know the rules about when to vote to delete answers. A lot of them have no idea what the policies for deleting posts are, or how the privilege (that they have) works. They earned reputation through answering questions, and suddenly have the ability to vote to delete posts.
There’s another way
So now we’ve identified the problem: Users earning privileges through actions that are entirely unrelated to the privilege itself.
And now we have an obvious solution.
Users should earn abilities and privileges through actions that are directly related to the ability they are earning.
Of course, that’s easier said than done. But at its core, that’s the guiding thought behind the Abilities system.
You want to be able to edit posts unilaterally? Great! First prove that you know what you’re doing with the editing system by getting a certain number of suggested edits approved.
You want to post a bunch of questions? Alright; but we’re going to rate-limit you a bit, just to make sure you’ve gotten the hang of this whole asking thing before letting you loose.
You want to close posts? Perfect, but first we’re going to need you to prove that you know when posts should be closed by working through flags first.
And so it goes; for each ability, you need to first prove that you know how to use a lower-powered version of it (and a couple have a small post requirement as well). Show that you know what you’re doing before you get handed the keys.
There are other factors to consider, as well; how do you effectively rate limit people without causing frustration? How hard is it to spin up a few sockpuppet accounts and have them all vote to delete a post? How far back in time should you go to calculate earning an ability?
We’re not going to get everything right immediately; we’re going to need to tweak thresholds, rework things as we see how they get used in practice, and maybe we’ll find that there are other things we actually do need to take into consideration. And of course, individual communities of varying sizes will want to adjust the thresholds for their specific needs. But we’ll keep working on it, and try to get to the point where everything works smoothly.
Abilities has the potential to allow Codidact to really set itself apart from Stack Exchange and other Q&A platforms, and I’m excited to see and help with all the work being done on it (especially by luap42). Here’s hoping we can go live with it soon!
