Case study: UX research on India’s most popular programming community
I spent last month doing user research of neoG.camp. With more than 19k members, it is arguably one of the most engaged programming communities of India. This case study describes my process and conclusions.
The beginning
It all started with a UI design course that I pursued at the beginning of this year. It gave me a good headstart and got me hooked on the world of design. Later I signed up for two more courses on Design Thinking and thus I embarked on my journey of becoming a UI/UX designer. While these courses were great to get a taste of the field, but the feeling of an engaged community was missing.
I soon found out about 10kdesigners and applied to become a part of their fourth cohort. It was a bright day when I saw my mailbox shining with an invite from them. This community is what I breathe day in and day out today. I’m an active part of this brilliant and supportive group of people learning UI/UX design together. You can read more about my journey here.
Overview of the project
This project was a part of a group assignment where our task was to find an internet community and figure out how people spend time there, their behavior, usage patterns, and gather findings that can be distilled into insights.
These usage patterns and findings would help us create user journey maps. Based on which we would be able to suggest some ideas/suggestions to solve the pain points we might discover in this process. Thereby aiding the community towards a better user experience.
Our Team
We are a team of five for this project and the best I could have wished for. The research project kicked off the same day we got the assignment. Despite other work commitments, we connected every day (or should I say night) to share the updates and progress on the research for the duration of this project.
Selecting a niche
With the growth of the internet and mobile penetration, we have observed a steep rise in the number of internet communities as well. The pandemic only fuelled this growth. Without deciding on a niche, this could quickly become very overwhelming.
We started our brainstorming session with all the popular community niches out there. No domain was left, starting from WhatsApp communities, to gaming, fitness, and whatnot. Again and again, we found ourselves drawn towards communities around cohort-based courses (CBC). And that’s what we decided in the end (more on this, later).
We narrowed down our choices based on their engagement in the particular niche. I came up with the following ideas.
Why did we choose Cohort Based Courses (CBC) as our niche?
Wait… before jumping on to that, first of all, what is a CBC?
What is a CBC?
Cohort-based learning is a collaborative learning style in which a group of individuals (a cohort) advances through an educational program together. A cohort environment is positively skewed towards peer-to-peer learning in addition to mentor-led live interaction.
CBC vs MOOC
Massive Open Online Courses provide another type of online learning experience where the focus is on the individual as compared to a full cohort. They started gaining popularity around 2010 and they continue to be so in different forms. If you want to learn Java you may find a course on Udemy for $10 or even less. But the completion rate of these online courses tends to be very low. As low as 3%.
This lead to the popularity of cohort-based courses (CBCs) where a group of individuals learn together and share the same goal. The feedback loop and hands-on training in the CBCs stand them apart from the lone learning experience of the MOOCs. And are they successful? Oh yes!
Completion rate is one of the determining factors to know whether a course is successful. As an individual, it is very easy to start a course, but completion is the key. And cohort-based courses with a whopping completion rate are ensuring that they are the future of online education. AltMBA, a CBC by Seth Godin, had a completion rate of 96%.
My experience at 10kdesigners
My own experience at 10kdesigners also played a big role in choosing this niche. Before joining the cohort at 10k, I signed up for some self-paced UI/UX courses on Udemy and Coursera. Though the content was good, the learning experience was lone and a feedback loop was missing. I was able to complete these courses, but it required a lot of motivation and deliberate effort. 10k flipped the coin for me. The community made the motivation implicit and the process made it effortless.
I wanted to understand how other CBCs were building their communities and whether there was a possibility of doing better.
Our goal
- Understand how people spend time in these communities. Their behavior, usage patterns, desires, and frustrations from the community.
- Conduct user interviews and learn how to do them effectively.
- Gather insights and pain points of the users.
- Come up with ideas and digital tools that would help the community to grow stronger and make the user experience better for the members.
- Validate or disprove our general hypothesis we had for the internet community. (We start with a hypothesis that you can read in the next section)
Focus Community — neoGcamp
We looked at a number of CBC communities around and decided to focus our research around the neoGcamp community.
Hypothesis and Assumptions
Before starting our research we jolted down a few hypotheses and assumptions that we had in our minds.
Secondary research
I came to know about neoGcamp recently from a friend, who is switching his career from civil engineering to programming. I got to know the sheer size of the community. I was fascinated to see how quickly it grew to 19000+ learners. All the participants were helping and growing together. This intrigued me a lot.
About neoGcamp
Founded by Tanvi Priya and Tanay Pratap in late 2020, neoGcamp provides a platform for people from no coding background to learn to program and be industry-ready in 6 months.
It all kicked with the #TeamTanayJobChallenge which started in March 2020. One had to finish a minimum of 5 projects, 3 blogs, and 2 reviews on their projects between March to May 2020 and they would be provided with an interview opportunity through jobchallenge.dev. A good number of people completed the challenge and even during the pandemic, this challenge got 15+ people placed in various companies.
This process surfaced two issues. People lacked the skillset to complete the given challenges effectively. Then, they had to be prepared to face the job interview. neoGcamp community was conceptualized to fix these points. The jobchallenge.dev later resulted in Elevate Labs which is known as Roc8 now.
Online presence
The neoGcamp community has an impressive online presence. From Discord to Reddit and from Youtube to Instagram. Tanay himself has a sizable following on Twitter and youtube which also feeds into the social presence of neoGcamp.
While interviewing members from the community most of them told me that they saw Tanay’s videos on youtube or Instagram live and got to know about neoGcamp.
Observing the engagement on these platforms I couldn’t help but notice how Tanay engages with the audience and answers all doubts and questions. Apart from this Tanay is also active on various podcasts where he shares his advice, experience, and stories.
Discord is the main platform for users with 19000+ active users. Members use this platform to ask their doubts, update about their progress and daily interactions. They have chaupal and chit-chat channels for meet-ups and live talks.
As we can see Tanay is an integral part of neoGcamp’s online presence. This works as the first point of contact for members to begin their neoGcamp journey.
Structure of neoGcamp
Since neoGcamp was founded in 2020, the structure has changed a number of times. We can see the evolution of the community with the help of the chart below.
The camp is structured in three stages.
levelZero: A free and open level, designed for all beginners. The levelZero is an enabler for beginners to learn to program.
levelOne: A structured six-month BootCamp for hardworking programmers. The entry is open for people completing the submission and qualify the interview.
Placements: After finishing levelOne, this stage gives an opportunity to get interview calls and get equipped to crack those interviews.
Competitive analysis for various other cohort-based courses for programming.
Primary research
We decided to kick off our primary research by deciding on the questions that we needed to ask community members. Some of them working on their final project for levelZero while the submission date was approaching and others were struggling to finish different projects with their full-time jobs/college education. We didn’t want to miss this opportunity by asking the wrong questions.
After brainstorming the questions we started arranging them. The questions were sorted according to the flow, and further divided into different sections for different members of the community.
During one of the Interviews, a community member said: After attending Tanay’s class for 4 hours (sometimes 5 hours), I would start working on the assignment straight up. I was never this excited for anything, to sit and code for 4 hours together.
That is the power of a good learning experience. I can also relate to him. For example at this instance, our team started brainstorming around 9:00 pm and for three hours straight we were discussing questions to be asked, how to get good insights from the community members.
Questions
Starting with a pool of questions we started narrowing down the questions that would make a flow in the interview and give us good insights. After interviewing a couple of members we derived the following questions. The questions acted as our guides, and they could change depending on the conversation or context.
User Interviews
I joined the neoGcamp discord community. It is open for everyone. I could see some active names here. Then I cross-checked these names on Twitter and LinkedIn. People who are active participants are usually active on different social networks as well, it helps them gain exposure. I made a list of potential community members for interview (which could cover different persona types according to our initial research) and approached them on the social network where they were active.
I recorded all the conversations on a spreadsheet where each column represented the stage of our conversation and context (where the conversation took place)— approached | replied / no response / declined | decided a time to meet | Notes. Every conversation was a learning experience. I experimented with different lines of introduction. The initial interviews were challenging. As I progressed with more interviews things became easier. The people I interviewed started referring other community members that I could interview later on. I had a bunch of learnings which you can read in the latter part of this case study.
I interviewed 10 members for 13+ hours of conversations.
User personas
While interviewing different cohort members at various stages of the neoGcamp we observed four personas.
Here is a visual to understand the different users in the camp.
User persona 1: Transitioning career switch into tech domain.
Current Status: Finished levelZero submission and waiting for levelOne interview.
User persona 2: 1+ year in the tech domain, direct interviewed through intermediate route
Current Status: Got selected in levelOne, Cohort starting from Jan’2022
User persona 3: 2+ years in the tech domain
Current Status: Neograds from the first cohort.
User persona 4: College graduate, Fresher.
Current Status: Neograds from the first cohort, got placed at Unicorn startup.
User journey map
User 1: Sahil Raza
Transitioning career switch into tech domain.
Current Status: Finished levelZero submission and waiting for levelOne interview.
User persona 2: Nithin M
1+ year in the tech domain, direct interviewed through intermediate route
Current Status: Got selected in levelOne, Cohort starting from Jan’2022
User persona 3:Nandini M
2+ years in the tech domain
Current Status: Neograds from the first cohort.
User persona 4: Kapil Maan
College graduate, Fresher.
Current Status: Neograds from the first cohort, got placed at Unicorn startup.
Hypothesis validation
We started working on the project with a list of hypotheses and assumptions. It was important to make a note of them at the beginning of the project so that once we were done with all the user interviews, we could come back to this list and validate our initial assumptions.
Ideas and suggestions
Taking care of the unanswered queries by the members
A commonly observed issue is unanswered queries by the members. Mentors and members try to solve most of the queries on the channel, but since the number of queries is a lot, sometimes they are left unanswered.
We can make a bot to repost unanswered queries within a specified time (24/48 hours). And once the queries are answered, they can be closed by the person who asked them or by the person who resolved them.
Recognition for resolving queries
The issue of unresolved queries described above can be further tackled by giving recognition to the most helpful members of the community. Karma points can help aid recognition. Think of it as StackOverflow for neoGcamp discord.
These points will be visible as badges, titles, etc. Gamification principles can be used to create a bot for this purpose, which will incentivize helping others by solving their doubts.
A common platform to share notes and videos from live lectures
The recordings of the live classes are shared with team captains[*] in LevelOne, which in turn is shared with other members of the group.
These recordings of the live sessions, notes, resources can be all shared on a common platform. At 10kdesigners we use Noggin for this purpose. It is user-friendly and well organized to share all the information along with assignment briefs, and submissions. That way, the delay in sharing the videos and the dependency on team captains for sharing the video recordings can be removed.
[*] Team-captains are assigned with every group of 18 members in LevelOne. A team captain is the go-to person in case of a query or a doubt. Team captains are selected on the basis of their skill set. They hold the responsibility of the group. They motivate members to finish the assignment on time.
Different modes of assignment projects for LevelZero
Participants of LevelZero come from a wide spectrum of skill-sets. There are people with no coding experience, as well as people who join the camp after their graduation in Computer Science. During my interviews I also met people who joined the camp after working in the domain for some time.
At the moment, all of these participants have to go through the same route in levelZero regardless of their skill levels (we are not referring to the intermediate route here)
A skill level test would assess their current level. We can push tailor-made assignments according to their results. This will make sure that the assignments are always interesting and challenging for them.
Association with the Alumni mentors and guest lecturers
With 300 people in the first cohort and ~1000 people in cohort 2, the community is growing exponentially. At the moment, the brand seems synonymous with Tanay. But for a community growing at this scale, it is important to include more faces in the brand placement. This could include guest lecturers, industry mentors, and alumni mentors.
The name and socials of the Alumni mentors (neograds) and guest lecturers can be uploaded on the page along with their current job designation or the projects they are working on.
My learnings from this research
- Selecting the community
After understanding the problem statement, we didn’t directly jump on selecting a community. We gave ourselves some time to understand and research different internet communities. This gave us a pretty good idea about active communities on the internet and we could draw parallels later while researching the NeoGcamp community.
2. Approaching people for the interviews
I love meeting new people and striking conversations. Interviewing members from the community was an absolute joy. But the challenge was how to approach these members at the beginning. Their final submission date approaching soon made it even more difficult. I learned that if you are genuinely interested in knowing a person they will be happy to share their insights with you too. I followed the cohort members on Twitter and LinkedIn, I saw their finished projects and later I approached them with a personalized request message for an interview and 90% of them responded despite their busy schedules.
And what an absolute delight it was to listen to their inspiring stories! Some snippets of our conversations.
3. The importance of documenting the process
From researching the communities to interviewing people from the community, the one important thing that I felt was to document everything. This made sure that I didn’t miss anything, or something that might look unuseful at that time but would prove important later. Also, our brain tricks to take shortcuts, to believe in the hypothesis we have, but UX research is a time-consuming and laborious job where you have to take care of the details.
4. Making friends
When we kick-started the research, we came up with different ideas. But what I absolutely loved was that despite different ideas we enjoyed working together. I learned a lot while working on this project. The brainstorming sessions for preparing a questionnaire for the interviews and approaching different community users were exciting.
We enjoyed late-night sessions with Venkat bringing her delicious dinner on the call sometimes. It brought smiles to our faces every time Nisheta brought Milo (her pet dog) on the call. Thanks to Arnav bringing his auto-layout wand out even on the draft sticky notes, we had everything organized. At the conclusion of this project, Siva gave a pleasant surprise with a fantastic hand-drawn illustration of all five of us together.
I would like to thank all the neoG.camp community members who participated in user interviews (some of them were up to 1.5 hours long); Tanvi Priya (founder neoG.camp), for answering all my queries and taking out time for a call despite being extremely busy with the admission process. A shoutout to Jash and Agrata for being very helpful mentors.
You did great if you are still with me till this point. And if you have come this far, you can perhaps share how do you find this case study? Did you learn any new insights? I’ll be happy to hear from you.
If you want people to become passionate, engaged in a field, transformed by an experience — you don’t test them, you don’t lecture them and you don’t force them. Instead, you create an environment where willing, caring individuals can find an experience that changes them.
Seth Godin
Let’s Connect
Are you working on something interesting that you want to collaborate on? Please feel free to reach out.
