A web based service provider whose aim is to give relevant solutions to program bugs, need to know which type of bugs are the most painful for developers. Raising the question as to what are the best ways to provide fast and useful solutions?
I am Andras Horvath, I joined Samebug six months ago as a data scientist and now I’d like to share some of my experience gained while measuring our performance and designing our data sources concerning user behaviour.
In case of Samebug the situation is the following: the company has built his knowledge base of stack traces and exceptions and provide a powerful search service. Developers can paste their whole stack traces to Samebug’s search box and find solutions to their crashes based on our analysis. We are deeply convinced that our knowledge accumulated during the years is of high value and our skill in analysing stack traces is unique. We also believe that the need for such a tool will be growing with the expansion of open source platforms and libraries in the software industry.
Still we have been looking for ways how to help the developer community more efficiently. The main task is to figure out how our services can be further developed so that programmers can use it in a friendly and understandable way. In other worlds we should be prepared to react to our visitors’ needs therefore we should explore somehow in advance the demands for this kind of service concerning their crashes.
Is it enough for them to use some crash monitoring tools available on the market or would they be happier with a tool that tries to give concrete solutions? Are they more willing to paste their stack traces in a search box or would they prefer to use a plugin in their IDE? What are their opinion about the usefulness of the solutions we give? After all this is the most relevant quality measure of our product.
If you are a developer of a service of this kind you have, of course, many opportunities to get feedback from your users. You can ask them directly through pop-ups or in email or via a questionnaire. Sometimes these are necessary but may be quite expensive and can be time consuming & intimidating for the users.
Probably you would also like to see what your users do on your site. From where they come, where they land on the site, what kind of interactions they perform and where they tend to move further until they leave it. It is then possible to draw conclusions from their behaviour regarding their demands. Which are the features they like, what other features they mark as unuseful, is there anything that seems to be in vain hunted for by them.
In that case you need to use some tracking analytic tool like Google Analytics (simply GA). Or you can run your own tracking system which gives even richer data to analyse than GA.
In either case you should track your events because if somebody wants to use the fancier tools of GA concerning user behaviour and conversions rates they are needed. For example you should be able to give GA the events to set up your specific goals to be measured.
One of the most interesting thing I learned at Samebug is the nature of web events and what can be read of them. I came from a field of social statistics where my database records were persons and households with data about their incomes and expenditure in some monetary value or in kilos, liters and pieces. These kind of data are mainly cross-sectional. You are lucky if having the chance to conduct a longitudinal survey and so the history of a household reveals itself from events like marriage, childbirth or some deducted monetary decision based on the data of consecutive month or years. Still this story is a much longer, slower and looser one. So you should always ensure that your methods are appropriate for the information you’d like to get.
Behaviour of visitors who are browsing the web with very strong determination to find solution to a specific problem and arriving at a promising site which offers some suitable tool is much different. It means a way faster and intensive event-flow. In our case they have their well specified intention: to find a quick and effective solution to a bug that blocks their daily work. They use intensively the web for the sake of this cause and one have not too much opportunity if he wants to grab hold of them.
For instance, if you are really interested in users’ demands not just in shoving them this or that way on your site and want to lean also on behaviour analysis not just on direct questions. Then and this is another important experience while working at Samebug, you have to lay out very profoundly the system of your tracking schema and especially the events which are your core data that a lot of the indices will be built upon.
The importance of planning I realize almost every day. As an example with the expansion of the features on the site it is more and more crucial to group my events because if I class every possible interaction in a separate category-action pair then I will be lost shortly. Without careful planning no reliable measurement is possible. We had a hard time with the team to find the place for planning our system of web-events because measurement is something that most people think about as just some follow-up thing.
If you hold on to your concept of building upon user demands you can use your A/B tests to measure not just the effect of some design changes but to explore your clients’ needs. It is also worth examining how many visitors return to the site and why. However, don’t think about them as client who are already satisfied with what they found there. Think about them instead as someones who are coming back in the hope of find something that is not yet there. This is an opportunity to try to find what they are missing, maybe it’s something that is in your capabilities to create.
It’s such a tremendous experience for me to work with web-events and draw conclusions based on observed user behaviour. This is the very moment when user feedback becomes more and more important for us and we are just about to ask our users more directly in some form. So if you have any comments on this or you have already tried Samebug in some way your opinion is highly appreciated as always, so feel free to leave them in the comments below. If you like this content please don’t forget to give it a clap and share it on your favourite social network.
