Note - The purpose of this template, for now, is for me to document it to get 1) feedback; and 2) for documentation.
A research paper is like a story. We need to have a story arc: introduction, rising, climax, ending. Each section is connected to each other. Each paragraph should naturally introduce to the next. Each sentence should flow to the next. Even the last sentence of the last paragraph should have an understandable flow and connection of thought to the next paragraph’s first sentence.
A research paper is also a report of our findings. It should be easy enough for both the reviewer (our undoubtedly gatekeepers of publishing our paper) and readers (who would eventually cite your paper for the next) to know what your research is about, what it contributes, and why do they care about it. They should also know where to find what information they need. It should be a delight to read our paper, not something that will frustrate the reader. Claims should always be backed up by evidence. If there’s no evidence, remove the claim or do the experiment (and report it) to back up your claim.
Some rules of thumb I follow:
- If we are going to have abbreviations, the first time we are going to use it would be the “introduction of the abbreviation” with the meaning. And then abbreviate away to our heart’s content
- For every subsection groups that we create, put a paragraph that outlines and concisely define each subsection before going to each of the subsection itself. It will be our mini-table of contents for those subsections and allow the reader to understand what part it is and decide if they would skip those subsections or not.
I hope this template would help would be research writers on how to better create a research report.
The template that I am using is what I have put based on my own writing experience mixed with my delight and frustrations as a reviewer and AC for CHI. I have to say that this is not yet fully tested (like I cannot say that it is 100% foolproof or how many papers have been published using this template out of the total number of papers that used this template) but this is at least what I want to have when I am reading a paper.
A SIGCHI related research paper that I am used to is usually divided into these sections: Abstract, Introduction, Related work, Methodology, Experimental Setup, Results, Discussions, Limitations & Future work, and Conclusion.
The abstract is a concise, usually a 250-word paragraph that describes what our paper is about. Usually, I would have the whole paper in sentences. It can be divided this way (each bullet point is a sentence.
- Background of the problem
- Problem statement
- What it does
- Experiment Setup and Result
- Insights found
- What it tries to solve
- What it does
- Experiment Setup and Result
- Insights found
This way we can already help the reader decide if it is related to their work or not. Besides, if they found our insights way early, it will intrigue them on how we came to that conclusion, making them hooked.
The introduction describes what the paper is about and why people should care about reading it. If we can’t make people care or understand why we need to do this research, then the paper will not be read. Usually, the introduction also is a longer form of abstract where it summarizes the whole body of the paper (prepping the reader what they will see with the rest of the paper) but it doesn’t take away the purpose of the abstract: a summary of what the paper is about.
For me, the introduction is composed of three parts: The background of the problem, the problem statement, and the contribution of the paper. This would at least have a structure of three (3) paragraphs. Sometimes, we can have more than 3 but not more than 5 paragraphs. We don’t want to bore the reader on minute details of the paper that can be found in other parts. The following subsections will describe each part in more detail.
The Background of the problem
It should set the tone of the current research landscape that every reader should agree on. It is the common ground that we and the readers should share. If the reader can’t agree on the first few statements here, we would have already lost them. This is our own “this is what is happening, and these are the current conditions that lead to the problem”. This paragraph then would lead up to…
The Statement of the Problem
This is where we describe what the problem is about and why it is particularly hard to solve. We can put a brief scenario here to describe why the problem is a problem. Make it relatable to readers who are also in our field that care with our problem. This part can also have or end with the research question we want to answer.
Here I have to note that a research question or problem is different than a practical problem. The research problem should be something that needs to be answered by knowledge of a specific topic to be able to find out how to answer a research question. The research problem can be a flawed understanding or an incomplete view of the space.
Answering the research question provides a solution to the practical problem and give insight and additional knowledge to the community. Once we have set the stage for the problem and the readers are already nodding in agreement that it is really indeed a non-trivial problem to be solved, we can lead them up to…
Here we state the contribution. Usually, it summarizes the contribution in the form of the claim or goal of the paper’s work and how it plans to prove the points of the claim. It also directly answers the research problem stated by stating that our work is the answer to give new insights or understanding on the questions surrounding the research problem.
Usually, it will start with “we propose a framework” or “a new understanding to the problem” or “a design” or “a tool” that tries to answer the research question proposed above and solves the problem it entails. It summarizes the findings (but not taking the conclusion’s job for it) while teasing out what the reader will find in the paper.
Now, most of the time we see the related work right after the introduction. Some papers put related work right after discussion. There are reasons for this and it depends on the story flow. But regardless of where it was put, it serves an important purpose in reporting our research: to show the novelty of our work and what is lacking in research body that will serve as motivation for us to do this work.
The question that needs to be answered is: what is the current view and landscape or understanding of the question and problem and what is still lacking? Thus, it should not be a laundry list of related research papers without a reason as to why they are related.
I usually have related works in groups of three. Categorizations can be of the following:
- papers that tried to answer the research question,
- papers that have a similar answer to my work or inspired my work
- Papers that have a similar output by answers a different question (really hard to find but might be worth it)
- We can also try a different kind of categorization like categorized by using solution type ‘a’, and solution type ‘b’, and so on and so forth.
To connect with the last paragraph of the introduction, we can start with “Many papers have tried working on the problem stated previously” or “Recent work has tried solving it using…”.
The ending note usually per group of papers or the whole related work section is the difference of our work to related work cited or a reiteration of contribution in the light of the previous work.
The methodology of the research is the walkthrough on how we did the research or find out how we came up with the tool, or design implication, or our contribution. The question that must be answered here is: “If I want to replicate our study, what are the necessary knowledge I need to know and steps I need to know to have a similar setup in the hopes that I will get the same data result.”
Another way of writing this part is to answer the question: “How did we found out the answer?” We might start it off with some theoretical framework that describes what theories were involved in understanding the framing of the problem and the solution. Sometimes, we can put our preliminary formative study here to discuss what observations did we notice to come up with the solution.
If our contribution is a tool, framework, algorithm, or design, we put it in detail here, based on either the theoretical framework or formative study that we have done or both.
Note: Do not put what coding language or computer we used here
The reader doesn’t need to know in great detail that we used Python flask or PHP Laravel or even AngularJS on implementing the solution unless the contribution is really about the library that we have made specifically for a particular language. It is enough to report in 1 small paragraph the apparatus of the proof-of-concept was done (and usually a link to the source code)
As for links to the source code repository, make sure that when submitting for review, it is anonymized. Either we submit it as supplementary evidence for review or put it in an anonymized repository. Either way, add instructions on how to set up and run so that it can be useful for the reviewer.
This is where we set up how we evaluated our methodology to back up our claim in the last paragraph of our introduction. Here we set up the hypothesis that we want to find out that will back up our claim. We also set up clearly what measurements or variables we need to measure that helps us accept or reject the null hypothesis.
The experimental setup should be sound and all bases should be covered: Why did we use between-subject or within-subject design; how did we recruit participants; what was the demographic; how many did we recruit; was there a compensation package; how much was the compensation; was this a new practice; what was the experimental procedure; how many hours; what were asked in the survey; what were the ordering of experiment; where there given any breaks; how were the measurements collected; what were the questions that we asked at the end; why were they asked; etc…
This is where we just show the results. This is also where a lot of reviewers and readers get confused because most of the results are buried with statistical jargon and numbers in paragraph form that it is almost impossible to read it in one take just to understand where to focus.
This is where my personal take comes in: I would like to know why a particular statistical method was used to describe the sample given the setup. (I just don’t know though what writing style is best to still show that is still good to read even when the results are being reported).
This is also where we should put graphs or tables that would make it easy to read in a first glance what the data is talking about.
What we don’t want to put here is the implication of the data (that’s where the discussion is for). We just want to paint a new observation point based on the experiment. This will be the basis of the next part.
This is where we now gather insights based on the affirmation or rejection of hypothesis and/or theoretical framework and preliminary observations. We also use post-experiment questionnaires, among other things, to corroborate the data and extract reasons out of it. In essence, this is the meat of the insight that we want to find out, which when summarized is the new knowledge that we are all waiting for.
The more non-trivial and interesting insights are, the more promising the paper is (at least for me). This is the climax, the top of the mountain, the Eureka moment where it says this is the moment we have been waiting for. This is where the culmination of what we have found and patch up the missing knowledge that would at least give new direction in this field that you are doing.
But before going full blast with this part, make sure that every part here is related to our claim of contribution and backed up by the data in the result. Anything that stretches the claim or data to support an insight will be questioned by reviewers and we might need more data to support your stretched claim. For checking, just ask: “is this sentence or paragraph backed up by theory or observation? What would be the questions that people will ask? Where would be the answers to those questions?”
Limitations and Future Work
Now, of course, we cannot cover all our bases and we have limitations on time, resources, and paper page to put all cases in our paper. The limitation can be seen as a disclaimer where we honestly put some conditions that were not covered that will make our claim a bit more generalizable. We can put here at least that our work is generalizable only at certain conditions, provided those conditions are also a big enough or important enough by the community. In essence, this is where it will stop the reviewers from telling: “Hey this is not generalizable to [this scenario]” which in the first place, not the scenario that we are working on (unless it is very much similar, then we should have worked on it).
It also opens up new avenues and direction of research for us or for other researchers to work on. Let’s put explicitly what direction can it take us or others if more research is done in this field. It will not only make the reader know that there is still more to come but also helps fellow researchers find something to work on.
This is the end of the paper, what we need to do here is to summarize the paper “in conclusion”. What it means is that after all that we have done: from telling and motivating ourselves to work on a problem, finding the solution, proving that our solution is working and useful, and gathering insights as to why it works; we have to summarize and reiterate bits and pieces into one or two paragraphs.
I use the conclusion to see what the paper is about more in detail (more than the introduction) on the contribution of the paper without the minute details. It is like another teaser on how the conclusion came about, making me read how did it happen (which I sometimes do when watching movies, I go at the end to decide if it is a good ending, and then I watch the middle to know how it happened).
If the introduction is more on fleshing out the motivation on why we should solve the problem and ending it with a teaser of what our contribution is; the conclusion is all about the contribution and what the summary of insights that we have observed that is important to the community.
Here we summarize what I think is a good template of what a paper can be for better reporting of observed data and extract insights out of it. We have discussed what parts of a paper should have based on patterns of published papers in SIGCHI-related venues. The parts are the following:
- Abstract — A concise description of the paper which includes the problem, contribution, how it was tested, what were the results, and what are the best insights that were discovered.
- Introduction — The introduction puts into perspective what the problem is and why it is important to have a better knowledge of the research question that causes the problem. The last part of the introduction is a teaser paragraph that tells the reader what the contribution of the work and describes the high-level solution to the problem.
- Related work — We use the related work to give a view on what is the current understanding that we have on the current problem we are solving and how do we position our work in relation to the related works cited.
- Methodology — It shows our readers how we came up with the solution and tells others how they can replicate the solution or research to see if they will find the same results.
- Experimental Setup — It shows our readers how did the experiment was set up. This allows our readers to say if the results are sound and believable.
- Results — It shows the results of the experiment: the empirical evidence of the paper. This needs to be written well in such a way that it is easily understood.
- Discussion — This is where we put our insights and explain why the results happened.
- Limitation and Future Work — This allows our fellow researchers to explore the field more and giving them guidance on where to look for next.
- Conclusion — This summarizes the entirety of the paper and insight, giving a conclusion to your story.
Again, this is for me to document what I am learning, as well as also get feedback. If you find this useful for your next research paper writing, don’t hesitate to give it a shoutout :)