This post is written by Sarita, in response to conversations with my Papers Co-Chairs (Juho and Siân), with the Editor team (Andrea, Andrés, Morgan, Sean, Uichin, and Sue), and with the Steering Committee (chaired by Amy). If you have feedback, concerns, or questions, email the Papers Chairs at: papers2020@cscw.acm.org
Information in this post is subject to change! Please refer to the CfP for official information, or email us if you still have questions.
In July 2019, we announced that CSCW would be changing! Please read that early announcement here: http://cscw.acm.org/2019/CSCW-2020-changes.html.
The CSCW Oct 2019 CfP is here: http://cscw.acm.org/2020/submit-papers.html.
This post is to give more details about some of the changes.
What is the difference between Editors and Papers Chairs? Editors are common in journals. A few years ago, CSCW shifted to publishing in the journal, the Proceedings of the ACM on Human Computer Interaction (PACM HCI), so the introduction of the Editor role is a natural choice. With the move to quarterly submissions, Editors will also help steward the peer review process.
Editors have three major responsibilities. 1) They will oversee (with Papers Co-Chairs) the assignment of ACs to papers; 2) they will review new submissions and assess whether they meet the criteria for a Desk Reject; and 3) they will be the contact people for ACs to help ACs make decisions about papers. Namely, if ACs are not sure whether a paper should receive an Accept with Minor Revision, Major Revision, or Reject (or if there is some related concern, like an ethical question), they will reach out to the Editors for assistance. Overall, Editors are overseeing the scholarship itself, via the review process.
Papers Chairs will do everything else. Papers Chairs recruit Associate Chairs, set up and manage PCS, oversee all the various phases of the review process, and set up the conference program itself each November. If you have a question and you aren’t sure about who to ask, email the Papers Chairs! Most likely, you are looking for Papers Chairs and not the Editors.
What happened to the PC meeting? We don’t have one anymore. A main benefit of the PC meeting was being able to discuss papers that were on the fence with a group of experts. This helped ACs make decisions about that particular paper and to establish norms more generally. We will lose some of that with the loss of PC meetings. What do we gain? PC meetings were difficult to orchestrate, and an immense demand on many people’s time, in terms of logistics, time changes, etc. Papers Chairs and ACs regain a significant amount of time with the removal of the PC meeting that can be directed towards the review process. The addition of the Editors is intended to make sure the ACs still can discuss papers with experts in the field and that Editors can work together to discern and shape norms within the community. We will have an in-person meeting for ACs each November that allows for high-level discussions as well.
What happened to numeric scores? We do not have numeric scores anymore. We think those introduced a lot of noise. Authors spent a lot of time trying to read into a 3.2 or 3.3 or 3.5 average; we’d like to focus on the scholarship. Very roughly, a 4 or 5 tended to indicate accept, a 1 or 2 tended to indicate reject, and a 3 tended to be a noisy signal. We now have Accept with Minor Revision which is likely to correlate with the 4 or 5, Reject which is likely to correlate with the 1 or 2, and Major Revision which is likely to correlate with the 3. These are of course rough mappings. Both categorization systems allow variance among reviewers which is what the discussion phase, along with Editor support, is intended to address.
There is not always a substantial difference between a 4 and 5 — both are likely accepts and strong papers. There may be a substantial difference between a 1 and a 2. A 2 could become a strong paper in a new submission cycle. A 1 could also do that, or it may just be a poor fit (in terms of topical fit or quality) to CSCW and the more useful advice for the authors would be to consider a different venue. We will encourage ACs to signal to authors, when possible and appropriate, if they encourage a resubmission (as a new submission) or not. In general, we will encourage ACs to use clear language for our global CSCW community.
What is the difference between Desk Reject, Quick Reject, and Reject? All three fall under a broad “Reject’ category overall. The difference is not in quality of submissions, necessarily, but rather in where in the process the rejection happens.
A Desk Reject is determined by Editors soon after submission, and is most likely because of egregious errors in the submission process, or possibly because a paper was resubmitted with no evidence of effort to make changes. Desk Rejects are not assigned ACs or reviewers.
A Quick Reject is determined by ACs. These are done if both ACs think the paper is not likely to have any chance at being accepted, and an Editor agrees with that assessment. Quick Rejects are not assigned reviewers.
A Reject goes through the typical review process.
What is a Major Revision? As before, authors will have the opportunity to revise their submission following feedback from reviewers. Revised submissions should be made at the next cycle after feedback is given. Major Revision papers will receive the same set of ACs and authors. This is the same process was before (which was previously referred to as Revise & Resubmit). Authors are required to highlight major changes (via track changes in Word, or via font color changes in LaTeX) and submit a highlighted change revision *and* a clean revision. This makes the review process easier for reviewers and ACs. There is also a summary of changes required with the revision. Major Revision submissions *must* be submitted in the immediate next cycle. For example, if the original submission was Oct 15, 2019 and it received a Major Revision decision, the revision must be submitted by the Jan 15, 2019 deadline. Missing that deadline will result in a rejection. The paper will no longer be under review at CSCW (but could be submitted as a new submission in a future cycle, per below).
What is a resubmission? If all or any part of a submission was previously submitted to CSCW at any time, authors will need to include what cycle it was submitted to (e.g., Oct 2019), a summary of reviewer feedback, and what changes were made (or why changes were not made). The purpose of this requirement is to discourage persistent resubmitting of papers without authors taking the time to incorporate feedback. Every so often that is a defensible choice, but most of the time it is not and it’s an incredible burden on an already over-burdened reviewing system. It’s also not fair to reviewers and ACs who took the time to offer feedback. The submission history will only be available to Papers Chairs and Editors, not to ACs or reviewers, so authors should have little concern about biases in the reviews they receive. The resubmission will be considered a new paper in PCS and will be assigned a new set of ACs and reviewers (though as is always the case, the same AC or reviewer could end up being a reviewer for a paper that they reviewed in a previous submission).
Can authors submit systems papers? Yes, we welcome systems papers! When submitting your paper, be sure to select the primary paradigm “Systems (e.g., novel systems, technical foundations, algorithms, tools that enable the building of new social and collaborative systems)”. Two of our six Editors are systems researchers who are invested in helping ACs to assess systems papers using criteria that are appropriate for systems work.
How does this work in PCS? James at PCS has put in a lot of work to update PCS for quarterly submissions (which is also in use by our friends over at IMWUT and ICWSM). Some parts of it work beautifully for quarterly submissions! Other parts are difficult for us to get right and there may be user errors in how we set it up. Please be patient as we work to optimize our use of PCS. If you find errors or confusing information, email the Papers Chairs! Very likely, it’s confusing for other people too and settings may in fact be wrong.
Each submission deadline is associated with a track in PCS. So the Oct 15, 2019 deadline is one track. The Jan 15, 2020 deadline is a new track. For authors, reviewers, and ACs, this means that you could have two *active* CSCW tracks at any given time. If you submit a paper to Oct 15 and receive Major Revision, you will submit the revision *back to the same original Oct 15 track in PCS* by Jan 15, 2020. If you also submit a different new paper to Jan 15, you will now have two active CSCW tracks in your PCS. The same idea goes for reviewers and ACs. Reviewers/ACs may need to bounce between reviewing a Major Revision for Oct 15 and reviewing new submissions for Jan 15.
Presenting at CSCW Presenting at CSCW is strongly encouraged but not required. You will be asked to indicate if you plan to present at CSCW when you submit your camera ready accepted paper.
How will this change my workflow? Quarterly models are good for authors. We hope this allows you to submit work when it is ready, and helps to encourage healthy relationships with deadlines. That might mean that you enjoy a little bit of motivation and adrenaline to meet the deadline, but that it should not significantly and negatively impact your wellbeing.
One major shift is that reviewing will happen more frequently, though the loads in each batch will typically be smaller on average. After talking with folks running ICWSM and IMWUT, we anticipate that reviewing may feel more relentless, if you’re accustomed to batch reviewing. If you come from a journal culture where reviews are ongoing and not associated with yearly deadlines, this shift will be more natural for you. We recommend that ACs and reviewers try to adapt their processes accordingly — instead of saving reviewing until close to the last minute, and then doing a batch process — build it into your workflow, the same way you do for teaching, reading, writing, committee meetings, etc. — and establish a default that there will always be a little bit of reviewing work to be done and that’s okay. Reviewing can be fun, to the extent that we get to read about new ideas and help authors develop those ideas through the review process. As with journals, we will also need to shift away from a temporal match between submitting and reviewing — across a year, authors should review 3x as many papers as they submit, in general. However, across a given quarter, we don’t expect, or necesarily want, the same people to be submitting and reviewing each others’ papers. We will keep reflecting about how to manage reviewer loads and welcome feedback from everyone about what has worked for them.
Questions? Feel free to email us: papers2020@cscw.acm.org
