Digital tools and workflow for managing my research activities
While the research work itself might seem a daunting task at times, I feel that the tools available to us should make it much easier. As I look at it, due to the advancement of technology, there have been two big jumps over the past years or so in terms of our day-to-day tools for research activities in general. The first jump was to move from the traditional paper and pen to digital. Currently, I am sure we are more comfortable typing the documents on a computer rather than writing it by hand and for reading too, the printed documents are slowly fading away in favor of displaying the content on a screen. The second jump, I feel is moving from ‘local’ computing to cloud-based workflows. What I see is that people around me have easily taken the first jump, while the second jump is still catching on. I mean using cloud-based service has become the default when it comes to entertainment activities on the web, but in terms of real ‘academic work’, it is still catching on. I felt the need to write this article to keep a track of the tools I am using for the research activities as well as kind of a reference for young people starting out in this area of work. In some sense, I am also writing this article so that maybe at a later stage, I can come back to it and wonder about the state of current tools used in research. Especially in the fast-changing world of technology, it is not unheard of things getting outdated in a span of years!
If I have to write about my research activities; they are mainly concentrated on the following aspects :
- Searching literature, archiving, collecting references.
- Managing the papers, annotating, citing, and reading them
- Development of codes, software, pipelines, and their documentation, etc.
- Writing the text in the form of research notes, research papers, revisions to manuscripts, proposals, applications, etc.
- Tools for managing the entire workflow.
Literature search, reading, and management.
The NASA- Astronomical Data System (ADS) has become one of the most useful tools for people involved in astronomical research. The entire database consists of millions of papers, articles, and other related research items which act as a ready library/database for the entire astronomical literature. We don’t usually need to go beyond the ADS for any literature search. The modern web UI of ADS is quite intuitive and it is super easy to get any literature there. Apart from the ADS, I scroll arXiv daily for new updates related to research. This daily habit also helps keep me up to date with current literature and a bunch of ideas to think upon over the week.
I use Mendeley for keeping the research papers together and also for reading and annotating them. It provides a desktop app as well as an online library which helps me sync all the documents. For bibliography, Mendeley automatically generates the .bib file for all the references available there, so it becomes a hassle-free experience. I wrote an article about Mendeley some time ago, which might be useful here.
Writing and note-taking
For writing the research results, a dedicated LaTex editor is necessary and very useful indeed. I use Overleaf for writing research papers and related documents such as telescope proposals, other proposals, etc. The benefit of using Overleaf is that the entire work is based in the cloud, so we don’t need to worry about the loss of data. Further, collaborating with my other colleagues is super easy as I just have to share a web link with them and we all can work on the document simultaneously.
For making notes/logs/any other documents, I use Markdown language. It is super easy to learn and much more flexible than any word processing software indeed. I use a web service known as HackMD for this purpose. The benefit of writing in markdown is that the output can be generated even in the form of an HTML webpage, which becomes very easy to share with other people and looks much more pleasing than a word document. Earlier, I used Google docs and its allied services. However, for making any presentation, Google Slides is still my preferred choice because of its flexibility. For quick notes, say while listening to a talk, or some ideas generated out of a random discussion over tea I prefer the Google keep app handy on my phone. I also use Zenpen as a notepad on my Desktop/Laptop as an easy to use online notepad.
My own computer
For offline work on my computer such as running codes, I prefer Ubuntu Linux, not because of any personal preference, but rather because of its ease of usage and minimal need to install additional packages. Currently, I am using Ubuntu 20.04 with Gnome 3.38 and it is running on Linux Kernel 5.4+. From the OS on my computer, I only expect a terminal, a text editor, and a web browser. If these three things are available…I feel rest all is a luxury!
For the terminal emulator, I use Terminator because of its easy access and high customization options. I particularly like the horizontal and vertical split functions as I can get as many terminals as I want on a single screen with this feature. I use atom as a general-purpose text editor. Again, with a wide variety of plugins available, atom is a versatile tool for people writing codes. One can push their code to GitHub directly from this editor and also share one’s window with anyone else with just a token! It is super easy to collaborate with other people with atom and I don’t see the use for any other editor currently. of course, editors like ‘vi’ have their own benefits, so if I have to log in to a remote desktop, ‘vi’ is my choice, apart from that I stick with atom on my computer.
Most of the computing needs are satisfied by the Python language in my case. I use the anaconda distribution, with the latest version of python running on my computer. While I am very impressed with the ubiquitous availability of Jupyter notebooks (read my article about Jupyter notebooks!); I still follow the traditional text editor- compiler path and have been able to run Jupyter notebooks only when needed in cloud-based environments, for example, Google Colab.
Storage
I don’t feel the need to purchase any hard drives right now. In fact, the only hard drive I own is the one inside my laptop, on which my OS exists. I am considering replacing it with a solid-state drive in due time. Apart from that, my storage needs are fulfilled by cloud-based services. I use a mixture of Google drive, Yandex disk, etc., and for the time being. I have never felt the need to get that extra hard drive. Of course, I use a lot of public data for my research, which is often available as an API query, and thus, I never felt the need to ‘store’ this kind of data on my computer. However, for the data generated from the telescopes, I am well within the limits the cloud services have set for me! I believe with the dropping prices in hardware, the hard drives will get even cheaper, which will cascade into the cloud services becoming cheaper than they are now, and owning a hard drive will be obsolete in the near future, at least for individuals.
Communication
Communication is one of the most important tasks for researchers. I use different software for this purpose. While I use a software known as Teamviewer for connecting multiple desktops together remotely, I prefer the SSH protocol to connect with desktops which allows me to do it! Often, I need to access the High Power Computing Facilities for which I use the SSH protocol quite easily.
With the COVID19 restrictions, the adoption of cloud-based services in almost every area including research activities has accelerated. In the current scenario, usual tasks such as the meetings with the supervisor, attending the conferences, or even defending a Ph.D. thesis, this online-only or online-preferred concept is here to stay for some time. Further, our group has its own Slack channel to post work updates and new papers or discussions that might come across. With the COVID19 pandemic, we have been drawn to use various meeting software for a lot of purposes. Many conferences have moved online. Again with a lot of services available, I use Google meet/skype/zoom/Webex/BlueJeans/jitsi. for face-to-face communication with other collaborators. The reason for listing all of this is because I don’t have any personal preference, and I often leave it to the other side to decide whatever software they want to use.
Managing the workflow
For managing my workflow, I use GitHub (see my article about GitHub here). When we start using the git-based workflow, there is no going back. The most important thing I have found is the ‘version control’ which makes sure I always have an option to go back in time and see the changes I make to any code. I keep a git repository for all my projects and push all the updates to GitHub as soon as any task is done. Further, all the links to the respective web services are present in the repository itself so that the access becomes seamless. My primary hard drive crashed last year, and it was wiped out entirely! However, thanks to the git-based workflow, I was able to resume the work as soon as I got a new hard drive and pulled everything onto that.
These are the major tools that I use on a daily basis. Of course, there are other tools too that find use here and there but the tools mentioned above are the ones that are in use on a daily basis and hence indispensable to my research work. With time, of course, there may be further development in the various software and thus these tools may be obsolete indeed as we have seen in the past too!!!
