A short note on avoiding typing your password at every push — The unsafe way of caching git credentials is: git config --global user.name "your username" git config --global user.password "your password" git config --global credential.helper store This is a general procedure that can be used for any Git server, but considering it’s storing your password in plain text this could be…

The raw output of NGS experiments should be backed, and format-specific compressor like “dsrc2” comes handy for this — Let’s skip the introductory comments about the frustration of seeing our disk quota almost full in any cluster we use for NGS data: it’s a well-known story, and as soon as we keep out of sight any unnecessary large file (.sam files anyone?) …

Two popular formats to store structured data, that are also commonly used in bioinformatics analyses. — Bioinformatics and text files Most bioinformatics file formats are simple text files, a famous example being the FASTA format to store sequences. Historically, most file formats were proposed to ad hoc address a specific need, resulting in a fragmented universe of formats. Examples of famous bioinformatics formats are the FASTA and FASTQ for sequences…

This article describes a Bash script to perform a simple task with some controls and best practices. — The problem Illumina NextSeq 500 will produce four separate files for each sample and we want to simply merge them in a single one (this is also true for HiSeq systems, but in that case the lanes are physical and it can be good to keep the file separate). The solution Illumina sequences are…

Using multiple threads to compress files but maintaining compatibility with gunzip — The gzip program is incredibly popular among Linux applications, in spite of some limitations. I think it fits incredibly well the Unix Philosophy of a single program, doing a single thing and doing it well. For example, we cannot compress a directory tree with gzip, but only single files. This…

Preventing unbound variables is good, but we need a way to tell if a variable has no content or has never been initialized — A variable is often referred to as a box with a name and a content. A command like the following: echo Hello $name Will print Hello, and then… the content of the box named ‘name’. If the box is empty Bash will print nothing, as expected. …

The minimum workflow for keeping file revisions under control — When working on a script (or manuscript), version control enables the possibility to restore the last working version. …

Sequencing data archives When you publish a paper based on nucleic acids sequencing it is mandatory to submit the “raw” data, i. e. all the reads produced without any filtering process, to a public repository. The two major repositories for NGS reads are the SRA (Short Reads Archive, hosted in the USA by…

Concepts and simple tools to evaluate the sequence coverage of a short reads dataset aligned against a reference genome. — Sequence coverage (or sequencing depth) refers to the number of reads that include a specific nucleotide of a reference genome. In the screenshot below a small region of the Human genome is shown in a genome browser, where the alignment of a re-sequencing experiment was loaded. …