Tags:

If Perl is Glorified Shell, Shell Scripts Are Dead. Long Live Shell Scripts

For those of you who don't know me, I'm your typical beginner bioinformatician. I can script with a few languages but strongly prefer 1990s-2000s dynamically-typed languages like Ruby and Python and write algorithms when nothing else is available. A little bit of webdev, databases, cloud, stats and stuff sprinkled in.

When I first started, my friends E and R mentored me in the basics of Illumina NGS data: quality control and mapping. My graduate school was fortunate enough to have a good Beowulf cluster so the bulk of my analyses needed to be submitted to a PBS/Torque grid. And thus began a quest for knowledge of linux, shell scripting tips, and eventual understanding of what tool/language to use for which tasks.

The shell is all some people ever need... but if you've ever seen how simple or complex CLIs from Python's argparse can be, you've probably gotten frustrated with the shell too. Sure yaourt and git are mostly bash/C, but it's pretty difficult to make similar level of UI/UX at the CLI for most pipelines in a world where tools, pipelines, and best practices are changing rapidly in a way that is meaningful or stable. In the world of bioinformatic pipelines, shell scripts are both the low-effort solution to provide cohesive workflows and yet difficult or ugly to test and maintain.

Yet Another Workflow Language?