Matt Ralston

Matt Ralston

M.Sc. Bioinfo + 5 years industry exp. - - - Currently writing a DNA sequence alignment, quasi-mapper, and graph algorithm and manuscript (for bioRxiv). I play math rock on my guitar, I write open-source software on GitHub, I pay my bills in the DE-NJ-PA tri-state area near Delco, and I'm launching a DE-based LLC for Univ. of Delaware and local NGS sequencing projects.

Cloud native architectures, containers, DevOps, and SysAdmin

Subject area expertise

For readers who are interested in my software development experience, visit /software. For readers who are interested in my /data_science roles, experience with bioinformatic algorithms and research related software, as well as my perspectives on architectures for application development and modeling perspectives, please visit my /data_science page. For readers interested in my expertise in systems administration and devops, please visit the /cloud_native page. For readers interested in more core life-science competencies, please visit /biosciences. For professionals who want to know more about my research experiences, check out /research. If you want, take a look at my /portfolio of public projects. If you are interested in contacting me about work opportunities please visit /contact.

Consider my recent blog post on [Docker containers and kubernetes)(/machine_learning_infrastructure_deployment_and_notebook_management) for a more current discussion of DevOps and k8s.

The discussion begins with consideration of classical (ca. 2000 server-side system administrative practices) systems administration. Most of the early web ran on bare metal on-prem, or on early cloud systems. Knowledge required includes OSX/Linux operating systems, HTTP2.0 communications, REST/SOAP systems, relational databases (MySQL, MariaDB, SQLite3, PostgreSQL, Oracle, Microsoft SQL server), filesystems (ext4, xfs, zfs, btrfs), source tarball installation, and more.

During the mid 2010s there was a movement to cloud-native platforms for price sensitivity. This gave rise to the modern git with Github/Gitlab, Google Colab, and Markdown-driven documentation style to software development. For some applications and systems, more complex configuration is necessary to ensure the proper interplay of meshed services. Enter the Docker container.

Most application code and service configurations are stashed away in a version control systems (VCS) such as git, ‘mercurial’, or ‘subversion’ svn repositories. The code and documentation provide implementation details for deploying an application, migrating data, and system maintainance.

During the period after the coronavirus pandemic of March 2020, the convention for cloud-native application architectures relies on some combination of VMs, containers, orchestration tools (Chef, Ansible, Puppet), and cloud-vendor (AWS, GCP, Azure) specific configuration.

Common knowledge would suggest that the best option for cloud agnostic and cloud-native architectures includes Docker and kubernetes at some level. Dockerfiles are considerably similar to shell scripts that would have done the heavy lifting of sysadmin roles in the 2000s, in that they primarily deal with shell-level configuration of environments for a specific single-feature single-purpose design.

devops

“Development Operations”, or simply devops, involves the configuration of system for application stacks within the context of the goals of the entire app. Primary tools for devops include Docker, kubernetes, Ansible, Chef, Puppet, and the boto3 AWS SDK for Python. Goals for devops positions include decreasing the load of technical debt, to make application subsystems reusable and long-lasting during a program’s lifecycle, and packaging services or subsystems for the application according to their goals diagrammed during requirements discovery and fine-tuning.

Docker containers

Containers, and more specifically Docker containers, use kernel virtualization in Linux systems to produce so-called “lightweight” virtualization. These containers are lightweight operating system images, stored as binary artifacts associated with repositories in the Docker image “registry”. Associated with a repostiory are container components, dependencies, and the Dockerfile, a recipe for creating the image artifacts and resulting operating system and application configuration from known and packaged assets in the registry’s repository.

From Wikipedia.org/wiki/Docker_(software):

“The Docker software as a service offering consists of three components:

devops vs sysadmin

Systems administration has traditionally involved subject matter expertise and experience with varieties of systems and their configurations. Reproducing a system configuration was a principle concern of early “lift-and-shift” cloud deployments and was often done through the knowledge of both local and cloud systems, storage options, caching, object storage (static assets), CDNs, and other subject area expertise.