Setup Rabix Composer Local Executor using WSL2

In the space of Data Science/Analytics, we rely heavily on automation to drive insights from several data sources. Docker, Python, node, JSON and YAML are a amongst a wide range of technologies used as part of the automation.

I recently came across Rabix and Common Workflow Language (CWL) in the space of biomedical research. As a developer, and not having come from a biomedical background, I wanted to explore opportunities for using Rabix/CWL.

After looking into Cancer Genomics Cloud and signing up, I started to get a feel for Rabix/CWL. I decided to setup a local Rabix/CWL playground on my Windows WSL2 environment. This would also be a prelude to some personal discovery work into CWL & Apache Airflow.

This article aims to cover the following:

WSL2 References & Prerequisites

It’s important to note that Linux distributions enabled with WSL2 are Windows hosted Virtual Machines which run using a Linux Kernel. As such, we’re able to run Linux apps natively, including Rabix.

My attempts to get the Windows version of Rabix working with a local executor, were unsuccessful. Perhaps tinkering with different Java versions might get you over the line if you decide to experiment with this approach.

Requirements

$ sudo apt install lsb-release
$ lsb_release -a
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster

Install/Configure XServer (MobaXterm)

Note: If you on shared network, setting “X11 remote access” requests to “full” may not be an appropriate option, in which case, select “restricted”

Installation of Rabix and Dependencies

$ sudo apt update
$ sudo apt-get install libgtk2.0-0 fuse mesa-utils \
wget libdbus-glib-1-2 kdialog desktop-file-utils xdg-utils \
software-properties-common gnupg libnss3

Install JDK8 (this version is listed as a requirement at the official Rabix site).

$ sudo mkdir -p /usr/lib/jvm 
$ sudo tar zxvf jdk-8u261-linux-x64.tar.gz -C /usr/lib/jvm
$ rm jdk-8u261-linux-x64.tar.gz
$ sudo update-alternatives \
--install "/usr/bin/java" "java" \
/usr/lib/jvm/jdk1.8.0_261/bin/java" 1
$ sudo update-alternatives \
--set java /usr/lib/jvm/jdk1.8.0_261/bin/java

Configure JAVA_HOME and Global Profile

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_261
export PATH=$JAVA_HOME/bin:$PATH

to the end of global profile file (or $HOME/.bashrc for individual profile)

$ sudo vi /etc/profile..
..
export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64
export PATH=$JAVA_HOME/bin:$PATH
$ source /etc/profile

Download Rabix

The latest release of Rabix can be found at the following link: https://github.com/rabix/composer/releases.

$ mkdir $HOME/rabix
$ cd $HOME/rabix
$ wget https://github.com/rabix/composer/releases/download/1.0.2/rabix-composer.1.0.2.AppImage -O rabix-composer.1.0.2.AppImage
$ sudo chmod 755 rabix-composer.1.0.2.AppImage

Add Display Server to ~/.bashrc

export DISPLAY=192.168.1.111:0.0
export LIBGL_ALWAYS_INDIRECT=1

Launch/Configure Rabix

$ cd $HOME/rabix
$ sudo ./rabix-composer.1.0.2.AppImage

Sample Workflows

Ensure Docker is running before attempting to run the following samples.

1. Running a Command from within a Docker Container

We are going to build a simple workflow which pulls an Alpine docker image and runs and spawns a container, from which we run an echo <message>. We'll be able to pass in the message parameter value during testing from Rabix.

cwlVersion: v1.0
class: CommandLineTool
label: Alpine Docker Image
baseCommand:
- echo
inputs:
- id: message
type: string
inputBinding:
position: 1
outputs:
- id: std_out
type: stdout
requirements:
- class: DockerRequirement
dockerPull: alpine
stdout: output.txt

You can use either of the following methods to view the log output:

For example, to view output written to file output.txt for the above job using Windows Explorer, replace the parameters and browser to the location using Windows Explorer.

\\wsl$\<linux distro name>
\home
\<Linux distro username>
\rabix
\Executions
\local
\<workflow name>
\<YYYY-MM-DD-HH-MM-SS>
\app-<YYYY-MM-DD-HH-MM-SS.SSS>
\root

2. Workflow with Embedded Python Script

The following workflow contains an inline/embedded python script, samplepy.py:

Proceeding as per the prior workflow:

cwlVersion: v1.0
class: CommandLineTool
label: Run an embedded Python script
hints:
DockerRequirement:
dockerPull: python:3
baseCommand: python

inputs:
script:
type: File
inputBinding:
position: 1
default:
class: File
basename: "samplepy.py"
contents: |-
var = "value of var"
print(var)

outputs:
results:
type: stdout

stdout: export.txt

Where to From Here?

Rabix and CWL have provided an additional perspective in the area of solution design/implementation. The way in which CWL is implemented promotes containerised, parameter driven components.

Having gained a basic knowledge of CWL workflows and establishing a local test environment, some possible areas I aim to cover off on in future are:

Learner. Interests include Cloud and Devops technologies.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store