Setup Rabix Composer Local Executor using WSL2

Image for post
Image for post

In the space of Data Science/Analytics, we rely heavily on automation to drive insights from several data sources. Docker, Python, node, JSON and YAML are a amongst a wide range of technologies used as part of the automation.

I recently came across Rabix and Common Workflow Language (CWL) in the space of biomedical research. As a developer, and not having come from a biomedical background, I wanted to explore opportunities for using Rabix/CWL.

After looking into Cancer Genomics Cloud and signing up, I started to get a feel for Rabix/CWL. I decided to setup a local Rabix/CWL playground on my Windows WSL2 environment. This would also be a prelude to some personal discovery work into CWL & Apache Airflow.

This article aims to cover the following:

WSL2 References & Prerequisites

It’s important to note that Linux distributions enabled with WSL2 are Windows hosted Virtual Machines which run using a Linux Kernel. As such, we’re able to run Linux apps natively, including Rabix.

My attempts to get the Windows version of Rabix working with a local executor, were unsuccessful. Perhaps tinkering with different Java versions might get you over the line if you decide to experiment with this approach.

Requirements

$ sudo apt install lsb-release
$ lsb_release -a
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster

Install/Configure XServer (MobaXterm)

Image for post
Image for post

Note: If you on shared network, setting “X11 remote access” requests to “full” may not be an appropriate option, in which case, select “restricted”

Image for post
Image for post
Image for post
Image for post

Installation of Rabix and Dependencies

Image for post
Image for post
$ sudo apt update
$ sudo apt-get install libgtk2.0-0 fuse mesa-utils \
wget libdbus-glib-1-2 kdialog desktop-file-utils xdg-utils \
software-properties-common gnupg libnss3

Install JDK8 (this version is listed as a requirement at the official Rabix site).

$ sudo mkdir -p /usr/lib/jvm 
$ sudo tar zxvf jdk-8u261-linux-x64.tar.gz -C /usr/lib/jvm
$ rm jdk-8u261-linux-x64.tar.gz
$ sudo update-alternatives \
--install "/usr/bin/java" "java" \
/usr/lib/jvm/jdk1.8.0_261/bin/java" 1
$ sudo update-alternatives \
--set java /usr/lib/jvm/jdk1.8.0_261/bin/java

Configure JAVA_HOME and Global Profile

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_261
export PATH=$JAVA_HOME/bin:$PATH

to the end of global profile file (or $HOME/.bashrc for individual profile)

$ sudo vi /etc/profile..
..
export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64
export PATH=$JAVA_HOME/bin:$PATH
$ source /etc/profile

Download Rabix

The latest release of Rabix can be found at the following link: https://github.com/rabix/composer/releases.

$ mkdir $HOME/rabix
$ cd $HOME/rabix
$ wget https://github.com/rabix/composer/releases/download/1.0.2/rabix-composer.1.0.2.AppImage -O rabix-composer.1.0.2.AppImage
$ sudo chmod 755 rabix-composer.1.0.2.AppImage

Add Display Server to ~/.bashrc

export DISPLAY=192.168.1.111:0.0
export LIBGL_ALWAYS_INDIRECT=1

Launch/Configure Rabix

$ cd $HOME/rabix
$ sudo ./rabix-composer.1.0.2.AppImage
Image for post
Image for post

Sample Workflows

Ensure Docker is running before attempting to run the following samples.

1. Running a Command from within a Docker Container

We are going to build a simple workflow which pulls an Alpine docker image and runs and spawns a container, from which we run an echo <message>. We'll be able to pass in the message parameter value during testing from Rabix.

cwlVersion: v1.0
class: CommandLineTool
label: Alpine Docker Image
baseCommand:
- echo
inputs:
- id: message
type: string
inputBinding:
position: 1
outputs:
- id: std_out
type: stdout
requirements:
- class: DockerRequirement
dockerPull: alpine
stdout: output.txt
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post

You can use either of the following methods to view the log output:

For example, to view output written to file output.txt for the above job using Windows Explorer, replace the parameters and browser to the location using Windows Explorer.

\\wsl$\<linux distro name>
\home
\<Linux distro username>
\rabix
\Executions
\local
\<workflow name>
\<YYYY-MM-DD-HH-MM-SS>
\app-<YYYY-MM-DD-HH-MM-SS.SSS>
\root
Image for post
Image for post
Image for post
Image for post

2. Workflow with Embedded Python Script

The following workflow contains an inline/embedded python script, samplepy.py:

Proceeding as per the prior workflow:

cwlVersion: v1.0
class: CommandLineTool
label: Run an embedded Python script
hints:
DockerRequirement:
dockerPull: python:3
baseCommand: python

inputs:
script:
type: File
inputBinding:
position: 1
default:
class: File
basename: "samplepy.py"
contents: |-
var = "value of var"
print(var)

outputs:
results:
type: stdout

stdout: export.txt
Image for post
Image for post
Image for post
Image for post

Where to From Here?

Rabix and CWL have provided an additional perspective in the area of solution design/implementation. The way in which CWL is implemented promotes containerised, parameter driven components.

Having gained a basic knowledge of CWL workflows and establishing a local test environment, some possible areas I aim to cover off on in future are:

Written by

Primarily a Learner/Coder with interests in Python, Cloud Technologies, Security and Automation. Pandas munching on Bamboo sticks give me the “Giggles” :))

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store