Setup Rabix Composer Local Executor using WSL2

In the space of Data Science/Analytics, we rely heavily on automation to drive insights from several data sources. Docker, Python, node, JSON and YAML are a amongst a wide range of technologies used as part of the automation.

I recently came across Rabix and Common Workflow Language (CWL) in the space of biomedical research. As a developer, and not having come from a biomedical background, I wanted to explore opportunities for using Rabix/CWL.

After looking into Cancer Genomics Cloud and signing up, I started to get a feel for Rabix/CWL. I decided to setup a local Rabix/CWL playground on my Windows WSL2 environment. This would also be a prelude to some personal discovery work into CWL & Apache Airflow.

This article aims to cover the following:

  • Installation of Rabix Composer on Windows 10, with WSL2 installed/enabled

WSL2 References & Prerequisites

It’s important to note that Linux distributions enabled with WSL2 are Windows hosted Virtual Machines which run using a Linux Kernel. As such, we’re able to run Linux apps natively, including Rabix.

My attempts to get the Windows version of Rabix working with a local executor, were unsuccessful. Perhaps tinkering with different Java versions might get you over the line if you decide to experiment with this approach.

Requirements

  • In order to access the Rabix Composer GUI from WSL2 VM, we need an Xserver running on the Windows host, which will render our display. For this article the Home Edition of Mobaxterm is used as an Xserver
$ sudo apt install lsb-release
$ lsb_release -a
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
  • Docker for Desktop needs to be installed, with WSL2 integration enabled for the respective Linux distribution

Install/Configure XServer (MobaXterm)

  • Download Mobaxterm

Note: If you on shared network, setting “X11 remote access” requests to “full” may not be an appropriate option, in which case, select “restricted”

  • We need to know the XServer IP address/details. These are required to direct Linux WSL2 to the location (DISPLAY=) where the Rabix display is to be sent
  • Make a note of the IP address as this will required it later on

Installation of Rabix and Dependencies

  • Open a prompt within our WSL2 Linux distro. Debian is used for this setup:
  • Install dependencies for X support
$ sudo apt update
$ sudo apt-get install libgtk2.0-0 fuse mesa-utils \
wget libdbus-glib-1-2 kdialog desktop-file-utils xdg-utils \
software-properties-common gnupg libnss3

Install JDK8 (this version is listed as a requirement at the official Rabix site).

$ sudo mkdir -p /usr/lib/jvm 
$ sudo tar zxvf jdk-8u261-linux-x64.tar.gz -C /usr/lib/jvm
$ rm jdk-8u261-linux-x64.tar.gz
$ sudo update-alternatives \
--install "/usr/bin/java" "java" \
/usr/lib/jvm/jdk1.8.0_261/bin/java" 1
$ sudo update-alternatives \
--set java /usr/lib/jvm/jdk1.8.0_261/bin/java

Configure JAVA_HOME and Global Profile

  • Add the following entries
export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_261
export PATH=$JAVA_HOME/bin:$PATH

to the end of global profile file (or $HOME/.bashrc for individual profile)

$ sudo vi /etc/profile..
..
export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64
export PATH=$JAVA_HOME/bin:$PATH
  • Activate the new configuration by exiting and logging back into WSL2 distro prompt, or by running:
$ source /etc/profile

Download Rabix

The latest release of Rabix can be found at the following link: https://github.com/rabix/composer/releases.

  • As of now, the latest release (1.0.2), can be downloaded to an appropriate subfolder via:
$ mkdir $HOME/rabix
$ cd $HOME/rabix
$ wget https://github.com/rabix/composer/releases/download/1.0.2/rabix-composer.1.0.2.AppImage -O rabix-composer.1.0.2.AppImage
  • Add execute privileges
$ sudo chmod 755 rabix-composer.1.0.2.AppImage

Add Display Server to ~/.bashrc

  • Using the address of the XServer we noted earlier on, add the following to the bottom of your ~/.bashrc file, replacing the IP address with your XServer's IP
export DISPLAY=192.168.1.111:0.0
export LIBGL_ALWAYS_INDIRECT=1
  • The LIBGL_ALWAYS_INDIRECT=1 variable enables OpenGL hardware rendering (if supported). If you experience issues in subsequent steps related to display output, try changing the "OpenGL acceleration" setting in the MobaXterm X11 Configuration tab from Hardware to Software

Launch/Configure Rabix

  • Start Rabix
$ cd $HOME/rabix
$ sudo ./rabix-composer.1.0.2.AppImage
  • Rabix frontend should appear in a Window on your host

Sample Workflows

Ensure Docker is running before attempting to run the following samples.

1. Running a Command from within a Docker Container

We are going to build a simple workflow which pulls an Alpine docker image and runs and spawns a container, from which we run an echo <message>. We'll be able to pass in the message parameter value during testing from Rabix.

  • Start by adding a local workspace from within Rabix My Projects --> Open a Project --> Select Folder
cwlVersion: v1.0
class: CommandLineTool
label: Alpine Docker Image
baseCommand:
- echo
inputs:
- id: message
type: string
inputBinding:
position: 1
outputs:
- id: std_out
type: stdout
requirements:
- class: DockerRequirement
dockerPull: alpine
stdout: output.txt
  • To test the workflow, click on the Test tab and for the message parameter, enter value Hello World
  • If all goes well, you should be able to see output similar to the following, indicating the workflow successfully completed
  • Detailed logs related to the workflow execution are stored at $HOME/rabix/Executions

You can use either of the following methods to view the log output:

  • WSL2 terminal using standard Linux commands, such as cat

For example, to view output written to file output.txt for the above job using Windows Explorer, replace the parameters and browser to the location using Windows Explorer.

\\wsl$\<linux distro name>
\home
\<Linux distro username>
\rabix
\Executions
\local
\<workflow name>
\<YYYY-MM-DD-HH-MM-SS>
\app-<YYYY-MM-DD-HH-MM-SS.SSS>
\root

2. Workflow with Embedded Python Script

The following workflow contains an inline/embedded python script, samplepy.py:

  • The script is created/populated and executed during the execution of the workflow inside a python:3 docker container

Proceeding as per the prior workflow:

  • Right click on rabix folder in the left hand pane and choose New CommandLineTool
cwlVersion: v1.0
class: CommandLineTool
label: Run an embedded Python script
hints:
DockerRequirement:
dockerPull: python:3
baseCommand: python

inputs:
script:
type: File
inputBinding:
position: 1
default:
class: File
basename: "samplepy.py"
contents: |-
var = "value of var"
print(var)

outputs:
results:
type: stdout

stdout: export.txt
  • Run the workflow
  • Check the output

Where to From Here?

Rabix and CWL have provided an additional perspective in the area of solution design/implementation. The way in which CWL is implemented promotes containerised, parameter driven components.

Having gained a basic knowledge of CWL workflows and establishing a local test environment, some possible areas I aim to cover off on in future are:

Learner. Interests include Cloud and Devops technologies.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store