Uppsala Multidisciplinary Center for Advanced Computational Science

Running LifeScope on UPPMAX

Table of contents:

Background

First off, LifeScope is not intended to be run in a multiuser HPC environments. The perfect LifeScope environment would be a research group's own private cluster, where you have full administrative access to the computer and the file system (the hard drives). Because of this, some tips and tricks are needed for running LifeScope efficiently in an HPC environment.

Set up and run LifeScope

Ok, here are the instructions to get LifeScope running. Skip the steps that you have already done. This guide is written for LifeScope 2.5.1.

Choose a random port number for your server

Since we only have 2 login nodes, we want to avoid 2 servers running on the same port. The port number of your server is permanent and it's probably hard to change it once LifeScope is installed, but it doesn't matter wich number you choose, as long as it's not the same as anyone else. So pick any number between 1024 and 65535 and you should be good to go. If you're low on imagination you can get a random number by pressing this link.

Remember this number, since you will need it everytime you connect to the server. It will be printed to the screen each time you start the server, so it's not hard to get reminded of it if you would forget it.

Connect to uppmax with port forwarding

Use the terminal in Linux/OSX, or MobaXterm for windows (www.mobaxterm.mobatek.net). The port forwarding will be active for as long as this connection is active, and will stop as soon as you terminate the session (i.e. close the window). Now you will have to choose which login node you want to install LifeScope on; milou1 or milou2. You will be connecting the this node every time you want to run LifeScope.

$ ssh -L [port number]:localhost:[port number] [username]@milou[1 or 2].uppmax.uu.se

Ex:

$ ssh -L 9998:localhost:9998 dahlo@milou1.uppmax.uu.se
or
$ ssh -L 15312:localhost:15312 dahlo@milou2.uppmax.uu.se 

where the port number is the random number you picked in the beginning.

Download the installation files

Go to http://www.lifetechnologies.com/se/en/home/technical-resources/software-downloads/lifescope-genomic-analysis-software.html and download the latest version of LifeScope. This guide is written for 2.5.1.

The files to download are

  • LifeScope™ Software v2.5.1
  • LifeScope™ Software v2.5.1 install script

Place them both in the same folder somewhere at uppmax.

Install LifeScope

First, we have to get the IP of the node you are installing LifeScope on. To do this, type

$ ifconfig eth0 

and find the IP of the node. It should be on the 2nd row, called inet addr

eth0 Link encap:Ethernet HWaddr 31:6A:A7:2B:3A:11
inet addr:130.18.238.20 Bcast:130.238.139.255 Mask:255.255.252.0
inet6 addr: ef80::3ee2:a1ff:fe1b:2a18/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:10089178457 errors:0 dropped:8156 overruns:0 frame:0
TX packets:12890635577 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:60720073404917 (55.2 TiB) TX bytes:44584699691158 (40.5 TiB)

Remember this IP, you will need it during the installation.

Go to the directory with the downloaded file in it and start the installation

$ sh install.sh

Here is an example of choices that worked for us:

Do you want to continue with this installation (y/n) [Y]? Y
 
Please select the operation you would like to perform (Q/q to quit) [1]: 1
 
If you are the LifeScope Admin user, do you want to continue with this installation (y/n) [Y]? Y
 
Select the type of LifeScope installation to be performed(1, 2 or 3)[1]: 1
 
Please enter the INSTALL location for LifeScope (Q/q to quit) [/share/apps]: /proj/b2013023/apps  
 
Please enter the REFERENCE FILE location for LifeScope (Q/q to quit) [/data/results/reference]: /proj/b2013023/apps/lifescope/ref 
 
Please enter the READS location for LifeScope (Q/q to quit) [/data/results/reads]: /proj/b2013023/apps/lifescope/reads 
 
Please enter the PROJECTS location for LifeScope (Q/q to quit) [/data/results/projects]: /proj/b2013023/apps/lifescope/projects 
 
Please enter the BAMS location for LifeScope (Q/q to quit) [/data/results/bams]: /proj/b2013023/apps/lifescope/bams 
 
Please enter the SCRATCH location for LifeScope (Q/q to quit) [/scratch]: 
 
Do you still want to use this location (y/n/q) [Y]? Y
 
(this is where you need the IP we checked earlier)
Please enter the IP address or Fully Qualified Domain Name (FQDN) for the local LifeScope Server (Q/q to quit): 130.18.238.20 
 
(this is where you enter the random number you picked earlier)
Please enter the Web URL Port address to be used to access the LifeScope Server (Q/q to quit) [9998]: 15312
 
Select the AUTHENTICATION REALM to be used with LifeScope (1, 2 or 3)[1]: 1
 
Please enter the QUEUE name to be used with LifeScope (Q/q to quit): node 
 
Please enter the NUMBER OF NODES available on the workstation. (Q/q to quit) [4]: 16
 
Available server cores per job for analysis:  (Q/q to quit) [11]: 16 
 
Available memory per job for analysis:  (Q/q to quit) [22]: 120
 
Do you want to change any of these entries (y/n/q) [N]? N
 
Do you want to continue installing the LifeScope package (y/n) [Y]? Y 

Look at the message at the end of the installation, about editing you PATH variable. Add this to the end of your ~/.bash_profile by running the following (remember to change the path in the command to the one you got in your message):

echo 'PATH=/proj/b2013023/apps/lifescope/bin:$PATH; export PATH' >> ~/.bash_profile 

This will make sure that your PATH variable is modified everytime you log in. To avoid logging out and then back in again, we just run the command in the terminal (remember to change the path in the command to the one you got in your message) .

PATH=/proj/b2013023/apps/lifescope/bin:$PATH; export PATH 

Modify the configuration

Now comes the tricky part where we have to modify the configuration files for them to work on uppmax.

We start with modifying the ui/LifeScope.html (relative your installation path, /proj/b2013023/apps/lifescope in my case). Change the 2 occurences of IP addresses in the file, change both to 127.0.0.1 . We have added comments in the code below to indicate line number (in 2.5.1), don't write these to the file, just edit the indicated options.

...
# line 61
<area shape="rect" coords="4,4,123,41" href="http://127.0.0.1:15312/LifeScopeClient.jnlp" target="_self" alt="Analysis Portal"> 
...
# line 65 
<area shape="rect" coords="5,5,123,41" href="http://127.0.0.1:15312/LifeScopeAdmin.jnlp" target="_self" alt="Admin Portal"> 
...

Next up is ui/LifeScopeClient.jnlp and ui/LifeScopeAdmin.jnlp. Change all IP addresses to 127.0.0.1. We have added comments in the code below to indicate line number (in 2.5.1), don't write these to the file, just edit the indicated options.

...
# line 2
<jnlp spec="1.0+" codebase="http://127.0.0.1:15312/" href="LifeScopeAdmin.jnlp">
...
# line 6
<homepage href="http://127.0.0.1:15312/"/> 
...
# line 29
<argument>server.address=http://127.0.0.1:15312/lifescope/</argument> 
...

The next file is etc/template/cluster.properties . We have added comments in the code below to indicate line number (in 2.5.1), don't write these to the file, just edit the indicated options.

...
# line 6
TORQUE.template.name=SLURM_TEMPLATE.ftl 
...
# line 31, add YOUR project id here
TORQUE.command.jobSubmit=(sbatch -A b2013023) 
...
# line 50
TORQUE.command.jobRemove=scancel 

Next file is a file that doen't exist yet. Create the file etc/template/SLURM_TEMPLATE.ftl and copy-paste the following to it:

#!/bin/bash -l
## Job name
#SBATCH -J ${jobName}-Cores${numberOfProcessorsPerNode}-mem${memoryStr}
## Declare job re-runnable
<#if reRunnable>#SBATCH --requeue</#if>
## Output files
<#if outputLog??>#SBATCH -o '${outputLog}'</#if>
<#if errorLog??>
#SBATCH -e '${errorLog}'
<#else>
#SBATCH -e '${outputLog}'
</#if>
## Resources
##SBATCH -t 24:00:00
#SBATCH -t ${wallTimeStr}
## Resources FOR DEBUG
# -l walltime=${wallTimeStr},nodes=${numberOfNodes}:ppn=${numberOfProcessorsPerNode},mem=${memoryStr}
<#if (memoryStr?length == 4 )>
<#assign UPPMAX_mem = memoryStr?substring(0,2) >
<#else>
<#assign UPPMAX_mem = memoryStr?substring(0,1) >
</#if>
 
<#if ((numberOfProcessorsPerNode > 1) && (UPPMAX_mem?number > 128))>
#SBATCH -p node -N ${numberOfNodes} -n ${numberOfProcessorsPerNode} -C fat
</#if>
<#if ((numberOfProcessorsPerNode > 1) && (UPPMAX_mem?number < 128))>
#SBATCH -p node -N ${numberOfNodes} -n ${numberOfProcessorsPerNode} -C thin
<#elseif numberOfProcessorsPerNode == 1 >
<#if ( UPPMAX_mem?number < 8 ) >
#SBATCH -p core -n 1
#mem=${memoryStr}
<#else>
#SBATCH -p node -n ${numberOfProcessorsPerNode}
#mem=${memoryStr}
</#if>
</#if>
<#if depends??>#SBATCH -d afterany:${depends}</#if>
 
<#include "common.ftl"> 

Starting the server

Now when all the editing is done, it's time to start the server!

$ lscope-server.sh start 

It should now be possible to open a web browser on your local computer and connect to the server using the address:

http://localhost:15312/LifeScope.html 

Remember to change the port number to the number you picked! Chrome asked me if i would like to download the jnpl file, and i did, and then run it from my computer. Everything started and workd as it should.

The first thing you sould do is log in to the admin interface and change the default password. The default login is:

user: lifescope
pass: lifescope

Epilogue

Since the IP of the node is hardcoded in the config files, it's important that you always start the LifeScope server on the same node every time. If you picked milou2 in the beginning, milou2 is the only node that will run LifeScope for you. The server will be running on the node until the node is restarted, or you manually shut the server down. If you have problems connecting to the server in the future, it could be that the login node was restarted and then you will have to connect to the node and start the server again. Same command as last time:

$ lscope-server.sh start

The port forwarding command is equally important. If you don't have a terminal open which is forwarding your port, you won't be able to connect to the server. Open a terminal, connect to the node, and minimize the terminal and start using the server in your web browser. Same command as last time:

$ ssh -L [port number]:localhost:[port number] [username]@milou[1 or 2].uppmax.uu.se

Ex:

$ ssh -L 9998:localhost:9998 dahlo@milou1.uppmax.uu.se
or
$ ssh -L 15312:localhost:15312 dahlo@milou2.uppmax.uu.se