Establish two ssh connections to reach the submission node

Zuguang Gu ( z.gu@dkfz.de )

2024-03-31

For us, if we are outside of DKFZ network, we first connect to DKFZ network by VPN, then we only need one ssh connection to reach the submission node. However, in some other institutes, if you are outside of network of your institute, you might first need to connect to your institute’s server via ssh (server A), then on server A, you connect to the job submission server, again via ssh (server B), which results in two ssh connections.

Of course, you cannot establish two nested ssh connections, but we can do some tricks on server A to run a pseudo bjobs which actually runs the real bjobs on server B. Maybe you have already guessed, we need to, on server A, run bjobs on server B via ssh without interactively entering the password.

Note since you are not under the same network as the submission node, you cannot submit jobs, however, you can query the job status by functions like bjobs, brecent or monitor().

The following instructions tell you how to configure bsub_opt so that you can use bsub package on your own computer outside your institute’s network.

Step 1: on server A, generate an ssh key so that you can connect from server A to B without entering password interactively.

On server A, in the shell:

ssh-keygen

Select where you store the key file (let’s assume it is ~/.ssh/id_rsa). Copy the key to the server B:

ssh-copy-id -i ~/.ssh/id_rsa user@serverB

Test whether you need to enter password if you go from server A to B:

ssh user@serverB

Step 2: On your own computer, use the following configuration. Note here we set the login_node while not submission_node because the submission node is not your login node.

config_foo =  function(user = NULL) {
    bsub_opt$login_node = "serverA"
    bsub_opt$submission_node = NULL  # so that you cannot submit jobs!
    
    if(!is.null(user)) bsub_opt$user = user

    # You need to find out the value of `LSF_ENVDIR` and `LSF_SERVERDIR` on the submission node.
    # Check the "Bash environment" section in ./configure_bsub_package.html
    ssh_envir = "source /etc/profile; export LSF_ENVDIR=your_LSF_ENVDIR; export LSF_SERVERDIR=your_LSF_SERVERDIR"

    # We create pseudo `bjobs`/`bparam`/`bkill` which actually call the real commands on server B
    bsub_opt$ssh_envir = c(
        "source /etc/profile",
        qq("alias bjobs=\"ssh -i your_ssh_key @{bsub_opt$user}@serverB '@{ssh_envir};bjobs'\""),
        qq("alias bparam=\"ssh -i your_ssh_key @{bsub_opt$user}@serverB '@{ssh_envir};bparam'\""),
        qq("alias bkill=\"ssh -i your_ssh_key @{bsub_opt$user}@serverB '@{ssh_envir};bkill'\""),
        qq("alias bsub=\"ssh -i your_ssh_key @{bsub_opt$user}@serverB '@{ssh_envir};bsub'\"")
    )
    invisible(NULL)
}
config_foo("your-user-name")

Now you can query job status with bsub package from command-line (by bjobs()/brecent()/…) or with the interactive job monitor (by monitor()).