Installing HTcondor in Ubuntu 16.04

Warning: I will write this post at the same time I learn, do not expect expertise.

Homepage of HTCondor project: HTcondor

In Ubuntu 16.04 Xenial, condor (htcondor since 1998), is available from official repositories. So it is enough to sudo apt-get install htcondor

At installation it triggers a gui (ncurses) configuration specific for Debian. We have the option to install condor in this computer self-contained, or configure it to join an existing cluster.

Because I am just learning by doing, and I have no cluster configured yet, I choose the self-contained option. The configuration exits at this point, and condor communications are set through a local address.

We can check and modify the configuration through some files:
/etc/condor/condor_config.local,/etc/condor/condor_config and in /etc/condor/config.d/ where there is a 00debconf file.
The priority is in reverse order, we can override any settings from condor_config.local.

After installation, we just run:
sudo condor_master and check that it is running with:
ps -aux | grep condor

We are now ready to submit some test jobs to our machine:
We can follow the official quick start guide using a bash file or instead use a good old “Hello World” in c++. We create the file hola.cpp:

#include <iostream>
int main(){
  std::cout << "Hola!" << std::endl;
}

We will compile with g++ hola.cpp -o hola.out.

The submission file for condor file:

# Unix submit description file
# hola.sub -- hello world in cpp

universe                = vanilla
executable              = hola.out
log                     = hola.log
output                  = outfile.txt
error                   = errors.txt
should_transfer_files   = Yes
when_to_transfer_output = ON_EXIT
queue

Note though, that this will work because our htcondor cluster is just one computer. Some executables would need to be relinked to be run in other computers.
This is where the Universes have something to say.
The standard universe requires that the executable is compiled using: condor_compile g++ hola.cpp -o hola.out
But sadly, there is no condor_compile script in the Ubuntu installation, so, we cannot use standard universe.

But, well we can use the default vanilla universe with our hola.out, because we know that it will have no problems with linking in the same machine.

For submission to the vanilla universe, we type in the directory where the executable and the submission file are: condor_submit hola.sub

We will get an OK message, and soon two files: error.txt (empty) and outfile.txt (with Hola!) files will appear.

If we want to submit 12 jobs at the same time, we can instead use this submission file:

# Multiple
universe                = vanilla
executable              = hola.out
log                     = hola.log.$(Process)
output                  = outfile.txt.$(Process)
error                   = errors.txt
should_transfer_files   = Yes
when_to_transfer_output = ON_EXIT
queue 12

This will generate 12 files, using the 12 CPU’s of my computer in parallel.

In the next post I will learn (and write at the same time) about how to join an existing cluster managed from a different computer.