Connection and file transfer

Requirements

Accounts are created at the initiative of the project coordinators via the Gitlab IDCS plateform. The user must also be registered in the Ecole Polytechnique's XAJAM directory.

Connection to Cholesky

Connection to the Cholesky front-end is done via ssh and SSH key authentification from :

  • SSH gateways or VPN clients of the laboratories of the Ecole Polytechnique member of the mesocenter ;
  • SSH gateways of external partners who are members of the mesocenter.
 ssh -J <your_login>@<your_ssh_laboratory_gateway> <your_cholesky_login>@cholesky-login.mesocentre.idcs.polytechnique.fr

You can configure your SSH client to simplify access to cholesky from everywhere. Edit your ~/.ssh/config file and add an entry like this :

Host cholesky-login
    Hostname cholesky-login.mesocentre.idcs.polytechnique.fr
    ProxyJump <your_ssh_laboratory_gateway>
    User <your_cholesky_login>
    IdentityFile ~/.ssh/<your_ssh_private_key>

and you can connect with :

user@mycomputer:~$ ssh cholesky-login

Important

The resources of this interactive node are shared among all connected users. This node is only reserved for compilation and debugging of scripts.

Note

To access with Ecole Polytechnique's VPN only for Ecole Polytechnique laboratory staff.

SSH key authentification

SSH key authentification is the only method to access on Cholesky. You have to generate a pair of keys following IDCS recommendations and then you have to add your SSH public key on your GITLAB IDCS account.

Home directory and Working directory

As a user, you have access to two separate directories for your own usage :

  • the users directory (or home directory or homedir) : /mnt/beegfs/home/laboratory/$USER where you can store your source files (code, light data). This volume is limited to 10 Go per user. You can refer to this directory with the environment variable $HOME.

  • the working directory (or workdir) : /mnt/beegfs/workdir/$USER where you can store your binaries and data. This volume is limited to 200 Go per user. You can refer to this directory with the environment variable $WORKDIR.

Cholesky storage uses the BeeGFS technologie to share data among all the nodes in the cluster. The calculation nodes have acces to your $HOME, $WORKDIR and PROJECT directories. There is no need for copying your files and executables to another location for execution.

Warning

Hedin, Hopper, Montblanc and Tianlong nodes can access to Cholesky BeeGFS storage ($WORKDIR and $HOME) but only through the ethernet network (not high speed InfiniBand network).

  • the dedicated working directory for Hedin, Hopper, Montblanc and Tianlong nodes : /mnt/nas/workdir/$USER where you can store binaries and data. This volume is limited to 200 Go per user. You can refer to this directory with the environment variable $WORKDIR_NAS.

As a rule of thumb, all your executables, local libraries, environment configuration files should be located in your $HOME directory and all your simulation datasets and results should be in your $WORKDIR or $WORKDIR_NAS directory.

Danger

There is no backup of the data stored on Cholesky. Any removed file is lost for ever. It is the user’s responsibility to keep a copy of the contents of their home and workdir in a safe place.

Note

Quotas are applied on the use of space on the HOME, WORKDIR directories through the use of unix groups. For the HOME directory, it is the user's home group (eg. firstname.lastname) and for the WORKDIR directory, it is the user's home group prefixed with the word work_ (eg. work_firstname.lastname). You can quickly check your file quota with the command cholesky_quota_user. If your computations require more space on your $WORKDIR directory, you can ask the cholesky's support team for quota extension.

Project directory

As a project manager, you could request a shared PROJECT directory : /mnt/beegfs/project/project_name where all members could share source files, binaries, and data. This request must be justified to the executive committee (generally at the same time as the submission of a project). By default, no project directory is dedicated to a project.

Danger

There is no backup of the data stored on Cholesky. Any removed file is lost for ever. It is the project manager’s responsibility to keep a copy of the contents of a PROJECT directory in a safe place.

Note

Quota is applied on the use of shared PROJECT directory through the use of unix groups. It is the project name prefixed with the word proj_ (eg. proj_project_name). Project managers can check their PROJECT quota(s) in detail with the command cholesky_quota_project.

Data transfer

In order to transfer files from the cluster or to the cluster, you can use the shell command scp from your computer (GNU/Linux or MacOS X). From Windows computer you can use pscp.exe (SCP for Windows) available here.

The following command copies the directory my_dir on the workdir of the user username.

user@mycomputer:~$ ls my_dir/
file01.txt  file02.txt
user@mycomputer:~$ scp -r my_dir username@cholesky-login.mesocentre.idcs.polytechnique.fr:/mnt/beegfs/workdir/username
Enter passphrase for key '/home/username/.ssh/id_rsa': 
file01.txt                                                                       100%  132KB 132.3KB/s   00:00
file01.txt                                                                       100%  132KB 132.3KB/s   00:00
user@mycomputer:~$

The following command makes a local copy of the directory my_dir from the homedir of the user username.

user@mycomputer:~$ scp -r username@cholesky-login.mesocentre.idcs.polytechnique.fr:/mnt/beegfs/home/laboratory/username/my_dir .
Enter passphrase for key '/home/username/.ssh/id_rsa': 
user@mycomputer:~$

To transfer bigger files (when the tranfer is long enough), you should use the shell command rsync with the options --partial --progress. The --partial option allows rsync to keep partially transferred files and thus, in case an error occurs, to restart the tranfer from the partial files and not from scratch. The --progress tells rsync to print information showing the progress of the transfer. The following command copies the directory my_dir on the workdir of the user username.