1. Obtaining and running mdtmFTP¶

There are different ways to obtain and run mdtmFTP:

to build, and install MDTM library and mdtmFTP programs from sources, or
to use Docker container images, or
to use Singularity container images.

This document discusses the source release. If you are looking for information on using Docker or Singuarity images, please find the manuals at http://mdtm.fnal.gov.

1.1. Requirements¶

mdtmFTP has been tested on Linux systems. The system must have a Linux kernel later than 3.12.23.

1.2. Building mdtm and mdtmFTP¶

To build mdtm library and mdtmFTP, the following tools are required:

C++ and C compilers.
GNU Autotools
Development packages for OpenSSL, numactl, hwloc, lvm2 mosquitto, json-c, krb5, ncurses.

1.2.1. Install dependencies¶

To install build tools and dependencies on RHEL and derivatives:

$ sudo yum install \
    gcc gcc-c++ make autoconf automake libtool m4 \
    numactl-devel hwloc-devel lvm2-devel libblkid-devel \
    openssl-devel mosquitto-devel json-c-devel \
    krb5-devel libtool-ltdl-devel ncurses-devel \
    libuuid-devel rrdtool-devel

On Debian and Ubuntu:

$ sudo apt install \
    gcc g++ make autoconf automake libtool m4 \
    libnuma-dev libhwloc-dev liblvm2-dev libblkid-dev \
    libssl-dev libmosquitto-dev libjson-c-dev \
    libkrb5-dev libltdl-dev libncurses5-dev \
    uuid-dev librrd-dev

1.2.2. Install mdtm library¶

Create a building directory:

$ mkdir ~/build
$ cd ~/build

Download mdtm-1.0.5.tar.gz software package from the project website, and extract it in ~/build.
Build and install mdtm library:

$ tar –xvz mdtm-1.0.5.tar.gz
$ cd mdtm-1.0.3
$ ./autogen.sh
$ ./configure --prefix=$prefix
$ make
$ sudo make install

1.2.3. Install mdtmFTP¶

Download mdtmFTP software package from the website.
Extract the downloaded mdtmftp-1.1.1.tar.gz archive to ~/build.
Build and install mdtmFTP:

$ tar xvfz mdtmftp-1.1.1.tar.gz
$ cd ~/build/mdtm
$ ./autogen.sh
$ ./configure --prefix=$prefix \
     PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:$PKG_CONFIG_PATH
$ make && make install

Two major mdtmFTP binary programs will be generated: $prefix/bin/mdtm-ftp-client - the mdtmFTP client executable, and $prefix/sbin/mdtm-ftp-server - the mdtmFTP server exectuable.

1.3. Configuring and running mdtmFTP server¶

1.3.1. Decide whether to run mdtmFTP server as root or non-root¶

You can run mdtmFTP server either as root, or as non-root. If you choose to run the server as non-root, please add the needed capabilities to the server so that it can function correctly.

Please see Running mdtmFTP server with or without root privilege for details on how to run mdtmFTP server as a non-root user.

1.3.2. Create configuration files¶

For versions >= 1.1.1, mdtmFTP is configured by using a JSON configuration file mdtmconfig.json. Please refer to Appendix: creating server mdtmconfig.json for details.
For versions < 1.1.1, mdtmFTP is configured by using two configuration files:
- mdtmconfig.xml. Please refer to Appendix: creating server mdtmconfig.xml for details.
- server.conf. Please refer to Appendix: creating server.conf for details.

1.3.3. Choose an authentication method¶

mdtmFTP supports two types of authentication:

Username/password authentication,
GSI certificate-based authentication

Please choose an authentication method that suits your needs. See Managing credentials for details.

1.3.4. Create local account(s) and grant user access to mdtmFTP server¶

First, create local user accounts in the host where mdtmFTP server will run.

Then, grant user access to mdtmFTP server:

For username/password authentication, you need create and configure a password file for mdtmFTP server. See Managing credentials: username/password authentication for details.
For GSI certificate-based authentication, you need create the mapping between user certificates and local user accounts by editing /etc/grid-security/grid-mapfile on the server side. See Managing credentials: GSI certificates for details.

1.3.5. Running mdtmFTP server¶

mdtmFTP server command syntax:

$ mdtm-ftp-server \
   -control-interface <ip_address> \
   -data-interface <ip_address> \
   -password-file <passwd_file> \
   -p <port_num> \
   -c <server.conf>

See section on Running mdtmFTP server for details.

1.4. Configuring and running mdtmFTP client¶

mdtmFTP client can be launched either as root or as non-root.

1.4.1. mdtmFTP client configuration file¶

For versions >= 1.1.1, mdtmFTP is configured by using a JSON configuration file mdtmconfig.json. Please refer to Appendix: creating client mdtmconfig.json for details.
For versions < 1.1.1, mdtmFTP is configured by using a XML configuration file mdtmconfig.xml. Please refer to Appendix: creating client mdtmconfig.xml for details.

1.4.2. Running mdtmFTP client¶

mdtmFTP client command syntax:

$ mdtm-ftp-client [-ipv6] -p <parallelism> [-splice] source_url destination_url

See section on Running mdtmFTP client for details.