1. Obtaining mdtmFTP image and running container¶

1.1. Requirements¶

System must have installed Docker, version 1.10 or higher. Get and install Docker from the project website at http://www.docker.com. Alternately, you may want to install Docker packages from your distribution’s package repository.

1.2. Get mdtmFTP Docker image¶

Download mdtmFTP Docker image either from the main repositry at Fermilab or from Docker Hub.

1.2.1. Fermilab repository¶

The mdtmFTP repository is at https://publicregistry.fnal.gov/. To access it, download the CA ceritifcate from https://bigdataexpress.fnal.gov/download/ca.crt.

The certificate has to be put in the appropriate path. For example, in Linux,

$ sudo mkdir -p /etc/docker/certs.d/publicregistry.fnal.gov
$ sudo cp ca.crt /etc/docker/certs.d/publicregistry.fnal.gov/

Then pull the image by running:

$ docker pull publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial

1.2.2. Docker hub¶

The mdtmFTP repository is at https://hub.docker.com/r/wenji/mdtm, and you can pull the image by running:

$ docker pull docker.io/wenji/mdtm:1.1.1-xenial

1.2.3. Docker image verification¶

Run docker images to check the image that you have pulled.

To verify the image is correct, run the mdtmFTP container:

$ docker run --rm publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial

This should shows messages about server configurations and current verion number.

libmdtm: mdtm_config generates /tmp/myserver.conf.
blocksize 4194304
direct 0
splice 0
monitor 0
mdtm_monitor_path /home/liangz/mdtm-test/data
libmdtm: mdtmconfig load mdtmconfig.json : succeed.
mdtmftp-ftp-server: 1.1.1

1.3. Configuring and running mdtmFTP server container¶

1.3.1. Create configuration files¶

For versions >= 1.1.1, mdtmFTP is configured by using a JSON configuration file mdtmconfig.json. Please refer to Appendix: creating server mdtmconfig.json for details.
For versions < 1.1.1, mdtmFTP is configured by using two configuration files:
- mdtmconfig.xml. Please refer to Appendix: creating server mdtmconfig.xml for details.
- server.conf. Please refer to Appendix: creating server.conf for details.

1.3.2. Choose an authentication method¶

mdtmFTP supports two types of authentication:

Username/password authentication,
GSI certificate-based authentication

Please choose an authentication method that suits your needs. See Managing credentials for details.

1.3.3. Create user account(s) and grant user access to mdtmFTP server¶

First, create user accounts within the container and in the host. See Appendix: adding users in docker environment for details.

Then, grant user access to mdtmFTP server:

For username/password authentication, you need create and configure a password file for mdtmFTP server. See Managing credentials: username/password authentication for details.
For GSI certificate-based authentication, you need create the mapping between user certificates and local user accounts by editing /etc/grid-security/grid-mapfile on the server side. See Managing credentials: GSI certificates for details.

1.3.4. Running mdtmFTP server¶

mdtmFTP server command syntax:

$ sudo docker run [OPTIONS] MDTMFTP_IMAGE \
              mdtm-ftp-server \
                  -control-interface <ip_address> \
                  -data-interface <ip_address> \
                  -password-file <passwd_file> \
                  -p <port_num> \
                  -l <log_file> \
                  -log-level <level>

See section on Running mdtmFTP server in docker environments for details.

1.4. Configuring and running mdtmFTP client¶

1.4.1. mdtmFTP client configuration file¶

For versions >= 1.1.1, mdtmFTP is configured by using a JSON configuration file mdtmconfig.json. Please refer to Appendix: creating client mdtmconfig.json for details.
For versions < 1.1.1, mdtmFTP is configured by using a XML configuration file mdtmconfig.xml. Please refer to Appendix: creating client mdtmconfig.xml for details.

1.4.2. Running mdtmFTP client¶

mdtmFTP client command syntax:

$ sudo docker run [OPTIONS] MDTMFTP_IMAGE \
              mdtm-ftp-client [-ipv6] -p <parallelism> <-splice> <src_url> <dst_url>

See section on Running mdtmFTP client in docker environments for details.