Running mdtmFTP client¶
1. Before starting data transfer, open a new terminal and create the mdtmFTP working directory
/home/mdtmwork
:
$ mkdir -p /home/mdtmwork && cd /home/mdtmwork
- Set up the credentials
mdtmFTP supports two approaches for authentication and authority between servers and clients: password and certificates.
2.1. Password
In this case, every client uses its own preset password to login to servers and each server should have legal
clients’ passwords saved in a file specified in mdtmFTP server command line option -password-file
. In the
distributed mdtmFTP container image, there already exists a default user mdtmftp
. We are about to use that
default user to take on data transfer jobs.
2.2. Certificates
To use X.509 security or GSI mechanism, each user has to create its own user certificate and place the CA certificate
for the host running mdtmFTP server in the path /etc/grid-security
. And the protocol gsiftp
should be also
identified in the command line of mdtmFTP client.
- See Appendix: setting up X.509 security on DTN for details on how to create user certificate and place host CA certificate.
- Running mdtmFTP client
Run the client application from the container following mdtmFTP client command syntax:
$ sudo docker run -ti --rm \
--net=host \
-v `pwd`:/mdtmwork \
publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial \
mdtm-ftp-client [-ipv6] -p <parallelism> -splice <src_url> <dst_url>
Those parameters are for:
-ipv6
enables IPv6 for data transfer. The default is IPv4.-p <parallelism>
specifies the number of parallel data streams.-splice
turns on “splice” feature.src_url
specifies source URL.dst_url
specifies destination URL.
There are two modes to run mdtmFTP client applications: client-server and third party. For the client-server, the mdtmFTP client transfer data between the remote mdtmFTP server and the local system; for the third party mode, the mdtmFTP client controls data transfer job between two remote mdtmFTP servers.
3.1 Client-Server mode
For example, the below command download folder from remote system to the local path /tmp
.
$ sudo docker run -ti --rm \
--net=host \
-v `pwd`:/mdtmwork \
publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial \
/usr/local/mdtmftp/HEAD/bin/mdtm-ftp-client -vb -p 4 ftp://mdtmftp:123456@bde1.fnal.gov:5050/data1/linux-4.13.4/ file:///tmp/
3.2 Third Party mode
For example, the below command download folder from remote system bde1 to another remote system bde2 in the path /tmp
.
$ sudo docker run -ti --rm \
--net=host \
-v `pwd`:/mdtmwork \
publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial \
/usr/local/mdtmftp/HEAD/bin/mdtm-ftp-client -vb -p 4 ftp://mdtmftp:123456@bde1.fnal.gov:5050/data1/linux-4.13.4/ ftp://mdtmftp:123456@bde2.fnal.gov:5050/data2/tmp/