Running mdtmFTP server¶
- Open a new terminal and create the mdtmFTP working directory
/home/mdtmwork
:
$ mkdir -p /home/mdtmwork && cd /home/mdtmwork
- Create mdmtFTP configuration files:
For versions >= 1.1.1, mdtmFTP is configured by using a JSON configuration file
mdtmconfig.json
. Please refer to Appendix: creating server mdtmconfig.json for details.For versions < 1.1.1, mdtmFTP is configured by using two configuration files:
mdtmconfig.xml
. Please refer to Appendix: creating server mdtmconfig.xml for details.server.conf
. Please refer to Appendix: creating server.conf for details.
- Set up the credentials
mdtmFTP supports two types of authentication method:
- Username/password authentication
- `GSI`_ certificate-based authentication.
3.1. Username/Password
In this case, every client uses its own preset password to login to servers and each server should have legal
clients’ passwords saved in a file specified in mdtmFTP server command line option -password-file
. In
mdtmFTP container image, a default user mdtmftp
has been configured. The password of the user mdtmftp
,
e.g. 123456
, can be added to the password file passfile
by running
$ docker run -u mdtmftp -v `pwd`:/mdtmwork publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial
/usr/local/mdtmftp/HEAD/sbin/globus-gridftp-password-nostdin >> passfile 123456
To create other users except mdtmftp
, login to the mdtmFTP container
$ docker run -ti --rm publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial /bin/bash
and then create users via useradd
command. Then add the password, i.e. 123456
, to the password file.
$ /usr/local/mdtmftp/HEAD/sbin/globus-gridftp-password-nostdin >> passfile 123456
3.2. Certificates
For better security, X.509 security, GSI, is used in mdtmFTP. Essentially, the host certificate as well as the CA certificate
should be created and put to the path /etc/grid-security
.
- See Appendix: setting up X.509 security on DTN for details on how to create host and CA certificates.
- Run the mdtmFTP server
Start the mdtmFTP server by running,
$ sudo docker run --name mdtmftp \
-it --rm \
-u root \
--net=host \
-v `pwd`:/mdtmwork \
-v /data1:/data1 \
--cap-add=ALL --cap-add=IPC_LOCK --cap-add=SYS_NICE \
--cap-add=SYS_ADMIN --cap-add=NET_ADMIN --cap-add=SYS_RESOURCES \
--security-opt seccomp:unconfined \
--privileged \
publicregistry.fnal.gov/bigdata_express/mdtmftp:1.1.1-xenial \
/usr/local/mdtmftp/HEAD/sbin/mdtm-ftp-server -data-interface 131.225.2.29 -password-file passfile -p 5050 -l mdtmftp.log -log-level all
In this example, the local data path /data1
is bind mounted into the /data1
in the container.
-v /data1:/data1
Running mdtmFTP server requires capabilities:
CAP_SYS_NICE
, to bind threads to cores.CAP_IPC_LOCK
, to lock memory.CAP_SYS_RESOURCE
, to increase pipe capacity.CAP_SYS_ADMIN
, to increase the maximum number of open files.
and security and privilege setup,
--security-opt seccomp:unconfined \
--privileged \
mdtmFTP server command syntax:
mdtm-ftp-server \
-control-interface <ip_address> \
-data-interface <ip_address> \
-password-file <passwd_file> \
-p <port_num> \
-l <log_file> \
-log-level <levels>
Among those options,
-control-interface <ip_address>
specifies a control interface for the server.-data-interface <ip_address>
specifies a data interface for the server.-password-file <passwd_file>
specifies a password file for username/password authentication.-p <port_num>
specifies a port that mdtmFTP server listens on.-l <log_file>
specifies a log file.-log-level <levels>
specifies the level of loggin.
If everything is all right, it then shows the output like,
[32]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83b5081558
[31]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83bd081558
[33]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83b1081558
[34]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83a9081558
mdtm_schedule_threads: device=enp4s0f0 cpu=0
mdtm_schedule_threads: device=enp4s0f0 cpu=1
mdtm_schedule_threads: device=nvme0n1 cpu=2
mdtm_schedule_threads: device=nvme0n1 cpu=3
And you can watch the log file in a different terminal,
$ sudo tail -f /home/mdtmwork/mdtmftp.log