.. _server-command-line: ============================================= Running mdtmFTP server in docker environments ============================================= mdtmFTP server command syntax ----------------------------- .. code-block:: shell mdtm-ftp-server \ -control-interface \ -data-interface \ -password-file \ -p \ -c Among those options, * ``-control-interface `` specifies a control interface for the server. * ``-data-interface `` specifies a data interface for the server. * ``-password-file `` specifies a password file for username/password authentication. * ``-p `` specifies a port that mdtmFTP server listens on. * ``-c `` specifies a configuration file to set data transfer parameters. Required Linux capabilities for mdtmFTP server ---------------------------------------------- For the purpose of performing permission checks, traditional UNIX implementations distinguish two categories of processes: `privileged` processes (whose effective user ID is 0, referred to as superuser or root), and `unprivileged` processes (whose effective UID is nonzero). Privileged processes bypass all kernel permission checks, while unprivileged processes are subject to full permission checking based on the process's credentials (usually: effective UID, effective GID, and supplementary group list). Starting with kernel 2.2, Linux divides the privileges traditionally associated with superuser into distinct units known as `capabilities`, which can be independently enabled and disabled. Running mdtmFTP server requires four capabilities: * ``CAP_SYS_NICE``, to bind threads to cores. * ``CAP_IPC_LOCK``, to lock memory. * ``CAP_SYS_RESOURCE``, to increase pipe capacity. * ``CAP_SYS_ADMIN``, to increase the maximum number of open files. Running mdtmFTP server in docker environments --------------------------------------------- Running mdtmFTP server in docker environments typically takes this form: .. code-block:: console $ sudo docker run \ [Name setting] \ [Operation mode settings] \ [User settings] \ [Network settings] \ [Volume settings] \ [Capabilities settings] \ [Security settings] \ [mdtmFTP docker image] \ [Path to mdtmFTP]mdtm-ftp-server [ARG...] Among these settings: * ``[Name setting]``. Sepcify a name for the container, e.g., ``--name mdtmftp-server``. * ``[Opertion mode settings]``. It is typically set to ``-ti`` to run the container in the foreground. * ``[User setting]``. Set the user for the container, e.g., ``-u mdtmftp-admin``. * ``[Network settings]``. It is typically set to ``--net=host`` to set the host's network stack inside the container. * ``[Volume setting]``. Set up bindmount volumes that link folders from inside the container to folders on the host machine. * ``[Capabilities settings]``. By default, Docker containers are unprivileged. You need to add the following Linux capabilitis to the container so that mdtmFTP server can run properly: - IPC_LOCK - SYS_NICE - SYS_ADMIN - NET_ADMIN - SYS_RESOURCE * ``[Security setting]``. For Docker 1.10 and 1.11, ``--security-opt seccomp:unconfined`` is required to add Linux capabilities. * ``[mdtmFTP docker image]``. Specify a particular docker image you want to run. With username/password authentication ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Here are the typical preparation steps before launching a mdtmFTP server with username/password authentication: * Open a new terminal and create the mdtmFTP working directory, e.g., ``/home/mdtmwork``. * Create mdmtFTP configuration files in the working directory * Add data transfer users in the docker environment and set the appropriate permissions. * Save the updated docker image if necessary. * Create a password file, e.g., ``passfile``, in the mdtmFTP working directory * Create a user account, e.g., ``mdtmftp-admin``, to run mdtmFTP docker image in host. * Prepare data transfer folders in host, e.g., ``/data1``. Set appropriate permissions to allow data transfer users to access this folder. Then, run the mdtmFTP server. Typically, the following folders in the host will be bindmount into the container: * mdtmFTP workig directory in the host, e.g., ``/home/mdtmwork`` --> ``/mdtmwork`` in the container. * Data transfer folders in the host, e.g,, ``/data1`` --> ``/data1`` in the container. .. code-block:: console $ cd /home/mdtmwork $ sudo docker run --name mdtmftp-server \ -it --rm \ -u mdtmftp-admin \ --net=host \ -v `pwd`:/mdtmwork \ -v /data1:/data1 \ --cap-add=ALL --cap-add=IPC_LOCK --cap-add=SYS_NICE \ --cap-add=SYS_ADMIN --cap-add=NET_ADMIN --cap-add=SYS_RESOURCE \ --security-opt seccomp:unconfined \ mdtmftp:1.1.1-xenial-updated \ /usr/local/mdtmftp/1.1.1/sbin/mdtm-ftp-server \ -data-interface 131.225.2.29 \ -password-file passfile \ -p 5050 \ -l mdtmftp.log \ -log-level all If everything is all right, it then shows the outputs like, .. code-block:: console [32]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83b5081558 [31]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83bd081558 [33]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83b1081558 [34]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83a9081558 mdtm_schedule_threads: device=enp4s0f0 cpu=0 mdtm_schedule_threads: device=enp4s0f0 cpu=1 mdtm_schedule_threads: device=nvme0n1 cpu=2 mdtm_schedule_threads: device=nvme0n1 cpu=3 And you can watch the log file in a different terminal, .. code-block:: console $ sudo tail -f /home/mdtmwork/mdtmftp.log With GSI certificate-based authentication ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Here are the typical preparation steps before launching a mdtmFTP server with GSI certificate-based authentication: * Open a new terminal and create the mdtmFTP working directory, e.g., ``/home/mdtmwork``. * Create mdmtFTP configuration files in the working directory * Add data transfer users in the docker environment and set the appropriate permissions. * Save the updated docker image if necessary. * Create ``/etc/grid-security`` in the host to grant access to mdtmFTP server. * Create a user account, e.g., ``mdtmftp-admin``, to run mdtmFTP docker image in host. * Prepare data transfer folder(s) in host, e.g., ``/data1``. Set appropriate permissions to allow data transfer users to access the folder(s). Then, run the mdtmFTP server. Typically, the following folders in the host will be bindmount into the container: * mdtmFTP workig directory in the host, e.g., ``/home/mdtmwork`` --> ``/mdtmwork`` in the container. * ``/etc/grid-security`` in the host --> ``/etc/grid-security`` in the container. * Data transfer folders in the host, e.g,, ``/data1`` --> ``/data1`` in the container. .. code-block:: console $ sudo docker run --name mdtmftpserver \ -it --rm \ -u mdtmftp-admin \ --net=host \ -v /etc/grid-security:/etc/grid-security \ -v `pwd`:/mdtmwork \ -v /data1:/data1 \ --cap-add=ALL --cap-add=IPC_LOCK --cap-add=SYS_NICE \ --cap-add=SYS_ADMIN --cap-add=NET_ADMIN --cap-add=SYS_RESOURCE \ --security-opt seccomp:unconfined \ mdtmftp:1.1.1-xenial-updated \ /usr/local/mdtmftp/1.1.1/sbin/mdtm-ftp-server \ -data-interface 131.225.2.29 \ -password-file passfile \ -p 5050 \ -l mdtmftp.log \ -log-level all If everything is all right, it then shows the outputs like, .. code-block:: console [32]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83b5081558 [31]mdtmthread/mdtm_io_thread.c 459: mdtm_net_io_thread: IO thread Ready on device enp4s0f0: evenq=0x7f83bd081558 [33]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83b1081558 [34]mdtmthread/mdtm_io_thread.c 336: mdtm_disk_io_thread: IO thread Ready on device nvme0n1: evenq=0x7f83a9081558 mdtm_schedule_threads: device=enp4s0f0 cpu=0 mdtm_schedule_threads: device=enp4s0f0 cpu=1 mdtm_schedule_threads: device=nvme0n1 cpu=2 mdtm_schedule_threads: device=nvme0n1 cpu=3 And you can watch the log file in a different terminal, .. code-block:: console $ sudo tail -f /home/mdtmwork/mdtmftp.log