======================
mdtmFTP Docker release
======================

About mdtmFTP
=============

To address challenges in high performancedata movement for large-scale
science, the Fermilab network research group has developed mdtmFTP, a
high-performance data transfer tool to optimize data transfer on
multicore platforms. mdtmFTP has a number of advanced features.

* First, it adopts a pipelined I/O design.  Data transfer tasks are
  carried out in a pipelined manner across multiple cores. Dedicated
  threads are spawned to perform network and disk I/O operations in
  parallel.

* Second, mdtmFTP uses multicore-aware data transfer middleware (MDTM)
  to schedule an optimal core for each thread, based on system
  configuration, to optimize throughput across the underlying
  multicore core platform.

* Third,  mdtmFTP  implements  a  large  virtual  file  mechanism  to
  efficiently  handle lots-of-small-files (LOSF) situations.

* Finally,   mdtmFTP   unitizes   optimization   mechanisms   such
  as   zero   copy, asynchronous  I/O,  batch  processing,  and
  pre-allocated  buffer  pools,  to  maximize performance.

This document describes using Docker container releases of mdtmFTP.
For manuals of source code or Singuarity releases of mdtmFTP, please
visit the project website: http://mdtm.fnal.gov.

Docker releases of mdtmFTP are at https://hub.docker.com/r/wenji/mdtm/.

For technical details about mdtmFTP, please refer to the paper:

Liang Zhang,Wenji Wu, Phil DeMar, Eric Pouyoul: `mdtmFTP and its
evaluation on ESNET SDN testbed`. Future Generation
Comp. Syst.79:199-204 (2018)


Intended audience
=================

This manual is intended for users and system administrators
responsible for installing, running, and managing DTNs.  The manual
assumes familiarity with multicore and `DTN concepts <https://fasterdata.es.net/science-dmz/DTN/>`_.


Contacts
========

* Wenji Wu (wenji@fnal.gov)
* Liang Zhang (liangz@fnal.gov)
* Phil DeMar (demar@fnal.gov)


Acknowledgements
================

mdtmFTP uses several `Globus <http://toolkit.globus.org/>`_
modules for rapid prototyping.  We sincerely thank Globus folks at
Argonne National Laboratory and University of Chicago.

Here is a list of Globus modules that mdtmFTP uses:

* GridFTP protocol module
* Globus xio module
* Globus security module
* Globus user interface

As of January 2018, Globus Toolkit is retired, and now it is succeeded by software maintained by
`Grid Community Forum <https://gridcf.org/>`_ (GCF).


.. toctree::
   :maxdepth: 2
   :numbered:
   :caption: Contents:

   docker-release.rst
   managing-credentials.rst
   server-command-line.rst
   client-command-line.rst
   support-for-ipv6.rst
   data-transfer-examples-docker.rst
   appendix-add-user.rst 
   appendix-configure-gsi.rst
   appendix-configure-server-json.rst
   appendix-configure-server-xml.rst
   appendix-configure-server-conf.rst
   appendix-configure-client-json.rst
   appendix-configure-client-xml.rst
   appendix-host-ca.rst

..
   Indices and tables
   ==================

   * :ref:`genindex`
   * :ref:`modindex`
   * :ref:`search`