How does MDTM work?

In operation, an MDTM application spawns three types of threads: management threads, Disk/storage I/O threads, network I/O threads:

  • Management threads execute and implement user request handling- and management-related functions and modules, including Data transfer applications’ native function & modules, Data transfer service interface, Request/data preprocessing, and Thread/flow management. 

  • Disk/storage I/O threads and network I/O threads execute the function of Data access and transmission.  MDTM uses dedicated threads to perform network and disk I/O operations (i.e., I/O-centric data transfer architecture).

    • Disk/storage reader thread. A reader thread reads data from disks or storage systems.

    • Disk/storage writer thread. A writer thread writes data to disks or storage systems.

    • Network sender threads. A sender thread sends data to networks via NIC.

    • Network receiver threads. A receiver thread receives data from network via NIC.

 

An MDTM data transfer application accesses MDTM middleware services explicitly via APIs.

 

In operation, an MDTM middleware daemon will be launched. It will provide two types of services:

  • Query service. A data transfer application queries system configuration and status through MDTM middleware, which can be used to determine number of threads that will be spawn and number of flows that a thread will handle for a data transfer request.

  • Scheduling service. MDTM middleware schedules and assign system resources based on the needs and requirements of data transfer applications (i.e., data transfer-centric scheduling). The MDTM middleware scheduling also takes into account other factors, including NUMA topology, I/O locality, and QoS.

 

  • Last modified
  • 07/18/2016