8. Appendix: creating server mdtmconfig.xml

mdtmconfig.xml configures a mdtmFTP server’s MDTM-related parameters. It is used for mdtmFTP server with versions < 1.1.1. The configuration file should be located at mdtmFTP server’s working directory.

8.1. Topology section

The syntax is defined as:

<Topology>
  <Device type=Device_Type numa=Numa_ID>Device_Name</device>
  ...
</Topology>

Device_Type refers to MDTM device type. MDTM defines three types of devices: network, block, and virtual.

  • Network refers to a network I/O device.

  • Block refers to a storage/disk I/O device.

  • Virtual refers to a virtual device, which is defined particularly for mdtmFTP server.

Numa_ID sets which NUMA node a device belongs to (i.e., NUMA location).

Device_Name specifies a device name.

MDTM middleware is typically able to detect physical I/O devices and their locations (i.e., which NUMA node that a I/O device belongs to) on a NUMA system. However, there are two cases that MDTM middleware cannot detect physical I/O devices or their locations correctly:

  1. In a fully virtualized environment, where information on physical I/O devices is not exposed to guest OS.

  2. Some vendors’ I/O devices may not comply to OS rules to expose device information properly.

Under these conditions, system admin should manually configure I/O devices and their NUMA locations.

Virtual device is defined particularly for mdtmFTP server to monitor data transfer status. mdtmFTP server spawns a dedicated management thread to collect and record data transfer statistics. The management thread is associated with a virtual device, which will be pinned to a specified NUMA node.

8.2. Online section

The syntax is defined as:

<Online>
  <Device>Device_Name</Device>
  ...
</Online>

This section specifies the I/O devices that are assigned for data transfer.

For example, assume a DTN has the following I/O devices:

  • Ethernet NIC devices

    • eth0 – configured for management access

    • eth1 – configured for WAN data transfer

  • Block I/O devices

    • /dev/sda – system disk

    • /dev/sdb – data repository for WAN data transfer

In this case, the online section would be defined as:

<Online>
  <Device>eth1</Device>
  <Device>sdb</Device>
</Online>

  • For network I/O devices, a user can run ifconfig to list network I/O devices available on the system.

  • For storage/disk IO devices, a user can run lsblk to list storage/disk I/O devices available on the system; and then run df to find out on which storage/disk I/O devices that a data transfer folder will be located.

    Assuming that a DTN system’s lsblk output is:

    $ lsblk
NAME                          MAJ:MIN  RM  SIZE RO  TYPE MOUNTPOINT
sda                             8:0     0  1.8T  0  disk
├─sda1                          8:1     0  500M  0  part /boot
└─sda2                          8:2     0  1.8T  0  part
├─scientific_bde1-root          253:0   0  50G   0  lvm  /
├─scientific_bde1-swap          253:1   0  4G    0  lvm  [SWAP]
└─scientific_bde1-home          253:2   0  1.8T  0  lvm  /home
loop0                           7:0     0  100G  0  loop
└─docker-253:0-203522131-pool   253:3   0  100G  0  dm
loop1                           7:1     0  2G    0  loop
└─docker-253:0-203522131-pool   253:3   0  100G  0  dm
nvme0n1                         259:0   0  1.1T  0  disk /data1

    And df output is:

    $ df
Filesystem                       1K-blocks  Used       Available  Use% Mounted on
/dev/mapper/scientific_bde1-root 52403200   15999428   36403772   31%  /
devtmpfs                         65855232   0          65855232   0%   /dev
/dev/nvme0n1                     1153584388 104952744  990009612  10%  /data1
/dev/mapper/scientific_bde1-home 1895386900 23602284   1871784616 2%   /home
/dev/sda1                        508588     376264     132324     74%  /boot

If /data1 is used as data transfer folder, the corresponding storage/disk I/O device is nvme0n1.

8.3. Thread section

The syntax is defined as:

<Threads threads=Default_Num>
  <Device type=Device_Type threads=Num>Device_Name</Device>
  ...
</Threads>

This section defines the number of threads that needs to be allocated for an I/O device. The number of threads allocated for an I/O device should be proportional to the device’s I/O bandwidth. The rule of thumb is that a thread can handle an I/O rate of 10Gbps. For example, four threads should be allocated for a 40GE NIC while one thread be allocated for a 10GE NIC.

Default_Num sets the default number of threads allocated for each I/O device.

If a different number of threads should be allocated for a particular I/O device, a separate entry for the device should to be specified here.

A virtual device should be allocated with 1 thread.

8.4. File section

The syntax is defined as:

<File segment=File_Size_Threshold>
</File>

MDTM splits a large file into segments, which are spread to different threads for disk and network operations to increase performance.

File_Size_Threshold sets a file size threshold. A file with a size that exceeds the threshold will be split into multiple segments, which are spread across I/O threads to be transferred in parallel.

Here is a sample mdtmconfig.xml file for mdtmFTP server:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
<?xml version="1.0" standalone="no" ?>
<Topology>
  <Device type="Virtual" numa="1">man</Device>
  <Device type="Network" numa="0">eth40.4020</Device>
</Topology>
<Online>
  <Device>eth40.4020</Device>
  <Device>sda</Device>
  <Device>man</Device>
</Online>
<Threads threads="1">
  <Device type="Network" threads="2">eth40.4020</Device>
  <Device type="Block" threads="2">sda</Device>
  <Device type="Virtual" threads="1">man</Device>
</Threads>
<File segment="2G">
</File>