D.2 Performance

For versions of FDR before 2.50, checks proceeded rapidly until physical memory was exhausted and then slowly until FDR ran out of memory, either because virtual-memory was exhausted or because of the 4GB (or less) limit on address-space imposed by the underlying 32-bit operating system. Improving raw performance in such circumstances involved ensuring that enough swap-space was available, and that as much physical memory as possible was fitted.

As disk sizes grew (and prices fell), this 4GB limit became a significant restriction; machines were routinely being shipped with more storage than FDR could exploit. Starting with version 2.50, we redesigned the components used to manage FDR’s largest data-structures with two main aims: to break the limit, and improve performance once physical memory was exceeded.

Rather than relying on the host OS, FDR now manages much of its storage explicitly. It pages data as required between an in-memory buffer (whose size is controlled by FDRPAGESIZE) and secondary storage areas (specified by FDRPAGEDIRS.) We recommend that FDRPAGESIZE should not exceed a quarter to a half of physical memory. For 64-bit platforms, FDRPAGESIZE can be be set to greater than 4Gb (G and M modify the value given in Gigabytes and Megabytes respectively).

If FDRPAGEDIRS is not set, then memory is used to provided the secondary storage; this gives behaviour comparable to that of FDR version 2.28 and allows FDR to continue to work acceptably on machines configured for earlier versions. However, if FDRPAGEDIRS is set to a colon-separated list of directories, then FDR creates one temporary file in each directory and uses those as secondary storage. Data is split across the files in proportion to the amount of free space in each directory when the check starts. Paging performance is best when using local file-systems (mounted asynchronously if the operating system permits). The use of network file-systems (NFS) for paging directories is possible, but not recommended. Errors may occur when the individual temporary files would exceed 4GB on an OS with a 32-bit file-system.

This document was generated by Phil Armstrong on May 17, 2012 using texi2html 1.82.