Uploaded image for project: 'DuraCloud'
  1. DuraCloud
  2. DURACLOUD-279

Bulk Image Transformation failures

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Fix Version/s: 0.8
    • Component/s: Services
    • Labels:
      None
    • Attachments:
      1
    • Comments:
      2

      Description

      The Bulk Image Transformation service has failed in numerous tests which has been over a set of TIFF images files supplied by Rhodes. This set includes 317 images, ranging in size from 2.35 MB to 265.42 MB. The primary test case has been converting to JP2 format using the source color space. The service has failed when running with 1 instance of each instance size. The service has also failed in a test running an xlarge instance with only 2 mappers.

      Watching the memory consumption on a small instance while the service is running shows that a significant amount of memory is consumed by the mogrify step (which is run using a bash script to call ImageMagick) - all swap space was consumed, and less than 10% of memory remained available.

      Testing has indicated that the size of images being transformed likely plays a role in the failures. Moving the two largest Rhodes files to another space (all of the rest of the tiffs being under 115 MB) allowed the transformation to complete successfully using 3 xlarge instances. Running the conversion over the same set with only 1 instance (regardless of size) still failed, however. The attached file shows the list of tiff files which were successfully transformed in these tests, as well as the files on which the tests failed. The large instance was run with two mappers, and the x-large instance with 4 mappers.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              bbranan Bill Branan
              Reporter:
              bbranan Bill Branan
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: