Uploaded image for project: 'DSpace'
  1. DSpace
  2. DS-3975

Checksum checker fails to complete with large repositories

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Code Review Needed (View Workflow)
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 6.0, 6.1, 6.2, 6.3
    • Fix Version/s: None
    • Component/s: API
    • Labels:
      None
    • Attachments:
      0
    • Comments:
      2
    • Documentation Status:
      Needed

      Description

      In repositories with many bitstreams, the checksum checker can fail to complete (hang or die) due to memory constraints. This problem appears to have been introduced with the move to Hibernate.

      Analysis

      We examined the issue in detail on a recent DSpace 6.x deployment and determined that it gets stuck in the following code: 

      {{{{ at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1230)
      at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101)
      at org.dspace.content.dao.impl.BitstreamDAOImpl.findBitstreamsWithNoRecentChecksum(BitstreamDAOImpl.java:91)
      at org.dspace.content.BitstreamServiceImpl.findBitstreamsWithNoRecentChecksum(BitstreamServiceImpl.java:365)
      at org.dspace.checker.MostRecentChecksumServiceImpl.updateMissingBitstreams(MostRecentChecksumServiceImpl.java:121)
      at org.dspace.checker.CheckerCommand.process(CheckerCommand.java:115)
      at org.dspace.app.checker.ChecksumChecker.main(ChecksumChecker.java:232)}}}}

      Notice that findBitstreamsWIthNoRecentChecksum returns a list, rather than an iterator of objects (or ids). This means it is relying on Hibernate to read, into memory, all Bitstream objects representing those that don't have a recent checksum.

      This is an impractical approach with large repositories, and results in the heap filling up and the process either hanging or dying.

      The approach that was used in DSpace 5.x (pre-Hibernate), was to call: 

      https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace-api/src/main/java/org/dspace/checker/BitstreamInfoDAO.java#L302
      ...which calls:
      https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace-api/src/main/java/org/dspace/checker/ChecksumHistoryDAO.java#L181
      ...which just executes a single SQL statement to insert necessary rows into the checksum_history table.

      In the move to Hibernate, this was turned into the following:

      https://github.com/DSpace/DSpace/blob/dspace-6_x/dspace-api/src/main/java/org/dspace/checker/MostRecentChecksumServiceImpl.java#L105

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            cwilper Chris Wilper
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated: