Uploaded image for project: 'DSpace'
  1. DSpace
  2. DS-4425

Indexing Performance is very slow since refactor. ITs take twice as long

    XMLWordPrintable

    Details

    • Attachments:
      0
    • Comments:
      2
    • Documentation Status:
      Not Required

      Description

      Since the Indexing refactor was merged in DS-4287 and https://github.com/DSpace/DSpace/pull/2612 , there has been a major impact in the performance of indexing/searching.

      This impact is most severe in our Integration tests, which now take twice as long to run as before the PR was merged.

      Prior to the merger of PR#2612, Travis CI builds/tests used to take roughly 25-30mins to complete.  After the merger, Travis CI builds/tests consistently take over 45+mins to complete. As Travis CI kills any builds that take longer than 50mins, this means that some builds are killed before completion.

      The Travis CI behaviors can be seen at: https://travis-ci.org/DSpace/DSpace/builds 

      PR#2612 was merged during Travis build #7495 (the first build that took much longer than 40mins).  All prior Travis builds took <30mins.

      I'm not sure myself why the performance is so much worse after this refactor.  It could be the code itself, or it might be related to the change to the Solr Schema (changing resource type from int to String). Needs investigation.

      Kevin Van de Velde (Atmire) and Ben Bosman , this may be of interest to you as you were both involved in DS-4287

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              kevin van de velde Kevin Van de Velde (Atmire)
              Reporter:
              tdonohue Tim Donohue
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: