Uploaded image for project: 'DSpace'
  1. DSpace
  2. DS-739

Improve performance of Lucene indexing

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None
    • Attachments:
      0
    • Comments:
      2
    • Documentation Status:
      In Description

      Description

      Adds a batch processing mode for Lucene indexes.

      Can be controlled by calling DSIndexer.setBatchProcessingMode(boolean).

      NB: If you set batch processing mode to true, ensure that you set it to 'false' at the end of the batch to flush any unwritten documents.

      The size of the batch can be controlled by setting a numeric value in dspace.cfg for the property: search.batch.documents

      By default, the size of the batch is 20 documents.

      Additionally, there is the possibility to create a 'delayed index flusher'. If a web application pushes multiple search requests (ie. a barrage or sword deposits, or multiple quick edits in the ui), then this will combine them into a single index update (up to the limit of the batch defined above).

      To use the delayed update, set the property 'search.index.delay' in dspace.cfg to the number of milliseconds to wait for an update. eg.

      search.index.delay = 5000

      will hold a Lucene update in a queue for up to 5 seconds. After 5 seconds - or the batch limit above is reached - all waiting updates will be written to the Lucene index.

        Attachments

          Activity

            People

            Assignee:
            grahamtriggs Graham Triggs
            Reporter:
            grahamtriggs Graham Triggs
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: