I am attempting to index a repository containing 250,000 items (metadata only).
I first ran index-discovery -b. On subsequent runs, I ran index-discovery.
The process runs quickly for the first several thousand items. After that the process becomes very sluggish. One one test that I ran overnight, the process ran out of memory and filled my log directory.
If I kill the process after about an hour and restart it, the process resumes and processes items quickly again.
In my testing, I have included the following code: https://github.com/DSpace/DSpace/pull/1131