Uploaded image for project: 'VIVO'
  1. VIVO
  2. VIVO-1669

RDFServiceGraph flush() : out of memory

    XMLWordPrintable

    Details

    • Attachments:
      0
    • Comments:
      2

      Description

      The append method of StringBuilder has a character count max limit of "Integer.MAX_VALUE",

      so the serialization in serializeGraph(Graph graph) is very likely to go in out of memory or out of bound.

      Tests based on real production data, with an upload of 2gb size rdf file show the need to adress this issue,

      something like this shoud do the job:

       

       
      if(!additionsGraph.isEmpty()) {
       String triples = "";
       StringBuilder sb = new StringBuilder(51000000);
       Iterator<Triple> tripIt = graph.find(null, null, null);
       while(tripIt.hasNext()) {
       sb.append(" \n");
       sb.append(serialize(tripIt.next()));
       if(sb.length()>50000000) {
       triples = sb.toString();
       sb = null;
       sb = new StringBuilder();
       changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
       RDFService.ModelSerializationFormat.N3, graphURI);
       triples = null;
       rdfService.changeSetUpdate(changeSet);
       changeSet = rdfService.manufactureChangeSet();
       }
       }
       triples = sb.toString();
       sb = null;
       changeSet.addAddition(RDFServiceUtils.toInputStream(triples), 
       RDFService.ModelSerializationFormat.N3, graphURI);
       triples = null;
       additionsGraph.clear();
       rdfService.changeSetUpdate(changeSet);
       }
      
      
      
      

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            alessandrogalasso alessandro galasso
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: