Talend DI
  1. Talend DI
  2. TDI-17255

Intermittent ConcurrentModificationException in runtime stats package

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Minor Minor
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ??? unknown ???
    • Labels:
      None
    • Environment:

      Seen on Windows 7 Pro 32 bit Java 1.6
      Seen on Solaris 10 Java 1.6

    • Found in Release:
    • Steps to reproduce:
      Hide

      Unknown - this is intermittent.
      The jobs in question used tMap, with lookups that store-temp-data to disk.
      The problem appears to occur more during memory shortages, particularly when the garbage collector is running frequently.

      Show
      Unknown - this is intermittent. The jobs in question used tMap, with lookups that store-temp-data to disk. The problem appears to occur more during memory shortages, particularly when the garbage collector is running frequently.
    • Product Build:
      63143
    • Java version:
      1.6
    • Platform:
      All
    • Operating System:
      Several
    • Resolution comment:
      Hide

      Incomplete issue. Feel free to open it again if needed, and if you can provide a way to reproduce.

      Thanks.

      Show
      Incomplete issue. Feel free to open it again if needed, and if you can provide a way to reproduce. Thanks.
    • Status Detail:
      Waiting for Reporter Information

      Description

      The following exception has occurred several times today on my laptop and a customer's testing machine.
      The jobs in question are characterised by tMaps with multiple lookups that store temp data on disk, and also in that they have often been garbage collecting a lot and running out of memory. (We have been stressing the job in order to tune memory consumption).

      The error is intermittent. I believe that it was suppressed by disabling stats collection for the job but since it's intermittent I cannot be 100% certain of this.

      Stack trace is:

      Starting job tMap_lookup_on_disk at 22:00 18/08/2011.

      [statistics] connecting to socket on port 3896
      [statistics] connected

            • job runs without issues for nearly 1 minute, processes many rows ****
            • ... and then it failed as follows ****

      Exception in component
      java.util.ConcurrentModificationException
      at java.util.LinkedList$ListItr.checkForComodification(Unknown Source)
      at java.util.LinkedList$ListItr.next(Unknown Source)
      at routines.system.RunStat.sendMessages(RunStat.java:248)
      at routines.system.RunStat.stopThreadStat(RunStat.java:225)
      at caci.tmap_lookup_on_disk_0_1.tMap_lookup_on_disk.tFileInputDelimited_4Process(tMap_lookup_on_disk.java:3115)
      at caci.tmap_lookup_on_disk_0_1.tMap_lookup_on_disk.tFileInputDelimited_2Process(tMap_lookup_on_disk.java:1361)
      at caci.tmap_lookup_on_disk_0_1.tMap_lookup_on_disk.runJobInTOS(tMap_lookup_on_disk.java:3287)
      at caci.tmap_lookup_on_disk_0_1.tMap_lookup_on_disk.main(tMap_lookup_on_disk.java:3161)
      [statistics] disconnected
      Job tMap_lookup_on_disk ended at 22:01 18/08/2011. [exit code=1]

        Activity

        Hide
        Rémy DUBOIS added a comment -

        Can you please attach your job. In the same time, can you please test the job again, de-activating the statistics ?

        Thanks.

        Show
        Rémy DUBOIS added a comment - Can you please attach your job. In the same time, can you please test the job again, de-activating the statistics ? Thanks.
        Hide
        Imad Rahman added a comment -

        I can't provide exports of these but I will try to reproduce the issue in a job of my own. If I succeed then I will post the testcase.

        We ran the jobs again with deactivated statistics, and they worked fine. But bear in mind that the problem was intermittent so this doesn't 100% prove the problem is the stats - I originally thought it was a problem in the stats because of the stack trace.

        Show
        Imad Rahman added a comment - I can't provide exports of these but I will try to reproduce the issue in a job of my own. If I succeed then I will post the testcase. We ran the jobs again with deactivated statistics, and they worked fine. But bear in mind that the problem was intermittent so this doesn't 100% prove the problem is the stats - I originally thought it was a problem in the stats because of the stack trace.

          People

          • Assignee:
            Rémy DUBOIS
            Reporter:
            Imad Rahman
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: