Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Blocker Blocker
    • Resolution: no change required
    • Affects Version/s: 3.2.3
    • Fix Version/s: None
    • Component/s: Studio
    • Labels:
    • Environment:

      Platform: windows server 2003

    • Found in Release:
    • OTRS ID:
      2010080310000068
    • Mantis ID:
      14767

      Description

      I have a simple job with input + tmap + output

      when I launched my job and I had a little bit of lines, it worked.
      Now, I have 5 000 000 lines to treat and it didin't work

      The error sent by talend is :
      Exception in thread "main" Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
      at java.util.concurrent.ConcurrentHashMap$Values.iterator(Unknown Source)
      at routines.system.RunStat.sendMessages(RunStat.java:155)
      at routines.system.RunStat.run(RunStat.java:128)
      at java.lang.Thread.run(Unknown Source)
      java.lang.OutOfMemoryError: Java heap space
      at com.mysql.jdbc.Buffer.getBytes(Buffer.java:198)
      at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:318)
      at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1345)
      at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2330)
      at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:427)
      at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2035)
      at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1421)
      at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1772)
      at com.mysql.jdbc.Connection.execSQL(Connection.java:2430)
      at com.mysql.jdbc.Connection.execSQL(Connection.java:2359)
      at com.mysql.jdbc.Statement.executeQuery(Statement.java:1227)
      at siad.fin_conso_mkg_02ds_0_1.FIN_conso_MKG_02DS.tMysqlInput_1Process(FIN_conso_MKG_02DS.java:3395)
      at siad.fin_conso_mkg_02ds_0_1.FIN_conso_MKG_02DS.tMysqlConnection_2Process(FIN_conso_MKG_02DS.java:929)
      at siad.fin_conso_mkg_02ds_0_1.FIN_conso_MKG_02DS.tMysqlConnection_1Process(FIN_conso_MKG_02DS.java:833)
      at siad.fin_conso_mkg_02ds_0_1.FIN_conso_MKG_02DS.runJobInTOS(FIN_conso_MKG_02DS.java:7965)
      at siad.fin_conso_mkg_02ds_0_1.FIN_conso_MKG_02DS.main(FIN_conso_MKG_02DS.java:7789)

        Activity

        Hide
        Shicong Hong added a comment -

        Hi
        How much memory do you have? Go to Windows->Preference->Talend->Run/debug and increase more memory.

        Best regards
        Shong

        Show
        Shicong Hong added a comment - Hi How much memory do you have? Go to Windows->Preference->Talend->Run/debug and increase more memory. Best regards Shong
        Hide
        fanny lepaul added a comment -

        Hi,

        my computer have 3 Gb of memory.
        in Windows->Preference->Talend->Run/debug, I have Xms256M and Xmx1024M.
        Anymore with these parameters it doesn't work.

        In the TISPE-win32-x86.ini file, I have
        -vmargs -Xms40m -Xmx1024m -XX:MaxPermSize=128m

        what could I do ?

        Show
        fanny lepaul added a comment - Hi, my computer have 3 Gb of memory. in Windows->Preference->Talend->Run/debug, I have Xms256M and Xmx1024M. Anymore with these parameters it doesn't work. In the TISPE-win32-x86.ini file, I have -vmargs -Xms40m -Xmx1024m -XX:MaxPermSize=128m what could I do ?
        Hide
        Shicong Hong added a comment -

        Hi

        • The JVM in Windows->Preference->Talend->Run/debug is set for executing the job. The TISPE-win32-x86.ini is set for running TIS studio itself.
        • For 3 GB of memory, you can set the JVM parameters in Windows->Preference- >Talend->Run/debug, Xms512M and Xmx2048M.
        • Do you do some join on tMap? Can you upload your job?
        • Have a try to decrease the number of commit every parameter on the advanced setttng tab of tMysqlOutput.

        Best regards
        Shong

        Show
        Shicong Hong added a comment - Hi The JVM in Windows->Preference->Talend->Run/debug is set for executing the job. The TISPE-win32-x86.ini is set for running TIS studio itself. For 3 GB of memory, you can set the JVM parameters in Windows->Preference- >Talend->Run/debug, Xms512M and Xmx2048M. Do you do some join on tMap? Can you upload your job? Have a try to decrease the number of commit every parameter on the advanced setttng tab of tMysqlOutput. Best regards Shong
        Hide
        Shicong Hong added a comment -

        Hi
        Also check the 'Enable stream" box on the advanced setting tab of tMysqlInput.

        Best regards
        Shong

        Show
        Shicong Hong added a comment - Hi Also check the 'Enable stream" box on the advanced setting tab of tMysqlInput. Best regards Shong
        Hide
        fanny lepaul added a comment -

        We have the same problem on TIS

        What can we do ?

        Show
        fanny lepaul added a comment - We have the same problem on TIS What can we do ?

          People

          • Assignee:
            Shicong Hong
            Reporter:
            fanny lepaul
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: