Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2541

Improve performance on dynamic column

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Hide

      1- Run attachment job 'JDBC_mysql_performance.zip' to prepare the testing data
      2- Run attachment job 'JDBC_mysql_performance2.zip' to retrieve 60 million records

      Show
      1- Run attachment job 'JDBC_mysql_performance.zip' to prepare the testing data 2- Run attachment job 'JDBC_mysql_performance2.zip' to retrieve 60 million records
    • All
    • Small

    Description

      Use tJDBCInput to retrieve 60 million records with dynamic column. Compare between tCompV0 and TCK framework

      Framework First Exec Second Exec Third Exec
      tCompV0 346s 337s 416s
      TCK 1112s 1124s 1293s

      The execution time of TCK tJDBCInput is twice more than tCompV0's.

      Attachments

        3. Screen Shot 2023-09-25 at 16.49.51.png
          1.96 MB
          Wei Wang
        4. Screen Shot 2023-09-27 at 10.05.19.png
          1.74 MB
          Wei Wang

        Issue Links

          is related to

          Work Item - Work item for the scum process TCOMP-2374 DIRecordVisitor to fill Studio Dynamic object's column name by tck Schema.Entry.getName which must follow tck name rule

          • Major - There are major errors in a system function. However, the error situation is not critical. The customer's business may be impaired by it.
          • Done

          Work Item - Work item for the scum process TCOMP-2375 Improve performance on Schema/Record

          • Major - There are major errors in a system function. However, the error situation is not critical. The customer's business may be impaired by it.
          • Done

          Bug - TCOMP-2732 Dynamic schema uses the value of the latest row for all rows

          • Major - There are major errors in a system function. However, the error situation is not critical. The customer's business may be impaired by it.
          • Done

          Activity

            People

              Unassigned Unassigned
              qyliu qiyan liu
              Wei Wang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: