Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-1925

Incorrect mapping of the parameters after arrays

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • 1.34.0
    • None
    • None
    • All
    • Small

    Description

      Looks like we have a small bug in org.talend.sdk.component.runtime.manager.configuration.ConfigurationMapper. The bug may be nondeterministic.
      During the recursive handling of configuration the class incorrectly handles the level of recursive.
      The current behavior is to clean all indexes of all levels when you exit the level of the array handling. Instead of that behavior it should clean only the indexes for the level that was handled and remain all above levels.

      That's why when "mapper" tries to handle ${index} for the parameter that inside array (upper level) after the handling of the array parameter (inner level) "indexes" map is empty and the resolve is unsuccess.

      Example:
      (1)configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      (2)configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE
      The row (1) was handled after the (2) in the result "mapper" don't have information on how to substitute the "${index}".

      Order of the nested parameters:

      An example for a draft connector.
      Original exception

      java.lang.IllegalArgumentException: Illegal character in query at index 544: 
      ApacheKudu://ApacheKuduOutput?configuration.createIfNotExist=true&configuration.dataset.datastore.masterAddresses[0].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[0].port=7051&configuration.dataset.datastore.masterAddresses[1].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[1].port=7151&configuration.dataset.datastore.masterAddresses[2].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[2].port=7251&configuration.dataset.tableName=testName1622458877099cAR&configuration.partitioning[${index}].rangePartitioning.columnName=int2Field&configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.value=1000&configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.bound=EXCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.value=1000&configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.bound=INCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.value=1000000&configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.bound=EXCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.value=1000000&configuration.partitioning[0].type=RANGE&configuration.partitioning[1].hashPartitioning.bucketNumber=8&configuration.partitioning[1].hashPartitioning.columnNames[0]=stringField&configuration.partitioning[1].type=HASH&configuration.primaryKey=stringField
      

      Pretty one:

      ApacheKudu://ApacheKuduOutput?configuration.createIfNotExist=true
      configuration.dataset.datastore.masterAddresses[0].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[0].port=7051
      configuration.dataset.datastore.masterAddresses[1].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[1].port=7151
      configuration.dataset.datastore.masterAddresses[2].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[2].port=7251
      configuration.dataset.tableName=testName1622456508150POa
      configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.value=1000
      configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.bound=EXCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.value=1000
      configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.bound=INCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.value=1000000
      configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.bound=EXCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.value=1000000
      configuration.partitioning[0].type=RANGE
      configuration.partitioning[1].hashPartitioning.bucketNumber=8
      configuration.partitioning[1].hashPartitioning.columnNames[0]=stringField
      configuration.partitioning[1].type=HASH
      configuration.primaryKey=stringField
      

      As result, we have this unresolved parameter

      configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      

      Attachments

        1. buildOk.png
          60 kB
          Oleksandr Zhelezniak
        2. nestedParamOrder.PNG
          25 kB
          Oleksandr Zhelezniak

        Activity

          People

            emmanuel_g emmanuel gallois
            ozhelezniak Oleksandr Zhelezniak
            Oleksandr Zhelezniak
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 4 hours
                4h