Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-1925

Incorrect mapping of the parameters after arrays

Details

    • Bug
    • Status: Done
    • Minor
    • Resolution: Fixed
    • None
    • 1.34.0
    • None
    • All
    • Small

    Description

      Looks like we have a small bug in org.talend.sdk.component.runtime.manager.configuration.ConfigurationMapper. The bug may be nondeterministic.
      During the recursive handling of configuration the class incorrectly handles the level of recursive.
      The current behavior is to clean all indexes of all levels when you exit the level of the array handling. Instead of that behavior it should clean only the indexes for the level that was handled and remain all above levels.

      That's why when "mapper" tries to handle ${index} for the parameter that inside array (upper level) after the handling of the array parameter (inner level) "indexes" map is empty and the resolve is unsuccess.

      Example:
      (1)configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      (2)configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE
      The row (1) was handled after the (2) in the result "mapper" don't have information on how to substitute the "${index}".

      Order of the nested parameters:

      An example for a draft connector.
      Original exception

      java.lang.IllegalArgumentException: Illegal character in query at index 544: 
      ApacheKudu://ApacheKuduOutput?configuration.createIfNotExist=true&configuration.dataset.datastore.masterAddresses[0].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[0].port=7051&configuration.dataset.datastore.masterAddresses[1].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[1].port=7151&configuration.dataset.datastore.masterAddresses[2].hostname=127.0.0.1&configuration.dataset.datastore.masterAddresses[2].port=7251&configuration.dataset.tableName=testName1622458877099cAR&configuration.partitioning[${index}].rangePartitioning.columnName=int2Field&configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.value=1000&configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.bound=EXCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.value=1000&configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.bound=INCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.value=1000000&configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.bound=EXCLUSIVE&configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.value=1000000&configuration.partitioning[0].type=RANGE&configuration.partitioning[1].hashPartitioning.bucketNumber=8&configuration.partitioning[1].hashPartitioning.columnNames[0]=stringField&configuration.partitioning[1].type=HASH&configuration.primaryKey=stringField
      

      Pretty one:

      ApacheKudu://ApacheKuduOutput?configuration.createIfNotExist=true
      configuration.dataset.datastore.masterAddresses[0].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[0].port=7051
      configuration.dataset.datastore.masterAddresses[1].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[1].port=7151
      configuration.dataset.datastore.masterAddresses[2].hostname=127.0.0.1
      configuration.dataset.datastore.masterAddresses[2].port=7251
      configuration.dataset.tableName=testName1622456508150POa
      configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.bound=INCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[0].upperBound.value=1000
      configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.bound=EXCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[1].lowerBound.value=1000
      configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.bound=INCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[1].upperBound.value=1000000
      configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.bound=EXCLUSIVE
      configuration.partitioning[0].rangePartitioning.bounds[2].lowerBound.value=1000000
      configuration.partitioning[0].type=RANGE
      configuration.partitioning[1].hashPartitioning.bucketNumber=8
      configuration.partitioning[1].hashPartitioning.columnNames[0]=stringField
      configuration.partitioning[1].type=HASH
      configuration.primaryKey=stringField
      

      As result, we have this unresolved parameter

      configuration.partitioning[${index}].rangePartitioning.columnName=int2Field
      

      Attachments

        Issue Links

          Activity

            People

              emmanuel_g emmanuel gallois
              ozhelezniak Oleksandr Zhelezniak
              Oleksandr Zhelezniak
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 4 hours
                  4h