Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-230

Add Flow variables support in Component Framework

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Work Item
    • Resolution: Fixed
    • Critical
    • 0.18.0
    • 0.16.0
    • None
    • All
    • 0.16.0, 0.18.0
    • GreenHopper Ranking:
      0|i0vnoj:
    • 9223372036854775807
    • Small
    • 20

    Description

      Some of old components have "Flow" variables. They are similar to "After" (Return) properties, but "Flow" properties are available only in current flow/subjob and are different for each record processed. "After" properties are available in different flow (after all records were processed).
      Both types of variables use the same mechanism of global map.

      For now new Component Framework doesn't support "Flow" variables, but they are required to port components on new Framework.

      During discussion "Flow variables" notion was transformed to "Out of band data" notion (OOB data)
      After this change TCOMP components may optionally output Root record

      This issue is related to 3 tasks:

      1. Definition task (TCOMP should provide means for components to define what OOB data component provide)
      2. Runtime task (TCOMP should provide means for components to output OOB data as well as usual data)
      3. Data Source task (Special case with Stand alone components should be addressed)

      Also this task should performed in several projects and modules:

      1. TCOMP
      2. Components (should be changed to output hierarchical record)
      3. DI codegen (should clean record from OOB data before passing to downstream component and post OOB data as DI flow variables)
      4. Studio (Should call TCOMP API to show OOB provided by component in Outline view)

      This issue includes following subtasks:

      1. Describe concept of Out of band data
      2. Add github wiki page, which describes how to implement component with/without OOB data
      3. Add means to define, what OOB data component provide
      4. Add utility methods to support component developer in component development which supports OOB
      5. Make changes in Studio according changes in TCOMP
      6. Make changes in codegen
      7. Make changes in all components

      Codegen-utils library:

      1. variableKey to VARIABLE_KEY transformation should be added to library (and removed from codegen templates )
      2. Analyze codegen to find code, which could be moved to library (to simplify codegen, and cover with junits)

      Attachments

        1. FlowUC1.png
          FlowUC1.png
          114 kB
        2. FlowUC2.png
          FlowUC2.png
          114 kB
        3. OutlineReturnVariables.png
          OutlineReturnVariables.png
          10 kB

        Issue Links

          Activity

            People

              igonchar Ivan Gonchar
              igonchar Ivan Gonchar
              Ivan Gonchar
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1 week, 3 days, 1 hour Original Estimate - 1 week, 3 days, 1 hour
                  1w 3d 1h
                  Remaining:
                  Time Spent - 3 weeks, 3 days, 3 hours, 30 minutes Remaining Estimate - 6 hours
                  6h
                  Logged:
                  Time Spent - 3 weeks, 3 days, 3 hours, 30 minutes Remaining Estimate - 6 hours
                  3w 3d 3h 30m