Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2291

Provide to streaming connectors the stop strategy applied

Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • All
    • GreenHopper Ranking:
      0|i2l0rv:
    • 9223372036854775807
    • Small
    • To be defined

    Description

      TCOMP-2226 introduces the StopStrategy.
      It has a part about timeout stop, it'll be really nice to have this value inside the connector, to allow the Source connector to handle timeout itself.
      Reason.
      For JMS brokers, interrupted exception leads to automatic rollback action, and instead of just stopping reading the next record because of timeout it rollbacks the reading of the previous record.

      As an idea, it's to pass this option right into the emitter method.
      We can either use an existing Option annotation or create a new one: Parameter("timeout) or smth like that.
      I think it should be an object to allow receive null as timeout if the timeout isn't supported on this platform.

          @Producer
          public Record next(@Option("timeout") Integer timeout) {
              try {
      

      UPD. Also, the framework should wait a bit to allow component handle timeout. Sounds complicated.
      timeout + grace period seconds

       

      [Update EG] according to work done by Oleksandr:

      Strategy parameters are passed to connector on connector's initialization to elaborate a strategy (according technology implemented).

      @PostConstruct
      public void init(@Option(Option.MAX_DURATION_PARAMETER) long maxDurationMs, @Option(Option.MAX_RECORDS_PARAMETER) long maxRecords ) {
         // create connector's specific strategy
      }

      With this feature, we may avoid record loss.
      See root issue linked issues to get more details of potential pitfalls.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ozhelezniak Oleksandr Zhelezniak
              emmanuel gallois, Oleksandr Zhelezniak
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: