Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2291

Provide to streaming connectors the stop strategy applied

Apply templateInsert Lucidchart DiagramXMLWordPrintable

    • All
    • GreenHopper Ranking:
      0|i2l0rv:
    • 9223372036854775807
    • Small
    • To be defined

      TCOMP-2226 introduces the StopStrategy.
      It has a part about timeout stop, it'll be really nice to have this value inside the connector, to allow the Source connector to handle timeout itself.
      Reason.
      For JMS brokers, interrupted exception leads to automatic rollback action, and instead of just stopping reading the next record because of timeout it rollbacks the reading of the previous record.

      As an idea, it's to pass this option right into the emitter method.
      We can either use an existing Option annotation or create a new one: Parameter("timeout) or smth like that.
      I think it should be an object to allow receive null as timeout if the timeout isn't supported on this platform.

          @Producer
          public Record next(@Option("timeout") Integer timeout) {
              try {
      

      UPD. Also, the framework should wait a bit to allow component handle timeout. Sounds complicated.
      timeout + grace period seconds

       

      [Update EG] according to work done by Oleksandr:

      Strategy parameters are passed to connector on connector's initialization to elaborate a strategy (according technology implemented).

      @PostConstruct
      public void init(@Option(Option.MAX_DURATION_PARAMETER) long maxDurationMs, @Option(Option.MAX_RECORDS_PARAMETER) long maxRecords ) {
         // create connector's specific strategy
      }

      With this feature, we may avoid record loss.
      See root issue linked issues to get more details of potential pitfalls.

            Unassigned Unassigned
            ozhelezniak Oleksandr Zhelezniak
            emmanuel gallois, Oleksandr Zhelezniak
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: