Uploaded image for project: 'Talend Component Kit'
  1. Talend Component Kit
  2. TCOMP-2190

Handle partial messages for large payloads in websocket communications

Apply templateInsert Lucidchart DiagramXMLWordPrintable

    • All
    • Small

      We meet this problem with the studio. The root cause is in the part of studio-integration plugin.
      WebSocketClient sends a request about job\node migration. It makes encodes values of the configuration with base64 and wraps them into ByteBuffer.
      WebClient-payload-creation
      WebSocketServer receives it but it has limitations. 8196 bytes. The communication isn't partial.
      And when our payload is bigger, for example, 10646, it can't be processed and an exception is thrown.
      As a user of Studio, I just see that Studio freezes for a couple of minutes and then I can't open the job.

      The buffer can be increased with option:

      org.apache.tomcat.websocket.DEFAULT_BUFFER_SIZE=12265
      

      But maybe, websocket can support partial payload (looks like it can, because I found in the decompiled code that it checks on the partialness of a payload).

      We met that problem with workday job migration: https://talend.slack.com/archives/GNLEZ369L/p1657045586838019

      UPD
      The idea of this ticket is to implement (enable if it's just a configuration) partial payload

            Unassigned Unassigned
            ozhelezniak Oleksandr Zhelezniak
            Oleksandr Zhelezniak
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: