Apply templateInsert Lucidchart Diagram
    XMLWordPrintable

Details

    • Backlog Task
    • Resolution: Fixed
    • Minor
    • None
    • None
    • None
    • Small

    Description

      A component should be able to provide runtime metrics to the job (counters like "how many records did I process and/or reject?".

      Beam has a limited facility to do counters called {{Aggregator}}s, which are only available in {{PTransform}}s based on {{DoFn}}s.

      More advanced counters can always be created by adding more {{PTransform}}s to a job (counter and aggregation steps).

      We should investigate the most efficient way to add metrics and monitoring to running and finished TCOMP jobs.

      Attachments

        Activity

          People

            Unassigned Unassigned
            tfion Thomas Fion
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: