com.hazelcast.jet.core

## Class WatermarkGenerationParams<T>

- java.lang.Object
- - com.hazelcast.jet.core.WatermarkGenerationParams<T>

- Type Parameters:
    `T` - event type
  All Implemented Interfaces:
    Serializable

      public final class WatermarkGenerationParams<T>
      extends Object
      implements Serializable

  A holder of functions and parameters Jet needs to determine when and how to insert watermarks into an event stream. These are the components:
  - `timestampFn`: extracts the timestamp from an event in the stream
  - `newWmPolicyFn`: a factory of watermark policy objects. Refer to its `documentation` for explanation.
  - `wmEmitPolicy`: a `policy object` that allows the processor to filter out redundant watermark items before emitting them. For example, a sliding/tumbling window processor doesn't need to observe more than one watermark item per frame.
  - `idleTimeoutMillis`: a measure to mitigate the issue with temporary lulls in a distributed event stream. It pertains to each partition of a data source independently. If Jet doesn't receive any events from a given partition for this long, it will mark it as "idle" and let the watermark in downstream vertices advance as if the partition didn't exist.
  - `wrapFn`: a function that transforms a given event and its timestamp into the item to emit from the processor. For example, the Pipeline API uses this to wrap items into `JetEvent`s as a way to propagate the event timestamps through the pipeline regardless of the transformation the user does on the event objects themselves.
  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static long` | `DEFAULT_IDLE_TIMEOUT` The default idle timeout in milliseconds.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `long` | `idleTimeoutMillis()` Returns the amount of time allowed to pass without receiving any events from a partition before marking it as "idle".
    `DistributedSupplier<? extends WatermarkPolicy>` | `newWmPolicyFn()` Returns the factory of the watermark policy objects.
    `static <T> WatermarkGenerationParams<T>` | `noWatermarks()` Returns watermark generation parameters that result in no watermarks being emitted.
    `DistributedToLongFunction<? super T>` | `timestampFn()` Returns the function that extracts the timestamp from the event.
    `WatermarkGenerationParams<T>` | `withEmitPolicy(WatermarkEmissionPolicy emitPolicy)` Returns new instance with emit policy replaced with the given argument.
    `WatermarkEmissionPolicy` | `wmEmitPolicy()` Returns the watermark emission policy, which decides how to suppress redundant watermarks.
    `static <T> WatermarkGenerationParams<T>` | `wmGenParams(DistributedToLongFunction<? super T> timestampFn, DistributedObjLongBiFunction<? super T,?> wrapFn, DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn, WatermarkEmissionPolicy wmEmitPolicy, long idleTimeoutMillis)` Creates and returns new watermark generation parameters.
    `static <T> WatermarkGenerationParams<T>` | `wmGenParams(DistributedToLongFunction<? super T> timestampFn, DistributedSupplier<? extends WatermarkPolicy> wmPolicy, WatermarkEmissionPolicy wmEmitPolicy, long idleTimeoutMillis)` Creates and returns a watermark generation parameters object.
    `DistributedObjLongBiFunction<? super T,?>` | `wrapFn()` Returns the function that transforms the received item and its timestamp into the emitted item.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_IDLE_TIMEOUT

          public static final long DEFAULT_IDLE_TIMEOUT

      The default idle timeout in milliseconds.
      See Also:
        Constant Field Values

  - ### Method Detail

    - #### wmGenParams

          public static <T> WatermarkGenerationParams<T> wmGenParams(@Nonnull
                                                                     DistributedToLongFunction<? super T> timestampFn,
                                                                     @Nonnull
                                                                     DistributedObjLongBiFunction<? super T,?> wrapFn,
                                                                     @Nonnull
                                                                     DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn,
                                                                     @Nonnull
                                                                     WatermarkEmissionPolicy wmEmitPolicy,
                                                                     long idleTimeoutMillis)

      Creates and returns new watermark generation parameters. To get parameters that result in no watermarks being emitted, call `noWatermarks()`.
      Parameters:
        `timestampFn` - function that extracts the timestamp from the event
        `wrapFn` - function that transforms the received item and its timestamp into the emitted item
        `newWmPolicyFn` - factory of the watermark policy objects
        `wmEmitPolicy` - watermark emission policy (decides how to suppress redundant watermarks)
        `idleTimeoutMillis` - the timeout after which a partition will be marked as idle. If <= 0, partitions will never be marked as idle.

    - #### wmGenParams

          public static <T> WatermarkGenerationParams<T> wmGenParams(@Nonnull
                                                                     DistributedToLongFunction<? super T> timestampFn,
                                                                     @Nonnull
                                                                     DistributedSupplier<? extends WatermarkPolicy> wmPolicy,
                                                                     @Nonnull
                                                                     WatermarkEmissionPolicy wmEmitPolicy,
                                                                     long idleTimeoutMillis)

      Creates and returns a watermark generation parameters object. To get parameters that result in no watermarks being emitted, call `noWatermarks()`.
      Parameters:
        `timestampFn` - function that extracts the timestamp from the event
        `wmPolicy` - factory of the watermark policy objects
        `wmEmitPolicy` - watermark emission policy (decides how to suppress redundant watermarks)
        `idleTimeoutMillis` - the timeout after which a partition will be marked as idle. If <= 0, partitions will never be marked as idle.

    - #### noWatermarks

          public static <T> WatermarkGenerationParams<T> noWatermarks()

      Returns watermark generation parameters that result in no watermarks being emitted. Only useful in jobs with streaming sources that don't do any aggregation. If there is an aggregation step in the job and you use these parameters, your job will keep accumulating the data without producing any output.

    - #### timestampFn

          @Nonnull
          public DistributedToLongFunction<? super T> timestampFn()

      Returns the function that extracts the timestamp from the event.

    - #### wrapFn

          @Nonnull
          public DistributedObjLongBiFunction<? super T,?> wrapFn()

      Returns the function that transforms the received item and its timestamp into the emitted item.

    - #### newWmPolicyFn

          @Nonnull
          public DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn()

      Returns the factory of the watermark policy objects.

    - #### wmEmitPolicy

          @Nonnull
          public WatermarkEmissionPolicy wmEmitPolicy()

      Returns the watermark emission policy, which decides how to suppress redundant watermarks.

    - #### idleTimeoutMillis

          public long idleTimeoutMillis()

      Returns the amount of time allowed to pass without receiving any events from a partition before marking it as "idle". When the partition becomes idle, the processor emits an `WatermarkCoalescer.IDLE_MESSAGE` to its output edges. This signals Jet that the watermark can advance as if the partition didn't exist.

      If supply a zero or negative value, partitions will never be marked as idle.

    - #### withEmitPolicy

          @Nonnull
          public WatermarkGenerationParams<T> withEmitPolicy(WatermarkEmissionPolicy emitPolicy)

      Returns new instance with emit policy replaced with the given argument.

Copyright © 2018. All rights reserved.