com.hazelcast.jet.core

## Class WatermarkPolicies

- java.lang.Object
- - com.hazelcast.jet.core.WatermarkPolicies

-     public final class WatermarkPolicies
      extends Object

  Utility class with factories of several useful watermark policies.

- - ### Method Summary

    All Methods Static Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static DistributedSupplier<WatermarkPolicy>` | `limitingLag(long lag)` Maintains a watermark that lags behind the top observed timestamp by the given amount.
    `static DistributedSupplier<WatermarkPolicy>` | `limitingLagAndDelay(long lag, long maxDelayMs)` Maintains a watermark that lags behind the top observed timestamp by at most the given amount and is additionally guaranteed to reach the timestamp of any given event within `maxDelayMs` of system time after observing it.
    `static DistributedSupplier<WatermarkPolicy>` | `limitingLagAndLull(long lag, long maxLullMs)` Maintains watermark that lags behind the top timestamp by the amount specified with `lag`.
    `static DistributedSupplier<WatermarkPolicy>` | `limitingTimestampAndWallClockLag(long timestampLag, long wallClockLag)` Maintains watermark that lags behind the top timestamp by at most `timestampLag` and behind wall-clock time by at most `wallClockLag`.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Method Detail

    - #### limitingLag

          @Nonnull
          public static DistributedSupplier<WatermarkPolicy> limitingLag(long lag)

      Maintains a watermark that lags behind the top observed timestamp by the given amount. In the case of a stream lull the watermark does not advance towards the top observed timestamp and remains behind it indefinitely.
      Parameters:
        `lag` - the desired difference between the top observed timestamp and the watermark

    - #### limitingLagAndDelay

          @Nonnull
          public static DistributedSupplier<WatermarkPolicy> limitingLagAndDelay(long lag,
                                                                                          long maxDelayMs)

      Maintains a watermark that lags behind the top observed timestamp by at most the given amount and is additionally guaranteed to reach the timestamp of any given event within `maxDelayMs` of system time after observing it.

      Keep in mind that the scope of the "top observed timestamp" is only the items received by an individual processor: the watermark of a processor using this policy will never advance beyond the timestamp it observed in isolation from the rest of the system. In the case of severe stream skew a given processor's watermark may lag significantly behind that of other processors and cause delays when getting coalesced with them in downstream processors. To overcome this you can also configure `JobConfig.setMaxWatermarkRetainMillis`.

      Parameters:
        `lag` - upper bound on the difference between the top observed timestamp and the watermark
        `maxDelayMs` - upper bound (in milliseconds) on how long it can take for the watermark to reach any observed event's timestamp

    - #### limitingTimestampAndWallClockLag

          @Nonnull
          public static DistributedSupplier<WatermarkPolicy> limitingTimestampAndWallClockLag(long timestampLag,
                                                                                                       long wallClockLag)

      Maintains watermark that lags behind the top timestamp by at most `timestampLag` and behind wall-clock time by at most `wallClockLag`. It assumes that the event timestamp is in milliseconds since Unix epoch and will use that fact to correlate it with wall-clock time acquired from the underlying OS. Note that wall-clock time is non-monotonic and sudden jumps that may occur in it will cause temporary disruptions in the functioning of this policy.

      In most cases the `limitingLagAndLull` policy should be preferred; this is a backup option for cases where some substreams may never see an event.

      Parameters:
        `timestampLag` - maximum difference between the top observed timestamp and the watermark
        `wallClockLag` - maximum difference between the current value of `System.currentTimeMillis` and the watermark

    - #### limitingLagAndLull

          @Nonnull
          public static DistributedSupplier<WatermarkPolicy> limitingLagAndLull(long lag,
                                                                                         long maxLullMs)

      Maintains watermark that lags behind the top timestamp by the amount specified with `lag`. Assumes that the event timestamp is given in milliseconds and will use that fact to correlate it with the passage of system time. There is no requirement on any specific point of origin for the timestamp, i.e., the zero value can denote any point in time as long as it is fixed.

      When the defined `maxLullMs` period elapses without observing more events, watermark will start advancing in lockstep with system time acquired from the underlying OS's monotonic clock.

      If no event is ever observed, watermark will advance from the initial value of `Long.MIN_VALUE`. Therefore this policy can be used only when there is a guarantee that each substream will emit at least one event that will initialize the timestamp. Otherwise the empty substream will hold back the processing of all other substreams by keeping the watermark below any realistic value. Configuring `WatermarkGenerationParams.idleTimeoutMillis()` will cope with this.

      Parameters:
        `lag` - the desired difference between the top observed timestamp and the watermark
        `maxLullMs` - maximum duration of a lull period before starting to advance watermark with system time

Copyright © 2018. All rights reserved.