com.hazelcast.jet.pipeline

## Interface StreamStageWithKey<T,K>

- Type Parameters:
    `T` - type of the stream items
    `K` - type of the key
  All Superinterfaces:
    GeneralStageWithKey<T,K>

      public interface StreamStageWithKey<T,K>
      extends GeneralStageWithKey<T,K>

  An intermediate step while constructing a windowed group-and-aggregate pipeline stage. It captures the grouping key and offers a method to specify the window definition.

- - ### Method Summary

    All Methods Instance Methods Abstract Methods Default Methods
    Modifier and Type | Method and Description
    --- | ---
    `<C> StreamStage<T>` | `filterUsingContext(ContextFactory<C> contextFactory, DistributedTriPredicate<? super C,? super K,? super T> filterFn)` Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it.
    `<C,R> StreamStage<R>` | `flatMapUsingContext(ContextFactory<C> contextFactory, DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn)` Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all the items from the `Traverser` it returns as the output items.
    `<C,R> StreamStage<R>` | `mapUsingContext(ContextFactory<C> contextFactory, DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn)` Attaches a mapping stage which applies the given function to each input item independently and emits the function's result as the output item.
    `default <V,R> StreamStage<R>` | `mapUsingIMap(IMap<K,V> iMap, DistributedBiFunction<? super T,? super V,? extends R> mapFn)` Attaches a `GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap`.
    `default <V,R> StreamStage<R>` | `mapUsingIMap(String mapName, DistributedBiFunction<? super T,? super V,? extends R> mapFn)` Attaches a `GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap` with the supplied name.
    `default <R> StreamStage<Map.Entry<K,R>>` | `rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)` A shortcut for: `aggregateRolling(aggrOp, Util::entry)`.
    `<R,OUT> StreamStage<OUT>` | `rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp, DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)` Attaches a rolling aggregation stage.
    `StageWithKeyAndWindow<T,K>` | `window(WindowDefinition wDef)` Adds the definition of the window to use in the group-and-aggregate pipeline stage being constructed.

    - ### Methods inherited from interface com.hazelcast.jet.pipeline.GeneralStageWithKey

      `customTransform, keyFn`

- - ### Method Detail

    - #### window

          @Nonnull
          StageWithKeyAndWindow<T,K> window(@Nonnull
                                                     WindowDefinition wDef)

      Adds the definition of the window to use in the group-and-aggregate pipeline stage being constructed.

    - #### mapUsingIMap

          @Nonnull
          default <V,R> StreamStage<R> mapUsingIMap(@Nonnull
                                                             String mapName,
                                                             @Nonnull
                                                             DistributedBiFunction<? super T,? super V,? extends R> mapFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a `GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap` with the supplied name. Jet will use the specified key function to retrieve a value from the map and pass it to the mapping function you supply, as the second argument.

      This stage is similar to `stageWithoutKey.mapUsingIMap()`, but here Jet knows the key and uses it to partition and distribute the input in order to achieve data locality. The value it fetches from the `IMap` is stored on the cluster member where the processing takes place. However, if the map doesn't use the default partitioning strategy, the data locality will be broken.

      Specified by:
        `mapUsingIMap` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `V` - type of the value in the `IMap`
        `R` - type of the output item
      Parameters:
        `mapName` - name of the `IMap`
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### mapUsingIMap

          @Nonnull
          default <V,R> StreamStage<R> mapUsingIMap(@Nonnull
                                                             IMap<K,V> iMap,
                                                             @Nonnull
                                                             DistributedBiFunction<? super T,? super V,? extends R> mapFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a `GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap`. It is not necessarily the map you provide here, but a map with the same name in the Jet cluster that executes the pipeline. Jet will use the specified key function to retrieve a value from the map and pass it to the mapping function you supply, as the second argument.

      This stage is similar to `stageWithoutKey.mapUsingIMap()`, but here Jet knows the key and uses it to partition and distribute the input in order to achieve data locality. The value it fetches from the `IMap` is stored on the cluster member where the processing takes place. However, if the map doesn't use the default partitioning strategy, the data locality will be broken.

      Specified by:
        `mapUsingIMap` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `V` - type of the value in the `IMap`
        `R` - type of the output item
      Parameters:
        `iMap` - the `IMap` to use as the context
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### mapUsingContext

          @Nonnull
          <C,R> StreamStage<R> mapUsingContext(@Nonnull
                                                        ContextFactory<C> contextFactory,
                                                        @Nonnull
                                                        DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a mapping stage which applies the given function to each input item independently and emits the function's result as the output item. The mapping function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`. If the mapping result is `null`, it emits nothing. Therefore this stage can be used to implement filtering semantics as well.

      Jet uses the `key-extracting function` specified on this stage for partitioning: all the items with the same key will see the same context instance (but note that the same instance serves many keys). One case where this is useful is fetching data from an external system because you can use a near-cache without duplicating the cached data.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `mapUsingContext` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `C` - type of context object
        `R` - the result type of the mapping function
      Parameters:
        `contextFactory` - the context factory
        `mapFn` - a stateless mapping function
      Returns:
        the newly attached stage

    - #### filterUsingContext

          @Nonnull
          <C> StreamStage<T> filterUsingContext(@Nonnull
                                                         ContextFactory<C> contextFactory,
                                                         @Nonnull
                                                         DistributedTriPredicate<? super C,? super K,? super T> filterFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it. The predicate function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`.

      Jet uses the `key-extracting function` specified on this stage for partitioning: all the items with the same key will see the same context instance (but note that the same instance serves many keys). One case where this is useful is fetching data from an external system because you can use a near-cache without duplicating the cached data.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `filterUsingContext` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `C` - type of context object
      Parameters:
        `contextFactory` - the context factory
        `filterFn` - a stateless filter predicate function
      Returns:
        the newly attached stage

    - #### flatMapUsingContext

          @Nonnull
          <C,R> StreamStage<R> flatMapUsingContext(@Nonnull
                                                            ContextFactory<C> contextFactory,
                                                            @Nonnull
                                                            DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all the items from the `Traverser` it returns as the output items. The traverser must be null-terminated. The mapping function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`.

      Jet uses the `key-extracting function` specified on this stage for partitioning: all the items with the same key will see the same context instance (but note that the same instance serves many keys). One case where this is useful is fetching data from an external system because you can use a near-cache without duplicating the cached data.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `flatMapUsingContext` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `C` - type of context object
        `R` - type of the output items
      Parameters:
        `contextFactory` - the context factory
        `flatMapFn` - a stateless flatmapping function
      Returns:
        the newly attached stage

    - #### rollingAggregate

          @Nonnull
          <R,OUT> StreamStage<OUT> rollingAggregate(@Nonnull
                                                             AggregateOperation1<? super T,?,? extends R> aggrOp,
                                                             @Nonnull
                                                             DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)

      Description copied from interface: `GeneralStageWithKey`
      Attaches a rolling aggregation stage. As opposed to regular aggregation, this stage emits the current aggregation result after receiving each item. For example, if your aggregation is summing and the input under a given key is `{2, 7, 8, -5}`, the output will be `{2, 9, 17, 12}`.

      This stage is fault-tolerant and saves its state to the snapshot.

      NOTE: if you plan to use an aggregate operation whose result size grows with input size (such as `toList` and your data source is unbounded, carefully consider the memory demands this implies. The result will keep growing forever.

      Specified by:
        `rollingAggregate` in interface `GeneralStageWithKey<T,K>`
      Type Parameters:
        `R` - type of the aggregate operation result
        `OUT` - type of the output item
      Parameters:
        `aggrOp` - the aggregate operation to perform
        `mapToOutputFn` - function that transforms the key and the aggregation result into the output item
      Returns:
        the newly attached stage

    - #### rollingAggregate

          @Nonnull
          default <R> StreamStage<Map.Entry<K,R>> rollingAggregate(@Nonnull
                                                                            AggregateOperation1<? super T,?,? extends R> aggrOp)

      Description copied from interface: `GeneralStageWithKey`
      A shortcut for:

      > `aggregateRolling(aggrOp, Util::entry)`.

      Specified by:
        `rollingAggregate` in interface `GeneralStageWithKey<T,K>`

Copyright © 2018. All rights reserved.