com.hazelcast.jet.pipeline

## Interface BatchStage<T>

- Type Parameters:
    `T` - the type of items coming out of this stage
  All Superinterfaces:
    GeneralStage<T>, Stage

      public interface BatchStage<T>
      extends GeneralStage<T>

  A stage in a distributed computation `pipeline` that will observe a finite amount of data (a batch). It accepts input from its upstream stages (if any) and passes its output to its downstream stages.

- - ### Method Summary

    All Methods Instance Methods Abstract Methods Default Methods
    Modifier and Type | Method and Description
    --- | ---
    `<R> BatchStage<R>` | `aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)` Attaches a stage that performs the given aggregate operation over all the items it receives.
    `default <T1,R0,R1> BatchStage<Tuple2<R0,R1>>` | `aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0, BatchStage<T1> stage1, AggregateOperation1<? super T1,?,? extends R1> aggrOp1)` Attaches a stage that co-aggregates the data from this and the supplied stage by performing a separate aggregate operation on each and emitting a `Tuple2` with their results.
    `default <T1,R0,R1,OUT> BatchStage<OUT>` | `aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0, BatchStage<T1> stage1, AggregateOperation1<? super T1,?,? extends R1> aggrOp1, DistributedBiFunction<? super R0,? super R1,? extends OUT> mapToOutputFn)` Attaches a stage that co-aggregates the data from this and the supplied stage by performing a separate aggregate operation on each and then passing both results to `mapToOutputFn`, which transforms them into the final output.
    `<T1,R> BatchStage<R>` | `aggregate2(BatchStage<T1> stage1, AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)` Attaches a stage that performs the given aggregate operation over all the items it receives from both this stage and `stage1` you supply.
    `default <T1,T2,R0,R1,R2> BatchStage<Tuple3<R0,R1,R2>>` | `aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0, BatchStage<T1> stage1, AggregateOperation1<? super T1,?,? extends R1> aggrOp1, BatchStage<T2> stage2, AggregateOperation1<? super T2,?,? extends R2> aggrOp2)` Attaches a stage that co-aggregates the data from this and the two supplied stages by performing a separate aggregate operation on each and emitting a `Tuple3` with their results.
    `default <T1,T2,R0,R1,R2,OUT> BatchStage<OUT>` | `aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0, BatchStage<T1> stage1, AggregateOperation1<? super T1,?,? extends R1> aggrOp1, BatchStage<T2> stage2, AggregateOperation1<? super T2,?,? extends R2> aggrOp2, DistributedTriFunction<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)` Attaches a stage that co-aggregates the data from this and the two supplied stages by performing a separate aggregate operation on each and then passing all three results to `mapToOutputFn`, which transforms them into the final output.
    `<T1,T2,R> BatchStage<R>` | `aggregate3(BatchStage<T1> stage1, BatchStage<T2> stage2, AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)` Attaches a stage that performs the given aggregate operation over all the items it receives from this stage as well as `stage1` and `stage2` you supply.
    `default AggregateBuilder1<T>` | `aggregateBuilder()` Offers a step-by-step API to build a pipeline stage that co-aggregates the data from several input stages.
    `default <R0> AggregateBuilder<R0>` | `aggregateBuilder(AggregateOperation1<? super T,?,? extends R0> aggrOp0)` Offers a step-by-step API to build a pipeline stage that co-aggregates the data from several input stages.
    `<R> BatchStage<R>` | `customTransform(String stageName, DistributedSupplier<Processor> procSupplier)` Attaches a stage with a custom transform based on the provided supplier of Core API `Processor`s.
    `default BatchStage<T>` | `distinct()` Attaches a stage that emits just the items that are distinct according to their definition of equality (`equals` and `hashCode`).
    `BatchStage<T>` | `filter(DistributedPredicate<T> filterFn)` Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it.
    `<C> BatchStage<T>` | `filterUsingContext(ContextFactory<C> contextFactory, DistributedBiPredicate<? super C,? super T> filterFn)` Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it.
    `<R> BatchStage<R>` | `flatMap(DistributedFunction<? super T,? extends Traverser<? extends R>> flatMapFn)` Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all the items from the `Traverser` it returns.
    `<C,R> BatchStage<R>` | `flatMapUsingContext(ContextFactory<C> contextFactory, DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn)` Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all items from the `Traverser` it returns as the output items.
    `<K> BatchStageWithKey<T,K>` | `groupingKey(DistributedFunction<? super T,? extends K> keyFn)` Specifies the function that will extract a key from the items in the associated pipeline stage.
    `<K,T1_IN,T1,R> BatchStage<R>` | `hashJoin(BatchStage<T1_IN> stage1, JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1, DistributedBiFunction<T,T1,R> mapToOutputFn)` Attaches to both this and the supplied stage a hash-joining stage and returns it.
    `<K1,K2,T1_IN,T2_IN,T1,T2,R> BatchStage<R>` | `hashJoin2(BatchStage<T1_IN> stage1, JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1, BatchStage<T2_IN> stage2, JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2, DistributedTriFunction<T,T1,T2,R> mapToOutputFn)` Attaches to this and the two supplied stages a hash-joining stage and returns it.
    `default HashJoinBuilder<T>` | `hashJoinBuilder()` Returns a fluent API builder object to construct a hash join operation with any number of contributing stages.
    `<R> BatchStage<R>` | `map(DistributedFunction<? super T,? extends R> mapFn)` Attaches a mapping stage which applies the supplied function to each input item independently and emits the function's result as the output item.
    `<C,R> BatchStage<R>` | `mapUsingContext(ContextFactory<C> contextFactory, DistributedBiFunction<? super C,? super T,? extends R> mapFn)` Attaches a mapping stage which applies the supplied function to each input item independently and emits the function's result as the output item.
    `default <K,V,R> BatchStage<R>` | `mapUsingIMap(IMap<K,V> iMap, DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)` Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap`.
    `default <K,V,R> BatchStage<R>` | `mapUsingIMap(String mapName, DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)` Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap` with the supplied name.
    `default <K,V,R> BatchStage<R>` | `mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap, DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)` Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `ReplicatedMap`.
    `default <K,V,R> BatchStage<R>` | `mapUsingReplicatedMap(String mapName, DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)` Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `ReplicatedMap` with the supplied name.
    `BatchStage<T>` | `merge(BatchStage<? extends T> other)` Attaches a stage that emits all the items from this stage as well as all the items from the supplied stage.
    `default BatchStage<T>` | `peek()` Adds a peeking layer to this compute stage which logs its output.
    `default BatchStage<T>` | `peek(DistributedFunction<? super T,? extends CharSequence> toStringFn)` Adds a peeking layer to this compute stage which logs its output.
    `BatchStage<T>` | `peek(DistributedPredicate<? super T> shouldLogFn, DistributedFunction<? super T,? extends CharSequence> toStringFn)` Attaches a peeking stage which logs this stage's output and passes it through without transformation.
    `<R> BatchStage<R>` | `rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)` Attaches a rolling aggregation stage.
    `BatchStage<T>` | `setLocalParallelism(int localParallelism)` Sets the preferred local parallelism (number of processors per Jet cluster member) this stage will configure its DAG vertices with.
    `BatchStage<T>` | `setName(String name)` Overrides the default name of the stage with the name you choose and returns the stage.

    - ### Methods inherited from interface com.hazelcast.jet.pipeline.GeneralStage

      `addTimestamps, addTimestamps, drainTo`

    - ### Methods inherited from interface com.hazelcast.jet.pipeline.Stage

      `getPipeline, name`

- - ### Method Detail

    - #### groupingKey

          @Nonnull
          <K> BatchStageWithKey<T,K> groupingKey(@Nonnull
                                                          DistributedFunction<? super T,? extends K> keyFn)

      Specifies the function that will extract a key from the items in the associated pipeline stage. This enables the operations that need the key, such as grouped aggregation. The key must not be null and must properly implement `equals()` and `hashCode()`.
      Specified by:
        `groupingKey` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - type of the key
      Parameters:
        `keyFn` - function that extracts the grouping key

    - #### map

          @Nonnull
          <R> BatchStage<R> map(@Nonnull
                                         DistributedFunction<? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a mapping stage which applies the supplied function to each input item independently and emits the function's result as the output item. If the result is `null`, it emits nothing. Therefore this stage can be used to implement filtering semantics as well.
      Specified by:
        `map` in interface `GeneralStage<T>`
      Type Parameters:
        `R` - the result type of the mapping function
      Parameters:
        `mapFn` - a stateless mapping function
      Returns:
        the newly attached stage

    - #### filter

          @Nonnull
          BatchStage<T> filter(@Nonnull
                                        DistributedPredicate<T> filterFn)

      Description copied from interface: `GeneralStage`
      Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it. Returns the newly attached stage.
      Specified by:
        `filter` in interface `GeneralStage<T>`
      Parameters:
        `filterFn` - a stateless filter predicate function
      Returns:
        the newly attached stage

    - #### flatMap

          @Nonnull
          <R> BatchStage<R> flatMap(@Nonnull
                                             DistributedFunction<? super T,? extends Traverser<? extends R>> flatMapFn)

      Description copied from interface: `GeneralStage`
      Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all the items from the `Traverser` it returns. The traverser must be null-terminated.
      Specified by:
        `flatMap` in interface `GeneralStage<T>`
      Type Parameters:
        `R` - the type of items in the result's traversers
      Parameters:
        `flatMapFn` - a stateless flatmapping function, whose result type is Jet's `Traverser`
      Returns:
        the newly attached stage

    - #### mapUsingContext

          @Nonnull
          <C,R> BatchStage<R> mapUsingContext(@Nonnull
                                                       ContextFactory<C> contextFactory,
                                                       @Nonnull
                                                       DistributedBiFunction<? super C,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a mapping stage which applies the supplied function to each input item independently and emits the function's result as the output item. The mapping function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`.

      If the mapping result is `null`, it emits nothing. Therefore this stage can be used to implement filtering semantics as well.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `mapUsingContext` in interface `GeneralStage<T>`
      Type Parameters:
        `C` - type of context object
        `R` - the result type of the mapping function
      Parameters:
        `contextFactory` - the context factory
        `mapFn` - a stateless mapping function
      Returns:
        the newly attached stage

    - #### filterUsingContext

          @Nonnull
          <C> BatchStage<T> filterUsingContext(@Nonnull
                                                        ContextFactory<C> contextFactory,
                                                        @Nonnull
                                                        DistributedBiPredicate<? super C,? super T> filterFn)

      Description copied from interface: `GeneralStage`
      Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it. The predicate function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `filterUsingContext` in interface `GeneralStage<T>`
      Type Parameters:
        `C` - type of context object
      Parameters:
        `contextFactory` - the context factory
        `filterFn` - a stateless filter predicate function
      Returns:
        the newly attached stage

    - #### flatMapUsingContext

          @Nonnull
          <C,R> BatchStage<R> flatMapUsingContext(@Nonnull
                                                           ContextFactory<C> contextFactory,
                                                           @Nonnull
                                                           DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn)

      Description copied from interface: `GeneralStage`
      Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all items from the `Traverser` it returns as the output items. The traverser must be null-terminated. The mapping function receives another parameter, the context object, which Jet will create using the supplied `contextFactory`.

      ### Interaction with fault-tolerant unbounded jobs

      If you use this stage in a fault-tolerant unbounded job, keep in mind that any state the context object maintains doesn't participate in Jet's fault tolerance protocol. If the state is local, it will be lost after a job restart; if it is saved to some durable storage, the state of that storage won't be rewound to the last checkpoint, so you'll perform duplicate updates.
      Specified by:
        `flatMapUsingContext` in interface `GeneralStage<T>`
      Type Parameters:
        `C` - type of context object
        `R` - the type of items in the result's traversers
      Parameters:
        `contextFactory` - the context factory
        `flatMapFn` - a stateless flatmapping function, whose result type is Jet's `Traverser`
      Returns:
        the newly attached stage

    - #### mapUsingReplicatedMap

          @Nonnull
          default <K,V,R> BatchStage<R> mapUsingReplicatedMap(@Nonnull
                                                                       String mapName,
                                                                       @Nonnull
                                                                       DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `ReplicatedMap` with the supplied name. The mapping function will receive it as the first argument.
      Specified by:
        `mapUsingReplicatedMap` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - type of the key in the `ReplicatedMap`
        `V` - type of the value in the `ReplicatedMap`
        `R` - type of the output item
      Parameters:
        `mapName` - name of the `ReplicatedMap`
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### mapUsingReplicatedMap

          @Nonnull
          default <K,V,R> BatchStage<R> mapUsingReplicatedMap(@Nonnull
                                                                       ReplicatedMap<K,V> replicatedMap,
                                                                       @Nonnull
                                                                       DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `ReplicatedMap`. It is not necessarily the map you provide here, but a replicated map with the same name in the Jet cluster that executes the pipeline. The mapping function will receive the replicated map as the first argument.
      Specified by:
        `mapUsingReplicatedMap` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - type of the key in the `ReplicatedMap`
        `V` - type of the value in the `ReplicatedMap`
        `R` - type of the output item
      Parameters:
        `replicatedMap` - the `ReplicatedMap` to use as context
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### mapUsingIMap

          @Nonnull
          default <K,V,R> BatchStage<R> mapUsingIMap(@Nonnull
                                                              String mapName,
                                                              @Nonnull
                                                              DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap` with the supplied name. The mapping function will receive it as the first argument.
      Specified by:
        `mapUsingIMap` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - type of the key in the `IMap`
        `V` - type of the value in the `IMap`
        `R` - type of the output item
      Parameters:
        `mapName` - name of the `IMap`
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### mapUsingIMap

          @Nonnull
          default <K,V,R> BatchStage<R> mapUsingIMap(@Nonnull
                                                              IMap<K,V> iMap,
                                                              @Nonnull
                                                              DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)

      Description copied from interface: `GeneralStage`
      Attaches a `GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>)` stage where the context is a Hazelcast `IMap`. It is not necessarily the map you provide here, but a map with the same name in the Jet cluster that executes the pipeline. The mapping function will receive the replicated map as the first argument.
      Specified by:
        `mapUsingIMap` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - type of the key in the `IMap`
        `V` - type of the value in the `IMap`
        `R` - type of the output item
      Parameters:
        `iMap` - the `IMap` to use as the context
        `mapFn` - the mapping function
      Returns:
        the newly attached stage

    - #### rollingAggregate

          @Nonnull
          <R> BatchStage<R> rollingAggregate(@Nonnull
                                                      AggregateOperation1<? super T,?,? extends R> aggrOp)

      Description copied from interface: `GeneralStage`
      Attaches a rolling aggregation stage. As opposed to regular aggregation, this stage emits the current aggregation result after receiving each item. For example, if your aggregation is summing and the input is `{2, 7, 8, -5}`, the output will be `{2, 9, 17, 12}`. The number of input and output items is equal.

      This stage is fault-tolerant and saves its state to the snapshot.

      NOTE 1: since the output for each item depends on all the previous items, this operation cannot be parallelized. Jet will perform it on a single member, single-threaded. Jet also supports `keyed rolling aggregation` which it can parallelize by partitioning.

      NOTE 2: if you plan to use an aggregate operation whose result size grows with input size (such as `toList` and your data source is unbounded, you must carefully consider the memory demands this implies. The result will keep growing forever.

      Specified by:
        `rollingAggregate` in interface `GeneralStage<T>`
      Type Parameters:
        `R` - result type of the aggregate operation
      Parameters:
        `aggrOp` - the aggregate operation to do the aggregation
      Returns:
        the newly attached stage

    - #### distinct

          @Nonnull
          default BatchStage<T> distinct()

      Attaches a stage that emits just the items that are distinct according to their definition of equality (`equals` and `hashCode`). There is no guarantee which one of equal items it will emit.
      Returns:
        the newly attached stage

    - #### merge

          @Nonnull
          BatchStage<T> merge(@Nonnull
                                       BatchStage<? extends T> other)

      Attaches a stage that emits all the items from this stage as well as all the items from the supplied stage. The other stage's type parameter must be assignment-compatible with this stage's type parameter.
      Parameters:
        `other` - the other stage whose data to merge into this one
      Returns:
        the newly attached stage

    - #### hashJoin

          @Nonnull
          <K,T1_IN,T1,R> BatchStage<R> hashJoin(@Nonnull
                                                         BatchStage<T1_IN> stage1,
                                                         @Nonnull
                                                         JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
                                                         @Nonnull
                                                         DistributedBiFunction<T,T1,R> mapToOutputFn)

      Description copied from interface: `GeneralStage`
      Attaches to both this and the supplied stage a hash-joining stage and returns it. This stage plays the role of the primary stage in the hash-join. Please refer to the `package javadoc` for a detailed description of the hash-join transform.
      Specified by:
        `hashJoin` in interface `GeneralStage<T>`
      Type Parameters:
        `K` - the type of the join key
        `T1_IN` - the type of `stage1` items
        `T1` - the result type of projection on `stage1` items
        `R` - the resulting output type
      Parameters:
        `stage1` - the stage to hash-join with this one
        `joinClause1` - specifies how to join the two streams
        `mapToOutputFn` - function to map the joined items to the output value
      Returns:
        the newly attached stage

    - #### hashJoin2

          @Nonnull
          <K1,K2,T1_IN,T2_IN,T1,T2,R> BatchStage<R> hashJoin2(@Nonnull
                                                                       BatchStage<T1_IN> stage1,
                                                                       @Nonnull
                                                                       JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
                                                                       @Nonnull
                                                                       BatchStage<T2_IN> stage2,
                                                                       @Nonnull
                                                                       JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
                                                                       @Nonnull
                                                                       DistributedTriFunction<T,T1,T2,R> mapToOutputFn)

      Description copied from interface: `GeneralStage`
      Attaches to this and the two supplied stages a hash-joining stage and returns it. This stage plays the role of the primary stage in the hash-join. Please refer to the `package javadoc` for a detailed description of the hash-join transform.
      Specified by:
        `hashJoin2` in interface `GeneralStage<T>`
      Type Parameters:
        `K1` - the type of key for `stage1`
        `K2` - the type of key for `stage2`
        `T1_IN` - the type of `stage1` items
        `T2_IN` - the type of `stage2` items
        `T1` - the result type of projection of `stage1` items
        `T2` - the result type of projection of `stage2` items
        `R` - the resulting output type
      Parameters:
        `stage1` - the first stage to join
        `joinClause1` - specifies how to join with `stage1`
        `stage2` - the second stage to join
        `joinClause2` - specifies how to join with `stage2`
        `mapToOutputFn` - function to map the joined items to the output value
      Returns:
        the newly attached stage

    - #### hashJoinBuilder

          @Nonnull
          default HashJoinBuilder<T> hashJoinBuilder()

      Description copied from interface: `GeneralStage`
      Returns a fluent API builder object to construct a hash join operation with any number of contributing stages. It is mainly intended for hash-joins with three or more enriching stages. For one or two stages prefer the direct `stage.hashJoinN(...)` calls because they offer more static type safety.
      Specified by:
        `hashJoinBuilder` in interface `GeneralStage<T>`

    - #### aggregate

          @Nonnull
          <R> BatchStage<R> aggregate(@Nonnull
                                               AggregateOperation1<? super T,?,? extends R> aggrOp)

      Attaches a stage that performs the given aggregate operation over all the items it receives. The aggregating stage emits a single item.
      Type Parameters:
        `R` - the type of the result
      Parameters:
        `aggrOp` - the aggregate operation to perform
      See Also:
        `AggregateOperations`

    - #### aggregate2

          @Nonnull
          <T1,R> BatchStage<R> aggregate2(@Nonnull
                                                   BatchStage<T1> stage1,
                                                   @Nonnull
                                                   AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)

      Attaches a stage that performs the given aggregate operation over all the items it receives from both this stage and `stage1` you supply. This variant requires you to provide a two-input aggregate operation (refer to its Javadoc for a simple example). If you can express your logic in terms of two single-input aggregate operations, one for each input stream, then you should use `stage0.aggregate2(aggrOp0, stage1, aggrOp1)` because it offers a simpler API and you can use the already defined single-input operations. Use this variant only when you have the need to implement an aggregate operation that combines the input streams into the same accumulator.

      The returned stage emits a single item.

      Type Parameters:
        `T1` - type of items in `stage1`
        `R` - type of the result
      Parameters:
        `aggrOp` - the aggregate operation to perform
      See Also:
        `AggregateOperations`

    - #### aggregate2

          @Nonnull
          default <T1,R0,R1,OUT> BatchStage<OUT> aggregate2(@Nonnull
                                                                     AggregateOperation1<? super T,?,? extends R0> aggrOp0,
                                                                     @Nonnull
                                                                     BatchStage<T1> stage1,
                                                                     @Nonnull
                                                                     AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
                                                                     @Nonnull
                                                                     DistributedBiFunction<? super R0,? super R1,? extends OUT> mapToOutputFn)

      Attaches a stage that co-aggregates the data from this and the supplied stage by performing a separate aggregate operation on each and then passing both results to `mapToOutputFn`, which transforms them into the final output.

      The returned stage emits a single item.

      Type Parameters:
        `T1` - type of the items in the other stage
        `R0` - type of the aggregated result for this stage
        `R1` - type of the aggregated result for the other stage
        `OUT` - the output item type
      Parameters:
        `aggrOp0` - aggregate operation to perform on this stage
        `stage1` - the other stage
        `aggrOp1` - aggregate operation to perform on the other stage
        `mapToOutputFn` - function to apply to the aggregated results

    - #### aggregate2

          @Nonnull
          default <T1,R0,R1> BatchStage<Tuple2<R0,R1>> aggregate2(@Nonnull
                                                                           AggregateOperation1<? super T,?,? extends R0> aggrOp0,
                                                                           @Nonnull
                                                                           BatchStage<T1> stage1,
                                                                           @Nonnull
                                                                           AggregateOperation1<? super T1,?,? extends R1> aggrOp1)

      Attaches a stage that co-aggregates the data from this and the supplied stage by performing a separate aggregate operation on each and emitting a `Tuple2` with their results.

      The returned stage emits a single item.

      Type Parameters:
        `T1` - type of the items in the other stage
        `R0` - type of the aggregated result for this stage
        `R1` - type of the aggregated result for the other stage
      Parameters:
        `aggrOp0` - aggregate operation to perform on this stage
        `stage1` - the other stage
        `aggrOp1` - aggregate operation to perform on the other stage

    - #### aggregate3

          @Nonnull
          <T1,T2,R> BatchStage<R> aggregate3(@Nonnull
                                                      BatchStage<T1> stage1,
                                                      @Nonnull
                                                      BatchStage<T2> stage2,
                                                      @Nonnull
                                                      AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)

      Attaches a stage that performs the given aggregate operation over all the items it receives from this stage as well as `stage1` and `stage2` you supply. This variant requires you to provide a three-input aggregate operation (refer to its Javadoc for a simple example). If you can express your logic in terms of two single-input aggregate operations, one for each input stream, then you should use `stage0.aggregate3(aggrOp0, stage1, aggrOp1, stage2, aggrOp2)` because it offers a simpler API and you can use the already defined single-input operations. Use this variant only when you have the need to implement an aggregate operation that combines the input streams into the same accumulator.

      The aggregating stage emits a single item.

      Type Parameters:
        `T1` - type of items in `stage1`
        `T2` - type of items in `stage2`
        `R` - type of the result
      Parameters:
        `aggrOp` - the aggregate operation to perform
      See Also:
        `AggregateOperations`

    - #### aggregate3

          @Nonnull
          default <T1,T2,R0,R1,R2,OUT> BatchStage<OUT> aggregate3(@Nonnull
                                                                           AggregateOperation1<? super T,?,? extends R0> aggrOp0,
                                                                           @Nonnull
                                                                           BatchStage<T1> stage1,
                                                                           @Nonnull
                                                                           AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
                                                                           @Nonnull
                                                                           BatchStage<T2> stage2,
                                                                           @Nonnull
                                                                           AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
                                                                           @Nonnull
                                                                           DistributedTriFunction<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)

      Attaches a stage that co-aggregates the data from this and the two supplied stages by performing a separate aggregate operation on each and then passing all three results to `mapToOutputFn`, which transforms them into the final output.

      The returned stage emits a single item.

      Type Parameters:
        `T1` - type of the items in `stage1`
        `T2` - type of the items in `stage2`
        `R0` - type of the aggregated result for this stage
        `R1` - type of the aggregated result for `stage1`
        `R2` - type of the aggregated result for `stage2`
        `OUT` - the output item type
      Parameters:
        `aggrOp0` - aggregate operation to perform on this stage
        `stage1` - the first additional stage
        `aggrOp1` - aggregate operation to perform on `stage1`
        `stage2` - the second additional stage
        `aggrOp2` - aggregate operation to perform on `stage2`
        `mapToOutputFn` - function to apply to the aggregated results

    - #### aggregate3

          @Nonnull
          default <T1,T2,R0,R1,R2> BatchStage<Tuple3<R0,R1,R2>> aggregate3(@Nonnull
                                                                                    AggregateOperation1<? super T,?,? extends R0> aggrOp0,
                                                                                    @Nonnull
                                                                                    BatchStage<T1> stage1,
                                                                                    @Nonnull
                                                                                    AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
                                                                                    @Nonnull
                                                                                    BatchStage<T2> stage2,
                                                                                    @Nonnull
                                                                                    AggregateOperation1<? super T2,?,? extends R2> aggrOp2)

      Attaches a stage that co-aggregates the data from this and the two supplied stages by performing a separate aggregate operation on each and emitting a `Tuple3` with their results.

      The returned stage emits a single item.

      Type Parameters:
        `T1` - type of the items in `stage1`
        `T2` - type of the items in `stage2`
        `R0` - type of the aggregated result for this stage
        `R1` - type of the aggregated result for `stage1`
        `R2` - type of the aggregated result for `stage2`
      Parameters:
        `aggrOp0` - aggregate operation to perform on this stage
        `stage1` - the first additional stage
        `aggrOp1` - aggregate operation to perform on `stage1`
        `stage2` - the second additional stage
        `aggrOp2` - aggregate operation to perform on `stage2`

    - #### aggregateBuilder

          @Nonnull
          default <R0> AggregateBuilder<R0> aggregateBuilder(AggregateOperation1<? super T,?,? extends R0> aggrOp0)

      Offers a step-by-step API to build a pipeline stage that co-aggregates the data from several input stages. The current stage will be already registered with the builder you get. You supply an aggregate operation for each input stage and in the output you get the individual aggregation results in a `Map.Entry(key, itemsByTag)`. Use the tag you get from `builder.add(stageN, aggrOpN)` to retrieve the aggregated result for that stage. Use `builder.tag0()` as the tag of this stage. You will also be able to supply a function to the builder that immediately transforms the `ItemsByTag` to the desired output type.

      This example counts the items in stage-0, sums those in stage-1 and takes the average of those in stage-2:

           BatchStage<Long> stage0 = p.drawFrom(source0);
           BatchStage<Long> stage1 = p.drawFrom(source1);
           BatchStage<Long> stage2 = p.drawFrom(source2);

           AggregateBuilder<Long> b = stage0.aggregateBuilder(
                   AggregateOperations.counting());
           Tag<Long> tag0 = b.tag0();
           Tag<Long> tag1 = b.add(stage1,
                   AggregateOperations.summingLong(Number::longValue));
           Tag<Double> tag2 = b.add(stage2,
                   AggregateOperations.averagingLong(Number::longValue));

           BatchStage<ItemsByTag> aggregated = b.build();
           aggregated.map(ibt -> String.format(
                   "Count of stage0: %d, sum of stage1: %d, average of stage2: %f",
                   ibt.get(tag0), ibt.get(tag1), ibt.get(tag2))
           );

    - #### aggregateBuilder

          @Nonnull
          default AggregateBuilder1<T> aggregateBuilder()

      Offers a step-by-step API to build a pipeline stage that co-aggregates the data from several input stages. The current stage will be already registered with the builder you get.

      This builder requires you to provide a multi-input aggregate operation. If you can express your logic in terms of single-input aggregate operations, one for each input stream, then you should use `stage0.aggregateBuilder(aggrOp0)` because it offers a simpler API. Use this builder only when you have the need to implement an aggregate operation that combines all the input streams into the same accumulator.

      This builder is mainly intended to build a co-aggregation of four or more contributing stages. For up to three stages, prefer the direct `stage.aggregateN(...)` calls because they offer more static type safety.

      To add the other stages, call `add(stage)`. Collect all the tags returned from `add()` and use them when building the aggregate operation. Retrieve the tag of the first stage (from which you obtained this builder) by calling `AggregateBuilder1.tag0()`.

      This example takes three streams of strings and counts the distinct strings across all of them:

           Pipeline p = Pipeline.create();
           BatchStage<String> stage0 = p.drawFrom(source0);
           BatchStage<String> stage1 = p.drawFrom(source1);
           BatchStage<String> stage2 = p.drawFrom(source2);
           AggregateBuilder1<String> b = stage0.aggregateBuilder();
           Tag<String> tag0 = b.tag0();
           Tag<String> tag1 = b.add(stage1);
           Tag<String> tag2 = b.add(stage2);
           BatchStage<Integer> aggregated = b.build(AggregateOperation
                   .withCreate(HashSet<String>::new)
                   .andAccumulate(tag0, (acc, item) -> acc.add(item))
                   .andAccumulate(tag1, (acc, item) -> acc.add(item))
                   .andAccumulate(tag2, (acc, item) -> acc.add(item))
                   .andCombine(HashSet::addAll)
                   .andFinish(HashSet::size));

    - #### peek

          @Nonnull
          default BatchStage<T> peek()

      Description copied from interface: `GeneralStage`
      Adds a peeking layer to this compute stage which logs its output. For each item the stage emits, it logs the result of its `toString()` method at the INFO level to the log category `com.hazelcast.jet.impl.processor.PeekWrappedP.<vertexName>#<processorIndex>`. The stage logs each item on whichever cluster member it happens to receive it. Its primary purpose is for development use, when running Jet on a local machine.
      Specified by:
        `peek` in interface `GeneralStage<T>`
      See Also:
        `GeneralStage.peek(DistributedPredicate, DistributedFunction)`, `GeneralStage.peek(DistributedFunction)`

    - #### peek

          @Nonnull
          BatchStage<T> peek(@Nonnull
                                      DistributedPredicate<? super T> shouldLogFn,
                                      @Nonnull
                                      DistributedFunction<? super T,? extends CharSequence> toStringFn)

      Description copied from interface: `GeneralStage`
      Attaches a peeking stage which logs this stage's output and passes it through without transformation. For each item the stage emits, it:
      1. uses the `shouldLogFn` predicate to see whether to log the item
      2. if yes, uses then uses `toStringFn` to get the item's string representation
      3. logs the string at the INFO level to the log category `com.hazelcast.jet.impl.processor.PeekWrappedP.<vertexName>#<processorIndex>`
      The stage logs each item on whichever cluster member it happens to receive it. Its primary purpose is for development use, when running Jet on a local machine.
      Specified by:
        `peek` in interface `GeneralStage<T>`
      Parameters:
        `shouldLogFn` - a function to filter the logged items. You can use `alwaysTrue()` as a pass-through filter when you don't need any filtering.
        `toStringFn` - a function that returns a string representation of the item
      See Also:
        `GeneralStage.peek(DistributedFunction)`, `GeneralStage.peek()`

    - #### peek

          @Nonnull
          default BatchStage<T> peek(@Nonnull
                                              DistributedFunction<? super T,? extends CharSequence> toStringFn)

      Description copied from interface: `GeneralStage`
      Adds a peeking layer to this compute stage which logs its output. For each item the stage emits, it:
      1. uses `toStringFn` to get a string representation of the item
      2. logs the string at the INFO level to the log category `com.hazelcast.jet.impl.processor.PeekWrappedP.<vertexName>#<processorIndex>`
      The stage logs each item on whichever cluster member it happens to receive it. Its primary purpose is for development use, when running Jet on a local machine.
      Specified by:
        `peek` in interface `GeneralStage<T>`
      Parameters:
        `toStringFn` - a function that returns a string representation of the item
      See Also:
        `GeneralStage.peek(DistributedPredicate, DistributedFunction)`, `GeneralStage.peek()`

    - #### customTransform

          @Nonnull
          <R> BatchStage<R> customTransform(@Nonnull
                                                     String stageName,
                                                     @Nonnull
                                                     DistributedSupplier<Processor> procSupplier)

      Description copied from interface: `GeneralStage`
      Attaches a stage with a custom transform based on the provided supplier of Core API `Processor`s.

      Note that the returned stage's type parameter is inferred from the call site and not propagated from the processor that will produce the result, so there is no actual type safety provided.

      Specified by:
        `customTransform` in interface `GeneralStage<T>`
      Type Parameters:
        `R` - the type of the output items
      Parameters:
        `stageName` - a human-readable name for the custom stage
        `procSupplier` - the supplier of processors

    - #### setLocalParallelism

          @Nonnull
          BatchStage<T> setLocalParallelism(int localParallelism)

      Description copied from interface: `Stage`
      Sets the preferred local parallelism (number of processors per Jet cluster member) this stage will configure its DAG vertices with. Jet always uses the same number of processors on each member, so the total parallelism automatically increases if another member joins the cluster.

      While most stages are backed by 1 vertex, there are exceptions. If a stage uses two vertices, each of them will have the given local parallelism, so in total there will be twice as many processors per member.

      The default value is -1 and it signals to Jet to figure out a default value. Jet will determine the vertex's local parallelism during job initialization from the global default and the processor meta-supplier's preferred value.

      Specified by:
        `setLocalParallelism` in interface `Stage`
      Returns:
        this stage

    - #### setName

          @Nonnull
          BatchStage<T> setName(@Nullable
                                         String name)

      Description copied from interface: `Stage`
      Overrides the default name of the stage with the name you choose and returns the stage. This can be useful for debugging purposes, to better distinguish pipeline stages in the diagnostic output.
      Specified by:
        `setName` in interface `Stage`
      Parameters:
        `name` - the stage name
      Returns:
        this stage

Copyright © 2018. All rights reserved.