com.hazelcast.jet.core

## Interface ProcessorMetaSupplier

- All Superinterfaces:
    Serializable
  Functional Interface:
    This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

      @FunctionalInterface
      public interface ProcessorMetaSupplier
      extends Serializable

  Factory of `ProcessorSupplier` instances. The starting point of the chain leading to the eventual creation of `Processor` instances on each cluster member:
  1. client creates `ProcessorMetaSupplier` as a part of the DAG;
  2. serializes it and sends to a cluster member;
  3. the member deserializes and uses it to create one `ProcessorSupplier` for each cluster member;
  4. serializes each `ProcessorSupplier` and sends it to its target member;
  5. the target member deserializes and uses it to instantiate as many instances of `Processor` as requested by the parallelism property on the corresponding `Vertex`.
  Before being asked to create `ProcessorSupplier`s this meta-supplier will be given access to the Hazelcast instance and, in particular, its cluster topology and partitioning services. It can use the information from these services to precisely parameterize each `Processor` instance that will be created on each member.
  Since:
    3.0

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Interface and Description
    --- | ---
    `static interface` | `ProcessorMetaSupplier.Context` Context passed to the meta-supplier at init time on the member that received a job request from the client.

  - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Default Methods
    Modifier and Type | Method and Description
    --- | ---
    `default void` | `close(Throwable error)` Called on coordinator member after execution has finished on all members, successfully or not.
    `static ProcessorMetaSupplier` | `forceTotalParallelismOne(ProcessorSupplier supplier)` Variant of `forceTotalParallelismOne(ProcessorSupplier, String)` where the node for the supplier will be chosen randomly.
    `static ProcessorMetaSupplier` | `forceTotalParallelismOne(ProcessorSupplier supplier, String partitionKey)` Wraps the provided `ProcessorSupplier` into a meta-supplier that will only use the given `ProcessorSupplier` on a single node.
    `Function<? super Address,? extends ProcessorSupplier>` | `get(List<Address> addresses)` Called to create a mapping from member `Address` to the `ProcessorSupplier` that will be sent to that member.
    `default void` | `init(ProcessorMetaSupplier.Context context)` Called on the cluster member that receives the client request, after deserializing the meta-supplier instance.
    `static ProcessorMetaSupplier` | `of(FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)` Factory method that creates a `ProcessorMetaSupplier` from the supplied function that maps a cluster member address to a `ProcessorSupplier`.
    `static ProcessorMetaSupplier` | `of(FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier, int preferredLocalParallelism)` Factory method that creates a `ProcessorMetaSupplier` from the supplied function that maps a cluster member address to a `ProcessorSupplier`.
    `static ProcessorMetaSupplier` | `of(int preferredLocalParallelism, ProcessorSupplier procSupplier)` Factory method that wraps the given `ProcessorSupplier` and returns the same instance for each given `Address`.
    `static ProcessorMetaSupplier` | `of(ProcessorSupplier procSupplier)` Wraps the provided `ProcessorSupplier` into a meta-supplier that will always return it.
    `static ProcessorMetaSupplier` | `of(SupplierEx<? extends Processor> procSupplier)` Factory method that wraps the given `Supplier<Processor>` and uses it as the supplier of all `Processor` instances.
    `static ProcessorMetaSupplier` | `of(SupplierEx<? extends Processor> procSupplier, int preferredLocalParallelism)` Factory method that wraps the given `Supplier<Processor>` and uses it as the supplier of all `Processor` instances.
    `static ProcessorMetaSupplier` | `preferLocalParallelismOne(ProcessorSupplier supplier)` Wraps the provided `ProcessorSupplier` into a meta-supplier that will always return it.
    `static ProcessorMetaSupplier` | `preferLocalParallelismOne(SupplierEx<? extends Processor> procSupplier)` Variant of `preferLocalParallelismOne(ProcessorSupplier)` where the supplied `SupplierEx<Processor>` will be wrapped into a `ProcessorSupplier`.
    `default int` | `preferredLocalParallelism()` Returns the local parallelism the vertex should be configured with.

- - ### Method Detail

    - #### preferredLocalParallelism

          default int preferredLocalParallelism()

      Returns the local parallelism the vertex should be configured with. The default implementation returns `Vertex.LOCAL_PARALLELISM_USE_DEFAULT`.

    - #### init

          default void init(@Nonnull
                            ProcessorMetaSupplier.Context context)
                     throws Exception

      Called on the cluster member that receives the client request, after deserializing the meta-supplier instance. Gives access to the Hazelcast instance's services and provides the parallelism parameters determined from the cluster size.
      Throws:
        `Exception`

    - #### get

          @Nonnull
          Function<? super Address,? extends ProcessorSupplier> get(@Nonnull
                                                                             List<Address> addresses)

      Called to create a mapping from member `Address` to the `ProcessorSupplier` that will be sent to that member. Jet calls this method with a list of all cluster members' addresses and the returned function must be a mapping that returns a non-null value for each given address.

      The method will be called once per job execution on the job's coordinator member. `init()` will have already been called.

    - #### close

          default void close(@Nullable
                             Throwable error)
                      throws Exception

      Called on coordinator member after execution has finished on all members, successfully or not. This method will be called after `ProcessorSupplier.close(Throwable)` has been called on all available members.

      If there is an exception during the creation of the execution plan, this method will be called regardless of whether the `init()` or `get()` method have been called or not. If this method throws an exception, it will be logged and ignored; it won't be reported as a job failure.

      If you rely on the fact that this method is run once per cluster, it can happen that it is not called at all, if the coordinator member crashed.

      Parameters:
        `error` - the exception (if any) that caused the job to fail; `null` in the case of successful job completion
      Throws:
        `Exception`

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(int preferredLocalParallelism,
                                                   @Nonnull
                                                   ProcessorSupplier procSupplier)

      Factory method that wraps the given `ProcessorSupplier` and returns the same instance for each given `Address`.
      Parameters:
        `preferredLocalParallelism` - the value to return from `preferredLocalParallelism()`
        `procSupplier` - the processor supplier

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(@Nonnull
                                                   ProcessorSupplier procSupplier)

      Wraps the provided `ProcessorSupplier` into a meta-supplier that will always return it. The `preferredLocalParallelism()` of the meta-supplier will be `Vertex.LOCAL_PARALLELISM_USE_DEFAULT`.

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(@Nonnull
                                                   SupplierEx<? extends Processor> procSupplier,
                                                   int preferredLocalParallelism)

      Factory method that wraps the given `Supplier<Processor>` and uses it as the supplier of all `Processor` instances. Specifically, returns a meta-supplier that will always return the result of calling `ProcessorSupplier.of(SupplierEx)`.
      Parameters:
        `procSupplier` - the supplier of processors
        `preferredLocalParallelism` - the value to return from `preferredLocalParallelism()`

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(@Nonnull
                                                   SupplierEx<? extends Processor> procSupplier)

      Factory method that wraps the given `Supplier<Processor>` and uses it as the supplier of all `Processor` instances. Specifically, returns a meta-supplier that will always return the result of calling `ProcessorSupplier.of(SupplierEx)`. The `preferredLocalParallelism()` of the meta-supplier will be `Vertex.LOCAL_PARALLELISM_USE_DEFAULT`.

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(@Nonnull
                                                   FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier,
                                                   int preferredLocalParallelism)

      Factory method that creates a `ProcessorMetaSupplier` from the supplied function that maps a cluster member address to a `ProcessorSupplier`.
      Parameters:
        `addressToSupplier` - the mapping from address to ProcessorSupplier
        `preferredLocalParallelism` - the value to return from `preferredLocalParallelism()`

    - #### of

          @Nonnull
          static ProcessorMetaSupplier of(@Nonnull
                                                   FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)

      Factory method that creates a `ProcessorMetaSupplier` from the supplied function that maps a cluster member address to a `ProcessorSupplier`. The `preferredLocalParallelism()` of the meta-supplier will be `Vertex.LOCAL_PARALLELISM_USE_DEFAULT`.

    - #### preferLocalParallelismOne

          @Nonnull
          static ProcessorMetaSupplier preferLocalParallelismOne(@Nonnull
                                                                          ProcessorSupplier supplier)

      Wraps the provided `ProcessorSupplier` into a meta-supplier that will always return it. The `preferredLocalParallelism()` of the meta-supplier will be one, i.e., no local parallelization.

      The parallelism will be overriden if the `Vertex.localParallelism(int)` is set to a specific value.

    - #### preferLocalParallelismOne

          @Nonnull
          static ProcessorMetaSupplier preferLocalParallelismOne(@Nonnull
                                                                          SupplierEx<? extends Processor> procSupplier)

      Variant of `preferLocalParallelismOne(ProcessorSupplier)` where the supplied `SupplierEx<Processor>` will be wrapped into a `ProcessorSupplier`.

    - #### forceTotalParallelismOne

          @Nonnull
          static ProcessorMetaSupplier forceTotalParallelismOne(@Nonnull
                                                                         ProcessorSupplier supplier)

      Variant of `forceTotalParallelismOne(ProcessorSupplier, String)` where the node for the supplier will be chosen randomly.

    - #### forceTotalParallelismOne

          @Nonnull
          static ProcessorMetaSupplier forceTotalParallelismOne(@Nonnull
                                                                         ProcessorSupplier supplier,
                                                                         @Nonnull
                                                                         String partitionKey)

      Wraps the provided `ProcessorSupplier` into a meta-supplier that will only use the given `ProcessorSupplier` on a single node. The node will be chosen according to the `partitionKey` supplied. This is mainly provided as a convenience for implementing non-distributed sources where data can't be read in parallel by multiple consumers. When used as a sink or intermediate vertex, the DAG should ensure that only the processor instance on the designated node receives any data, otherwise an `IllegalStateException` will be thrown.

      The vertex containing the `ProcessorMetaSupplier` must have a local parallelism setting of 1, otherwise {code IllegalArgumentException} is thrown.

      Parameters:
        `supplier` - the supplier that will be wrapped
        `partitionKey` - the supplier will only be created on the node that owns the supplied partition key
      Returns:
        the wrapped `ProcessorMetaSupplier`
      Throws:
        `IllegalArgumentException` - if vertex has local parallelism setting of greater than 1

Copyright © 2020. All rights reserved.