Package com.hazelcast.jet.python

Interface PythonExtension.PythonStage<S extends com.hazelcast.jet.pipeline.GeneralStage<String>>

Type Parameters:
S - Type of the stage which is extended with PythonExtension
Enclosing interface:
PythonExtension
public static interface PythonExtension.PythonStage<S extends com.hazelcast.jet.pipeline.GeneralStage<String>>
Pipeline stage with ability to call Python user-defined functions.
Since:
5.7

  • Method Details

    • map

      @Nonnull S map(@Nonnull PythonServiceConfig cfg)
      Adds a "map using Python" pipeline stage. See PythonServiceConfig for more details.

    • map

      @Nonnull S map()
      Adds a "map using Python" pipeline stage which uses PythonServiceConfig configured using fluent builder API in this stage.
      See Also:

    • maxBatchSize

      @Nonnull PythonExtension.PythonStage<S> maxBatchSize(int maxBatchSize)
      Sets the maximum size of a batch for a single request to Python handler.
      Parameters:
      maxBatchSize - the maximum size of a batch for a single request

    • baseDir

      @Nonnull PythonExtension.PythonStage<S> baseDir(@Nonnull String baseDir)
      Sets the base directory where the Python files reside. When you set this, also set the name of the handler module to identify the location of the handler function (named transform_list() by convention).

      If all you need to deploy to Jet is in a single file, you can call handlerFile(java.lang.String) instead.

    • handlerFile

      @Nonnull PythonExtension.PythonStage<S> handlerFile(@Nonnull String handlerFile)
      Sets the Python handler file. It must contain the handler function. If your Python work is in more than one file, call baseDir(java.lang.String) instead.

    • handlerModule

      @Nonnull PythonExtension.PythonStage<S> handlerModule(@Nonnull String handlerModule)
      Sets the name of the Python module that has the function that transforms Jet pipeline data.

    • handlerFunction

      @Nonnull PythonExtension.PythonStage<S> handlerFunction(@Nonnull String handlerFunction)
      Overrides the default name of the Python function that transforms Jet pipeline data. The default name is "transform_list". It must be defined in the module you configured with handlerModule(java.lang.String), must take a single argument that is a list of strings, and return another list of strings which has the results of transforming each item in the input list. There must be a strict one-to-one match between the input and output lists.

    • channelFn

      @Nonnull PythonExtension.PythonStage<S> channelFn(@Nonnull com.hazelcast.function.BiFunctionEx<String,Integer,? extends io.grpc.ManagedChannelBuilder<?>> channelFn)
      Sets the channel function. The function receives a host+port tuple, and it's supposed to return a configured instance of ManagedChannelBuilder. You can use this to configure the channel, for example to configure the maximum message size etc.

      The default value is NettyChannelBuilder::forAddress.