Package com.hazelcast.jet.avro

Class AvroSources

java.lang.Object
com.hazelcast.jet.avro.AvroSources
public final class AvroSources extends Object
Contains factory methods for Apache Avro sources.
Since:
Jet 3.0

  • Method Summary

    Modifier and Type
    Method
    Description
    static <D> com.hazelcast.jet.pipeline.BatchSource<D>
    files(String directory, com.hazelcast.function.BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)
    Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
    static <D> com.hazelcast.jet.pipeline.BatchSource<D>
    files(String directory, Class<D> datumClass)
    Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass.
    static <D> AvroSourceBuilder<D>
    filesBuilder(String directory, com.hazelcast.function.SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
    Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.
    static <D> AvroSourceBuilder<D>
    filesBuilder(String directory, Class<D> recordClass)
    Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • filesBuilder

      @Nonnull public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull String directory, @Nonnull Class<D> recordClass)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
      Type Parameters:
      D - the type of the datum
      Parameters:
      directory - parent directory of the files
      recordClass - the class to read

    • filesBuilder

      @Nonnull public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull String directory, @Nonnull com.hazelcast.function.SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
      Type Parameters:
      D - the type of the datum
      Parameters:
      directory - parent directory of the files
      datumReaderSupplier - the supplier of datum reader which reads records from the files

    • files

      @Nonnull public static <D> com.hazelcast.jet.pipeline.BatchSource<D> files(@Nonnull String directory, @Nonnull Class<D> datumClass)
      Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass. If datumClass implements SpecificRecord, SpecificDatumReader is used to read the records, ReflectDatumReader is used otherwise.

    • files

      @Nonnull public static <D> com.hazelcast.jet.pipeline.BatchSource<D> files(@Nonnull String directory, @Nonnull com.hazelcast.function.BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)
      Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.