## Uses of Interface
com.hazelcast.jet.pipeline.BatchSource

- Packages that use BatchSource
  Package | Description
  --- | ---
  com.hazelcast.jet.avro | Apache Avro file read/write support for Hazelcast Jet.
  com.hazelcast.jet.hadoop | Apache Hadoop HDFS read/write support for Hazelcast Jet.
  com.hazelcast.jet.pipeline | The Pipeline API is Jet's high-level API to build and execute distributed computation jobs.
  com.hazelcast.jet.pipeline.test | This package contains various mock sources to help with pipeline testing and development.
  com.hazelcast.jet.s3 | AWS S3 read/write support for Hazelcast Jet.

- - ### Uses of BatchSource in com.hazelcast.jet.avro

    Methods in com.hazelcast.jet.avro that return BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `BatchSource<D>` | AvroSourceBuilder.`build()` Convenience for `AvroSourceBuilder.build(BiFunctionEx)`.
    `<T> BatchSource<T>` | AvroSourceBuilder.`build(BiFunctionEx<String,? super D,T> mapOutputFn)` Builds a custom Avro file `BatchSource` with supplied components and the output function `mapOutputFn`.
    `static <D> BatchSource<D>` | AvroSources.`files(String directory, BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)` Convenience for `AvroSources.filesBuilder(String, SupplierEx)` which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
    `static <D> BatchSource<D>` | AvroSources.`files(String directory, Class<D> datumClass)` Convenience for `AvroSources.filesBuilder(String, Class)` which reads all the files in the supplied directory as specific records using supplied `datumClass`.

  - ### Uses of BatchSource in com.hazelcast.jet.hadoop

    Methods in com.hazelcast.jet.hadoop that return BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `static <K,V> BatchSource<Map.Entry<K,V>>` | HdfsSources.`hdfs(org.apache.hadoop.mapred.JobConf jobConf)` Convenience for `HdfsSources.hdfs(JobConf, BiFunctionEx)` with `Map.Entry` as its output type.
    `static <K,V,E> BatchSource<E>` | HdfsSources.`hdfs(org.apache.hadoop.mapred.JobConf jobConf, BiFunctionEx<K,V,E> projectionFn)` Returns a source that reads records from Apache Hadoop HDFS and emits the results of transforming each record (a key-value pair) with the supplied mapping function.

  - ### Uses of BatchSource in com.hazelcast.jet.pipeline

    Methods in com.hazelcast.jet.pipeline that return BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `static <T> BatchSource<T>` | Sources.`batchFromProcessor(String sourceName, ProcessorMetaSupplier metaSupplier)` Returns a bounded (batch) source constructed directly from the given Core API processor meta-supplier.
    `BatchSource<T>` | SourceBuilder.Batch.`build()` Builds and returns the batch source.
    `BatchSource<String>` | FileSourceBuilder.`build()` Convenience for `FileSourceBuilder.build(BiFunctionEx)`.
    `<T> BatchSource<T>` | FileSourceBuilder.`build(BiFunctionEx<String,String,? extends T> mapOutputFn)` Builds a custom file `BatchSource` with supplied components and the output function `mapOutputFn`.
    `static <K,V> BatchSource<Map.Entry<K,V>>` | Sources.`cache(String cacheName)` Returns a source that fetches entries from a Hazelcast `ICache` with the given name and emits them as `Map.Entry`.
    `static BatchSource<String>` | Sources.`files(String directory)` A source to read all files in a directory in a batch way.
    `static <T> BatchSource<T>` | Sources.`jdbc(String connectionURL, String query, FunctionEx<? super ResultSet,? extends T> createOutputFn)` Convenience for `Sources.jdbc(SupplierEx, ToResultSetFunction, FunctionEx)`.
    `static <T> BatchSource<T>` | Sources.`jdbc(SupplierEx<? extends Connection> newConnectionFn, ToResultSetFunction resultSetFn, FunctionEx<? super ResultSet,? extends T> createOutputFn)` Returns a source which connects to the specified database using the given `newConnectionFn`, queries the database and creates a result set using the the given `resultSetFn`.
    `static <T> BatchSource<T>` | Sources.`list(IList<? extends T> list)` Returns a source that emits items retrieved from a Hazelcast `IList`.
    `static <T> BatchSource<T>` | Sources.`list(String listName)` Returns a source that emits items retrieved from a Hazelcast `IList`.
    `static <K,V> BatchSource<Map.Entry<K,V>>` | Sources.`map(IMap<? extends K,? extends V> map)` Returns a source that fetches entries from the given Hazelcast `IMap` and emits them as `Map.Entry`.
    `static <T,K,V> BatchSource<T>` | Sources.`map(IMap<? extends K,? extends V> map, Predicate<? super K,? super V> predicate, FunctionEx<? super Map.Entry<K,V>,? extends T> projection)` Convenience for `Sources.map(IMap, Predicate, Projection)` which uses a `FunctionEx` as the projection function.
    `static <T,K,V> BatchSource<T>` | Sources.`map(IMap<? extends K,? extends V> map, Predicate<? super K,? super V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)` Returns a source that fetches entries from the given Hazelcast `IMap`.
    `static <K,V> BatchSource<Map.Entry<K,V>>` | Sources.`map(String mapName)` Returns a source that fetches entries from a local Hazelcast `IMap` with the specified name and emits them as `Map.Entry`.
    `static <T,K,V> BatchSource<T>` | Sources.`map(String mapName, Predicate<? super K,? super V> predicate, FunctionEx<? super Map.Entry<K,V>,? extends T> projection)` Convenience for `Sources.map(String, Predicate, Projection)` which uses a `FunctionEx` as the projection function.
    `static <T,K,V> BatchSource<T>` | Sources.`map(String mapName, Predicate<? super K,? super V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)` Returns a source that fetches entries from a local Hazelcast `IMap` with the specified name.
    `static <K,V> BatchSource<Map.Entry<K,V>>` | Sources.`remoteCache(String cacheName, ClientConfig clientConfig)` Returns a source that fetches entries from the Hazelcast `ICache` with the specified name in a remote cluster identified by the supplied `ClientConfig` and emits them as `Map.Entry`.
    `static <T> BatchSource<T>` | Sources.`remoteList(String listName, ClientConfig clientConfig)` Returns a source that emits items retrieved from a Hazelcast `IList` in a remote cluster identified by the supplied `ClientConfig`.
    `static <K,V> BatchSource<Map.Entry<K,V>>` | Sources.`remoteMap(String mapName, ClientConfig clientConfig)` Returns a source that fetches entries from the Hazelcast `IMap` with the specified name in a remote cluster identified by the supplied `ClientConfig` and emits them as `Map.Entry`.
    `static <T,K,V> BatchSource<T>` | Sources.`remoteMap(String mapName, ClientConfig clientConfig, Predicate<? super K,? super V> predicate, FunctionEx<? super Map.Entry<K,V>,? extends T> projection)` Convenience for `Sources.remoteMap(String, ClientConfig, Predicate, Projection)` which use a `FunctionEx` as the projection function.
    `static <T,K,V> BatchSource<T>` | Sources.`remoteMap(String mapName, ClientConfig clientConfig, Predicate<? super K,? super V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)` Returns a source that fetches entries from a remote Hazelcast `IMap` with the specified name in a remote cluster identified by the supplied `ClientConfig`.

    Methods in com.hazelcast.jet.pipeline with parameters of type BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `<T> BatchStage<T>` | Pipeline.`drawFrom(BatchSource<? extends T> source)` Returns a pipeline stage that represents a bounded (batch) data source.

  - ### Uses of BatchSource in com.hazelcast.jet.pipeline.test

    Methods in com.hazelcast.jet.pipeline.test that return BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `static <T> BatchSource<T>` | TestSources.`items(Iterable<? extends T> items)` Returns a batch source which iterates through the supplied iterable and then terminates.
    `static <T> BatchSource<T>` | TestSources.`items(T... items)` Returns a batch source which iterates through the supplied items and then terminates.

  - ### Uses of BatchSource in com.hazelcast.jet.s3

    Methods in com.hazelcast.jet.s3 that return BatchSource
    Modifier and Type | Method and Description
    --- | ---
    `static <T> BatchSource<T>` | S3Sources.`s3(List<String> bucketNames, String prefix, Charset charset, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, BiFunctionEx<String,String,? extends T> mapFn)` Creates an AWS S3 `BatchSource` which lists all the objects in the bucket-list using given `prefix`, reads them line by line, transforms each line to the desired output object using given `mapFn` and emits them to downstream.
    `static BatchSource<String>` | S3Sources.`s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier)` Convenience for `S3Sources.s3(List, String, Charset, SupplierEx, BiFunctionEx)`.

Copyright © 2020. All rights reserved.