Hazelcast Jet Enterprise Readme
===============================

About Hazelcast Jet
-------------------

Hazelcast Jet Enterprise is a cloud-native, distributed stream
processing engine. It handles both unbounded (stream) and bounded 
(batch) data sources.


What's Included
---------------

* `bin/jet-start`: starts a new Jet instance
* `bin/jet-stop`: stops all Jet instances running on this machine
* `bin/jet`: submits and manages jobs
* `bin/jet-cluster-admin`: manages the Jet cluster (for example, shutting
  down the cluster)
* `bin/jet-cluster-cp-admin`: manages the CP Subsystem in the Jet cluster
* `config/hazelcast-jet.yaml`: The jet configuration file
* `config/hazelcast.yaml`: The IMDG configuration used by the Jet instance
* `config/hazelcast-client.yaml`: The client configuration file used by the 
  Jet Command Line client
* `config/log4j.properties`: Logging configuration used by the Jet Instance
* `opt`: Optional extensions for Jet. You can include them in the classpath by moving
them to the lib folder.
    
Quickstart
----------

### 1. Configure the License Key

Set the environment variable `JET_LICENSE_KEY`, for example:

```
$ export JET_LICENSE_KEY=JET_ENTERPRISE#<the_rest_of_your_key>
```

There's also a place to set it in the configuration file, but since
the configuration file is more likely to be sent around or published on
GitHub, it is generally safer to set the environment variable instead.


### 2. Start a Jet Instance

On a terminal prompt, with the unpacked distribution's root as your current
directory, type:

```
$ bin/jet-start
```

### 3. (Optional) Start a Second Instance to Form a Cluster

Repeat the step above. Eventually lines like the following should appear
in the log output, confirming that your new instance has successfully
discovered the first one and formed a cluster with it:

```
Members {size:2, ver:2} [
	Member [192.168.0.2]:5701 - 399aa674-cc73-4bae-8451-67943efc4a66 this
	Member [192.168.0.2]:5702 - 79b6399b-433f-4d61-b0d9-38ec121af07b
]
```

Note: By default, Jet instances use IP multicast to discover each other. 
This network feature is often disabled in managed cloud environments. In 
this case you can enable the TCP-IP join inside `config/hazelcast.yaml`
or use a mechanism specific to the cloud provider you're using. See the
configuration file for a template with all the available self-discovery
mechanisms.
 
You can also review the section on setting up clusters in 
[Hazelcast IMDG's reference manual](https://docs.hazelcast.org/docs/4.0/manual/html-single/index.html#setting-up-clusters)

### 4. Submit a Job

In order to validate your cluster setup, submit the provided Hello World
example:

```
$ bin/jet submit examples/hello-world.jar
```

This job creates a stream of random numbers and then calculates the
top 10 largest values observed so far, writing the results to an IMap.
It periodically prints the contents of the IMap.
 
You can play around with the various fault tolerance features of Jet by
adding and removing Jet instances to the cluster and observe how the job
is affected. You can also monitor the job from the command line :

```
$ bin/jet list-jobs
```

Additional Libraries
--------------------

To use an additional library provided in the opt folder, move it to the
`lib` folder. Jet will automatically use it the next time you start it.

Additional Information
----------------------

* [Hazelcast Jet Enterprise on GitHub](https://github.com/hazelcast-jet-enterprise)
* [Hazelcast Jet on GitHub](https://github.com/hazelcast-jet), the repo includes [code examples](https://github.com/hazelcast/hazelcast-jet/tree/master/examples)
* [Hazelcast Jet Homepage](https://jet.hazelcast.org)
* [Hazelcast Jet Reference Manual](https://jet-start.sh/docs)
