This document lists the new features, enhancements, fixed issues and, removed or deprecated features for Hazelcast IMDG 4.2.z releases. The numbers in the square brackets refer to the issues in Hazelcast's GitHub repositories.

==== 4.2.6 ====

## Enhancements

* Upgrade jackson-databind to 2.14.0. #22391

## Fixes

* Fixed an issue where replication over WAN was failing on the source cluster members, when there are multiple batch publishers configured in a single WAN replication. #22496
* Fixed a memory leak due to incomplete clean-up of backup replica sync operations. #22406
* Fixed the cluster failure occurred after requesting healthcheck of a member using REST API while the `hazelcast.socket.buffer.direct` property is enabled. #21702
* Improved connection handling. #21643
* Fixed an issue where a cluster could not be formed when security is enabled, various client permissions are set,
and multiple members are started simultaneously. #21508

==== 4.2.5 ====

If you’re using eviction or expiration for maps, you must set the per-entry-stats-enabled property to true to fix an issue in version 4.2.x where some map entries are not always evicted or expired according to their time-to-live or maximum idle duration configurations.

## Enhancements

* Introduced a system property for allowing you to audit that all the Hazelcast instances running in your environment have the instance tracking file name set correctly in the configuration. See the note in Instance Tracking. #19929
* Enabled XXE (XML External Entity Reference) protection for XMLInputFactory. The issue was reported through https://huntr.dev/bounties/d63972a2-b910-480a-a86b-d1f75d24d563/. #20942
* The probe level for most of the network related statistics has been changed to "DEBUG" to decrease the pressure on Management Center; now they are not sent to Management Center by default. If you want to see these statistics, you need to set the "hazelcast.metrics.debug.enabled" property to true. #21275

## Fixes

* Fixed an issue where the entry memory statistics of a map with an external data store (MapStore) were being calculated incorrectly for the following scenario:
  ** The map is backed by High-Density Memory Store, i.e., in-memory format is NATIVE.
  ** Map entries are evicted using IMap.evict() and then loaded using loadAll().
  ** Map Statistics screen in Management Center shows increased memory usage which gives a false impression as if there is a memory leak.
  #4649
* Fixed an issue where the statistics like puts and removals were not increasing when these operations are executed through Transactional interface. #21105
* Fixed an issue where Hazelcast clients, which have only the IP address of a member to connect (but the member also has a hostname), were not able to connect to the cluster. #20631
* Hazelcast’s memcached implementation was interpreting the number values and parameters for incr and decr wrongly (numbers were being converted into byte arrays instead of decimals). This has been fixed by making these commands' implementations strictly follow the memcached protocol specification. #19676
* Fixed an issue where the totalPublishes statistics for the Reliable Topic data structure were always generated as 0. #19656
* Fixed an issue where the map.clear() and cache.clear() methods were evicting all entries in all near caches of all the maps in a cluster, not only the map on which these methods are called. #19501
* Fixed an issue that caused some map entries to remain in Hazelcast even when you configured eviction or expiration. #19483

## Removed/Deprecated Features

The following system properties have been deprecated:

* hazelcast.client.statistics.enabled
* hazelcast.client.statistics.period.seconds

==== 4.2.4 =====

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.17.0. No changes were made to the IMDG code.

Note: The 4.2.4 release notes for the Hazelcast distribution on Maven wrongly states that log4j2 version has been updated to 2.15.0; it should be 2.17.0.

==== 4.2.3 ====

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.15.0. No changes were made to the IMDG code.

==== 4.2.2 ====

## Fixes

* Fixed an issue where the client state listener was not properly working with failover clients (in blue-green deployments); it was failing with invalid configuration exception. #19115
* Fixed an issue where the LDAP login modules could not properly handle multi-value attributes: when multiple values, e.g., roles, are referenced by an LDAP attribute, only the first value was taken into account by the login modules. #4151

==== 4.2.1 ====

## Enhancements

* Enhanced the getPartitionGroupStrategy() method to have cluster members as arguments so that useful partitioning strategies can be implemented by accessing the members using this method. #18904
* The log message for infinite cluster connection timeout is clearer now. Previously, it was represented as the value of Long.MAX_VALUE. #18648
* Introduced a new mechanism in the background expiration tasks; now a thread local array controls the allocations for these tasks otherwise which may cause increased garbage collection pressure and CPU usage spikes when you use aggressive expiration configurations, e.g., low time-to-live values. #18507

## Fixes

* Fixed an issue where the health monitor was incorrectly showing the value for free metadata memory. #18953
* Fixed an issue where the updates made to a persistent map store might be lost when the write coalescing is enabled. #18929
* Some merge policies like LatestUpdateMergePolicy for the map and WAN replication configurations require the per-entry statistics to be enabled. Previously, this configuration inconsistency was causing the related member to fail at runtime. Now, the Hazelcast member fails to start, i.e., fast fails, in such a case. #18875
* Fixed an issue where the maximum size policy for a map was being ignored when the policy is PER_NODE and the cluster is scaled down (due to losing or killing a member). #18873
* Fixed an issue where the map’s Near Cache was setting its maximum size as 10.000 even if the configured eviction policy is NONE. #18812
* The LRU eviction policy now takes last access time value into account to prevent premature removal of the lately added but not yet accessed map entries. #18624

## Contributors

We would like to thank the contributors from our open source community who worked on this release:

* Lenny Primak

==== 4.2 ====

## New Features

Hazelcast IMDG Open Source New Features:

* Sorting Support for Hazelcast SQL: Added the support of SQL sorting feature. By using the newly implemented ORDER BY, LIMIT, and OFFSET constructions, you can see the query results being ordered in ascending or descending fashion, limit the count of results, or skip some results. See the documentation for examples.
* Node Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for Kubernetes. You can create partition groups according to the name of the node which is provided by this plugin and on which the containers/pods run. See the NODE_AWARE section of the IMDG Reference Manual for more information.
* Placement Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for AWS. You can group members according to their placement metadata provided by the cloud providers, such as rack, fault domain, power sources, network, and resources of a virtual machine in a zone. See the PLACEMENT_AWARE section of the IMDG Reference Manual for more information.
* Dynamic Log Level Support: Added the support of changing Hazelcast log levels without restarting cluster members. See the Dynamically Changing Log Levels section.

## Breaking Changes

BETA API Changes:

* Being in its BETA stage, we improved the GenericRecord API, and performed various related changes in the serialization API due to these improvements:
  ** GenericRecord#read() and GenericRecord.Builder#write() methods have been renamed as get() and set(), respectively.
  ** GenericRecord.Builder has been moved to its own class as GenericRecordBuilder.
  ** UTF has been renamed as String for the following methods, which are now deprecated:
    ClassDefinitionBuilder.addUTFField()
    ClassDefinitionBuilder.addUTFArrayField()
    PortableWriter.writeUTF()
    PortableWriter.writeUTFArray()
    PortableReader.readUTF()
    PortableReader.readUTFArray()
    ObjectDataOutput.writeUTF()
    ObjectDataOutput.writeUTFArray()
    ObjectDataInput.readUTF()
    ObjectDataInput.readUTFArray()
  ** UTF has been renamed as String for the following without deprecation:
    GenericRecord.readUTF()
    GenericRecord.readUTFArray()
    GenericRecordBuilder.writeUTF()
    GenericRecordBuilder.writeUTFArray()
    #18100

Default Settings Change:
* Seeing that getting entry level statistics in a map has not been a common use case and it has been creating unnecessary memory overhead that can be otherwise used for storing user data, per-entry statistics of maps now are disabled by default; they were enabled in the previous releases. See the Enhancements section below.
* The default value for cluster connection timeout for clients has been set as -1 (infinite timeout): This is the timeout value for the client to give up to connect to the current cluster. For this new default value, the client will not stop trying to connect to the target cluster. This change has been made to prevent the clients from shutting down when a cluster is unavailable; otherwise (previous default value was 2 minutes), it often meant restarting the whole application. The reason for keeping the previous default value has been not to break the backward compatibility; with this release we introduced this breaking change which would be better for users. See here for more information. #18094

## Enhancements

Hazelcast IMDG Open Source Enhancements:

* Per-Entry Statistics of Maps: Introduced a configuration option (per-entry-stats-enabled) for retrieving per-entry statistics of a map, which is disabled by default. See the Accessing Map and Entry statistics section.
* Default Serializer Overriding: Added the ability to override Hazelcast’s built-in serializers. Also added the serialization support of java.util.Optional. See the Serialization chapter.

The following are the improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

* Added support of named parameters in SQL’s HazelcastFunction. #18328
* Added the HTTPS/TLS support for phone home pings. Also enhanced the phone home data by adding the following information:
  ** Java classpath on which the member runs
  ** Detailed client information such as total connection durations and client versions
  ** Total number of created proxies for each distributed object service
  #18337, #18326, #18307, #18287
* Improved the LIKE query predicate so that it now supports indexes. #18289
* Added management-permission to the client permissions that defines which Management Center client principals/endpoints are allowed to perform management tasks. #18264
* Added the temporal types support for indexes by introducing converters for the missing types (LocalTime, LocalDate, LocalDateTime and OffsetDateTime) #18244
* Added the support of bitmap index statistics to be propagated to local map statistics. #18220
* Improved the map store configuration such that it is enabled unless you explicitly disable it; this improvement has been introduced to eliminate the inconsistencies between map store’s declarative and programmatic configurations. #18217
* Implemented the High-Density Metadata Store to store all the metadata for on-heap and off-heap storages. This new store is based on BinaryElasticHashMap. A new type of record called HDMetadataRecord has been introduced for this purpose that references the key and value part of the metadata. #18186
* Added the ability to register the class definition of the parent generic record and check the class definition compatibility for nested portable fields. #18180
* Improved the SQL engine so that it attempts to re-use the already deserialized values stored in an index; this has been improved the map scan operations performance when there is at least one index on an IMap. #18172
* Added the iterator() and iterable() methods with different parameter combinations to the map. #18126
* Added the support of non-nullable columns for the SQL service. #18114
* Implemented the fetch and offset SQL clauses without pushing the operator on the individual member. #18091
* Introduced a new serialization mechanism for SQL row data. Previously, every value was being serialized as Data which was inefficient and the non-Java clients could not read some types, e.g., decimal. With this new mechanism, a new custom codec SqlPageCodec has been introduced that serializes values using a custom built-in list serializer depending on the type of the column. #18089
* Added Nonnull annotations to StreamSerializer. #18071
* Added the ability to cancel queries on the client side when there are active requests sent to the member; this enhancement facilitates the future implementation of a non-blocking SQL client handler. #18047
* Added the support of OBJECT type in the comparison operators in Hazelcast’s SQL service. #18016
* Implemented the ORDER BY clause for the SQL service; it supports the ASC and DESC sortings with limitations. #18013
* Added the support of remainder operation for Hazelcast’s SQL service. #17997
* Added the support of NOT LIKE expression to Hazelcast’s SQL service. #17996
* Introduced a new SQL threading model to improve its performance. #17985
* Introduced the support of schemas for the SQL public API. #17953
* Implemented the IMap.entrySet() method for the partition ID set. #17937
* Improved the discovery by external smart clients in the cloud environments: the clients now only need to know the address of any member (or that of a load balancer if members are exposed via load balancer). #17895
* Introduced a configuration property to ignore errors during enabling the XXE protection. This protection works with JAXP 1.5 (Java 7 Update 40) and newer. When an older JAXP implementation is added to the classpath, e.g., Xerces and Xalan, an exception is thrown. The newly introduced property, namely hazelcast.ignoreXxeProtectionFailures, allows you to ignore those exceptions. #17839
* Added the missing replicatedmap-permission support to the XML and YAML configuration handlers. #17810
* Replaced Charset with StandardCharsets to be used in JSON querying. #17741
* Implemented migration listener for the Java client. #17713
* Introduced tenant control when creating JCache caches. #17673
* Added the BigDecimal,LocalTime,LocalDate,LocalDateTime,OffsetDateTime types to the Portable Serialization. #17257
* Introduced the unparkAll() method for event journal read operations. #14081
* Eliminated the possibility of losing WAN events when rebalancing the cluster by introducing a new, specialized WAN queue. #3941

## Fixes

* Fixed a regression issue where the locked and expired entry keys could not be reached over indexes. #18386
* Improved the deserialization for indexes: when having multiple indexes on a map, each time an entry is put in the map and thus to the index, the entry is deserialized for each index instead just once. This was causing performance issues. #18343
* Fixed an issue where SQL’s ORDER BY statement was failing if the field is indexed and there are other clauses such as WHERE in the query. #18341
* Fixed an issue where GenericRecord could not be queried when the in-memory format of the map is OBJECT. #18336
* Fixed a discrepancy between the behaviors of index-scan and full-scan in maps; the eviction of the idle map entries was not stable in IMDG 4.x series due to this discrepancy. #18334
* Fixed an issue where the health monitor was logging no values for garbage collection metrics. #18317
* Fixed an issue that occurred when there is Near Cache configured for both the member and client sides for the same map and serialize-keys option is false. #18312
* Fixed an issue where the transactions was not throwing TransactionTimeOutException in case of a timeout. #18305
* Fixed an issue where Hazelcast IMDG was not picking the hazelcast.xml configuration file from the download package but from the current working directory where IMDG has been started. #18304
* Fixed the race condition occurring during the serialization of writes for the copy-on-write data structures. #18285
* Fixed an issue where the clients, in a blue/green deployment, were hanging while reconnecting to the alternative cluster due to mishandling of member list. #18276
* Fixed an issue where CachedQueryEntry could not be serialized since it didn’t have a default serializer. #18238
* Fixed an issue where the Java client was not receiving membership events in its membership listener when a member with Hot Restart Persistence enabled is restarted. #18234
* Fixed a regression issue: when overlapping wildcard configurations are defined declaratively, the most specific one was inheriting attributes from the more generic one during parsing. #18187
* Fixed an issue where the non-UTF characters in a JSON value was causing query failures. #18183
* Fixed an issue where tasks were not running in parallel when they are submitted to all the cluster members in some scenarios. #18107
* Fixed an issue that prohibited Hazelcast from being used as Tomcat session manager when it is also deployed in a web application context: When a client application that uses Hazelcast tries to connect to the Hazelcast cluster and this application’s web sessions are persisted using Hazelcast’s Tomcat session manager, the session manager could not connect to the cluster. This has been fixed by improving Hazelcast’s service loader mechanism. #18103
* When the in-memory format of a map is NATIVE and the uploaded user code has missing classes (in case the user code deployment feature is used), the resulting exception could not be seen on the client side when a map query is run. This was causing the client to hang indefinitely and fixed by improving the failure handling for this case. #18081
* Fixed an issue where the queue items were being delivered more than once when they are reproduced after a member leaves the cluster. #18057
* Fixed the syntax for inMemoryFormat variable in the MapConfig.toString() method. #17976
* Fixed a failure which happened when a client is recreated with the same client failover configuration after creating a map that has the default near cache eviction configuration. #17952
* Fixed several issues when handling the SQL expressions. The fixes include not relying on Apache Calcite for inference and coercion anymore and introducing custom operand checker implementations provided by every operator. #17947
* Fixed an issue where the metrics for map hits statistics in Management Center were decreasing as the map entries are being expired. #17930
* Fixed an issue where the clients were opening two connections to the same member when the member is behind a private network. #17844
* Fixed an issue where the failures in SessionAwareSemaphore was preventing the acquired permits from their releases. #17826
* Fixed an issue where the Javadoc of release methods for session-aware semaphore structure was incorrectly addressing "threads" instead of "Hazelcast instances". #17823
* Fixed an issue where the remove() and delete() operations of maps were not updating the local map statistics. #17771
* Fixed the metrics unit for cache statistics to be declared in microseconds. #17742
* Fixed a failure when retrieving the member state before the member becomes ACTIVE on Kubernetes using Helm charts. #17773
* Fixed an issue where the parsing of configurations for some Hazelcast features, such as Hot Restart and user code deployment, was overriding the existing configuration values. #18036, #18034, #17675, #17681, #17885, #17917, #17923, #17924, #17940, #17945, #17946, #17875, #17878, #17969, #17971, #17972
* Removed InetSocketAddressCache from the Java client code so that the client can continue to work, while preserving the behaviors in a Blue/Green Deployment scenario; the Java client was not able to resolve the new address of a restarted member, e.g., for a setup in Docker environment. #17239
* Fixed an issue where the statistics entry for WAN replication state of a member was retrieving the hardcoded STOPPED value instead of the configured initial publisher state. #3834

## Removed/Deprecated Features

* The Symmetric Encryption feature has been deprecated. You can use the TLS/SSL protocol to establish an encrypted communication across your Hazelcast cluster.
* The following system properties have been deprecated:
  ** hazelcast.partition.group.rack
  ** hazelcast.partition.group.host
  ** hazelcast.hotrestart.free.native.memory.percentage

## Contributors

We would like to thank the contributors from our open source community who worked on this release:

* Lenny Primak
* Farinu Taiwo
* Abdullah Cevik
* Piotr Jasina
* Franz Wimmer
* Lukáš Kvídera
* Aleksey Kaurov
* Aaron Riekenberg
* Tomasz Gaweda