GridGain Documentation

The GridGain Developer Hub

Welcome to the GridGain developer hub. You'll find comprehensive guides and documentation to help you start working with GridGain as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    

Network Segmentation

Network segmentation (a.k.a. "split-brain" problem) is a tough problem to deal with in distributed systems and must be accounted for (especially in a production system with ACID guarantees). The cluster may become segmented because of temporary network problems when nodes (or groups of nodes) become isolated from the rest of the topology - and every such isolated group would "act" as if it is the only one left and therefore must proceed unaware of other groups. If not addressed, such segmentation can cause inconsistent clusters and, what’s worse, inconsistent data. GridGain addresses this issue by making sure that no inconsistent write is allowed into the system if segmentation occurs, while writes that are certain to be consistent are still allowed to proceed.

In GridGain, each node checks network segment individually, using configured segmentation resolvers. Segmentation resolvers are pluggable and can be tailored to any environment. GridGain comes with several implementations out of the box. Segment check is performed in following cases:

  • Before the start of discovery SPI.
  • When any node leaves topology.
  • When any node in topology fails.
  • Periodically (see IgniteConfiguration.setSegmentCheckFrequency(long)).

Each segmentation resolver checks the segment for validity. Typically, a resolver should run a light-weight single check (i.e. one IP address or one shared folder). Compound segment checks may be performed using several resolvers. If a segmentation resolver determines that the local grid node belongs to an incorrect segment, the node will act in accordance with the configured segmentation policy.

Refer to SegmentationPolicy in JavaDoc for more information on available policies.

Configuration

Here is the list of IgniteConfiguration properties to configure a segmentation logic. All configuration parameters below are optional:

Note that segmentation check is disabled by default. It is because segmentation resolvers are always custom (for a specific environment, network configuration, etc.) and therefore cannot be made generic. It is highly recommended, however, to enable network segmentation checks in production systems performing data processing.

Setter Method
Description
Default Value

setSegmentationPolicy(GridSegmentationPolicy)

Segmentation policy.

STOP

setSegmentationResolvers(GridSegmentationResolver[])

Segmentation resolvers.

The following segmentation resolver implementations are built-in:

  • ReachabilitySegmentationResolver
  • SharedFsSegmentationResolver
  • TcpSegmentationResolver

null

setSegmentCheckFrequency(long)

Network segment check frequency.

10000ms

setSegmentationResolveAttempts(int)

Sets segmentation resolve attempts count.

2

setWaitForSegmentOnStart(boolean)

Wait for segment on start flag.

true

setAllSegmentationResolversPassRequired(boolean)

All segmentation resolvers pass required flag. If flag is set to true a single failure of any resolver will trigger network segmentation event.

true

Segmentation Resolver

Network segmentation resolver is defined by the GridSegmentationResolver interface.

User implementation should implement only one method isValidSegment(). Each segmentation resolver checks the segment for validity, using its inner logic. As we mentioned, ideally, a resolver should run a light-weight single check (i.e. one IP address or one shared folder). Compound segment checks may be performed using several resolvers.

Note that GridGain supports logical segmentation and is not limited to network related segmentation only. For example, a particular segmentation resolver can check for a specific application or service present on the network and mark the topology as segmented in case it is not available. In other words, you can equate the service outage with network outage via segmentation resolution and employ the unified approach in dealing with these types of problems.

Built-In Segmentation Resolvers

GridGain has the following built-in segmentation resolvers. You can specify multiple resolvers, in which case, all the specified resolvers will be checked. Use setAllSegmentationResolversPassRequired(boolean) to make sure that all resolvers must pass the segmentation check. Otherwise, the segment is declared valid if it passes any one of the below-mentioned segmentation resolver checks.

Resolver
Description

TcpSegmentationResolver

Segmentation resolver implementation that checks whether node is in the correct segment or not by establishing TCP connection to configured host and port and immediately closing it. This is a multi-purpose resolver as it can be used to check connectivity to any web server (e.g. try to connect to port 80), data grid (e.g. establish TCP connection to JDBC port), etc…

SharedFsSegmentationResolver

Segmentation resolver implementation that checks whether node is in the correct segment or not by writing to and reading from shared directory.

ReachabilitySegmentationResolver

Segmentation resolver implementation that uses java.net.InetAddress.isReachable(NetworkInterface, int, int) to check whether node is in the correct segment or not.

Segmentation Events

GridGain has the following built-in event types to notify on segmentation events:

  • EVT_NODE_SEGMENTED
  • EVT_NODE_RECONNECTED

As with any other events in the system, a user can subscribe to these events locally and query the events from remote nodes in a distributed fashion.

Segmentation Policies

GridGain has the following built-in segmentation policies.

Policy
Description

RESTART_JVM

When segmentation policy is RESTART_JVM, all event listeners will receive EVT_NODE_SEGMENTED event and then the JVM will be restarted.
Note, that this will work only if GridGain is started with CommandLineStartup via standard ignite.[sh|bat] shell script.

STOP

When segmentation policy is STOP, all event listeners will receive EVT_NODE_SEGMENTED event and then the particular node will be stopped via call a to Ignition.stop(GRID_NAME, true) method.

NOOP

When segmentation policy is NOOP, all event listeners will receive EVT_NODE_SEGMENTED event and it is up to the user to implement the logic to handle this event.

Network Segmentation


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.