extreme scale – Geek 2.0 https://www.stormacq.com Some not-so random thoughts about this small IT World Mon, 15 Oct 2018 14:03:30 +0000 en-US hourly 1 https://wordpress.org/?v=4.9.8 WebSphere Liberty Profile Cluster Sharing an In-Memory Data Grid https://www.stormacq.com/websphere-liberty-profile-cluster-sharing-an-in-memory-data-grid/ https://www.stormacq.com/websphere-liberty-profile-cluster-sharing-an-in-memory-data-grid/#respond Sun, 14 Oct 2012 13:48:35 +0000 http://www.stormacq.com/?p=521 WebSphere Liberty Profile is a fast, lightweight and simple Java web application container allowing developer to develop, test and deploy applications easily.  In my previous articles, I explained how to install Liberty Profile on Mac and how to develop and deploy your first REST based services.

Liberty Profile is a standalone Java container.  It is not designed to be included in larger deployments based on WebSphere Application Server  ND cells.

However, Liberty Profile can take benefit of a shared persistence engine to store HTTP Session data. This allows two or more independent Liberty Profile instances to share a common user session for web applications.  When one instance fails, the surviving instances can continue to serve user requests as-is nothing happened.

Persistent data store might be a relational database (such as Derby used for development purposes) or a in-memory data grid. In-Memory Data Grid are software solutions providing in-memory data storage, replicated across different containers (or machines). Many IMDG solutions are available from different vendors or in open-source.  Most common ones are MemCached, Terracotta (Software AG), Coherence (Oracle) and IBM’s WebSphere eXtreme Scale.

If you are totally new to eXtreme Scale, I would recommend to read some basic information about its architecture before continuing to read this article.

Configuring WebSphere Application Server (WAS – full profile) to store HTTP Session in a eXtreme Scale container is a matter of three clicks in WAS admin console.  It is slightly more complicate with Liberty Profile, just a few configuration steps described below.

There are four different ways to install eXtreme Scale (XS) with Liberty :

  • Run XS Container in a separate JVM or separate machine than Liberty Profile
  • Run XS Container inside the same JVM as Liberty Profile
  • Use Liberty Profile as client for an XS container
  • Configure Liberty Profile to store HTTP Session data to an XS container

In this article, I will show you how to configure Liberty Profile to

  1. Start an XS server within the same JVM as Liberty profile
  2. Store HTTP Session data in this in-memory data grid,allowing to create clusters of Liberty Profile Instances

My final architecture is depicted in the image below.

0. Download and Install Liberty Profile and eXtreme Scale for Liberty Profile (both solutions are available at no charge from IBM – with forum based and peer-to-peer support only).

  • Liberty Profile installation is described in my previous blog entry.
  • eXtreme Scale for Liberty Profile installation is just a matter of unzipping the file in the directory above wlp

1. Create two servers instances

cd wlpBLOG
sst:wlpBLOG sst$ ./bin/server create ServerONE
Server ServerONE created.
sst:wlpBLOG sst$ ./bin/server create ServerTWO
Server ServerTWO created.

2. Change default HTTP Port in both server.xml so that the two instances can run in parallel

<httpEndpoint host="localhost" httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>

3. Add two features in server.xml for each server.  One to tell Liberty to run an XS server embedded.  And one to tell Liberty to use XS as HTTP Session store for web applications.

<!-- Enable features -->
<featureManager>
   <feature>jsp-2.2</feature>
   <feature>localConnector-1.0</feature>
   <feature>eXtremeScale.server-1.0</feature>
   <feature>eXtremeScale.web-1.0</feature>
</featureManager>

4. Configure the the WXS container inside Liberty Profile : add WXS configuration in Liberty Profile

<!-- Configuration for XS Server -->
<xsServer isCatalog="true" serverName="XS_ServerONE"/>

<!-- Configuration for Web Application XS HTTP Session data storage -->
<xsWebApp catalogHostPort="localhost:2809"
    objectGridType="REMOTE" 
    replicationInterval="0"
    reuseSessionId="true"
    securityEnabled="true"
    sessionTableSize="0"/>

5. Configure the the WXS container inside Liberty Profile : add XML configuration files in WLP runtime directory

In the directory WLP_HOME/usr/servers/ServerONE, create a “grids” directory and drop those two files

deployment.xml
<?xml version="1.0" encoding="UTF-8"?>
<deploymentPolicy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ibm.com/ws/objectgrid/deploymentPolicy ../deploymentPolicy.xsd"
xmlns="http://ibm.com/ws/objectgrid/deploymentPolicy">

<objectgridDeployment objectgridName="session">
<mapSet name="sessionMapSet" numberOfPartitions="47" minSyncReplicas="0" maxSyncReplicas="0" maxAsyncReplicas="1" developmentMode="false" placementStrategy="FIXED_PARTITIONS">
<map ref="objectgridSessionMetadata"/>
<map ref="objectgridSessionAttribute.*"/>
<map ref="objectgridSessionTTL.*"/>
</mapSet>
</objectgridDeployment>
</deploymentPolicy>

 

objectgrid.xml
<?xml version="1.0" encoding="UTF-8"?>
<objectGridConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ibm.com/ws/objectgrid/config ../objectGrid.xsd"
xmlns="http://ibm.com/ws/objectgrid/config">
<objectGrids>
<objectGrid name="session" txTimeout="30">
<bean id="ObjectGridEventListener" className="com.ibm.ws.xs.sessionmanager.SessionHandleManager"/>
<backingMap name="objectgridSessionMetadata" pluginCollectionRef="objectgridSessionMetadata" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="LAST_ACCESS_TIME" timeToLive="3600" copyMode="COPY_TO_BYTES"/>
<backingMap name="objectgridSessionAttribute.*" template="true" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="NONE" copyMode="COPY_TO_BYTES"/>
<backingMap name="objectgridSessionTTL.*" template="true" readOnly="false" lockStrategy="PESSIMISTIC" ttlEvictorType="LAST_ACCESS_TIME" timeToLive="3600" copyMode="COPY_TO_BYTES"/>
</objectGrid>
</objectGrids>
<backingMapPluginCollections>
<backingMapPluginCollection id="objectgridSessionMetadata">
<bean id="MapEventListener" className="com.ibm.ws.xs.sessionmanager.MetadataMapListener"/>
</backingMapPluginCollection>
</backingMapPluginCollections>
</objectGridConfig>

6. Tell Liberty’s session manager to reuse the same session ID for all user’s requests, even if handled by different JVM (See Liberty’s documentation for more details)

<httpSession idReuse="true"/>

7. Start Liberty Profile

sst:wlpBLOG sst$ ./bin/server start ServerONE
Server ServerONE started with process ID 11769.

In the logs, wait for the following line

[AUDIT ] CWWKF0011I: The server ServerONE is ready to run a smarter planet.

8. Create & Deploy a simple JSP file for testing

Create a Dynamic Web Project in Eclipse, and add the following index.jsp page

<%@page contentType="text/html" pageEncoding="UTF-8"%>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Liberty Profile Cluster Demo</title>
</head>
<body>
<h1>Liberty Profile - eXtreme Scale HTTP Session Demo!</h1>
<%
Integer count;
Object o = session.getAttribute("COUNT");
if (o != null) {
count = (Integer) o;
count = count + 1;
} else {
count = 1;
}
session.setAttribute("COUNT", count);
%>
<h3>This counter is increased each time the page is loaded.  Its value is stored in the <code>HttpSession</code></h3>
<h3><font color="#FF0000">Counter = <%=count%></font></h3>
<h4>Page server by cluster instance : <font color="#FF0000"><b><%= System.getProperty("wlp.server.name") %></b></font></h4>
<br/>
Page generated at = <%=new java.util.Date().toString()%><br/>
<br/>
</body>
</html>

Then deploy the WAR to the server instance (example of creating a WAR and deploying it to Liberty is given in my previous blog post)

9. Test, open your favorite browser and connect to http://localhost:9080/

You should see the following screen

Each time you will refresh the page (CTRL-R), the counter should be increased by one

Congrats, you have your first instance up and running, let’s now configure a second instance.

Repeat Steps 2-7 on a second Liberty instance to create a second cluster member.  Remember to change the following

  • The name of the instance
  • The HTTP and HTTPS ports used by Liberty Profile (step 2 above)
  • The WXS configuration – only one catalog server is needed (step 3 above, change isCatalog=”no”)
  • You do not need to copy the XML files in the grids directory of the second instance (step 5) – This is only required on the instance running XS’ Catalog Server

Then deploy your test application to instance #2.  To test your application, point your browser to

http://localhost:9081/<YOUR APPLICATION NAME>

You should see a page similar to the one shown at step 9 above.  Try to alternatively reload the page from ServerONE and the page from ServerTWO : you should see the session counter to increase in a sequence across the two server instances.

You’ve just created your first Liberty Profile cluster with two instances and a shared in-memory grid for HTTP session storage.

I leave you as an exercise to install and configure a load balancer in front of these two instances.  Hint : I am using the open-source balance for demo / test purpose.

If you find errors / typos in this (long) article, let me know – I will fix them – Thanks !

Enjoy !

 

]]>
https://www.stormacq.com/websphere-liberty-profile-cluster-sharing-an-in-memory-data-grid/feed/ 0
Distributed Caching, also on Mac OS X https://www.stormacq.com/distributed-caching-also-on-mac-os-x/ https://www.stormacq.com/distributed-caching-also-on-mac-os-x/#respond Sat, 14 Jul 2012 06:20:27 +0000 http://www.stormacq.com/?p=490 IBM’s distributed caching system, WebSphere eXtreme Scale (formerly ObjectGrid) is a distributed, transactional object caching system for elastic scalability and extreme performance.

It can store any type of data and provides REST API as long with Java (HashMap, JPA, Hibernate, Spring) APIs.  It also natively integrates with WebSphere Application Server and WebSphere Liberty Profile to cache HTTP session data.

It is supported on most platforms and – because it is a pure JavaSE application, it also works on Mac OS X, although this platform is not officially supported by IBM.

How to get started ?

  • Download eXtreme Scale trial and unzip
  • In a Terminal, go to product directory
  • cd ObjectGrid/gettingstarted
  • Run the Catalog Server
  • ./runcat.sh
  • Open another Terminal window and start an ObjectGrid server
  • ./runcontainer.sh server0
  • Repeat the last step to create several instances of ObjectGrid server
  • Then experiment with client script.  It provides basic CRUD operations from command line
  • ./runclient.sh i key value

Congrats, you managed to setup a multi instance grid, in-memory cache system on your Mac.

To further understand how it works and how you can programmatically interact with the cache, refer to eXtreme Scale documentation.

Next step will be to demonstrate how eXtreme Scale integrates with Liberty to create a multi instance cluster with shared HTTP Session. Stay Tuned.

Enjoy !

 

 

]]>
https://www.stormacq.com/distributed-caching-also-on-mac-os-x/feed/ 0