On heap cache vs off-heap cache

On-heap cache vs Off-heap cache
Radek Grębski
@RadekGrebski
https://github.com/rgrebski

PRESENTATION PLAN
1. Heap vs off-heap
2. Heap memory
3. Off-heap memory
3.1 Memory mapped file
3.2 Unsafe and ByteBuffers
3.3 Off-heap memory advantages
4. Cache:
4.1 Chronicle
4.2 Hazelcast
4.3 Redis
5. Comparison

2. HEAP MEMORY
JVM memory
JDK8
S1Eden S0 Old Generation
Heap (-Xmx)
Metaspace
(-XX:MaxMetaspaceSize)
Native memory
JVM Process memory
OS memory
Young Generation

Objects on heap
Source: http://www.ibm.com/developerworks/library/j-codetoheap/
Integer (64-bit JVM): 7:1 (28 bytes)
9:1 (36 bytes)
Integer (32-bit JVM):
3:1 overhead ratio
16 bytes
String (32-bit JVM):
3.75:1
60 bytes
int[1]
2. HEAP MEMORY

3. OFF-HEAP MEMORY
Off-heap (native) memoryJVM Process
OS memory
Other processes /
unallocated
byte[]
byte[]

3.1. MEMORY MAPPED FILE
Off-heap memoryJVM Process
OS memory
JVM Process
/tmp/myFile.dat
byte[]
byte[] byte[]
byte[]

3.2. UNSAFE AND BYTEBUFFERS
How to allocate memory using standard Java classes?
java.nio.ByteBuffer:
HeapByteBuffer (on-heap, up to 2gb)
DirectByteBuffer (off-heap, up to 2gb)
MappedByteBuffer (off-heap, up to 2gb, persisted)
sun.misc.Unsafe
public native long allocateMemory(long allocationSizeInBytes);

ByteBuffer.allocate ( <2GB )
JvmUtils.verifyJvmArgumentsPresent("-Xmx2g");
ByteBuffer byteBuffer = ByteBuffer.allocate((int) ByteUtil.GB);
byteBuffer.putChar('a') //2bytes, position =0
.putInt(123) //4bytes, position = 2(0 + 2(char))
.put("test".getBytes("UTF-8")); //6 => 2(char) + 4(integer)
byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length];
char charA = byteBuffer.getChar(/*address*/ 0); // 'a'
int int123 = byteBuffer.getInt(/*address*/ 2); //123
byteBuffer.position(6); //set cursor position
byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"

Unallocating ByteBuffer memory
public static void callCleaner(ByteBuffer byteBuffer){
//DirectByteBuffer.cleaner().clean()
Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner");
cleanerMethod.setAccessible(true);
Object cleaner = cleanerMethod.invoke(byteBuffer);
Method cleanMethod = cleaner.getClass().getMethod("clean");
cleanMethod.setAccessible(true);
cleanMethod.invoke(cleaner);
}

Allocating memory using sun.misc.Unsafe
sun.misc.Unsafe::getUnsafe():
public static Unsafe getUnsafe() {
Class cc = sun.reflect.Reflection.getCallerClass(2);
if (cc.getClassLoader() != null)
throw new SecurityException("Unsafe");
return theUnsafe;
}
Creating an instance of Unsafe:
public static Unsafe createUnsafe() {
Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();
unsafeConstructor.setAccessible(true);
return unsafeConstructor.newInstance();
}
Memory allocation:
long memorySizeInBytes = ByteUtil.GB * 4;
long startAddress = unsafe.allocateMemory(memorySizeInBytes);
unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0);
unsafe.freeMemory(startAddress);

Memory mapped file using Java API
File mappedFile = new File("/tmp/mappedFile.tmp");
mappedFile.delete();
try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) {
long buffer8MB = 8 * ByteUtil.MB;
MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0,
buffer8MB);
long startAddress = 0;
long elementsToPut = 200_000_000;
for (long counter = 0; counter < elementsToPut; counter++) {
if (!mappedByteBuffer.hasRemaining()) {
startAddress += mappedByteBuffer.position();
mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress,
buffer8MB);
}
mappedByteBuffer.putLong(counter);
}
}
Time: 1,068 s
Filesize: 1,49 GB

Memory mapped file content

Big endian vs Little endian
0xCAFEBABE
Address 00 01 02 03
Big endian CA FE BA BE
Address 00 01 02 03
Little endian BE BA FE CA

ByteBuffer endianess
@Test
public void testEndianess() throws NoSuchMethodException, IllegalAccessException,
InvocationTargetException {
ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4);
bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN);
ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4);
littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN);
littleEndianByteBuffer.putInt(0xCAFEBABE);
bigEndianByteBuffer.putInt(0xCAFEBABE);
String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array());
String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array());
assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE");
assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA");
}

3.3. OFF-HEAP MEMORY ADVANTAGES
Off-heap storage
Pros and cons
No GC!
Manual GC!
Memory leaks
Persistence
IPC
Latency
Durability
Scalability (1TB+)

4. CACHE
HashMap
Collections.synchronizedMap(..)
ConcurrentHashMap

4.1. CHRONICLE
http://chronicle.software
Chronicle Map Chronicle Set Chronicle Queue
Chronicle Logger Java Thread Affinity

4.1. CHRONICLE MAP
Creating off-heap map instance
private Map<String, OffHeapUser> chronicleMap;
private static final String KEY_SAMPLE = "12345678901234567890";
private static final long MAX_ENTRIES = 10_000_000;
{
chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class)
.averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length)
.constantValueSizeBySample(new OffHeapUserSample())
.createPersistedTo(new File("/tmp/mappedFile.bin"))
.entries((long) (MAX_ENTRIES))
.create();
}

4.1. CHRONICLE MAP
Complex structures
public interface OffHeapUser {
String getUsername();
void setUsername(@MaxSize(30) String username);
long getAccountValidUntil();
void setAccountValidUntil(long accountValidUntil);
void setRoleAt(@MaxSize(2) int index, Role role);
Role getRoleAt(@MaxSize(2) int index);
static interface Role {
String getRole();
void setRole(@MaxSize(10) String role);
}
}

4.1. CHRONICLE MAP
Creating value instance
//fill the data
long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS;
String username = RandomStringUtils.randomAlphabetic(20);
OffHeapUser offHeapUser = chronicleMap.newValueInstance();
offHeapUser.setAccountValidUntil(accountValidUntil);
offHeapUser.setUsername(username);
OffHeapUser.Role role0 = offHeapUser.getRoleAt(0);
role0.setRole("Role0");
OffHeapUser.Role role1 = offHeapUser.getRoleAt(1);
role0.setRole("Role1");
//put
chronicleMap.put("someKey", offHeapUser);
//get
OffHeapUser offHeapUserActual = chronicleMap.get("someKey");

4.1. CHRONICLE MAP
Throughput
Key = „u:0123456789”, value = counter
*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.
Source: https://github.com/OpenHFT/Chronicle-Map
0
20
40
60
80
100
120
140
160
180
10 000 000 50 000 000 250 000 000 1 250 000 000
ThroughputMupd/s
Map entries
Throughput - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory

4.1. CHRONICLE MAP
Memory usage
OutOfMemory
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
10 000 000 50 000 000 250 000 000 1 250 000 000
MemoryinGB
Map entries
Memory used - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap

4.1. CHRONICLE MAP
GC pauses
0,0
5,0
10,0
15,0
20,0
25,0
30,0
35,0
40,0
45,0
50,0
10 000 000 50 000 000 250 000 000 1 250 000 000
WorstGCpauseinseconds
Map entries
Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap
Cronicle Map
ConcurrentHashMap
OutOfMemory

4.2. HAZELCAST
On-heap cache
Off-heap support (High Density Memory - commercial)
List, Map, Set, Queue
Topics
Executor Service
Dynamic clustering
Transactional

4.2. HAZELCAST
Example
@Test
public void hazelcastClusterTest(){
Config hazelcastConfig = new Config();
HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig);
HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig);
Map<String, String> node1Map = hazelcastInstance1.getMap("someMapName");
Map<String, String> node2Map = hazelcastInstance2.getMap("someMapName");
node1Map.put("key", "value");
Assertions.assertThat(node2Map.get("key")).isEqualTo("value");
}
Output:
Members [2] {
Member [192.168.1.23]:5701 this
Member [192.168.1.23]:5702
}

4.3. REDIS
REmote DIctionary Server
Key/Value cache + store
ANSI C
Used by:
StackOverflow
GitHub
Twitter
Instagram
Alibaba
Clients for almost all programming languages

4.3. REDIS
Redis data types
HashMaps (hset, hget)
LinkedList (lpush, ltrim, lrange)
Queue (lpush, rpop, brpop)
Topics (publish, subscribe)
Set (sadd, srem)
SortedSet (zadd, zrem)

4.3. REDIS
Redis Java clients
Jedis
Redisson
Aredis
JDBC-Redis
Jredis
Lettuce
RedisClient

4.3. REDIS
Redis HashMap example
@Test
public void testRedisMap(){
Jedis jedis = new Jedis("localhost");
// Pipeline pipeline = jedis.pipelined();
// pipeline.multi();
jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek");
jedis.hset("user:1", "lastName", "Grebski");
jedis.hset("user:1", "email", "rgrebski@gmail.com");
// pipeline.sync()
Map<String, String> mapFromRedis = jedis.hgetAll("user:1");
assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek");
assertThat(mapFromRedis)
.hasSize(3)
.contains(entry("firstName", "Radek"))
.contains(entry("lastName", "Grebski"))
.contains(entry("email", "rgrebski@gmail.com"));
}

5. COMPARISON
Chronicle Hazelcast Redis
Deployment model Embedded Embedded / Separate Separate
Replication Yes Yes Master/Slave
Topics (pub/sub) No Yes Yes
Executor Service No Yes No
Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically)
Java collections support Yes Yes Yes (Redisson)
Clustering No Yes Yes (since 3.0)
Distributed events No Yes Yes
Features

5. COMPARISON
R/W Performance
Chronicle Hazelcast Redis ConcurrentHashMap
Write (2mln entries) 2 555 583 178 491 110 668 1 100 715
Read (2mln entries) 937 207 214 638 156 177 4 092 769
0
500 000
1 000 000
1 500 000
2 000 000
2 500 000
3 000 000
3 500 000
4 000 000
4 500 000
Ops/s
R/W Performance
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

5. PERFORMANCE COMPARISON
GC Pauses
Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)
Chronicle Hazelcast Redis ConcurrentHashMap
GC pauses summarized 39 1452 480 726
GC pause max 8 28 15 360
0
200
400
600
800
1000
1200
1400
1600
Timeinmillis

Which cache should I use in my app ?

On heap cache vs off-heap cache

More Related Content

What's hot (20)

Similar to On heap cache vs off-heap cache (20)

Recently uploaded (20)

On heap cache vs off-heap cache