Fail high load with Open Source

SMP Capacity

3100+ пользователей

Ожидание

• 1 приложение
• 1 БД
• 2 сервера

SMP Capacity

3100+ пользователей

Реальность

• 3+ приложения
• 3 БД
• 6 серверов

Architecture

• Java
• Tomcat
• SMP
• Postgres

SMP

• Log4j
• Artemis
• Infinispan
• Hibernate
• Lucene

Java CodeCache

        #
        java -XX:ReservedCodeCacheSize=512m -version
    

MBean

JIT Compiler. Live

JIT Compiler. Dead

Tomcat

Tomcat connector

Tomcat non blocking

Log4j

Log4j

• 80Gb/day
• 7Gb/hour
• NAS storage

SMP loves JMS

SMP loves JMS

• 8+ queses
• 3k messages/min

Large messages

Artemis Failover

• FS
• Replication
• JDBC

Artemis replication

Artemis replication

Spring JMS

• JmsTemplate

        context.getBean("jmsTemplete").send();
    

• AbstractMessageListenerContainer

        public class listener 
        extends AbstractMessageListenerContainer {
            public void onMessage(Message message, Session session) {}
        }  
    

Artemis connection

        //      Spring JMS integration
        
        // listener acquires a connection
        listener.onMessage() 

        //jmsTemplate acquires a connection
        context.getBean("jmsTemplete").send(); 


        // This is a connection factory
        public class ActiveMQConnectionFactory {

            // really create a connection 
            protected synchronized ActiveMQConnection 
                    createConnectionInternal(final String username,
                                            final String password,
                                            final boolean isXA,
                                            final int type) {}
        }

Artemis contention

JFR recording for 40 min

• 4500+ waits
• 17 min waits

Here comes cluster

Artemis contention

JFR recording for 40 min

• 1000 waits
• 3 min waits

Infinispan

Infinispan async

Infinispan replication

Lucene

Lucene Directory

• MMapDirectory
• Infinispan Lucene Directory

Lucene Directory

FSDirectory is a solid option
15% + faster. for free.

Connection pool

Connection pools: c3p0

Connection pools: HikariCP

HikariCP

C3P0
2000 waits per 40min

Hikari
0 waits per 40min

HikariCP is ultra fast connection pool. Proven.

Hibernate executeUpdate

Session session = SessionFactory.getCurrentSession();
Query query = session.createSQLQuery("update tbl_servicall ...");
query.executeUpdate();

query = session.createQuery("update servicecall ...")
query.executeUpdate();
            

Hibernate partitioning

Magic of indexes

Postgres replication

Outcome

• Internet posts could be wrong
• Bugs you found is only your problem
• Every project has a roadmap