Async servlets. 9 years later

Sync is great!

• Не требует настройки
• Потоков хватит всем
• Есть MBean для настройки

Sync cons

• Простой способ сделать DoS
• Каждый поток - расход память

Sync Servlet

Sync Servlet

Async Servlet

Async in 5 minutes

public class AsyncServlet extends HttpServlet {
    @Override
    public void service(HttpServletRequest req, HttpServletResponse resp) {


        // set container thread free
        final AsyncContext context = request.startAsync();
        
        // process request in an App thread
        executor.submit(()->{

            try {
                // call AsyncServlet to process
                serviceInt(context.getRequest(), context.getResponse());
            }
            finally {
                // close user request / response / etc
                context.complete();
            }

        })
    }
}

AsyncContext

public interface AsyncContext {

    public ServletRequest getRequest();
    public ServletResponse getResponse();

    public void complete();

    // what is dispatch??
    public void dispatch();
}

What is dispatch

Configuration

web.xml

    NocacheFilter
    ru.naumen.core.server.NocacheFilter
    
    true




    jwt-servlet
    ru.naumen.sec.server.jwt.provider.JwtTokenServlet

    true
    

Configuration

Annotations

@WebServlet(urlPatterns = { "/async" }, asyncSupported=true)
public class AsyncServlet extends HttpServlet {
}

@WebFilter("/*", asyncSupported=true)
public class MyFilter implements Filter {
}

It works!

It works!

with Tomcat 8

RequestDispatcher

public class AsyncServlet extends HttpServlet {

    @Override
    public void service(HttpServletRequest req, HttpServletResponse resp) {

        final AsyncContext context = request.startAsync();
        executor.submit(()->{
            try { 
                
                // WTF ?!?!

                context.getRequest().getRequestDispatcher("/denied.jsp")
                    .forward(request, response);
            }
            
            finally {
                context.complete();
            }
        });
    }
}

RequestDispatcher

RequestDispatcher

public class AsyncServlet extends HttpServlet {

    @Override
    public void service(HttpServletRequest req, HttpServletResponse resp) {
        final AsyncContext context = request.startAsync();
        
        executor.submit(()->{
            boolean forward = false;

            try { 
                forward = true;
                
                // no RequestDispatcher needed
                context.dispatch("/denied.jsp");
            }        
            finally {

                if(!forward)
                    context.complete();

            }
        });
    }
}

Servlet filters. Sync

Servlet filters

public class NocacheFilter implements Filter
{
    @Override
    public void doFilter(ServletRequest request, final ServletResponse response,
                         FilterChain chain) throws IOException, ServletException
    {
        // do pre processing
        try {

            // call handlers
            chain.doFilter(request, response);

        } finally {
            // do post processing
        }
    }

Servlet filters. Async

// from Spring Security with love!

Servlet filters. Final

// somewhere in filter chain
chain.doFilter(new WrappedRequest(request), 
            new WrappedResponse(response));

// async servlet
public void service(HttpServletRequest req, HttpServletResponse resp) {
    AsyncContext context = request.startAsync();

    // at execution moment
    // is context.getRequest() wrapped ? 
    // is context.getRequest() equals to req ?
    
    context.getRequest().getInputStream().available();
    
    context.complete();
}

// from JavaMelody with love!

Error handling

public class AsyncServlet extends HttpServlet {

    @Override
    public void service(HttpServletRequest req, HttpServletResponse resp) {
        final AsyncContext context = request.startAsync();
        executor.submit(()->{
            try {

                doSomeWorkAndGenerateException(); 

            }        
            finally {
                context.complete();

                // what user will get in case of error?
            }
        });
    }
}

Error handling

public void service(HttpServletRequest req, HttpServletResponse resp) {
    
    // if an error has occured during async processing
    if(req.getAttribute(RequestDispatcher.ERROR_EXCEPTION)){
        throw new ServletException();
    }

    // if everything is ok
    final AsyncContext context = request.startAsync();
    executor.submit(()->{
        // doSomeWorkAndGenerateException(); 
        
        catch(Throwable tr) {
            context.getRequest().setAttribute(
                RequestDispatcher.ERROR_EXCEPTION, tr);
            // send request back
            context.dispatch();
            return;
        }
        // in case everything is ok
        context.complete();
    });
}

Tests are falling

• Тест не находит dom-element'а
• Ответ не соответствует запросу

Tomcat IOException

• Response is already committed
• Broken pipe

Response commit

    public interface HttpServletResponse {
        
        public PrintWriter getWriter();

        public ServletOutputStream getOutputStream();

        public int getBufferSize();

        public void resetBuffer();

        public void flushBuffer();
    }

Another server

• OpenLiberty
• WildFly
• Jetty

Request attributes

public interface HttpServletRequest {
    public String getRequestURI();
    public String getContextPath();
    public String getPathInfo();
    public String getServletPath();
    public String getQueryString();
}

public interface AsyncContext {

/**
* The name of the request attribute under which the original
* .... is made available to the target of a
* {@link #dispatch(String)} or {@link #dispatch(ServletContext,String)} 
*/

 static final String ASYNC_REQUEST_URI="javax.servlet.async.request_uri";
 static final String ASYNC_CONTEXT_PATH="javax.servlet.async.context_path";
 static final String ASYNC_PATH_INFO="javax.servlet.async.path_info";
 static final String ASYNC_SERVLET_PATH="javax.servlet.async.servlet_path";
 static final String ASYNC_QUERY_STRING="javax.servlet.async.query_string";
}

Jetty works!

• org.eclipse.jetty.io.EofException - Response committed
• java.io.IOException - Broken pipe

EofException: SetContentLength

    public interface HttpServletResponse {
        
        public void setContentLength(int len);
        public void setContentLengthLong(long len);
        
        // name = "Content-Length"
        public void addHeader(String name, String value);
        public void setIntHeader(String name, int value);
    }

IOException: broken channel

public void service(HttpServletRequest req, HttpServletResponse resp) {
    // if everything is ok
    final AsyncContext context = request.startAsync();
    executor.submit(()->{
        try{
            doSomeWorkAndGenerateException(); 
        }
        catch(Throwable tr) {
            // send request back
            context.dispatch();
            return;
        }
        // in case everything is ok
        context.complete();
    });
}

Async request flow

// avoid setContentLength()
// start
AsyncContext context = request.startAsync();

try{
    // avoid setContentLength()
    doWork();
}
catch(Throwable e) {
    // store e in the request attribute
    context.dispatch();
}
finally {
    if(!forward){
        // no flush - no data
        response.flush();
        // cleanup
        context.complete();
    }
}

Spring MVC Async

• Specification: undetermined behavior
• Servers: every server has its own issues
• Developers: set content length, hide errors

What is async really for?

• Gain throughput
• Gain resource utilization

What is async really for?

• Resiliency

Resiliency