Friday, September 14, 2012

weblogic.socket.MaxMessageSizeExceededException appearing when Managed Server Attempting To Send larger message to Admin Server.

weblogic.socket.MaxMessageSizeExceededException appearing when Managed Server Attempting To Send larger message to Admin Server.


We used to get the following exception in the weblogic managed servers log file frequently

 ####<Mar 20, 2012 12:22:47 PM EST> <Error> <Socket> <ifappc105p> <AdminServer> <ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'> <> <> <a33374ea6b7bab0f:17f6d3b7:1343f18f4ea:-8000-000000000000008a> <1323912167710> <BEA-000403> <IOException occurred on socket: Socket[addr=ifappc105p-vip1/172.105.200.105,port=8000,localport=55103]
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '10000480' bytes exceeds the configured maximum of: '10000000' bytes for protocol: 't3'.
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '10000480' bytes exceeds the configured maximum of: '10000000' bytes for protocol: 't3'
at weblogic.socket.BaseAbstractMuxableSocket.incrementBufferOffset(BaseAbstractMuxableSocket.java:174)
at weblogic.rjvm.t3.MuxableSocketT3.incrementBufferOffset(MuxableSocketT3.java:351)


The reason is the weblogic managed server tries to send the message size more than 10 MB and failing with the following exception. It is observed that every single time this message is observed in the logs, it is followed by the respective managed server who is sending that message, losing connectivity with the admin server.

The root cause is the RJVM messages used to get piled up due to default watches trap events that’s comes installed and it used to grow in larger size causing the broken communication between managed servers and AdminServer

The solution to resolve this is to disable the default WLDF configured in the server.

Steps to disable the default WLDF configured in the server

  • Login into admin console with weblogic user.
  • Click Lock & Edit
  • Expand Diagnostics tab 

  • Click on Diagnostic Modules


  • Click on Module-FMWDFW


  • Click on Watches and Notifications


  • Click on Deadlock


  • Uncheck the below highlighted (Enable Watch)

  • Save the changes and continue the same for remaining 2 components as well. 
               StuckThread  and UncheckedException 
  • Save the changes.
  • Restart all servers (admin and managed)
The default message size of all the servers can also be increased. The configure can be changed in the protocol section of the server configuration from the weblogic console as shown below




This stabilized the connectivity issue between the managed servers and the Admin Server.


3 comments:

  1. Hi Albin,
    This is a very useful information, i want to know if we can analyses these RJVM messages where does these messages get stored and how to view them.

    In our case we are still seeing the Max Message even after disabling the diagnostics as describe by you.
    Thanks in Advance.


    ReplyDelete
  2. Hi sachin,

    Are you still encountering the issue? What fix did you applied?

    ReplyDelete
  3. Hi Albin,
    Is there some way I can enable the compression of messages at t3 socket level.

    Thanks
    Ankit

    ReplyDelete