AMQ6109S: An internal WebSphere MQ error has occurred

We have upgraded MQ from 9.0 to 9.2 and the installation is successful .When we start QMGR we got below error .

AMQ6109S: An internal WebSphere MQ error has occurred

If we get these kind of error then verify FFDC logs and QMGR logs to see more details . Also try to create a test manager and see if you are able to create and start .

we verified any mq related seamphores running using ipcs | grep -i mqm

If its running, remove all the seamphores related to mq using ipcrm -m <pid>

Please Be careful to list out only mq seamphores and delete mq seamphores
only. In the server might be few other seamphores related to other apps also
running.

Here Example:

ipcs | grep -i mqm

m 317718551 0x007750d0 –rw-rw—- mqm mqm
m 24 0x007750f0 –rw-rw—- mqm mqm
m 25 0x007750f4 –rw-rw-rw- mqm mqm
m 26 0x00775129 –rw-rw—- mqm mqm
m 27 0x00775130 –rw-rw-rw- mqm mqm
m 28 0x00775139 –rw-rw-rw- mqm mqm
m 29 0x0077513a –rw-rw-rw- mqm mqm
m 30 0x00775155 –rw-rw—- mqm mqm
m 31 0x00775156 –rw-rw—- mqm mqm
m 32 0x00775157 –rw-rw—- mqm mqm
m 33 0x00775158 –rw-rw—- mqm mqm
m 34 0x00775159 –rw-rw—- mqm mqm
m 35 0x0077515a –rw-rw—- mqm mqm
m 36 0x0077515b –rw-rw—- mqm mqm

ipcrm -m 317718551

….

ipcrm -m 36

if you have hunders or thousands of semaphoers then it is difficult to remove one by one using ipcrm -m <pid>

ipcrm |grep -i mqm | awk ‘{ print $2 }’ | xargs ipcrm -m

or send pid to file using ipcrm |grep -i mqm | awk ‘{ print $2 }’ >/tmp/pid

then run below

while read line; do 
   ipcrm -m $line
done</tmp/pid

Now verify again with ipcrm |grep -i mqm

Once you clear the seamphores. Start the qmgr if the issue is due to semaphores then QMGR will start successfully now .

This is a typical scenario. An administrator might end their queue manager and listeners but forget to end one or more of their MQ applications.

A problem can arise if that application is bound to the queue manager using shared memory bindings  and has not made an MQI call for a while. Application processes like this can remain attached to MQ's shared memory segments even though the queue manager has ended.

In this scenario, and supposing the application did not make another MQI call, it will remain attached to the shared memory. The administrator will find that the queue manager later refuses to start, and error messages will be written by the strmqm command. The reason is that, during strmqm, the queue manager startup code detects the presence of those old applications and fails the startup processing.

error in the logs looks like An error has been detected and the IBM MQ error recoding routine called . the failing process is process XXXX .

The error messages from strmqm do state that the application(s) should be ended to allow the queue manager to start. The error states also mention the process id (pid) to make it easy to find the process concerned and end it.

In our case concerned process id is printed in the logs .