Streaming queue feature in IBM MQ 9.3

IBM MQ 9.3.0 adds the streaming queues feature, which allows you to configure a queue to put a near-identical copy of every message to a second queue. Streaming queues can be useful in certain scenarios, where you need to create a copy of your messages. For example:

  • Streaming messages to Apache Kafka using the Kafka Connect source connector for IBM MQ.
  • Performing analysis on the data going through the system.
  • Storing messages for recovery at a later time.
  • Capturing a set of messages to use in development and test systems.
  • Consuming IBM MQ event messages from the system event queues, and sending additional copies to other queues or topics.

Refer below diagram for illustrating the streaming queue

How to configure Steaming queue ?

Configuration is done on a queue-by-queue basis. For example if messages are currently delivered to QUEUE1 and you would like duplicates of those messages streamed to QUEUE2 you could set the new STREAMQ attribute as follows:

ALTER QL(QUEUE1) STREAMQ(QUEUE2) STREAMQOS(BESTEF)

The STREAMQOS attribute allows you to choose between the 2 streaming queue operating modes:

  • Best Effort (BESTEF)
    • This option is the default and is the best option for scenarios where you do not want the original application to be affected by the streaming queue feature. Messages will be delivered to the original queue (in our case that’s QUEUE1) regardless of any problems encountered when streaming messages to QUEUE2. So if QUEUE2 is full when a message is put by the application it will be delivered to QUEUE1 for processing even though the queue manager isn’t able to stream a copy to QUEUE2.
    • Set using ALTER QL(QUEUE1) STRMQOS(BESTEF)
  • Must Duplicate (MUSTDUP)
    • This option is best for scenarios where it is important that each message delivered to the original queue is also delivered to the stream queue. If there is a problem streaming a message to QUEUE2 then the message won’t be delivered to QUEUE1 and the putting application will receive an appropriate error code. The application will need to retry its put just like it would if QUEUE1 had a problem.
    • Set using ALTER QL(QUEUE1) STRMQOS(MUSTDUP)

The messages sent to the stream queue are almost identical with a small number of exceptions. For example, the following aspects of the stream queue messages are identical to the original message:

  • Message ID
  • Correlation ID
  • Message Properties
  • Application Data

The following parts of the stream queue message may be different to the original:

  • Report options that cause the queue manager to generate new messages are removed
  • Expiry is set to unlimited regardless of the expiry of the original message. If the stream queue has a CAPEXPRY value configured then that value will be applied to the streamed message.

If we want more copies of message then ??

the streaming queue cam point to a local or alias queue . if we point to alias queue then followed by a topic then we will subscribe as many subscribers we want . In this way we can duplicate but message id is not identical across all the messages delivered from the topic to subscribers . Please note this and adjust accordingly to the application requirements .;

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *