Dequeue is stop - PL/SQL Program Units

Hi , Suddenly our dequeue process is not picking up data from Qtable even though we have #50 records are "Ready" stage. Is there any way we can check dequeue is up and running. We keep getting the message during dequeue process.\++++No more messages for Dequeue in Queue table XXPSO_MDM_PDH_Q_TBL_IN Waiting for next iteration. No more messages for Dequeue in Queue table XXPSO_MDM_PDH_Q_TBL_IN Waiting for next iteration. No more messages for Dequeue in Queue table XXPSO_MDM_PDH_Q_TBL_IN till MAX Run Time. Program ran for Maximum Run time : 10(Minutes) and stoped to process another item payloads which are in Queue.Next Run will process those payloads . ++++ Thanks,Raj

This forum is for question about Oracle MOOC (Massive OnLine Open Course) called "PL/SQL Program Units". If you have questions regarding a specific product, please ask it on the appropiate forum inside the community.


TTL and scheduled tasks

Hi all,
I've a problem in processing a jms queue. I'm scheduling the jobs and putting these into a jms Q, the problem is with the TTL. The jobs are almost the same on every schedule and the same messages after a X time are continuously "killed" because the consumers are too slow to consume each message.
It's something like if I have 1 consumer.. 5 messages with a TTL of '3'.. and the consumer takes '1' to complete the job. the 5 messages are almost always the same (these are scheduled).. and I've always processed only the first 3 and killed the other 2...
do you know how to solve this problem? 
i don't understand something about your question.
a scheduled job sends a message to your queue for processing. the message event triggers a message listener MBD to take the message off the queue and process it. it's an asynchronous process, so the listener can take as long as it likes to complete the process. it doesn't send any reply back to the originator of the message.
sounds like you need to check your listener pool settings. if the messages are backing up in the queue, you should add more MDBs to the listener pool.

Artificially delaying messages

We would like to delay queue messages by about 2-3 seconds so that our listner is only delivered the message after this "artificial delay". The external systems that provide messages that we in turn insert into a queue can not handle such a quick reply to a message.
With out having to change our architecture from using a queue listener to creating a MessageConsumer that needs recreating for every message, does anyone know of a quick trick to delaying messages in a queue. A JBossMQ specific solution is also welcome.
Using a selector does not solve the issue because the selection criteria changes with time.
In sumary, is there a way to have a message inserted into a queue only delivered to the consumer after x number of seconds.
Maybe a better way to view this questions is:
Is there a way to schedule the delivery of a message? For example, can you insert a message in a queue and say, delivery to the consumer after 2 hours?
I have searched around and I have not found any selector variable that can indicate the current time.
I was thinking of a selector like:
JMSTimestamp <currentTimeMillis - 60000
where currentTimeMillis is a variable replaced by the current system time in miliseconds (i.e. like sysdate in Oracle). This selector would schedule delivery of every message to consumers one minute after the message has been inserted in the queue.
Is there such a variable as currentTimeMillis? If not, I have to continually recreate the selector and update currentTimeMillis! 
This would be far from elegant, but you could have your listener pull the message from the queue immediately and then use a TimerTask (if the client is in J2SE) or the EJB Timer service (if the client is an MDB) to schedule your business logic a few seconds down the road. 
Tricky and yes, does not seem so elegant.
I can not use AUTO_ACKNOWLEDGE in this case. Also this could get the Timer very busy. There is nothing that stops messages from being pulled out of the queue since the queue consumer would return to get another message once the message is added to the timer.
So I guess a JMS wish list for the selector is a CURRENT_TIME variable...
Safer to have a home made DB queue with a select that only pulls out messages that are older than x seconds. 
One possible solution, which admittedly is JMS provider dependent, is to make use of timeouts and dead letter queues etc.
e.g. you could use a timeout on the messages, say 2-3 seconds - then some JMS providers, like ActiveMQ will send messages that timeout onto a dead letter queue which you could then subscribe from etc.

JMS Message Selector

Im using Sun App Server 8.2 and have a jms queue.
I have 3 timer-triggered ejbs which each have a different message selector.
Every 5 minutes they open a connection to the queue and receive messages from the queue based on their message selector until there are no more left, or the timeout (5secs) is reached.
Most of the time it works ok, but I am experiencing a scenario where one of the ejbs will be processing merrily, but will not see the messages on the queue.
But an arbitary amount of time later, the messages will be received.
For example, I sent 10000 messages into the system, 5000 for each ejb. 9990 messages were processed as normal, but the final ten were left on the system for 15 hours (even though the ejb was processing every 5 minutes, it reported that there were no messages). The messages were fully acknowledged, and I could see them on the queue using the qBrowser demo app.
Anyone experienced this? Or am I doing something wrong?
were any exceptions thrown during the processing?
also, what happens if the message processing takes longer than 5 min? does the timer calls overlap, or is the next one delayed? 
No - no exceptions are thrown.
Also the timer does not fire until the processing is finished (it is an interval, so ony fires once).
The timer is then initialised again at the end of processing.

JMS Fair Queueing

I need to implement a Fair Queueing system based on JMS. The scenario is : 10 Companies submiting requests with varying priorities (say high, medium and low). The requirement is to process these requests fairly, which is deemed to be at a throughput of 100 messages/sec per customer, irrelevant of the priority. However, per customer the requests should be processes in a ratio of 5 high to 1 medium, and 5 medium to 1 low.
In order to fulfil this i'd have a Q per customer, and a single MDB that would filter dynamically (i know, not nice... or do-able by the looks of things) and process it's filter by keeping a simple count of the request priorities.
Has anyone done something similar? Any help would be greatly appreciated.
Brainstorming a few ideas (not without flaws)...
Idea 1
3 MDBs each with a different message selector
MDB1 processes P1 messages
MDB2 processes P2 messages
MDB3 processes P3 messages
You'll have to get your customers to set a JMS Header (not JMSPriority) with values for 'P1, P2 or P3.
The tricky part is working out how to force the 25:5:1 ratio. A rough way to do this would be to create 25 instances of MDB1, 5 instances of MDB2 and 1 instance of MDB3.
In this case you're relying on the container to actually create 25 instances of MDB1 and 5 instances of MDB2, but there is no guarantee it will do this. If the container is maxed out, you may only have 10:5:1. You're also relying on thread contention to do the prioritising for you, which won't necessarily give you the ration you need. Your container may also end up with 240 MDB concurrent instances to manage.
Idea 2 (similar to your filter idea)
Use an MDB to intercept the messages and re-asses the priority before sending to the real queue and real MDB, i.e.
1. Customer sends message with JMSPriority 0 (high)
2. MDB1 reads messages in priority order
3. MDB1 re-evaluates priority based on the last X messages and decides it is now priority 2
4. MDB1 sends the message to the outbound queue
5. MDB2 reads the messages in priority order
This has the problem when a message is assigned a low priority, it will remain a low priority and therefore may never be processed if your system is busy. You've also introduced a single MDB bottleneck.
Idea 3
The MDB simply writes all messages to a database
A second process (Session Bean with a Timer perhaps) reads the messages from the database, performs some inteligent ordering and processing them accordingly.
This solution may not suite your performance criteria and would be tricky to implement in a cluster.
Idea 4
Don't use MDB's - instead have a "battery" of QueueReceivers for each customer and a "manager" performing a kind of round robin.
Receiver1 reads only P1 messages using the MessageSelector
Receiver2 reads only P2 messagse using the MessageSelector
Receiver3 reads only P3 messages using the MessageSelector
// Manager application does something like...
while (true)
  while (R1.moreMessages()  && R1.getMsgCount <= 25)
       // If wait time is exceeded assum no more messages
   While (R2.moreMessages() && R2.getMsgCount <= 5)
   While (R3.moreMessages() && R3.getMsgCount <= 1)
}In order to meet your 100msg/s/customer (which depending on message size and how long your processing takes might be extremely tough to meet), you may need to have the several instances of the above. 
Thanks Steve, I'm going to give your QueueReceiver Battery a go. It is probably the closest to a Fair Queueing mechanism. 
Its an interesting problem :) I agree with pretty much everything Steve just said.
Another alternative could be a slight twist on 3 and 4; you might want to separate out the reordering/scheduling of the messages from the code actually processing them (as you might have many JVMs running etc).
So a hybrid of 3 and 4 could be a single transacted queue consumer that receives messages & puts them into a RAM buffer to keep track of the ratios etc; then when the right ratio of messages are right (or a timeout fires), they are forwarded on to the ultimate destination & the messages that are sent are acknowledged - then the transaction commits and things start over again for the next batch.
Open Source SOA 
I think your problem statement is flawed, actually. What if there are no higher-priority messages - do they really want the lower-priority messages to wait until the quota of higher-priorities has been filled? What if the actual ratio of messages is not in the ratio 25:5:1?
It seems to me that a HP message should have 5 times the probability of being read as against an LP message, which I suppose you can achieve by adroit use of a variable MessageSelector expression on a single queue, but regardless of that if there is a message there to be read you should certainly read it. 
The point of fair queueing is just that. The problem is that i need to process 100 messages / sec regardless of what the priority is. So if there were 1000 of each type of message they should be processed in the ratio 25:5:1 at 100m/s untill the queues were depleted... If there were however only low-priority messages they would also be processed at 100m/s. 
Agreed but the solutions provided so far don't accomplish that: they block LP messages until 5 MP and 25 HP messages have been read regardless of how long that takes and regardless of the actual distribution of message priorities. If your queues are always full and the priorities are always distributed 25:5:1 there is no issue, I am just wondering what is supposed to happen if only LP messages exist for a while. Gotta be fair to them messages as well! 
Look again. None of the solutions block until all 25HP messages have been received.
Solution 1. has lots of MDBs which run concurrently processing messags as they arrive. No blocking there.
Solution 2. MDB1 re-assess the priorty based on the number of HP/MP/LP messages received and moves them to another queue. It should do this by keeping track of the number of each message type, and not by holding onto MP/LP messages until 25 have arrived. MDB2 reads the messages in this new priority order. No blocking here either.
Solution 3 Reads the messages from a database, reassess priority and processes them in the new order. No blocking.
Solution 4. Reads up to 25 HP messages, and does block for a short wait time (but not until all 25 HP messages have been read).
So if there was a wait time of 0.01s and (worst case) each HP message was 0.009s apart the application would have wasted roughly 0.009s * 25 =~ 0.225s when it could have been processing MP/LP messages. This may be releaved by having more than one instance of the battery class, so that while one instance is waiting other instances are processing. You would need to tune the wait time though.

Dead queue processing

Hi All
I would like to know how to go about dead queue processing
First a very basic question as i am new to messaging systems.
1) When a message sent from a remote queue (QueueMgr1) to another queue on (QueueMgr2) fails and some how reaches dead queue, it will reach dead queue on which QM, (saying dead queues are configured on both QM's).
2) How do we process the dead queue messages, is there any standard practice.?
3) If I want to resend the messages from dead queue to the orginal intended rceipient how do I do it? Is it simply receive the messages from dead queue and put it on the queue. Or is some level of processing needs to be done to get the original message back? How do I retrieve the reason for failure from the message? Is there a JMS API call for the same?