Monitoring Postfix Email Queue and add Push Notifications
Reading time: 6 minutes
Monitoring email queues on your favorite email server is an essential task of an email admin, as rising queues are often an indication of problems. First, you need to understand how queues work, manage them, and get emails enqueued. Then build a shell script to monitor them. In the last step, you will learn how to get smartphone push notifications by sending emails to a third-party push service.
Queue Basics
Sending and receiving emails is by default done asynchronously. Both are handled by your email service like Postfix, Exim, Sendmail, etc. We will focus on Postfix, but you can map this on any email system by simply replacing the line for accessing the mail queue.
As the mail service process them asynchronously, they need to be queued before and maybe during processing for good reasons:
- preprocessing like IP blacklists, spam filters, antivirus, filters, etc.
- network problems, delays, or temporary rejects
- not reachable, overloaded, or simple broken email systems you try to send emails to
By using queues, you should have some crucial things in mind about Queueing theory (Wikipedia):
- Queues may get full or overflow, and at some point, it always will
- A big queue size (number of enqueued emails) can significantly slow down processing if the number of enqueued emails rise
Let’s focus on the queue size in our email service and how to monitor this number. But first, we need to know how to access the Postfix email queues and what we can do with them.
Accessing Queues and Email Messages in Queues
First of all, we want to list enqueued emails in our system. For Postfix, we can do this with one of these commands:
root@server $ mailq
Mail queue is empty
root@server $ postqueue -p
Mail queue is empty
“Mail queue is empty” means there is nothing to process. To see some enqueued mail, you can try this from your command line:
root@server $ mail -s 'testing message subject' someone@gsmail.com <<< 'testing message body' && mailq
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
2A0324780066 251 Wed Nov 4 13:29:07 root
someone@gsmail.com
-- 0 Kbytes in 1 Request.
Now you can see an enqueued email. If your queue has some other mails, you will see them too. If you choose a not existing domain as I did with gsmail.com you can rerun the command to see processing information for your email:
root@server $ postqueue -p
-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------
E900C4780073 462 Wed Nov 4 13:36:46 root@mail.myserver.local
(Host or domain name not found. Name service error for name=gsmail.com type=MX: Host not found, try again)
someone@gsmail.com
As we have some unprocessable email in a queue, we can try to access the message by Queue ID E900C4780073 and see what is inside:
root@server $ postcat -q E900C4780073
*** ENVELOPE RECORDS deferred/E/E900C4780073 ***
message_size: 462 210 1 0 462 0
message_arrival_time: Wed Nov 4 14:36:46 2020
create_time: Wed Nov 4 14:36:46 2020
named_attribute: rewrite_context=local
sender_fullname: root
sender: root@mail.myserver.local
*** MESSAGE CONTENTS deferred/E/E900C4780073 ***
Received: by mail.myserver.local (Postfix, from userid 0)
id E900C4780073; Wed, 4 Nov 2020 14:36:46 +0100 (CET)
To: someone@gsmail.com
Subject: testing message subject
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Message-Id: <20201104133646.E900C4780073@mail.myserver.local>
Date: Wed, 4 Nov 2020 14:36:46 +0100 (CET)
From: root@mail.myserver.local (root)
testing message body
*** HEADER EXTRACTED deferred/E/E900C4780073 ***
named_attribute: encoding=8bit
named_attribute: dsn_orig_rcpt=rfc822;someone@gsmail.com
original_recipient: someone@gsmail.com
recipient: someone@gsmail.com
*** MESSAGE FILE END deferred/E/E900C4780073 ***
Deleting emails from Postfix queue
To remove unwanted or for testing generated emails from the Postfix queue, you can use the command postsuper and the Queue ID to remove a message:
root@server $ postsuper -d E900C4780073
postsuper: E900C4780073: removed
postsuper: Deleted: 1 message
Be careful when viewing and deleting emails, as you could be breaking the law. But we need all these commands to test our monitoring system for Postfix queues. Keep on reading.
Script to monitor the queue size
First of all, we want to write a bash script to see the queue size. We use absolute paths to commands to reduce possible errors due to missing paths in $PATH. Please consider adjusting the paths for your system. By executing these commands, we can see the queue size:
root@server $ /usr/bin/mailq | /usr/bin/tail -n1 | /usr/bin/gawk '{print $5}'
root@server $
If our queue is empty, we get an empty result. Now let’s enqueue an email like we did above and see what the script will tell us:
root@server $ mail -s 'testing message subject' someone@gsmail.com <<< 'testing message body'
root@server $ /usr/bin/mailq | /usr/bin/tail -n1 | /usr/bin/gawk '{print $5}'
1
We enqueued an email, and now we can see “1” as a return value - great! Now we write a simple bash script to print out a message if the queue is bigger than 0:
#!/bin/bash
mailq_count="$(/usr/bin/mailq | /usr/bin/tail -n1 | /usr/bin/gawk '{print $5}')"
# If variable is empty, then the queue is empty -> set it to zero
if [ -z "$mailq_count" ]; then
mailq_count=0
fi
if [ "$mailq_count" -gt 0 ]; then
echo "Mail count on mail.myserver.local is ${mailq_count}" | /usr/bin/mail -s "mail.myserver.local queue size: ${mailq_count}" notify.me@somemail.local
fi
This script is pretty simple and should send an email if the mail queue is greater than 0. You can adjust the number for your needs. For low traffic servers, something between 10 and 20 should be the right choice.
Now you can adjust the script for your needs and use the commands above to put some messages in a queue and remove them for testing.
You can replace the notification mail line for simpler testing:
if [ "$mailq_count" -gt 0 ]; then
echo "Mail count on mail.myserver.local is ${mailq_count}"
fi
Try it for yourself and meet your expectations.
Running the script by a cron job
Since our script is working, we want to call it every minute and notify us if
the queue exceeds our preset value. We can do this through a cron job entry in a
new file like /etc/cron.d/mailqueue-watchdog:
*/1 * * * * root nice -n 19 /usr/local/sbin/mailqueue_size.sh
You can use the commands above and add or delete some emails for testing with a not existing domain, as I did with domain name gsmail.com.
Getting push notifications
Receiving emails to be notified if the queue is in trouble is not the best decision as an admin gets many emails; that’s why we want to use push notifications through a different service.
You can use Pushover or any other service for sending a copy of the queue size email as a push message to your favorite smartphone. We don’t want to trust another service in such a critical situation fully; that’s why we’ve set up to receive an email and a push notification too.
Before Pushover I used IFTTT to receive push notifications, which should be fine too.
Summary
Using a shell script, you can monitor your server queues and get notified by hitting a limit. Through a third-party service, you can get push notifications for this particular case.
Happy monitoring and, always small queue sizes - your MailMum Team!