technology and zen of life

“A heisenbug (named after the Heisenberg Uncertainty Principle) is a computer bug that disappears or alters its characteristics when an attempt is made to study it.”

Exchange 2013 Disk Cleaning

After some troubles with mailflow due to Back Pressure on my relatively small virtual machine for Exchange, I had to try somewhat harder to reclaim disk space inside the Exchange VM. After the steps described below, I freed nearly 10 GB.

When you use Exchange inside a Virtual Machine, you might want to keep the size of the VM under control, so you can backup the VM with more ease.

Manual directory check

Manually empty these directories:

  • C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Logging\\Diagnostics\\DailyPerformanceLogs
    Ref.: Richard SchwendimanExchange 2013 Logging and Space Requirements, 2013
  • C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Logging\\lodctr_backups
    Ref.: Tony RedmondFreeing up disk space after Exchange 2010 installations, 2010
  • C:\\Program Files\\Microsoft\\Exchange Server\\V15\\TransportRoles\\Logs
  • C:\\Windows\\Temp
  • C:\\inetpub\\logs\\LogFiles

Limit transaction log size

  • Use backup software that clears the transaction logs.
  • Use other backup methods and enable circular logging.

Clean the WinSXS folder

The WinSXS folder holds often more than is needed.
Ref.: Lefteris KarafilisWinSXS cleanup in Windows 2012 and Windows 8, 2013

Get-WindowsFeature | where-object{$_.Installed -eq 0 -and $_.InstallState -eq 'Available'} | uninstall-windowsfeature -remove

dism /online /cleanup-image /SPSuperseded
dism /online /cleanup-image /StartComponentCleanup
dism /online /cleanup-image /StartComponentCleanup /ResetBase

Use specific tools

Exchange Back Pressure external SMTP test and warning

Your Exchange server will stop receiving email messages when the disk utilization is high. This is called Back Pressure. There are various ways to prevent such a dreadful situation, such as enabling circular logging and monitoring for lack of server resources, but you might want a warning message when this happens. It makes no sense to buy 1-processor servers. It is better to choose a configuration with two processors, and keep in mind that buying a server with the simplest Xeon, you will put two CPU cooler. The relatively-recent Intel Xeon E3-1200 v6 processor should be powerful enough for small and medium enterprises, get Xeon Server Buyers Guide can handle data-intensive processes and maintain the speed that Intel is known for.

Below you find a Python script. Run it periodically, e.g. using cron on Linux, and you are set. Don’t forget to change the variables for your own email address and servers.


# Tests an Exchange transport server for Back Pressure
# using SMTP probe connections.
# Run (periodically) from another server.
# This script will also send a warning email message,
# using another mailserver.
# I suggest you set an Outlook rule for automatic deletion
# of the test messages when you run this script periodically.

# Evert Mouw <>
# 2014-01-04

# variables used for the target that will be tested
FROM="" # should be from AN EXTERNAL DOMAIN to the Exchange server
SUBJECT="test: back pressure"
BODY="This is a test message. Please ignore."
DEBUG=0 # either 0 (no SMTP debugging) or 1

# variables used to send yourself a warning message
WARNSUBJECT="Back Pressure on Exchange server " + SERVER
WARNBODY="Exchange server " + SERVER + " is experiencing Back Pressure and cannot receive (external) emails any longer."

import smtplib

	print ("You are a moron.")

MSG="""To: %s
From: %s
Subject: %s


WARNMSG="""To: %s
From: %s
Subject: %s


if (PORT==465):
	smtp = smtplib.SMTP_SSL(SERVER, PORT)
	smtp = smtplib.SMTP(SERVER)

	smtp.sendmail (FROM, TO, MSG)
except smtplib.SMTPException, error:
	if "Insufficient system resources" in str(error):
		print (WARNBODY)
		if (WARNPORT==465):
			warnsmtp = smtplib.SMTP_SSL(WARNSERVER, WARNPORT)
			warnsmtp = smtplib.SMTP(WARNSERVER)
		warnsmtp.sendmail (WARNFROM, WARNTO, WARNMSG)
	print (error)
	#print (repr(error))

Installation notes for Postfix, SASL and Unbound on RHEL 6

Today I finished installing and configuring a filtering mailserver that does virus- and spamfiltering. In fact, it only does filtering, and then passes the mail on to internal mail servers.

I hit a few hurdles and these installation notes are meant for my own future reference. I publish them on this weblog so others might benefit as well.


A Scientific Linux (RHEL clone) 6.4 64-bit installation, virtualized inside a KVM host. I use macvtap/macvlan networking and virtio drivers. The EPEL repository is activated.

I used the Spamassassin Ultimate Setup Guide written by Warren Togami to get most of the system up.

Fixing SASL support for Postfix

My SASL-Postfix configuration

As the mailserver is mainly a spam filter and frontend SMTP receiver, I do not run Dovecot or other IMAP servers on the box. So I opted for the Cyrus SASL components.

Furthermore I use the sasldb plugin to have more secure password mechanisms over the line (I don’t care about the unencrypted storage as my SMTP password differ from user passwords). Manage SMTP users with saslpasswd2 (db stored in /etc/sasldb2).

Install SASL support:

yum install cyrus-sasl*

cat /etc/sasl2/smtpd.conf

pwcheck_method: auxprop
auxprop_plugin: sasldb

cat /etc/postfix/ | grep sasl

smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes
smtpd_sasl_security_options = noanonymous, noplaintext

Postfix error: no SASL authentication mechanisms

less /var/log/messages

filter postfix/smtpd[12154]: fatal: no SASL authentication mechanisms

Install all “mechanisms”:

yum install cyrus-sasl*

Check mechanism files using:

ls /usr/lib64/sasl2/

Postfix error: sql_select option missing

less /var/log/messages

Oct 17 12:41:35 filter postfix/smtpd[3852]: sql_select option missing
Oct 17 12:41:35 filter postfix/smtpd[3852]: auxpropfunc error no mechanism available
Oct 17 12:41:35 filter postfix/smtpd[3852]: auxpropfunc error invalid parameter supplied

According to CarbonCopy, “I found this on another site and it worked fine”

The workaround for the above problem is actually quite simple, really. You simply remove all the libsql* files in /usr/local/lib/sasl2. That’s it. Personally, I moved them all into a folder I created in that directory on the off chance that I may need them in the future. Obviously, this is going to wreak havoc if you are using libsasl for more than one app’s authentication and the other one uses sql. But, if you are using it as I am, where I’m only using sasl for smtp auth through an alternate mechanism (authdaemond in my case), then this will end the spamming of your log file.

Seems indeed to be a problem, as I have cyrus-sasl-sql installed.

yum list installed | grep cyrus-sasl

So remove it with:

yum remove cyrus-sasl-sql

And the error message is gone. (Check by restarting postfix and checking /var/log/messages).

Unbound SELINUX problem with root.anchor

A filtering mailserver will do lots of DNS lookups, because of the large amount of spammers who use all kind of (bogus) host names, and because of DNS based blacklists. You want to cache DNS requests so your mail flow will not me needlessly delayed by nameserver requests.

Of course a spam- and virusfiltering server needs to be tought. Do not disable SELINUX. If you don’t know selinux, watch this great video: SELinux For Mere Mortals, by Thomas Cameron, Red Hat Summit 2012.

I choose Unbound because of it’s reputation of being fast, modern, and secure. And it is developed in The Netherlands

Installation of Unbound

yum install unbound (EHEL repo)

That installs these packages:

  • ldns-1.6.16-2.el6.x8664.rpm – libevent-1.4.13-4.el6.x8664.rpm – unbound-1.4.21-1.el6.x86_64.rpm
  • unbound-libs-1.4.21-1.el6.x86_64.rpm

Error reading root.anchor

But the unbound service did not start successfully. There was a selinux problem accourding to /var/log/messages:

Oct 17 12:57:42 filter unbound: [4004:0] error: unable to open /var/lib/unbound/root.anchor for reading: Permission denied
Oct 17 12:57:42 filter unbound: [4004:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.anchor
Oct 17 12:57:42 filter unbound: [4004:0] error: validator: error in trustanchors config
Oct 17 12:57:42 filter unbound: [4004:0] error: validator: could not apply configuration settings.
Oct 17 12:57:42 filter unbound: [4004:0] error: module init for module validator failed
Oct 17 12:57:42 filter unbound: [4004:0] fatal error: failed to setup modules

This specific error might be somewhat related to other selinux packaging bugs, for examble bug 963067 and bug 896599.

Fix using audit2allow

Googling for “audit2allow el6? tells me it’s in policycoreutils-python, so let’s install that:

yum install policycoreutils-python

And this is how we are going to fix it by creating a selinux policy:

grep unbound /var/log/audit/audit.log | grep root | audit2allow -M unbound-fix-anchor
semodule -i unbound-fix-anchor.pp

(Note that I first did a “grep anchor” but that didn’t do the trick, so then I tried a “grep root”.)

And yes it works

Configure Unbound for a mailserver

I love a large DNS cache to speed up mail delivery and lower network traffic from my mailserver. Luckely, there is an Unbound optimization guide. Add the text below to /etc/unbound/unbound.conf

    # more cache memory, rrset=msg*2
    # Due to malloc overhead, the total memory usage is likely
    # to rise to double (or 2.5x) the total cache memory!!
    rrset-cache-size: 100m
    msg-cache-size: 50m

Furthermore, you might want to give internal mailservers the local IP addresses they have. Add to /etc/unbound/local.d/internal-mailservers.conf

local-data: " IN A"
local-data: "bifrost. IN A"

Configure Postfix to use the local DNS caching nameserver

Of course, don’t forget to make sure your mailserver uses the DNS caching nameserver. There are various methods to do so:

  • edit /etc/resolv.conf (if you don’t use Network Manager), set nameserver
  • configure using Network Manager
  • edit /etc/sysconfig/network-scripts/ifcfg-eth0, set BOOTPROTO=dhcp and PEERDNS=no (if PEERDNS is disabled, then /etc/resolv.conf will not be modified when this interface gets its DHCP address)
  • when using DHCP, set a custom DHCP reservation with DNS server for the mailserver

nofail and nobootwait mount options in fstab prevent boot problems

Problem description: You (re)boot your computer, e.g. a headless server. The boot process can halt or exit to rescue mode when an external disk is unavailable or when an internal but non-critical disk is out of order.

  • mountall tries to automount all entries from fstab that have the defaults or auto mount options. It will halt the boot process if such entries cannot be mounted, except when the mount option nobootwait is given.
  • fsck tries to do a filesystem check on all entries from fstab that have the sixth field set to 1 or 2. Non-critical drives typically have this field set to 2. It will halt the boot process if such filesystems cannot be checked, except when the mount option nofail is given.

A possible disadvantage of nobootwait is that, if the disk is actually present, it will do the filecheck in the background and continue booting. That might be exactly what you want for e.g. external drives, but if other services depend on the drive, then those services can fail.

Another disadvantage of nobootwait is the fact that it is not supported by all Linux distributions. For example, using Scientific Linux (RHEL) 6.4, I cannot mount a partition that has the nobootwait set in fstab.

According to the fsck man page, this is what nofail does:

fsck normally does not check whether the device actually exists before calling a file system specific checker. Therefore non-existing devices may cause the system to enter file system repair mode during boot if the filesystem specific checker returns a fatal error. The /etc/fstab mount option nofail may be used to have fsck skip non-existing devices. fsck also skips non-existing devices that have the special file system type auto

Ubuntu has a bug listed: “mountall ignores nofail mount option”. That makes sense, as mountall has not implemented nofail. It uses nobootwait. The bug reporters argue that both mountall and fsck should make use of the nofail option, and I agree with them.

So, if you want to be on the safe side, you

  • should use both nobootwait and nofail.
  • should test this with a manual umount followed by a mount.
  • should have no services depending on non-critial disks.

Below an example from my own SE 6.4 server using only the nofail option, because mounting with the nobootwait option present gives an error.

LABEL=Series  /mnt/filer/Series  xfs  auto,nofail,nodev,noexec,nouser,noatime  0  2


Enable Internet Calendar Publishing for Exchange 2013

If you want to share your Exchange 2013 calendar, or export your calendar to an iCal ICS file, read on. This is what I did on my calendaring server named Verdandi.

In 2011, I wrote in the article “Calendaring: Exchange can publish to ICS, defaults to insecure” on this website:

ICS (iCal) is a file format for calendars. Most calendaring software uses it. For example, you can import / export it to Google Calendar. If you have a Google Calendar, you can “publish” your calendar to an ICS file, so you can use it with other software like Mozilla Calendar or web services. Read the rest of this entry »

How to add a custom filetype to Linux


In this example, we will add support for MHTML / MHT files.

Note that the MIME type for MHTML is not well agreed upon. Used MIME types include:

  • multipart/related
  • application/x-mimearchive
  • message/rfc822

According to a StackOverflow question, message/rfc822 should be used. A recent discussion on FreeDesktop, however, recommends application/x-mimearchive as a subtype of multipart/related. I will use message/rfc822 in the examples below and have added this unclarity to the Wikipedia article. Read the rest of this entry »

Get SQL Agent service starting

Today I was struggling with Microsoft’s SQL Server. I could not get the SQL Agent service starting. Maybe my experiences can help you.


Sometimes the SQL Agent service will not start, and it is quite hard to get the cause of this when the Error log doesn’t show much. The hints below were gathered from various websites and online forums, and some of them were tested with a Microsoft SQL Server 2008 R2 stand-alone instance running on Windows 7.

General testing

To get some idea of what is wrong, do this in a command shell: Read the rest of this entry »

Installing Haiku directly to a disk partition


Haiku is something like Windows or Linux: an Operating System (OS). Some geeks like to play with alternative operating systems; it you are one of such geeks, you might want to give Haiku a try. Haiku is very fast and easy to use, but currently there are not many applications you can use with Haiku.

Haiku is a new open-source operating system that specifically targets personal computing. Inspired by the BeOS, Haiku is fast, simple to use, easy to learn and yet very powerful.

In this post, I will describe how to install Haiku on a spare harddisk partition directly without using a CD-ROM or USB memory key. I furthermore assume that you already installed Linux on another partition. Read the rest of this entry »


The views expressed on this blog are personal. We do not claim to be a representative voice of the views of any organisation whatsoever. We are not responsible for the content present on the blogs to which we have linked.Views expressed are solely that of the author and does not reflect a collective opinion of contributors.