Back in 2018 CloudFlare reported massive amplification attacks from UDP port 11211 on a scale never seen before. Zimbra quickly updated their wiki with How to Block Memcache Attack and if you haven’t taken the advice yet, you probably should!
2 years later we still see this kinds of attacks, aldo on much lower scale, but still enough to makes Zimbra administrators worried.
In this guide we’ll go over this DDOS attack and show you ways to protect your Zimbra Mail Server from being used in such attacks but also protect it from those same attacks.
What is an Amplification Attack?
An IP-spoofing capable attacker sends forged requests to a vulnerable UDP server. The UDP server, not knowing the request is forged, politely prepares the response. The problem happens when thousands of responses are delivered to an unsuspecting target host, overwhelming its resources – most typically the network itself.
Blocking Memcached Exploit
Detailed in this blog post, Zimbra Memcached may face “memcrashd” attack on port 11211. By-default Memcached listens on the server IP address which is accessible in the network and on the Internet if there is no firewall.
Zimbra Wiki
For Zimbra Single Server Installation
Configure memcached to listen on 127.0.0.1 only to avoid this attack. Use below commands.
su - zimbra /opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedBindAddress 127.0.0.1 /opt/zimbra/bin/zmprov ms `zmhostname` zimbraMemcachedClientServerList 127.0.0.1
Restart memcached:
zmmemcachedctl restart
For Zimbra Multi Server Installation
On zimbra multi server setup, workaround is to block traffic on port 11211 from Internet and allow only from zimbra proxy servers. First you need to enable/start iptables or ufw on the server. Make sure zimbra’s other ports are not blocked in the firewall.
Run below commands in the given sequence on ALL memcached servers.
Iptables rules for Redhat based servers
Drop all connections to port 11211.
iptables -I INPUT -p udp --dport 11211 -j DROP iptables -I INPUT -p tcp --dport 11211 -j DROP
Accept connections from localhost.
iptables -I INPUT -p udp -s 127.0.0.1 --dport 11211 -j ACCEPT iptables -I INPUT -p tcp -s 127.0.0.1 --dport 11211 -j ACCEPT
Accept connections from other proxy servers. Run below two commands for each proxy server IP in your zimbra setup.
iptables -I INPUT -p udp -s <Proxy IP> --dport 11211 -j ACCEPT iptables -I INPUT -p tcp -s <Proxy IP> --dport 11211 -j ACCEPT
UFW rules for Ubuntu servers
Drop all connections to port 11211.
ufw deny 11211
Accept connections from localhost.
ufw allow from 127.0.0.1 to any port 11211
Accept connections from other proxy servers. Run below two commands for each proxy server IP in your zimbra setup.
ufw allow from <Proxy1 IP> to any port 11211
Testing
UDP – Below command should give an empty response when executed from any host except proxy servers allowed above. If you see non-empty response that contains PID etc details, then your server is vulnerable.
echo -en "\x00\x00\x00\x00\x00\x01\x00\x00stats\r\n" | nc -q1 -u <IP of zimbra memcache server> 11211
[Above command may fail on redhat servers because nc does not recognize “-q” option. You can use “-w 1 ” instead.]
TCP – With below command you should not be able to connect when executed from any host except proxy servers allowed above. If you are able to connect, your server is vulnerable.
telnet <IP of zimbra memcache server> 11211
The attack description is taken from CloudFlare’s blog and the code snippets are taken from Zimbra’s Wiki page.
