Mailman and captcha

The libwebsockets.org mailing list signup page (at https://libwebsockets.org/mailman/listinfo/libwebsockets ) has been targeted by a botnet trying to use automated signups via google.

Nothing made it to the list, but the mail server is filled with doomed attempts to verify against the generated emails, eg

S'sqoonart+yjjzxqku@outlook.com'
S'sqoonart+wziudy@outlook.com'
S'sqoonart+clkqaj@outlook.com'
S'sqoonart+xxohsyye@outlook.com'
S'sqoonart+uvubjuwp@outlook.com'
S'sqoonart+ezpikxd@outlook.com'
S'sqoonart+eaiqnjs@outlook.com'
S'sqoonart+mvpboqse@outlook.com'
S'sqoonart+tiai@outlook.com'
S'sqoonart+ieamtf@outlook.com'
S'sqoonart+prdpbmp@outlook.com'

looking closer they’re being generated by signups from a wide range of IPs POSTing the mailman signup form with nonsense names and passwords.

This causes our server to make a lot of bad requests to the mail hosts (in good faith). So it seems we should enable a captcha on the signup page.

No captcha support in mailman

Mailman does not support captcha, despite botnets are scanning the net looking for mailman signup pages to spam. I guess Hyperkitty has taken over dev interest, but I am okay with mailman. Googling around found this page

https://www.dragonsreach.it/2014/05/03/adding-recaptcha-support-to-mailman/

Where the author has already suffered this problem in 2014 and he provides a somewhat corrupted patch and info on how to patch mailman… this is a bit painful since we are patching distro python that is subject to being overwritten by package upgrades. But since mailman itself doesn’t want to support captcha it is the only choice.

The rest of this post is about how to actually do that successfully, based on Andrea Veri’s original blog post.

Broken package for python-recaptcha-client

The first problem following those instructions is the dependent package python-recaptcha-client that it relies on cannot be recognized as something you can include from Python. In fact as pointed out at http://mailman.9.n7.nabble.com/Mailman-2-1-23-and-reCAPTCHA-td46468.html#a46474 you must perform:

 $ sudo touch /usr/lib/python2.7/site-packages/recaptcha/__init__.py

to provide the missing indication that the content is actually a python package at all; the Fedora package has the __init.py__ in a subdir, which causes python to ignore it.

Broken patch

The next problem is the patch has been mangled…. quoted items in angle-brackets have been snipped. This isn’t just an html rendering issue: they are missing in the page source on Andrea’s site. The fixed patch is here

--- listinfo.py 2017-08-12 04:12:29.953487616 +0200
+++ listinfo.py 2017-08-12 04:53:00.071483277 +0200
@@ -23,6 +23,7 @@
 import os
 import cgi
 import time
+import sys

 from Mailman import mm_cfg
 from Mailman import Utils
@@ -32,6 +33,8 @@
 from Mailman.htmlformat import *
 from Mailman.Logging.Syslog import syslog

+from recaptcha.client import captcha
+
 # Set up i18n
 _ = i18n._
 i18n.set_language(mm_cfg.DEFAULT_SERVER_LANGUAGE)
@@ -227,6 +230,7 @@
     replacements['<mm-displang-box>'] = displang
     replacements['<mm-lang-form-start>'] = mlist.FormatFormStart('listinfo')
     replacements['<mm-fullname-box>'] = mlist.FormatBox('fullname', size=30)
+    replacements['<mm-recaptcha-javascript>'] = captcha.displayhtml(mm_cfg.RECAPTCHA_PUBLIC_KEY, use_ssl=True)

     # Do the expansion.
     doc.AddItem(mlist.ParseTags('listinfo.html', replacements, lang))
--- subscribe.py    2017-08-12 04:14:44.143487376 +0200
+++ subscribe.py    2017-08-12 04:45:08.608484119 +0200
@@ -32,6 +32,9 @@
 from Mailman.UserDesc import UserDesc
 from Mailman.htmlformat import *
 from Mailman.Logging.Syslog import syslog
+from recaptcha.client import captcha
+
+

 SLASH = '/'
 ERRORSEP = '\n\n<p>'
@@ -122,6 +125,16 @@
              os.environ.get('HTTP_X_FORWARDED_FOR',
              os.environ.get('REMOTE_ADDR',
                             'unidentified origin')))
+
+    captcha_response = captcha.submit(
+        cgidata.getvalue('recaptcha_challenge_field', ""),
+        cgidata.getvalue('recaptcha_response_field', ""),
+        mm_cfg.RECAPTCHA_PRIVATE_KEY,
+        remote,
+        )
+    if not captcha_response.is_valid:
+        results.append(_('Invalid captcha'))
+
     # Are we checking the hidden data?
     if mm_cfg.SUBSCRIBE_FORM_SECRET:
         now = int(time.time())

Change dir to /usr/lib/mailman/Mailman/Cgi (for Fedora) before applying the patch.

This patch is correct against mailman-2.1.21.

You also need to modify the html and add your captcha keys to env vars in the mm_cfg.py as pointed out in the original article.

Troubleshooting

If problems are coming, at least on Fedora, although mailman puts out some scary “low level error” html, it also puts the details / backtrace down /var/log/mailman/error/.

Maintaining

Once it’s working, this is actually very fragile against updated mailman package from Fedora. In the abscence of a better idea I disabled updating mailman by creating an /etc/yum.conf containing

exclude=mailman

and keep an eye out when updating for mailman getting listed as excluded.