{"id":1218,"date":"2011-01-14T23:30:09","date_gmt":"2011-01-14T22:30:09","guid":{"rendered":"http:\/\/t-machine.org\/?p=1218"},"modified":"2011-01-15T15:55:38","modified_gmt":"2011-01-15T14:55:38","slug":"howto-prevent-seo-scam-referrer-traffic-and-install-mod-security-on-debian","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2011\/01\/14\/howto-prevent-seo-scam-referrer-traffic-and-install-mod-security-on-debian\/","title":{"rendered":"HOWTO: Prevent SEO scam Referrer traffic &#8230; AND &#8230; Install Mod-Security on Debian"},"content":{"rendered":"<p>UPDATE: there were several bugs in my original version &#8211; by Debian standards, ModSecurity is damn hard to configure correctly, mainly because the Debian packager has left out so much that&#8217;s essential! This version is fully tested and working&#8230;<\/p>\n<p><a href=\"http:\/\/www.modsecurity.org\/\">Mod Security<\/a> is an awesome, open-source product for Apache that will protect your webserver against attackers, using a custom rules-language that lets you easily filter for any kind of website attack. Even better, it comes with a pre-built (and regularly updated) set of &#8220;official&#8221; default rules for cutting out the majority of common internet attacks.<\/p>\n<p>But, pretty shocking &#8230; I tried 10 different tutorials \/ HOWTO&#8217;s for this, and each one was wrong. Out of the 10, 6 of them lead to fundamentally insecure \/ misconfigured systems.<\/p>\n<p>Mostly it&#8217;s the vendor&#8217;s fault for providing huge long-winded webpages in place of basic install instructions. Partly, it&#8217;s the Debian packager&#8217;s fault for both mis-packaging, and also &#8220;forgetting&#8221; to document what they&#8217;d done (e.g most of the README&#8217;s are empty. Grr!). Whatever. Here&#8217;s my HOWTO for doing it correctly, and picking up the excellent default security rules, that *should* work with most installs of Debian.<br \/>\n<!--more--><br \/>\n(Also, for future reference, the vendor install docs are empty on their own website. But, if you click random links, you <a href=\"http:\/\/www.owasp.org\/index.php\/Category:OWASP_ModSecurity_Core_Rule_Set_Project#tab=Installation\">*eventually* find some of the missing pieces<\/a>)<\/p>\n<h4>Before you start<\/h4>\n<p>Debian packager screwed up, didn&#8217;t document this: the main package has no config files. If you install libapache-mod-security, it also installs mod-security-common (note: no apache mentioned), which contains most of the config for this thing (but still not all). Yeah. It&#8217;s a bit odd, but that&#8217;s how security-based packages sometimes work, so not that unusual.<\/p>\n<p>Still, the main package SHOULD contain the root config file. It doesn&#8217;t &#8211; tragically, you have to write that yourself<\/p>\n<h4>Partial install: debian packages<\/h4>\n<p>All you need is &#8220;libapache-mod-security&#8221;. This will also pull-down the other packages needed. So far as I can tell, this is *correctly* packaged by the maintainer &#8211; I got no errors for missing packages.<\/p>\n<h4>Remained of install: config files<\/h4>\n<p>The main config file is missing.<\/p>\n<p>Where do you want to put this? (debian packager: please note + fix this!)<\/p>\n<p>&#8230;Personally, I&#8217;d go for the debian standard location: \/etc\/apache2\/conf.d\/<\/p>\n<p>What do you call it?  (debian packager: please note + fix this!)<\/p>\n<p>&#8230;if you compile the vendor-suppled source-code, it&#8217;ll call the file &#8220;mod-security2.conf&#8221;, so let&#8217;s use that.<\/p>\n<p>Contents?<\/p>\n<blockquote><p>\n\t&lt;IfModule mod_security2.c&gt;<br \/>\n\t# NB: this is the minimal-possible example; see bottom of this post for a more complete example<\/p>\n<p>\t# NB: these are DEBIAN-SPECIFIC &#8211; only recommended if you used the official Debian package:<br \/>\n               Include \/usr\/share\/doc\/mod-security-common\/examples\/rules\/*.conf<br \/>\n               Include \/usr\/share\/doc\/mod-security-common\/examples\/rules\/base_rules\/*.conf<br \/>\n\tSecDebugLog \/var\/log\/modsec_debug.log<br \/>\n\tSecAuditLog \/var\/log\/modsec_audit.log<\/p>\n<p>       &lt;\/IfModule&gt;\n<\/p><\/blockquote>\n<p>Also, create empty versions of those log files (again, the Debian maintainer is supposed to do this!)<\/p>\n<ol>\n<li>touch \/var\/log\/modsec_debug.log\n<li>touch \/var\/log\/modsec_audit.log\n<li>chown www-data:www-data \/var\/log\/modsec*\n<\/ol>\n<p>After creating the main config file, don&#8217;t forget to do an apache:<\/p>\n<blockquote><p>\n\/etc\/init.d\/apache2 restart\n<\/p><\/blockquote>\n<p>to pick up the new config<\/p>\n<h4>Add the user-modified config files<\/h4>\n<p>Again, this is something that should have been done by the Debian packager &#8211; it appears to be necessary as part of the main config if you are following the &#8220;official&#8221; guidelines (which are optional).<\/p>\n<p>I accidentally discovered <a href=\"http:\/\/blog.modsecurity.org\/2007\/02\/handling-false.html\">this blog post<\/a>, and used that info to &#8220;infer&#8221; what these files should be called, and where they should be listed in the main config file.<\/p>\n<p>Create:<\/p>\n<ol>\n<li>\/etc\/modsecurity\/\n<li>\/etc\/modsecurity\/modsecurity_crs_15_customrules.conf<br \/>\n<LI>\/etc\/modsecurity\/modsecurity_crs_60_customrules.conf\n<\/ol>\n<p>Modify: \/etc\/apache.d\/conf.d\/mod-security2.conf<\/p>\n<ol>\n<li>Anywhere in that ifModule stanza, add:\n<ul>\n<li>Include \/etc\/modsecurity\/*.conf\n<\/ul>\n<\/ol>\n<h4>Finally: local configuration<\/h4>\n<p>Personally, I gave up. Any security system where customization is hard is doomed to failure. Instead, I replaced the &#8220;Include &#8230; *.conf&#8221; rules with a manual include of the named files I wanted.<\/p>\n<p>I created a new file in \/etc\/modsecurity &#8211; the only requirement *appears* to be that it ends in &#8220;.conf&#8221;, but to be safe I named it with the same convention as ModSecurity&#8217;s other files: &#8220;modsecurity_crs_60_my_anti_spam_rules.conf&#8221;.<\/p>\n<p>I put just onehand-made custom rule in there:<\/p>\n<blockquote><p>\n# To all those &#8220;SEO Marketers&#8221; who think they&#8217;re clever: leave me alone!<br \/>\n# ADAM: this is written to fast-fail: as soon as it skips past the &#8220;domain name&#8221; part of the referrer, it quits and lets the request through<br \/>\n# &#8230;but if it finds any of these SEO spammers, it&#8217;ll 500 them, with no explanation given<br \/>\n# (tested and confirmed working)<br \/>\nSecRule REQUEST_HEADERS:REFERER &#8220;http:\/\/[^\/]*(holdem|poker|casino|porn|seo|miley|seller|pokemon|replica|buy|health|shop|dental|exposed|sex|forex|pussy|realty|sale|pills|download|invest|hosting|videos|cheap)&#8221; &#8220;t:lowercase,deny,nolog,status:500&#8221;\n<\/p><\/blockquote>\n<p>(I got the idea for using ModSecurity for this from <a href=\"http:\/\/mediakey.dk\/~cc\/block-referer-spam\/\">this page<\/a>. I&#8217;d never thought of using MS for that &#8211; but thought it was a great idea. I scanned some of the local referer logs for common marketing \/ scam referrers, and used the most-commonly-occurring ones to write the custom rule you see above.<\/p>\n<h4>Testing your custom rules<\/h4>\n<p>wget is your friend.<\/p>\n<p>e.g.:<\/p>\n<blockquote><p>\nwget  http:\/\/your.website.com &#8211;referer=http:\/\/seo-bastards-must-die.com<br \/>\nwget  http:\/\/your.website.com &#8211;referer=http:\/\/google.com\n<\/p><\/blockquote>\n<p>For the first, you should see a result printed to screen saying &#8220;HTTP request sent, awaiting response&#8230; 500 Internal Server Error&#8221;<\/p>\n<p>For the second, you should see: &#8220;HTTP request sent, awaiting response&#8230; 200 OK&#8221;<\/p>\n<h4>Adding mod-security to logrotate<\/h4>\n<p>Again &#8230; the Debian package should have done this automatically, but tragically doesn&#8217;t.<\/p>\n<p>Create the file:<\/p>\n<blockquote><p>\n\/etc\/logrotate.d\/mod-security\n<\/p><\/blockquote>\n<p>And set the contents as:<\/p>\n<blockquote><p>\n\/var\/log\/modsec_debug.log \/var\/log\/modsec_audit.log {<br \/>\n\tdaily<br \/>\n\tmissingok<br \/>\n\trotate 10<br \/>\n\tcompress<br \/>\n\tdelaycompress<br \/>\n\tnotifempty<br \/>\n\tcreate 640 www-data adm<br \/>\n}\n<\/p><\/blockquote>\n<h4>PS: &#8230; for future reference, my personal mod-security2.conf<\/h4>\n<p>NB: this is the template I started with, then added lines after the &#8220;ADAM: disabling&#8230;&#8221; line<\/p>\n<blockquote><p>\n&lt;IfModule mod_security2.c&gt;<\/p>\n<p>        # NB: these are DEBIAN-SPECIFIC &#8211; only recommended if you used the official Debian package:<br \/>\n        # &#8230; this will include the official &#8220;normal&#8221; settings, including the one that enables the system!<br \/>\n        Include \/usr\/share\/doc\/mod-security-common\/examples\/rules\/*.conf<\/p>\n<p>        # ADAM: disabling; instead I&#8217;ll add on an individual basis, thanks!<br \/>\n        #Include \/usr\/share\/doc\/mod-security-common\/examples\/rules\/base_rules\/*.conf<\/p>\n<p>        # Add custom rules from this directory:<br \/>\n        Include \/etc\/modsecurity\/*.conf<\/p>\n<p>        # Tell it where to put logfiles &#8230; maybe should be in \/var\/log\/apache2\/* ?<br \/>\n        SecDebugLog \/var\/log\/modsec_debug.log<br \/>\n        SecAuditLog \/var\/log\/modsec_audit.log<\/p>\n<p>        # Tell it where to put the Data dir it uses for some long-term processing<br \/>\n        # (if you omit this, you&#8217;ll get your error log for Apache filling up VERY fast with complaints!)<br \/>\n        SecDataDir \/var\/run\/modsecurity<br \/>\n&lt;\/IfModule&gt;\n<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>UPDATE: there were several bugs in my original version &#8211; by Debian standards, ModSecurity is damn hard to configure correctly, mainly because the Debian packager has left out so much that&#8217;s essential! This version is fully tested and working&#8230; Mod Security is an awesome, open-source product for Apache that will protect your webserver against attackers, [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,45],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/1218"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=1218"}],"version-history":[{"count":0,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/1218\/revisions"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=1218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=1218"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=1218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}