apache-fakegooglebot jail bans real Google bots

Bug #1708197 reported by Mikkel Kirkgaard Nielsen
24
This bug affects 5 people
Affects Status Importance Assigned to Milestone
fail2ban (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

On a mostly fresh Ubuntu 16.04 server with fail2ban the apache-fakegooglebot jail falsely detect accesses by authentic google bots as a fakes. This is because the ignore script at /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot depends on fail2ban python code installed only for python3 but the shebang dictates it being run using the default python at /usr/bin/python which is 2.7.

$ /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot 66.249.69.54
Traceback (most recent call last):
  File "/etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot.distrib", line 32, in <module>
    is_googlebot(process_args(sys.argv))
  File "/etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot.distrib", line 17, in process_args
    from fail2ban.server.filter import DNSUtils
ImportError: No module named fail2ban.server.filter

Forcing python3 gives the expected result (66.249.69.54 is a google webcrawler and should be ignored):
$ /usr/bin/python3 /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot 66.249.69.54 && echo $?
0

Explicitly calling python3 in the shebang should solve it (distributed script backed up in .distrib):
$ diff /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot{.distrib,}
1c1
< #!/usr/bin/python
---
> #!/usr/bin/python3

A more correct fix detecting the python version used by fail2ban was committed upstream almost a year ago (apache-fakegooglebot seems to be the only use case); https://github.com/fail2ban/fail2ban/issues/1506

Error in /var/log/fail2ban.log
2017-08-02 13:50:16,010 fail2ban.action [1565]: ERROR /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot 66.249.69.54 -- stdout: b''
2017-08-02 13:50:16,010 fail2ban.action [1565]: ERROR /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot 66.249.69.54 -- stderr: b'Traceback (most recent call last):\n File "/etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot", line 32, in <module>\n is_googlebot(process_args(sys.argv))\n File "/etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot", line 17, in process_args\n from fail2ban.server.filter import DNSUtils\nImportError: No module named fail2ban.server.filter\n'
2017-08-02 13:50:16,010 fail2ban.action [1565]: ERROR /etc/fail2ban/filter.d/ignorecommands/apache-fakegooglebot 66.249.69.54 -- returned 1
2017-08-02 13:50:16,010 fail2ban.filter [1565]: INFO [apache-fakegooglebot] Found 66.249.69.54
2017-08-02 13:50:16,770 fail2ban.actions [1565]: NOTICE [apache-fakegooglebot] Ban 66.249.69.54

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

$ apt-cache policy fail2ban
fail2ban:
  Installed: 0.9.3-1
  Candidate: 0.9.3-1
  Version table:
 *** 0.9.3-1 500
        500 http://dk.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        500 http://dk.archive.ubuntu.com/ubuntu xenial/universe i386 Packages
        100 /var/lib/dpkg/status

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in fail2ban (Ubuntu):
status: New → Confirmed
Revision history for this message
Anthony Geoghegan (anthony-geoghegan) wrote :

Confirming this bug also affects users of Ubuntu 16.04.4 LTS - which has the same version of fail2ban (0.9.3-1).

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.