RewriteRule of "^$" is broken

As reported the problem was solved upstream by Apache.

This is the bugreport :
https://bz.apache.org/bugzilla/show_bug.cgi?id=53929

Please use the provided patch:
https://bz.apache.org/bugzilla/attachment.cgi?bugid=53929&action=viewall

As this bug is really impacting the way mod_rewrite works (works incorrectly) this fix would be very much appreciated.

After upgrading a couple of servers to use Ubuntu 14.04 / Apache 2.4.7, we now have a lot of mod_rewrite problems. (a lot as in many websites).

Status changed to 'Confirmed' because the bug affects multiple users.

Thank you for taking the time to report this bug and helping to make Ubuntu better.

Can someone familiar with the issue please follow the steps documented at https://wiki.ubuntu.com/StableReleaseUpdates#Procedure? In particular the user impact, test case and regression potential sections. This is to ensure that we don't unnecessarily or accidentally regress existing users.

As this is fixed in 2.4.9, this presumably affects Trusty only, so marking Fix Released in Vivid and adding a Trusty task.

[Test Case]

Setup
Apache 2.4.7
* mod_rewrite
* mod_ajp
* mod_dir

Tomcat
* Listening on Port 9001

Apache with a .htaccess in the example.net VirtualHost

  RewriteEngine On
  RewriteRule ^(.*)$ ajp://localhost:9001/$1 [P]

Expected:
Return from Tomcat

  HTTP Status 404 - /

Reality:
Return from Tomcat

  HTTP Status 404 - /index.html

Workaround for this particular setup was to either disable mod_dir or disable DirectoryIndex in .htaccess.

Or on VirtualHost context use ProxyPass.

  ProxyPass / ajp://localhost:9001/
  ProxyPassReverse / ajp://localhost:9001/

[Impact]

With DirectoryIndex disabled:
<pre>
[Thu Apr 30 13:55:18.761066 2015] [rewrite:trace3] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] strip per-dir prefix: /home/www-data/example.net/ ->
[Thu Apr 30 13:55:18.761191 2015] [rewrite:trace3] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] applying pattern '^(.*)$' to uri ''
[Thu Apr 30 13:55:18.761215 2015] [rewrite:trace2] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] rewrite '' -> 'ajp://localhost:9001/'
[Thu Apr 30 13:55:18.761232 2015] [rewrite:trace2] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] escaped URI in per-dir context for proxy, ajp://localhost:9001/ -> ajp://localhost:9001/
[Thu Apr 30 13:55:18.761245 2015] [rewrite:trace2] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] forcing proxy-throughput with ajp://localhost:9001/
[Thu Apr 30 13:55:18.761259 2015] [rewrite:trace1] [pid 31422] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38052] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb50a0/initial] [perdir /home/www-data/example.net/] go-ahead with proxy request proxy:ajp://localhost:9001/ [OK]
</pre>

With DirectoryIndex enabled:
<pre>
[Thu Apr 30 13:58:37.954876 2015] [rewrite:trace3] [pid 31419] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38156] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb10a0/initial] [perdir /home/www-data/example.net/] strip per-dir prefix: /home/www-data/example.net/ ->
[Thu Apr 30 13:58:37.954930 2015] [rewrite:trace3] [pid 31419] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38156] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb10a0/initial] [perdir /home/www-data/example.net/] applying pattern '^(.*)$' to uri ''
[Thu Apr 30 13:58:37.954947 2015] [rewrite:trace2] [pid 31419] mod_rewrite.c(468): [client XXX.XXX.XXX.XXX:38156] XXX.XXX.XXX.XXX - - [example.net/sid#7fc6ddd849f8][rid#7fc6ddcb10a0/initial] [perdir /home/www-data/example.net/] rewrite '' -> 'ajp://localhost:9001/'
[Thu Apr 30 13:58:37.954959 ...

This bug is truly a pain, it really needs to be fixed.

The only acceptable workaround is not to use Apache < 2.4.9, which really means don't use Trusty (unfortunately it can't die off soon enough ... it's LTS and it's everywhere now).

This is real misery to support.

Attaching debdiff with patch applied.

 Updated Description for SRU proposal.

NACK on the debdiff. It doesn't use the actual fix that went into Apache 2.4. It uses a proposed patch from the bug that wasn't the way it was ultimately fixed.

Please prepare a new debdiff with the following commit:

https://github.com/apache/httpd/commit/f0529e54b8d889322b5113eb623e263556bfa28e

Thanks!

Replaced the debdiff with one that uses the fix from apache 2.4

https://github.com/apache/httpd/commit/f0529e54b8d889322b5113eb623e263556bfa28e

Thanks Wesley, this looks good.

Some minor changes please:

1) Please could you add an Origin: header to the debdiff? Something like "Origin: upstream, https://github.com/apache/httpd/commit/f0529e54b8d889322b5113eb623e263556bfa28e" or "Origin: backport, https://github.com/apache/httpd/commit/f0529e54b8d889322b5113eb623e263556bfa28e" depending on whether you had to tweak the diff or not. Sorry I didn't mention this on IRC before - I saw the Closes/LP thing on a quick glance but I didn't review properly.

2) Please could you note what's going on with the new DirectoryCheckHandler directive in the debian/changelog entry? In particular, I think it should answer: should users expect any behaviour changes when updating and not changing anything; and what users need to do to fix the bug after the update is applied (set DirectoryCheckHandler to something?)

3) Given that the directive is being added, it should probably be noted in the Regression Potential section, along with stating whether default behaviour is being changed or not, and we should probably add a test case to the Test Case section to make sure that the path that shouldn't change is also exercised to ensure that it indeed hasn't changed.

I think I'm asking for 2 and 3 really because I don't feel that I yet have clarity on exactly which way round the new directive works, and what expectations are with regard to behaviour changes on the SRU update (rather than against 2.2). I understand that parity against 2.2 is needed to fix the bug, but from an SRU and regression perspective it's behaviour changes on the SRU update itself that I'm bothered about. Essentially, it comes down to: how are we ensuring that no user will scream at us for breaking behaviour in pushing this SRU?

I'm sorry that this is a bit more complicated that I originally expected when I asked you to look at this because of the behaviour issue.

With these changes, assuming it builds and you've tested it then I'm happy to sponsor an upload. Thanks!

Log from IRC:

11:30 <rbasak> mdeslaur: around? I'm looking at sponsoring bug 1394403 - as you're looked at it before I'd like your opinion please.

11:30 <ubottu> bug 1394403 in apache2 (Ubuntu Trusty) "RewriteRule of "^$" is broken" [Medium,Confirmed] https://launchpad.net/bugs/1394403

11:31 <rbasak> when I asked magicalChicken to look at it I didn't realise the upstream fix would add a configuration directive. But it looks like it's safe as it defaults to the same behaviour. Had you considered this already? Does it also look reasonable to you?

11:31 <rbasak> I also think we should include the documentation update in our backport - better than not having it in the SRU IMHO.

12:10 <mdeslaur> rbasak: I'm ok with the new config option...I've added options before to packages as security updates, so it's not like we haven't done it before. The option will change the behaviour though, but cases where it will break something are unlikely

12:10 <mdeslaur> rbasak: for the documentation, meh...if it were man pages, I'd push for it...but the static web documentation, meh

12:10 <mdeslaur> rbasak: especially since there are localized versions of the documentation and we'd only be u...

Updated debdiff with origin information for the patch

The updated debdiff looks great. I particularly appreciate the very clearly written changelog that explains exactly what the deal is with the new directive, the behaviour change and the default. Uploaded - thank you!

SRU team: please note the behaviour change. We've concluded that it's OK (I'm taking into account both Wesley's and Marc's views here) but if you disagree there is always the option of adjusting the default for SRU purposes only. However, there is a documentation and user confusion cost to doing that by diverging from upstream on the default, so I've gone with what Wesley has submitted.

Awesome, thanks!

Hello Matthias, or anyone else affected,

Accepted apache2 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apache2/2.4.7-1ubuntu4.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Wesley, have you gotten a chance to test the package in trusty-proposed?

I can confirm that the package in trusty-proposed resolves the issue for me in a trusty vm with the default values set. I think it should be good to go into trusty-updates.

This bug was fixed in the package apache2 - 2.4.7-1ubuntu4.6

---------------
apache2 (2.4.7-1ubuntu4.6) trusty; urgency=medium

  * d/p/fix_rewrite_rule.patch: Add a configurable option to keep mod_dir from
    running when another handler is set. This makes default behavior
    consistant with 2.2, and fixes (LP: #1394403)
    - This adds the configuration option "DirectoryCheckHandler" which is
      present in apache 2.4.8 and later versions. The default value is
      "DirectoryCheckHandler Off".
    - This will change default behavior. Instead of mod_dir running even if
      other rules are being run, mod_dir will only run when no other rules
      have been processed by default. This is the expected behavior of
      mod_dir, and is consistant with the behavior of mod_dir in apache
      versions < 2.4 and > 2.4.8, and so the default value of this
      configuration option will correct the bug.
    - The current default behavior, which is considered to be a bug, can be
      kept by setting "DirectoryCheckHandler On".

 -- Wesley Wiedenmeier <email address hidden> Tue, 18 Aug 2015 09:36:21 -0500

The verification of the Stable Release Update for apache2 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Hi, apparently only part of the fix was included, http://svn.apache.org/viewvc?view=revision&revision=1557641 seems to be missing. That is the part that keeps mod_dir from firing when mod_rewrite just fired.

Rewrites in .htaccess are still affected after upgrade to 2.4.7-1ubuntu4.6.

Matthias

@Matthias

It would have been nice if you could have verified the proposed package as instructed in the bug, rather than waiting for the update to be released first!

Let's re-open the Trusty task on the basis of your comment.

Presumably the test case Wesley determined does not cover your failure case then? Please could you update it, and commit to testing any future proposed package yourself when we can next work on this bug to save us from wasting time again?

Actually Incomplete is probably more appropriate, since we don't have a reproducer right now.

To reproduce (s. attachment):

vhost bla.conf (requires a2enmod proxy_http)
    enables RewriteEngine/RewriteLogging and Proxy
   DocumentRoot /var/www/bla

/var/www/bla/.htaccess:
RewriteRule ^(.*)$ http://10.110.110.29/$1 [P]

no other files in /var/www/bla/

request / (as in attached reproduce-1394403.sh):
$ (printf "GET / HTTP/1.0\r\nHost: bla\r\n\r\n"; sleep 1) | netcat localhost 80

in the ErrorLog (/var/log/apache2/bla_error.log) you can see that after the first
  go-ahead with proxy request proxy:http://10.110.110.29/ [OK]
mod_rewrite is called a second time for the same request, but this time
with /index.html instead of just /
This is from mod_dir not honoring the final decision of mod_rewrite ([P]).

Matthias

Thanks for the test case, that works really well. Sorry that the next commit in upstream was not included in the SRU, I should have caught that.

At the version that's in the repos now I do see a change in behaviour in the log files between when 'DirectoryCheckHandler Off' is set and when it is set on.

With DirectoryCheckHandler On I get:
http://paste.ubuntu.com/12551255

And with it set to Off I get:
http://paste.ubuntu.com/12551264

So to me it looks like the issue is resolved, but I will prepare a new debdiff with the rest of the upstream fix tomorrow.

This bug was fixed in the package apache2 - 2.4.7-1ubuntu4.13

---------------
apache2 (2.4.7-1ubuntu4.13) trusty-security; urgency=medium

  * SECURITY UPDATE: proxy request header vulnerability (httpoxy)
    - debian/patches/CVE-2016-5387.patch: don't pass through HTTP_PROXY in
      server/util_script.c.
    - CVE-2016-5387
  * This update does _not_ contain the changes from (2.4.7-1ubuntu4.12) in
    trusty-proposed.

 -- Marc Deslauriers <email address hidden> Thu, 14 Jul 2016 08:40:55 -0400

This bug was autoclosed by mistake. The update that just got pushed to the -security pocket does _not_ contain the changes required to fix this issue. Re-opening.

Hi,

Sorry for the delay in getting the second part of the fix backported and about the confusion over which commits from upstream should be backported.

I have a updated debdiff that brings in the second part of the fix from upstream at:
http://svn.apache.org/viewvc?view=revision&revision=1557641

Note though, that the modification to include/ap_mmn.h from that revision was not included in the attached debdiff. The change to that file was only to increment the minor version number for the modules included in this build. Since it was a bump to minor number there is no major incompatibility. Since there have been many changes to upstream between the last minor increment in module version number here, it did not make sense to modify the version number as it had been modified in the upstream revision. If it turns out that it would be good to modify the minor version in another way, please let me know and I will make that change.

Before I had tested the version of apache2 currently in trusty-updates with the provided test case and it appeared to me that everything was working as expected. Once this package has built I will repeat this test and will see if I can update the testcase slightly to show a difference in behavior between apache with this fix and the version with the older fix if none is immediately obvious.

The package is building in a ppa at: ppa:wesley-wiedenmeier/test

If anyone would like to test with the most recent debdiff that would be appreciated.

Thanks

Thanks Wesley for resuming work in this.
I can confirm that your new debdiff aligns apache 2.4.7 to newer upstream releases.

My original "mod_rewrite->[P] inside .htaccess" issue remains only partly fixed. But that's not your fault or Ubuntus, it appears unfixed even in upstream.

With mod_rewrite->[P] rules in .htaccess, you need to set "DirectoryCheckHandler On" to keep mod_dir from interfering (even when "Options -Indexes" is set), whereas rules in vhost definitions just work.

This could be corrected by comparing the handler string against "proxy-server" in addition to just REDIRECT_HANDLER_NAME ("redirect-handler"); see attached debdiff, just a single line modified.

Alas, it seems my initial testing back then against newer apache version was broken. While I _thought_ I had found correct behaviour with mod_rewrite->proxy in .htaccess in 2.4.10 (Debian Testing at that time or pkgsrc), I don't get that now with 2.4.23 (Debian Testing). Even there I need to set "DirectoryCheckHandler On". Probably confused some xterms and compared with 2.2.x instead. Sorry for that.

With the modified debdiff, mod_rewrite->[P] in .htaccess works the same again as it does in vhost definitions.

Not sure if that fix is eligible for incorporation in a 14.04 release though, as it diverges from upstream (closer to ideal, but still diverging).

Regards
Matthias

In case you are still interested in this, here is a "blubb" vhost definition which contains the mod_rewrite->[P] rule (as opposed to having it in .htaccess as with the "bla" vhost).

The error log then will show termination after the first
  go-ahead with proxy request

whereas the bla host will fire some more lines with index.html initiated by mod_dir.

Regards
Matthias

Hi Matthias,

Thanks for helping figure out what was going on with the differing results in the test case.
Since the new debdiff contains the rest of the patch for the original issue I'll go ahead and continue the sru process for that debdiff.

Hopefully the second issue with rewrite directives will be fixed by upstream soon.
If you don't already have a bug open on the upstream bugtracker it would be good to open one, then hopefully we can get the fix into debian and ubuntu.

Since you have a patch for the .htaccess bug, upstream should be willing to accept it.

It probably wouldn't be possible to sru a patch before it is accepted into upstream, but once it is it should be fine to do.

Thanks

Upstream bug is fix released, and states that the fix is available in apache 2.4.9. Trusty has 2.4.7 atm.

Trusty ended standard support April 2019