Otdfbt

I recently set up a rewrite to serve .webp files in place of .jpg files whenever possible. It seemed to be working correctly until today we noticed that a newly uploaded .jpg image was returning a 404 even though the image was present. I did some checking and found that the server was rewriting the .jpg URL to .webp, and then getting a 404 because the .webp hadn't been created yet.

I then moved the rule from vhost_ssl.conf over to .htaccess and did a bunch of tinkering until I found something that worked, then moved that solution back to vhost_ssl.conf. But I still don't understand why the original rewrite sometimes failed. Can anyone review and shed some light on the subject?

Original rewrite:

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %DOCUMENT_ROOT/%1.webp -f
RewriteRule (.+).(?:jpe?g|png)$ $1.webp [NC,T=image/webp,E=webp,L]

Final rewrite:

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %REQUEST_URI (.+).(?:jpe?g|png)$
RewriteCond %DOCUMENT_ROOT/%1.webp -f
RewriteRule .* %1.webp [NC,T=image/webp,E=webp,L]

The only real difference between the two methods is that in the new method I check if the request URI matches jpeg/png before the rewrite condition that checks if the file exists. I think the new method is less efficient since the rewrite rule matches everything, but at least the condition above filters it, and it's in vhost_ssl.conf which is better than .htaccess.

Some other notes:

• During testing, I changed the original rewrite to a 301 redirect so I could see the file it thought it found. Sure enough, the file was the same as the .jpg file, just with a .webp extension - correct path and all (even though the .webp didn't really exist).


• Also during testing, I tried changing the rewrite condition to RewriteCond %DOCUMENT_ROOT/%1.webp_asdfasdfasf -fjust to verify that it would in fact fail, and it did. So it knew that the completely bogus file didn't exist, but it still thought the .webp file did exist.

tl;dr

RewriteCond %DOCUMENT_ROOT/%1.webp -f

It should be $1, not %1 in the RewriteCond TestString. Although it's unclear why this behaviour should have only recently changed (or maybe it hasn't?).

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %DOCUMENT_ROOT/%1.webp -f
RewriteRule (.+).(?:jpe?g|png)$ $1.webp [NC,T=image/webp,E=webp,L]

This should never have worked, unless you had a file called ".webp" (with no basename, starting with a dot) in the document root. If you did, then it would have always rewritten the jpeg/png URL to <whatever>.webp, whether it existed or not.

So, this would seem to suggest you do have a file called .webp (no basename) in the document root? (And/or you are seeing a cached redirect for some requests? See below...)

The %1 backreference in the 2nd RewriteCond TestString should be $1, not %1. %1 is a backreference to the first captured group in the last matched CondPattern - of which there is none, so %1 is always empty. So, it will always test the existence of /path/to/document-root/.webp. $1 on the other hand is a backreference to the first captured group in the RewriteRule pattern, ie. the filesystem path/basename, excluding the file extension.

You have used $1 in the RewriteRule substitution - you should have used the same in the preceding condition.

Note, that if these directives are in a Virtual Host context then you should either... remove the slash in the RewriteCond TestString, for example:

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %DOCUMENT_ROOT$1.webp -f
RewriteRule (.+).(?:jpe?g|png)$ $1.webp [NC,T=image/webp,E=webp,L]

OR, exclude the slash in the RewriteRule captured group. For example:

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %DOCUMENT_ROOT/$1.webp -f
RewriteRule ^/(.+).(?:jpe?g|png)$ /$1.webp [NC,T=image/webp,E=webp,L]

Otherwise, you'll end up with a double slash in the TestString that the OS will need to resolve.

RewriteCond %HTTP_ACCEPT image/webp
RewriteCond %REQUEST_URI (.+).(?:jpe?g|png)$
RewriteCond %DOCUMENT_ROOT/%1.webp -f
RewriteRule .* %1.webp [NC,T=image/webp,E=webp,L]

This achieves the same as simply using $1 in the original rule block (as mentioned above). Curious that you changed the backreference in the substitution to %1, but knew to leave the backreference in the condition unchanged (or was this simply overlooked)?

During testing, I changed the original rewrite to a 301 redirect ...

Always test with 302 (temporary) redirects. 301s are cached persistently by the browser, so can cause caching issues later. (In fact, this can account for "intermittent" behaviour when the request hits the browser cache.)