[Shorewall-users] Tproxy + Squid + IPv6

[Sam]

Hi again.

Spent the last week getting my home network(s) online with IPv6. I think
I'm on the finishing stretch. One last issue has popped up that I am not
sure how to fix.

I'm running squid in transparent proxy mode via tproxy. Had it like that
for years on ipv4.

I've pretty much just followed the guide at the bottom here:
http://shorewall.org/Shorewall_Squid_Usage.html

I've attached a shorewall6 dump for good measure

Before I enabled the tproxy rules over ipv6 (and thus not using squid),
all tests on this site passed: http://test-ipv6.com/

Once I got the tproxy rules enabled, one test started failing.

That was the "Test IPv6 large packet" test. Your browser basically
fetches a url with 1600 characters in it. I shortened it and added it
here: http://preview.tinyurl.com/y9vy2j3u

I can fetch that url fine without squid and tproxy. But once it is
enabled, I can't. Looking at tcpdump, I see the request made goes out of
my wan nic, what comes back is an icmp "packet too big" response. That
icmp packet then flows back out (through shorewall) to the computer on
the lan that made the request. I'm thinking since squid intercepted the
HTTP request, that the icmp response should be going to squid. So I
don't know if this is just an issue of iptable rules or something else
at play here. Any thoughts? Googling for squid + mtu+ ipv6 + tproxy
doesn't give me too many results other than someone with the same issue
here (which never responds back with what the fix was):
http://squid-web-proxy-cache.1019090.n4.nabble.com/TPROXY-Timeouts-on-Select-Websites-td4657073.html


I've not found any websites that are proxied that don't work. Only issue
seems to be with the ipv6 test website. So perhaps I can ignore this...


Regards,
Samuel Smith

[Tom Eastep]

Post by Sam
Hi again.
Spent the last week getting my home network(s) online with IPv6. I think
I'm on the finishing stretch. One last issue has popped up that I am not
sure how to fix.
I'm running squid in transparent proxy mode via tproxy. Had it like that
for years on ipv4.
http://shorewall.org/Shorewall_Squid_Usage.html
I've attached a shorewall6 dump for good measure
Before I enabled the tproxy rules over ipv6 (and thus not using squid),
all tests on this site passed: http://test-ipv6.com/
Once I got the tproxy rules enabled, one test started failing.
That was the "Test IPv6 large packet" test. Your browser basically
fetches a url with 1600 characters in it. I shortened it and added it
here: http://preview.tinyurl.com/y9vy2j3u
I can fetch that url fine without squid and tproxy. But once it is
enabled, I can't. Looking at tcpdump, I see the request made goes out of
my wan nic, what comes back is an icmp "packet too big" response. That
icmp packet then flows back out (through shorewall) to the computer on
the lan that made the request. I'm thinking since squid intercepted the
HTTP request, that the icmp response should be going to squid. So I
don't know if this is just an issue of iptable rules or something else
at play here. Any thoughts? Googling for squid + mtu+ ipv6 + tproxy
doesn't give me too many results other than someone with the same issue
http://squid-web-proxy-cache.1019090.n4.nabble.com/TPROXY-Timeouts-on-Select-Websites-td4657073.html
I've not found any websites that are proxied that don't work. Only issue
seems to be with the ipv6 test website. So perhaps I can ignore this...

FWIW, my configuration also fails this test and I've noticed no problems.

-Tom

[Tom Eastep]

Post by Tom Eastep

Post by Sam
Hi again.
Spent the last week getting my home network(s) online with IPv6. I think
I'm on the finishing stretch. One last issue has popped up that I am not
sure how to fix.
I'm running squid in transparent proxy mode via tproxy. Had it like that
for years on ipv4.
http://shorewall.org/Shorewall_Squid_Usage.html
I've attached a shorewall6 dump for good measure
Before I enabled the tproxy rules over ipv6 (and thus not using squid),
all tests on this site passed: http://test-ipv6.com/
Once I got the tproxy rules enabled, one test started failing.
That was the "Test IPv6 large packet" test. Your browser basically
fetches a url with 1600 characters in it. I shortened it and added it
here: http://preview.tinyurl.com/y9vy2j3u
I can fetch that url fine without squid and tproxy. But once it is
enabled, I can't. Looking at tcpdump, I see the request made goes out of
my wan nic, what comes back is an icmp "packet too big" response. That
icmp packet then flows back out (through shorewall) to the computer on
the lan that made the request. I'm thinking since squid intercepted the
HTTP request, that the icmp response should be going to squid. So I
don't know if this is just an issue of iptable rules or something else
at play here. Any thoughts? Googling for squid + mtu+ ipv6 + tproxy
doesn't give me too many results other than someone with the same issue
http://squid-web-proxy-cache.1019090.n4.nabble.com/TPROXY-Timeouts-on-Select-Websites-td4657073.html
I've not found any websites that are proxied that don't work. Only issue
seems to be with the ipv6 test website. So perhaps I can ignore this...

FWIW, my configuration also fails this test and I've noticed no problems.

This is apparently a known limitation of interception caching -- see
http://wiki.squid-cache.org/SquidFaq/InterceptionProxy?highlight=%28PMTU%29.

-Tom

[Tuomo Soini]

I tested dual stack configuration (ipv4 and ipv6) with transparent
proxy some years ago and ended up decision:

Time of transparent proxy is long gone. Especially with dual stack
transparent proxy makes things a lot worse. There are quite a few sites
with ipv6 address so that web site doesn't actually work at all with
ipv6. With transparent proxy in place browsers can't fail back to to
ipv4 rendering all these sites unavailable.

Tom: I think this should be noted on documentation too.

Reason for the issue is browser creates tcp connection with proxy, not
with remote site so browser doesn't know tcp connection failed with
destination site - so ipv6 to ipv4 fallback can't work.

[Simon Hobson]

Post by Tuomo Soini
Reason for the issue is browser creates tcp connection with proxy, not
with remote site so browser doesn't know tcp connection failed with
destination site - so ipv6 to ipv4 fallback can't work.

I'm not sure that's still the case - Happy Eyeballs has been updated a bit over the years. AIUI, it doesn't attempt a connection and then fall back if it fails - it makes two connections (via 4 & 6) and waits to see which one gives an answer first.

[Tuomo Soini]

On Sun, 2 Jul 2017 12:14:14 +0100

Post by Simon Hobson

Post by Tuomo Soini
Reason for the issue is browser creates tcp connection with proxy,
not with remote site so browser doesn't know tcp connection failed
with destination site - so ipv6 to ipv4 fallback can't work.

I'm not sure that's still the case - Happy Eyeballs has been updated
a bit over the years. AIUI, it doesn't attempt a connection and then
fall back if it fails - it makes two connections (via 4 & 6) and
waits to see which one gives an answer first.

Exactly. That's why it is so bad idea to do transparent proxy. Both get
connect, ipv6 answers page not reachable and ipv4 gives real page - and
of course page not reachable from squid is faster response....

[Simon Hobson]

Post by Tuomo Soini

Post by Simon Hobson
I'm not sure that's still the case - Happy Eyeballs has been updated
a bit over the years. AIUI, it doesn't attempt a connection and then
fall back if it fails - it makes two connections (via 4 & 6) and
waits to see which one gives an answer first.

Exactly. That's why it is so bad idea to do transparent proxy. Both get
connect, ipv6 answers page not reachable

Ah, that's really bad - it should really just wait and time out the connection when it's connection upstream times out. Ah you sure there's no option for this ?

Post by Tuomo Soini
and ipv4 gives real page - and
of course page not reachable from squid is faster response....

[Sam]

Post by Tuomo Soini
I tested dual stack configuration (ipv4 and ipv6) with transparent
Time of transparent proxy is long gone. Especially with dual stack
transparent proxy makes things a lot worse. There are quite a few sites
with ipv6 address so that web site doesn't actually work at all with
ipv6. With transparent proxy in place browsers can't fail back to to
ipv4 rendering all these sites unavailable.

I have not noticed any issues yet. I've debated on ditching squid years
ago as everything was moving to https only. Squid can apparently break
through https now though, but I have not looked into it.

My biggest issue is I've been stuck on ATT dsl 6mbit internet for years
now. Thankfully I managed to get upgraded to 10mbit just a couple of
months ago but I doubt I'll ever get higher than that for many years. So
every bit of data that is fetched locally is a nice thing. I miss the
days when maps.google.com was http only. I used to cache all the images
to squid and that page would totally fly.

--Sam

[Simon Hobson]

This post might be inappropriate. Click to display it.

[Tom Eastep]

Post by Tuomo Soini
I tested dual stack configuration (ipv4 and ipv6) with transparent
Time of transparent proxy is long gone. Especially with dual stack
transparent proxy makes things a lot worse. There are quite a few sites
with ipv6 address so that web site doesn't actually work at all with
ipv6. With transparent proxy in place browsers can't fail back to to
ipv4 rendering all these sites unavailable.
Tom: I think this should be noted on documentation too.
Reason for the issue is browser creates tcp connection with proxy, not
with remote site so browser doesn't know tcp connection failed with
destination site - so ipv6 to ipv4 fallback can't work.

I've added a caution to the top of the Shorewall Squid document.

-Tom

[Ralf Schenk]

Hello,

I had the Tproxy + SQUID config for ipv4 and ipv6, too and can confirm
the problem with Path-MTU Discovery. That simple does'nt work. So ipv6
sites that need reduced MTU simply will hang. In my opinion thats a
problem in xt_TPROXY/kernel not squid which should act on Packet-Too-Big
ICMP Types for the Tproxy connection.

See:
http://www1.gr.squid-cache.org/mail-archive/squid-users/201210/0217.html

My only solution was to disable transparent proxy for ipv6.

Bye

Post by Tom Eastep

Post by Tuomo Soini
I tested dual stack configuration (ipv4 and ipv6) with transparent
Time of transparent proxy is long gone. Especially with dual stack
transparent proxy makes things a lot worse. There are quite a few sites
with ipv6 address so that web site doesn't actually work at all with
ipv6. With transparent proxy in place browsers can't fail back to to
ipv4 rendering all these sites unavailable.
Tom: I think this should be noted on documentation too.
Reason for the issue is browser creates tcp connection with proxy, not
with remote site so browser doesn't know tcp connection failed with
destination site - so ipv6 to ipv4 fallback can't work.

I've added a caution to the top of the Shorewall Squid document.
-Tom
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Shorewall-users mailing list
https://lists.sourceforge.net/lists/listinfo/shorewall-users

--
*Ralf Schenk*
fon +49 (0) 24 05 / 40 83 70
fax [phone]
mail ****@databay.de* <mailto:***@databay.de>

*Databay AG*
Jens-Otto-Krag-Straße 11
D-52146 Würselen
*www.databay.de* <http://www.databay.de>

Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202
Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm.
Philipp Hermanns
Aufsichtsratsvorsitzender: Wilhelm Dohmen

------------------------------------------------------------------------