Discussion:
[Shorewall-users] DNAT Problem
Jørn Eriksen
2016-12-29 16:18:01 UTC
Permalink
Hello there,

Got Leaf Bering uClibc with Shorewall 5.0.12.1

Compile go OK however when Shorewall do iptables restore I get the
following message
iptables-restore: line 41 failed
ERROR: iptables-restore Failed. Input is in
/var/lib/.iptables-restore-input

shorewall restart debug give this:
iptables: No chain/target/match by that name.
ERROR: Command "/sbin/iptables --wait -t nat -A net_dnat -p 6
--dport 2020 -j DNAT --to-destination 172.16.176.39:22" Failed

The rule I've added is this:
DNAT net loc:172.16.176.39:22 tcp 2020

Anyone got an idea as to why this is happening?

Jørn
Tom Eastep
2016-12-29 21:59:04 UTC
Permalink
Post by Jørn Eriksen
Hello there,
Got Leaf Bering uClibc with Shorewall 5.0.12.1
Compile go OK however when Shorewall do iptables restore I get the
iptables-restore Failed. Input is in
/var/lib/.iptables-restore-input
shorewall restart debug give this: iptables: No chain/target/match
by that name. ERROR: Command "/sbin/iptables --wait -t nat -A
net_dnat -p 6 --dport 2020 -j DNAT --to-destination
172.16.176.39:22" Failed
The rule I've added is this: DNAT net loc:172.16.176.39:22
tcp 2020
Anyone got an idea as to why this is happening?
Have you posted on the Bering uClibc mailing list? There seems to be a
problem with module loading in the latest release of Bering.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
j***@jorneriksen.com
2016-12-30 07:52:11 UTC
Permalink
Post by Tom Eastep
Have you posted on the Bering uClibc mailing list? There seems to be a
problem with module loading in the latest release of Bering.
Not yet - however I do know how to load modules but I'm not a kernel wiz,
so a pointer to a module name would be appreciated. I've checked the
obvious "nat" named ones and "conntrac" ones.

J
Post by Tom Eastep
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Post by Jørn Eriksen
Hello there,
Got Leaf Bering uClibc with Shorewall 5.0.12.1
Compile go OK however when Shorewall do iptables restore I get the
iptables-restore Failed. Input is in
/var/lib/.iptables-restore-input
shorewall restart debug give this: iptables: No chain/target/match
by that name. ERROR: Command "/sbin/iptables --wait -t nat -A
net_dnat -p 6 --dport 2020 -j DNAT --to-destination
172.16.176.39:22" Failed
The rule I've added is this: DNAT net loc:172.16.176.39:22
tcp 2020
Anyone got an idea as to why this is happening?
Have you posted on the Bering uClibc mailing list? There seems to be a
problem with module loading in the latest release of Bering.
- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJYZYcnAAoJEJbms/JCOk0QickP/3jN90Ds6akG8JtRcKJGUvwl
I79Y3Ht+wdDcsXP2vZGUjFS1SxSWTCQS5x+f62GW2MaNFs0KKKPxL4P16fS4aL3I
INVj1zOz/tj0qJDdySYwkp1RqDD7IZ41WEDCZvKUb9WK0bM1mhHsH25w24R+vK+b
gg+Kq6J9lL03GDZBX6dqD0GWvWXD6gOzv/vFfAfFSIa+uLX5Wffq+SGpDLqqkS3q
fxGReZy6qOFSDAQ6UpJZ3IeZmz88HaBi+46wdVrokTsd9nziBTk4Eei9NZaXtnS2
51VPrS5Pkr/qqnonGzqnAZXehEPU34dI+P4rOTV1fLI9tcot+QScPDqPj6kkjtDk
NXB2fLj+IZFX7mDdHER/QdpbvXvw4vJXEPDCxFgRy4LhTvQgYZ7tVw1XLKonEXW0
fDGlgavtTIRDvHilZ+Z1iJFCkPRLvvkjYRdiDd1C+IUBeN8s9d/eE2Fa+67NXNlv
PT+TyOp4GQSnpovHhJmJz0vV0b23LpdEYwiqWWY3EV0/2Jpp/nhMJqQ52g8tmWhq
pG8e/61Xb0FIPINkd+tSMEdUsAir1JFwmoW9HOQg7JQu4PHnf+CTdtfJA3ECOMW9
KEGRsf3w7i6JP2UV37+tM9XRt+1p7i3T3Cdl65jPFZrdw5YAFCJaqsVMecWhYPWa
C25A5933MwZgGP4+vbIo
=7ln6
-----END PGP SIGNATURE-----
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Shorewall-users mailing list
https://lists.sourceforge.net/lists/listinfo/shorewall-users
Tom Eastep
2016-12-30 16:54:55 UTC
Permalink
Post by j***@jorneriksen.com
Post by Tom Eastep
Have you posted on the Bering uClibc mailing list? There seems to
be a problem with module loading in the latest release of
Bering.
Not yet - however I do know how to load modules but I'm not a
kernel wiz, so a pointer to a module name would be appreciated.
I've checked the obvious "nat" named ones and "conntrac" ones.
That should be all that is needed.

After this failure, what is the output of:

fgrep net_dnat /var/lib/.iptables-restore-input

Thanks,
- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Tom Eastep
2016-12-30 17:01:13 UTC
Permalink
Post by Tom Eastep
Post by j***@jorneriksen.com
Post by Tom Eastep
Have you posted on the Bering uClibc mailing list? There seems
to be a problem with module loading in the latest release of
Bering.
Not yet - however I do know how to load modules but I'm not a
kernel wiz, so a pointer to a module name would be appreciated.
I've checked the obvious "nat" named ones and "conntrac" ones.
That should be all that is needed.
fgrep net_dnat /var/lib/.iptables-restore-input
And be sure that the xt_nat module is loaded.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Tom Eastep
2016-12-30 17:05:29 UTC
Permalink
Post by Tom Eastep
Post by Tom Eastep
Post by j***@jorneriksen.com
Post by Tom Eastep
Have you posted on the Bering uClibc mailing list? There
seems to be a problem with module loading in the latest
release of Bering.
Not yet - however I do know how to load modules but I'm not a
kernel wiz, so a pointer to a module name would be appreciated.
I've checked the obvious "nat" named ones and "conntrac"
ones.
That should be all that is needed.
fgrep net_dnat /var/lib/.iptables-restore-input
And be sure that the xt_nat module is loaded.
And nf_nat and nf_nat_ipv4

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Erich Titl
2017-01-02 20:12:05 UTC
Permalink
Hi Jorn
Post by j***@jorneriksen.com
Post by Tom Eastep
Have you posted on the Bering uClibc mailing list? There seems to be a
problem with module loading in the latest release of Bering.
Not yet - however I do know how to load modules but I'm not a kernel wiz,
so a pointer to a module name would be appreciated. I've checked the
obvious "nat" named ones and "conntrac" ones.
The module loading process has been heavily modified, but just for 6.x.
6.x loads the modules from a squashfs system and does not have the
moddb.lrp stuff anymoer.
You should not face problems with 5.x

cheers

Erich
Philip Le Riche
2017-01-03 14:51:07 UTC
Permalink
I've been trying without success on and off for some while to modify an
existing Shorewall configuration for the purposes of a school lesson on
Internet routing, using traceroute.

I originally set up the firewall to protect the school network from a
bunch of Raspberry Pis, operated "headless" from school PCs using VNC or
ssh, thus we had 3 zones:

#ZONE TYPE OPTIONS IN OUT
fw firewall
schl ipv4
pinet ipv4

The idea is to run traceroute from the Pis, but since since traceroute
is blocked by the school firewall/proxy I've added a mobile data dongle
and a new zone giving me unfiltered Internet access:
inet ipv4

My interfaces file now looks like this:
schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
pinet enp2s0 tcpflags,nosmurfs,routefilter,logmartians
inet ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional

In my providers file I've defined a provider "raw" for the unfiltered
mobile data interface:
#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS
raw 1 1 - ppp0

I've been trying both regular traceroute (udp/33434-33523) and
traceroute -P 253 (protocol 253), and so I'm using mangle to mark all
such packets coming from the Pi network (and from the firewall while I'm
at it, for testing purposes):
#ACTION SOURCE DEST PROTO PORT(S) SOURCE USER TEST
# PORT(S)
MARK(1) enp2s0 - udp 33434:33523 - - -
MARK(1) enp2s0 - 253 - - - -
MARK(1) $FW - udp 33434:33523 - - -
MARK(1) $FW - 253 - - - -

And in rtrules I'm directing marked packets at provider raw:
SOURCE DEST PROVIDER PRIORITY MARK
enp2s0 - raw 11000 1
lo - raw 11000 1

In my rules file I've allowed traceroute from pinet and $FW to inet:
#
# pinet -> inet
# Allow traceroute only
#
ACCEPT pinet inet udp 33434:33523
ACCEPT pinet inet 253

#
# $FW -> inet
#
#ACTION SOURCE DEST PROTO DEST SOURCE RATE USER/
# PORT(S) PORT(S) LIMIT GROUP
ACCEPT $FW inet udp 33434:33523
ACCEPT $FW inet 253

Since the mobile data dongle hasn't connected by the time Shorewall
starts on a reboot, I have to do a shorewall restart, and also if I plug
in the dongle at any time after booting.

However, there still seems to be an error or omission in my logic as
traceroute on the firewall Pi still shows it routing through the school
network, as evidenced by the ip addresses reported (as far as they go),
and traceroute on a Pi shows nothing beyond the pinet firewall interface.
Perhaps you can provide me with that lightbulb moment which seems to be
evading me.

Regards - Philip
Tom Eastep
2017-01-03 17:22:15 UTC
Permalink
Post by Philip Le Riche
I've been trying without success on and off for some while to
modify an existing Shorewall configuration for the purposes of a
school lesson on Internet routing, using traceroute.
I originally set up the firewall to protect the school network from
a bunch of Raspberry Pis, operated "headless" from school PCs using
#ZONE TYPE OPTIONS IN OUT fw
firewall schl ipv4 pinet ipv4
The idea is to run traceroute from the Pis, but since since
traceroute is blocked by the school firewall/proxy I've added a
mobile data dongle and a new zone giving me unfiltered Internet
access: inet ipv4
My interfaces file now looks like this: schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0 pinet
enp2s0 tcpflags,nosmurfs,routefilter,logmartians inet
ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional
In my providers file I've defined a provider "raw" for the
unfiltered mobile data interface: #NAME NUMBER MARK
DUPLICATE INTERFACE GATEWAY OPTIONS raw 1 1
- ppp0
I've been trying both regular traceroute (udp/33434-33523) and
traceroute -P 253 (protocol 253), and so I'm using mangle to mark
all such packets coming from the Pi network (and from the firewall
while I'm at it, for testing purposes): #ACTION SOURCE DEST
PROTO PORT(S) SOURCE USER TEST #
PORT(S) MARK(1) enp2s0 - udp 33434:33523 - -
- MARK(1) enp2s0 - 253 - - - - MARK(1)
$FW - udp 33434:33523 - - - MARK(1) $FW
- 253 - - - -
And in rtrules I'm directing marked packets at provider raw: SOURCE
DEST PROVIDER PRIORITY MARK enp2s0 - raw
11000 1 lo - raw 11000 1
In my rules file I've allowed traceroute from pinet and $FW to
inet: # # pinet -> inet # Allow traceroute only # ACCEPT
pinet inet udp 33434:33523 ACCEPT pinet
inet 253
# # $FW -> inet # #ACTION SOURCE DEST PROTO DEST
SOURCE RATE USER/ #
PORT(S) PORT(S) LIMIT GROUP ACCEPT $FW inet udp
33434:33523 ACCEPT $FW inet 253
Since the mobile data dongle hasn't connected by the time
Shorewall starts on a reboot, I have to do a shorewall restart, and
also if I plug in the dongle at any time after booting.
However, there still seems to be an error or omission in my logic
as traceroute on the firewall Pi still shows it routing through the
school network, as evidenced by the ip addresses reported (as far
as they go), and traceroute on a Pi shows nothing beyond the pinet
firewall interface. Perhaps you can provide me with that lightbulb
moment which seems to be evading me.
You need en01 to be the primary provider and ppp0 to be the fallback
provider.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Philip Le Riche
2017-01-06 11:52:57 UTC
Permalink
Thanks, Tom, for the rapid response.

I don't have easy access to the firewall in question so I've set up an
equivalent network at home. In the providers file I've added the primary
option to the school network and fallback to the mobile data, though I
don't actually want it to fall back.

Now, on starting Shorewall, I get WARNING: interface ppp0 is not useable.

Is ther a log file which will shed a little more light on that?

In fact Linux won't start the ppp0 session unless I do shorewall clear
before plugging the dongle in - I'm not sure I got that at school, but
with shorewall clear and if I set eno1 down (the "school" network) I can
browse the net through the dongle.

I now have interfaces:
schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
pinet enx00e04c534458 tcpflags,nosmurfs,routefilter,logmartians
inet ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional

providers:
#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS
raw 1 1 - ppp0 detect fallback
school 2 - - eno1 192.168.1.1 primary

mangle:
#ACTION SOURCE DEST PROTO PORT(S) SOURCE USER TEST
# PORT(S)
MARK(1) enx00e04c534458 - udp 33434:33523 - - -
MARK(1) enx00e04c534458 - 253 - - - -
MARK(1) lo - udp 33434:33523 - - -
MARK(1) lo - 253 - - - -

rtrules:
#SOURCE DEST PROVIDER PRIORITY MARK
enx00e04c534458 - raw 11000 1
lo - raw 11000 1

masq:
#INTERFACE:DEST SOURCE ADDRESS PROTO PORT(S)
IPSEC MARK USER/ SWITCH ORIGINAL
# GROUP DEST
eno1 192.168.2.0/24 192.168.1.2

and in rules I simply modified the DNAT rules for the Pis to reflect the
different IP addressing scheme (I didn't mention that before).

Best regards - Philip
Post by Tom Eastep
Post by Philip Le Riche
I've been trying without success on and off for some while to
modify an existing Shorewall configuration for the purposes of a
school lesson on Internet routing, using traceroute.
I originally set up the firewall to protect the school network from
a bunch of Raspberry Pis, operated "headless" from school PCs using
#ZONE TYPE OPTIONS IN OUT fw
firewall schl ipv4 pinet ipv4
The idea is to run traceroute from the Pis, but since since
traceroute is blocked by the school firewall/proxy I've added a
mobile data dongle and a new zone giving me unfiltered Internet
access: inet ipv4
My interfaces file now looks like this: schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0 pinet
enp2s0 tcpflags,nosmurfs,routefilter,logmartians inet
ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional
In my providers file I've defined a provider "raw" for the
unfiltered mobile data interface: #NAME NUMBER MARK
DUPLICATE INTERFACE GATEWAY OPTIONS raw 1 1
- ppp0
I've been trying both regular traceroute (udp/33434-33523) and
traceroute -P 253 (protocol 253), and so I'm using mangle to mark
all such packets coming from the Pi network (and from the firewall
while I'm at it, for testing purposes): #ACTION SOURCE DEST
PROTO PORT(S) SOURCE USER TEST #
PORT(S) MARK(1) enp2s0 - udp 33434:33523 - -
- MARK(1) enp2s0 - 253 - - - - MARK(1)
$FW - udp 33434:33523 - - - MARK(1) $FW
- 253 - - - -
And in rtrules I'm directing marked packets at provider raw: SOURCE
DEST PROVIDER PRIORITY MARK enp2s0 - raw
11000 1 lo - raw 11000 1
In my rules file I've allowed traceroute from pinet and $FW to
inet: # # pinet -> inet # Allow traceroute only # ACCEPT
pinet inet udp 33434:33523 ACCEPT pinet
inet 253
# # $FW -> inet # #ACTION SOURCE DEST PROTO DEST
SOURCE RATE USER/ #
PORT(S) PORT(S) LIMIT GROUP ACCEPT $FW inet udp
33434:33523 ACCEPT $FW inet 253
Since the mobile data dongle hasn't connected by the time
Shorewall starts on a reboot, I have to do a shorewall restart, and
also if I plug in the dongle at any time after booting.
However, there still seems to be an error or omission in my logic
as traceroute on the firewall Pi still shows it routing through the
school network, as evidenced by the ip addresses reported (as far
as they go), and traceroute on a Pi shows nothing beyond the pinet
firewall interface. Perhaps you can provide me with that lightbulb
moment which seems to be evading me.
You need en01 to be the primary provider and ppp0 to be the fallback
provider.
-Tom
Philip Le Riche
2017-01-10 20:50:16 UTC
Permalink
I'm afraid I'm still struggling with this, though I made a minor
breakthrough when I realised I hadn't added a masq rule for the raw
interface, and the ppp0 not useable problem has gone away. (It seems I
have to connect it with shorewall clear then start shorewall.) Anyway,
my home test setup now seems to be working like the school firewall.

(To recap, Raspberry Pis on zone pinet are accessed by PCs in zone schl
using ssh and vnc, and access the Internet via schl and the school
gateway. Traceroute traffic (only) from Pis and the firewall is to be
routed to a 3rd zone containing a mobile data dongle to give unfiltered
Internet access.)

Traceroute is now routed correctly from the Pis, but on the firewall
traceroute reports Send: Operation not permitted. (I have the same rules
with pinet and $FW as source to allow traceroute.) Also, web access from
both the Pis and the firewall is now broken. However a PC on schl can
still access a Pi.

My providers file is now:
#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS
raw 1 1 - ppp0 -
school 2 - - eno1 192.168.1.1 primary

If I add option fallback to provider raw, that fixes web from both the
Pis and the firewall but breaks traceroute. (I didn't think it was a
good idea but tried it anyway.)

I've read providers(5) and Multiple Internet Connections several times
and spent a good few hours trying to get it to work but there seems to
be something that I still haven't correctly understood. Any help would
be greatly appreciated.

For reference, my other relevant shorewall files are:
mangle:
#ACTION SOURCE DEST PROTO PORT(S) SOURCE USER TEST
# PORT(S)
MARK(1) enx00e04c534458 - udp 33434:33523 - - -
MARK(1) enx00e04c534458 - 253 - - - -
MARK(1) $FW - udp 33434:33523 - - -
MARK(1) $FW - 253 - - - -

rtrules:
#SOURCE DEST PROVIDER PRIORITY MARK
enx00e04c534458 - raw 11000 1
lo - raw 11000 1

zones:
fw firewall
schl ipv4
pinet ipv4
inet ipv4

interfaces:
schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0
pinet enx00e04c534458 tcpflags,nosmurfs,routefilter,logmartians
inet ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional

Regards - Philip
Post by Philip Le Riche
Thanks, Tom, for the rapid response.
I don't have easy access to the firewall in question so I've set up an
equivalent network at home. In the providers file I've added the
primary option to the school network and fallback to the mobile data,
though I don't actually want it to fall back.
<snip snip>
Tom Eastep
2017-01-10 21:05:39 UTC
Permalink
Post by Philip Le Riche
I'm afraid I'm still struggling with this, though I made a minor
breakthrough when I realised I hadn't added a masq rule for the
raw interface, and the ppp0 not useable problem has gone away. (It
seems I have to connect it with shorewall clear then start
shorewall.) Anyway, my home test setup now seems to be working like
the school firewall.
(To recap, Raspberry Pis on zone pinet are accessed by PCs in zone
schl using ssh and vnc, and access the Internet via schl and the
school gateway. Traceroute traffic (only) from Pis and the firewall
is to be routed to a 3rd zone containing a mobile data dongle to
give unfiltered Internet access.)
Traceroute is now routed correctly from the Pis, but on the
firewall traceroute reports Send: Operation not permitted. (I have
the same rules with pinet and $FW as source to allow traceroute.)
Also, web access from both the Pis and the firewall is now broken.
However a PC on schl can still access a Pi.
My providers file is now: #NAME NUMBER MARK DUPLICATE
INTERFACE GATEWAY OPTIONS raw 1 1 - ppp0
- school 2 - - eno1 192.168.1.1 primary
If I add option fallback to provider raw, that fixes web from both
the Pis and the firewall but breaks traceroute. (I didn't think it
was a good idea but tried it anyway.)
I've read providers(5) and Multiple Internet Connections several
times and spent a good few hours trying to get it to work but there
seems to be something that I still haven't correctly understood.
Any help would be greatly appreciated.
#ACTION SOURCE DEST PROTO PORT(S) SOURCE USER
TEST # PORT(S) MARK(1) enx00e04c534458 -
udp 33434:33523 - - - MARK(1) enx00e04c534458 -
253 - - - - MARK(1) $FW - udp 33434:33523
- - - MARK(1) $FW - 253 - - - -
rtrules: #SOURCE DEST PROVIDER PRIORITY MARK
enx00e04c534458 - raw 11000 1 lo - raw
11000 1
zones: fw firewall schl ipv4 pinet ipv4 inet ipv4
interfaces: schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0 pinet
enx00e04c534458 tcpflags,nosmurfs,routefilter,logmartians inet
ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional
Philip,

Please:

a) Set fallback on the raw provider.
b) Shorewall reload
c) Try a traceroute from a Pi
d) 'shorewall dump > dump'
e) Send me the 'dump' file.

Thanks,
- -Tom

- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Philip Le Riche
2017-01-10 21:55:00 UTC
Permalink
Hi Tom -

Thanks for the greased-lightning response again, and here's the dump.

Many thanks - Philip
Post by Tom Eastep
Post by Philip Le Riche
I'm afraid I'm still struggling with this, though I made a minor
breakthrough when I realised I hadn't added a masq rule for the
raw interface, and the ppp0 not useable problem has gone away. (It
seems I have to connect it with shorewall clear then start
shorewall.) Anyway, my home test setup now seems to be working like
the school firewall.
(To recap, Raspberry Pis on zone pinet are accessed by PCs in zone
schl using ssh and vnc, and access the Internet via schl and the
school gateway. Traceroute traffic (only) from Pis and the firewall
is to be routed to a 3rd zone containing a mobile data dongle to
give unfiltered Internet access.)
Traceroute is now routed correctly from the Pis, but on the
firewall traceroute reports Send: Operation not permitted. (I have
the same rules with pinet and $FW as source to allow traceroute.)
Also, web access from both the Pis and the firewall is now broken.
However a PC on schl can still access a Pi.
My providers file is now: #NAME NUMBER MARK DUPLICATE
INTERFACE GATEWAY OPTIONS raw 1 1 - ppp0
- school 2 - - eno1 192.168.1.1 primary
If I add option fallback to provider raw, that fixes web from both
the Pis and the firewall but breaks traceroute. (I didn't think it
was a good idea but tried it anyway.)
I've read providers(5) and Multiple Internet Connections several
times and spent a good few hours trying to get it to work but there
seems to be something that I still haven't correctly understood.
Any help would be greatly appreciated.
#ACTION SOURCE DEST PROTO PORT(S) SOURCE USER
TEST # PORT(S) MARK(1) enx00e04c534458 -
udp 33434:33523 - - - MARK(1) enx00e04c534458 -
253 - - - - MARK(1) $FW - udp 33434:33523
- - - MARK(1) $FW - 253 - - - -
rtrules: #SOURCE DEST PROVIDER PRIORITY MARK
enx00e04c534458 - raw 11000 1 lo - raw
11000 1
zones: fw firewall schl ipv4 pinet ipv4 inet ipv4
interfaces: schl eno1
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0 pinet
enx00e04c534458 tcpflags,nosmurfs,routefilter,logmartians inet
ppp0
tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,optional
Philip,
a) Set fallback on the raw provider.
b) Shorewall reload
c) Try a traceroute from a Pi
d) 'shorewall dump > dump'
e) Send me the 'dump' file.
Thanks,
-Tom
Tom Eastep
2017-01-11 00:38:43 UTC
Permalink
Post by Philip Le Riche
Hi Tom -
Thanks for the greased-lightning response again, and here's the dump.
It looks to me like the traceroute packets are going out of ppp0 but
that there are no responses. Can you confirm that using tcpdump?

Thanks,
- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Philip Le Riche
2017-01-11 11:21:40 UTC
Permalink
Hi Tom -

Here are a couple of pcaps on ppp0 from wireshark, one with ppp0 as
fallback (traceroute from the Pi doesn't work but web does) and with
ppp0 with no options (traceroute works but web doesn't).

In both cases you can see the udp packets going out and icmp timeouts
coming back but with fallback they don't seem to make it back to the Pi.
It looks like shorewall isn't opening the reverse path. Hopefully the
inconsistent web behaviour is another consequence of the same problem.

Several other problems which may or may not be related:
1. traceroute getting send: operation not permitted when run from the
firewall itself.
2. Mobile data dongle not starting with shorewall running - possibly the
same problem as 1.
3. dhcpd not starting reliably - possibly a startup sequence problem -
it's worked the last twice and I didn't record the message but was
something about no available NICs to serve on.

Thanks again - Philip
Post by Tom Eastep
Post by Philip Le Riche
Hi Tom -
Thanks for the greased-lightning response again, and here's the dump.
It looks to me like the traceroute packets are going out of ppp0 but
that there are no responses. Can you confirm that using tcpdump?
Thanks,
-Tom
Tom Eastep
2017-01-11 16:48:24 UTC
Permalink
Post by Philip Le Riche
Hi Tom -
Here are a couple of pcaps on ppp0 from wireshark, one with ppp0
as fallback (traceroute from the Pi doesn't work but web does) and
with ppp0 with no options (traceroute works but web doesn't).
In both cases you can see the udp packets going out and icmp
timeouts coming back but with fallback they don't seem to make it
back to the Pi. It looks like shorewall isn't opening the reverse
path. Hopefully the inconsistent web behaviour is another
consequence of the same problem.
Several other problems which may or may not be related: 1.
traceroute getting send: operation not permitted when run from the
firewall itself. 2. Mobile data dongle not starting with shorewall
running - possibly the same problem as 1. 3. dhcpd not starting
reliably - possibly a startup sequence problem - it's worked the
last twice and I didn't record the message but was something about
no available NICs to serve on.
Turn off route filtering. From the dump:

/proc
...
/proc/sys/net/ipv4/conf/eno1/rp_filter = 1
/proc/sys/net/ipv4/conf/ppp0/rp_filter = 1


You have 'routefilter' specified on both provider interfaces. From
shorewall-interfaces(5):

Note
There are certain cases where routefilter cannot be used on an interface:

If USE_DEFAULT_RT=Yes in shorewall.conf[12](5) and the interface is
listed in shorewall-providers[18](5).

If there is an entry for the interface in shorewall-providers[18](5)
that doesn't specify the balance option.

...

Set 'routefilter=0' for both interfaces.

- -Tom

- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Tom Eastep
2017-01-11 19:46:22 UTC
Permalink
Post by Philip Le Riche
Hi Tom -
Several other problems which may or may not be related: 1.
traceroute getting send: operation not permitted when run from the
firewall itself.
As pointed out in http://www.shorewall.org/MultiISP.html, packet
marking is unreliable when applied to connections originating from the
firewall. Try using the '-i' option of traceroute from the firewall.
Post by Philip Le Riche
2. Mobile data dongle not starting with shorewall running -
possibly the same problem as 1.
No clue -- are there any 'Shorewall' messages logged when this occurs?
Post by Philip Le Riche
3. dhcpd not starting reliably - possibly a startup sequence
problem - it's worked the last twice and I didn't record the
message but was something about no available NICs to serve on.
Sounds like a startup sequencing issue. Can't tell without seeing the
messages.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Philip Le Riche
2017-01-11 21:31:30 UTC
Permalink
Great - thanks Tom! Removing routefilter from the 2 outbound interfaces
did the trick. I can now do both traceroute and http from the Pi, and
the -i option fixed traceroute on the firewall itself. I would have
given up long before stumbling across routefilter.

I haven't seen the dhcpd startup problem again so I assume that's gone
away. However the mobile dongle startup seems to be getting more
unreliable but that seems to be USB problem not a shorewall one assuming
the kernel USB and networking stacks are completely disjoint.
(/var/log/messages shows it sometimes recognising the mass storeage
device and/or the CDROM on the dongle but not the GSM modem, or
detecting it as a serial device but not doing anything with it).

Thanks again for excellent support.

Best regards - Philip
Post by Tom Eastep
Post by Philip Le Riche
Hi Tom -
Several other problems which may or may not be related: 1.
traceroute getting send: operation not permitted when run from the
firewall itself.
As pointed out in http://www.shorewall.org/MultiISP.html, packet
marking is unreliable when applied to connections originating from the
firewall. Try using the '-i' option of traceroute from the firewall.
Post by Philip Le Riche
2. Mobile data dongle not starting with shorewall running -
possibly the same problem as 1.
No clue -- are there any 'Shorewall' messages logged when this occurs?
Post by Philip Le Riche
3. dhcpd not starting reliably - possibly a startup sequence
problem - it's worked the last twice and I didn't record the
message but was something about no available NICs to serve on.
Sounds like a startup sequencing issue. Can't tell without seeing the
messages.
-Tom
Tom Eastep
2017-01-11 22:53:20 UTC
Permalink
Post by Philip Le Riche
Great - thanks Tom! Removing routefilter from the 2 outbound
interfaces did the trick. I can now do both traceroute and http
from the Pi, and the -i option fixed traceroute on the firewall
itself. I would have given up long before stumbling across
routefilter.
I haven't seen the dhcpd startup problem again so I assume that's
gone away. However the mobile dongle startup seems to be getting
more unreliable but that seems to be USB problem not a shorewall
one assuming the kernel USB and networking stacks are completely
disjoint. (/var/log/messages shows it sometimes recognising the
mass storeage device and/or the CDROM on the dongle but not the GSM
modem, or detecting it as a serial device but not doing anything
with it).
Thanks again for excellent support.
Glad to hear that it is working.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Philip Le Riche
2017-01-12 16:28:46 UTC
Permalink
Slightly off-topic, but in case anyone picked up on my problem with the
mobile data dongle, if it doesn't connect by itself the trick seems to
be to eject the CDROM that it throws at you. It then presents the GSM
modem and connects. Whether or not shorewall is started, stopped or
clear is irrelevant but it needs to be restarted after ppp0 has come up.

Regards - Philip
Post by Tom Eastep
Post by Philip Le Riche
Great - thanks Tom! Removing routefilter from the 2 outbound
interfaces did the trick. I can now do both traceroute and http
from the Pi, and the -i option fixed traceroute on the firewall
itself. I would have given up long before stumbling across
routefilter.
I haven't seen the dhcpd startup problem again so I assume that's
gone away. However the mobile dongle startup seems to be getting
more unreliable but that seems to be USB problem not a shorewall
one assuming the kernel USB and networking stacks are completely
disjoint. (/var/log/messages shows it sometimes recognising the
mass storeage device and/or the CDROM on the dongle but not the GSM
modem, or detecting it as a serial device but not doing anything
with it).
Thanks again for excellent support.
Glad to hear that it is working.
-Tom
Tom Eastep
2017-01-12 16:54:33 UTC
Permalink
Post by Philip Le Riche
Slightly off-topic, but in case anyone picked up on my problem with
the mobile data dongle, if it doesn't connect by itself the trick
seems to be to eject the CDROM that it throws at you. It then
presents the GSM modem and connects. Whether or not shorewall is
started, stopped or clear is irrelevant but it needs to be
restarted after ppp0 has come up.
If ppp0 is defined as 'optional' in the interfaces file, all that is
required with recent versions is 'shorewall enable ppp0'.

- -Tom
- --
Tom Eastep \ When I die, I want to go like my Grandfather who
Shoreline, \ died peacefully in his sleep. Not screaming like
Washington, USA \ all of the passengers in his car
http://shorewall.net \________________________________________________
Continue reading on narkive:
Loading...