s***@iotti.biz
2016-10-23 07:09:41 UTC
Hi all
I am experimenting with traffic shaping. I use 5.0.13.1 on CentOS 7 with
added xtables-addons. I have BASIC_FILTERS=Yes . Currently I am trying to
use ipsets in tcfilter, but I stumbled in 3 distinct errors which I think
might be bugs in Tc.pm. I sort of solved 2 of these, and I would like to
review them and possibly make them in the source. For the remaining one, I
would like to ask if there is a solution.
Issue #1, if I put the following line in tcfilters:
1:140 +wuhosts
I obtain the following error:
Setting up Traffic Control...
Object "ipset(wuhosts" is unknown, try "tc help".
ERROR: Command "tc ipset(wuhosts src) flowid 1:140" Failed
Restoring Shorewall...
Initializing...
Processing /etc/shorewall/init ...
Processing /etc/shorewall/tcclear ...
Setting up Route Filtering...
Setting up Martian Logging...
Setting up Accept Source Routing...
Setting up Proxy ARP...
Setting up Traffic Control...
IPv4 Forwarding Enabled
Processing /etc/shorewall/restored ...
done.
Shorewall restored from /var/lib/shorewall/.try
/usr/share/shorewall/lib.common: line 93: 8663 Terminated
$SHOREWALL_SHELL $script $options $@
Spreading some print STDERR "\n$rule\n" in process_tc_filter2 in Tc.pm, it
seems the problem is that the rule to be applied is reset when the parser
encounters an ipset, where the new clause should be added. This results in a
tc rule like this, which is clearly invalid:
'\
ipset\(wuhosts src\)'
The following patch seems to solve the issue:
--- /usr/share/perl5/vendor_perl/Shorewall/Tc.pm.orig 2016-10-23
07:41:55.000000000 +0200
+++ /usr/share/perl5/vendor_perl/Shorewall/Tc.pm 2016-10-23
08:11:15.913612498 +0200
@@ -1518,7 +1518,7 @@
$rule .= ' and' if $have_rule;
if ( $source =~ /^\+/ ) {
- $rule = join( '', "\\\n ", handle_ematch( $source, 'src' ) );
+ $rule .= join( '', "\\\n ", handle_ematch( $source, 'src' ) );
} else {
my @parts = decompose_net_u32( $source );
For reference, the tc command results like this:
'filter add dev eth1 protocol ip parent 1:0 prio 1 basic match\
ipset\(wuhosts src\)'
Issue #2, Another problem I encounter is that I have to manually pre-create
the wuhosts set, otherwise I get another error:
Setting up Traffic Control...
ipset: unknown set name 'wuhosts'
... ipset(wuhosts >>dst)<< ...
... ipset(>>wuhosts<< dst)...
Usage: ipset(SETNAME FLAGS)
where: SETNAME:= string
FLAGS := { FLAG[,FLAGS] }
FLAG := { src | dst }
Example: 'ipset(bulk src,dst)'
Illegal "ematch"
ERROR: Command "tc filter add dev ifb0 protocol ip parent 2:0 prio 1
basic match ipset(wuhosts dst) flowid 2:140" Failed
I am sure there is support in shorewall to automatically create ipsets when
needed, because for example I use dynamic zones which are implemented with
ipsets (and in fact it calls add_ipset($ipset); ).
Inspired by that I tried to simply do something like this:
--- Tc.pm.orig 2016-10-23 07:41:55.000000000 +0200
+++ Tc.pm 2016-10-23 08:55:24.529013933 +0200
@@ -1517,8 +1517,9 @@
if ( $source ne '-' ) {
$rule .= ' and' if $have_rule;
- if ( $source =~ /^\+/ ) {
+ if ( $source =~ /^\+(\S+)/ ) {
$rule .= join( '', "\\\n ", handle_ematch( $source, 'src' ) );
+ add_ipset($1);
} else {
my @parts = decompose_net_u32( $source );
but it gave me this weird error. Apparently I have to know something more
about ipset handling in Shorewall. Could this be addressed?
Optimizing Ruleset...
Creating iptables-restore input...
Use of uninitialized value $capability in hash element at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4937.
Use of uninitialized value $capability in hash element at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4924.
ERROR: Internal error in Shorewall::Config::detect_capability at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4926 at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 1466.
Shorewall::Config::fatal_error('Internal error in
Shorewall::Config::detect_capability at /us...') called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 1506
Shorewall::Config::assert('') called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4926
Shorewall::Config::detect_capability(undef) called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4939
Shorewall::Config::have_capability(undef) called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4551
Shorewall::Config::IPSet_Match_Counters() called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4927
Shorewall::Config::detect_capability('IPSET_MATCH_COUNTERS') called
at /usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4939
Shorewall::Config::have_capability('IPSET_MATCH_COUNTERS') called at
/usr/share/perl5/vendor_perl/Shorewall/Chains.pm line 8268
Shorewall::Chains::ensure_ipsets('wuhosts') called at
/usr/share/perl5/vendor_perl/Shorewall/Chains.pm line 8348
Shorewall::Chains::create_save_ipsets() called at
/usr/share/perl5/vendor_perl/Shorewall/Compiler.pm line 370
Shorewall::Compiler::generate_script_3(':none:') called at
/usr/share/perl5/vendor_perl/Shorewall/Compiler.pm line 923
Shorewall::Compiler::compiler('script',
'/var/lib/shorewall/.reload', 'directory', '/etc/shorewallConWinUpdSets',
'verbosity', 1, 'timestamp', 0, 'debug', ...) called at
/usr/libexec/shorewall/compiler.pl line 142
Issue #3, more like #1, if I put the following line in tcfilters:
2:140 - +wuhosts
I get the error:
Compiling /etc/shorewallConWinUpdSets/tcfilters...
WARNING: Degenerate filter ignored /etc/shorewallConWinUpdSets/tcfilters
(line 11)
Without going too deep, it seems that in Tc.pm, when parsing a rule with a
set in DEST, the rule is treated as if it was void (degenerate). A change
like this seems to address the issue:
--- Tc.pm.orig 2016-10-23 07:41:55.000000000 +0200
+++ Tc.pm 2016-10-23 09:01:15.129492591 +0200
@@ -1558,8 +1558,9 @@
}
}
- $have_rule = 1;
}
+
+ $have_rule = 1;
}
if ( $have_rule ) {
Thank you
Luigi
I am experimenting with traffic shaping. I use 5.0.13.1 on CentOS 7 with
added xtables-addons. I have BASIC_FILTERS=Yes . Currently I am trying to
use ipsets in tcfilter, but I stumbled in 3 distinct errors which I think
might be bugs in Tc.pm. I sort of solved 2 of these, and I would like to
review them and possibly make them in the source. For the remaining one, I
would like to ask if there is a solution.
Issue #1, if I put the following line in tcfilters:
1:140 +wuhosts
I obtain the following error:
Setting up Traffic Control...
Object "ipset(wuhosts" is unknown, try "tc help".
ERROR: Command "tc ipset(wuhosts src) flowid 1:140" Failed
Restoring Shorewall...
Initializing...
Processing /etc/shorewall/init ...
Processing /etc/shorewall/tcclear ...
Setting up Route Filtering...
Setting up Martian Logging...
Setting up Accept Source Routing...
Setting up Proxy ARP...
Setting up Traffic Control...
IPv4 Forwarding Enabled
Processing /etc/shorewall/restored ...
done.
Shorewall restored from /var/lib/shorewall/.try
/usr/share/shorewall/lib.common: line 93: 8663 Terminated
$SHOREWALL_SHELL $script $options $@
Spreading some print STDERR "\n$rule\n" in process_tc_filter2 in Tc.pm, it
seems the problem is that the rule to be applied is reset when the parser
encounters an ipset, where the new clause should be added. This results in a
tc rule like this, which is clearly invalid:
'\
ipset\(wuhosts src\)'
The following patch seems to solve the issue:
--- /usr/share/perl5/vendor_perl/Shorewall/Tc.pm.orig 2016-10-23
07:41:55.000000000 +0200
+++ /usr/share/perl5/vendor_perl/Shorewall/Tc.pm 2016-10-23
08:11:15.913612498 +0200
@@ -1518,7 +1518,7 @@
$rule .= ' and' if $have_rule;
if ( $source =~ /^\+/ ) {
- $rule = join( '', "\\\n ", handle_ematch( $source, 'src' ) );
+ $rule .= join( '', "\\\n ", handle_ematch( $source, 'src' ) );
} else {
my @parts = decompose_net_u32( $source );
For reference, the tc command results like this:
'filter add dev eth1 protocol ip parent 1:0 prio 1 basic match\
ipset\(wuhosts src\)'
Issue #2, Another problem I encounter is that I have to manually pre-create
the wuhosts set, otherwise I get another error:
Setting up Traffic Control...
ipset: unknown set name 'wuhosts'
... ipset(wuhosts >>dst)<< ...
... ipset(>>wuhosts<< dst)...
Usage: ipset(SETNAME FLAGS)
where: SETNAME:= string
FLAGS := { FLAG[,FLAGS] }
FLAG := { src | dst }
Example: 'ipset(bulk src,dst)'
Illegal "ematch"
ERROR: Command "tc filter add dev ifb0 protocol ip parent 2:0 prio 1
basic match ipset(wuhosts dst) flowid 2:140" Failed
I am sure there is support in shorewall to automatically create ipsets when
needed, because for example I use dynamic zones which are implemented with
ipsets (and in fact it calls add_ipset($ipset); ).
Inspired by that I tried to simply do something like this:
--- Tc.pm.orig 2016-10-23 07:41:55.000000000 +0200
+++ Tc.pm 2016-10-23 08:55:24.529013933 +0200
@@ -1517,8 +1517,9 @@
if ( $source ne '-' ) {
$rule .= ' and' if $have_rule;
- if ( $source =~ /^\+/ ) {
+ if ( $source =~ /^\+(\S+)/ ) {
$rule .= join( '', "\\\n ", handle_ematch( $source, 'src' ) );
+ add_ipset($1);
} else {
my @parts = decompose_net_u32( $source );
but it gave me this weird error. Apparently I have to know something more
about ipset handling in Shorewall. Could this be addressed?
Optimizing Ruleset...
Creating iptables-restore input...
Use of uninitialized value $capability in hash element at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4937.
Use of uninitialized value $capability in hash element at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4924.
ERROR: Internal error in Shorewall::Config::detect_capability at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4926 at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 1466.
Shorewall::Config::fatal_error('Internal error in
Shorewall::Config::detect_capability at /us...') called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 1506
Shorewall::Config::assert('') called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4926
Shorewall::Config::detect_capability(undef) called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4939
Shorewall::Config::have_capability(undef) called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4551
Shorewall::Config::IPSet_Match_Counters() called at
/usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4927
Shorewall::Config::detect_capability('IPSET_MATCH_COUNTERS') called
at /usr/share/perl5/vendor_perl/Shorewall/Config.pm line 4939
Shorewall::Config::have_capability('IPSET_MATCH_COUNTERS') called at
/usr/share/perl5/vendor_perl/Shorewall/Chains.pm line 8268
Shorewall::Chains::ensure_ipsets('wuhosts') called at
/usr/share/perl5/vendor_perl/Shorewall/Chains.pm line 8348
Shorewall::Chains::create_save_ipsets() called at
/usr/share/perl5/vendor_perl/Shorewall/Compiler.pm line 370
Shorewall::Compiler::generate_script_3(':none:') called at
/usr/share/perl5/vendor_perl/Shorewall/Compiler.pm line 923
Shorewall::Compiler::compiler('script',
'/var/lib/shorewall/.reload', 'directory', '/etc/shorewallConWinUpdSets',
'verbosity', 1, 'timestamp', 0, 'debug', ...) called at
/usr/libexec/shorewall/compiler.pl line 142
Issue #3, more like #1, if I put the following line in tcfilters:
2:140 - +wuhosts
I get the error:
Compiling /etc/shorewallConWinUpdSets/tcfilters...
WARNING: Degenerate filter ignored /etc/shorewallConWinUpdSets/tcfilters
(line 11)
Without going too deep, it seems that in Tc.pm, when parsing a rule with a
set in DEST, the rule is treated as if it was void (degenerate). A change
like this seems to address the issue:
--- Tc.pm.orig 2016-10-23 07:41:55.000000000 +0200
+++ Tc.pm 2016-10-23 09:01:15.129492591 +0200
@@ -1558,8 +1558,9 @@
}
}
- $have_rule = 1;
}
+
+ $have_rule = 1;
}
if ( $have_rule ) {
Thank you
Luigi