New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[22.03] jsonfilter yields unreliable results #8703
Comments
mikma: The problem is caused by a bug in the array parsing. It can be solved with the following patch. https://patchwork.ozlabs.org/project/openwrt/list/?series=244999 |
OpenWrt 19.07 release is EOL, try to reproduce the issue with latest supported release and feel free to ask for issue reopening if the problem is still present, thanks. |
This problem still exists in 22.03. Please reopen. Below is another example that would trigger the bug: { "interface": "lan", "up": true, "pending": false, "available": true, "autostart": true, "dynamic": false, "uptime": 455693, "l3_device": "br-lan", "proto": "dhcp", "device": "br-lan", "metric": 0, "dns_metric": 0, "delegation": true, "ipv4-address": [ { "address": "192.168.123.58", "mask": 24 } ], "ipv6-address": [ ], "ipv6-prefix": [ ], "ipv6-prefix-assignment": [ ], "route": [ { "target": "0.0.0.0", "mask": 0, "nexthop": "192.168.123.1", "source": "192.168.123.58\/32" } ], "dns-server": [ "192.168.123.1" ], "dns-search": [ ], "neighbors": [ ], "inactive": { "ipv4-address": [ ], "ipv6-address": [ ], "route": [ ], "dns-server": [ ], "dns-search": [ ], "neighbors": [ ] }, "data": { "dhcpserver": "192.168.123.1", "hostname": "foo", "leasetime": 43200 } }
{ "interface": "lan6", "up": true, "pending": false, "available": true, "autostart": true, "dynamic": false, "uptime": 455689, "l3_device": "br-lan", "proto": "dhcpv6", "device": "br-lan", "metric": 0, "dns_metric": 0, "delegation": false, "ipv4-address": [ ], "ipv6-address": [ { "address": "1234:123:1234:1234:1234:123:1234:1234", "mask": 64, "preferred": 172773, "valid": 172773 } ], "ipv6-prefix": [ ], "ipv6-prefix-assignment": [ ], "route": [ { "target": "1234:123:1234:1234::", "mask": 64, "nexthop": "::", "metric": 256, "valid": 172773, "source": "::\/0" }, { "target": "::", "mask": 0, "nexthop": "fe80::1234:1234:1234:1234", "metric": 512, "valid": 1173, "source": "1234:123:1234:1234:1234:123:1234:1234\/64" } ], "dns-server": [ "1234:123:1234:1234::1", "1234:123:1234:1234::1" ], "dns-search": [ ], "neighbors": [ ], "inactive": { "ipv4-address": [ ], "ipv6-address": [ ], "route": [ ], "dns-server": [ ], "dns-search": [ ], "neighbors": [ ] }, "data": { "passthru": "0000000000000000000000000000000000000001" } }
{ "interface": "wg0", "up": true, "pending": false, "available": true, "autostart": true, "dynamic": false, "uptime": 1079353, "l3_device": "wg0", "proto": "wireguard", "updated": [ "addresses" ], "metric": 0, "dns_metric": 0, "delegation": false, "ipv4-address": [ { "address": "100.100.100.2", "mask": 24 } ], "ipv6-address": [ ], "ipv6-prefix": [ ], "ipv6-prefix-assignment": [ ], "route": [ ], "dns-server": [ ], "dns-search": [ ], "neighbors": [ ], "inactive": { "ipv4-address": [ ], "ipv6-address": [ ], "route": [ ], "dns-server": [ ], "dns-search": [ ], "neighbors": [ ] }, "data": { } } Save the above into a file named <test.json jsonfilter -a -e '@[@.proto="wireguard"].interface' would produce nothing, where the expected result would be |
@ynezz ping |
Same issue at openwrt/package/utils/jsonfilter/Makefile Lines 3 to 11 in 9dc46d6
Here's my sample data:
My temp workaround: add # /usr/bin/wireguard_watchdog
...
# query ubus for all active wireguard interfaces
# wg_ifaces=$(ubus -S call network.interface dump | jsonfilter -e '@.interface[@.up=true]' | jsonfilter -a -e '@[@.proto="wireguard"].interface' | tr "\n" " ")
wg_ifaces=$(ubus -S call network.interface dump | jsonfilter -e '@.interface[@.up=true]' | grep wireguard | jsonfilter -a -e '@[@.proto="wireguard"].interface' | tr "\n" " ")
... |
one can filter with one call to mitigate the bug in jsonfilter:
|
@ynezz I didn't quite understand the label release/22.03 I can confirm fully that the issue exists on 21.02.3 but does not on 22.03.2. |
Eliminate the extra call to jsonfilter to enumerate active WireGuard interfaces to avoid potential bugs and be more efficient. This change was originally proposed by @pputerla in openwrt#11649 as a workaround for openwrt#8703. Since 22.03.2 the issue seems to be fixed so @pputerla closed the pull request, but I think it's still very worth it as it is more efficient and robust. Signed-off-by: Rio <riobard@users.noreply.github.com>
This issue still exists in 23.05.2, today I ran into it. I fixed it by installing the “jq” package and replacing
with
I can also confirm that #12344 fixes my problem by using the following replacement:
I attached some example data that triggers the error: example_data.json
|
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: #8703 Fixes: #11649 Fixes: #12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io> (cherry picked from commit 33f15dd)
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: #8703 Fixes: #11649 Fixes: #12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io> (cherry picked from commit 33f15dd)
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt/openwrt#8703 Fixes: openwrt/openwrt#11649 Fixes: openwrt/openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
This bug appears to be fixed in today's snapshot build of jsonfilter 2024-01-23-594cfa86-1. Tested with a handful of previously problematic files and the test cases all work now. Below is my worst test, as snort produces randomly ordered output with this command and previously jsonfilter would produce somewhere on the order of 10-40 output lines with each invocation.
|
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt#8703 Fixes: openwrt#11649 Fixes: openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt#8703 Fixes: openwrt#11649 Fixes: openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt#8703 Fixes: openwrt#11649 Fixes: openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt#8703 Fixes: openwrt#11649 Fixes: openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io> (cherry picked from commit 33f15dd)
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt/openwrt#8703 Fixes: openwrt/openwrt#11649 Fixes: openwrt/openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io> (cherry picked from commit 33f15dd6d41873b02eb8895b8886763659f1390c)
013b75ab0598 jsonfilter: drop legacy json-c support 594cfa86469c main: fix spurious premature parse aborts in array mode Fixes: https://bugs.openwrt.org/?task_id=3683 Fixes: openwrt/openwrt#8703 Fixes: openwrt/openwrt#11649 Fixes: openwrt/openwrt#12344 Signed-off-by: Jo-Philipp Wich <jo@mein.io> (cherry picked from commit 33f15dd6d41873b02eb8895b8886763659f1390c)
arnysch:
Use attached file jinp1 as input and run this command:
jsonfilter -a -e '@[@.proto="wireguard"].interface' <jout1
=> nothing is output (empty output), which is incorrect
Now use attached file jinp2, which is jinp1 with lines slighty reordered:
jsonfilter -a -e '@[@.proto="wireguard"].interface' <jout2
=> "wgvpn" is output, which is correct, this time.
To this end it runs these commands:
ubus -S call network.interface dump | jsonfilter -e '@.interface[@.up=true]' | jsonfilter -a -e '@[@.proto="wireguard"].interface'
The second jsonfilter call in this line erroneously does not show a result.
The attached files jinp1, jinp2 contain example input for the 2nd jsonfilter call
The text was updated successfully, but these errors were encountered: