Error: Not signer by this signature (avail light node)

Nirnaeth · October 23, 2023, 6:59pm

I am running a Light client and synced from scratch. After synced I frequently see an error messages which is shown in the screenshot. How to diagnose and fix this?
OS: Ubuntu 22.04 (arm64)
Light node version: v1.7.2

config.yaml

log_level = “info”

http_server_host = “127.0.0.1”

http_server_port = 7000

secret_key = { seed = “avail” }

port = 37000

full_node_ws = [“ws://127.0.0.1:9944”]

app_id = 0

confidence = 92.0

avail_path = “avail_path”

bootstraps = [“/ip4/127.0.0.1/tcp/39000/quic-v1/12D3KooWMm1c4pzeLPGkkCJMAgFbsfQ8xmVDusg272icWsaNHWzN”]

sync_start_block = 0

log_format_json = true

dan · October 23, 2023, 8:14pm

Hi Nirnaeth, thanks for reporting.

Can you share a bit more about your whole setup, including the node? It looks like the light client is connecting to a local node. What version is the node, and what network is it a part of? Is there anything in the node logs, or any other details you might share? Thanks

Nirnaeth · October 23, 2023, 9:59pm

I have used this command to run my node
./avail-light-linux-aarch64 --config config.yaml --network biryani
I can not connect my node to kate network cuz it wont accepted as a parameter. I also did not check my bootstrap node and looks like it is a local node I just coppied from official setup steps and pasted it. As I mentioned using Ubuntu 22.04 and node version is 1.7.2

Nirnaeth · October 25, 2023, 8:36am

Update:
Now I discovered how to run it properly I think. I only parced sync_start_block = 0 variable via config.yaml cuz I want to keep the whole state of the network. Now sync started and everything is good so far. I used:

./avail-light-linux-aarch64 --config config.yaml --network biryani

command to start node and config.yaml is:

# config.yaml
sync_start_block = 0

Nirnaeth · October 25, 2023, 6:24pm

Update 2:

My light client crashed again like this:

2023-10-25T10:51:21.321056Z  INFO avail_light::sync_client: App index Ok(DataLookup { index: [(AppId(0), 0..1)] }) block_number=28426 
2023-10-25T10:51:21.321897Z  INFO avail_light::sync_client: Synced block header block_number=28426 elapsed=141.801823ms 
2023-10-25T10:51:21.327010Z  INFO avail_light::sync_client: Number of cells fetched from DHT: 0 block_number=28426 
2023-10-25T10:51:21.358866Z  INFO avail_light::subscriptions: Header no.: 253539 
2023-10-25T10:51:21.359092Z  INFO avail_light::subscriptions: New validator set: ([3d162da5f7fc62c8c89f12c5f856058a39999ea297275ad14b04a26d8323db6c (5DSoND2v...), 3cbcdaca210805aa6e91d23b3f47d7023050e12a0d8e04ff0790e4deba323ee2 (5DSLqCQ5...), 3d32d484ec90ac54f5ba76e17d80d50288237823daa0f4657e54abeb0de7dc9b (5DSwssJA...), 5aec12feb2d6bde49d6cb52a137fd804f9ee21f4e904c47eca6875458406dace (5E7vJ6DE...), e21e6d1e01a97ca335a9bc0a5a8f29ded049f4a6b57c1e54e9931e9218ea4612 (5HBBiaD2...), ad1d463182b7571294c69acc1d63d608957e1e550186dd511b63fd195c6ddc5a (5FygqXVL...), 461b509c94fed87b7c7d0140fac669ebbb6ea18c99b74b1251bb7637b648a7d2 (5DedKhrf...)], 235) 
2023-10-25T10:51:21.360210Z  INFO avail_light::subscriptions: New justification at block no.: 253539, hash: 0xcca638a1517ddef2870dc61a3e07b1025395b8a90bf53f316d71d01218ef1200 
2023-10-25T10:51:21.362695Z ERROR avail_light::subscriptions: Not signed by this signature! 
2023-10-25T10:51:21.364125Z  INFO Server::run{addr=127.0.0.1:7000}: warp::server: close time.busy=26.8µs time.idle=8518s 
Error: Not signed by this signature!

robin-rrt · October 30, 2023, 7:06am

Could you try spinning it up again, clearing the avail_path folder first, manually (a flag for that is coming up in the new version of LC).

Nirnaeth · October 30, 2023, 5:10pm

OK. Started from scracth. Current config.yaml is like this:

# config.yaml
log_level = "info"
http_server_host = "127.0.0.1"
http_server_port = 7000
secret_key = { seed = "avail" }
port = 37000
tcp_port_reuse = false
autonat_only_global_ips = false
autonat_throttle = 2
autonat_retry_interval = 10
autonat_refresh_interval = 30
autonat_boot_delay = 10
identify_protocol = "/avail_kad/id/1.0.0"
identify_agent = "avail-light-client/rust-client"
bootstraps = ["/ip4/13.51.79.255/udp/39000/quic-v1/12D3KooWE2xXc6C2JzeaCaEg7jvZLogWyjLsB5dA3iw5o3KcF9ds"]
relays = ["/ip4/13.49.44.246/udp/39111/quic-v1/12D3KooWBETtE42fN7DZ5QsGgi7qfrN3jeYdXmBPL4peVTDmgG9b"]
full_node_ws = ["ws://127.0.0.1:9944"]
app_id = 0
confidence = 92.0
avail_path = "avail_path"
ot_collector_endpoint = "http://otelcollector.avail.tools:4317"
log_format_json = false
block_matrix_partition = "1/20"
disable_proof_verification = false
disable_rpc = false
query_proof_rpc_parallel_tasks = 8
max_cells_per_rpc = 30
dht_parallelization_limit = 20
put_batch_size = 100
block_processing_delay = 0
sync_start_block = 0
sync_finality_enable = true
record_ttl = 86400
publication_interval = 43200
replication_interval = 10800
replication_factor = 20
connection_idle_timeout = 30
query_timeout = 60
query_parallelism = 3
caching_max_peers = 1
disjoint_query_paths = false
max_kad_record_number = 2400000
max_kad_record_size = 8192
max_kad_provided_keys = 1024

Nirnaeth · October 31, 2023, 10:45pm

Update

Now a different crash occured.

2023-10-31T21:21:41.285451Z  INFO avail_light::subscriptions: Header no.: 281296
2023-10-31T21:21:41.285967Z  INFO avail_light::subscriptions: New justification at block no.: 281296, hash: 0xa6f474416cf6fe8fab7c68bfb50c0e97a7bfe289fb6c12c2e9c4cb544dc7deca
2023-10-31T21:21:41.304303Z  INFO avail_light::subscriptions: Number of matching signatures: 7/7 for block 281296
2023-10-31T21:21:41.304504Z  INFO avail_light::subscriptions: Sending finalized block 281296
2023-10-31T21:21:41.304973Z  INFO avail_light::light_client: Processing finalized block block_number=281296 block_delay=0
2023-10-31T21:21:41.305093Z  INFO avail_light::light_client: Random cells generated: 4 block_number=281296 cells_requested=4
2023-10-31T21:21:48.819728Z ERROR jsonrpsee_core::client::async_client: [backend]: Networking or low-level protocol error: WebSocket connection error: connection closed
2023-10-31T21:21:48.823219Z ERROR avail_light::subscriptions: All senders dropped!
2023-10-31T21:21:48.824527Z ERROR avail_light::sync_finality: Cannot sync finality Couldn't get header for 0xa71d…1165
2023-10-31T21:21:48.824608Z ERROR avail_light::sync_finality: Cannot send error message: channel closed
2023-10-31T21:21:48.829837Z  INFO Server::run{addr=127.0.0.1:7000}: warp::server: close time.busy=46.8µs time.idle=9681s
Error: All senders dropped!

robin-rrt · November 1, 2023, 8:47am

I’ve tagged the team to look into this, they will get back with you on this soon

Nirnaeth · November 1, 2023, 9:17am

btw previous error just occured again

2023-11-01T04:51:21.611312Z  INFO avail_light::subscriptions: Header no.: 282642 
2023-11-01T04:51:21.611519Z  INFO avail_light::subscriptions: New validator set: ([3d162da5f7fc62c8c89f12c5f856058a39999ea297275ad14b04a26d8323db6c (5DSoND2v...), 3cbcdaca210805aa6e91d23b3f47d7023050e12a0d8e04ff0790e4deba323ee2 (5DSLqCQ5...), 3d32d484ec90ac54f5ba76e17d80d50288237823daa0f4657e54abeb0de7dc9b (5DSwssJA...), 5aec12feb2d6bde49d6cb52a137fd804f9ee21f4e904c47eca6875458406dace (5E7vJ6DE...), e21e6d1e01a97ca335a9bc0a5a8f29ded049f4a6b57c1e54e9931e9218ea4612 (5HBBiaD2...), ad1d463182b7571294c69acc1d63d608957e1e550186dd511b63fd195c6ddc5a (5FygqXVL...), 461b509c94fed87b7c7d0140fac669ebbb6ea18c99b74b1251bb7637b648a7d2 (5DedKhrf...)], 262) 
2023-11-01T04:51:21.612617Z  INFO avail_light::subscriptions: New justification at block no.: 282642, hash: 0x05bbd74e746eefd11706aedbd039f4104a313a0cfb7e6636d5c84cf676598907 
2023-11-01T04:51:21.615055Z ERROR avail_light::subscriptions: Not signed by this signature! 
2023-11-01T04:51:21.620930Z  INFO Server::run{addr=127.0.0.1:7000}: warp::server: close time.busy=47.2µs time.idle=21963s 
Error: Not signed by this signature!

Nirnaeth · November 10, 2023, 5:58am

Updated to 1.7.3 and joined goldberg network. So far no issues occured.

robin-rrt · November 10, 2023, 4:25pm

Thanks for the update. If you run into anything, please do post

Nirnaeth · November 13, 2023, 5:29pm

Hello. I want to inform you that I faced the same error on 1.7.3 (using default config) with aarch64 built ubuntu 22.04. Here the logs:

2023-11-11T18:58:07.637710Z  INFO avail_light::api::v2: Message published to clients topic=HeaderVerified published=0 failed=0
2023-11-11T18:58:23.577722Z  INFO avail_light::network::rpc::event_loop: New justification at block no.: 18159, hash: 0x847050473a42d131f6122a42d2e521e66adff539abd181748e048ab4cd92cf73
2023-11-11T18:58:23.578363Z  INFO avail_light::network::rpc::event_loop: Header no.: 18159
2023-11-11T18:58:23.582567Z  INFO avail_light::network::rpc::event_loop: Number of matching signatures: 8/10 for block 18159
2023-11-11T18:58:23.582749Z  INFO avail_light::network::rpc::event_loop: Sending finalized block 18159
2023-11-11T18:58:23.582969Z  INFO avail_light::api::v2: Message published to clients topic=HeaderVerified published=0 failed=0
2023-11-11T18:58:41.752152Z  INFO avail_light::network::rpc::event_loop: Header no.: 18160
2023-11-11T18:58:41.753894Z  INFO avail_light::network::rpc::event_loop: New justification at block no.: 18160, hash: 0x4c59833ad43aba61394628b5e4e6fabe96c654d0e2b9812c4e5a08eaf96682f2
2023-11-11T18:58:41.775249Z  INFO avail_light::network::rpc::event_loop: Number of matching signatures: 7/10 for block 18160
2023-11-11T18:58:41.775621Z  INFO avail_light::network::rpc::event_loop: Sending finalized block 18160
2023-11-11T18:58:41.775905Z  INFO avail_light::api::v2: Message published to clients topic=HeaderVerified published=0 failed=0
2023-11-11T18:58:58.807749Z  INFO avail_light::light_client: Number of cells fetched from DHT: 1 block_number=17339 cells_from_dht=1
2023-11-11T18:58:58.899865Z  INFO avail_light::light_client: Number of cells fetched from RPC: 3 block_number=17339 cells_from_rpc=3
2023-11-11T18:58:58.951375Z  INFO avail_light::light_client: Completed 4 verification rounds block_number=17339 elapsed=55.50933701s
2023-11-11T18:58:58.951974Z  INFO avail_light::light_client: Confidence factor: 93.75 block_number=17339 confidence=93.75
2023-11-11T18:58:58.952383Z  INFO avail_light::light_client: Partition cells received block_number=17339 partition_retrieve_time_elapsed=364ns partition_cells_fetched=3
2023-11-11T18:59:03.461943Z  INFO avail_light::network::rpc::event_loop: New justification at block no.: 18161, hash: 0xe00a9572e45b48d89580227f77cc9655c5b5e811f13d586659af36da297a30b5
2023-11-11T18:59:03.464171Z  INFO avail_light::network::rpc::event_loop: Header no.: 18161
thread 'tokio-runtime-worker' panicked at /home/runner/work/avail-light/avail-light/src/network/rpc/event_loop.rs:310:57:
called `Result::unwrap()` on an `Err` value: Not signed by this signature!
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

robin-rrt · November 15, 2023, 8:30am

Thanks for flagging this. We’re trying to solve this as we speak.

fLuXx · November 21, 2023, 8:39am

Should be fixed in the upcoming release. If you happen to see it again, please report again, as it should not happen again.

Topic		Replies	Views
Error: Failed to receive error message (Light Node) Q&A/Troubleshooting	1	161	November 15, 2023
WARN avail_light::network::rpc::subscriptions: Signature verification fails with default set_id 215, trying alternatives Q&A/Troubleshooting	1	134	March 18, 2024
ERROR avail_light: Bootstrap process: 0: the request timed out Developers	2	249	February 12, 2024
Bash Shell Install Fullnode & Light Client Node Operators clash-of-nodes , goldberg	3	195	March 29, 2024
Starting Light Client failed Light Client	1	236	April 5, 2024

Error: Not signer by this signature (avail light node)

config.yaml

Related topics