Message: RPC Subscription stream should be infinite

Developers Q&A/Troubleshooting
1

I am running avail light client on ubuntu 22.04 amd64 architecture LC version is v1.7.4. My client crashed like this:

2023-12-15T10:33:49.633435Z  INFO avail_light::network: Cells fetched from DHT block_number=14448 cells_total=8 cells_fetched=0 cells_verified=0 fetch_elapsed=10.002151687s proof_verification_elapsed=203.551µs
2023-12-15T10:33:49.754989Z  INFO avail_light::network: Cells fetched from RPC block_number=14448 cells_total=8 cells_fetched=8 cells_verified=8 fetch_elapsed=110.676771ms proof_verification_elapsed=10.789979ms
2023-12-15T10:33:49.755207Z  INFO avail_light::sync_client: App index Ok(DataLookup { index: [(AppId(0), 0..1)] }) block_number=14449
2023-12-15T10:33:49.755222Z  INFO avail_light::sync_client: Synced block header block_number=14449 elapsed=16.831µs
2023-12-15T10:33:49.755257Z  INFO avail_light::api::v2: Message published to clients topic=ConfidenceAchieved published=0 failed=0
2023-12-15T10:33:50.851187Z  INFO avail_light::network: Cells fetched from DHT block_number=14449 cells_total=8 cells_fetched=0 cells_verified=0 fetch_elapsed=1.095730806s proof_verification_elapsed=213.59µs
2023-12-15T10:33:50.946020Z  INFO avail_light::network: Cells fetched from RPC block_number=14449 cells_total=8 cells_fetched=8 cells_verified=8 fetch_elapsed=83.210699ms proof_verification_elapsed=11.595919ms
2023-12-15T10:33:50.950271Z  INFO avail_light::sync_client: App index Ok(DataLookup { index: [(AppId(0), 0..12)] }) block_number=14450
2023-12-15T10:33:50.950282Z  INFO avail_light::sync_client: Synced block header block_number=14450 elapsed=12.393µs
2023-12-15T10:33:50.950312Z  INFO avail_light::api::v2: Message published to clients topic=ConfidenceAchieved published=0 failed=0
2023-12-15T10:33:50.958860Z  INFO avail_light::network::p2p::event_loop: Cell upload success rate for block 14448: 8/8. Duration: 1
2023-12-15T10:33:52.046668Z  INFO avail_light::network::p2p::event_loop: Cell upload success rate for block 14449: 8/8. Duration: 1
The application panicked (crashed).
Message:  RPC Subscription stream should be infinite
Location: /root/avail-light/src/network/rpc/event_loop.rs:217

My LC was syncing from block 0. I have also created a github issue which includes the crash report too. Hope this helps.

2 Likes
2

I’m not sure but it could be about your bandwith capacity

3

Hmm how can I confirm this. The machines is dedicated one and another node runs with different port communications. I dont think that they are conflicting each other.

4

Looking at the logs, I don’t think there is a conflict either, but 2 different nodes may have increased the network traffic and caused the bandwith capacity to be exceeded. :thinking:

5

I dont think so cuz its 1Gbps. I saw 1.7.5 stable release will update and observe

1 Like
6

I see if you say so looks like it’s not about that, lets ask devs

7

I did not updated to 1.7.5 but I synced with the main branch and built new binary. Now sync speed is blazing fast and no issue with this brand new 1.7.4 binary. I will keep observe and close this if no issue comes up.

1 Like
8

Glad that works, sounds great! :+1:

9

Issue still remains but it is very rare.

10

Issue came back again with latest version (v1.7.6) amd64 ubuntu 22.04

2024-02-12T11:25:49.965889Z  INFO avail_light::network::rpc::event_loop: New justification at block no.: 406872, hash: 0xb5fa792b6f785146f6a308aceca8348cec9422f9cb99ad17beabda02909e4a7e
2024-02-12T11:25:50.025922Z  INFO avail_light::network: Cells fetched from RPC block_number=406871 cells_total=10 cells_fetched=10 cells_verified=0 fetch_elapsed=192.246026ms proof_verification_elapsed=59.104482ms
2024-02-12T11:25:50.026302Z ERROR avail_light::light_client: Failed to fetch 10 cells block_number=406871
2024-02-12T11:25:50.026614Z  INFO avail_light::api::v2: Message published to clients topic=ConfidenceAchieved published=0 failed=0
2024-02-12T11:25:51.088296Z  INFO avail_light::network::rpc::event_loop: Header no.: 406872
2024-02-12T11:25:51.089310Z  INFO avail_light::network::rpc::event_loop: Number of matching signatures: 5/7 for block 406872, set_id 184
2024-02-12T11:25:51.089331Z  INFO avail_light::network::rpc::event_loop: Storing finality checkpoint at block 406872
2024-02-12T11:25:51.089561Z  INFO avail_light::network::rpc::event_loop: Sending finalized block 406872
2024-02-12T11:25:51.089654Z  INFO avail_light::light_client: Sleeping for 14.998662506s seconds
2024-02-12T11:25:51.089846Z  INFO avail_light::api::v2: Message published to clients topic=HeaderVerified published=0 failed=0
2024-02-12T11:25:54.671171Z ERROR avail_light: Error: The application panicked (crashed).
Message:  RPC Subscription stream should be infinite
Location: /home/runner/work/avail-light/avail-light/src/network/rpc/event_loop.rs:248

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
2024-02-12T11:25:54.671197Z  INFO avail_light::network::p2p::event_loop: Exiting event loop...
Error:
   0: Running Light Client encountered an error
   1: Panic occurred, shuting down

Location:
   src/bin/avail-light.rs:501

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
11

Try the latest version found here please: v1.7.9

1 Like
12

I am monitoring the light client for some time. Working without any issue looks like issue is fixed!

2 Likes
13

Awesome! That’s great news

1 Like