Rejected

Requested:

2.77K KSM

#194 Monitoring of public RPC/WSS Endpoints

Proposer:

HqRc...fVZn

in Democracy

Beneficiary:

(2.77K KSM)

16th May '23

(Edited)

Description

AI Summary

Votes Bubble

Timeline

Evaluation

On Chain Info

Stats

Dear community,

EDIT: The proposal now seeks $32,640.00 for RPC monitoring only.

I am resubmitting a proposal for Monitoring of public RPC/WSS Endpoints & Node Database Analytics. This proposal was previous submitted as referendum 175 but failed due to HANC's interference.

Unable to gauge his feedback, I have made the followings revisions:

I am funding provision of servers for the DB Analytics/Restoration, this reduces the overall cost of the proposal by ~25%
Reduce the slippage from 10% to 5% in light of a recent market correction

Do note that slippage as well as any excess KSM beyond the asking USD amount is refundable to the treasury.

The goals of the proposal remain the same:

To become an independent monitor of all WSS providers, displaying key metrics in a visual manner (Dashboard)
To monitor DB growth, sizes, bloat and restoration times for various DB configurations.

Kind Regards,

Users are saying...

Based on all comments and replies

Overall 50 % of users are feeling optimistic. A strong Aye for the OpenGov treasury proposal, as Will/Paradox is a dedicated member with a proven track record of contributing to the ecosystem. The focus on development and talent acquisition makes the project valuable and low-risk, with potential for a good return on investment. Will has the necessary skills and commitment to make this project a success, and the community stands to benefit from his expertise. Good luck, Will!

Overall 25 % of users are feeling neutral. In a recent discussion, a proposal to monitor RPC nodes in the Kusama network has sparked debate. The proposal aims to improve performance and reliability by monitoring various metrics such as connection speeds, database sizes, and restoration times. While some community members support the initiative as a necessary step, others question the need for such monitoring and its associated costs. The proposer has responded by adjusting the scope of the proposal and clarifying its intentions, emphasizing the importance of ensuring a high-quality user experience in the network....

Overall 25 % of users are feeling against it. Voting against ref. 175 and discussing unnecessary monitoring of IBP nodes and database sizes for Kusama Governance. Opposing the allocation of funds for over-optimization and excessive monitoring that does not provide significant value. Rejecting the proposal to fund RPC dashboard and additional features through Kusama treasury. Switching from Nay to Aye only after hardware/database portions were dropped from the proposal.

AI-generated from comments

FLKB...svZR

16th May '23

Voted Aye

You got my Aye support, good luck, Will

StakerSpace

17th May '23

(Edited)

We voted Nay on ref. 175 and I had a personal discussion with Will (Paradox) on Element that I would provide information in the next referendum re-post which will also be again be a Nay, despite the overwhelming community support.

There is the IBP network monitor that checks whether nodes are healthy and on the right version their ping-times. Pretty much a complete dashboard. Even though is checks only the IBP nodes. This passed Kusama Governance in ref 35 and funded the most performant nodes and as the community go-to servers for all relay-chains and system chains. My point being I don't believe there is a need to monitor all public WSS nodes, this I believe is a waste of the KSM treasury funds and I think it won't have any traction except for the RPC/WSS speedfetishj geeks, that would last for 1 min of fun.
The other main functionality other point to monitor the DB sizes, restoration times of various DB configuration, which doesn't even cover the default setting of the node software in the proposal(256 blocks). Let's say I'm a validator and I require the last 256 blocks of state pruning. On our nodes this is currently ~250GB, if I’m looking to be a member of the 1KV member I’m not going to rent a server with exactly 251GB storage. No you rent/buy something that can last for 1-2 years minimum if you’re serious about it. And the state doesn’t jump +10GB a day. These values can be easily written down somewhere and last for 1-2 years as valid numbers. As is the case now in the Polkadot Wiki This can always be expanded of course by community members.
Regarding the restoration times, this can vary from 5-15 minutes depending on your hardware and connection. Don’t think exact numbers are needed here and would be just statistics that I don’t believe is worth 2700 KSM

If anyone want’s to dig deep into these numbers and all KSM/DOT community members do on a daily rate, we can update the wiki or create a new section that needs community input. But running 24 different database sizing options is would be in my opinion again a waste of KSM public funds over it's value add

Hide replies

HqRc...fVZn

17th May '23

Hey StakerSpace,

We voted Nay on ref. 175 and I had a personal discussion with Will (Paradox) on Element that I would provide information in the next referendum re-post which will also be again be a Nay, despite the overwhelming community support.

Thank you for taking the time to respond in light of voting against as you phrased it, overwhelming community support.

Point 1:

There is the IBP network monitor that checks whether nodes are healthy and on the right version their ping-times. Pretty much a complete dashboard. Even though is checks only the IBP nodes. This passed Kusama Governance in ref 35 and funded the most performant nodes and as the community go-to servers for all relay-chains and system chains.

The scope of this project not only checks ping times but other metrics associated with performance and compliance. Examples of this would be execution times, connection saturation, metadata and block depth checks. There's more than just being able to connect, it is how well the service performs and does it meet the requirements of the "unwritten" SLA.

I stand to be corrected but please also note that the allocation granted to the IBP for this task is $75,000 with another $254,000 for other development. The scope of the project I am presenting is greater than basic monitoring, it is imo more holistic and comes in at a lower cost. This is in no way showing a lack of appreciation for what the IBP is trying to do and many of the members support this proposal as an alternative check.

The consensus of one is never a good thing, as we've seen with ref 175.

My point being I don't believe there is a need to monitor all public WSS nodes, this I believe is a waste of the KSM treasury funds and I think it won't have any traction except for the RPC/WSS speedfetishj geeks, that would last for 1 min of fun.

I am presenting a solution based on my experience both evaluating, using and hosting said services. From a development standpoint we need to rely on a standard for these nodes, we can't have pruned nodes in operation and we need some conformity when it comes to metadata. Quality of service matters and this is not only ping related.

Point 2

The other main functionality other point to monitor the DB sizes, restoration times of various DB configuration, which doesn't even cover the default setting of the node software in the proposal(256 blocks). Let's say I'm a validator and I require the last 256 blocks of state pruning.

To my knowledge and as-was the default for snapshots, validators utilize pruning of the last 1000 states. In an effort to reduce on the number of configurations I omitted 256 state pruning as it is rarely used in practice. I am actually shocked that you use otherwise. Per my previous experience and a show of how this analysis helps, the size different between state pruning of 1000 and 256 is negligible.

I also challenge you with regards to the growth of even pruned databases, as we have seen in the past this growth can and has increased to almost 450GB+ per instance. This 'issue' was transitory but caused problems for some resulting in chilling. This is another one of those 'from experience' points that as I validator I hoped that you would understand.

I am a contributor to the Wiki myself and a person who lends support to the new entrants. The more metrics that can be extracted from real-time data helps keep the wiki relevant. We need to think smart and we have seen the benefits of this in other areas like extractions for chain constants.

Point 3

Regarding the restoration times, this can vary from 5-15 minutes depending on your hardware and connection. Don’t think exact numbers are needed here and would be just statistics that I don’t believe is worth 2700 KSM

This is somewhat misrepresenting the figures. 2700 KSM is not allocated to restoration times alone, 2700 KSM encompasses the entire scope including points that you've raised in 1 and 2. I have also utilized one of your references to demonstrate that there are larger budgets allocated to smaller scoped projects.

Speaking directly to the point, monitoring of restoration times is useful especially when your time for reacting without consequence is < 1hr. We can not run on the assumption that the restoration time is constant and only identify deviation at the last moment. That is not good IT.

Last point:

If anyone want’s to dig deep into these numbers and all KSM/DOT community members do on a daily rate, we can update the wiki or create a new section that needs community input. But running 24 different database sizing options is would be in my opinion again a waste of KSM public funds over it's value add

This focuses on a part of the proposal which I am now funding a significant portion of the cost myself. I have expressed the value of having real-time updates to the Wiki, previous cases in which things did not go right and DB sizes increased beyond 'norm' by significant amounts.

Overall your feedback has demonstrated that I need to explain the technical aspects more, even to those who I may have assumed to have technical knowledge. As I mentioned before, I am a regular user, developer, provider and curator so I do have an appreciation for the need from different facets. I'll try to improve on this.

Thank you for your time, I appreciate the opportunity to respond.

HqRc...fVZn

17th May '23

Voted Aye

There is some feedback that light-clients is the way-to-go and that supporting this proposal is in some way not supporting light clients. I assure you this is not the case and monitoring of bootnodes is on the roadmap at no additional cost.

I firstly ask, if you are a supporter of light clients are you using it now? If so what is your experience with it and do you think it is ready for the main stage? The answers would vary depending on your role within the ecosystem, I am confident to say that MOST consumers still use RPC nodes and that most would not enjoy the daily light client experience.

Like many I would like to see a world where light clients are used more frequently but it is not the case right now. Light client implementation on Polkadot.JS (at least) is very slow and substrate connect utilizes bootnodes provided by one entity (Parity). As light client technology matures we should see improvements with usability and we should see more bootnodes added. The focus of the project is intended to shift towards monitoring same including other aspects that some take for granted like SSL peer availability which is also important. These things are already on the roadmap!

The lack of readiness is not only my opinion but one that was shared in the discussion of the previous referendum. It was also discussed by Tom during the last AAG session. It is easy to say "we don't need RPC nodes, we need light clients" but it is not the reality nor do I see it being the case for the next 12 months (at least).

StakerSpace

17th May '23

(Edited)

as you phrased it, overwhelming community support.

Just to correct, this was your phrase on Element :)

consensus of one is never a good thing

I understand this HACN user is a hit-piece and a good meme, but maybe this user is also the other sound/side of this gang-gang behaviour that is forming in this community that has lost its self in micro-processes of RPCs and trying to squeeze performance and create dashboards where effort and funding exceeds to result. Plus there are plenty of RPC's that have worked at all times in Polkadot.JS. and where IBP is already holding them to an SLA. So why do we need to monitor the unwritten SLA's?

execution times, connection saturation

these sound like an extension of connection speeds/health. As I said the funds are used for the over optimisation that is not needed.

validators utilize pruning of the last 1000 states. In an effort to reduce on the number of configurations I omitted 256 state pruning as it is rarely used in practice. I am actually shocked that you use otherwise.

No need to be shocked it's the default setting of polkadot node software with --validator flag on...

I stand by what I wrote in my first comment. It's great you picked it from the start when none asked you to create a DB growth table, but extending that into a treasury proposal with extra features and a holistic RPC dashboard is not what I think should be funded by the Kusama treasury.

Hide replies

HqRc...fVZn

17th May '23

Hey Staker Space,

Nice to hear from you again. There's some retort's in your response that I don't think would yield for a good discussion so I'll try to reply to the any points that you've made.

Plus there are plenty of RPC's that have worked at all times in Polkadot.JS. and where IBP is already holding them to an SLA. So why do we need to monitor the unwritten SLA's?

The problem with WSS nodes is a concern, at least it is for Jaco who is now putting further checks on RPC providers before they are added to Polkadot.JS. We need to ensure that we are receiving good service, you may not see this but those interacting with the services see it. The IBP is a good programme but a singular provider which would also be monitored by this service. This proposal is not meant to shame anyone but to help identify issues.

these sound like an extension of connection speeds/health. As I said the funds are used for the over optimisation that is not needed.

It actually isn't, some of these things are the reason why you may have a bad Polkadot.JS experience or transactions take a long time to execute. We tend to think "Kusama's running slow" but it can be something else. Again, this is meant to help up-lift and not destroy.

No need to be shocked it's the default setting of polkadot node software with --validator flag on...

Yes, but many also maintain the --pruning=1000 option for compatibility with snapshots. I think this could be a preference thing but I am very confident that many use --pruning=1000 option or equivalent thereof with their validators. To invalidate the underlying point, I'll give consideration to adding --pruning=256 to the proposal :D

It's great you picked it from the start when none asked you to create a DB growth table, but extending that into a treasury proposal with extra features and a holistic RPC dashboard is not what I think should be funded by the Kusama treasury.

When I first displayed this, the reaction was positive and there were those asking for additional functionality. To your remark about funding, a then council member (JAM) indicated that if I ever required funding that I should seek it via Treasury Proposal.

E5qF...tqrg

25th May '23

The team stopped by AAG on May 22. See the clip! 👇 (9:36) https://twitter.com/TheKusamarian/status/1661770597792088065

Full replay here.

HqRc...fVZn

26th May '23

Voted Aye

Dear community,

I appreciate that the value the second part of the proposal is contentious and given that this is a tight vote, I am removing it from the scope. The proposal now seeks** $32,640.00 for RPC monitoring only**.

It is my intention to proceed with this referendum but with anticipation of revised funding. Any excess would be refunded to the treasury within 12 hrs of receipt. I believe that my years of service to the community affords me a level of trust that I would make good on my promises.

CC: Tom.StakePlus, HACN

Regards,

HERT...afdU

26th May '23

Voted Aye

We were against the database growth portions of this being funded by treasury and brought this up during discussions. Since Will has dropped the hardware / database portions of this and is only seeking funding for the RPC portions we have switched our vote from Nay to Aye.

EJYe...YgVL

26th May '23

There are countless reasons why one could vote for or against this (or many other) proposals.

However we would like to add the following angle:

Will/Paradox is already a valued member, bound to our community, and this project seems to be more a Development project than an infrastructure project, which is the value we have been receiving from Paradox so far.

Projects come, go, and evolve. but the core asset is people and talent. We are very happy to see that Paradox is taking on this new project on a Development space, and hope we can benefit from Paradox on this dimension too.

As I understand Will has a degree in Computer Science and will use this project as a way to migrate existing competence to our opensource stack.

If we are going to do a purely economic assessment of the project, then it is only fair to include typical costs of talent acquisition and talent development. Then the cost of the project is very reasonable. And because Paradox is already bound to our community the investment in talent is likely to remain with us.

Considering these points we see little risk in this proposal. One way or another, I think it is very likely that we will get return on our investment.

EwR2...jtt4

27th May '23

Voted Aye

We are a strong Aye for two reasons.

Will/Paradox is a committed dotsama member. He has been steadfast and passionate in expanding the ecosystem by shepherding new members(including when we were newbies). We believe - we know, his development will be both judicious in the use of money and will be for the common good.
As an Infrastructure provider ourselves, we understand the value in a monitoring system. A neutral monitoring system can help defend and improve the quality of the infrastructure - which is the foundation we believe Blockchain applications will build on.

HUj6...GLD6

13th Aug '23

Hello,

We are in the process of validating a true need for a service to assist teams with crafting and completing successful treasury proposals, so they can focus on building. We would love to hear about your experience with this proposal. If you are willing to take a few minutes, please fill out this form about your experience with the OpenGov treasury proposal process: https://forms.gle/MwDij4adXEQd7Um79

Feel free to leave out any details that your team is not comfortable with sharing, but the more info you can provide, the better we will be able to assess the potential need for our services.

For more info, follow us on Twitter/X: https://twitter.com/OpenGovAssist

Discover similar proposals

#509

E5qF...tqrg

Deciding