#339 dotreasury maintenance from 05.2021 to 07.2021 and part of M3 features development
Copied from post 816.
The MA30 price at subscan is 203.077 at my proposal time, so we request 7850/203.077 = 38.66 KSM.
dotreasury aims to introduce retrospect mechanism for Kusama/Polkadot treasury, while you can see it as a treasury explorer. Check here for this project history.
Maintenance
Server
Server fees for 05.2021 and 07.2021: (8vCPU + 16G + 1 TB SSD + 10Mbps) * 3 = $450 * 3 = $1350
services run in this server:
- 2 polkadot archive node instance for Polkadot and Kusama
- 1 polkadot-collator archive node instance for Statemine
- 1 mongodb instance
- dotreasury
- scan scripts for production and staging environments
- dotreasury restful server for production and staging environments
- metadata scan scripts for production and staging. We use these scripts to scan store the raw block/events data, so we don't have to read them from the node.
- Statescan
- scan scripts for production and staging environments
- Statescan restful server for production and staging environments
- Price tracker scripts
- Some other scripts like monitors
We'd say this server is a little tired, and we may increase 1 or 2 servers in next season.
Continuous development
We are still actively developing to improve it. Please check the code diff from last delivery. Our work includes:
- We improved the way to check the target calls recursively, so we won't miss some actions without events, check code here
- Improve page performance by only connecting 1 node when page init
- Add funded projects and related links for proposals manually
- Continually track the polkadot.js releases and update dotreasury dependencies to make it work for latest runtime
- Some UI improvements and bugs, like show voting members on proposal list, fiat value punctuation, link recognition, etc.
We request 10 days cost for 1 developer, and it's $500 * 10 = $5000.
Part of M3 work
You can see the original M3 plan here and a discussion we raise about beneficiary credit model. We still can not decide the best model for a credit score, but we can first let the grade action happen. That means councilors and any token holders can grade 1-5 for a proposal and leave a short message, and they have to sign it with polkadot.js extension which we will store to IPFS. Of course anybody can see these grades with the verified signatures.
We request $1500 for this feature development.
Metrics
Please check metrics from cloudflare. The visitors are not so many. Sorry that our social media followers are very few and maybe we need some work to promote it.
Total cost
The work is for both kusama and polkadot, but to make the on-chain work simple, we apply from kusama and polkadot in turn every 3 monthes.
+-----------------------------------------+---------------+-------+
| Description | Time | Cost |
+-----------------------------------------+---------------+-------+
| Server(8vCPU + 16G + 1 TB SSD + 10Mbps) | last 3 months | $1350 |
+-----------------------------------------+---------------+-------+
| Continuous maintenance development | last 3 months | $5000 |
+-----------------------------------------+---------------+-------+
| Support grading and IPFS storage | 2 weeks | $1500 |
+-----------------------------------------+---------------+-------+
| Total | | $7850 |
+-----------------------------------------+---------------+-------+
I will calc the KSM amount when propose it on-chain with the MA30 price if no problems.
OpenSquare team, we develop public goods and are dedicated to facilitate the collaboration between projects and developers.
Show More
Overall 100 % of users are feeling neutral. Thank you for your response! We appreciate your insights on optimizing costs in a decentralized environment. Your suggestion of using smaller Kubernetes (K8S) clusters to save costs and improve flexibility is an interesting approach. It's true that companies like AWS, Azure, or Google may offer more options and advanced managed features, but they can be costlier compared to alternatives such as Hetzner. Considering the potential benefits of a small K8S cluster, it could indeed provide you with more horsepower for roughly 1/4 of the costs while reducing the risk of having all your...See More
AI-generated from comments
Do you mind mentioning a word about your cloud costs? You mention $450 and I see $80 in my "pricing list" for specs close to what you mention.
You also mentioned:
We'd say this server is a little tired, and we may increase 1 or 2 servers in next season.
Could you be more concrete? I fail to see where the all load is required and I feel you could optimize your costs (and the treasury costs on the way) using for instance a 2-3 smaller nodes K8s. The achive nodes costs in block storage (those costs can be split) but the rest of the load does not seem that high.
Hi, @chevdor , thanks for your question.
We use the Ali-Cloud ecs service. A screen shot of the one month fee can be checked bellow, and feel free to login to it and pick the same spec server. Sorry, the site has not English translation for this part. It's 3365RMB, about $519.
To reduce the cost, I paied 1 year cost in advance so I can get a discount, please check my payment bellow. Honestly we added a develop server for some routine develop work, and sorry I just ignored many details to save editting time. In total, one month average fee is over $400. I took the risk to not be funded by treasury, so make it up to $450.
Maybe the luxury part is the bandwidth. 10M maybe currently too much for current visitors. I asked some friends that AWS maybe more expensive for same spec cloud server. Can I know your cloud server vendor, so I can try and switch to that.
The MongoDB took too much memory, especially when we scan the block data and do insertion very frequently. We tried a MongoDB cloud service, but finally abandon it since it will be more expensive than that server itself.
Thanks for your answer @opensquare
Don't worry, I totally understand that optimizing costs is not a priority when a project starts. But the treasury is funding several projects and needs to keep an eye on the costs and honnestly, I prefer the treasury to pay developer work than 3rd party clould providers (whatever they are) too much when we can avoid it.
I could not see any of the screenshots you mentioned.
From the description of your infra, it sounds to me that you could save and get ready to be more flexible with the use of a small K8S cluster that would also help you scale better later.
Look at DO for instance (this is NOT a referral link...). The machine I linked costs $40/ month. Make it a 3 nodes cluster and you have more horse power than your current config for roughly 1/4 of the costs. You may need a couple of extra goodies such as a loadbalancer but you would have to add lots of storage to reach $450. Another benefit is that you no longer have all on one single machine (that may fail...).
There are even cheaper options but you will then tap more into the "DIY" solutions that involve more to do on your side (which has a cost as well).
I could not see any of the screenshots you mentioned.
Sorry, they only can see by login first and selecting the target ecs, and the other one can not be reached in the site since it's my private bill.
Anyway, I'll try DO and thanks for your suggestion.
In general, you will find that companies like AWS, Azure or Google are more expensive. The main reason is that they provide more options and advanced managed features. If you need those, this is great, if you don't however, some alternatives may be cheaper.
For the single server option, Hetzner is very competitive. I suggested DO above as I think they are a nice middle option between raw metal server (where Hetzner will be cheaper and more powerful) and something like AWS with more "managed" option but it does not sound you use those.
It's a bit late for this now but OnFinality would be able to provide you with API keys for unrestricted access to our Polkadot, Kusama, and Statemine archival nodes. This would allow you to dramatically reduce your ongoing costs (i'm guessing the archive nodes are the bulk of the costs for your infrastructure) as well as allow the talented team at OpenSquare to focus more on your product.
We're both supported by the treasury and have the same goals, so it makes no sense to duplicate work or resources here?
Additionally, since we have multiple clusters around the world, your users would receive better performance when the connect to the node on page init, since we load balance that connection to the nearest cluster?
@james_bayly Thanks.
Currently we run an archive node just in one machine with the scan scripts for the lowest network latency. Actually we tried onFinality and Patract public nodes, still slow in our case. Maybe some kinds of whitelist can solve some problems. Our server is in Singapore and vendor is Ali Cloud.
While I encourage the reuse of source and libs and the offer from @james_bayly may indeed help cut the costs, this is a rather different option.
In a decentralized and trustless environment, it does make sense to use your own nodes.
Also going for a pruned vs archive node does make a difference today and this difference will increase over time.
Dotreasury support grading now!
You may know dotreasury as a explorer or dashboard for Kusama|Polkadot treasury. Actually it's more than that, aiming to introduce retrospect mechanism for current treasury. Please check the post 352 for the original discussions. In general we have following opinions:
- ROI should be measured for treasury expense.
- Every community members should have ways to evaluate treasury expense, while the evaluation should be well saved and verified.
- It's better to create a transparent and fair way to measure most participants' credit which will help the treasury and even the whole ecosystem long running.
We raised a discussion about how to build the credit score and received some very good suggestions. But we prefer to let the grade behavior happen before a maybe biased score, so we implemented grading first with the part fund of proposal #103.
Ecosystem fit
Again please check here for the original ideas. It will help the whole ecosystem by following means.
- Provide a way for anyone(councilor, external reviewer, community member)'s feedback to be stored permanently and easily verified.
- Some post-evaluation mechanism will give a supervision to beneficiaries for better delivery.
- Previous verified evaluation will bring some reference to new enrolled councilors, cost for new evaluation will be reduced.
- Councilors approve or reject proposals, it provide a way for community members to give their judgement for proposals(even before council motion).
- It will become easier for anyone to see what kinds of proposal, and what work is welcomed and get high evaluation.
How to grade
Currently we support grade treasury proposals and projects. So for Kusama，
- Open proposals page or projects page.
- Choose a proposal or project and open the detail page, and you'll see the grading panel.
- Evalulate the proposal or project, and click the stars next to the 'Your grading' text.
- Leave some reason for your grade if you want to.
- Click the 'Sign' button, then it will raise the polkadot extension and require your sign.
- Your signed msg will be stored to IPFS and listed bellow. Sample for project, Sample for proposal.
Note:
- No account registraction required, just a Kusama|Polkadot account.
- Your 2nd grading will override the 1st one.
- Currently we didn't implement methods for canceling grading.
- We suggest to grade the project if there are several proposals funded for one project. Of course you can still grade one of the proposals for the partial work.
Call for candidates
Maybe it's always sensitive to evaluate others, but OpenSquare will be very glad to be the 1st candidate.
Check our funded projects:
Dotreasury proposals:
Statescan proposals：
We will be appreciated for any grading to OpenSquare, and don't worry, we will not complain that :).If any beneficiaries want to be the candidates, maybe just leave your project or proposals as comments.
Worries or FAQs
-
Engagement Yes, we worry a little about it. It's always not easy without token incentive. We hope councilors give grades at your ease.
-
Future score Maybe start from some simple and easy method as experiment, but let's foucus on this first. It's meaningless to calculate score with very few opinions.
-
Bounties or tips Currently we didn't implement them because bounty is so less and tip is usually small, but we can add them if there are expected.
Feedback: it would be nice if grading was shared or at least could be viewed simultaneously between "Projects" and "Proposals" and if Projects have multiple proposals.
Discover similar proposals
