Subsquid - Public data indexing Infrastructure for Polkadot and Kusama (Q3 2022)
Proponent: Subsquid Lab Official - FAyqRVLZmXVUU9pNeG8w8SxS5xRgN8FRGeWLL6u9S32TMHc
Short description: Ongoing costs for the running of high performance, scalable, and reliable data indexing infrastructure for most of the Polkadot and Kusama ecosystem (and growing).
Raw data: Archive Infrastructure metrics - July, August, September
Requested KSM: 1,447.28 KSM*
*added 33.33 KSM due to bond associated with our proposal in governance 1.0. This proposal supersedes the proposal made with governance 1.0 = https://kusama.polkassembly.io/treasury/246 . Further we added another 6.66 KSM due to an issue with u128 conversion for the requested amount in Polkadot JS.
Previous proposal:
https://kusama.polkassembly.io/treasury/171
https://polkadot.polkassembly.io/treasury/134
Motivation
Archives are an important piece of Polkadot and Kusama data infrastructure provided by Subsquid. They provide access to on-chain data in GraphQL format for all major parachains on Kusama and Polkadot, with more chains getting support as the ecosystem grows. They are being used as a data source for Squids and for efficient data exploration and ad-hoc queries. An up-to-date list of Archives is kept in a public Archive Registry
Achievements
At the end of Q2 Subsquid released a new generation of Archives, together with a major version of the SDK, both code named “Fire Squid”.
The improvements brought by this release include:
- Substantial performance increase, with the ability to deploy a new Archive for a chain as big as Kusama, and the indexing to be ready in roughly a day
- Fire Squid Archives support batch requests, which lead to a hundredfold increase in performance of indexing project connecting to them (squids)
- Fire Squid Archives expose two separate gateways, configured and tuned for different uses (one for data exploration by humans, the other one specifically for programmatic and batch access)
- Fire Squid Archives “unwrap” proxy, sudo and batch calls, tracing the origin, but making them much easier to access them
Subsquid has also engaged in continuously improving the product, iteratively working on adding features, such as EVM Transactions indexing, and subscriptions to queries in the Graphql endpoints of indexing projects created with the Subsquid SDK (commonly called squids).
Statistics
Between 2022-09-03 (when tracking has started) and 2022-09-30 Subsquid’s Archive infrastructure has collectively:
- Served a total of 129,468,038.2 (~129 million) queries in the month of September alone
- Our 7 day average peaks above 5 million queries and adjusts around 3.3 million queries per day at the end of the period
Data from the month of October confirms the linear trend, allowing us to make assumptions for the months of July and August, given the steady growth of queries over time shown.
Full report here
Comments (3)
Requested
Proposal Passed
Summary
0%
Aye
0%
Nay
Aye (259)0.0 DOT
Support0.0 DOT
Nay (8)0.0 DOT
Hi, regarding this section "Further we added another 6.66 KSM due to an issue with u128 conversion for the requested amount in Polkadot JS" - This is actually refundable to you at the end of the referendum even if you made a mistake with that referendum. You can claim it back or someone else can claim it on your behalf and it goes back to the depositor's address. As for the rest of your proposal, I'm biased because I'm an advocate of anything to do with Databases and Data Analysis as I used to work in that sector so I support your proposal and I know you've been in the Dotsama ecosystem for quite some time and parachain teams love your work. I vote Aye with 3x conviction on this. Good luck !
Would your team have any interest working with the infrastructure builders program to deploy these indexer nodes on our hardware and for us to provide the endpoint to access them? We can include this in our mission and work with you to deploy / maintain them as your dedicated infrastructure team. If you're unfamiliar with the IBP, it's a new infrastructure program, you can view the proposal below. In simplest terms, we're building a decentralized global private cloud footprint to provide core infrastructure services to the community. https://docs.google.com/document/d/16USQYVhlyAlrU829EUB2TRoqUC0nnfoS\_uCdZ84HT8k
Hey! I handed it over to the right team member. We will come back to you. Thanks a lot!