Polkassembly - Patract Hub's treasury proposal for Himalia v0.1 & v0.2 (WASM contract sdks in Go and Python) #370

Patract Hub's treasury proposal for Himalia v0.1 & v0.2 (WASM contract sdks in Go and Python)

5 years ago

Summary

At present, Substrate has a large number of client SDKs of different languages to interact with the blockchains. However, these client SDKs are oriented to the Substrate-based chain, so the functions provided by them are often limited to the APIs and primitive under the Substrate framework. All of these client SDKs do not support smart contracts.

In the Polkadot ecosystem, not all parachains need to include smart contracts support. In most cases, Runtime can already solve most certain application scenarios. Therefore, the client SDKs for Substrate does not have to support smart contracts. On the other hand, based on the principle of minimization, this type of smart contract SDKs should be developed in the form of an independent library. This is the development goal of Himalia.

A smart contract-based DApp not only needs to develop the contract on the chain, but also needs to develop user-oriented front-end interfaces and micro-services, which means that developers need to interact with contracts from different terminals and based on multiple development languages.

Himalia will provide FRAME Contracts SDK supports in a multi-language environment to support WASM DApp development. Himalia will be Patract Hub's 7th project for whole WASM contract development support. DApp needs to choose different technology stacks based on its form. For example, a service that provides users with contract status queries is often developed based on Golang, and a contract-based blockchain game can be developed in C# based on game engines such as Unity3D. This means that to expand the application scenarios of DApp, we need SDKs in as many environments as possible. Currently Himalia plans to implement SDK support for the following languages:

v0.1(3 weeks): PatractGo, Golang contract SDK, to support micro-service development based on contract status. PatractGo will be based on Centrifuge's GSRPC, which has already support Substract 2.0.
v0.2(2 weeks): PatractPy, Python contract SDK, to support the development of scripts interacting with contracts, including automated scripts to support testing. PatractPy will be based on Polkascan's Python Substrate Interface, which has already support Substract 2.0.
v0.3(4 weeks): PatractJ, Java/Scala/Kotlin contract SDK, to support back-end development based on JVM, and also rely on Emeraldpay's PolkaJ to complete the support development for Substrate 2.0.
v0.4(3 weeks): PatractN, C# contract SDK, to support applications based on DotNet/Mono development and contract interaction, especially game development based on Unity3D, also rely on Darkfriend77's SubstrateNetApi or Usetech's polkadot_api_dotnet to complete the support development for Substrate 2.0.

v0.1 for PatractGo

PatractGo is a Golang contract SDK. In addition to supporting the basic interactive API with the contract, it mainly supports the development of micro-services based on the contract status. For example, for common ERC20 contracts, a service can be developed based on PatractGo to synchronize all transfer information to the database, and based on the database to implement the statistics and processing of transfer data. In addition, developers can also develop some command-line tools based on Golang to assist in testing and debugging. We will be independent of GSRPC and encapsulate the API provided by GSRPC , to adapt to different substrate-based chains.

Introduction

Most contract behaviors are highly related to context. In addition to interacting with the chain, user-oriented contract applications also need to provide users with current relevant context status information:

+--DAPP-Front-End--------------+        +---Chain-------------------------+
|                              |        |                                 |
| +----+  +------------------+ |        | +-------+     +-------+         |
| |    |  |                  | | Commit | |       |     |       |         |
| |    |  |   Polkadot-JS    +------------> Node  +---->+ Node  |         |
| |    +->+                  | |   Tx   | |       |     |       |         |
| |    |  |                  | |        | +-------+     +----+-++         |
| |    |  +------------------+ |        |                    ^ |          |
| | UI |                       |        +---------------------------------+
| |    |  +------------------+ |                             | |
| |    |  |                  | |        +--DAPP-Server--------------------+
| |    |  |                  | |  Push  | +--------+     +-----v-------+  |
| |    +<-+   Model          +<-----------+        +-----+             |  |
| |    |  |                  | |        | | Server |     |  PatractGo  |  |
| |    |  |                  +------------>        +-----+             |  |
| +----+  +------------------+ | Query  | +----+---+     +-----+-------+  |
+------------------------------+        |      |               |          |
                                        |      |         +-----v-------+  |
                                        |      |         |             |  |
                                        |      +-------->+   DataBase  |  |
                                        |                |             |  |
                                        |                +-------------+  |
                                        |                                 |
                                        +---------------------------------+

PatractGo is mainly responsible for implementing micro-services in a DApp. Unlike querying the state of the chain API, PatractGo can monitor the calls and events generated by the specified contract. Developers can obtain the state storage based on this information to maintain consistent state with the chain. Through data services based on a typical API-DB architecture, the front-end DApp can efficiently and concisely obtain the state on the chain as context information.

Based on the API of chain nodes, PatractGo obtains block information and summarizes and filters it, and sends contract-related messages and events based on metadata analysis to the handler protocol specified by the developer. For example, for a typical ERC20 contract, the developer can use the channel to subscribe to all transfer events that occur, and then synchronize them into the database, so that other microservices can provide services corresponding to the token data of the account, such as querying the current token holding distribution and other logics.

Therefor, PatractGo will achieve the following support:

Complete the secondary packaging of the contract module interface, complete operations such as put_code, call, instantiate, etc.
Parse the metadata.json information of the contract, and support the automatic generation of http service interface for the metadata corresponding contract
Scanning and monitoring support of the contract status on the chain for statistics and analysis
Basic command line tool support for native interaction with the contract, mainly used to test the security of the contract
SDK development examples for ERC20 contract support

Design

PatractGo consists of the following packages:

patractgo/metadata contract metadata processing, and metadata-based contract processing
patractgo/rpc/native re-encapsulation of the contract module interface to provide the contract-related interaction based on chain RPC
patractgo/rpc implement the interaction with the contract based on metadata
patractgo/rest implements an http service based on metadata to interact with the contract
patractgo/scanner Scanning support for contract status on chain
patractgo/observer Monitoring support for contract status on the chain
patractgo/contracts/erc20 supports ERC20 contracts
patractgo/cmd/patractgo a command line tool for interacting with contracts
patractgo/cmd/pdumper monitors contract status and events and synchronizes them to the database
patractgo/examples SDK instance

v0.2 for PatractPy

PatractPy is a contract SDK to support the development of Python scripts that interact with contracts, including automated scripts to support testing. Unlike PatractGo, PatractPy is mainly for script development, so PatractPy mainly completes contract-related RPC interfaces, and completes contract deployment and instantiation-related operations. At the same time, PatractPy will implement PatractGo-based interaction and contract status monitoring support.

As a result, PatractPy will achieve the following support:

Complete the secondary packaging of the contract module interface, complete operations such as put_code, call, instantiate, etc.
Contract-based metadata information and contract interaction
Based on PatractGo, provide HTTP service for contract-based metadata information and contract interaction
Based on PatractGo, provide Scanning and monitoring support for contract to do statistics and analysis
Provide a SDK development example for ERC20 contract

Detailed timeline for v0.1 ~ v0.2 (5 weeks, 1 Dec ~ 5 Jan)

M1: v0.1 PatractGo（2 developers * 3 weeks)

Initialize the project and complete the native contract interaction API
Complete the contract interaction API and HTTP contract interaction service based on metadata.json
Complete contract status scan support
Monitor service support and complete the example to write into the database
Complete ERC20 contract support and complete the corresponding command line tools
Improve sample code and unboxing documentation

M2: v0.2 PatractPy (2 developers * 2 weeks)

Initialize the project and complete the native contract interaction API
Complete the contract interaction API based on metadata.json
Through PatractGo, complete the contract interaction API based on metadata.json
Through PatractGo, complete the monitoring service support, and complete the example to write into the database
Complete ERC20 contract support
Improve sample code and unboxing documentation

M3&M4: v0.3&v0.4, PatractJ & PatractN.

We have finished the design for the Java and .NET SDK, but because the chain SDK teams hasn't completed the Substrate 2.0 support , so we may need to wait for them to finish it and apply later, so the timeline maybe uncertain.

Cost of v0.1 & 0.2 (10 developers * weeks)

Operating activities: $2000 ( Rent and Devices: $200 per developer * week )
Employee payments: $19000 ($1900 per developer * week)
—————————— +
Total Cost: $21000
Exchange Rate: $45 / KSM
Treasury Proposal: 466 KSM

Verification of v0.1: Report & Github Source & Jupiter and Canvas testnet:

Through unit test cases, based on Docker and verify all interactive APIs
Through unit test cases, all contract events and calls can be synchronized and written to the log and database through the monitoring service
Provide sample code for RPC package and Observer package, based on the README document, you can complete the test environment and debug the sample code

Verification of v0.2: Report & Github Source & Jupiter and Canvas testnet:

Through unit test cases, based on Docker and http service to verify all interactive APIs
Provide sample code, based on the README document, you can complete the test environment and debug the sample code

Comments (6)

5 years ago

I'm generally in favor, but a little worried about long-term maintenance of four separate SDKs for a pallet that's evolving as quickly as Contracts is. Will these SDKs use ink! in any way, or will they just be for interacting with a chain that has the contracts pallet installed? I guess I'm struggling a little with figuring out when and why someone would use these SDKs that are specific to contracts, when they're not dealing with ink! and instead depend on a wasm compilation of a contract for put_code from somewhere else. Can you maybe list some user stories, as in, when exactly and how someone might use each of these SDKs specifically, just to help me understand?

5 years ago

@swader For first question: No need to worry about our determine for long-term maintenance:. We support the whole aspects of the WASM contract development ecosystem. The SDK part must be supplemented. After we develop different SDKs, we will integrate them into the Redspot ecosystem. So as long as the Redspot ecosystem exists, different versions of the SDK will be maintained together.

For second question: These SDKs can be used for any parachain that integrates Pallet-Contracts. The contract SDK and the chain SDK are not in a competitive relationship, but the contract SDK depends on the chain SDK and provide upper-level encapsulation. It is similar to the existing polkadot.js, but we can further extend the functions based on polkadot.js for the ink! functional components.

For third question: If there is only a chain SDK, then the ink! contract interaction with this chain can only be done through the three methods put_code, call, and instantiate. For example, in the call of an ERC20 contract, all methods such as transfer, approve can only be analyzed by the developer himself to parse its metadata, and assemble parameters like "0xab1203..." as the call parameter. This process is complicated and error-prone. The contract SDK is to encapsulate this process. When an ERC20 contract is called, it can be written directly in the form of erc20.transfer(...).send() instead of api.call("0xab1203...") Form. This kind of encapsulation of the contract is the same as the contract code that can be called automatically generated according to the contract ABI in Ethereum's web3j, in order to simplify the process of calling the contract.

On the other hand, the contract SDK can be extended with more functions. For example, the Go version contract SDK can be packaged into a form that can provide services. Unlike isolated chain SDKs for different languages, the Himalia project is a whole, and each SDK and SDK-based applications will collaborate and divide works to assist contract-based application development.

PleaseLogin to comment