View All Discussion

Patract Hub's treasury report for Himalia v0.1 & v0.2 (WASM contract sdks in Go and Python)

patract
5 years ago

5 weeks ago，Patract Hub applied a treasury proposal #61 for Himalia v0.1&v0.2, Now, we have finished all the work and you can review our codebase at：

PatractGo mainly completes the contract-related interfaces and APIs, and supports scanning and monitoring (observer) of events generated by the contract. PatractGo is based on Centrifuge's GSRPC. In the development process, in order to make the API more convenient to use and embed other services, we encapsulated GSRPC, providing API call based on context style. In addition, some interfaces that are not provided by GSRPC but are more useful for development and debugging have been added, such as encode and decode of AccountID in ss58 format, and unit test environment support.

PatractGo further encapsulates common ERC20 contracts, including API and monitoring support. This is convenient for processing ERC20 contracts, and can be used as an example to guide the development of other contract APIs. In the future, PatractGo will develop a code generator that automatically generates the corresponding go code based on the metadata of the contract. Since the logic of this generation process is the same for go and java, this part of the function is postponed to complete with PatractJ. An example of ERC20 is provided in the current version to show how to make packaging calls for a specific contract. For more information about PatractGo, please refer to the following function description and sample code in the project.

PatractGo also provieds the following founctions:

  • dump provides an example of the ERC20 contract status synchronization tool. Dump will scan all events of the specified ERC20 contract. The number of tokens corresponding to the account is synchronized to a local database, and a simple query function is provided. This example can be used as an example of Oberver, and at the same time, it can be used as a tool to analyze the Token distribution of a specified ERC20 contract.
  • rest use [github.com/patractlabs/go-patract/rest](https://github.com /patractlabs/go-patract/tree/master/rest) module. rest builds the url corresponding to the message based on metadata. Users can use this service to build the unsigned Extrinsic binary data of the specified contract, which can be sent to the trusted environment Third-party tool for signing. Based on this model, when contract developers use the Golang to develop contract back-end services, they can put the private key in an independent network environment and development environment to protect the security of the private key. In some scenarios, this processing is necessary , such as the account management of an exchange. As a developer-oriented SDK, programs containing PatractGo and other SDK codes may not be able to run in some sensitive environments. Therefore, rest provides an HTTP-based service interface, which is also convenient for developers to access existing authorization system.

PatractPy mainly completes the support of the unit test environment and scans and monitors the events generated by the contract. PatractPy is based on Polkascan's Python Substrate Interface. In the v0.11.8 version of py-substrate-interface, they have provided good support for contract API, so PatractPy mainly supplements some of the missing functions of py-substrate-interface, such as subscription API and deserialization support for contract events. PatractPy can automatically construct a python contract object based on the message part of the metadata, and has a method to call the contract, similar to the implementation of api-contracts in polkadot.js. Therefore, after loading metadata, PatractPy can simply call the contract according to the definition of the contract's own method, without the need for static languages such as Java or Golang to provide a method of interacting with the contract through code generation. On the other hand, PatractPy completes the scanning and monitoring of events generated by the contract, and its implementation can also be used to process events from other modules.

As the most important function of PatractPy, we hope that it can support our Europa and complete unit testing for contracts based on it. On the one hand, as the realization of the test case of the PatractPy project itself, on the other hand, we can use python to implement the test case code for the contract based on the PatractPy to test and reproduce the test cases in complex scenarios, plus complete Python ecological test support can make smart contract development more convenient and efficient.

In the future, PatractGo and PatractPy will be integrated into our Redspot in the form of plug-ins. Therefore, for a contract development project, you can use Redspot to build a contract development framework, and then use Redspot's own polkadot.js or use a plug-inPatractPy to quickly and easily perform contract debugging and integration testing in our Europa, and then use PatractGo to call the contract. After the entire system has been debugged in the test environment, use the backend required by PatractGo to develop the contract and use it in the production environment. The next version of PatractJ and PatractN will be integrated into the Redspot system in the same way.

1. Himalia's future development plan

  • M1: v0.1 PatractGo for Golang
  • M2: v0.2 PatractPy for Python
  • M3: v0.3 PatractJ for JVM
  • M4: v0.4 PatractN for .NET

2. PatractGo Report

Based on PatractGo，we can interact with the contracts very easily。The design goal of PatractGo is closer to that the contract owner build its own business logic to interact with the contract on the chain through PatractGo, which is not suitable for the scenario of contract debugging. If you need to debug the contract, it is recommended to use ParactPy or polkadot.js for quick testing.

For PatractGo, we split the functions into three components:

  • Contracts：provides the functions to interact with contracts
  • Rest：provides a way to generate an offline signature for contracts.
  • Observer: listens contract events and parses events by contract metadata.

2.1 API for Contracts

By Native Contract API，We Can Call Contracts Runtime Earlier。

Put Contracts Code to chain：

   // read the code wasm from file
   codeBytes, err := ioutil.ReadFile("/path/to/contracts.wasm")
   if err != nil {
      return err
   }

   // create the api
   cApi, err := rpc.NewContractAPI(env.URL())
   
   // read the abi(metadata) for contract
   metaBz, err := ioutil.ReadFile("/path/to/contracts_metadata.json")
   cApi.WithMetaData(metaBz)

   // create context with from auth, like Alice
   ctx := api.NewCtx(context.Background()).WithFrom(authKey)

   // put code
   _, err = cApi.Native().PutCode(ctx, codeBytes)
   
   // do next steps

Get Code from chain：

   codeHash := readCodeHash() // get code hash

   var codeBz []byte
   
	if err := cApi.Native().Cli.GetStorageLatest(&codeBz,
		"Contracts", "PristineCode",
		[]byte(codeHash), nil); err != nil {
		return err
   }
   
   // codeBz is now code

Instantiate Contract in chain:

	var endowment uint64 = 1000000000000

	// Instantiate
	_, contractAccount, err := cApi.Instantiate(ctx,
		types.NewCompactBalance(endowment),
		types.NewCompactGas(test.DefaultGas),
		contracts.CodeHashERC20,
		types.NewU128(totalSupply),
   )

api will return contractAccount, which can use it to call the contract.

For a contract, we can read or exec messages. Currently CallToRead and CallToExec implement a more basic level of encapsulation for contract calls, so contract developers need to write corresponding packaging functions according to the contract method, such as the ERC20 contract under the directory of PatractGo/contracts/erc20 example.

Read the total_supply of ERC20 contract, no request params:

	var res types.U128

	err := a.CallToRead(ctx,
		&res,
		a.ContractAccountID,
		[]string{"total_supply"},
	)

Read the balance_of of AccountID for ERC20 contract:

	req := struct {
		Address types.AccountID
	}{
		Address: owner,
	}

	var res types.U128

	err := a.CallToRead(ctx,
		&res,
		ContractAccountIDForERC20,
		[]string{"balance_of"},
		req,
	)

Call Transfer:

	toParam := struct {
		Address AccountID
	}{
		Address: to,
	}

	valueParam := struct {
		Value U128
	}{
		Value: amt,
	}

	return a.CallToExec(ctx,
		a.ContractAccountID,
		types.NewCompactBalance(0),
		types.NewCompactGas(test.DefaultGas),
		[]string{"transfer"},
		toParam, valueParam,
	)

These behaviors will be automatically generated after the auto contract code generator is completed, without the developers needing to care about this part.

2.2 REST API for Contract

We can use rest to get unsigned raw byte data for contract call, it can help to build an offline signature for contract.

can use this for example: rest

start the rest server:

go run ./examples/rest

to get data:

curl -X POST \
  'http://localhost:8899/erc20/exec/transfer?isOffline=true&contract=5HKinTRKW9THEJxbQb22Nfyq9FPWNVZ9DQ2GEQ4Vg1LqTPuk' \
  -H 'content-type: application/json' \
  -d '{
	"nonce":1,
	"chain_status":{
		"spec_version":1,
		"tx_version":1,
		"block_hash":"0xc20f241b61039e5685d118c7fbc8b27210153c21eee7686a9466f22e01281114",
		"genesis_hash":"0xc20f241b61039e5685d118c7fbc8b27210153c21eee7686a9466f22e01281114"
	},
	"contract":"5HKinTRKW9THEJxbQb22Nfyq9FPWNVZ9DQ2GEQ4Vg1LqTPuk",
	"origin":"5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
	"gas_limit":"500000000000",
	"args":{
		"to":"5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty",
		"value":"100000000"
	}
}'

2.3 observer

For a contract, we need observer events for the contract, can use observer to build a contract events observer service:

a complete example: observer

Also can take the dumper for example, it dump events by contacts to a db.

...

    // create observer
	o := observer.New(logger, *flagURL)
	ctx, cancelFunc := context.WithCancel(context.Background())

...
    // other init functions
...

    // create a handler for process events in chain
	h := observer.NewEvtHandler()
	h = h.WithContractExecution(func(l log.Logger, height uint64, evt types.EventContractsContractExecution) {
		data := evt.Data

		l.Debug("handler contract execution", "height", height)

        // for golang we need process each diff types for event
		typ := metadata.GetEvtTypeIdx(data)
		switch typ {
		case 0:
			var transfer erc20.EventTransfer
			err := metaData.Spec.Events.DecodeEvt(metaData.NewCtxForDecode(data).WithLogger(l), &transfer)
			if err != nil {
				logger.Error("evt decode transfer error", "err", err, "height", height)
			}
			logger.Info("transfer event", "evt", transfer)
		case 1:
			var approve erc20.EventApproval
			err := metaData.Spec.Events.DecodeEvt(metaData.NewCtxForDecode(data).WithLogger(l), &approve)
			if err != nil {
				logger.Error("evt decode approve error", "err", err, "height", height)
			}
			logger.Info("approve event", "evt", approve)
		}
	})

    // watcher events
	if err := o.WatchEvent(ctx, h); err != nil {
		logger.Error("watch event error", "err", err)
		return
	}

...

2.4 Support for ERC20 Contract

In github.com/patractlabs/go-patract/contracts/erc20, Complete ERC20 contract support.

...

		erc20API := erc20.New(rpcAPI, contractAccountID)
        ctx := rpc.NewCtx(context.Background()).WithFrom(signature.TestKeyringPairAlice)
        
		// transfer alice to bob
		aliceTotal, err := erc20API.BalanceOf(ctx, test.AliceAccountID)

		// transfer
		_, err = erc20API.Transfer(ctx, bob, types.NewBalanceByU64(100))

		bobBalance, err := erc20API.BalanceOf(ctx, bob)
        aliceNewTotal, err := erc20API.BalanceOf(ctx, test.AliceAccountID)

...

2.4 Tools

We make some tools for developing constracts, like:

3. PatractPy Report

For Unittest, should install Europa at first.

europa --version
europa 0.1.0-3f71403-x86_64-linux-gnu

All of test passed by Europa environment.

The design goal of PatractPy is to simplify the repetitive work of the interaction between developers and the contracts, and to monitor contract events, so as to provide a simplified way to quickly call and debug the contract. Therefore, the function of PatractPy will be similar to the api-contracts in polkadot.js. It can load contract instances according to the metadata of the contract, and automatically provide corresponding methods to the python contract instances according to the content of the metadata. Developers can call these generated methods to interact with the contract on the chain.

3.1 Basic Apis For Contracts

As polkascan's Python Substrate Interface has provide some support to contract api, so we not need to improve the apis for contract calls, but there is some api to add:

  • SubstrateSubscriber is a subscriber support to subscribe data changes in chain, for example, the events in chain.
  • get_contract_event_type add event decode support for contracts.

The basic api split into 2 parts:

  • Contract, include:
    • contractExecutor: This api could construct an extrinsic to call a contract, would be packed into a block and change state.
    • contractReader: This api could construct a rpc request to call a contract, do not pack into a block and do not change any state.
    • contractCreator: This api is used for instantiate a contract and holding the WASM code and metadata, receive following parameters:
      • gas_limit
      • endowment
      • deployment_salt (salt parameter in instantiate)
    • ContractAPI: This api is used for call a contract, is a wrapper for contractExecutor andcontractReader, developers could use this api to react with contracts. This api could create a instance depends on the metadata, auto generate the contract access functions based on the contract. And the auto-gen functions receive the parameters which defined in contracts, besides receive following common parameters:
      • gas_limit
      • value (notice, if current call's payable is false, this value must be 0)
    • ContractFactory: This api is used for constructing a contract instance in python, and developer could use this instance to access contract. This is a wrapper for contractCreator and ContractAPI
  • Observer, include:
    • ContractObserver: This api is used for listen the events in contracts.

Based on the api provided above, Python developers can refer to the following case to access the contract. The following example shows a contract instance loaded through the metadata of ERC20. Developers can directly call methods like transfer that are automatically created by metadata to send transactions to the node or call the node's rpc to return the contract execution result. On the other hand, the following example also shows how to monitor the events of a contract:

import os
from substrateinterface import SubstrateInterface, Keypair
from patractinterface.contract import ContractAPI, ContractFactory
from patractinterface.observer import ContractObserver

def main():
    # use [europa](https://github.com/patractlabs/europa) as test node endpoint, notice `type_registry` should set correctly.
    substrate=SubstrateInterface(url='ws://127.0.0.1:9944', type_registry_preset="default", type_registry={'types': {'LookupSource': 'MultiAddress'}})
    # load deployer key
    alice = Keypair.create_from_uri('//Alice')
    bob = Keypair.create_from_uri('//Bob')
    # 1. load a contract from WASM file and metadata.json file (Those files is complied by [ink!](https://github.com/paritytech/ink))
    # in this example, we use `ink/example/erc20` contract as example.
    contract = ContractFactory.create_from_file(
            substrate=substrate, # should provide a subtrate endpoint
            code_file= os.path.join(os.path.dirname(__file__), 'res', 'erc20.wasm'),
            metadata_file= os.path.join(os.path.dirname(__file__), 'res', 'erc20.json')
        )
    # upload code to chain directly
    res = contract.put_code(alice)
    print("update code hash{} res:{}".format(contract.code_hash.hex(), res.is_succes))
    # 2. instantiate the uploaded code as a contract instance
    erc20_ins = contract.new(alice, 1000000 * (10 ** 15), endowment=2*10**10, gas_limit=20000000000, deployment_salt="0x12")
    # 2.1 create a observer to listen event
    observer = ContractObserver(erc20_ins.contract_address, erc20_ins.metadata, substrate)
    # 3. send a transfer call for this contract
    res = erc20_ins.transfer(alice, bob.ss58_address, 100000, gas_limit=20000000000)
    print('transfer res', res.is_succes)

    def on_transfer(num, evt):
        print("on_transfer in {} : {} {} {}".format(num, evt['from'], evt['to'], evt['value']))

    def on_approval(num, evt):
        print("on_approval in {} : {} {} {}".format(num, evt['owner'], evt['spender'], evt['value']))
    # 4 set event callback 
    observer.scanEvents(handlers={
        'Transfer': on_transfer,
        'Approve': on_approval
    })

if __name__ == "__main__":
    main()
    pass

3.2 ContractObserver

ContractObserver can observer events for a contract:

substrate=SubstrateInterface(url="ws://127.0.0.1:9944", type_registry_preset='canvas')
contract_metadata = ContractMetadata.create_from_file(
    metadata_file=os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'),
    substrate=substrate
)
observer = ContractObserver("0x8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48", contract_metadata, substrate)

# for some handlers
observer.scanEvents()

The handler function can take the erc20 support as a example.

3.3 ERC20 API

In addition to the normal use of metadata to construct contract objects to access the contract, developers can also package their own access methods to the contract according to the basic api provided by PatractPy.

ERC20 api provide a wrapper to erc20 contract exec, read and observer events, it can be a example for contracts api calling.


# init api
substrate=SubstrateInterface(url="ws://127.0.0.1:9944", type_registry_preset='canvas')

contract_metadata = ContractMetadata.create_from_file(
    metadata_file=os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json'),
    substrate=substrate
)

alice = Keypair.create_from_uri('//Alice')
bob = Keypair.create_from_uri('//Bob')

# erc20 api
erc20 = ERC20.create_from_contracts(
    substrate= substrate, 
    contract_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.wasm'),
    metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json')
)

# deplay a erc20 contract
erc20.putAndDeploy(alice, 1000000 * (10 ** 15))

# read total supply
total_supply = erc20.totalSupply()

# transfer
erc20.transferFrom(alice,
    fromAcc=alice.ss58_address, 
    toAcc=bob.ss58_address, 
    amt=10000)

erc20.transfer(alice, bob.ss58_address, 10000)

# get balance
alice_balance = erc20.balanceOf(alice.ss58_address)

# approve
erc20.approve(alice, spender=bob.ss58_address, amt=10000)

# get allowance
alice_allowance = erc20.allowance(alice.ss58_address, bob.ss58_address)

ERC20Observer is a event observer for erc20 contract:

observer = ERC20Observer.create_from_address(
    substrate = substrate, 
    contract_address = contract_address,
    metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json')
)

def on_transfer(num, fromAcc, toAcc, amt):
    logging.info("on_transfer in block[{}] : {} {} {}".format(num, fromAcc, toAcc, amt))

def on_approval(num, owner, spender, amt):
    logging.info("on_approval in block[{}] : {} {} {}".format(num, owner, spender, amt))

observer.scanEvents(on_transfer = on_transfer, on_approval = on_approval)

3.4 Unittest Node Environment

PatractPy can support write contract unittest by node environment.

At First We need install europa.

from patractinterface.contracts.erc20 import ERC20
from patractinterface.unittest.env import SubstrateTestEnv

class UnittestEnvTest(unittest.TestCase):
    @classmethod
    def setUp(cls):
        # start env or use canvas for a 6s block
        cls.env = SubstrateTestEnv.create_europa(port=39944)
        cls.env.startNode()

        cls.api = SubstrateInterface(url=cls.env.url(), type_registry_preset=cls.env.typ())
        cls.alice = Keypair.create_from_uri('//Alice')
        cls.bob = Keypair.create_from_uri('//Bob')

        cls.erc20 = ERC20.create_from_contracts(
            substrate= cls.substrate, 
            contract_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.wasm'),
            metadata_file= os.path.join(os.path.dirname(__file__), 'constracts', 'ink', 'erc20.json')
        )
        cls.erc20.putAndDeploy(alice, 1000000 * (10 ** 15))

        return

    def tearDown(cls):
        cls.env.stopNode()

    def test_transfer(self):
        self.erc20.transferFrom(alice,
            fromAcc=alice.ss58_address, 
            toAcc=bob.ss58_address, 
            amt=10000)
        # some more test case

if __name__ == '__main__':
    unittest.main()

By example, we can use python to write testcase for some complex logics, by Europa, we can test the contracts for python scripts.

4. Recap of verification of v0.1 and v0.2

v0.1 PatractGo

  • Initialize the project and complete the native contract interaction API
  • Complete the contract interaction API and HTTP contract interaction service based on metadata.json
  • Complete contract status scan support
  • Monitor service support and complete the example to write into the database
  • Complete ERC20 contract support and complete the corresponding command line tools
  • Improve sample code and unboxing documentation

v0.2 PatractPy

  • Initialize the project and complete the native contract interaction API
  • Complete the contract interaction API based on metadata.json
  • Complete ERC20 contract support
  • Improve sample code and unboxing documentation

We will propose v0.3 and v0.4 later after some time for research.

Comments (0)

PleaseLogin to comment
    Polkassembly - Patract Hub's treasury report for Himalia v0.1 & v0.2 (WASM contract sdks in Go and Python) #428