Off-Chain P8e Contracts & SDK
A tutorial on Provenance Blockchain’s off-chain SDK
Provenance Blockchain Contract Tooling
Provenance Blockchain has two contract systems. The first is the blockchain-based cosmwasm smart contract system. The second, and the focus of this article, is the off-chain “p8e” contract system. The difference between these contract systems lies primarily in their use cases. All information operated on by a smart contract is essentially public and that puts us in a pickle if our input and/or output needs to remain private. Luckily, the p8e contract system provides us a solution to this problem with a JVM SDK and an object store, keeping data off the public blockchain, while recording the hashes of execution results on-chain. For later access to the private data, we can get the hashes from the blockchain and retrieve the true data from our private object store.
The SDK allows us to define our own contracts that can be used to store and process data on Provenance Blockchain. Once we have defined our contracts, we can generate a jar file and register it with the object store in a process known as bootstrapping. During this process, hashes of the jar and the contained contracts are calculated and stored on chain. We can later import this contract jar into our application and execute a contract against some data.
Interestingly, an execution doesn’t actually use the local contract dependency and instead downloads the contract jar from object store using the contract hash from our local dependency. This provides certainty that a contract is running specific code that is verifiable via hash value. The result of the contract execution is an “envelope” that contains metadata about the contract that was executed and hashes of the input data and result. We can then easily submit this envelope data to the blockchain which is saved as a scope on Provenance Blockchain.
Later, we can load the scope data which will contain some metadata and our contract results in the form of hashes. We use the SDK to “hydrate” the private data from object store by providing the relevant hashes in a secure request. Provided our keys have access and the hashes exist, our raw data will be returned.
It’s kind of a lot… but I’ll try to sum it up: The p8e contract system allows us to access or reason about private data and record to the blockchain the data hashes of the exact inputs, results, and code executed.
The Code
We’re gonna create a little demo app for this tutorial. I’ll be using Kotlin and Kotlin DSL gradle scripts, but you should be fine using any JVM language or build system for your own implementations. All of the code for this tutorial can be found here.
Ok, that’s it! Let’s dive in.
Local Infrastructure
First things first, we are going to need a local environment to “do all the things” and docker is here to save the day. We’ll use docker to manage the following:
- Provenance Blockchain Network — this is our base
- Provenance Object Store — manages data access and contract jar storage
- Postgres — used by object store (and maybe whatever you’re building)
We need to set up configuration for our local node, but we don’t need to get bogged down in it. I recommend copying the docker folder from the linked GitHub repo. Here are the main bits:
/docker/provenance/config This folder holds all of the local chain configuration. If you want to ensure certain accounts exist, have some Hash, or the local chain reflects a particular setup - you can do it here.
/docker/db-init This folder holds SQL scripts required for configuring the object store db (and maybe your app).
/docker/docker-compose.yaml We’re utilizing docker compose to be able to quickly start and stop our local environment.
/docker/env
This folder holds environment variables that we will be using down the line when bootstrapping and executing contracts. More on this later.
One more thing: The object store image is stored on GitHub container registry. This requires some level of setup to properly download images. If you have any problems spinning up the local environment due to this, please take a look at this guide.
Create our subprojects
For this tutorial, we’re going to do everything in a single project. Let’s go ahead and create the following 3 subprojects:
- application — where we will do our contract executions and data access
- contract — where the contracts will live
- proto — where the data model will live
Setup ./build.gradle.kts
Before we get to defining our data model and contracts, we first need to configure the p8e gradle plugin by making some updates to our parent project’s gradle file. Let’s start with the plugin section
plugins {
kotlin("jvm") version "1.6.20"
id("io.provenance.p8e.p8e-publish") version "0.6.3"
}The p8e gradle plugin requires some buildscript dependencies so let’s add those as well
buildscript {
repositories {
mavenCentral()
maven { url = uri("<https://javadoc.jitpack.io>") }
}
}Last, but not least is the bootstrapping configuration.
fun p8eParty(publicKey: String): P8ePartyExtension = P8ePartyExtension().also { it.publicKey = publicKey }
p8e {
// Package locations that the ContractHash and ProtoHash source files will be written to.
language = "kt" // defaults to "java"
contractHashPackage = "io.p8e.demo.contract"
protoHashPackage = "io.p8e.demo.contract"
contractProject = "contract"
locations = mapOf(
"local" to P8eLocationExtension().also {
it.osUrl = System.getenv("OS_GRPC_URL")
it.provenanceUrl = System.getenv("PROVENANCE_GRPC_URL")
it.encryptionPrivateKey = System.getenv("ENCRYPTION_PRIVATE_KEY")
it.signingPrivateKey = System.getenv("SIGNING_PRIVATE_KEY")
it.chainId = System.getenv("CHAIN_ID")
it.txFeeAdjustment = "2.0"
it.audience = mapOf(
"local1" to p8eParty("0A41046C57E9E25101D5E553AE003E2F79025E389B51495607C796B4E95C0A94001FBC24D84CD0780819612529B803E8AD0A397F474C965D957D33DD64E642B756FBC4"),
"local2" to p8eParty("0A4104D630032378D56229DD20D08DBCC6D31F44A07D98175966F5D32CD2189FD748831FCB49266124362E56CC1FAF2AA0D3F362BF84CACBC1C0C74945041EB7327D54"),
"local3" to p8eParty("0A4104CD5F4ACFFE72D323CCCB2D784847089BBD80EC6D4F68608773E55B3FEADC812E4E2D7C4C647C8C30352141D2926130D10DFC28ACA5CA8A33B7BD7A09C77072CE"),
"local4" to p8eParty("0A41045E4B322ED16CD22465433B0427A4366B9695D7E15DD798526F703035848ACC8D2D002C1F25190454C9B61AB7B243E31E83BA2B48B8A4441F922A08AC3D0A3268"),
"local5" to p8eParty("0A4104A37653602DA20D27936AF541084869B2F751953CB0F0D25D320788EDA54FB4BC9FB96A281BFFD97E64B749D78C85871A8E14AFD48048537E45E16F3D2FDDB44B"),
"smart_key1" to p8eParty("0A4104B4495B4A4F24F70650E9104EA409B6108740C376FC2625BA0B5085DD12E4BC13EE34C8BFFBBC1762B20E79B74257D32F31409F3E56372CB9B671B590BC46F287"),
)
}
)
}Ummm… what are all these environment variables and hex values? Great question! Remember the /docker/env folder? In that folder is a file called bootstrap.env and, later on when we are bootstrapping our contracts, that file will be referenced to fill in these variables. The hex values, on the other hand, are correlated to the private key values also found in that env file.
Protos
This step is all about defining the data for our contracts and the p8e contract system is designed around protobufs.
(If you aren’t super familiar with protos, you can learn more here)
Setup ./proto/build.gradle.kts
To get our project to build protobufs, we need to tell gradle to use the protobuf plugin.
plugins {
kotlin("jvm")
id("com.google.protobuf") version "0.8.18"
}Next, we include the protobuf dependency.
dependencies {
// at compile time we need access to ProtoHash on the classpath
compileOnly("io.provenance.scope:contract-base:0.4.9")
implementation("com.google.protobuf:protobuf-kotlin:3.20.0")
}Ensure that we include the built protos into our jar (java and kotlin builders).
sourceSets {
main {
java {
srcDir("build/generated/source/proto/main/java")
srcDir("build/generated/source/proto/main/kotlin")
}
}
}And finally we’ll do some configuration for the protobuf compiler.
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile>().all{
kotlinOptions{
freeCompilerArgs =listOf("-Xopt-in=kotlin.RequiresOptIn")
}
}protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.20.0"
}
generateProtoTasks { // to get kotlin dsl proto builders
all().forEach {
it.builtins {
id("kotlin")
}
}
}
}
Proto Definition
At this point, gradle should be more than happy to build our protos! We just need to define what they are — let’s make some protos for storing some super basic loan data.
syntax = "proto3";package demo;option java_package = "io.p8e.demo.proto";
option java_outer_classname = "LoanData";message Loan {
string id = 1;
string type = 2;
string originator = 3;
}message Servicer {
string id = 1;
string name = 2;
}
Contracts
Now that we have our data structures defined, we can go ahead and write some contracts!
Setup ./contract/build.gradle.kts
In our gradle file, we need to import our proto module jar, the google protobuf dependencies, and the SDK contract module
dependencies {
implementation(project(":proto"))
implementation("com.google.protobuf:protobuf-kotlin:3.20.0")
implementation("io.provenance.scope:contract-base:0.4.9")
}…and that’s it! Simple 👍
Contract Definitions
A little more background… the Provenance Blockchain p8e contract system stores data in a structure known as a scope. The scopes themselves have a specification definition that controls what data contracts are able to create or modify. That definition is… well… defined by us, so let’s start there.
const val scopeNamespace = "io.p8e.demo.Loan"@ScopeSpecificationDefinition(
uuid = "c25680aa-32d5-4652-969f-26992222e167",
name = scopeNamespace,
description = "A demo loan contract",
partiesInvolved = [ORIGINATOR] //Specifications.PartyType.ORIGINATOR
)
class DemoLoanScopeSpecification : P8eScopeSpecification()
I’ve defined a scope namespace as its own variable so that it is reusable — nothing crazy here. The annotated class just below it, however, is all of the magic. The annotation defines how we can identify this scope spec as well as what entities can create it.
It is important to note, that the name and uuid should remain unchanged — if they are changed, you may find that your contracts are no longer compatible with scopes created under the original specification. If you do find yourself needing to update the identification information, you’ll need to update the legacy scopes to conform to the new information. That, however, is outside the scope of this guide.
Another important part of the definition is the partiesInvolved field. When we work with the SDK, we need to define who we are as an entity in the scheme of the p8e contract environment. In our example, we are saying that only an ORIGINATOR can create a scope of this specification. Don’t worry, we’ll get into how to configure that later.
Now that we have a specification, let’s go ahead and define a contract that allows us to create a DemoLoanScopeSpecification scope.
@Participants(roles = [ORIGINATOR])
@ScopeSpecification(names = [scopeNamespace])
open class CreateLoanScopeContract : P8eContract() { @Function(invokedBy = ORIGINATOR)
@Record(name = "loan")
open fun loan(@Input(name = "loan") loan: LoanData.Loan) = loan @Function(invokedBy = ORIGINATOR)
@Record(name = "servicer")
open fun servicer(@Input(name = "servicer") servicer: LoanData.Servicer) = servicer
}
Ok! Awesome! There’s some more “stuff” to unpack here. Let’s begin with the class annotations.
@Participants is similar to partiesInvolved in that it defines what entities are required to execute this contract. Once again, we are saying that only an ORIGINATOR can execute this contract. An important note, a participant defined in a contract definition does not need to appear as an involved party in the scope specification.
@ScopeSpecification is the way we can link back to our earlier scope specification definition. It lets the SDK know whether or not a contract can be executed on a particular scope spec.
One last note, both of those fields are lists. While beyond the boundaries of this tutorial, we can define contracts that require more than one affiliate type and/or are valid with more than one scope specification.
Let’s move on to the method annotations…
@Function simply tells the SDK who is required to provide what data. Here, it is all the ORIGINATOR, but this could be different if multiple affiliate types were required.
@Record is another part of the scope structure. While the scope tells us something about the stored data, a record is the actual data being stored. We provide a name by which we can access or modify the data. It’s worth mentioning that each record on a scope is unique, so writing to an existing record will completely overwrite any old data that may exist.
So we have the ability to create a loan using our scope specification and first contract, but we often need to modify data after it has been created. Let’s go ahead and define a contract for updating the loan servicer.
@Participants(roles = [ORIGINATOR])
@ScopeSpecification(names = [scopeNamespace])
open class UpdateLoanScopeServicerContract(
@Record(name = "servicer") val existingServicer: LoanData.Servicer
) : P8eContract() { @Function(invokedBy = ORIGINATOR)
@Record(name = "servicer")
open fun servicer(@Input(name = "servicer") servicer: LoanData.Servicer) = servicer
}
This isn’t all that different except for one thing — the constructor for our contract requires an input. You’ll notice that input also has the @Record attribute - in our case, this means that this contract requires the “servicer” record to exist on this scope, otherwise the contract execution will fail. You’ll notice that our contract methods have access to the existing record data and the new input data in our contract methods. While we are simply replacing data in this example, you could have more complex logic that leverages both sets of data, calls an external API, runs validation, etc.
Application
Ok, we’ve done all the prep work! Now we can go ahead and build a little app to create, modify, and access scope data stored on Provenance Blockchain.
Setup ./application/build.gradle.kts
First we’ll add the application plugin. We’re also going to import our other projects, some sdk modules, and grpc dependencies to interact with object store and Provenance Blockchain. Finally, we’ll provide a little configuration for our application plugin.
plugins {
application
kotlin("jvm")
}dependencies {
// our data model and contracts
implementation(project(":proto"))
implementation(project(":contract")) // p8e sdk modules
implementation("io.provenance.scope:sdk:0.4.9")
implementation("io.provenance.scope:util:0.4.9")
// sdk needs an slf4j implementation - choosing no logger
implementation("org.slf4j:slf4j-nop:1.7.36") // gprc client + protos for PBC
implementation("io.provenance:proto-kotlin:1.8.0")
implementation("io.provenance.client:pb-grpc-client-kotlin:1.1.1")
implementation("io.provenance.hdwallet:hdwallet:0.1.15") // grpc for OS and PBC
implementation("io.grpc:grpc-protobuf:1.45.1")
implementation("io.grpc:grpc-stub:1.45.1")
}application {
mainClassName = "io.p8e.demo.MainKt"
}
Do All The Things
To start, let’s define a data class that represents our loan. Such a class is required to hydrate our scope data later in this section.
data class Loan(
@Record("loan") val loan: LoanData.Loan,
@Record("servicer") val servicer: LoanData.Servicer
)Once again, we use the @Record annotation to help the SDK determine which records map to which field.
Alright, now let’s do some setup so we can actually create, modify, and access our data on chain (and in object store)
// grpc connection to provenance blockchain
val pbcClient = PbClient("chain-local", URI("grpc://localhost:9090"), GasEstimationMethod.MSG_FEE_CALCULATION)// sdk client for contract execution and object store interactions
private val signer = fromMnemonic(
NetworkType("tp", "m/44'/1'/0'/0/0"),
"stable payment cliff fault abuse clinic bus belt film then forward world goose bring picnic rich special brush basic lamp window coral worry change"
)
private val encryptionPrivateKey = "0A2100EF4A9391903BFE252CB240DA6695BC5F680A74A8E16BEBA003833DFE9B18C147".toJavaPrivateKey()
private val signingPrivateKey = "0A2100EF4A9391903BFE252CB240DA6695BC5F680A74A8E16BEBA003833DFE9B18C147".toJavaPrivateKey()
private val affiliate = Affiliate(
encryptionKeyRef = DirectKeyRef(ECUtils.toPublicKey(encryptionPrivateKey)!!, encryptionPrivateKey),
signingKeyRef = DirectKeyRef(ECUtils.toPublicKey(signingPrivateKey)!!, signingPrivateKey),
partyType = Specifications.PartyType.ORIGINATOR,
)
private val config = ClientConfig(
cacheJarSizeInBytes = 0L,
cacheSpecSizeInBytes = 0L,
cacheRecordSizeInBytes = 0L,
osGrpcUrl = URI("grpc://localhost:8090"),
mainNet = false,
)
val sdkClient = Client(SharedClient(config = config), affiliate)
The first bit, pbcClient is what we will use to interact with Provenance Blockchain. The second portion of code is setting up the sdk for our contract executions. The private key values were pulled right out of /docker/env/bootstrap.env . Also, notice that when we set up our affiliate, we specify the ORIGINATOR party type as our contracts require this role.
Next, we’re going to make a few helper methods for us to:
- execute a contract and send the results as a transaction to Provenance Blockchain
- get the error message from a failed transaction
- load a scope from Provenance Blockchain
- hydrate a
Loanfrom object store with our scope record
// execute the contract and send to PBC
fun executeContractAndSendAsTx(session: Session): ServiceOuterClass.BroadcastTxResponse =
(sdkClient.execute(session) as SignedResult).let { executionResult ->
val messages = executionResult.messages.map { Any.pack(it, "") }
TxOuterClass.TxBody.newBuilder().addAllMessages(messages).build()
}.let { txBody ->
pbcClient.estimateAndBroadcastTx(
txBody,
signers = listOf(BaseReqSigner(signer)),
mode = ServiceOuterClass.BroadcastMode.BROADCAST_MODE_BLOCK
)
}// get the error message, if a tx has failed
fun Abci.TxResponse.getError(): String =
logsList.filter { it.log.isNotBlank() }.takeIf { it.isNotEmpty() }?.joinToString("; ") { it.log }
?: rawLog// fetch the scope from PBC
fun loadScope(scopeUuid: UUID): ScopeResponse =
scopeRequest {
scopeId = scopeUuid.toString()
includeSessions = true
includeRecords = true
}.let { request ->
pbcClient.metadataClient.withDeadlineAfter(10, TimeUnit.SECONDS).scope(request)
}// hydrate data from object store
fun hydrateLoan(scopeUuid: UUID): Loan =
loadScope(scopeUuid).let { scope -> sdkClient.hydrate(Loan::class.java, scope) }
You’ll notice that I chose a UUID as the scope id data type and that isn’t an accident — Provenance Blockchain requires scope ids to be a UUID. However, the restrictions are loosened when loading a scope as we can search for the UUID or the scope’s bech32 address.
At this point, all the support code is complete! Let’s go ahead and create some methods to execute our CreateLoanScopeContract and UpdateLoanScopeServicerContract.
fun executeCreateLoanScopeContract(
scopeUuid: UUID,
loan: LoanData.Loan,
servicer: LoanData.Servicer
): ServiceOuterClass.BroadcastTxResponse { // a session is the representation in a scope of a contract execution
val session = sdkClient
.newSession(CreateLoanScopeContract::class.java, DemoLoanScopeSpecification::class.java)
.setScopeUuid(scopeUuid)
.addProposedRecord("loan", loan)
.addProposedRecord("servicer", servicer)
.build() // the contract and send the execution results to provenance blockchain
return executeContractAndSendAsTx(session)
}
First up is CreateLoanScopeContract - Since we are using a helper to execute the contract and send the transaction to Provenance Blockchain, all we need to do is set up a session. “What is a session?”, you may ask - it is the blockchain abstraction of a contract execution. Similar to a scope representing some data, the session represents some action on that data.
Since our scope should not exist, you will notice that we must specify the scope specification definition in newSession along with the type of contract we are going to run. Following that, we set the scope and provide the loan and servicer records that we intend to store.
fun executeUpdateLoanScopeServicerContract(
scopeUuid: UUID,
servicer: LoanData.Servicer
): ServiceOuterClass.BroadcastTxResponse { // fetch the scope from PBC
val scope = loadScope(scopeUuid) // a session is the representation in a scope of a contract execution
val session = sdkClient
.newSession(UpdateLoanScopeServicerContract::class.java, scope)
.addProposedRecord("servicer", servicer)
.build() // the contract and send the execution results to provenance blockchain
return executeContractAndSendAsTx(session)
}
The method for executing UpdateLoanScopeServicerContract is going to be similar. The main difference here is in the newSession method - since the scope is already created, we must provide the scope itself (instead of a scope specification definition). Additionally, we don’t need to explicitly set the scope id.
Ok, the time has come! Let’s go ahead and write our main method to do all this contract stuff.
fun main() {
// this will be the uuid of our example scope
val scopeUuid = UUID.randomUUID() // the data we want to save
val loanRecord = loan {
id = scopeUuid.toString()
type = "SOME_LOAN_TYPE"
originator = "IM_A_BANK"
}
val servicerRecord = servicer {
id = UUID.randomUUID().toString()
name = "IM_A_LOAN_SERVICER"
} println("Creating scope record for loan $scopeUuid...")
val createResponse = executeCreateLoanScopeContract(scopeUuid, loanRecord, servicerRecord)
if (createResponse.txResponse.code == 0) {
println("Scope: $scopeUuid")
println("Tx Hash: ${createResponse.txResponse.txhash}")
} else { //error
throw IllegalStateException(createResponse.txResponse.getError())
} println("Checking the data stored on chain for loan $scopeUuid...")
hydrateLoan(scopeUuid).let { data ->
println("Loan Id: ${data.loan.id}")
println("Loan Type: ${data.loan.type}")
println("Loan Originator: ${data.loan.originator}")
println("Loan Servicer Id: ${data.servicer.id}")
println("Loan Servicer Name: ${data.servicer.name}")
} // data for the new servicer
val newServicer = servicer {
id = UUID.randomUUID().toString()
name = "IM_A_DIFFERENT_LOAN_SERVICER"
} println("Updating the servicer record for loan $scopeUuid...")
val updateResponse = executeUpdateLoanScopeServicerContract(scopeUuid, newServicer)
if (updateResponse.txResponse.code == 0) {
println("Scope: $scopeUuid")
println("Tx Hash: ${updateResponse.txResponse.txhash}")
} else { //error
throw IllegalStateException(updateResponse.txResponse.getError())
} println("Checking the data stored on chain for loan $scopeUuid...")
hydrateLoan(scopeUuid).let { data ->
println("Loan Id: ${data.loan.id}")
println("Loan Type: ${data.loan.type}")
println("Loan Originator: ${data.loan.originator}")
println("Loan Servicer Id: ${data.servicer.id}")
println("Loan Servicer Name: ${data.servicer.name}")
}
}
To walk through everything that is happening here, we first set up our data, execute the contract, and send the results to Provenance Blockchain. After this, we end up with our createResponse which will contain some information about the transaction that was sent. If the transaction was successful, a code of 0 is returned and we print out the tx hash for our own knowledge. If there is an error, a nonzero code will exist and we throw an exception with the failure message.
Provided everything succeeded, we then hydrate the loan data. As previously mentioned, this process includes loading the scope from Provenance Blockchain and retrieving data from object store by providing the record hashes in the scope. Just to know what’s going on, we go ahead and print the output.
Finally, we create a new Servicer so that we can update the loan. We execute the contract, submit the results, and handle the response just as before. Once again, we hydrate the loan data and check. At this point, we should be able to look at the printed messages and see that the servicer id and name has changed.
Running the Application
Time to bring it home! As an FYI, I’m going to assume that your local infrastructure was copied and there is a docker folder in your project’s root directory.
Open a terminal instance in your project’s root folder and spin up our local environment.
docker-compose -f ./docker/docker-compose.yaml up -dNote — If this step fails because the object store image cannot be pulled from GitHub container registry, please follow this guide.
Next, we are going to publish our p8e contracts.
source docker/env/bootstrap.env && ./gradlew p8eClean p8eBootstrap --infoFinally, we run our app!
./gradlew application:runOnce the app code has completed, you can test out provenanced - a utility in the Provenance Blockchain image. With this utility, we can check for some of the blockchain information that was output (tx hashes, scope ids, etc).
#get a transaction
docker exec -it provenance provenanced query tx <tx hash>#get a scope
docker exec -it provenance provenanced query metadata scope --include-sessions --include-records <scope id>
Once you’re all done, you can spin down the local environment.
docker-compose -f ./docker/docker-compose.yaml downOur local docker environment also utilizes volumes. This allows us to bring it up and down without needing to reinitialize anything or bootstrap the contracts all the time. If you find yourself wanting to reset your local environment, you can prune the docker volumes like so.
docker-compose -f ./docker/docker-compose.yaml down -vThere you go! Thank you for following along! I hope this sheds some light on the utility of the p8e contract system.
ANTHONY FREMUTH
Anthony is a software engineer from Florida working on solutions that interact with or provide accessibility to Provenance Blockchain. He is currently contributing to the lending and R&D efforts at Figure. Outside of work he enjoys reading, cooking, and traveling.
