Skip to content

Commit 857169d

Browse files
wilwadeshannonwells
wilwade
and
shannonwells
authored
Schemas Pallet Documentation (#1968)
# Goal The goal of this PR is to improve the documentation of the Pallets and make that documentation be able to be used on docs.frequency.xyz. Part of frequency-chain/docs#59 # Discussion - Schemas Pallet --------- Co-authored-by: Shannon Wells <[email protected]>
1 parent 80a5407 commit 857169d

File tree

3 files changed

+105
-32
lines changed

3 files changed

+105
-32
lines changed

‎common/primitives/src/schema.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,10 +48,10 @@ pub enum PayloadLocation {
4848
#[derive(Copy, Clone, RuntimeDebug, PartialEq, Eq, Encode, Decode, MaxEncodedLen, TypeInfo)]
4949
pub enum SchemaSetting {
5050
/// Schema setting to enforce append-only behavior on payload.
51-
/// Applied to schemas of type `PayloadLocation::Itemized` or `PayloadLocation::Paginated`.
51+
/// Applied to schemas of type `PayloadLocation::Itemized`.
5252
AppendOnly,
5353
/// Schema may enforce signature requirement on payload.
54-
/// Applied to schemas of type `PayloadLocation::Paginated`.
54+
/// Applied to schemas of type `PayloadLocation::Itemized` or `PayloadLocation::Paginated`.
5555
SignatureRequired,
5656
}
5757

‎pallets/schemas/README.md

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Schemas Pallet
2+
3+
The Schemas Pallet provides universal schema registration for data flowing through Frequency.
4+
5+
## Summary
6+
7+
This pallet provides an on chain repository for schemas, thereby allowing participants of the network to flexibly interact and exchange messages with each other with the correct human intent and data structure.
8+
All Messages and Stateful Storage content must be attached to a Schema Identifier so that the content can be correctly parsed and interpreted.
9+
10+
### Data Structure vs Human Intent
11+
12+
Schemas provide for both consistent data structures, but also human intent of the message.
13+
Some schemas may be structurally the same, but have a different interpretation of the contents.
14+
For example, two schemas might both only have a hash for contents, but one is a recording of the hash for time validation purposes, while the other is to mark an off-chain vote.
15+
16+
### Schema Parameters
17+
18+
- Model: The actual JSON representing the data structure.
19+
- Model Type: The type of serialization used. (Parquet, Avro, etc...)
20+
- Settings: Various options for the Schema like signature requirements.
21+
- Payload Location: The location the data for this Schema is stored.
22+
23+
#### Model Types
24+
25+
- [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet): Designed for lists and when a Provider is collecting items from many different MSAs and publishing them together.
26+
- [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary): Useful for most generic data structures.
27+
28+
#### Settings
29+
30+
- [`AppendOnly`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly)
31+
- Prior data is immutable and all new data is appended to existing data.
32+
- For Payload Locations: `Itemized`
33+
- [`SignatureRequired`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired)
34+
- An MSA control key signature is required instead of a delegation.
35+
- For Payload Locations: `Itemized` or `Paginated`
36+
37+
#### Payload Locations
38+
39+
- [`OnChain`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.OnChain): Data is stored directly in the Messages pallet data storage, usually as `AvroBinary`.
40+
- [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS): Data is stored in IPFS and Messages pallet stores the CID.
41+
- [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized): Data is stored in the Stateful Storage pallet as an array of individual items.
42+
- [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated): Data is stored in the Stateful Storage pallet as a list of paged blobs.
43+
44+
### Mainnet vs Testnet Schema Creation
45+
46+
Mainnet schemas must be approved by the Frequency Council.
47+
This is to prevent malicious schemas and increase the documentation around the schemas available.
48+
49+
On Testnets, schemas can be created by anyone, so there are _no_ guarantees around schema correctness or quality.
50+
If you want to use a particular schema on a testnet, it is suggested that you use specific Schema Ids or publish the needed schema yourself.
51+
52+
### Actions
53+
54+
The Schemas pallet provides for:
55+
56+
- Registering or proposing new Schemas.
57+
- Retrieving schemas by their Id or name.
58+
- Validating a Schema model.
59+
- Retrieving last registered Schema Id.
60+
61+
## Interactions
62+
63+
### Extrinsics
64+
65+
| Name/Description | Caller | Payment | Key Events | Runtime Added |
66+
| ----------------------------------------------------------------------------------------------------- | ----------------------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
67+
| `set_max_schema_model_bytes`<br />Governance action to alter the maximum byte length of Schema models | Governance | Tokens | [`SchemaMaxSizeChanged`](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/enum.Event.html#variant.SchemaMaxSizeChanged) | 1 |
68+
| `propose_to_create_schema_v2`<br />Creates a proposal to the Frequency Council for a new schema | Token Account | Tokens | [`Proposed`](https://paritytech.github.io/polkadot-sdk/master/pallet_collective/pallet/enum.Event.html#variant.Proposed) | 66 |
69+
| `create_schema_via_governance_v2`<br />Governance action version of `create_schema_v3` | Frequency Council | Tokens | [`SchemaCreated`](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/enum.Event.html#variant.SchemaCreated) | 66 |
70+
| `create_schema_v3`<br />Creates a new Schema. | Mainnet: Governance<br />Testnet: Token Account | Tokens | [`SchemaCreated`](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/enum.Event.html#variant.SchemaCreated) | 1 |
71+
| `propose_to_create_schema_name`<br />Creates a Council proposal to set the name of a Schema | Token Account | Tokens | [`Proposed`](https://paritytech.github.io/polkadot-sdk/master/pallet_collective/pallet/enum.Event.html#variant.Proposed) | 1 |
72+
| `create_schema_name_via_governance`<br />Governance action to set the name of a Schema | Frequency Council | Tokens | [`SchemaNameCreated`](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/enum.Event.html#variant.SchemaNameCreated) | 66 |
73+
74+
See [Rust Docs](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/struct.Pallet.html) for more details.
75+
76+
### State Queries
77+
78+
| Name | Description | Query | Runtime Added |
79+
| --------------------------------- | ------------------------------------------------------------------- | -------------------------------- | ------------- |
80+
| Get Max Current Schema Identifier | Fetch current Schema Identifier maximum | `currentSchemaIdentifierMaximum` | 1 |
81+
| Get Schema Model Max Bytes | Fetch maximum number of bytes per Schema Model as set by Governance | `governanceSchemaModelMaxBytes` | 1 |
82+
| Get a Schema Info | Fetch the metadata and settings for a schema | `schemaInfos` | 62 |
83+
| Get Schema Ids by Name | Fetch matching Schemas Ids by namespace and name | `schemaNameToIds` | 62 |
84+
| Get Schema Payload/Model | Fetch the payload/model JSON for the specified Schema | `schemaPayloads` | 62 |
85+
86+
See the [Rust Docs](https://frequency-chain.github.io/frequency/pallet_schemas/pallet/storage_types/index.html) for additional state queries and details.
87+
88+
### RPCs
89+
90+
Note: May be restricted based on node settings and configuration.
91+
92+
| Name | Description | Call | Node Version |
93+
| --------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ |
94+
| Get Schema by Id | Retrieves the schema for the given Schema Id | [`getBySchemaId`](https://frequency-chain.github.io/frequency/pallet_schemas_rpc/trait.SchemasApiServer.html#tymethod.get_by_schema_id) | v1.0.0+ |
95+
| Check Schema Validity | Validates a schema model and returns “true” if the model is correct | [`checkSchemaValidity`](https://frequency-chain.github.io/frequency/pallet_schemas_rpc/trait.SchemasApiServer.html#tymethod.check_schema_validity) | v1.0.0+ |
96+
| Get Schema Versions | Returns an array of schema versions | [`getVersions`](https://frequency-chain.github.io/frequency/pallet_schemas_rpc/trait.SchemasApiServer.html#tymethod.get_versions) | v1.10.0+ |
97+
98+
\* Must be enabled with off-chain indexing
99+
100+
See [Rust Docs](https://frequency-chain.github.io/frequency/pallet_schemas_rpc/trait.SchemasApiServer.html) for more details.

‎pallets/schemas/src/lib.rs

Lines changed: 3 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,18 @@
1-
//! # Schemas Pallet
2-
//! The Schemas pallet provides functionality for handling schemas.
1+
//! Schema Management for Message and Stateful Storage
32
//!
3+
//! ## Quick Links
44
//! - [Configuration: `Config`](Config)
55
//! - [Extrinsics: `Call`](Call)
66
//! - [Runtime API: `SchemasRuntimeApi`](../pallet_schemas_runtime_api/trait.SchemasRuntimeApi.html)
77
//! - [Custom RPC API: `SchemasApiServer`](../pallet_schemas_rpc/trait.SchemasApiServer.html)
88
//! - [Event Enum: `Event`](Event)
99
//! - [Error Enum: `Error`](Error)
10-
//!
11-
//! ## Overview
12-
//!
13-
//! This pallet provides an on chain repository for schemas, thereby allowing participants of the
14-
//! network to flexibly interact and exchange messages with each other without facing the challenge
15-
//! of sharing, managing and validating messages as well as schemas between them.
16-
//!
17-
//! <b>NOTE</b>: In this pallet we define the <b>payload</b> structure that is used in <a href="../pallet_messages/index.html">Messages Pallet</a>.
18-
//!
19-
//! The Schema pallet provides functions for:
20-
//!
21-
//! - Registering a new schema.
22-
//! - Setting maximum schema model size by governance.
23-
//! - Retrieving latest registered schema id.
24-
//! - Retrieving schemas by their id.
25-
//!
26-
//!
27-
//! ## Terminology
28-
//!
29-
//! - **Schema:** The structure that defines how a Message is stored and structured.
30-
//! - **Schema Model:** Serialization/Deserialization details of the schema
31-
//! - **Schema Model Type:** The type of the following Serialization/Deserialization. It can be
32-
//! Avro, Parquet or ...
10+
#![doc = include_str!("../README.md")]
3311
//!
3412
//! ## Implementations
3513
//!
3614
//! - [`SchemaValidator`](../common_primitives/schema/trait.SchemaValidator.html): Functions for accessing and validating Schemas. This implementation is what is used in the runtime.
3715
//! - [`SchemaProvider`](../common_primitives/schema/trait.SchemaProvider.html): Allows another pallet to resolve schema information.
38-
//!
39-
//! ## Genesis config
40-
//!
41-
//! The Schemas pallet depends on the [`GenesisConfig`].
42-
//!
4316
// Substrate macros are tripping the clippy::expect_used lint.
4417
#![allow(clippy::expect_used)]
4518
#![cfg_attr(not(feature = "std"), no_std)]

0 commit comments

Comments
 (0)