Skip to content

Commit c277cb6

Browse files
authored
DSNP on Frequency (#197)
The pre-release of DSNP on Frequency https://github.com/LibertyDSNP/spec/milestone/3
1 parent d85ae8f commit c277cb6

File tree

10 files changed

+217
-9
lines changed

10 files changed

+217
-9
lines changed

.yaspellerrc.json

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,14 +51,15 @@
5151
"Kotlin",
5252
"LibertyDSNP",
5353
"Lookups",
54+
"MSAs",
5455
"Markdown",
5556
"Mastodon",
5657
"MessagePack",
5758
"MongoDB",
5859
"NodeJS",
5960
"Nonces",
6061
"OpenZeppelin",
61-
"pre-1",
62+
"Payload",
6263
"PrivateBroadcast",
6364
"PrivateGraph",
6465
"PrivateGraphChange",
@@ -69,6 +70,7 @@
6970
"Reddit",
7071
"RocksDB",
7172
"SQLite",
73+
"TypeScript",
7274
"URL[s]?",
7375
"UpgradeableBeacon",
7476
"WebM",
@@ -92,6 +94,7 @@
9294
"ecrecover",
9395
"emoji",
9496
"enum[s]?",
97+
"extrinsics",
9598
"followee",
9699
"foundational",
97100
"fromAddress",
@@ -100,8 +103,8 @@
100103
"hostnames",
101104
"https",
102105
"implementers",
103-
"indexers",
104106
"inReplyTo",
107+
"indexers",
105108
"internationalization",
106109
"keccak",
107110
"keyList",
@@ -116,6 +119,7 @@
116119
"officiation",
117120
"parseable",
118121
"permissioned",
122+
"pre-1",
119123
"prepended",
120124
"prosumer",
121125
"regex",
@@ -139,14 +143,14 @@
139143
"timestamps",
140144
"toAddress",
141145
"tombstoned",
142-
"TypeScript",
143146
"unencrypted",
144-
"unfollow[s]?",
145147
"unfollow(s|ed|ing)",
148+
"unfollow[s]?",
146149
"unicode",
147150
"url",
148151
"validator",
149152
"vanishingly",
150-
"versioning"
153+
"versioning",
154+
"websocket"
151155
]
152156
}

pages/DSNP/Announcements.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ That produced Batch Publications that were generic and disconnected from the use
7070
They could be submitted to the chain via anyone not just delegates or users.
7171

7272
In DSNP v1.1, Announcement signatures were removed in favor of the implementation being responsible for the connection between the on-chain signature and the user.
73-
The expected and [EVM implementation](../Ethereum/Validation.md) is that the implementation chain requires that the transaction that produces a Batch be performed by the user or delegate directly.
73+
Implementations require that the transaction that produces a Batch be performed by the user or delegate directly.
7474
This created batches that are delegate specific, but allows for faster testing of the validity of individual Announcements in a Batch.
7575

7676
For more information see [DIP-145](https://github.com/LibertyDSNP/spec/issues/145).

pages/Ethereum/Overview.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
# Ethereum/EVM Compatible DSNP Implementation
22
__Version 1.1.0__
33

4+
### Deprecation Warning: EVM implementation is deprecated in favor of the [Frequency Implementation](../Frequency/Overview.md)
5+
46
DSNP on Ethereum is designed using smart contracts and log messages.
57
Smart contracts are used for identity and delegation.
68
Log messages are used for announcement publishing via batches.

pages/Frequency/Identity.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Identity
2+
3+
## Purpose
4+
5+
1. Specify how DSNP Identifiers map to Frequency.
6+
1. Specify how the DSNP delegation maps to Frequency.
7+
1. Detail how Frequency matches the ownership requirements of DSNP.
8+
9+
## Details
10+
11+
DSNP uses [Message Source Accounts (MSAs)](https://libertydsnp.github.io/frequency/pallet_msa/index.html) that map directly to a DSNP Identity.
12+
13+
## Identifier
14+
15+
MSAs each receive a unique unsigned 64 bit identifier that is used as the DSNP Id.
16+
These MSAs are pseudo-anonymous identifiers that are associated with one or more public keys.
17+
MSAs can be created by the user or on behalf of the user using a signed authorization.
18+
19+
### Creating a DSNP User Id
20+
21+
1. Generate a [compatible key pair](https://wiki.polkadot.network/docs/learn-keys) in your wallet.
22+
1. Create a Message Source Account (MSA) on Frequency using one of the `create` extrinsics.
23+
1. Wait for the transaction to process.
24+
25+
### Retrieving a DSNP User Id from a Public Key
26+
27+
1. Use the Frequency RPC call `msa_getMsaId` to retrieve the MSA Id from the public key generated in step 1.
28+
1. The MSA Id is the DSNP Id and can be used to generate the [DSNP User URI](../DSNP/Identifiers.md#dsnp-user-uri).
29+
30+
## Ownership
31+
32+
Frequency has a strong ownership model for MSAs.
33+
Ownership of an MSA is controlled by the associated keys.
34+
MSAs are the source for [Messages](https://libertydsnp.github.io/frequency/pallet_messages/index.html) either directly or via delegation.
35+
36+
## Delegation
37+
38+
Frequency allows for delegation to others called [Providers](https://libertydsnp.github.io/frequency/pallet_msa/index.html).
39+
Delegation comes with permissions that grant publishing of specific Schemas to a provider.
40+
Since each [Announcement Type](../DSNP/Announcements.md) has a [set Schema Id](./Publishing.md), a user may permission specific Announcement Types.
41+
42+
When a user delegates to a Provider, that delegation is either validated on-chain or can be validated off-chain using the `msa_checkDelegations` RPC call.
43+
Users can revoke a delegation at anytime without any fees, and revocation is locked to the point the transaction is included in a block.

pages/Frequency/Overview.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Frequency DSNP Implementation (Pre-Release)
2+
__Version pre-1.0.0__
3+
4+
_Final release is pending release of [Frequency v1.0.0](https://github.com/LibertyDSNP/frequency)_
5+
6+
DSNP on Frequency is built using Message Source Accounts (MSAs) for identity and delegation with Schemas defined for each Announcement Type.
7+
Frequency uses Schemas to define how to create and pass messages.
8+
9+
## Links
10+
11+
- [Frequency Documentation](https://libertydsnp.github.io/frequency/)
12+
- [Frequency GitHub](https://github.com/LibertyDSNP/frequency)
13+
14+
## Schemas
15+
16+
Official DSNP Schemas may be found in [GitHub](https://github.com/LibertyDSNP/schemas).
17+
18+
## Libraries
19+
<!-- yaspeller ignore:start -->
20+
| Name | Language(s) |
21+
| --- | --- |
22+
| [TODO]() | JavaScript/TypeScript |
23+
<!-- yaspeller ignore:end -->
24+
25+
<!--- Uncomment for pre-release changes and prefix the version with `pre-[next version]`
26+
## Prerelease Changelog
27+
28+
- [DIP-###](https://github.com/LibertyDSNP/spec/issues/###)
29+
30+
--->
31+
## Releases
32+
33+
| Version | Description | DSNP Compatibility | Release Date | Changelog |
34+
| --- | --- | --- | --- | --- |
35+
| `pre-1.0.0` | Pre-Release | 1.1.x | 2022-09-19 | N/A |
36+
<!--
37+
| [1.0.0](https://github.com/LibertyDSNP/spec/tree/Frequency-v1.0.0) | Initial Release | 1.1.x | 2022-10-15 | [Changelog](https://github.com/LibertyDSNP/spec/releases/tag/Frequency-v1.0.0) |
38+
-->

pages/Frequency/Publishing.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Announcement Publishing
2+
3+
On Frequency, [Announcements](../DSNP/Announcements.md) are mapped to schemas which in turn publish Frequency Messages.
4+
Frequency Messages are either direct Graph Changes from a particular user, or a Batch Publication with a multitude of users possible.
5+
6+
<!-- Links to https://libertydsnp.github.io/frequency should be updated with links to docs.frequency.xyz when able to be -->
7+
8+
| Announcement Type Enum | Announcement | Batched | Schema Id Mainnet | Schema Id Rococo | Frequency Model Type | Frequency Payload Location |
9+
| --- | --- | --- | --- | --- | --- | --- |
10+
| 0 | [Tombstone](../DSNP/Types/Tombstone.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
11+
| 1 | [Graph Change](../DSNP/Types/GraphChange.md) | no | TBD | TBD | [`AvroBinary`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`OnChain`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.OnChain) |
12+
| 2 | [Broadcast](../DSNP/Types/Broadcast.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
13+
| 3 | [Reply](../DSNP/Types/Reply.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
14+
| 4 | [Reaction](../DSNP/Types/Reaction.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
15+
| 5 | [Profile](../DSNP/Types/Profile.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
16+
| 6 | [Update](../DSNP/Types/Update.md) | YES | TBD | TBD | [`Parquet`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://libertydsnp.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
17+
18+
Source code for each schema is also located in the [LibertyDSNP/schemas](https://github.com/LibertyDSNP/schemas) repository.
19+
20+
## DSNP Non-Batched Announcements
21+
22+
Frequency uses an on-chain data structure for storage of `GraphChange` Announcements.
23+
Each individual announcement is submitted using the [`add_onchain_message`](https://libertydsnp.github.io/frequency/pallet_messages/pallet/enum.Call.html#variant.add_onchain_message) extrinsic.
24+
25+
## DSNP Batched Announcements
26+
27+
Frequency uses [DSNP Batch Publications](../DSNP/BatchPublications.md) for Announcements that are batched.
28+
The parquet file is stored on [IPFS](https://ipfs.io/), but it is discovered through the Frequency Message.
29+
30+
Frequency Messages maintain the metadata of where and when the Batch Publication was published and which Provider MSA published it.
31+
It is the publisher's responsibility to maintain the [IPFS pin](https://docs.ipfs.tech/concepts/glossary/#pinning) so that the batch file is continuously available.
32+
33+
DSNP Batch Publications [MUST be validated](./Validation.md) upon fetching to ensure data and permission integrity.
34+
35+
## Ordering
36+
37+
Frequency Messages are well ordered within a Schema
38+
39+
1. Frequency: Block number ascending
40+
2. Frequency: Block index ascending
41+
3. DSNP Standard: Order Announcements in a Batch Publication File by row appearance order
42+
43+
### Ordering Across Schemas
44+
45+
Frequency provides complete ordering metadata for Messages across Schemas.
46+
Block index is unique per Message within the same block.
47+
48+
### Human Order
49+
50+
Due to the asynchronous nature of networks and batching, it is possible that the canonical ordering of Announcements is wrong from a human viewpoint.
51+
With dependent Announcements, where one Announcement refers to another Announcement, the order may be inferred differently than the canonical ordering.
52+
It is left to user interfaces to handle these situations.
53+
54+
55+
## Retrieval
56+
57+
Frequency nodes provide an RPC interface [`get_messages_by_schema`](https://libertydsnp.github.io/frequency/pallet_messages_rpc/trait.MessagesApiClient.html#method.get_messages_by_schema) with paginated responses that differ based on the Schema.
58+
59+
Frequency nodes can provide a websocket interface that will emit an event for each block that has one or more messages of a given schema in that block.
60+
The [`MessagesStored`](https://libertydsnp.github.io/frequency/pallet_messages/pallet/enum.Event.html#variant.MessagesStored) event can be used to know when to call the RPC interface to retrieve the messages.
61+
62+
See the [Frequency Documentation](https://libertydsnp.github.io/frequency/pallet_messages_rpc/trait.MessagesApiClient.html#method.get_messages_by_schema) for more details on Message retrieval.

pages/Frequency/Validation.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Announcement Validation
2+
3+
DSNP Announcements are validated differently depending on the type of Announcement.
4+
Batched Announcements are off-chain and MUST be validated at read time.
5+
Non-batched Announcements (example: Graph Change), are on-chain and are validated at write time and do not need to be re-validated at read time.
6+
7+
## On-chain Announcements
8+
9+
Creation of an Announcement as an on-chain Frequency Message requires that the delegation be validated before adding the message to the chain.
10+
Thus, any Graph Change Announcements read from a trusted node, can be trusted to have been made by the DSNP identity or a delegate of the identity at the time the Frequency Message was sent.
11+
12+
### On-Chain Announcement Types
13+
- Graph Change
14+
15+
16+
## Batch Publication Announcements
17+
18+
DSNP Announcements that are in a Batch Publication MUST be validated at read time.
19+
20+
### Batched Announcement Types
21+
- Tombstone
22+
- Broadcast
23+
- Reply
24+
- Reaction
25+
- Profile
26+
27+
<!-- Links to https://libertydsnp.github.io/frequency should be updated with links to docs.frequency.xyz when able to be -->
28+
29+
## Batch Publication Frequency Message Validation
30+
31+
The [Frequency Message](https://libertydsnp.github.io/frequency/common_primitives/messages/struct.MessageResponse.html) for a Batch Publication has several important fields for validation:
32+
33+
| Field | Description |
34+
| ----- | ----------- |
35+
| `provider_msa_id` | MSA Id of the provider sending the message |
36+
| `cid` | The [Content IDentifier v1](https://github.com/multiformats/cid/) for IPFS content |
37+
| `payload_length` | Expected length of the content from IPFS |
38+
| `block_number` | Block number that the message was recorded on the chain. |
39+
40+
### File Validation
41+
42+
1. Retrieve the file from the IPFS network using the `cid`.
43+
1. (Required for non-trusted IPFS nodes) Verify the file hash by [comparing it to the hash included in the `cid`](https://docs.ipfs.tech/concepts/hashing/).
44+
1. Verify that the byte length of the retrieved file matches the `payload_length`.
45+
46+
### Publication Announcements Validation
47+
48+
1. Collect the unique set of `fromId` values.
49+
1. Use the [Custom RPC `msa_checkDelegations`](https://libertydsnp.github.io/frequency/pallet_msa_rpc/trait.MsaApiClient.html#method.check_delegations) with the `fromId` values as the `delegator_msa_ids` and the `provider_msa_id` at the `block_number`.
50+
1. The `fromId` values that `msa_checkDelegations` verifies as having a delegation at `block_number` are valid Announcements.
51+
52+
## Announcement Duplicates
53+
54+
Duplicate Announcements MUST be rejected or ignored.

pages/Implementations.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
# DSNP Implementations
22

3-
- [Ethereum/EVM Compatible](Ethereum/Overview.md) (Official)
3+
- [Frequency](Frequency/Overview.md) (Official)
4+
- [Ethereum/EVM Compatible](Ethereum/Overview.md) (Deprecated)

pages/SUMMARY.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,11 @@
1717
- [Update](DSNP/Types/Update.md)
1818
- [Serializations](DSNP/Serializations.md)
1919
- [DSNP Implementation Specs](Implementations.md)
20-
- [EVM](Ethereum/Overview.md)
20+
- [Frequency](Frequency/Overview.md)
21+
- [Identity](Frequency/Identity.md)
22+
- [Publishing](Frequency/Publishing.md)
23+
- [Validation](Frequency/Validation.md)
24+
- [EVM (Deprecated)](Ethereum/Overview.md)
2125
- [Identity](Ethereum/Identity.md)
2226
- [Identifiers](Ethereum/Identifiers.md)
2327
- [Factory](Ethereum/IdentityFactory.md)

pages/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ There are three core specifications currently.
1616
| Name | Version | Description |
1717
| --- | --- | --- |
1818
| [DSNP](DSNP/Overview.md) | 1.1.0 | The implementation-agnostic DSNP specification |
19-
| [DSNP Implementations](Implementations.md) | - | Implementation-specific specifications |
19+
| [DSNP on Frequency (Pre-release)](Frequency/Overview.md) | pre-1.0.0 | Frequency implementation for DSNP specification |
2020
| [Activity Content](ActivityContent/Overview.md) | 1.1.0 | A specification for DSNP-referenced content (subset of W3C Activity Streams) |
2121

2222
Each specification is divided into several modules.

0 commit comments

Comments
 (0)