Skip to content

Commit c407aba

Browse files
wesbiggsWes Biggs
and
Wes Biggs
authored
DIP-263 Use User Data for Public Keys (#276)
Problem ======= See #263 Solution ======== - Moved old Public Key Announcement to "Migrated Announcements". - Added Avro type PublicKey. - Updated User Data section to define key agreement and assertion method key types. --------- Co-authored-by: Wes Biggs <[email protected]>
1 parent f782332 commit c407aba

File tree

10 files changed

+79
-37
lines changed

10 files changed

+79
-37
lines changed

.spellcheckerdict.txt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ DSNP
2525
DSNP-compatible
2626
DSNP-referenced
2727
ECIES
28+
EdDSA
2829
[Ee]num(s)?
2930
Ethereum
3031
Extrinsics
@@ -67,7 +68,7 @@ Polkadot
6768
Poly1305
6869
pre-configured
6970
PRId([ABs])?
70-
publicKey
71+
[Pp]ublicKey
7172
repo
7273
RFC[1-9][0-9]*
7374
Ristretto
@@ -78,6 +79,7 @@ schemaless
7879
Schnorrkel
7980
SDK
8081
SHA-256
82+
SHA-512
8183
Stateful
8284
stringified
8385
subkey

pages/DSNP/Announcements.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,11 @@ Each Announcement has an enumerated type for use when separating out a stream of
1717
| 4 | [Reaction](Types/Reaction.md) | a public visual reply to a Broadcast | no | no |
1818
| 5 | [Profile](Types/Profile.md) | a profile | YES | no |
1919
| 6 | [Update](Types/Update.md) | an update to content| YES | no |
20-
| 7 | [Public Key](Types/PublicKey.md) | a public key for secure communication | no | no |
20+
| 7 | ~~[Public Key](Types/PublicKey.md)~~<sup>b</sup> | ~~a public key for secure communication~~ | ~~no~~ | ~~no~~ |
2121

22-
<sup>a</sup> Since DSNP version 1.2, social graph changes use [User Data](UserData.md) operations as described in the [Graph](Graph.md) section.
22+
<sup>a</sup> Since DSNP version 1.2, social graph changes use [User Data](UserData.md) operations as described in the [Graph](Graph.md) section.
23+
24+
<sup>b</sup> Since DSNP version 1.3, public keys use [User Data](UserData.md) operations.
2325

2426
## Announcement Validation
2527

pages/DSNP/Operations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Compliant implementations may respond to error conditions either synchronously,
3434
| <a id="remove-control-key">Remove Control Key</a> | YES | User | Key | [Control Key Removal Record](Records.md#control-key-removal) |
3535
| <a id="publish-announcement">Publish Announcement</a> | no* | User OR Delegate | [Announcement](Announcements.md) | [Announcement Published Record](Records.md#announcement-published) |
3636
| <a id="publish-batch">Publish Batch</a> | no* | User OR Delegate | [Announcement Type](Announcements.md#announcement-types), [Batch Publication](BatchPublications.md) URL, Batch Publication Content Hash | [Batch Published Record](Records.md#batch-published) |
37-
| <a id="get-user-data">Get User Data</a> | no | Any | User's Identifier, Set of Requested [User Data Types](UserData.md#user-data-types) | Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) with optional [Key Identifiers](Types/PublicKey.md#keyid) |
37+
| <a id="get-user-data">Get User Data</a> | no | Any | User's Identifier, Set of Requested [User Data Types](UserData.md#user-data-types) | Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) with optional key identifiers of encryption keys for each chunk |
3838
| <a id="replace-user-data">Replace User Data</a> | no | User OR Delegate | User's Identifier, [Key Identifier](Types/PublicKey.md#keyid), Map of [User Data Types](UserData.md#user-data-types) to [Data Chunks](UserData.md#data-chunks) | [User Data Replaced Record](Records.md#user-data-replaced) |
3939

4040
\* For each Announcement Type, an implementation may support one or both of these operations.

pages/DSNP/Types/PublicKey.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Public Key Announcement
22

3+
<mark>_Since DSNP version 1.3, public keys use [User Data](../UserData.md) operations._</mark>
4+
35
A Public Key Announcement is a way to note a new cryptographic key that can be used in DSNP to secure and verify the authenticity of communications.
46

57
The most recently published key (if one exists) for a given key type should be treated as the active key of that key type.

pages/DSNP/Types/PublicKeyUserData.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Public Key
2+
3+
Represents an encoding of a public key, one half of a cryptographic key pair.
4+
5+
## Serialization
6+
7+
PublicKey object serialization MUST conform to the following [Avro](https://avro.apache.org) schema:
8+
9+
```
10+
{
11+
"namespace": "org.dsnp",
12+
"name": "PublicKey",
13+
"type": "record",
14+
"fields": [
15+
{
16+
"name": "publicKey",
17+
"type": "bytes",
18+
"doc": "Multicodec public key"
19+
}
20+
]
21+
}
22+
```
23+
24+
## Generation
25+
26+
### publicKey
27+
28+
- MUST be a public key of an allowed key type for the associated User Data type, encoded in `multicodec` format
29+
30+
The byte encoding consists of a [multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv) key identifier (as a varint) followed by the public key's binary data in the codec's described format.
31+
32+
#### Allowed Key Types
33+
34+
| User Data Type | Allowed Algorithms ([multicodec](https://github.com/multiformats/multicodec/blob/master/table.csv)) | Purpose |
35+
| --- | --- | --- |
36+
| `keyAgreementPublicKeys` | `x25519-pub` | A Curve25519 public key that can be used in key exchange protocols to generate a shared secret |
37+
| `assertionMethodPublicKeys` | `ed25519-pub` | A public key for the EdDSA signature scheme using SHA-512 and Curve25519 that can be used to verify cryptographic signatures |

pages/DSNP/UserData.md

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,11 @@ DSNP implementations MUST support the following User Data Types:
1919
| <a name="private-follows">`privateFollows`</a> | 1.2 | `curve25519xsalsa20poly1305` | [`DEFLATE`](https://en.wikipedia.org/wiki/Deflate) | [GraphEdge](Types/GraphEdge.md) |
2020
| <a name="private-connections">`privateConnections`</a> | 1.2 | `curve25519xsalsa20poly1305` | [`DEFLATE`](https://en.wikipedia.org/wiki/Deflate) | [GraphEdge](Types/GraphEdge.md) |
2121
| <a name="private-connection-prids">`privateConnectionPRIds`</a> | 1.2 | NONE | NONE | [PRId](Types/PRId.md) |
22+
| <a name="key-agreement-public-keys">`keyAgreementPublicKeys`</a> | 1.3 | NONE | NONE | [PublicKey](Types/PublicKeyUserData.md) |
23+
| <a name="assertion-method-public-keys">`assertionMethodPublicKeys`</a> | 1.3 | NONE | NONE | [PublicKey](Types/PublicKeyUserData.md) |
2224

2325
Data for each data type is initially formatted as a stream of Avro objects that should conform to the schema specified.
26+
A DSNP system MAY limit the number of objects allowed for a given user data type; if so, this MUST be documented.
2427
Avro file- and block-level information (including in-stream schema) is omitted.
2528
The Avro stream is then compressed and/or encrypted as specified.
2629

@@ -29,7 +32,7 @@ In the specification of cryptographic operations below, relevant methods from th
2932

3033
## Data Chunks
3134

32-
Because blockchain systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks.
35+
Because consensus systems often have specific limits to the amount of data that can be included in a given transaction, operations on user data deal with the data in discrete chunks.
3336
As implementation strategies may vary, implementations MUST define their own maximum chunk size in bytes to be used in the operations described below.
3437

3538
## Entity Tags
@@ -46,8 +49,8 @@ The Replace User Data Operation takes the following parameters:
4649

4750
* A DSNP User Id
4851
* Implementations MUST ensure that the principal invoking this Operation is this user, or a transparent chain of delegation from the user to the principal exists.
49-
* A [Key Identifier](Types/PublicKey.md#keyid) for the `keyAgreement` key pair used to encrypt any private data in the operation.
50-
(If only unencrypted user data types are included, the key identifier is optional.)
52+
* The index of the `keyAgreementPublicKeys` key pair used to encrypt any private data in the operation.
53+
(If only unencrypted user data types are included, the key index may be omitted.)
5154
* A map containing the set of data types to update as the keys, and tuples consisting of (1) the schema version used to encode the data type, and (2) a list where each element includes a data chunk and its associated entity tag, as the values.
5255

5356
If the Operation is successful, any previous data associated with the user for each data type included in the input MUST be removed and replaced by the new data.
@@ -61,9 +64,9 @@ Data chunks should be generated for each included data type using the following
6164
3. For each chunk generated, the application should then:
6265
1. If the data type requires compression, apply the compression codec noted.
6366
1. If the data type requires encryption,
64-
1. Retrieve the user's active (most recently announced) `keyAgreement` public key, U<sub>public</sub>.
65-
The `keyId` in the announcement should match the key identifier provided for this Operation.
66-
If no key exists, one should be created and published as an Announcement before invoking the Operation.
67+
1. Retrieve the user's active `keyAgreementPublicKey` key, U<sub>public</sub>, and note its index.
68+
If no key exists, one should be created and published as User Data before invoking the Operation.
69+
By convention, the key with the highest index (the last object in the Avro stream) is the active key.
6770
1. Create a sealed box (a payload encrypted with a symmetric key derived from an ephemeral key pair, and accompanied by the ephemeral public key), as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal`, using U<sub>public</sub>.
6871
1. Include the previous `etag` value for the chunk. If the chunk is new, `etag` should be set to `null`.
6972
If any chunks are to be deleted, they should be included in the input identified with the existing `etag` and a `null` value for the data.
@@ -151,13 +154,13 @@ The Get User Data Operation takes the following parameters:
151154
* Note: While _writing_ user data is reserved for the user and any delegates, anyone on the network can read any user's data (though it may be encrypted).
152155
* The User Data Types (by system name) that should be retrieved.
153156

154-
The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key identifier. (Note that this is the same general structure as the input data for [Replace User Data](#replace-user-data-operation), for each requested data type.
157+
The operation returns a mapping of User Data Type to data chunks, with each data chunk annotated with an entity tag and (optionally) a key index. (Note that this is the same general structure as the input data for [Replace User Data](#replace-user-data-operation), for each requested data type.
155158
If no chunks for a requested data type exist, an implementation MAY omit that data type from the response.
156159

157160
To transform the data from the output to Avro binary records, a consumer should apply the following algorithm to each data type included:
158161
1. Determine the relevant encryption algorithm, compression codec, and object schema from the User Data Type and version noted.
159162
1. For each chunk,
160-
1. If encryption is indicated, decrypt the chunk data using the user's secret key (identified using the key identifier) as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal_open`.
163+
1. If encryption is indicated, decrypt the chunk data using the user's secret key (identified using the key index) as in the [libsodium](https://doc.libsodium.org/public-key_cryptography/sealed_boxes) function `crypto_box_seal_open`.
161164
1. If compression is required, uncompress the chunk data using the specified codec.
162165
1. Deserialize the uncompressed data to logical records according to the Avro object schema.
163166
1. Retain the chunk's `etag` value if needed for any updates.

pages/Frequency/Overview.md

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -19,18 +19,19 @@ Official schemas may be found in [GitHub](https://github.com/LibertyDSNP/schemas
1919

2020
<!-- These ids are duplicated here for quick reference. -->
2121

22-
| Name | Schema Id Mainnet | Schema Id Testnet (Paseo) | Schema Id Testnet (Rococo) |
23-
| --- | --- | --- | --- |
24-
| [Tombstone](./Publishing.md) | 1 | 1 | 1 |
25-
| [Broadcast](./Publishing.md) | 2 | 2 | 2 |
26-
| [Reply](./Publishing.md) | 3 | 3 | 3 |
27-
| [Reaction](./Publishing.md)| 4 | 4 | 4 |
28-
| [Profile](./Publishing.md) | 6 | 6 | 5 |
29-
| [Update](./Publishing.md)| 5 | 5 | 6 |
30-
| [Public Key](./Publishing.md)| 7 | 7 | 18 |
31-
| [Public Follows](./UserData.md)| 8 | 8 | 13 |
32-
| [Private Follows](./UserData.md) | 9 | 9 | 14 |
33-
| [Private Connections](./UserData.md) | 10 | 10 | 15 |
22+
| Name | Schema Name | Schema Id Mainnet | Schema Id Testnet (Paseo) | Schema Id Testnet (Rococo) |
23+
| --- | --- | --- | --- | --- |
24+
| [Tombstone](./Publishing.md) | `dsnp.tombstone` | 1 | 1 | 1 |
25+
| [Broadcast](./Publishing.md) | `dsnp.broadcast` | 2 | 2 | 2 |
26+
| [Reply](./Publishing.md) | `dsnp.reply` | 3 | 3 | 3 |
27+
| [Reaction](./Publishing.md)| `dsnp.reaction` | 4 | 4 | 4 |
28+
| [Profile](./Publishing.md) | `dsnp.profile` | 6 | 6 | 5 |
29+
| [Update](./Publishing.md)| `dsnp.update` | 5 | 5 | 6 |
30+
| [Key Agreement Public Keys](./UserData.md)| `dsnp.public-key-key-agreement` | 7 | 7 | 18 |
31+
| [Public Follows](./UserData.md)| `dsnp.public-follows` | 8 | 8 | 13 |
32+
| [Private Follows](./UserData.md) | `dsnp.private-follows` | 9 | 9 | 14 |
33+
| [Private Connections](./UserData.md) | `dsnp.private-connections` | 10 | 10 | 15 |
34+
| [Assertion Method Public Keys](./UserData.md)| `dsnp.public-key-assertion-method` | TBD | 11 | 100 |
3435

3536
<!--
3637
### Obsolete
@@ -50,4 +51,4 @@ Official schemas may be found in [GitHub](https://github.com/LibertyDSNP/schemas
5051

5152
| Last Update Date | Frequency Release | DSNP Version |
5253
| --- | --- | --- |
53-
| 2023-07-17 | 1.5.2+ | 1.2.0 |
54+
| 2024-04-12 | 1.10.0+ | pre-1.3.0 |

pages/Frequency/Publishing.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
# Announcement Publishing
22

3-
On Frequency, [Announcements](../DSNP/Announcements.md) are mapped to Schemas which in turn publish Frequency Messages or Stateful Storage changes.
3+
On Frequency, [Announcements](../DSNP/Announcements.md) are mapped to Schemas which in turn publish Frequency Messages.
44
Frequency Messages are either individual Announcements from a particular user, or a Batch Publication with a multitude of possible users.
5-
Frequency Stateful Storage is either direct Announcements from a particular user or [User Data](./UserData.md) changes.
65

76
<!-- Links to https://frequency-chain.github.io/frequency should be updated with links to docs.frequency.xyz when able to be -->
87

@@ -16,7 +15,6 @@ Frequency Stateful Storage is either direct Announcements from a particular user
1615
| 4 | [Reaction](../DSNP/Types/Reaction.md) | Batched | Mainnet: 4<br />Testnet (Paseo): 4<br />Testnet (Rococo): 4 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
1716
| 5 | [Profile](../DSNP/Types/Profile.md) | Batched | Mainnet: 6<br />Testnet (Paseo): 6<br />Testnet (Rococo): 5 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
1817
| 6 | [Update](../DSNP/Types/Update.md) | Batched | Mainnet: 5<br />Testnet (Paseo): 5<br />Testnet (Rococo): 6 | [`Parquet`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.Parquet) | [`IPFS`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.IPFS) |
19-
| 7 | [Public Key](../DSNP/Types/PublicKey.md) | [Stateful](./UserData.md#announcements) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 18 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) |
2018

2119
Source code for each schema is located in the [LibertyDSNP/schemas](https://github.com/LibertyDSNP/schemas) repository.
2220

pages/Frequency/UserData.md

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,23 @@
11
# Frequency User Data
22

3-
On Frequency, User Data and select Announcements are mapped to Schemas which use [Stateful Storage](https://frequency-chain.github.io/frequency/pallet_stateful_storage/index.html) for storage and retrieval of the data.
3+
On Frequency, User Data is mapped to Schemas which use [Stateful Storage](https://frequency-chain.github.io/frequency/pallet_stateful_storage/index.html) for storage and retrieval of the data.
44

55
## User Data Sets
66

77
<!-- Update ./Overview.md if a Schema Id is updated -->
88

99
| User Data Set | Deployed Schema Ids | Frequency Model Type | Frequency Payload Location | Settings |
1010
| --- | --- | --- | --- | --- |
11+
| [Public Key (Key Agreement)](../DSNP/Types/PublicKeyUserData.md) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 7 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |
1112
| [Public Follows](../DSNP/Graph.md#public-follows) | Mainnet: 8<br />Testnet (Paseo): 8<br />Testnet (Rococo): 13 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
1213
| [Private Follows](../DSNP/Graph.md#private-follows) | Mainnet: 9<br />Testnet (Paseo): 9<br />Testnet (Rococo): 14 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
1314
| [Private Connections](../DSNP/Graph.md#private-connections) | Mainnet: 10<br />Testnet (Paseo): 10<br />Testnet (Rococo): 15 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Paginated`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Paginated) | None |
15+
| [Public Key (Assertion Method)](../DSNP/Types/PublicKeyUserData.md) | Mainnet: TBD <br />Testnet (Paseo): 11<br />Testnet (Rococo): 100 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |
1416

15-
[Pseudonymous Relationship Identifiers](./../DSNP/Graph.md#pseudonymous-relationship-identifiers) (PRIds) are stored along side Private Connections in the same Stateful Storage page.
17+
[Pseudonymous Relationship Identifiers](./../DSNP/Graph.md#pseudonymous-relationship-identifiers) (PRIds) are stored alongside Private Connections in the same Stateful Storage page.
1618

1719
Source code for each schema is located in the [LibertyDSNP/schemas](https://github.com/LibertyDSNP/schemas) repository.
1820

19-
## Announcements
20-
21-
| Announcement | Deployed Schema Ids | Frequency Model Type | Frequency Payload Location | Settings |
22-
| --- | --- | --- | --- | --- |
23-
| [Public Key](../DSNP/Types/PublicKey.md) | Mainnet: 7<br />Testnet (Paseo): 7<br />Testnet (Rococo): 7 | [`AvroBinary`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.ModelType.html#variant.AvroBinary) | [`Itemized`](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.PayloadLocation.html#variant.Itemized) | [Append Only](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.AppendOnly), [Signature Required](https://frequency-chain.github.io/frequency/common_primitives/schema/enum.SchemaSetting.html#variant.SignatureRequired) |
24-
2521
## Read Operation Mapping
2622

2723
Stateful data is retrieved via state queries (`pallet.stateQuery`) or RPC calls (`pallet.rpcCall()`).

0 commit comments

Comments
 (0)