Skip to content

Commit 20a3680

Browse files
authored
Merge pull request #48 from Cysharp/circular-reference
Add support Circular Reference
2 parents 3db1e8e + e0a609e commit 20a3680

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+1237
-394
lines changed

README.md

Lines changed: 63 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ Other than performance, MemoryPack has these features.
2121
* Deserialize into existing instance
2222
* Polymorphism(Union) serialization
2323
* limited version-tolerant(fast/default) and full version-tolerant support
24+
* Circular reference serialization
2425
* PipeWriter/Reader based streaming serialization
2526
* TypeScript code generation and ASP.NET Core Formatter
2627
* Unity(2021.3) IL2CPP Support via .NET Source Generator
@@ -528,6 +529,58 @@ By checking the differences in this file, dangerous schema changes can be preven
528529
* member order change
529530
* member deletion
530531

532+
Circular Reference
533+
---
534+
MemoryPack also supports circular reference. This allows the tree objects to be serialized as is.
535+
536+
```csharp
537+
// to enable circular-reference, use GenerateType.CircularReference
538+
[MemoryPackable(GenerateType.CircularReference)]
539+
public partial class Node
540+
{
541+
[MemoryPackOrder(0)]
542+
public Node? Parent { get; set; }
543+
[MemoryPackOrder(1)]
544+
public Node[]? Children { get; set; }
545+
}
546+
```
547+
548+
For example, [System.Text.Json preserve-references](https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/preserve-references) code will become like here.
549+
550+
```csharp
551+
// https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/preserve-references?pivots=dotnet-7-0
552+
Employee tyler = new()
553+
{
554+
Name = "Tyler Stein"
555+
};
556+
557+
Employee adrian = new()
558+
{
559+
Name = "Adrian King"
560+
};
561+
562+
tyler.DirectReports = new List<Employee> { adrian };
563+
adrian.Manager = tyler;
564+
565+
var bin = MemoryPackSerializer.Serialize(tyler);
566+
Employee? tylerDeserialized = MemoryPackSerializer.Deserialize<Employee>(bin);
567+
568+
Console.WriteLine(tylerDeserialized?.DirectReports?[0].Manager == tylerDeserialized); // true
569+
570+
[MemoryPackable(GenerateType.CircularReference)]
571+
public partial class Employee
572+
{
573+
[MemoryPackOrder(0)]
574+
public string? Name { get; set; }
575+
[MemoryPackOrder(1)]
576+
public Employee? Manager { get; set; }
577+
[MemoryPackOrder(2)]
578+
public List<Employee>? DirectReports { get; set; }
579+
}
580+
```
581+
582+
`GenerateType.CircularReference` has the same characteristics as version-tolerant. However, as an additional constraint, only parameterless constructors are allowed. Also, object reference tracking is only done for objects marked with `GenerateType.CircularReference`. If you want to track any other object, wrap it.
583+
531584
CustomFormatter
532585
---
533586
If implements `MemoryPackCustomFormatterAttribute<T>`, you can configure to use custom formatter to MemoryPackObject's member.
@@ -871,15 +924,16 @@ Unity version is not supported CustomFormatter and ImmutableCollections.
871924

872925
Binary wire format specification
873926
---
874-
The type of `T` defined in `Serialize<T>` and `Deserialize<T>` is called C# schema. MemoryPack format is not self described format. Deserialize requires the corresponding C# schema. Seven types exist as internal representations of binaries, but types cannot be determined without a C# schema.
927+
The type of `T` defined in `Serialize<T>` and `Deserialize<T>` is called C# schema. MemoryPack format is not self described format. Deserialize requires the corresponding C# schema. These types exist as internal representations of binaries, but types cannot be determined without a C# schema.
875928

876929
Endian must be `Little Endian`. However reference C# implementation does not care endianness so can not use on big-endian machine. However modern computers are usually little-endian.
877930

878-
There are seven types of format.
931+
There are eight types of format.
879932

880933
* Unmanaged struct
881934
* Object
882935
* Version Tolerant Object
936+
* Circular Reference Object
883937
* Tuple
884938
* Collection
885939
* String
@@ -901,6 +955,13 @@ Object has 1byte unsigned byte as member count in header. Member count allows `0
901955

902956
Version Tolerant Object is similar as Object but has byte length of values in header. varint follows these spec, first sbyte is value or typeCode and next X byte is value. 0 to 127 = unsigned byte value, -1 to -120 = signed byte value, -121 = byte, -122 = sbyte, -123 = ushort, -124 = short, -125 = uint, -126 = int, -127 = ulong, -128 = long.
903957

958+
### Circular Reference Object
959+
960+
`(byte memberCount, [varint byte-length-of-values...], varint referenceId, [values...])`
961+
`(250, varint referenceId)`
962+
963+
Circular Reference Object is similar as Version Tolerant Object but if memberCount is 250, next varint(unsigned-int32) is referenceId. If not, after byte-length-of-values, varint referenceId is written.
964+
904965
### Tuple
905966

906967
`(values...)`

sandbox/Benchmark/Benchmark.csproj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
<ImplicitUsings>enable</ImplicitUsings>
77
<AllowUnsafeBlocks>True</AllowUnsafeBlocks>
88
<Nullable>enable</Nullable>
9-
9+
<TieredPGO>true</TieredPGO>
1010
<IsPackable>false</IsPackable>
1111
</PropertyGroup>
1212

sandbox/Benchmark/Micro/GetLocalVsStaticField.cs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ public GetLocalVsStaticField()
2424
[Benchmark(Baseline = true)]
2525
public void GetFromProvider()
2626
{
27-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
27+
using var state = MemoryPackWriterOptionalStatePool.Rent(MemoryPackSerializeOptions.Default);
28+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
2829
for (int i = 0; i < 100; i++)
2930
{
3031
writer.GetFormatter<int>().Serialize(ref writer, ref i);
@@ -35,7 +36,8 @@ public void GetFromProvider()
3536
[Benchmark]
3637
public void GetFromLocal()
3738
{
38-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
39+
using var state = MemoryPackWriterOptionalStatePool.Rent(MemoryPackSerializeOptions.Default);
40+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
3941
var provider = writer.GetFormatter<int>();
4042
for (int i = 0; i < 100; i++)
4143
{

sandbox/Benchmark/Micro/RawSerialize.cs

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,8 @@ public byte[] HandMemoryPackWriterEmpty()
7171
bufWriter = staticWriter = new ReusableLinkedArrayBufferWriter(true, true);
7272
}
7373

74-
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), MemoryPackSerializeOptions.Default);
74+
var state = MemoryPackWriterOptionalStatePool.Rent(null);
75+
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), state);
7576
try
7677
{
7778
if (value == null)
@@ -94,6 +95,7 @@ public byte[] HandMemoryPackWriterEmpty()
9495
finally
9596
{
9697
bufWriter.Reset();
98+
state.Reset();
9799
}
98100
}
99101

@@ -106,7 +108,8 @@ public byte[] HandMemoryPackWriterHeaderOnly()
106108
bufWriter = staticWriter = new ReusableLinkedArrayBufferWriter(true, true);
107109
}
108110

109-
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), MemoryPackSerializeOptions.Default);
111+
var state = MemoryPackWriterOptionalStatePool.Rent(null);
112+
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), state);
110113
try
111114
{
112115
if (value == null)
@@ -128,6 +131,7 @@ public byte[] HandMemoryPackWriterHeaderOnly()
128131
finally
129132
{
130133
bufWriter.Reset();
134+
state.Reset();
131135
}
132136
}
133137

@@ -140,7 +144,8 @@ public byte[] HandMemoryPackWriterHeaderInt3()
140144
bufWriter = staticWriter = new ReusableLinkedArrayBufferWriter(true, true);
141145
}
142146

143-
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), MemoryPackSerializeOptions.Default);
147+
var state = MemoryPackWriterOptionalStatePool.Rent(null);
148+
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), state);
144149
try
145150
{
146151
if (value == null)
@@ -162,6 +167,7 @@ public byte[] HandMemoryPackWriterHeaderInt3()
162167
finally
163168
{
164169
bufWriter.Reset();
170+
state.Reset();
165171
}
166172
}
167173

@@ -174,7 +180,8 @@ public byte[] HandMemoryPackWriterHeaderInt3String1()
174180
bufWriter = staticWriter = new ReusableLinkedArrayBufferWriter(true, true);
175181
}
176182

177-
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), MemoryPackSerializeOptions.Default);
183+
var state = MemoryPackWriterOptionalStatePool.Rent(null);
184+
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), state);
178185
try
179186
{
180187
if (value == null)
@@ -196,6 +203,7 @@ public byte[] HandMemoryPackWriterHeaderInt3String1()
196203
finally
197204
{
198205
bufWriter.Reset();
206+
state.Reset();
199207
}
200208
}
201209

@@ -208,7 +216,8 @@ public byte[] HandMemoryPackFull()
208216
bufWriter = staticWriter = new ReusableLinkedArrayBufferWriter(true, true);
209217
}
210218

211-
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), MemoryPackSerializeOptions.Default);
219+
var state = MemoryPackWriterOptionalStatePool.Rent(null);
220+
var writer = new MemoryPackWriter<ReusableLinkedArrayBufferWriter>(ref bufWriter, bufWriter.DangerousGetFirstBuffer(), state);
212221
try
213222
{
214223
if (value == null)
@@ -230,6 +239,7 @@ public byte[] HandMemoryPackFull()
230239
finally
231240
{
232241
bufWriter.Reset();
242+
state.Reset();
233243
}
234244
}
235245

sandbox/Benchmark/Micro/StaticAbstractVsFormatter.cs

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,17 @@ public StaticAbstractVsFormatter()
1414
this.value = new IntClass { Value = 999999 };
1515
this.bufferWriter = new ArrayBufferWriter<byte>(99999);
1616

17-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
17+
using var state = MemoryPackWriterOptionalStatePool.Rent(null);
18+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
1819
this.formatter = writer.GetFormatter<IntClass>();
1920
}
2021

2122
[Benchmark]
2223
public void WriteValue()
2324
{
2425
bufferWriter.Clear();
25-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
26+
using var state = MemoryPackWriterOptionalStatePool.Rent(null);
27+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
2628

2729
writer.WriteValue(value); // GetFormatter<T>.Serialize(ref writer, ref value);
2830
}
@@ -31,23 +33,26 @@ public void WriteValue()
3133
public void FormatterSerialize()
3234
{
3335
bufferWriter.Clear();
34-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
36+
using var state = MemoryPackWriterOptionalStatePool.Rent(null);
37+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
3538
formatter.Serialize(ref writer, ref value!); // IMemoryPackFormatter<T>.Serialize(ref writer, rf value)
3639
}
3740

3841
[Benchmark(Baseline = true)]
3942
public void WritePackable()
4043
{
4144
bufferWriter.Clear();
42-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
45+
using var state = MemoryPackWriterOptionalStatePool.Rent(null);
46+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
4347
writer.WritePackable(value); // T.Serialize(ref writer, ref value);
4448
}
4549

4650
[Benchmark]
4751
public void Direct()
4852
{
4953
bufferWriter.Clear();
50-
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, MemoryPackSerializeOptions.Default);
54+
using var state = MemoryPackWriterOptionalStatePool.Rent(null);
55+
var writer = new MemoryPackWriter<ArrayBufferWriter<byte>>(ref bufferWriter, state);
5156
writer.WriteUnmanagedWithObjectHeader(1, value.Value);
5257
}
5358

0 commit comments

Comments
 (0)