# MessagePack for C# (.NET, .NET Core, Unity, Xamarin) [![Build Status](https://camo.githubusercontent.com/f36b27088bf1e3dc77e07869b3b039192c059763b3f11021cf924578f3e63a4c/68747470733a2f2f6465762e617a7572652e636f6d2f696c73303038362f4d6573736167655061636b2d4353686172702f5f617069732f6275696c642f7374617475732f4d6573736167655061636b2d4353686172702d43493f6272616e63684e616d653d6d6173746572)](https://dev.azure.com/ils0086/MessagePack-CSharp/_build/latest?definitionId=2&branchName=master) The extremely fast [MessagePack](http://msgpack.org/) serializer for C#. It is 10x faster than [MsgPack-Cli](https://github.com/msgpack/msgpack-cli) and outperforms other C# serializers. MessagePack for C# also ships with built-in support for LZ4 compression - an extremely fast compression algorithm. Performance is important, particularly in applications like games, distributed computing, microservices, or data caches. ![Perf comparison graph](https://cloud.githubusercontent.com/assets/46207/23835716/89c8ab08-07af-11e7-9183-9e9415bdc87f.png) MessagePack has a compact binary size and a full set of general purpose expressive data types. Please have a look at the [comparison with JSON, protobuf, ZeroFormatter section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#comparison) and learn [why MessagePack C# is the fastest](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#performance). ## Table of Contents - [Installation](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#installation) - [NuGet packages](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#nuget-packages) - [Unity](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#unity) - [Migration notes from v1.x](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#migration-notes-from-v1x) - [Quick Start](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#quick-start) - [Analyzer](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#analyzer) - [Built-in supported types](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#built-in-supported-types) - [Object Serialization](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#object-serialization) - [DataContract compatibility](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#datacontract-compatibility) - [Serializing readonly/immutable object members (SerializationConstructor)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#serializing-readonlyimmutable-object-members--serializationconstructor) - [Serialization Callback](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#serialization-callback) - [Union](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#union) - [Dynamic (Untyped) Deserialization](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#dynamic-untyped-deserialization) - [Object Type Serialization](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#object-type-serialization) - [Typeless](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#typeless) - [Security](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#security) - [Performance](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#performance) - [Deserialization Performance for different options](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#deserialization-performance-for-different-options) - [String interning](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#string-interning) - [LZ4 Compression](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#lz4-compression) - [Attributions](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#attributions) - [Comparison with protobuf, JSON, ZeroFormatter](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#comparison-with-protobuf-json-zeroformatter) - [Hints to achieve maximum performance when using MessagePack for C#](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#hints-to-achieve-maximum-performance-when-using-messagepack-for-c) - [Use indexed keys instead of string keys (Contractless)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#use-indexed-keys-instead-of-string-keys-contractless) - [Create own custom composite resolver](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#create-own-custom-composite-resolver) - [Use native resolvers](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#use-native-resolvers) - [Be careful when copying buffers](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#be-careful-when-copying-buffers) - [Choosing compression](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#choosing-compression) - [Extensions](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#extensions) - [Experimental Features](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#experimental-features) - [High-Level API (`MessagePackSerializer`)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#high-level-api-messagepackserializer) - [Multiple MessagePack structures on a single `Stream`](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#multiple-messagepack-structures-on-a-single-stream) - [Low-Level API (`IMessagePackFormatter`)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#low-level-api-imessagepackformattert) - [Primitive API (`MessagePackWriter`, `MessagePackReader`)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#primitive-api-messagepackwriter-messagepackreader) - [`MessagePackReader`](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#messagepackreader) - [`MessagePackWriter`](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#messagepackwriter) - [Main Extension Point (`IFormatterResolver`)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#main-extension-point-iformatterresolver) - [MessagePackFormatterAttribute](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#messagepackformatterattribute) - [IgnoreFormatter](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#ignoreformatter) - [Reserved Extension Types](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#reserved-extension-types) - [Unity support](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#unity-support) - [AOT Code Generation (support for Unity/Xamarin)](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#aot-code-generation-support-for-unityxamarin) - [RPC](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#rpc) - [MagicOnion](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#magiconion) - [StreamJsonRpc](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#streamjsonrpc) - [How to build](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#how-to-build) - [Author Info](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#author-info) ## Installation This library is distributed via NuGet. Special [Unity support](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#unity) is available, too. We target .NET Standard 2.0 with special optimizations for .NET Core 2.1+, making it compatible with most reasonably recent .NET runtimes such as Core 2.0 and later, Framework 4.6.1 and later, Mono 5.4 and later and Unity 2018.3 and later. The library code is pure C# (with Just-In-Time IL code generation on some platforms). ### NuGet packages To install with NuGet, just install the `MessagePack` package: ```ps1 Install-Package MessagePack ``` Install the optional C# [analyzers](https://github.com/MessagePack-CSharp/MessagePack-CSharp/blob/master/doc/analyzers/index.md) package to get warnings about coding mistakes and automatic fix suggestions to save you time: ```ps1 Install-Package MessagePackAnalyzer ``` There are also a range of official and third party Extension Packages available (learn more in our [extensions section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#extensions)): ```ps1 Install-Package MessagePack.ReactiveProperty Install-Package MessagePack.UnityShims Install-Package MessagePack.AspNetCoreMvcFormatter ``` ### Unity For Unity projects, the [Releases](https://github.com/MessagePack-CSharp/MessagePack-CSharp/releases) page provides downloadable `.unitypackage` files. When using in Unity IL2CPP or Xamarin AOT environments, please carefully read the [pre-code generation section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#aot). ### Migration notes from v1.x If you were using MessagePack for C# v1.x, check out the ["How to update to our new v2.x version"](https://github.com/MessagePack-CSharp/MessagePack-CSharp/blob/master/doc/migration.md) document. ## Quick Start Define the struct or class to be serialized and annotate it with a `[MessagePackObject]` attribute. Annotate members whose values should be serialized (fields as well as properties) with `[Key]` attributes. ```csharp [MessagePackObject] public class MyClass { // Key attributes take a serialization index (or string name) // The values must be unique and versioning has to be considered as well. // Keys are described in later sections in more detail. [Key(0)] public int Age { get; set; } [Key(1)] public string FirstName { get; set; } [Key(2)] public string LastName { get; set; } // All fields or properties that should not be serialized must be annotated with [IgnoreMember]. [IgnoreMember] public string FullName { get { return FirstName + LastName; } } } ``` Call `MessagePackSerializer.Serialize/Deserialize` to serialize/deserialize your object instance. You can use the `ConvertToJson` method to get a human readable representation of any MessagePack binary blob. ```csharp class Program { static void Main(string[] args) { var mc = new MyClass { Age = 99, FirstName = "hoge", LastName = "huga", }; // Call Serialize/Deserialize, that's all. byte[] bytes = MessagePackSerializer.Serialize(mc); MyClass mc2 = MessagePackSerializer.Deserialize(bytes); // You can dump MessagePack binary blobs to human readable json. // Using indexed keys (as opposed to string keys) will serialize to MessagePack arrays, // hence property names are not available. // [99,"hoge","huga"] var json = MessagePackSerializer.ConvertToJson(bytes); Console.WriteLine(json); } } ``` By default, a `MessagePackObject` annotation is required. This can be made optional; see the [Object Serialization section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#object-serialization) and the [Formatter Resolver section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#resolvers) for details. ## Analyzer The MessagePackAnalyzer package aids with: 1. Automating definitions for your serializable objects. 1. Produces compiler warnings upon incorrect attribute use, member accessibility, and more. ![analyzergif](https://cloud.githubusercontent.com/assets/46207/23837445/ce734eae-07cb-11e7-9758-d69f0f095bc1.gif) If you want to allow a specific custom type (for example, when registering a custom type), put `MessagePackAnalyzer.json` at the project root and change the Build Action to `AdditionalFiles`. ![image](https://cloud.githubusercontent.com/assets/46207/23837427/8a8d507c-07cb-11e7-9277-5a566eb0bfde.png) An example `MessagePackAnalyzer.json`: ```json [ "MyNamespace.FooClass", "MyNameSpace.BarStruct" ] ``` ## Built-in supported types These types can serialize by default: * Primitives (`int`, `string`, etc...), `Enum`s, `Nullable<>`, `Lazy<>` * `TimeSpan`, `DateTime`, `DateTimeOffset` * `Guid`, `Uri`, `Version`, `StringBuilder` * `BigInteger`, `Complex`, `Half` * `Array[]`, `Array[,]`, `Array[,,]`, `Array[,,,]`, `ArraySegment<>`, `BitArray` * `KeyValuePair<,>`, `Tuple<,...>`, `ValueTuple<,...>` * `ArrayList`, `Hashtable` * `List<>`, `LinkedList<>`, `Queue<>`, `Stack<>`, `HashSet<>`, `ReadOnlyCollection<>`, `SortedList<,>` * `IList<>`, `ICollection<>`, `IEnumerable<>`, `IReadOnlyCollection<>`, `IReadOnlyList<>` * `Dictionary<,>`, `IDictionary<,>`, `SortedDictionary<,>`, `ILookup<,>`, `IGrouping<,>`, `ReadOnlyDictionary<,>`, `IReadOnlyDictionary<,>` * `ObservableCollection<>`, `ReadOnlyObservableCollection<>` * `ISet<>`, * `ConcurrentBag<>`, `ConcurrentQueue<>`, `ConcurrentStack<>`, `ConcurrentDictionary<,>` * Immutable collections (`ImmutableList<>`, etc) * Custom implementations of `ICollection<>` or `IDictionary<,>` with a parameterless constructor * Custom implementations of `IList` or `IDictionary` with a parameterless constructor You can add support for custom types, and there are some official/third-party extension packages for: * ReactiveProperty * for Unity (`Vector3`, `Quaternion`, etc...) * F# (Record, FsList, Discriminated Unions, etc...) Please see the [extensions section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#extensions). `MessagePack.Nil` is the built-in type representing null/void in MessagePack for C#. ## Object Serialization MessagePack for C# can serialize your own public `class` or `struct` types. By default, serializable types must be annotated with the `[MessagePackObject]` attribute and members with the `[Key]` attribute. Keys can be either indexes (`int`) or arbitrary strings. If all keys are indexes, arrays are used for serialization, which offers advantages in performance and binary size. Otherwise, MessagePack maps (dictionaries) will be used. If you use `[MessagePackObject(keyAsPropertyName: true)]`, then members do not require explicit `Key` attributes, but string keys will be used. ```csharp [MessagePackObject] public class Sample1 { [Key(0)] public int Foo { get; set; } [Key(1)] public int Bar { get; set; } } [MessagePackObject] public class Sample2 { [Key("foo")] public int Foo { get; set; } [Key("bar")] public int Bar { get; set; } } [MessagePackObject(keyAsPropertyName: true)] public class Sample3 { // No need for a Key attribute public int Foo { get; set; } // If want to ignore a public member, you can use the IgnoreMember attribute [IgnoreMember] public int Bar { get; set; } } // [10,20] Console.WriteLine(MessagePackSerializer.SerializeToJson(new Sample1 { Foo = 10, Bar = 20 })); // {"foo":10,"bar":20} Console.WriteLine(MessagePackSerializer.SerializeToJson(new Sample2 { Foo = 10, Bar = 20 })); // {"Foo":10} Console.WriteLine(MessagePackSerializer.SerializeToJson(new Sample3 { Foo = 10, Bar = 20 })); ``` All public instance members (fields as well as properties) will be serialized. If you want to ignore certain public members, annotate the member with a `[IgnoreMember]` attribute. Please note that any serializable struct or class must have public accessibility; private and internal structs and classes cannot be serialized! The default of requiring `MessagePackObject` annotations is meant to enforce explicitness and therefore may help write more robust code. Should you use an indexed (`int`) key or a string key? We recommend using indexed keys for faster serialization and a more compact binary representation than string keys. However, the additional information in the strings of string keys can be quite useful when debugging. When classes change or are extended, be careful about versioning. `MessagePackSerializer` will initialize members to their `default` value if a key does not exist in the serialized binary blob, meaning members using reference types can be initialized to `null`. If you use indexed (`int`) keys, the keys should start at 0 and should be sequential. If a later version stops using certain members, you should keep the obsolete members (C# provides an `Obsolete` attribute to annotate such members) until all other clients had a chance to update and remove their uses of these members as well. Also, when the values of indexed keys "jump" a lot, leaving gaps in the sequence, it will negatively affect the binary size, as `null` placeholders will be inserted into the resulting arrays. However, you shouldn't reuse indexes of removed members to avoid compatibility issues between clients or when trying to deserialize legacy blobs. Example of index gaps and resulting placeholders: ```csharp [MessagePackObject] public class IntKeySample { [Key(3)] public int A { get; set; } [Key(10)] public int B { get; set; } } // [null,null,null,0,null,null,null,null,null,null,0] Console.WriteLine(MessagePackSerializer.SerializeToJson(new IntKeySample())); ``` If you do not want to explicitly annotate with the `MessagePackObject`/`Key` attributes and instead want to use MessagePack for C# more like e.g. [Json.NET](https://github.com/JamesNK/Newtonsoft.Json), you can make use of the contractless resolver. ```csharp public class ContractlessSample { public int MyProperty1 { get; set; } public int MyProperty2 { get; set; } } var data = new ContractlessSample { MyProperty1 = 99, MyProperty2 = 9999 }; var bin = MessagePackSerializer.Serialize( data, MessagePack.Resolvers.ContractlessStandardResolver.Options); // {"MyProperty1":99,"MyProperty2":9999} Console.WriteLine(MessagePackSerializer.ConvertToJson(bin)); // You can also set ContractlessStandardResolver as the default. // (Global state; Not recommended when writing library code) MessagePackSerializer.DefaultOptions = MessagePack.Resolvers.ContractlessStandardResolver.Options; // Now serializable... var bin2 = MessagePackSerializer.Serialize(data); ``` If you want to serialize private members as well, you can use one of the `*AllowPrivate` resolvers. ```csharp [MessagePackObject] public class PrivateSample { [Key(0)] int x; public void SetX(int v) { x = v; } public int GetX() { return x; } } var data = new PrivateSample(); data.SetX(9999); // You can choose either StandardResolverAllowPrivate // or ContractlessStandardResolverAllowPrivate var bin = MessagePackSerializer.Serialize( data, MessagePack.Resolvers.DynamicObjectResolverAllowPrivate.Options); ``` If you want to use MessagePack for C# more like a BinaryFormatter with a typeless serialization API, use the typeless resolver and helpers. Please consult the [Typeless section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#typeless). Resolvers are the way to add specialized support for custom types to MessagePack for C#. Please refer to the [Extension point section](https://github.com/MessagePack-CSharp/MessagePack-CSharp/raw/master/#resolvers). ## DataContract compatibility You can use `[DataContract]` annotations instead of `[MessagePackObject]` ones. If type is annotated with `DataContract`, you can use `[DataMember]` annotations instead of `[Key]` ones and `[IgnoreDataMember]` instead of `[IgnoreMember]`. Then `[DataMember(Order = int)]` will behave the same as `[Key(int)]`, `[DataMember(Name = string)]` the same as `[Key(string)]`, and `[DataMember]` the same as `[Key(nameof(member name)]`. Using `DataContract`, e.g. in shared libraries, makes your classes/structs independent from MessagePack for C# serialization. However, it is not supported by the analyzers nor in code generation by the `mpc` tool. Also, features like `UnionAttribute`, `MessagePackFormatter`, `SerializationConstructor`, etc can not be used. Due to this, we recommend that you use the specific MessagePack for C# annotations when possible. ## Serializing readonly/immutable object members (SerializationConstructor) MessagePack for C# supports serialization of readonly/immutable objects/members. For example, this struct can be serialized and deserialized. ```csharp [MessagePackObject] public struct Point { [Key(0)] public readonly int X; [Key(1)] public readonly int Y; public Point(int x, int y) { this.X = x; this.Y = y; } } var data = new Point(99, 9999); var bin = MessagePackSerializer.Serialize(data); // Okay to deserialize immutable object var point = MessagePackSerializer.Deserialize(bin); ``` `MessagePackSerializer` will choose the constructor with the best matched argument list, using argument indexes index for index keys, or parameter names for string keys. If it cannot determine an appropriate constructor, a `MessagePackDynamicObjectResolverException: can't find matched constructor parameter` exception will be thrown. You can specify which constructor to use manually with a `[SerializationConstructor]` annotation. ```csharp [MessagePackObject] public struct Point { [Key(0)] public readonly int X; [Key(1)] public readonly int Y; [SerializationConstructor] public Point(int x) { this.X = x; this.Y = -1; } // If not marked attribute, used this(most matched argument) public Point(int x, int y) { this.X = x; this.Y = y; } } ``` ### C# 9 `record` types C# 9.0 record with primary constructor is similar immutable object, also supports serialize/deserialize. ```csharp // use key as property name [MessagePackObject(true)]public record Point(int X, int Y); // use property: to set KeyAttribute [MessagePackObject] public record Point([property:Key(0)] int X, [property: Key(1)] int Y); // Or use explicit properties [MessagePackObject] public record Person { [Key(0)] public string FirstName { get; init; } [Key(1)] public string LastName { get; init; } } ``` ### C# 9 `init` property setter limitations When using `init` property setters in _generic_ classes, [a CLR bug](https://github.com/neuecc/MessagePack-CSharp/issues/1134) prevents our most efficient code generation from invoking the property setter. As a result, you should avoid using `init` on property setters in generic classes when using the public-only `DynamicObjectResolver`/`StandardResolver`. When using the `DynamicObjectResolverAllowPrivate`/`StandardResolverAllowPrivate` resolver the bug does not apply and you may use `init` without restriction. ## Serialization Callback Objects implementing the `IMessagePackSerializationCallbackReceiver` interface will received `OnBeforeSerialize` and `OnAfterDeserialize` calls during serialization/deserialization. ```csharp [MessagePackObject] public class SampleCallback : IMessagePackSerializationCallbackReceiver { [Key(0)] public int Key { get; set; } public void OnBeforeSerialize() { Console.WriteLine("OnBefore"); } public void OnAfterDeserialize() { Console.WriteLine("OnAfter"); } } ``` ## Union MessagePack for C# supports serializing interface-typed and abstract class-typed objects. It behaves like `XmlInclude` or `ProtoInclude`. In MessagePack for C# these are called `Union`. Only interfaces and abstracts classes are allowed to be annotated with `Union` attributes. Unique union keys are required. ```csharp // Annotate inheritance types [MessagePack.Union(0, typeof(FooClass))] [MessagePack.Union(1, typeof(BarClass))] public interface IUnionSample { } [MessagePackObject] public class FooClass : IUnionSample { [Key(0)] public int XYZ { get; set; } } [MessagePackObject] public class BarClass : IUnionSample { [Key(0)] public string OPQ { get; set; } } // --- IUnionSample data = new FooClass() { XYZ = 999 }; // Serialize interface-typed object. var bin = MessagePackSerializer.Serialize(data); // Deserialize again. var reData = MessagePackSerializer.Deserialize(bin); // Use with e.g. type-switching in C# 7.0 switch (reData) { case FooClass x: Console.WriteLine(x.XYZ); break; case BarClass x: Console.WriteLine(x.OPQ); break; default: break; } ``` Unions are internally serialized to two-element arrays. ```csharp IUnionSample data = new BarClass { OPQ = "FooBar" }; var bin = MessagePackSerializer.Serialize(data); // Union is serialized to two-length array, [key, object] // [1,["FooBar"]] Console.WriteLine(MessagePackSerializer.ConvertToJson(bin)); ``` Using `Union` with abstract classes works the same way. ```csharp [Union(0, typeof(SubUnionType1))] [Union(1, typeof(SubUnionType2))] [MessagePackObject] public abstract class ParentUnionType { [Key(0)] public int MyProperty { get; set; } } [MessagePackObject] public class SubUnionType1 : ParentUnionType { [Key(1)] public int MyProperty1 { get; set; } } [MessagePackObject] public class SubUnionType2 : ParentUnionType { [Key(1)] public int MyProperty2 { get; set; } } ``` Please be mindful that you cannot reuse the same keys in derived types that are already present in the parent type, as internally a single flat array or map will be used and thus cannot have duplicate indexes/keys. ## Dynamic (Untyped) Deserialization When calling `MessagePackSerializer.Deserialize