Gord0.ChunkyMonkey.CodeGenerator 2.0.28

ChunkyMonkey - C# Object Chunking Source Generator

Introduction

ChunkyMonkey is a C# code generator that generates code, at build time, to split an object containing collection properties into chunks. It also provides the ability to merge the chunks back into a single object instance.

Supported collection types

• System.Array
• System.Collections.Generic.Dictionary<K,V>
• System.Collections.Generic.List<T>
• System.Collections.Generic.HashSet<T>
• System.Collections.Generic.SortedDictionary<TKey, TValue>
• System.Collections.Generic.SortedList<TKey, TValue>
• System.Collections.Generic.SortedSet<T>
• System.Collections.Immutable.ImmutableArray<T>
• System.Collections.Immutable.ImmutableHashSet
• System.Collections.Immutable.ImmutableList<T>
• System.Collections.ObjectModel.Collection<T>
• System.Collections.ObjectModel.ObservableCollection<T>
• System.Collections.ObjectModel.ReadOnlyCollection<T>
• System.Collections.ObjectModel.ReadOnlyObservableCollection<TKey>
• System.Collections.Specialized.NameValueCollection
• System.Collections.Specialized.StringCollection

Use Case

• You need to break down a large API request into smaller pieces (chunks) - to avoid breaking the maximum request body size limit of your web server.

<br>

Walkthrough

Imagine you have a class like this:

public class CreateUserRequest
{
    public string Username { get; set; }
    public DateTime DateOfBirth { get; set; }	
    public List<int> FavouriteNumbers { get; set; }
    public string[] FavouriteFilms { get; set; }
    public Dictionary<string, string> Attributes { get; set; }
}

If a CreateUserRequest object is for a user who has lot of favourite numbers, films and attributes, you may want to split the object into chunks to reduce the size of the request object. [Yes, this is a very contrived example!]

ChunkyMonkey generates two methods within partial classes, where TInstance is the name of the class.

📝 The method isn't generic; the generated code contains the actual class type rather than TInstance.

public IEnumerable<TInstance> Chunk(int chunkSize)

public TInstance MergeChunks(IEnumerable<TInstance> chunks)

IEnumerable<TInstance> Chunk(int chunkSize)

• The chunk size is the maximum number of values held in chunked collection properties.
• Only partial and non-static classes can be chunked. This is because the code generates generates a partial class containing the methods above for each chunked class.
• Non-chunkable properties are repeated for each chunked instance.
• If a collection (list, collection, array or dictionary) property's values are exhausted, subsequent chunks will contain an empty collection for the property value.
• Only top-level collection properties are chunked. Nested properties are not chunked.

Example of usage

• Let's examine the use of the Chunk method on the CreateUserRequest class with a chunk size of 3.

Input to Chunks(3)

Output for Chunks(3)

Chunk #1

Chunk #2

Chunk #3

<br>

TInstance MergeChunks(IEnumerable<TInstance> chunks)

This generated method merges a set of chunks back into a single instance.

<br>

Using within a .NET project

1. Add the ChunkyMonkey NuGet package to your C# project containing classes for which you'd like to generate the Chunk and MergeChunks methods.

   Environment	Command
   .NET CLI	dotnet add package Gord0.ChunkyMonkey.CodeGenerator
   VS Package Manager Console	NuGet\Install-Package Gord0.ChunkyMonkey.CodeGenerator
   VS Package Manager Console	<PackageReference Include="Gord0.ChunkyMonkey.CodeGenerator" Version="x.y.z" />

2. There are two ways of indicating to ChunkyMonkey that you would like the Chunk and MergeChunks to be generated for a class:

   Method	Attribute	Description
   Class-level	<ul><li>[Chunk] defaults to Accessibility.Public</li><li>[Chunk(Accessibility.Public)]</li><li>[Chunk(Accessibility.All)]	Add the [Chunk] attribute to the class for which you'd like ChunkyMonkey to generate the Chunk and MergeChunks methods. Code will be generated to chunk all public properties for supported collection types. The optional Accessibility argument determines whether chunking methods will be written for all properties, or just those with public getters and setters (default)</li></ul>
   Property-level	<ul><li>[ChunkMember]</li></ul>	Add the [ChunkMember] attribute to one or more properties of a class for which you'd like ChunkyMonkey to generate the Chunk and MergeChunks methods. Code will be generated to chunk the properties for supported collection types if they have been decorated with ChunkMember, regardless of getter and setter visibility.

   💡 Tip: Be sure to define the class as a partial class - otherwise, you will receive a compiler error when you build your project.

   using Gord0.ChunkMonkey.Attributes;
   
   namespace TestProject
   {
       [Chunk]
       public partial class Person
       {
           public string? Name { get; set; }
   
           public DateTime? DateOfBirth { get; set; }
   
           public string[]? PhoneNumbers { get; set; }
   
           public List<string>? RecentAddresses { get; set; }
       }
   }
   

   OR

   using Gord0.ChunkMonkey.Attributes;
   
   namespace TestProject
   {
       public partial class Person
       {
           public string? Name { get; set; }
   
           public DateTime? DateOfBirth { get; set; }
   
           [ChunkMember]
           public string[]? PhoneNumbers { get; set; }
   
           [ChunkMember]
           public List<string>? RecentAddresses { get; set; }
       }
   }  
   

   📝 Note: If your classes/DTOs live in a separate project, you should add the Gord0.ChunkyMonkey.Attributes package to that project. This package provides the ChunkAttribute and ChunkMemberAttribute types.

3. Build your project.

4. To view the generated partial classes:

   • Expand the Dependencies nodes under your project in the Solution Explorer.
   • Expand the Analyzers node.
   • Expand the Gord0.ChunkyMonkey.CodeGenerator node.
   • Expand the Gord0.ChunkyMonkey.CodeGenerator.ChunkyMonkeyGenerator node.
   • Now you will see the generated partial classes, each containing the Chunk and MergeChunks methods. The generated classes are called <ClassName>_ChunkeyMonkey.g.cs

5. The output for the above Person class would be a file called Person_ChunkeyMonkey.g.cs:

   using System;
   using System.Collections.Generic;
   using System.Collections.ObjectModel;
   using System.Collections.Immutable;
   using System.Linq;
   
   namespace TestProject
   {
       public partial class Person
       {
           /// <summary>
           /// Chunks the instance into multiple instances based on the specified chunk size.
           /// </summary>
           /// <param name="chunkSize">The size of each chunk.</param>
           /// <returns>An enumerable of chunked instances.</returns>
           public IEnumerable<Person> Chunk(int chunkSize)
           {
               // Find the length of the biggest collection.
               long biggestCollectionLength = new long[] {
                   (this.PhoneNumbers is not null) ? this.PhoneNumbers.Length : 0, 
                   (this.RecentAddresses is not null) ? this.RecentAddresses.Count : 0
               }.Max();
   
               for (int i = 0; i < biggestCollectionLength; i += chunkSize)
               {
                   var instance = new Person();
                   instance.Name = this.Name;
                   instance.DateOfBirth = this.DateOfBirth;
                   {
                       if (this.PhoneNumbers is not null)
                       {
                           instance.PhoneNumbers = this.PhoneNumbers.Skip(i).Take(chunkSize).ToArray();
                       }
                   }
   
                   if (this.RecentAddresses is not null)
                   {
                       instance.RecentAddresses = this.RecentAddresses.Skip(i).Take(chunkSize).ToList();
                   }
   
                   yield return instance;
               }
           }
   
           /// <summary>
           /// Merges the specified chunks into a single instance.
           /// </summary>
           /// <param name="chunks">The chunks to merge.</param>
           /// <returns>The merged instance.</returns>
           public static Person MergeChunks(IEnumerable<Person> chunks)
           {
               var instance = new Person();
   
               long chunkNumber = 0;
               string? lastValue_Name = default;
               global::System.DateTime? lastValue_DateOfBirth = default;
               List<string>? tempArrayList_PhoneNumbers = null;
   
               foreach(var chunk in chunks)
               {
                   if (chunkNumber > 0)
                   {
                       if (lastValue_Name != chunk.Name)
                       {
                           throw new InvalidDataException("Chunks contain different values for non-chunked property 'Name'");
                       }
                       if (lastValue_DateOfBirth != chunk.DateOfBirth)
                       {
                           throw new InvalidDataException("Chunks contain different values for non-chunked property 'DateOfBirth'");
                       }
                   }
   
                   instance.Name = chunk.Name;
                   lastValue_Name = chunk.Name;
   
                   instance.DateOfBirth = chunk.DateOfBirth;
                   lastValue_DateOfBirth = chunk.DateOfBirth;
   
                   if (chunk.PhoneNumbers is not null)
                   {
                       if (tempArrayList_PhoneNumbers is null)
                       {
                           tempArrayList_PhoneNumbers = [];
                       }
   
                       tempArrayList_PhoneNumbers.AddRange(chunk.PhoneNumbers);
                   }
   
                   if (chunk.RecentAddresses is not null)
                   {
                       if (instance.RecentAddresses is null)
                       {
                           instance.RecentAddresses = new List<string>();
                       }
   
                       if (chunk.RecentAddresses is not null)
                       {
                           foreach(var value in chunk.RecentAddresses)
                           {
                               instance.RecentAddresses.Add(value);
                           }
                       }
                   }
   
                   chunkNumber++;
               }
   
               if (tempArrayList_PhoneNumbers is not null)
               {
                   instance.PhoneNumbers = tempArrayList_PhoneNumbers.ToArray();
               }
               else
               {
                   instance.PhoneNumbers = null;
               }
   
               return instance;
           }
      }
   }
   

<br>

Future Enhancements

• Support System.Collections.Generic.Stack<T>
• Support System.Collections.Generic.Queue<T>
• Support System.Collections.Generic.PriorityQueue<T>
• Support System.Collections.Specialized.BitArray
• Support Memory<T>
• Support ReadOnlyMemory<T>

Versions