CsvImporterToolkit 1.0.30

CsvImporterToolkit

CsvImporterToolkit helps you import CSV files safely and quickly.

It validates rows, compares against existing data, and returns clear results such as New, Changed, Unchanged, and Invalid rows.

If you are new to the package, start with README.EASY.md first.

Highlights

• Fluent field configuration with Set(...).Required().Type(...).OnChange(...)
• File validation: extension, empty file, and required headers
• Row validation: required fields, datatype checks, custom validators
• Structured row output: RowError, RowChange, RowState
• Change tracking against existing data by unique key
• Import summary counts: total, valid, invalid, new, changed, unchanged, duplicate conflicts
• Optional skip unchanged rows
• Pluggable comparison strategies (DefaultComparer, CaseInsensitiveStringComparer, TrimmedStringComparer)

Start Here (2 Minutes)

1. Create a model that inherits ImportableBase.
2. Configure columns using Importer<T>.Set(...).
3. Call ImportAsync(...) with your CSV stream or file.
4. Use result.Summary and grouped result helpers like result.InvalidRows.

If you want the easiest possible walkthrough, use README.EASY.md.

Install

<PackageReference Include="CsvImporterToolkit" Version="1.0.30" />

For local testing, add a local source in NuGet.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="local-packages" value="C:\angular\CSV nuget\nupkgs" />
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </packageSources>
</configuration>

Minimal Usage (No ClassMap Required)

Use this when CSV headers match model property names.

using CsvImporterToolkit.Configuration;
using CsvImporterToolkit.Models;
using CsvImporterToolkit.Services;

var importer = new Importer<PersonImportRow>();
importer.Set("Id", x => x.Id).Required();
importer.Set("Name", x => x.Name).Required();
importer.Set("Age", x => x.Age)
    .Required()
    .Type(typeof(int))
    .OnChange((oldValue, newValue) => $"Age changed from {oldValue} to {newValue}");

var existing = new List<PersonImportRow>
{
    new() { Id = "1", Name = "Alice", Age = "30" }
};

var service = new CsvImportService();
await using var fileStream = File.OpenRead("people.csv");

var result = await service.ImportAsync<PersonImportRow>(
    fileStream,
    "people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: existing,
    options: new ImportOptions { SkipUnchangedRows = true });

foreach (var row in result.FileData)
{
    foreach (var error in row.Errors)
    {
        Console.WriteLine($"Row {error.RowNumber}, Column {error.ColumnName}: {error.Message}");
    }

    foreach (var change in row.Changes)
    {
        Console.WriteLine($"Row {change.RowNumber}, Column {change.ColumnName}: {change.Message}");
    }
}

Console.WriteLine($"Changed: {result.Summary.ChangedRows}, New: {result.Summary.NewRows}, Invalid: {result.Summary.InvalidRows}");

public sealed class PersonImportRow : ImportableBase
{
    public string Id { get; set; } = string.Empty;
    public string Name { get; set; } = string.Empty;
    public string Age { get; set; } = string.Empty;
}

ASP.NET Usage (IFormFile)

The service supports direct import from an uploaded file with optional comparison against existing data.

public sealed class UploadRequest
{
    [Required]
    public IFormFile File { get; set; } = default!;
}

public async Task<IResult> ImportPeople([FromForm] UploadRequest request, CsvImportService service)
{
    var importer = new Importer<PersonImportRow>();
    importer.Set("Id", x => x.Id).Required();
    importer.Set("Name", x => x.Name).Required();

    // --- Option 1: no existing data, all rows come back as New ---
    var result = await service.ImportAsync<PersonImportRow>(
        request.File,
        importer,
        uniqueColumnName: "Id",
        existingData: null);

    // --- Option 2: existing data as IEnumerable<T> ---
    var existingRows = new List<PersonImportRow>
    {
        new() { Id = "1", Name = "Alice", Age = "30" }
    };
    var result2 = await service.ImportAsync<PersonImportRow>(
        request.File,
        importer,
        uniqueColumnName: "Id",
        existingData: existingRows);

    // --- Option 3: existing data as paginated items ---
    var existingList = await dbContext.People.ToListAsync();
    var result3 = await service.ImportAsync<PersonImportRow>(
        request.File,
        importer,
        uniqueColumnName: "Id",
        existingData: existingList);

    // Paginated response example:
    // PaginatedResponse<PersonImportRow> paged = ...;
    // var result4 = await service.ImportAsync<PersonImportRow>(
    //     request.File, importer, "Id", existingData: paged.Items);

    return Results.Ok(result3);
}

Path-based Usage

If your backend already stores files on disk, call path-based overloads directly.

var result = await service.ImportAsync<PersonImportRow>(
    @"C:\\imports\\people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: existing,
    options: new ImportOptions { SkipUnchangedRows = true });

Standalone Row Comparison (Version 1.0.18+)

If you have already parsed CSV rows and want to compare them with existing data separately:

// Parse CSV independently or get from another source
var service = new CsvImportService();
var parsedRows = new List<PersonImportRow> { /* your parsed rows */ };

var existing = new List<PersonImportRow>
{
    new() { Id = "1", Name = "Alice", Age = "30" }
};

var comparedRows = await service.CompareAsync<PersonImportRow>(
    parsedRows,
    existing,
    keySelector: x => x.Id);

// Compare against a different existing model type
var existingEntities = new List<PersonEntity>
{
    new() { PersonCode = "1", FullName = "Alice", AgeText = "30" }
};

var comparedCrossTypeRows = await service.CompareAsync<PersonImportRow, PersonEntity>(
    parsedRows,
    existingEntities,
    fileKeySelector: x => x.Id,
    existingKeySelector: x => x.PersonCode,
    configure: map =>
    {
        map.Map(x => x.Name, x => x.FullName).Named("Name");
        map.Map(x => x.Age, x => x.AgeText).Named("Age");
    });

// Each row now has State, Errors, and Changes populated
foreach (var row in comparedRows)
{
    Console.WriteLine($"State: {row.State}");
    foreach (var change in row.Changes)
    {
        Console.WriteLine($"  {change.ColumnName}: {change.OldValue} -> {change.NewValue}");
    }
}

Advanced Usage (With ClassMap)

Use ImportAsync<T, TMap> only when you need custom CsvHelper mapping, renamed headers, or advanced conversion logic.

Notes

• .Type(typeof(int)) remains unchanged and supported.
• Version 1.0.9 adds direct IFormFile and file path overloads for ImportAsync.
• Version 1.0.12: Automatic header mapping from Importer rules. CSV headers with spaces (e.g., "Vehicle no") automatically map to PascalCase properties (e.g., "VehicleNo"). No ClassMap needed.
• Version 1.0.19: Added duplicate key detection for imported rows and existing data.
• Version 1.0.25: Added lifecycle hooks for parsing, validation, and comparison checkpoints.
• Version 1.0.26: Added grouped result helpers on FileResult for easier row filtering.
• Version 1.0.28: Added cross-type ImportAsync overloads with key selectors and comparison mapping.
• The library does not write UI output. It returns structured data and messages.
• If you update README or metadata for NuGet, publish a new package version.

Lifecycle Hooks (Version 1.0.25+)

Use lifecycle hooks to inject business logic during row processing.

var importer = new Importer<PersonImportRow>();
importer.Set("Id", x => x.Id).Required();
importer.Set("Name", x => x.Name).Required();

importer.OnRowParsed((row, rowNumber) =>
{
    // Called before validation for each parsed row.
});

importer.BeforeValidate((row, rowNumber) =>
{
    // Called right before ValidateRow.
});

importer.AfterValidate((row, rowNumber) =>
{
    // Called after validation and can inspect row.Errors.
});

importer.BeforeCompare((row, rowNumber) =>
{
    // Called only for rows that passed validation.
});

importer.AfterCompare((row, rowNumber) =>
{
    // Called after state assignment: New / Changed / Unchanged.
});

Better Result Helpers (Version 1.0.26+)

Use grouped helper properties on FileResult to avoid manual Where filters.

var result = await service.ImportAsync<PersonImportRow>(
    fileStream,
    "people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: existingRows);

var newRows = result.NewRows;
var changedRows = result.ChangedRows;
var invalidRows = result.InvalidRows;
var notFoundRows = result.NotFoundRows;
var warningRows = result.WarningRows;

var unchangedRows = result.GetRowsByState(RowState.Unchanged);

Scenario Cookbook (Phase 9)

These examples cover the most common real-world adoption scenarios.

1) Multiple header aliases

var importer = new Importer<PersonImportRow>();
importer.Set("Customer Id", new[] { "CustomerID", "cust_id", "customer-id" }, x => x.Id).Required();
importer.Set("Full Name", "Name", x => x.Name).Required();
importer.Set("Age", x => x.Age).Type(typeof(int));

2) IFormFile import in ASP.NET

public async Task<IResult> ImportPeople([FromForm] IFormFile file, CsvImportService service)
{
    var importer = new Importer<PersonImportRow>();
    importer.Set("Id", x => x.Id).Required();
    importer.Set("Name", x => x.Name).Required();

    var existing = await dbContext.People
        .Select(x => new PersonImportRow { Id = x.Code, Name = x.Name, Age = x.Age.ToString() })
        .ToListAsync();

    var result = await service.ImportAsync<PersonImportRow>(
        file,
        importer,
        uniqueColumnName: "Id",
        existingData: existing,
        options: new ImportOptions { SkipUnchangedRows = true });

    return Results.Ok(result);
}

3) File-path import

var result = await service.ImportAsync<PersonImportRow>(
    @"C:\\imports\\people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: existingRows);

4) Paginated existing data

var page = await repository.GetPeoplePageAsync(pageNumber: 1, pageSize: 1000);

var result = await service.ImportAsync<PersonImportRow>(
    fileStream,
    "people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: page.Items);

5) Cross-type comparison (DTO vs entity)

var comparedRows = await service.CompareAsync<PersonImportRow, PersonEntity>(
    fileRows,
    existingEntities,
    fileKeySelector: x => x.Id,
    existingKeySelector: x => x.PersonCode,
    configure: map =>
    {
        map.Map(x => x.Name, x => x.FullName).Named("Name");
        map.Map(x => x.Age, x => x.AgeText).Named("Age");
    });

// Full import + compare in one step (cross-type)
var importResult = await service.ImportAsync<PersonImportRow, PersonEntity>(
    fileStream,
    "people.csv",
    importer,
    existingEntities,
    fileKeySelector: x => x.Id,
    existingKeySelector: x => x.PersonCode,
    keyColumnName: "Id",
    configure: map =>
    {
        map.Map(x => x.Name, x => x.FullName).Named("Name");
        map.Map(x => x.Age, x => x.AgeText).Named("Age");
    });

6) Duplicate handling

foreach (var row in result.FileData)
{
    if (row.DuplicateInFile || row.DuplicateInExistingData)
    {
        Console.WriteLine($"Duplicate conflict on Id {row.Id}");
    }
}

Console.WriteLine($"Duplicate rows: {result.Summary.DuplicateRows}");

7) Not-found rows

foreach (var row in result.NotFoundRows)
{
    Console.WriteLine($"Missing in DB: {row.Id}");
}

8) Custom transformations and parsing

var importer = new Importer<PersonTypedImportRow>();
importer.Set("Id", x => x.Id).Required().Normalize();
importer.Set("Name", x => x.Name).Normalize().DefaultValue("Unknown");
importer.Set("Age", x => x.Age)
    .DefaultValue("0")
    .ParseWith(value => int.TryParse(value, out var parsed) ? parsed : 0);

9) Lifecycle hooks + result helpers together

importer.BeforeValidate((row, rowNumber) =>
{
    if (string.IsNullOrWhiteSpace(row.Id))
    {
        row.AddError(rowNumber, "Id", "Id is required before validation.");
    }
});

var result = await service.ImportAsync<PersonImportRow>(
    fileStream,
    "people.csv",
    importer,
    uniqueColumnName: "Id",
    existingData: existingRows);

Console.WriteLine($"Changed rows: {result.ChangedRows.Count}");
Console.WriteLine($"Invalid rows: {result.InvalidRows.Count}");

10) Recommended API response shape

return Results.Ok(new
{
    result.Summary,
    New = result.NewRows,
    Changed = result.ChangedRows,
    Invalid = result.InvalidRows,
    NotFound = result.NotFoundRows,
    Warnings = result.WarningRows
});

This helps UI clients render grouped tabs without additional server-side filtering.