TheDataCity.Utilities 1.0.31

TheDataCity.Utilities

A .NET utilities library providing structured logging, timing helpers, string utilities, company web-text cleaning, and DuckDB table definition generation.

Installation

dotnet add package TheDataCity.Utilities

Or via Package Manager Console:

Install-Package TheDataCity.Utilities

Quick Start

Logging

using TheDataCity.Utilities;

// Initialize logger once at startup
Log.InitialiseWithConsole("MyApp");

// Use Log
Log.Info("Processing started");
Log.Info("Processed {Count} items", itemCount);
Log.Error("Something went wrong");
Log.Warning("This is a warning");
Log.Debug("Debug information");

Timing Operations

using TheDataCity.Utilities;

// Automatic timing with using statement
using (new TimingHelper("Processing data"))
{
    // Your code here
    // Automatically logs completion time when disposed
}

// Or use the static Timing class
var (result, elapsed) = Timing.TimeOperation("Calculate result", () => {
    return PerformCalculation();
});

String Utilities

using TheDataCity.Utilities;

// Get first word from a string
string firstWord = StringUtilities.GetFirstWord("Hello World"); // "Hello"

// Normalise website URLs
string normalised = StringUtilities.NormaliseWebsite("https://www.example.com/"); // "example.com"

// Normalise company names (period / ampersand spacing)
string normalised = StringUtilities.NormaliseCompanyName("A.B.C. & Co."); // "ABC & Co"

// SearchName / name-filter needle (drop punctuation except &, collapse spaces)
string searchName = StringUtilities.NormaliseSearchName("Reno's of Wise, LLC").ToUpperInvariant(); // "RENOS OF WISE LLC"

Company web-text cleaning

using TheDataCity.Utilities;

(string? cleaned, CleaningStats stats) = CompanyTextCleaner.CleanText(rawWebText);
string status = CompanyTextCleaner.QualityStatusName(stats.QualityStatus);

CompanyTextCleaner is the canonical C# cleaner for company website text (boilerplate strip, foreign-char strip, paragraph/sentence dedup, n-gram phrase dedup, quality classification). Prefer this package over local copies.

DuckDB Table Generation

using TheDataCity.Utilities.Attributes.DuckDB;
using TheDataCity.Utilities.DuckDB;

[TableName("custom_table_name")]
public class MyModel
{
    [ColumnName("id")]
    [NotNull]
    public int Id { get; set; }

    [DuckDBType("VARCHAR")]
    public string Name { get; set; }

    [ExcludeFromTable]
    public string IgnoredProperty { get; set; }
}

// Generate CREATE TABLE statement
string createTableSql = TableDefinitionGenerator.GenerateCreateTable<MyModel>();

Features

• Structured Logging - Source-generated logging using .NET ILogger with automatic timestamp formatting
• Timing Helpers - Automatic timing and logging of code blocks with intelligent operation name transformation
• String Utilities - String manipulation and normalisation methods for common use cases
• Company Web-Text Cleaning - Shared CompanyTextCleaner for pipeline web-text cleaning
• DuckDB Integration - Generate DuckDB table definitions from C# models with customisable attributes

Requirements

• .NET 10.0 or later

License

MIT License