Dequeueable.AmazonSQS 1.0.0

Dequeueable.AmazonSQS

This project is an opinionated framework build for the Amazon (AWS) Simple Queue Service (SQS):

• Build as a Console App
• Being able to use optimized alpine/dotnet images
• Have the freedom to use Keda or any other scalers to retrieve queue messages

This framework can run as a listener or job:

• Listener: Highly scalable queue listener that will be invoked automatically when new messages are detected on the SQS.
• Job: Framework that depends on external queue triggers, eg; KEDA. When the host is started, new messages on the SQS are being retrieved and executed. After execution the host will shutdown automatically.

Getting started

Scaffold a new project, you can either use a console or web app.

1. Add a class that implements the IAmazonSQSFunction.
2. Add .AddAmazonSQSServices<TFunction>() in the DI container.
3. Specify how you want to run your service:
   • Add .RunAsJob() in the DI container of your app to run the host as a job.
   • Add .RunAsListener() in the DI container of your app to run the app as a back ground listener.

function.cs:

internal class TestFunction : IAmazonSQSFunction
    {
        public Task ExecuteAsync(Message message, CancellationToken cancellationToken)
        {
            // Put your magic here!
        }
    }

program.cs:

await Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
    services
    .AddAmazonSQSServices<TestFunction>()
    .RunAsJob(options =>
    {
        options.VisibilityTimeoutInSeconds = 300;
        options.BatchSize = 4;
    });
})
.RunConsoleAsync();

Configurations

You can configure the host via the appsettings.json or the IOptions pattern during registration.

Appsettings

Use the Dequeueable section to configure the settings:

"Dequeueable": {
     "QueueUrl": "https://sqs.<region>.amazonaws.com/<id>/<queuename>"
  }

Options

await Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
    services
    .AddAmazonSQSServices<TestFunction>()
    // .RunAsListener(options =>
    .RunAsJob(options =>
    {
        options.VisibilityTimeoutInSeconds = 300;
        options.BatchSize = 4;
    });
})
.RunConsoleAsync();

Settings

The library uses the IOptions pattern to inject the configured app settings.

Job options

Listener options

Authentication

The queue client is constructed with the credentials loaded from the application's default configuration, using the FallbackCredentialsFactory.GetCredentials().

Custom AmazonSQSClientFactory

There are plenty ways to construct the AmazonSQSClient, and not all are by default supported. You can override the default implementations to retrieve the queue client by implementing the IAmazonSQSClientFactory. You still should register your custom factory in your DI container, specific registration order is not needed:

internal class MyCustomQueueFactory : IAmazonSQSClientFactory
    {
        private AmazonSQSClient? _client;
        public AmazonSQSClient Create() => _client ??= new AmazonSQSClient(Amazon.RegionEndpoint.CNNorth1);
    }

Singleton

The application can run as distributed singleton. The Amazon SQS message group ID is used to processed the messages one by one, in a strict order relative to the message group. Therefore a fifo queue is required. Both the Job as the Listener services can run as singleton by defining this during registration:

await Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
    services
    .AddAmazonSQSServices<TestFunction>()
    .RunAsJob(options =>
    {
        // some options
    })
    .AsSingleton();
})
.RunConsoleAsync();

Timeouts

Visibility Timeout Queue Message

The visibility timeout of the queue messages is automatically updated. It will be updated when the half VisibilityTimeoutInSeconds option is reached. Choose this setting wisely to prevent talkative hosts. When renewing the timeout fails, the host cannot guarantee if the message is executed only once. Therefore the CancelationToken is set to Cancelled. It is up to you how to handle this scenario!

Sample

• Job Console app
• Listener Console app