High-performance and reliable .NET library for background tasks, designed for distributed applications.
- Scheduled tasks with cron, intervals and any custom schedulers
- Queue-based task execution
- Transactional creation of multiple tasks
- Configurable execution order for multiple tasks
- Groups of tasks with sequential execution
- Retry policies for failed tasks
- Multi-Queues
- Configurable middlewares pipeline for executing background tasks code
- OpenTelemetry-compatible metrics and tracing
- Proper operation in distributed applications
- Fault tolerance and component failure resilience
- High performance
- Low resource consumption on both .NET application and database sides
To use Jobby, install the Jobby.Core package and a storage package (currently only PostgreSQL is supported):
dotnet package add Jobby.Core
dotnet package add Jobby.Postgres
For ASP.NET Core integration, also install the Jobby.AspNetCore package:
dotnet package add Jobby.AspNetCore
To define a background task, implement the IJobCommand interface for the task parameters and IJobCommandHandler for the task logic:
public class SendEmailCommand : IJobCommand
{
// Properties can contain any parameters to be passed to the task
public string Email { get; init; }
// Return a unique name identifying the task
public static string GetJobName() => "SendEmail";
}
public class SendEmailCommandHandler : IJobCommandHandler<SendEmailCommand>
{
// Dependency injection is supported when using Jobby.AspNetCore
private readonly IEmailService _emailService;
public SendEmailCommandHandler(IEmailService logger)
{
_logger = logger;
}
public async Task ExecuteAsync(SendEmailCommand command, JobExecutionContext ctx)
{
// Implement your task logic here
// command - task parameters
// ctx - contains cancellationToken and additional task execution info
}
} To add Jobby to an ASP.NET Core application, use the AddJobbyServerAndClient extension method:
builder.Services.AddSingleton<NpgsqlDataSource>(NpgsqlDataSource.Create(databaseConnectionString));
builder.Services.AddJobbyServerAndClient(jobbyBuilder =>
{
// Specify assemblies containing your IJobCommand and IJobCommandHandler implementations
jobbyBuilder.AddJobsFromAssemblies(typeof(SendEmailCommand).Assembly);
// Configure jobby
jobbyBuilder.ConfigureJobby((serviceProvider, jobby) => {
jobby.UsePostgresql(serviceProvider.GetRequiredService<NpgsqlDataSource>());
jobby.UseServerSettings(new JobbyServerSettings
{
// Maximum number of concurrently executing tasks
MaxDegreeOfParallelism = 10,
// Maximum number of tasks fetched from queue per query
TakeToProcessingBatchSize = 10,
});
jobby.UseDefaultRetryPolicy(new RetryPolicy
{
// Maximum number of task execution attempts
MaxCount = 3,
// Delays between retry attempts (in seconds)
IntervalsSeconds = [1, 2]
});
});
}); Full ASP.NET Core example: Jobby.Samples.AspNet.
For non-ASP.NET Core usage, create a JobbyBuilder instance:
var jobbyBuilder = new JobbyBuilder();
jobbyBuilder
.UsePostgresql(dataSource)
// scopeFactory - your custom scope factory implementation
.UseExecutionScopeFactory(scopeFactory)
.AddJobsFromAssemblies(typeof(SendEmailCommand).Assembly);
// Service for creating tasks
var jobbyClient = builder.CreateJobbyClient();
// Background task execution service
var jobbyServer = builder.CreateJobbyServer();
jobbyServer.StartBackgroundService(); // Start background service
//...
jobbyServer.SendStopSignal(); // Stop service Full console application example: Jobby.Samples.CliJobsSample.
The library provides the IJobbyStorageMigrator service to create the required tables in the database and automatically update their structure when transitioning to a new version.
When using Jobby.AspNetCore, the service is available through the DI container and can be called at service startup:
//...
app.MapControllers();
// Create or update jobby storage schema
var jobbyStorageMigrator = app.Services.GetRequiredService<IJobbyStorageMigrator>();
jobbyStorageMigrator.Migrate();Without using Jobby.AspNetCore, the IJobbyStorageMigrator service can be obtained from the JobbyBuilder.GetStorageMigrator method.
For full examples, refer to the links: Jobby.Samples.AspNet and Jobby.Samples.CliJobsSample.
Use the IJobbyClient service to enqueue tasks (available via DI in ASP.NET Core or from JobbyBuilder otherwise).
var command = new SendEmailCommand { Email = "some@email.com" };
// Enqueue task for execution as soon as possible
await jobbyClient.EnqueueCommandAsync(command);
// Enqueue task for execution no earlier than specified time
await jobbyClient.EnqueueCommandAsync(command, DateTime.UtcNow.AddHours(1)); For transactional creation of multiple tasks:
var jobs = new List<JobCreationModel>
{
jobbyClient.Factory
.Create(new SendEmailCommand { Email = "first@email.com" }),
jobbyClient.Factory
.Create(new SendEmailCommand { Email = "second@email.com" }),
};
await jobbyClient.EnqueueBatchAsync(jobs); To enforce strict execution order:
var sequenceBuilder = jobbyClient.Factory.CreateSequenceBuilder();
// Tasks will execute in strict order
sequenceBuilder.Add(jobbyClient.Factory
.Create(new SendEmailCommand { Email = "first@email.com" }));
sequenceBuilder.Add(jobbyClient.Factory
.Create(new SendEmailCommand { Email = "second@email.com" }));
var jobs = sequenceBuilder.GetJobs();
await jobbyClient.EnqueueBatchAsync(jobs); When creating a task, you can specify a group ID, and Jobby will ensure that no more than one task within each group is executed at any given time.
await jobbyClient.EnqueueCommandAsync(command, new JobOpts
{
SerializableGroupId = "SomeGroupId"
});The next task in the group will only start after the current task completes, whether successfully or unsuccessfully. If, in the event of an unsuccessful completion, you need to block the execution of any tasks from the same group, you must set the LockGroupIfFailed flag when creating the task:
await jobbyClient.EnqueueCommandAsync(command, new JobOpts
{
SerializableGroupId = "SomeGroupId",
LockGroupIfFailed = true
});For EF Core integration:
public class YourDbContext : DbContext
{
// Add DbSet for JobCreationModel
public DbSet<JobCreationModel> Jobs { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<JobCreationModel>().ToTable("jobby_jobs");
modelBuilder.Entity<JobCreationModel>().HasKey(x => x.Id);
// Apply snake_case naming convention for other configurations
}
}
// Enqueue via EF
var command = new SendEmailCommand { Email = "some@email.com" };
var jobEntity = jobbyClient.Factory.Create(command);
_dbContext.Jobs.Add(job);
await _dbContext.SaveChangesAsync(); EF Core example: Jobby.Samples.AspNet.
// Scheduled tasks are defined similarly to regular tasks
public class RecurrentJobCommand : IJobCommand
{
public static string GetJobName() => "SomeRecurrentJob";
}
public class RecurrentJobHandler : IJobCommandHandler<RecurrentJobCommand>
{
public async Task ExecuteAsync(SendEmailCommand command, JobExecutionContext ctx)
{
// Your scheduled task logic
}
}
// Schedule task using cron expression
// Will execute every 5 minutes
var command = new RecurrentJobCommand();
await jobbyClient.ScheduleRecurrentAsync(command, "*/5 * * * *"); Failed tasks can be retried according to configured policies.
A RetryPolicy defines:
- Maximum total execution attempts
- Delays between retries (in seconds)
var retryPolicy = new RetryPolicy
{
// Maximum total execution attempts
// Value of 3 means 1 initial attempt + 2 retries
MaxCount = 3,
// Delays between retry attempts
// First retry after 1 second, second after 2 seconds
IntervalsSeconds = [1, 2]
};
// IntervalSeconds doesn't require all values
// Example for 10 retries every 10 minutes:
retryPolicy = new RetryPolicy
{
MaxCount = 11,
IntervalsSeconds = [600]
}; Policies can be global or task-specific:
jobbyBuilder
// Default policy for all tasks
.UseDefaultRetryPolicy(defaultPolicy)
// Custom policy for SendEmailCommand
.UseRetryPolicyForJob<SendEmailCommand>(specialRetryPolicy); Jobby allows you to distribute tasks across independent queues. For example, it can be useful to allocate tasks that need to be executed on time (such as recurrent jobs) to a separate queue, or tasks that are very heavy and require reduced parallelism.
By default, all tasks go to the default queue.
The queue can be specified when creating a task:
jobbyClient.Enqueue(command, new JobOpts { QueueName = "special_queue"});Or defined in the command class by implementing the IHasDefaultJobOptions interface:
class SomeCommand : IJobCommand, IHasDefaultJobOptions
{
public string GetJobName() => "JobName";
// Queue for non-recurrent jobs
public JobOpts GetOptionsForEnqueuedJob() => new() { QueueName = "special_queue" };
// Queue for recurrent jobs
public RecurrentJobOptions GetOptionsForRecurrentJob() => new() { QueueName = "special_queue" };
}Additionally, when configuring the library, you can specify a default queue for all recurrent jobs:
builder.Services.AddJobbyServerAndClient((IAspNetCoreJobbyConfigurable jobbyBuilder) =>
{
jobbyBuilder
.AddJobsFromAssemblies(typeof(DemoJobCommand).Assembly)
// separate queue for all recurrent tasks
.UseQueueForAllRecurrent("recurrent")By default, JobbyServer only executes tasks from the default queue. To run tasks from other queues, you need to specify them during configuration in the UseServerSettings method:
builder.Services.AddJobbyServerAndClient((IAspNetCoreJobbyConfigurable jobbyBuilder) =>
{
// ...
jobbyBuilder.ConfigureJobby((sp, jobby) =>
{
jobby
.UseServerSettings(new JobbyServerSettings
{
// Here you specify the queues
// from which tasks should be executed
Queues = [
new QueueSettings { QueueName = "default" },
new QueueSettings { QueueName = "recurrent" },
new QueueSettings
{
QueueName = "heavy",
// If needed, you can reduce the degree of parallelism for a separate queue
MaxDegreeOfParallelism = 1,
}
]
})It is possible to wrap background task handler calls with your own middlewares.
To create a middleware, you need to implement the IJobbyMiddleware interface:
public class SomeMiddleware : IJobbyMiddleware
{
public async Task ExecuteAsync<TCommand>(TCommand command, JobExecutionContext ctx, IJobCommandHandler<TCommand> handler)
where TCommand : IJobCommand
{
// Logic to be executed before the background task call can be placed here
// ....
await handler.ExecuteAsync(command, ctx);
// Logic to be executed after the background task call can be placed here
// ....
}
}Middleware supports dependency injection through the constructor.
Configuration:
builder.Services.AddJobbyServerAndClient(jobbyBuilder =>
{
jobbyBuilder.ConfigureJobby((serviceProvider, jobby) => {
// ...
jobby.ConfigurePipeline(pipeline => {
// This is how a singleton middleware without dependencies is added
pipeline.Use(new SomeMiddleware());
// This is how a singleton middleware with non-scoped dependencies can be added
// In this case, the SomeMiddleware type must be registered in the DI container!
pipeline.Use(serviceProvider.GetRequiredService<SomeMiddleware>());
// This is how a scoped middleware or middleware with scoped dependencies can be added
// In this case, the SomeMiddleware type must be registered in the DI container!
pipeline.Use<SomeMiddleware>();
});
});
});More examples: Jobby.Samples.AspNet.
Jobby collects several metrics about background job execution on a given instance:
jobby.inst.jobs.started- number of started jobsjobby.inst.jobs.completed- number of successfully completed jobsjobby.inst.jobs.retried- number of job retries scheduled after a failurejobby.inst.jobs.failed- number of jobs that failed after the last retry attempt plus the number of failed launches of recurrent jobsjobby.inst.jobs.duration- execution time histogram of background jobs
To enable metric collection, you must call the UseMetrics method during configuration:
builder.Services.AddJobbyServerAndClient((IAspNetCoreJobbyConfigurable jobbyBuilder) =>
{
jobbyBuilder.ConfigureJobby((sp, jobby) =>
{
jobby
.UseMetrics() // Enable metric collection
// ...
});
});In OpenTelemetry, Jobby metrics are added as follows:
builder.Services
.AddOpenTelemetry()
.WithMetrics(builder => {
// Add all metrics from Jobby to OpenTelemetry
builder.AddMeter(JobbyMeterNames.GetAll());
});In the Jobby.Samples.AspNet example, metric collection is enabled with export to Prometheus format via the /metrics endpoint.
To enable tracing, you should call the UseTracing method during configuration:
builder.Services.AddJobbyServerAndClient((IAspNetCoreJobbyConfigurable jobbyBuilder) =>
{
jobbyBuilder.ConfigureJobby((sp, jobby) =>
{
jobby
.UseTracing() // Execute jobs within an Activity
// ...
});
});You can enable the export of Jobby job traces via OpenTelemetry as follows:
builder.Services
.AddOpenTelemetry()
.ConfigureResource(resource => resource.AddService(serviceName: "Jobby.Samples.AspNet"))
.WithTracing(builder =>
{
builder.AddConsoleExporter();
// Add Jobby job execution traces to OpenTelemetry
builder.AddSource(JobbyActivitySourceNames.JobsExecution);
});In the Jobby.Samples.AspNet example, metric collection with export to stdout is enabled.