Introducing Long Running Tasks

At Ampt, we're always looking for ways to improve the developer experience and provide more capabilities to enhance Node.js apps in the cloud. Today, we're excited to announce a new task interface for the @ampt/sdk. This new interface offers a more flexible, reliable, performant, and reusable way to define and run periodic tasks in your applications.

Previously, developers could schedule tasks using Ampt's schedule interface, but there were a number of limitations. Most notably, schedules could only run for a maximum of five minutes. While this was fine for a number of use cases, we recognized the need for greater flexibility, especially for intensive workloads like nightly ETL jobs. The new task interface moves beyond this limitation, providing a more robust solution for managing and scheduling tasks that require significantly longer execution times.

In addition to longer running times, the task interface was designed to support reusability of task definitions. This allows developers to define a task once, then schedule or run it as many times as they want with the ability to pass additional context into each execution.

The Power of Tasks

Tasks are designed to accommodate applications that require recurring and one-off task execution. Whether it's running nightly ETL jobs, executing database aggregation scripts, performing data enrichment via third-party API calls, or scheduling a one-off workflow in the future, the task interface offers a powerful solution.

One reason for the previous execution limit was because we used AWS Lambda to execute the schedules. As part of our private beta, we set the limit to 5 minutes, but due to the underlying implementation, we would only have been able to increase this to a maximum of 15 minutes anyway. Also, Lambda functions are not the most cost effective or performant way to execute long-running tasks, so a new, more robust, and smarter solution was needed.

Tasks now utilize AWS Fargate alongside AWS Lambda to ensure you have the necessary resources, memory, and runtime capabilities to optimally run your workloads. Ampt detects timeouts and other indicators to seamlessly transition your tasks to AWS Fargate when necessary, all without changing a single line of code. This not only facilitates long-running tasks, but enables future optimization capabilities.

This unique "smart compute" capability allows Ampt to ensure optimal application performance by smartly deploying to a combination of AWS Lambda, AWS App Runner, and now, AWS Fargate. As a developer, you focus on writing your business logic, and Ampt handles the rest, automatically selecting the best infrastructure for your application's needs.

Defining and Running Tasks in Code

Define a task using the new task interface from the Ampt SDK:

javascriptCopy code
import { task } from '@ampt/sdk';

// Use default configuration
const myTask = task('my task', (event) => {
  console.log('Running my task');
});

// Or with configuration
const myTask2 = task('my task 2', { timeout: 30000 }, (event) => {
  console.log('Running my task');
});

Tasks can be set to run periodically using .every() and .cron() methods:

javascriptCopy code
myTask.every('1 hour', { foo: 'bar' });
myTask2.cron('0 0 ? * TUE *', { foo: 'bar' });

Tasks can also be executed immediately or at a scheduled time in the future. In either case, the tasks run asynchronously in the background.

javascriptCopy code
await myTask.run({ foo: 'bar' });
await myTask.run('1 hour', { foo: 'bar' }); // run 1 hour later XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Embrace the Power of Tasks

We're looking forward to seeing the solutions you'll build with this functionality. For detailed information about the new task feature, visit our documentation. Sign up to start building now. Also, be sure to follow along with our latest updates and join our growing community on Discord. We're looking forward to building, learning, and innovating together.