Xem thêm

Using Python scheduler to schedule asynchronous tasks

Task scheduling is a common practice in modern software infrastructure that helps automate non-critical tasks by running them at a fixed time or interval. By automating tasks such as regular data cleanups or media conversion...

Task scheduling is a common practice in modern software infrastructure that helps automate non-critical tasks by running them at a fixed time or interval. By automating tasks such as regular data cleanups or media conversion in batches, organizations can increase operational efficiency and allocate more resources to customer-facing needs.

Python, with its simplicity and wide range of use cases, is a popular programming language for task scheduling. In this article, we will explore how to use the sched module in Python to schedule tasks.

Python scheduling using the sched module

There are multiple Python packages and modules available for task scheduling. These libraries can also repeat the execution of your script over fixed intervals, ensuring that important maintenance routines are run consistently.

Some prominent use cases of task scheduling in Python include:

  • Scheduling shutdowns/restarts: You can schedule your systems to automatically shut down or restart at certain times in the day to refresh memory and other resources.
  • Running routine cleanups: You can schedule jobs to clear your database of ghost records or delete duplicate files from your file storage.
  • Generating reports: You can schedule jobs to generate routine reports, such as a weekly head count roster for your company.
  • Batch processing: You can batch multiple operations and run them at fixed time intervals for efficient resource and time utilization.

The sched module

Among the available packages and modules for task scheduling in Python, the sched module is a built-in, general-purpose event scheduler. It is useful for one-off tasks and is thread-safe.

Here's an example of how you can use the sched module to schedule print operations with delays:

import sched
import time

def print_task():
    print("Task executed at:", time.time())

scheduler = sched.scheduler(time.time, time.sleep)

scheduler.enter(2, 1, print_task)
scheduler.enter(5, 1, print_task)
scheduler.enter(10, 1, print_task)

scheduler.run()

The enter() function is used to schedule tasks. It takes three mandatory arguments:

  • delay: The number of seconds to wait before executing the task (measured from the time the run() function is called).
  • priority: The priority of the task (the lower the number, the higher the priority).
  • action: The function to be called when the task is executed.

You can also pass in two additional optional arguments:

  • argument: A sequence that carries the positional arguments for the action.
  • kwargs: A dictionary that carries the keyword arguments for the action.

Whenever the action function is executed, it is called in the following manner:

action(*argument, **kwargs)

Let's take a practical example of scheduling tasks that print exchange rate values against given base currencies:

import sched
import time

def print_exchange_rates(base_currency):
    # Code to fetch exchange rate values against the given base currency
    print("Exchange rates for", base_currency, "are:", time.time())

scheduler = sched.scheduler(time.time, time.sleep)

scheduler.enter(2, 1, print_exchange_rates, argument=("USD",))
scheduler.enter(5, 1, print_exchange_rates, argument=("EUR",))
scheduler.enter(10, 1, print_exchange_rates, argument=("GBP",))

scheduler.run()

In this example, we schedule the print_exchange_rates() function to be executed at different times with varying delays and base currencies.

However, the sched module does not support recurring schedules out-of-the-box, and you need to keep your host system running to execute the scheduled tasks on time. To address these limitations and implement task scheduling more conveniently, let's explore a powerful platform called Airplane.

Python scheduling using Airplane

Airplane offers a simple and elegant solution for task scheduling. It provides a powerful platform to set up tasks that can be executed manually or on a recurring schedule. Tasks can be single business operations or multi-step functions that anyone on your team can use. Let's see how you can implement the previous example using Airplane.

Prerequisites

First, create an Airplane account and install the Airplane CLI. This will help you package and deploy your script to the Airplane cloud.

Once installed, run the following command to ensure that you're logged in and ready to go:

airplane login

A browser window will open, asking you to log in to your Airplane account. Once done, you can create a new folder for the Airplane Task to be initialized in.

1. Initialize a new task

Initialize a new Airplane Task by running the following command:

airplane init

You will be asked for some basic information about your task. Enter the required details.

Replace the contents of the exchange_rate_job.py file with the following code:

import time

def print_exchange_rates(base_currency):
    # Code to fetch exchange rate values against the given base currency
    return "Exchange rates for" + base_currency + "are:" + str(time.time())

def main():
    exchange_rates = []
    exchange_rates.append(print_exchange_rates("USD"))
    exchange_rates.append(print_exchange_rates("EUR"))
    exchange_rates.append(print_exchange_rates("GBP"))

    return exchange_rates

if __name__ == "__main__":
    print(main())

In this script, we have eliminated the code related to the sched module since Airplane handles the scheduling.

2. Deploy your task

The next step is to deploy your script as a task on the Airplane servers. Run the following command:

airplane deploy

The Airplane CLI will prompt you to create a new task with a unique slug. Follow the instructions to complete the deployment process.

3. Test the deployed task

Once the deployment succeeds, you can head over to your Airplane dashboard to see the new task in your library.

Click on the task, and you will find options to execute and schedule it. Click on "Execute task" to see your task in action.

You will notice that the output generated from your Python script is intelligently converted into a tabular format, making it easy to analyze. Airplane also allows you to export this data into a CSV file directly from the portal.

4. Schedule recurring tasks

After setting up and testing your task, it's time to schedule it. Go back to the task details page and click the "New schedule" button.

You will be provided with options to choose the frequency of task execution. For complete control, you can choose the "Cron" schedule option and enter the routine in cron syntax.

Once you have chosen a schedule, click "Create schedule" to create it, and the task will be executed accordingly.

You can go back to the task details page to view the executions of the task and click on any recent run to view its logs and output.

Unlike traditional scheduling methods in Python, you don't need to keep your host machine running to run the scheduled tasks. Airplane simplifies the task scheduling process and allows you to focus on other aspects of your application.

In conclusion, task scheduling is an important aspect of application management. While the Python sched module provides a simple solution for scheduling tasks in a local Python runtime, Airplane offers a more convenient and efficient way to schedule tasks. With Airplane, you can easily schedule and manage tasks without the need to maintain a scheduler instance manually.

1