Sesam Workflows Knowledge Centre Help library Using Sesam with Python Async Programming
Asynchronous Programming in SesamCommands with OneWorkflow
SesamCommands, a Python package, seamlessly integrates with the OneWorkflow Python package to enhance workflow management. The core of OneWorkflow is designed around asynchronous programming, a powerful technique that optimizes HTTP requests and file transfers, enhancing efficiency and responsiveness. This asynchronous foundation allows SesamCommands to perform tasks concurrently, improving overall performance. To delve deeper into the asynchronous programming concept in Python, please refer to the Python Async Programming section.
In many of our workflow examples, SesamCommands employs a minimalistic approach, as demonstrated below for the Standard Python environment:
asyncio.run(
run_managed_commands_in_parallel_async(
client=one_workflow_client,
commands_info=[cmd_info],
)
This snippet showcases the use of asyncio.run, a high-level function that executes a coroutine and returns its result. It schedules the provided coroutine on the event loop and finalizes it upon completion.
In a Jupyter notebook, due to its native support for asynchronous operations, the same statement can be used slightly differently:
await run_managed_commands_in_parallel_async(
client=one_workflow_client,
commands_info=[cmd_info],
)
This snippet showcases the use of the await keyword, which is used to pause the execution of the coroutine until the awaited coroutine completes. It allows other coroutines to run in the meantime, thus enabling asynchronous behavior and improving the overall efficiency of the program.
Regardless of the Python environment, be it Jupyter or Standard, the coroutine run_managed_commands_in_parallel_async adeptly manages the scheduling and execution of other coroutines. It handles HTTP requests and file transfer operations efficiently. With this understanding, let's explore the practical application of these concepts within the OneWorkflow framework.
Async HTTP Requests
OneWorkflow uses the asyncio library to make asynchronous HTTP requests. This allows it to initiate a request and execute other tasks without waiting for the server's response. Once the response is received, OneWorkflow can then handle it. This is particularly useful when OneWorkflow needs to make multiple requests, as it can send them all at once and then handle the responses as they come in, rather than waiting for each response before sending the subsequent request.
File Uploads and Downloads
OneWorkflow also uses async programming to upload and download files. This allows it to initiate a file transfer and then execute other tasks without waiting for the transfer to complete. Once the file is fully transferred, OneWorkflow can then handle it. This is particularly useful when dealing with large files, as it allows OneWorkflow to better use its resources by not blocking execution while waiting for the file transfer to complete.
Using async programming, OneWorkflow can provide a more responsive and efficient service, making it a powerful tool for managing workflows.
Python Async Programming
Introduction
Asynchronous programming is a coding technique that enables a program to carry out multiple tasks simultaneously without waiting for one task to complete before starting the next. In Python, async programming mainly utilizes the asyncio library, which offers a structure for creating concurrent code using the async and await keywords.
Why Use Async Programming?
- Improved Performance: Async programming can significantly improve the performance of applications that perform many I/O operations, such as web servers, by allowing them to handle multiple tasks simultaneously.
- Non-blocking I/O: Unlike traditional synchronous code, which can block the execution of a program while waiting for I/O operations to complete, async code can continue executing other tasks during these waits.
- Efficient Resource Utilization: Async programming uses system resources better, mainly when dealing with many I/O-bound tasks, like network requests or file operations.
Basic Concepts
async and await
- async: Used to define a coroutine, a function that can pause and resume its execution.
- await: Used to pause the execution of a coroutine until the result of an awaited operation is available.
Example
Here's a simple example to illustrate async programming in Python:
import asyncio
async def say_hello():
print("Hello")
await asyncio.sleep(1)
print("World")
# Running the async function
asyncio.run(say_hello())
Event Loop
The event loop is the core of every asyncio application. It is responsible for executing async tasks, managing I/O operations, and scheduling callbacks.
Key Libraries and Tools
- asyncio: The standard library for writing asynchronous code in Python.
- aiohttp: A popular library for making asynchronous HTTP requests.
Further Reading
OneWorkflow Uses Async Programming
OneWorkflow, like many modern applications, relies heavily on I/O operations such as making HTTP requests and uploading or downloading files. These operations can be time-consuming and, if handled synchronously, can significantly slow down the application. This is where asynchronous programming comes in.
Asynchronous programming allows OneWorkflow to initiate an I/O operation, such as an HTTP request or file download, and then continue executing other code without waiting for the operation to complete. Once the operation is complete, OneWorkflow can then handle the result. This allows OneWorkflow to handle multiple I/O operations concurrently, improving performance and responsiveness.