Python asyncio cancel unawaited coroutines

So given a bit of a complex setup, which is used to generate a list of queries to be run semi parallel (using a semaphore to not run too many queries at the same time, to not ddos the server).

i have an (in itself async) function that creates a number of queries:

async def run_query(self, url):
    async with self.semaphore:
        return await some_http_lib(url)

async def create_queries(self, base_url):
   # ...gathering logic is ofc a bit more complex in the real setting
   urls = await some_http_lib(base_url).json()
 
   coros = [self.run_query(url) for url in urls]  # note: not executed just yet
   return coros

async def execute_queries(self):
   queries = await self.create_queries('/seom/url')
   _logger.info(f'prepared {len(queries)} queries')
   results = []
   done = 0
   for query in queries:
       # ath this point, the request is actually triggered
       result = await query

       # ...some postprocessing

       if not result['success']:
           raise QueryException(result['message'])  # ...internal exception
       
       done  += 1
       _logger.info(f'{done} of {len(queries)} queries done')
       results.append(result)
    
   return results

Now this works very nicely, executing exactly as i planned and i can handle an exception in one of the queries by aborting the whole operation.

async def run():
    try:
       return await QueryRunner.execute_queries()
    except QueryException:
       _logger.error('something went horribly wrong')
       return None

The only problem is that the program is terminated, but leaves me with the usual RuntimeWarning: coroutine QueryRunner.run_query was never awaited, because the queries later in the queue are (rightfully) not executed and as such not awaited.

Is there any way to cancel these unawaited coroutines? Would it be otherwise possible to supress this warning?



Read more here: https://stackoverflow.com/questions/66278106/python-asyncio-cancel-unawaited-coroutines

Content Attribution

This content was originally published by Mr.Manhattan at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: