我想非常快地连接到很多不同网站的列表.我使用asyncio以异步方式执行此操作,现在想要添加超时,以便在需要花费太长时间来响应时忽略连接.
我该如何实现?
import ssl import asyncio from contextlib import suppress from concurrent.futures import ThreadPoolExecutor import time @asyncio.coroutine def run(): while True: host = yield from q.get() if not host: break with suppress(ssl.CertificateError): reader, writer = yield from asyncio.open_connection(host[1], 443, ssl=True) #timout option? reader.close() writer.close() @asyncio.coroutine def load_q(): # only 3 entries for debugging reasons for host in [[1, 'python.org'], [2, 'qq.com'], [3, 'google.com']]: yield from q.put(host) for _ in range(NUM): q.put(None) if __name__ == "__main__": NUM = 1000 q = asyncio.Queue() loop = asyncio.get_event_loop() loop.set_default_executor(ThreadPoolExecutor(NUM)) start = time.time() coros = [asyncio.async(run()) for i in range(NUM)] loop.run_until_complete(load_q()) loop.run_until_complete(asyncio.wait(coros)) end = time.time() print(end-start)
(旁注:有人知道如何优化这个吗?)
您可以将调用包装到open_connection
in中asyncio.wait_for
,这允许您指定超时:
with suppress(ssl.CertificateError): fut = asyncio.open_connection(host[1], 443, ssl=True) try: # Wait for 3 seconds, then raise TimeoutError reader, writer = yield from asyncio.wait_for(fut, timeout=3) except asyncio.TimeoutError: print("Timeout, skipping {}".format(host[1])) continue
请注意,在TimeoutError
引发时,open_connection
协程也会被取消.如果您不希望它被取消(虽然我认为您确实希望在这种情况下取消它),您已将呼叫包裹起来asyncio.shield
.