Python 并发编程（上） (4)

不建议：无限制的创建线程。

import threading def task(video_url): pass url_list = ["www.xxxx-{}.com".format(i) for i in range(30000)] for url in url_list: t = threading.Thread(target=task, args=(url,)) t.start() # 这种每次都创建一个线程去操作，创建任务的太多，线程就会特别多，可能效率反倒降低了。

建议：使用线程池

示例1：

import time from concurrent.futures import ThreadPoolExecutor # pool = ThreadPoolExecutor(100) # pool.submit(函数名,参数1，参数2，参数...) def task(video_url,num): print("开始执行任务", video_url) time.sleep(5) # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) url_list = ["www.xxxx-{}.com".format(i) for i in range(300)] for url in url_list: # 在线程池中提交一个任务，线程池中如果有空闲线程，则分配一个线程去执行，执行完毕后再将线程交还给线程池；如果没有空闲线程，则等待。 pool.submit(task, url,2) print("END")

示例2：等待线程池的任务执行完毕。

import time from concurrent.futures import ThreadPoolExecutor def task(video_url): print("开始执行任务", video_url) time.sleep(5) # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) url_list = ["www.xxxx-{}.com".format(i) for i in range(300)] for url in url_list: # 在线程池中提交一个任务，线程池中如果有空闲线程，则分配一个线程去执行，执行完毕后再将线程交还给线程池；如果没有空闲线程，则等待。 pool.submit(task, url) print("执行中...") pool.shutdown(True) # 等待线程池中的任务执行完毕后，在继续执行 print(\'继续往下走\')

示例3：任务执行完任务，再干点其他事。

import time import random from concurrent.futures import ThreadPoolExecutor, Future def task(video_url): print("开始执行任务", video_url) time.sleep(2) return random.randint(0, 10) def done(response): print("任务执行后的返回值", response.result()) # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) url_list = ["www.xxxx-{}.com".format(i) for i in range(15)] for url in url_list: # 在线程池中提交一个任务，线程池中如果有空闲线程，则分配一个线程去执行，执行完毕后再将线程交还给线程池；如果没有空闲线程，则等待。 future = pool.submit(task, url) future.add_done_callback(done) # 是子主线程执行，传入参数是函数。 # 可以做分工，例如：task专门下载，done专门将下载的数据写入本地文件。

示例4：最终统一获取结果。

import time import random from concurrent.futures import ThreadPoolExecutor,Future def task(video_url): print("开始执行任务", video_url) time.sleep(2) return random.randint(0, 10) # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) future_list = [] url_list = ["www.xxxx-{}.com".format(i) for i in range(15)] for url in url_list: # 在线程池中提交一个任务，线程池中如果有空闲线程，则分配一个线程去执行，执行完毕后再将线程交还给线程池；如果没有空闲线程，则等待。 future = pool.submit(task, url) future_list.append(future) pool.shutdown(True) for fu in future_list: print(fu.result())

案例：基于线程池下载豆瓣图片。

26044585,Hush,https://hbimg.huabanimg.com/51d46dc32abe7ac7f83b94c67bb88cacc46869954f478-aP4Q3V 19318369,柒十一,https://hbimg.huabanimg.com/703fdb063bdc37b11033ef794f9b3a7adfa01fd21a6d1-wTFbnO 15529690,Law344,https://hbimg.huabanimg.com/b438d8c61ed2abf50ca94e00f257ca7a223e3b364b471-xrzoQd 18311394,Jennah·,https://hbimg.huabanimg.com/4edba1ed6a71797f52355aa1de5af961b85bf824cb71-px1nZz 18009711,可洛爱画画,https://hbimg.huabanimg.com/03331ef39b5c7687f5cc47dbcbafd974403c962ae88ce-Co8AUI 30574436,花姑凉~,https://hbimg.huabanimg.com/2f5b657edb9497ff8c41132e18000edb082d158c2404-8rYHbw 17740339,小巫師,https://hbimg.huabanimg.com/dbc6fd49f1915545cc42c1a1492a418dbaebd2c21bb9-9aDqgl 18741964,桐末tonmo,https://hbimg.huabanimg.com/b60cee303f62aaa592292f45a1ed8d5be9873b2ed5c-gAJehO 30535005,TANGZHIQI,https://hbimg.huabanimg.com/bbd08ee168d54665bf9b07899a5c4a4d6bc1eb8af77a4-8Gz3K1 31078743,你的老杨,https://hbimg.huabanimg.com/c46fbc3c9a01db37b8e786cbd7174bbd475e4cda220f4-F1u7MX 25519376,尺尺寸,https://hbimg.huabanimg.com/ee29ee198efb98f970e3dc2b24c40d89bfb6f911126b6-KGvKes 21113978,C-CLong,https://hbimg.huabanimg.com/7fa6b2a0d570e67246b34840a87d57c16a875dba9100-SXsSeY 24674102,szaa,https://hbimg.huabanimg.com/0716687b0df93e8c3a8e0925b6d2e4135449cd27597c4-gWdv24 30508507,爱起床的小灰灰,https://hbimg.huabanimg.com/4eafdbfa21b2f300a7becd8863f948e5e92ef789b5a5-1ozTKq 12593664,yokozen,https://hbimg.huabanimg.com/cd07bbaf052b752ed5c287602404ea719d7dd8161321b-cJtHss 16899164,一阵疯,https://hbimg.huabanimg.com/0940b557b28892658c3bcaf52f5ba8dc8402100e130b2-G966Uz 847937,卩丬My㊊伴er彎,https://hbimg.huabanimg.com/e2d6bb5bc8498c6f607492a8f96164aa2366b104e7a-kWaH68 31010628,慢慢即漫漫,https://hbimg.huabanimg.com/c4fb6718907a22f202e8dd14d52f0c369685e59cfea7-82FdsK 13438168,海贼玩跑跑,https://hbimg.huabanimg.com/1edae3ce6fe0f6e95b67b4f8b57c4cebf19c501b397e-BXwiW6 28593155,源稚生,https://hbimg.huabanimg.com/626cfd89ca4c10e6f875f3dfe1005331e4c0fd7fd429-9SeJeQ 28201821,合伙哼哼,https://hbimg.huabanimg.com/f59d4780531aa1892b80e0ec94d4ec78dcba08ff18c416-769X6a 28255146,漫步AAA,https://hbimg.huabanimg.com/3c034c520594e38353a039d7e7a5fd5e74fb53eb1086-KnpLaL 30537613,配䦹,https://hbimg.huabanimg.com/efd81d22c1b1a2de77a0e0d8e853282b83b6bbc590fd-y3d4GJ 22665880,日后必火,https://hbimg.huabanimg.com/69f0f959979a4fada9e9e55f565989544be88164d2b-INWbaF 16748980,keer521521,https://hbimg.huabanimg.com/654953460733026a7ef6e101404055627ad51784a95c-B6OFs4 30536510,“西辞”,https://hbimg.huabanimg.com/61cfffca6b2507bf51a507e8319d68a8b8c3a96968f-6IvMSk 30986577,艺成背锅王,https://hbimg.huabanimg.com/c381ecc43d6c69758a86a30ebf72976906ae6c53291f9-9zroHF 26409800,CsysADk7,https://hbimg.huabanimg.com/bf1d22092c2070d68ade012c588f2e410caaab1f58051-ahlgLm 30469116,18啊全阿,https://hbimg.huabanimg.com/654953460733026a7ef6e101404055627ad51784a95c-B6OFs4 17473505,椿の花,https://hbimg.huabanimg.com/0e38d810e5a24f91ebb251fd3aaaed8bb37655b14844c-pgNJBP 19165177,っ思忆゜♪,https://hbimg.huabanimg.com/4815ea0e4905d0f3bb82a654b481811dadbfe5ce2673-vMVr0B 16059616,格林熊丶,https://hbimg.huabanimg.com/8760a2b08d87e6ed4b7a9715b1a668176dbf84fec5b-jx14tZ 30734152,sCWVkJDG,https://hbimg.huabanimg.com/f31a5305d1b8717bbfb897723f267d316e58e7b7dc40-GD3e22 24019677,虚无本心,https://hbimg.huabanimg.com/6fdfa9834abe362e978b517275b06e7f0d5926aa650-N1xCXE 16670283,Y-雨后天空,https://hbimg.huabanimg.com/a3bbb0045b536fc27a6d2effa64a0d43f9f5193c177f-I2vHaI 21512483,汤姆2,https://hbimg.huabanimg.com/98cc50a61a7cc9b49a8af754ffb26bd15764a82f1133-AkiU7D 16441049,笑潇啸逍小鱼,https://hbimg.huabanimg.com/ae8a70cd85aff3a8587ff6578d5cf7620f3691df13e46-lmrIi9 24795603,⁢⁢⁢⁢⁢v,https://hbimg.huabanimg.com/a7183cc3a933aa129d7b3230bf1378fd8f5857846cc5-3tDtx3 29819152,妮玛士珍多,https://hbimg.huabanimg.com/ca4ecb573bf1ff0415c7a873d64470dedc465ea1213c6-RAkArS 19101282,陈勇敢❤,https://hbimg.huabanimg.com/ab6d04ebaff3176e3570139a65155856871241b58bc6-Qklj2E 28337572,爱意随风散,https://hbimg.huabanimg.com/117ad8b6eeda57a562ac6ab2861111a793ca3d1d5543-SjWlk2 17342758,幸运instant,https://hbimg.huabanimg.com/72b5f9042ec297ae57b83431123bc1c066cca90fa23-3MoJNj 18483372,Beau染,https://hbimg.huabanimg.com/077115cb622b1ff3907ec6932e1b575393d5aae720487-d1cdT9 22127102,栽花的小蜻蜓,https://hbimg.huabanimg.com/6c3cbf9f27e17898083186fc51985e43269018cc1e1df-QfOIBG 13802024,LoveHsu,https://hbimg.huabanimg.com/f720a15f8b49b86a7c1ee4951263a8dbecfe3e43d2d-GPEauV 22558931,白驹过隙丶梨花泪う,https://hbimg.huabanimg.com/e49e1341dfe5144da5c71bd15f1052ef07ba7a0e1296b-jfyfDJ 11762339,cojoy,https://hbimg.huabanimg.com/5b27f876d5d391e7c4889bc5e8ba214419eb72b56822-83gYmB 30711623,雪碧学长呀,https://hbimg.huabanimg.com/2c288a1535048b05537ba523b3fc9eacc1e81273212d1-nr8M4t 18906718,西霸王,https://hbimg.huabanimg.com/7b02ad5e01bd8c0a29817e362814666a7800831c154a6-AvBDaG 31037856,邵阳的小哥哥,https://hbimg.huabanimg.com/654953460733026a7ef6e101404055627ad51784a95c-B6OFs4 26830711,稳健谭,https://hbimg.huabanimg.com/51547ade3f0aef134e8d268cfd4ad61110925aefec8a-NKPEYX import os import requests from concurrent.futures import ThreadPoolExecutor def download(file_name, image_url): res = requests.get( url=image_url, headers={ "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" } ) # 检查images目录是否存在？不存在，则创建images目录 if not os.path.exists("images"): # 创建images目录 os.makedirs("images") file_path = os.path.join("images", file_name) # 2.将图片的内容写入到文件 with open(file_path, mode=\'wb\') as img_object: img_object.write(res.content) # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) with open("mv.csv", mode=\'r\', encoding=\'utf-8\') as file_object: for line in file_object: nid, name, url = line.split(",") file_name = "{}.png".format(name) pool.submit(download, file_name, url) import os import requests from concurrent.futures import ThreadPoolExecutor def download(image_url): res = requests.get( url=image_url, headers={ "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" } ) return res def outer(file_name): def save(response): res = response.result() # 写入本地 # # 检查images目录是否存在？不存在，则创建images目录 if not os.path.exists("images"): # 创建images目录 os.makedirs("images") file_path = os.path.join("images", file_name) # # 2.将图片的内容写入到文件 with open(file_path, mode=\'wb\') as img_object: img_object.write(res.content) return save # 创建线程池，最多维护10个线程。 pool = ThreadPoolExecutor(10) with open("mv.csv", mode=\'r\', encoding=\'utf-8\') as file_object: for line in file_object: nid, name, url = line.split(",") file_name = "{}.png".format(name) fur = pool.submit(download, url) fur.add_done_callback(outer(file_name))# 传入参数，可以使用闭包的形式。 7.单例模式（扩展）

面向对象 + 多线程相关的一个面试题（以后项目和源码中会用到）。

之前写一个类，每次执行 类() 都会实例化一个类的对象。

class Foo: pass obj1 = Foo() obj2 = Foo() print(obj1,obj2)

以后开发会遇到单例模式，每次实例化类的对象时，都是最开始创建的那个对象，不再重复创建对象。