当需要知道某一时刻有多少对象被使用,以及他们是否被垃圾收集时,通过对堆的查看,可以很好的得到结果。
安装
pip install guppy3代码修改如下
import time import numpy as np # import imageio # import PIL # import matplotlib.pyplot as plt from guppy import hpy # import cv2 as cv from functools import wraps x1, x2, y1, y2 = -1.8, 1.8, -1.8, 1.8 c_real, c_imag = -0.62772, -0.42193 def timefn(fn): @wraps(fn) def measure_time(*args, **kwargs): t1 = time.time() result = fn(*args, **kwargs) t2 = time.time() print("@timefn:" + fn.__name__ + " took " + str(t2 - t1), " seconds") return result return measure_time def calculate_z_serial_purepython(maxiter, zs, cs): output = [0] * len(zs) for i in range(len(zs)): n = 0 z = zs[i] c = cs[i] while n < maxiter and abs(z) < 2: z = z * z + c n += 1 output[i] = n return output def calc_pure_python(desired_width, max_itertions): x_step = (float(x2 - x1)) / float(desired_width) y_step = (float(y2 - y1)) / float(desired_width) x, y = [], [] ycoord = y1 while ycoord < y2: y.append(ycoord) ycoord += y_step xcoord = x1 while xcoord < x2: x.append(xcoord) xcoord += x_step print("heapy after creating y and x lists of floats") hp = hpy() h = hp.heap() print(h) print("") zs, cs = [], [] for ycoord in y: for xcoord in x: zs.append(complex(xcoord, ycoord)) cs.append(complex(c_real, c_imag)) print("heapy after creating zs and cs using complex numbers") h = hp.heap() print(h) print("") print(f"Length of x: {len(x)}") print(f"Total elements: {len(zs)}") start_time = time.time() output = calculate_z_serial_purepython(max_itertions, zs, cs) end_time = time.time() secs = end_time - start_time print("calculate_z_serial_purepython took", secs, "seconds") print("") print("heapy after calling calculate_z_serial_purepython") h = hp.heap() print(h) assert sum(output) == 33219980 if __name__ == "__main__": calc_pure_python(desired_width=1000, max_itertions=300)在使用的时候发现不能 import imageio 这个库,不然调用 hp.heap() 的时候会直接退出。。。。
输出
heapy after creating y and x lists of floats Partition of a set of 96564 objects. Total size = 12355685 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 27588 29 4021242 33 4021242 33 str 1 25226 26 1920104 16 5941346 48 tuple 2 12595 13 962362 8 6903708 56 bytes 3 6336 7 912831 7 7816539 63 types.CodeType 4 5855 6 796280 6 8612819 70 function 5 922 1 789656 6 9402475 76 type 6 255 0 499248 4 9901723 80 dict of module 7 922 1 496880 4 10398603 84 dict of type 8 514 1 284608 2 10683211 86 set 9 529 1 276160 2 10959371 89 dict (no owner) <248 more rows. Type e.g. '_.more' to view.> heapy after creating zs and cs using complex numbers Partition of a set of 2096566 objects. Total size = 93750677 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 2000003 95 64000096 68 64000096 68 complex 1 536 0 17495680 19 81495776 87 list 2 27588 1 4021242 4 85517018 91 str 3 25226 1 1920104 2 87437122 93 tuple 4 12595 1 962362 1 88399484 94 bytes 5 6336 0 912831 1 89312315 95 types.CodeType 6 5855 0 796280 1 90108595 96 function 7 922 0 789656 1 90898251 97 type 8 255 0 499248 1 91397499 97 dict of module 9 922 0 496880 1 91894379 98 dict of type <248 more rows. Type e.g. '_.more' to view.> Length of x: 1000 Total elements: 1000000 calculate_z_serial_purepython took 24.96058201789856 seconds heapy after calling calculate_z_serial_purepython Partition of a set of 2196935 objects. Total size = 104561033 bytes. Index Count % Size % Cumulative % Kind (class / dict of class) 0 2000003 91 64000096 61 64000096 61 complex 1 537 0 25495744 24 89495840 86 list 2 27588 1 4021242 4 93517082 89 str 3 102343 5 2870796 3 96387878 92 int 4 25226 1 1920104 2 98307982 94 tuple 5 12595 1 962362 1 99270344 95 bytes 6 6336 0 912831 1 100183175 96 types.CodeType 7 5855 0 796280 1 100979455 97 function 8 922 0 789656 1 101769111 97 type 9 255 0 499248 0 102268359 98 dict of module <248 more rows. Type e.g. '_.more' to view.>可以发现:
在创建了 zs 和 cs 列表后,内存增长了大约 80M, 2000003 个复数对象消耗了 64000096 字节内存,占用了当前大部分的内存。
第 3 段中,计算完集合后占用了 104M 的内存,除了之前的复数,现在还保存了大量的整数,列表中的项目也增多了。
hpy.setrelheap() 可以用来创建一个断点,当后续调用 hpy.heap() 时,会产生一个跟这个断点的差额,这样可以略过断点前产生的内存分配。
小节这篇文章介绍了一些对于代码运行时内存的分析方法,相信通过合理运用这些方法对代码进行分析修改,能写出性能更优的代码。
这是彩蛋