以下是代码:
import numpy as np from keras.preprocessing import image from keras.applications import inception_v3 from keras import backend as K from PIL import Image # Load pre-trained image recognition model model = inception_v3.InceptionV3() # Grab a reference to the first and last layer of the neural net model_input_layer = model.layers[0].input model_output_layer = model.layers[-1].output # Choose an ImageNet object to fake # The list of classes is available here: https://gist.github.com/ageitgey/4e1342c10a71981d0b491e1b8227328b # Class #859 is "toaster" object_type_to_fake = 859 # Load the image to hack img = image.load_img("cat.png", target_size=(299, 299)) original_image = image.img_to_array(img) # Scale the image so all pixel intensities are between [-1, 1] as the model expects original_image /= 255. original_image -= 0.5 original_image *= 2. # Add a 4th dimension for batch size (as Keras expects) original_image = np.expand_dims(original_image, axis=0) # Pre-calculate the maximum change we will allow to the image # We'll make sure our hacked image never goes past this so it doesn't look funny. # A larger number produces an image faster but risks more distortion. max_change_above = original_image + 0.01 max_change_below = original_image - 0.01 # Create a copy of the input image to hack on hacked_image = np.copy(original_image) # How much to update the hacked image in each iteration learning_rate = 0.1 # Define the cost function. # Our 'cost' will be the likelihood out image is the target class according to the pre-trained model cost_function = model_output_layer[0, object_type_to_fake] # We'll ask Keras to calculate the gradient based on the input image and the currently predicted class # In this case, referring to "model_input_layer" will give us back image we are hacking. gradient_function = K.gradients(cost_function, model_input_layer)[0] # Create a Keras function that we can call to calculate the current cost and gradient grab_cost_and_gradients_from_model = K.function([model_input_layer, K.learning_phase()], [cost_function, gradient_function]) cost = 0.0 # In a loop, keep adjusting the hacked image slightly so that it tricks the model more and more # until it gets to at least 80% confidence while cost < 0.80: # Check how close the image is to our target class and grab the gradients we # can use to push it one more step in that direction. # Note: It's really important to pass in '0' for the Keras learning mode here! # Keras layers behave differently in prediction vs. train modes! cost, gradients = grab_cost_and_gradients_from_model([hacked_image, 0]) # Move the hacked image one step further towards fooling the model hacked_image += gradients * learning_rate # Ensure that the image doesn't ever change too much to either look funny or to become an invalid image hacked_image = np.clip(hacked_image, max_change_below, max_change_above) hacked_image = np.clip(hacked_image, -1.0, 1.0) print("Model's predicted likelihood that the image is a toaster: {:.8}%".format(cost * 100)) # De-scale the image's pixels from [-1, 1] back to the [0, 255] range img = hacked_image[0] img /= 2. img += 0.5 img *= 255. # Save the hacked image! im = Image.fromarray(img.astype(np.uint8)) im.save("hacked-image.png")运行后,它会吐出一个会欺骗神经网络的图像:
$ python3 generated_hacked_image.py Model's predicted likelihood that the image is a toaster: 0.00072% [ .... a few thousand lines of training .... ] Model's predicted likelihood that the image is a toaster: 99.4212%注意:如果你没有GPU,可能需要几个小时的时间。如果你有正确配置了Keras和CUDA的GPU,则运行时间不超过两分钟。
现在我们来测试一下这个黑客图像:
$ python3 predict.py This is a toaster with 98.09% confidence!我们成功了!我们已经欺骗神经网络让它认为这只猫是烤面包机!
黑客图像可以做什么?创建了一个黑客图像被称为“生成对抗实例”。我们有意制作一个数据,让机器学习模型分类错误。这是一个巧妙的花招,但为什么这在现实世界中很重要呢?
研究表明,这些黑客图像有一些令人惊讶的属性:
即使被打印在纸上黑客图像仍然可以欺骗神经网络!因此,你可以使用这些黑客图像来欺骗物理摄像机或扫描仪,而不仅仅是欺骗直接上传图像文件的系统。
欺骗了一个神经网络的图像往往也能欺骗设计完全不同的其他神经网络,如果他们接受了相似的数据训练。
所以我们可以用这些黑客图片做很多事情!