使用Fuzzing进行漏洞挖掘的入门方法介绍（part2）
导语：在本节中，我们将着眼于改善part1的Fuzzer的性能。
0x01 基本介绍
在本节中，我们将着眼于改善part1的Fuzzer的性能。这意味着不会有任何大的变化，我们只是在寻求对上一篇文章中已有内容的改进。这意味着我们的fuzzer仍然会是一个非常基本的突变Fuzzer。在这篇文章中，我们不会真正修改多线程或多个处理函数，我们会将其保存下来，以备后继之用。
我觉得有必要在此添加一个免责声明，因为我不是专业开发人员。在这一点上，我只是对编程没有足够的经验，因此无法以经验丰富的程序员的方式提高性能的机会。我将使用我的原始技能和有限的编程知识来改进我们之前的Fuzzer。编写的代码将不会很漂亮，也不会完美，但是它将比我们之前的文章更好。所有测试都是在带有1个CPU和1个内核的x86 Kali VM的VMWare Workstation上完成的。
我在这里“更好”的意思是我们可以更快地遍历n个Fuzzing的迭代，我们将花一些时间完全重写Fuzzer，使用一种很酷的语言，并在以后使用更高级的Fuzzing技术。
0x02 分析之前的Fuzzer
让我们再看一下上一篇文章中的Fuzzer（为测试目的进行了一些小的更改）：
#!/usr/bin/env python3
import sys
import random
from pexpect import run
from pipes import quote
# read bytes from our valid JPEG and return them in a mutable bytearray
def get_bytes(filename):
f = open(filename, "rb").read()
return bytearray(f)
def bit_flip(data):
num_of_flips = int((len(data) - 4) * .01)
indexes = range(4, (len(data) - 4))
chosen_indexes = []
# iterate selecting indexes until we've hit our num_of_flips number
counter = 0
while counter < num_of_flips:
chosen_indexes.append(random.choice(indexes))
counter += 1
for x in chosen_indexes:
current = data[x]
current = (bin(current).replace("0b",""))
current = "0" * (8 - len(current)) + current
indexes = range(0,8)
picked_index = random.choice(indexes)
new_number = []
# our new_number list now has all the digits, example: ['1', '0', '1', '0', '1', '0', '1', '0']
for i in current:
new_number.append(i)
# if the number at our randomly selected index is a 1, make it a 0, and vice versa
if new_number[picked_index] == "1":
new_number[picked_index] = "0"
else:
new_number[picked_index] = "1"
# create our new binary string of our bit-flipped number
current = ''
for i in new_number:
current += i
# convert that string to an integer
current = int(current,2)
# change the number in our byte array to our new number we just constructed
data[x] = current
return data
def magic(data):
magic_vals = [
(1, 255),
(1, 255),
(1, 127),
(1, 0),
(2, 255),
(2, 0),
(4, 255),
(4, 0),
(4, 128),
(4, 64),
(4, 127)
]
picked_magic = random.choice(magic_vals)
length = len(data) - 8
index = range(0, length)
picked_index = random.choice(index)
# here we are hardcoding all the byte overwrites for all of the tuples that begin (1, )
if picked_magic[0] == 1:
if picked_magic[1] == 255: # 0xFF
data[picked_index] = 255
elif picked_magic[1] == 127: # 0x7F
data[picked_index] = 127
elif picked_magic[1] == 0: # 0x00
data[picked_index] = 0
# here we are hardcoding all the byte overwrites for all of the tuples that begin (2, )
elif picked_magic[0] == 2:
if picked_magic[1] == 255: # 0xFFFF
data[picked_index] = 255
data[picked_index + 1] = 255
elif picked_magic[1] == 0: # 0x0000
data[picked_index] = 0
data[picked_index + 1] = 0
# here we are hardcoding all of the byte overwrites for all of the tuples that being (4, )
elif picked_magic[0] == 4:
if picked_magic[1] == 255: # 0xFFFFFFFF
data[picked_index] = 255
data[picked_index + 1] = 255
data[picked_index + 2] = 255
data[picked_index + 3] = 255
elif picked_magic[1] == 0: # 0x00000000
data[picked_index] = 0
data[picked_index + 1] = 0
data[picked_index + 2] = 0
data[picked_index + 3] = 0
elif picked_magic[1] == 128: # 0x80000000
data[picked_index] = 128
data[picked_index + 1] = 0
data[picked_index + 2] = 0
data[picked_index + 3] = 0
elif picked_magic[1] == 64: # 0x40000000
data[picked_index] = 64
data[picked_index + 1] = 0
data[picked_index + 2] = 0
data[picked_index + 3] = 0
elif picked_magic[1] == 127: # 0x7FFFFFFF
data[picked_index] = 127
data[picked_index + 1] = 255
data[picked_index + 2] = 255
data[picked_index + 3] = 255
return data
# create new jpg with mutated data
def create_new(data):
f = open("mutated.jpg", "wb+")
f.write(data)
f.close()
def exif(counter,data):
command = "exif mutated.jpg -verbose"
out, returncode = run("sh -c " + quote(command), withexitstatus=1)
if b"Segmentation" in out:
f = open("crashes2/crash.{}.jpg".format(str(counter)), "ab+")
f.write(data)
print("Segfault!")
#if counter % 100 == 0:
# print(counter, end="\r")
if len(sys.argv) < 2:
print("Usage: JPEGfuzz.py ")
else:
filename = sys.argv[1]
counter = 0
while counter < 1000:
data = get_bytes(filename)
functions = [0, 1]
picked_function = random.choice(functions)
picked_function = 1
if picked_function == 0:
mutated = magic(data)
create_new(mutated)
exif(counter,mutated)
else:
mutated = bit_flip(data)
create_new(mutated)
exif(counter,mutated)
counter += 1你可能会注意到一些变化，做的一些修改：
· 每100次迭代就将迭代计数器的打印语句注释掉，
· 添加了打印语句可以将一些段错误信息通知给我们，
· 硬编码的1k迭代，
· 新增了这一行：picked_function = 1，以便我们消除测试中的随机性，并且仅使用了一种突变方法（bit_flip()）
让我们使用一些性能分析工具来运行此版本的Fuzzer，可以真正分析在程序执行中花费了多少时间。
我们可以利用cProfilePython模块，查看在1000次Fuzzing迭代中花费的时间。如果你还记得的话，该程序会将文件路径参数传递给有效的JPEG文件，因此完整的命令行语法为：python3 -m cProfile -s cumtime JPEGfuzzer.py ~/jpegs/Canon_40D.jpg。
还应注意，添加此cProfile工具可能会降低性能。我没有使用它进行测试，对于我们在本文中使用的迭代大小，它似乎并没有太大的不同。
运行此命令后，将看到程序输出，并可以看到执行期间花费最多时间的地址。
2476093 function calls (2474812 primitive calls) in 122.084 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
33/1 0.000 0.000 122.084 122.084 {built-in method builtins.exec}
1 0.108 0.108 122.084 122.084 blog.py:3()
1000 0.090 0.000 118.622 0.119 blog.py:140(exif)
1000 0.080 0.000 118.452 0.118 run.py:7(run)
5432 103.761 0.019 103.761 0.019 {built-in method time.sleep}
1000 0.028 0.000 100.923 0.101 pty_spawn.py:316(close)
1000 0.025 0.000 100.816 0.101 ptyprocess.py:387(close)
1000 0.061 0.000 9.949 0.010 pty_spawn.py:36(__init__)
1000 0.074 0.000 9.764 0.010 pty_spawn.py:239(_spawn)
1000 0.041 0.000 8.682 0.009 pty_spawn.py:312(_spawnpty)
1000 0.266 0.000 8.641 0.009 ptyprocess.py:178(spawn)
1000 0.011 0.000 7.491 0.007 spawnbase.py:240(expect)
1000 0.036 0.000 7.479 0.007 spawnbase.py:343(expect_list)
1000 0.128 0.000 7.409 0.007 expect.py:91(expect_loop)
6432 6.473 0.001 6.473 0.001 {built-in method posix.read}
5432 0.089 0.000 3.818 0.001 pty_spawn.py:415(read_nonblocking)
7348 0.029 0.000 3.162 0.000 utils.py:130(select_ignore_interrupts)
7348 3.127 0.000 3.127 0.000 {built-in method select.select}
1000 0.790 0.001 1.777 0.002 blog.py:15(bit_flip)
1000 0.015 0.000 1.311 0.001 blog.py:134(create_new)
1000 0.100 0.000 1.101 0.001 pty.py:79(fork)
1000 1.000 0.001 1.000 0.001 {built-in method posix.forkpty}
-----SNIP-----对于这种类型的分析，我们并不会在乎有多少段错误，因为并没有真正地对突变方法进行过多的修改或比较不同的方法。当然这里会有一些随机性，因为崩溃将需要额外的处理。
我仅截取了我们累计花费超过1.0秒的代码部分，你可以看到到目前为止在blog.py:140(exif)的时间最多，总共122秒中有118秒。
可以看到，在该函数下的大部分时间都与该函数直接相关，我们可以从pty和pexpect的使用中看到对该模块修改，通过从subprocess模块改写我们的Popen函数，看看我们是否可以在这里提高性能！
这是重新定义的exif()函数：
def exif(counter,data):
p = Popen(["exif", "mutated.jpg", "-verbose"], stdout=PIPE, stderr=PIPE)
(out,err) = p.communicate()
if p.returncode == -11:
f = open("crashes2/crash.{}.jpg".format(str(counter)), "ab+")
f.write(data)
print("Segfault!")
#if counter % 100 == 0:
# print(counter, end="\r")执行效果：
2065580 function calls (2065443 primitive calls) in 2.756 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
15/1 0.000 0.000 2.756 2.756 {built-in method builtins.exec}
1 0.038 0.038 2.756 2.756 subpro.py:3()
1000 0.020 0.000 1.917 0.002 subpro.py:139(exif)
1000 0.026 0.000 1.121 0.001 subprocess.py:681(__init__)
1000 0.099 0.000 1.045 0.001 subprocess.py:1412(_execute_child)
-----SNIP-----重新定义的exif()函数的该Fuzzer仅在2秒内完成了相同的工作量！旧的Fuzzer：122秒，新的Fuzzer：2.7秒。
0x03 进一步改进Python代码
尝试使用Python继续改进我们的Fuzzer。首先，我们会完成一个经过优化的Python Fuzzer，以实现50,000次Fuzzing迭代，并将cProfile再次使用该模块来获取有关花费时间的一些细粒度统计信息。
102981395 function calls (102981258 primitive calls) in 141.488 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
15/1 0.000 0.000 141.488 141.488 {built-in method builtins.exec}
1 1.724 1.724 141.488 141.488 subpro.py:3()
50000 0.992 0.000 102.588 0.002 subpro.py:139(exif)
50000 1.248 0.000 61.562 0.001 subprocess.py:681(__init__)
50000 5.034 0.000 57.826 0.001 subprocess.py:1412(_execute_child)
50000 0.437 0.000 39.586 0.001 subprocess.py:920(communicate)
50000 2.527 0.000 39.064 0.001 subprocess.py:1662(_communicate)
208254 37.508 0.000 37.508 0.000 {built-in method posix.read}
158238 0.577 0.000 28.809 0.000 selectors.py:402(select)
158238 28.131 0.000 28.131 0.000 {method 'poll' of 'select.poll' objects}
50000 11.784 0.000 25.819 0.001 subpro.py:14(bit_flip)
7950000 3.666 0.000 10.431 0.000 random.py:256(choice)
50000 8.421 0.000 8.421 0.000 {built-in method _posixsubprocess.fork_exec}
50000 0.162 0.000 7.358 0.000 subpro.py:133(create_new)
7950000 4.096 0.000 6.130 0.000 random.py:224(_randbelow)
203090 5.016 0.000 5.016 0.000 {built-in method io.open}
50000 4.211 0.000 4.211 0.000 {method 'close' of '_io.BufferedRandom' objects}
50000 1.643 0.000 4.194 0.000 os.py:617(get_exec_path)
50000 1.733 0.000 3.356 0.000 subpro.py:8(get_bytes)
35866791 2.635 0.000 2.635 0.000 {method 'append' of 'list' objects}
100000 0.070 0.000 1.960 0.000 subprocess.py:1014(wait)
100000 0.252 0.000 1.902 0.000 selectors.py:351(register)
100000 0.444 0.000 1.890 0.000 subprocess.py:1621(_wait)
100000 0.675 0.000 1.583 0.000 selectors.py:234(register)
350000 0.432 0.000 1.501 0.000 subprocess.py:1471()
12074141 1.434 0.000 1.434 0.000 {method 'getrandbits' of '_random.Random' objects}
50000 0.059 0.000 1.358 0.000 subprocess.py:1608(_try_wait)
50000 1.299 0.000 1.299 0.000 {built-in method posix.waitpid}
100000 0.488 0.000 1.058 0.000 os.py:674(__getitem__)
100000 1.017 0.000 1.017 0.000 {method 'close' of '_io.BufferedReader' objects}
-----SNIP-----50,000次迭代总共花费了141秒的时间，与之前的处理的相比，这是很棒的性能。之前，我们花了122秒来进行1,000次迭代！再次仅过滤了花费超过1.0秒的时间，我们看到exif()再次花费了大部分时间，但是由于bit_flip()花费了25秒钟的累积时间，我们也看到了一些性能问题，让我们尝试对该函数进行一些优化。
继续分析一下bit_flip()函数：
def bit_flip(data):
num_of_flips = int((len(data) - 4) * .01)
indexes = range(4, (len(data) - 4))
chosen_indexes = []
# iterate selecting indexes until we've hit our num_of_flips number
counter = 0
while counter < num_of_flips:
chosen_indexes.append(random.choice(indexes))
counter += 1
for x in chosen_indexes:
current = data[x]
current = (bin(current).replace("0b",""))
current = "0" * (8 - len(current)) + current
indexes = range(0,8)
picked_index = random.choice(indexes)
new_number = []
# our new_number list now has all the digits, example: ['1', '0', '1', '0', '1', '0', '1', '0']
for i in current:
new_number.append(i)
# if the number at our randomly selected index is a 1, make it a 0, and vice versa
if new_number[picked_index] == "1":
new_number[picked_index] = "0"
else:
new_number[picked_index] = "1"
# create our new binary string of our bit-flipped number
current = ''
for i in new_number:
current += i
# convert that string to an integer
current = int(current,2)
# change the number in our byte array to our new number we just constructed
data[x] = current
return data该函数有点臃肿，通过使用更好的逻辑，可以大大简化它。我发现，在我有限的经验中，编程经常是这种情况，你可以拥有所需的所有奇特的深奥编程知识，但是如果程序背后的逻辑不健全，则程序的性能将受到影响。
让我们减少执行类型转换的次数，例如，将int转换为str，反之亦然，然后减少编辑器中的代码。我们可以使用重新定义的bit_flip()函数来完成所需的操作，如下所示：
def bit_flip(data): length = len(data) - 4 num_of_flips = int(length * .01) picked_indexes = [] flip_array = [1,2,4,8,16,32,64,128] counter = 0 while counter < num_of_flips: picked_indexes.append(random.choice(range(0,length))) counter += 1 for x in picked_indexes: mask = random.choice(flip_array) data[x] = data[x] ^ mask return data
如果使用此新函数并监视结果，则将获得以下性能列表：
59376275 function calls (59376138 primitive calls) in 135.582 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
15/1 0.000 0.000 135.582 135.582 {built-in method builtins.exec}
1 1.940 1.940 135.582 135.582 subpro.py:3()
50000 0.978 0.000 107.857 0.002 subpro.py:111(exif)
50000 1.450 0.000 64.236 0.001 subprocess.py:681(__init__)
50000 5.566 0.000 60.141 0.001 subprocess.py:1412(_execute_child)
50000 0.534 0.000 42.259 0.001 subprocess.py:920(communicate)
50000 2.827 0.000 41.637 0.001 subprocess.py:1662(_communicate)
199549 38.249 0.000 38.249 0.000 {built-in method posix.read}
149537 0.555 0.000 30.376 0.000 selectors.py:402(select)
149537 29.722 0.000 29.722 0.000 {method 'poll' of 'select.poll' objects}
50000 3.993 0.000 14.471 0.000 subpro.py:14(bit_flip)
7950000 3.741 0.000 10.316 0.000 random.py:256(choice)
50000 9.973 0.000 9.973 0.000 {built-in method _posixsubprocess.fork_exec}
50000 0.163 0.000 7.034 0.000 subpro.py:105(create_new)
7950000 3.987 0.000 5.952 0.000 random.py:224(_randbelow)
202567 4.966 0.000 4.966 0.000 {built-in method io.open}
50000 4.042 0.000 4.042 0.000 {method 'close' of '_io.BufferedRandom' objects}
50000 1.539 0.000 3.828 0.000 os.py:617(get_exec_path)
50000 1.843 0.000 3.607 0.000 subpro.py:8(get_bytes)
100000 0.074 0.000 2.133 0.000 subprocess.py:1014(wait)
100000 0.463 0.000 2.059 0.000 subprocess.py:1621(_wait)
100000 0.274 0.000 2.046 0.000 selectors.py:351(register)
100000 0.782 0.000 1.702 0.000 selectors.py:234(register)
50000 0.055 0.000 1.507 0.000 subprocess.py:1608(_try_wait)
50000 1.452 0.000 1.452 0.000 {built-in method posix.waitpid}
350000 0.424 0.000 1.436 0.000 subprocess.py:1471()
12066317 1.339 0.000 1.339 0.000 {method 'getrandbits' of '_random.Random' objects}
100000 0.466 0.000 1.048 0.000 os.py:674(__getitem__)
100000 1.014 0.000 1.014 0.000 {method 'close' of '_io.BufferedReader' objects}
-----SNIP-----从指标中可以看到，此时我们仅在bit_flip()花费了14秒钟的累积时间！在上一个回合中，我们在这里花了25秒，这几乎是现在的两倍。在我看来，我们在优化方面做得很好。
既然已经有了理想的Python测试，让我们继续将Fuzzer移植到新语言C ++并测试性能。
0x04 C ++中的新Fuzzer
首先，我们继续测试优化的python Fuzzing程序，进行100,000次Fuzzing迭代，看看总共花费多长时间。
118749892 function calls (118749755 primitive calls) in 256.881 seconds
仅256秒即可进行10万次迭代！这个数据打破了我们以前的Fuzzer。
我会逐个函数地进行介绍，并描述它们的实现。
//
// this function simply creates a stream by opening a file in binary mode;
// finds the end of file, creates a string 'data', resizes data to be the same
// size as the file moves the file pointer back to the beginning of the file;
// reads the data from the into the data string;
//
std::string get_bytes(std::string filename)
{
std::ifstream fin(filename, std::ios::binary);
if (fin.is_open())
{
fin.seekg(0, std::ios::end);
std::string data;
data.resize(fin.tellg());
fin.seekg(0, std::ios::beg);
fin.read(&data[0], data.size());
return data;
}
else
{
std::cout << "Failed to open " << filename << ".\n";
exit(1);
}
}此函数只是从目标文件中检索一个字节字符串，在我们的测试情况下，该字符串仍为Canon_40D.jpg。
//
// this will take 1% of the bytes from our valid jpeg and
// flip a random bit in the byte and return the altered string
//
std::string bit_flip(std::string data)
{
int size = (data.length() - 4);
int num_of_flips = (int)(size * .01);
// get a vector full of 1% of random byte indexes
std::vector picked_indexes;
for (int i = 0; i < num_of_flips; i++)
{
int picked_index = rand() % size;
picked_indexes.push_back(picked_index);
}
// iterate through the data string at those indexes and flip a bit
for (int i = 0; i < picked_indexes.size(); ++i)
{
int index = picked_indexes[i];
char current = data.at(index);
int decimal = ((int)current & 0xff);
int bit_to_flip = rand() % 8;
decimal ^= 1 << bit_to_flip;
decimal &= 0xff;
data[index] = (char)decimal;
}
return data;
}此函数与Python脚本中的bit_flip()函数直接等效。
//
// takes mutated string and creates new jpeg with it;
//
void create_new(std::string mutated)
{
std::ofstream fout("mutated.jpg", std::ios::binary);
if (fout.is_open())
{
fout.seekp(0, std::ios::beg);
fout.write(&mutated[0], mutated.size());
}
else
{
std::cout << "Failed to create mutated.jpg" << ".\n";
exit(1);
}
}该函数将简单地创建一个临时mutated.jpg文件，类似于我们在Python脚本中的create_new()函数。
//
// function to run a system command and store the output as a string;
// https://www.jeremymorgan.com/tutorials/c-programming/how-to-capture-the-output-of-a-linux-command-in-c/
//
std::string get_output(std::string cmd)
{
std::string output;
FILE * stream;
char buffer[256];
stream = popen(cmd.c_str(), "r");
if (stream)
{
while (!feof(stream))
if (fgets(buffer, 256, stream) != NULL) output.append(buffer);
pclose(stream);
}
return output;
}
//
// we actually run our exiv2 command via the get_output() func;
// retrieve the output in the form of a string and then we can parse the string;
// we'll save all the outputs that result in a segfault or floating point except;
//
void exif(std::string mutated, int counter)
{
std::string command = "exif mutated.jpg -verbose 2>&1";
std::string output = get_output(command);
std::string segfault = "Segmentation";
std::string floating_point = "Floating";
std::size_t pos1 = output.find(segfault);
std::size_t pos2 = output.find(floating_point);
if (pos1 != -1)
{
std::cout << "Segfault!\n";
std::ostringstream oss;
oss << "/root/cppcrashes/crash." << counter << ".jpg";
std::string filename = oss.str();
std::ofstream fout(filename, std::ios::binary);
if (fout.is_open())
{
fout.seekp(0, std::ios::beg);
fout.write(&mutated[0], mutated.size());
}
else
{
std::cout << "Failed to create " << filename << ".jpg" << ".\n";
exit(1);
}
}
else if (pos2 != -1)
{
std::cout << "Floating Point!\n";
std::ostringstream oss;
oss << "/root/cppcrashes/crash." << counter << ".jpg";
std::string filename = oss.str();
std::ofstream fout(filename, std::ios::binary);
if (fout.is_open())
{
fout.seekp(0, std::ios::beg);
fout.write(&mutated[0], mutated.size());
}
else
{
std::cout << "Failed to create " << filename << ".jpg" << ".\n";
exit(1);
}
}
}get_output将C ++字符串作为参数，并将在操作系统上运行该命令并捕获输出。然后，该函数将输出作为字符串返回给调用函数exif()。
exif()将获取输出并查找Segmentation fault或Floating point exception错误，然后如果发现错误，则将这些字节写入文件并将其保存为crash.
//
// simply generates a vector of strings that are our 'magic' values;
//
std::vector vector_gen()
{
std::vector magic;
using namespace std::string_literals;
magic.push_back("\xff");
magic.push_back("\x7f");
magic.push_back("\x00"s);
magic.push_back("\xff\xff");
magic.push_back("\x7f\xff");
magic.push_back("\x00\x00"s);
magic.push_back("\xff\xff\xff\xff");
magic.push_back("\x80\x00\x00\x00"s);
magic.push_back("\x40\x00\x00\x00"s);
magic.push_back("\x7f\xff\xff\xff");
return magic;
}
//
// randomly picks a magic value from the vector and overwrites that many bytes in the image;
//
std::string magic(std::string data, std::vector magic)
{
int vector_size = magic.size();
int picked_magic_index = rand() % vector_size;
std::string picked_magic = magic[picked_magic_index];
int size = (data.length() - 4);
int picked_data_index = rand() % size;
data.replace(picked_data_index, magic[picked_magic_index].length(), magic[picked_magic_index]);
return data;
}
//
// returns 0 or 1;
//
int func_pick()
{
int result = rand() % 2;
return result;
}这些函数也非常类似于我们的Python实现。vector_gen()只是创建了“magic value”向量，然后magic()使用该向量的后续函数会随机选择一个索引，然后将有效jpeg中的数据替换为变异数据。
func_pick()非常简单，只返回a 0或a1即可，使我们的Fuzzer可以随机bit_flip()或magic()变异有效的jpeg。为了保持一致，让我们的Fuzzer暂时只选择一个bit_flip()，方法function = 1是在程序中添加一条临时行，以便与Python测试匹配。
这是到目前为止执行所有代码的main()函数：
int main(int argc, char** argv)
{
if (argc < 3)
{
std::cout << "Usage: ./cppfuzz \n";
std::cout << "Usage: ./cppfuzz Canon_40D.jpg 10000\n";
return 1;
}
// start timer
auto start = std::chrono::high_resolution_clock::now();
// initialize our random seed
srand((unsigned)time(NULL));
// generate our vector of magic numbers
std::vector magic_vector = vector_gen();
std::string filename = argv[1];
int iterations = atoi(argv[2]);
int counter = 0;
while (counter < iterations)
{
std::string data = get_bytes(filename);
int function = func_pick();
function = 1;
if (function == 0)
{
// utilize the magic mutation method; create new jpg; send to exiv2
std::string mutated = magic(data, magic_vector);
create_new(mutated);
exif(mutated,counter);
counter++;
}
else
{
// utilize the bit flip mutation; create new jpg; send to exiv2
std::string mutated = bit_flip(data);
create_new(mutated);
exif(mutated,counter);
counter++;
}
}
// stop timer and print execution time
auto stop = std::chrono::high_resolution_clock::now();
auto duration = std::chrono::duration_cast(stop - start);
std::cout << "Execution Time: " << duration.count() << "ms\n";
return 0;
}我们从命令行参数获取有效的JPEG进行变异和许多Fuzzing迭代。然后，在std::chrono名称空间中建立了一些计时机制，以计时程序执行所需的时间。
我们在这里只是通过选择bit_flip()类型突变来作弊，但这也是我们在Python中所做的，因此希望进行“ Apples to Apples”比较。
继续进行100,000次迭代，并将其与256秒的Python Fuzzing进行比较。
运行C ++Fuzzer，我们将获得以毫秒为Execution Time: 172638ms单位的打印时间。
因此，使用新的C ++ Fuzzer轻松地超越了之前的Python Fuzzer！继续在这里做一些数学运算：172/256 = 67％。因此，使用C ++实现大约要快33。
让我们以优化的Python和C ++ Fuzzing工具为Fuzzer，并选择一个Fuzzing目标！
0x05 选择一个 Fuzzing 目标
我们的操作环境是Kali Linux，看一下/usr/bin/exiv2中的内容。
root@kali:~# exiv2 -h
Usage: exiv2 [ options ] [ action ] file ...
Manipulate the Exif metadata of images.
Actions:
ad | adjust Adjust Exif timestamps by the given time. This action
requires at least one of the -a, -Y, -O or -D options.
pr | print Print image metadata.
rm | delete Delete image metadata from the files.
in | insert Insert metadata from corresponding *.exv files.
Use option -S to change the suffix of the input files.
ex | extract Extract metadata to *.exv, *.xmp and thumbnail image files.
mv | rename Rename files and/or set file timestamps according to the
Exif create timestamp. The filename format can be set with
-r format, timestamp options are controlled with -t and -T.
mo | modify Apply commands to modify (add, set, delete) the Exif and
IPTC metadata of image files or set the JPEG comment.
Requires option -c, -m or -M.
fi | fixiso Copy ISO setting from the Nikon Makernote to the regular
Exif tag.
fc | fixcom Convert the UNICODE Exif user comment to UCS-2. Its current
character encoding can be specified with the -n option.
Options:
-h Display this help and exit.
-V Show the program version and exit.
-v Be verbose during the program run.
-q Silence warnings and error messages during the program run (quiet).
-Q lvl Set log-level to d(ebug), i(nfo), w(arning), e(rror) or m(ute).
-b Show large binary values.
-u Show unknown tags.
-g key Only output info for this key (grep).
-K key Only output info for this key (exact match).
-n enc Charset to use to decode UNICODE Exif user comments.
-k Preserve file timestamps (keep).
-t Also set the file timestamp in 'rename' action (overrides -k).
-T Only set the file timestamp in 'rename' action, do not rename
the file (overrides -k).
-f Do not prompt before overwriting existing files (force).
-F Do not prompt before renaming files (Force).
-a time Time adjustment in the format [-]HH[:MM[:SS]]. This option
is only used with the 'adjust' action.
-Y yrs Year adjustment with the 'adjust' action.
-O mon Month adjustment with the 'adjust' action.
-D day Day adjustment with the 'adjust' action.
-p mode Print mode for the 'print' action. Possible modes are:
s : print a summary of the Exif metadata (the default)
a : print Exif, IPTC and XMP metadata (shortcut for -Pkyct)
t : interpreted (translated) Exif data (-PEkyct)
v : plain Exif data values (-PExgnycv)
h : hexdump of the Exif data (-PExgnycsh)
i : IPTC data values (-PIkyct)
x : XMP properties (-PXkyct)
c : JPEG comment
p : list available previews
S : print structure of image
X : extract XMP from image
-P flgs Print flags for fine control of tag lists ('print' action):
E : include Exif tags in the list
I : IPTC datasets
X : XMP properties
x : print a column with the tag number
g : group name
k : key
l : tag label
n : tag name
y : type
c : number of components (count)
s : size in bytes
v : plain data value
t : interpreted (translated) data
h : hexdump of the data
-d tgt Delete target(s) for the 'delete' action. Possible targets are:
a : all supported metadata (the default)
e : Exif section
t : Exif thumbnail only
i : IPTC data
x : XMP packet
c : JPEG comment
-i tgt Insert target(s) for the 'insert' action. Possible targets are
the same as those for the -d option, plus a modifier:
X : Insert metadata from an XMP sidecar file .xmp
Only JPEG thumbnails can be inserted, they need to be named
-thumb.jpg
-e tgt Extract target(s) for the 'extract' action. Possible targets
are the same as those for the -d option, plus a target to extract
preview images and a modifier to generate an XMP sidecar file:
p[[, ...]] : Extract preview images.
X : Extract metadata to an XMP sidecar file .xmp
-r fmt Filename format for the 'rename' action. The format string
follows strftime(3). The following keywords are supported:
:basename: - original filename without extension
:dirname: - name of the directory holding the original file
:parentname: - name of parent directory
Default filename format is %Y%m%d_%H%M%S.
-c txt JPEG comment string to set in the image.
-m file Command file for the modify action. The format for commands is
set|add|del [[] ].
-M cmd Command line for the modify action. The format for the
commands is the same as that of the lines of a command file.
-l dir Location (directory) for files to be inserted from or extracted to.
-S .suf Use suffix .suf for source files for insert command.现在我们Fuzzer中的命令字符串将类似于exiv2 pr -v mutated.jpg。
继续进行更新，以了解是否可以在更加复杂的目标上找到更多漏洞。值得一提的是，目前已经支持此目标，而不是像我们上一个目标（Github上一个不受支持的7年历史项目）上发现漏洞的简单二进制文件。
这个目标已经被更高级的Fuzzer所Fuzzing，你可以简单地通过谷歌搜索“ ASan exiv2”之类的东西，并获得大量的Fuzzer信息，在二进制文件中创建段错误，并将ASan输出作为错误转发到github存储库。这是我们最后一个目标的重要一步。
在Github上的exiv2](https://github.com/Exiv2/exiv2) [exiv2网站](https://www.exiv2.org/)
0x06 Fuzzing新目标
让我们从经过改进的Python Fuzzer开始，并监视其50,000次迭代的性能。除了分段错误检测外，添加一些监视浮点异常的代码。我们的新exif()函数如下所示：
def exif(counter,data):
p = Popen(["exiv2", "pr", "-v", "mutated.jpg"], stdout=PIPE, stderr=PIPE)
(out,err) = p.communicate()
if p.returncode == -11:
f = open("crashes2/crash.{}.jpg".format(str(counter)), "ab+")
f.write(data)
print("Segfault!")
elif p.returncode == -8:
f = open("crashes2/crash.{}.jpg".format(str(counter)), "ab+")
f.write(data)
print("Floating Point!")查看来自python3 -m cProfile -s cumtime subpro.py ~/jpegs/Canon_40D.jpg的输出：
75780446 function calls (75780309 primitive calls) in 213.595 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
15/1 0.000 0.000 213.595 213.595 {built-in method builtins.exec}
1 1.481 1.481 213.595 213.595 subpro.py:3()
50000 0.818 0.000 187.205 0.004 subpro.py:111(exif)
50000 0.543 0.000 143.499 0.003 subprocess.py:920(communicate)
50000 6.773 0.000 142.873 0.003 subprocess.py:1662(_communicate)
1641352 3.186 0.000 122.668 0.000 selectors.py:402(select)
1641352 118.799 0.000 118.799 0.000 {method 'poll' of 'select.poll' objects}
50000 1.220 0.000 42.888 0.001 subprocess.py:681(__init__)
50000 4.400 0.000 39.364 0.001 subprocess.py:1412(_execute_child)
1691919 25.759 0.000 25.759 0.000 {built-in method posix.read}
50000 3.863 0.000 13.938 0.000 subpro.py:14(bit_flip)
7950000 3.587 0.000 9.991 0.000 random.py:256(choice)
50000 7.495 0.000 7.495 0.000 {built-in method _posixsubprocess.fork_exec}
50000 0.148 0.000 7.081 0.000 subpro.py:105(create_new)
7950000 3.884 0.000 5.764 0.000 random.py:224(_randbelow)
200000 4.582 0.000 4.582 0.000 {built-in method io.open}
50000 4.192 0.000 4.192 0.000 {method 'close' of '_io.BufferedRandom' objects}
50000 1.339 0.000 3.612 0.000 os.py:617(get_exec_path)
50000 1.641 0.000 3.309 0.000 subpro.py:8(get_bytes)
100000 0.077 0.000 1.822 0.000 subprocess.py:1014(wait)
100000 0.432 0.000 1.746 0.000 subprocess.py:1621(_wait)
100000 0.256 0.000 1.735 0.000 selectors.py:351(register)
100000 0.619 0.000 1.422 0.000 selectors.py:234(register)
350000 0.380 0.000 1.402 0.000 subprocess.py:1471()
12066004 1.335 0.000 1.335 0.000 {method 'getrandbits' of '_random.Random' objects}
50000 0.063 0.000 1.222 0.000 subprocess.py:1608(_try_wait)
50000 1.160 0.000 1.160 0.000 {built-in method posix.waitpid}
100000 0.519 0.000 1.143 0.000 os.py:674(__getitem__)
1691352 0.902 0.000 1.097 0.000 selectors.py:66(__len__)
7234121 1.023 0.000 1.023 0.000 {method 'append' of 'list' objects}
-----SNIP-----我们总共花费了213秒，却没有发现任何漏洞，我们在完全相同的情况下运行C ++ Fuzzer并监视输出。
在这里，我们得到了类似的时间，但是有了很多改进：
root@kali:~# ./blogcpp ~/jpegs/Canon_40D.jpg 50000 Execution Time: 170829ms
这是一个相当重要的改进，时间为43秒，这比我们的Python时间节省了20％。
让C ++ Fuzzer运行一会儿，看看是否发现任何错误:)。
0x07 新目标上的漏洞
在可能再次运行Fuzzer10秒钟之后，我得到了以下终端输出：
root@kali:~# ./blogcpp ~/jpegs/Canon_40D.jpg 1000000 Floating Point!
看来我们已经满足了浮点例外的要求，cppcrashes目录中应该有一个jpg。
root@kali:~/cppcrashes# ls crash.522.jpg
通过运行exiv2以下示例来确认漏洞：
root@kali:~/cppcrashes# exiv2 pr -v crash.522.jpg File 1/1: crash.522.jpg Error: Offset of directory Image, entry 0x011b is out of bounds: Offset = 0x080000ae; truncating the entry Warning: Directory Image, entry 0x8825 has unknown Exif (TIFF) type 68; setting type size 1. Warning: Directory Image, entry 0x8825 doesn't look like a sub-IFD. File name : crash.522.jpg File size : 7958 Bytes MIME type : image/jpeg Image size : 100 x 68 Camera make : Aanon Camera model : Canon EOS 40D Image timestamp : 2008:05:30 15:56:01 Image number : Exposure time : 1/160 s Aperture : F7.1 Floating point exception
我们确实发现了一个新漏洞！非常令人兴奋。我们应该在exiv2的Github 上向开发人员报告此错误。
0x08 研究总结
我们首先用Python优化了Fuzzer，然后用C ++重写了它。我们获得了巨大的性能优势，甚至在一个更难的目标上发现了一些新的漏洞。
为了获得乐趣，让我们比较一下原始的Fuzzer在50,000次迭代中的性能：
123052109 function calls (123001828 primitive calls) in 6243.939 seconds
如你所见，6243秒比我们的C ++Fuzzing基准170秒要慢得多。
0x09 附录
只是将C ++ Fuzzer移植到C上，我自己做了一些适度的改进。我所做的逻辑更改之一是，仅从原始有效映像中收集一次数据，然后在每次Fuzzing迭代时将数据复制到新分配的缓冲区中，然后对新分配的缓冲区执行变异操作。与C ++ Fuzzer相比，该C ++ Fuzzer的C版本表现良好。这是两者进行200,000迭代的比较（你可以忽略崩溃结果，因为此Fuzzer非常笨拙且随机性为100％）：
h0mbre:~$ time ./cppfuzz Canon_40D.jpg 200000 real 10m45.371s user 7m14.561s sys 3m10.529s h0mbre:~$ time ./cfuzz Canon_40D.jpg 200000 real 10m7.686s user 7m27.503s sys 2m20.843s
因此，经过200,000迭代，我们最终节省了大约35-40秒，这在我的测试中非常典型。因此，仅需进行少量逻辑更改并使用较少的C ++提供的抽象，就可以节省大量sys时间，我们将速度提高了大约5％。
监视子进程
完成C重写后，我去Twitter寻求有关性能改进的建议。AFL的开发者@lcamtuf向我解释说，我不应该在我的代码中使用popen()，因为它生成了一个shell并执行了abysmally。这是我寻求帮助的代码段：
void exif(int iteration) {
FILE *fileptr;
//fileptr = popen("exif_bin target.jpeg -verbose >/dev/null 2>&1", "r");
fileptr = popen("exiv2 pr -v mutated.jpeg >/dev/null 2>&1", "r");
int status = WEXITSTATUS(pclose(fileptr));
switch(status) {
case 253:
break;
case 0:
break;
case 1:
break;
default:
crashes++;
printf("\r[>] Crashes: %d", crashes);
fflush(stdout);
char command[50];
sprintf(command, "cp mutated.jpeg ccrashes/crash.%d.%d",
iteration,status);
system(command);
break;
}
}如你所见，我们使用popen()，运行shell命令，然后关闭指向子进程的文件指针，并返回退出状态以使用WEXITSTATUS宏进行监视。我正在过滤一些我不关心的退出代码，例如253，0和1，并希望看到一些与我们的C ++ Fuzzer甚至是段错误已经发现的浮点错误有关的代码。@lcamtuf建议不要使用popen()调用fork()来生成一个子进程，execvp()让该子进程执行一个命令，然后最终使用waitpid()来等待子进程终止并返回退出状态。
由于在此系统调用路径中没有适当的shell程序，因此我还必须打开一个/dev/null句柄并调用dup2()。我还使用WTERMSIG宏来检索在WIFSIGNALED宏返回true时终止子进程的信号，这表明我们遇到了段错误或浮点异常等。因此，现在更新函数如下所示：
void exif(int iteration) {
char* file = "exiv2";
char* argv[4];
argv[0] = "pr";
argv[1] = "-v";
argv[2] = "mutated.jpeg";
argv[3] = NULL;
pid_t child_pid;
int child_status;
child_pid = fork();
if (child_pid == 0) {
// this means we're the child process
int fd = open("/dev/null", O_WRONLY);
// dup both stdout and stderr and send them to /dev/null
dup2(fd, 1);
dup2(fd, 2);
close(fd);
execvp(file, argv);
// shouldn't return, if it does, we have an error with the command
printf("[!] Unknown command for execvp, exiting...\n");
exit(1);
}
else {
// this is run by the parent process
do {
pid_t tpid = waitpid(child_pid, &child_status, WUNTRACED |
WCONTINUED);
if (tpid == -1) {
printf("[!] Waitpid failed!\n");
perror("waitpid");
}
if (WIFEXITED(child_status)) {
//printf("WIFEXITED: Exit Status: %d\n", WEXITSTATUS(child_status));
} else if (WIFSIGNALED(child_status)) {
crashes++;
int exit_status = WTERMSIG(child_status);
printf("\r[>] Crashes: %d", crashes);
fflush(stdout);
char command[50];
sprintf(command, "cp mutated.jpeg ccrashes/%d.%d", iteration,
exit_status);
system(command);
} else if (WIFSTOPPED(child_status)) {
printf("WIFSTOPPED: Exit Status: %d\n", WSTOPSIG(child_status));
} else if (WIFCONTINUED(child_status)) {
printf("WIFCONTINUED: Exit Status: Continued.\n");
}
} while (!WIFEXITED(child_status) && !WIFSIGNALED(child_status));
}
}你可以看到，这大大提高了200,000迭代基准测试的性能：
h0mbre:~$ time ./cfuzz2 Canon_40D.jpg 200000 real 8m30.371s user 6m10.219s sys 2m2.098s
结果汇总
· C ++ Fuzzer – 310次迭代/秒
· C Fuzzer – 329次/秒（+ 6％）
· C Fuzzer 2.0 – 392次迭代/秒（+ 26％）
我在这里上传了完整代码：https://github.com/h0mbre/Fuzzing/tree/master/JPEGMutation
本文翻译自：https://h0mbre.github.io/Fuzzing-Like-a-Caveman-2/#如若转载，请注明原文地址：
发表评论
提供云计算服务