The Big Zeus Family Similarity Showdown

TL;DR: I took the data of Zeus-like families available in Malpedia and then used IDA Pro & BinDiff to calculate their similarities.
Last Update (2018-01-14): This is the first version. It will be potentially updated with more versions / families as they are inventorized.

Content:

1. Result Overview
2. Methodology
3. Additional Notes
4. Changelog

1) Result Overview

Generated on 2018-01-14.
Legend:

	>0.90 BinDiff similarity
	>0.80 BinDiff similarity
	>0.70 BinDiff similarity
	>0.60 BinDiff similarity
	>0.50 BinDiff similarity
	>0.40 BinDiff similarity
	>0.30 BinDiff similarity
	>0.20 BinDiff similarity
	>0.10 BinDiff similarity
	>=0.0 BinDiff similarity

2) Methodology

Having worked on several of these families over the years, retrospectively I was interested in how similar their code actually is.
I then thought, this might be a cool idea for an experiment / blog post demonstrating one of the use-cases for my current project Malpedia. :)
Next, I remembered that I had written a crude collection of scripts in 2012 that I called "BatchBinDiff" back then and gave it a shot to use them in this endevour.
So why not dust it off and do a pretty diagram?

My first step was to take all dumps for the above families from Malpedia and convert them to IDBs.
While there are certainly hundreds of thousands packed samples for the families, the above is a pretty good collection of single (unpacked!) samples representing some of the versions observed over time, which makes the comparison way more manageable.
However, since IDA Pro's function recognition performance on memory dumps vs clean files is not as good, I had to manually improve the code coverage. For the 52 files listed above, that took about 4 hours of pressing "C", "P", and occasionally "U". :D

With the IDBs at hand, I started toying around with "BatchBinDiff". I found a working set up in one of my VirtualBox VMs with IDA Pro 6.4 and BinDiff 4.0.
The first step was to create BinExport files for the IDBs. This can be achieved with a little idc script invoking the BinExporter IDA Plugin.
Using the freshly produced BinExport files, one can now use Zynamics' BinDiffDeluxe.exe in order to directly compare BinExport files on the command line.
Since we can only do direct comparisons, this means we have to execute BinDiffDeluxe n*(n-1)/2 = 1.326 times to calculate values for every pair once.
The runtime for this was about 35 minutes. Adding the BinExports on top, we look at a good 40 minutes.
It is quite obvious that this method does not scale well because of the quadratic complexity, but as BinDiff (IMHO) is still the best code-based binary similarity tool, it's at least a workaround to generate some ground-truth (about 900MB in text files with details about all matches).
And BinDiffDeluxe.exe is probably not widely known and didn't receive as much love in the past as it actually deserves. ;)

3) Additional Notes

Will I release the raw files/data? Maybe, I have to think about it and consider some factors regarding this.

Fow now, this is just a dump of raw results.
I may have a closer look into some of the versions seemingly being outliers and try to explain what's happening there.
For example, for VM Zeus 2.0.0.0+, I assume this is related to the introduced API call obfuscation scheme.

I am aware that there are more families and versions that could be included here.
The following are on my list for future updates of this experiment:

• Zeus Sphinx - has hard-linked OpenSSL with low detection by IDA, so I have to tag a couple thousand functions by hand
• ZLoader / Zeus Terdot - I simply have not enough coverage insight into this family
• PandaBanker - is this even based on Zeus? :D

If you have further ideas or comments, let me know!

4) Changelog

2018-01-14 Initial release: 52 versions of 10 zeus-like families compared.