The Big Zeus Family Similarity Rundown v2

TL;DR: I took the data of Zeus-like families available in Malpedia and then used IDA Pro & BinDiff to calculate their similarities.
Last Update (2018-05-22): This is the 2nd version. I improved the ordering to group directly connected subfamilies while trying to maintain temporal order, it now includes 99 versions of 14 families.

Content:

Result Overview
Methodology
Additional Notes
Credits
Changelog

1) Result Overview

Zeus 2.0.8.9 (20110522)

Murofet 0.0.0.27 (20120120)

Murofet 0.0.0.0 (20121112)

Murofet 0.0.0.0 (20130408)

GameoverP2P (20130904)

GameoverDGA (20140709)

GameoverDGA (20140904)

GameoverDGA (20141003)

Ice IX 1.0.5.0 (20110814)

Ice IX 1.2.6.0 (20120120)

Citadel 1.3.0.0 (20120220)

Citadel 1.3.3.1 (20120315)

Citadel 1.3.4.0 (20120430)

Citadel 1.3.4.5 (20120615)

Citadel 1.3.5.1 (20121108)

Citadel 1.1.0.0 (20130815)

Citadel 1.2.0.0 (20131010)

Citadel 1.3.4.0 (20140123)

Citadel (Atmos) 0.0.1.0 (20141218)

Citadel (Atmos) 0.0.1.1 (20150622)

KINS 3.2.3.1 (20130703)

KINS 3.2.5.2 (20130811)

KINS 3.2.8.1 (20130908)

KINS 3.2.9.1 (20130927)

KINS 3.3.1.0 (20131112)

KINS 3.3.2.0 (20131118)

KINS 3.3.6.0 (20131214)

KINS 3.3.6.1 (20140227)

KINS 3.3.7.0 (20140306)

KINS 4.6.9.0 (20140529)

VM Zeus 2.0.9.14 (20111121)

VM Zeus 2.0.9.15 (20120225)

VM Zeus 2.0.9.15 (20141115)

VM Zeus 2.0.9.15 (20141216)

VM Zeus 2.0.9.13 (20130211)

VM Zeus 1.0.0.1 (20130314)

VM Zeus 1.0.0.1 (20130515)

VM Zeus 1.0.0.2 (20131101)

VM Zeus 1.0.0.2 (20131124)

VM Zeus 1.0.0.4 (20140125)

VM Zeus 1.0.0.5 (20140222)

VM Zeus 1.0.0.5 (20140303)

VM Zeus 1.0.0.5 (20140316)

VM Zeus 1.0.1.0 (20140512)

VM Zeus 1.0.1.0 (20140515)

VM Zeus 2.0.0.0 (20140609)

VM Zeus 2.0.0.0 (20140921)

VM Zeus 2.0.c.0 (20150523)

VM Zeus 2.0.e.0 (20150701)

VM Zeus 3.1.0.0 (20150807)

VM Zeus 3.2.0.0 (20151029)

VM Zeus 3.3.0.0 (20151107)

PandaBanker 2.1.0 (20160124)

PandaBanker 2.1.1 (20160208)

PandaBanker 2.1.2 (20160302)

PandaBanker 2.1.3 (20160307)

PandaBanker 2.2.2 (20160429)

PandaBanker 2.2.3 (20160607)

PandaBanker 2.2.4 (20160621)

PandaBanker 2.2.5 (20160628)

PandaBanker 2.2.6 (20160824)

PandaBanker 2.2.7 (20160910)

PandaBanker 2.2.8 (20160921)

PandaBanker 2.2.1.0 (20161011)

PandaBanker 2.2.9 (20161025)

PandaBanker 2.2.1.1 (20161209)

PandaBanker 2.2.1.3 (20170125)

PandaBanker 2.2.1.4 (20170215)

PandaBanker 2.3.2 (20170315)

PandaBanker 2.3.4 (20170504)

PandaBanker 2.4.1 (20170606)

PandaBanker 2.4.2 (20170616)

PandaBanker 2.5.0 (20170829)

PandaBanker 2.5.1 (20170830)

PandaBanker 2.5.2 (20170907)

PandaBanker 2.5.3 (20170819)

PandaBanker 2.5.4 (20170922)

PandaBanker 2.5.6 (20171016)

PandaBanker 2.5.7 (20171025)

PandaBanker 2.6.0 (20171204)

PandaBanker 2.6.1 (20171207)

PandaBanker 2.6.2 (20180122)

PandaBanker 2.6.3 (20180126)

PandaBanker 2.6.4 (20180214)

PandaBanker 2.6.5 (20180314)

PandaBanker 2.6.6 (20180321)

PandaBanker 2.6.7 (20180418)

PandaBanker 2.6.8 (20180425)

PandaBanker 2.6.9 (20180511)

Zeus MailSniffer (20160904)

FlokiBot 0.0.0.a (20160926)

FlokiBot 0.0.0.c (20161005)

FlokiBot 0.0.0.d (20161115)

Zeus Sphinx 1.5.5.0 (20160527)

Chthonic 2.23.10.1 (-)

Chthonic 2.23.12.3 (-)

Chthonic 2.23.12.5 (-)

Chthonic 2.23.14.3 (-)

Generated on 2018-01-14.
Legend:

>0.90 BinDiff similarity

>0.80 BinDiff similarity

>0.70 BinDiff similarity

>0.60 BinDiff similarity

>0.50 BinDiff similarity

>0.40 BinDiff similarity

>0.30 BinDiff similarity

>0.20 BinDiff similarity

>0.10 BinDiff similarity

>=0.0 BinDiff similarity

2) Methodology

Having worked on several of these families over the years, retrospectively I was interested in how similar their code actually is.
I then thought, this might be a cool idea for an experiment / blog post demonstrating one of the use-cases for my current project Malpedia. :)
Next, I remembered that I had written a crude collection of scripts in 2012 that I called "BatchBinDiff" back then and gave it a shot to use them in this endevour.
So why not dust it off and do a pretty diagram?

My first step was to take all dumps for the above families from Malpedia and convert them to IDBs.
While there are certainly hundreds of thousands packed samples for the families, the above is a pretty good collection of single (unpacked!) samples representing some of the versions observed over time, which makes the comparison way more manageable.
However, since IDA Pro's function recognition performance on memory dumps vs clean files is not as good, I had to manually improve the code coverage. For the files listed above, that took about 8 hours of pressing "C", "P", and occasionally "U". :D IDAscope helped as well.

With the IDBs at hand, I started toying around with "BatchBinDiff". I found a working set up in one of my VirtualBox VMs with IDA Pro 6.4 and BinDiff 4.0.
The first step was to create BinExport files for the IDBs. This can be achieved with a little idc script invoking the BinExporter IDA Plugin.
Using the freshly produced BinExport files, one can now use Zynamics' BinDiffDeluxe.exe in order to directly compare BinExport files on the command line.
Since we can only do direct comparisons, this means we have to execute BinDiffDeluxe n*(n-1)/2 = 4.851 times to calculate values for every pair once.
The runtime for this was about 80 minutes. Adding the BinExports on top, we look at a good 90 minutes.
It is quite obvious that this method does not scale well because of the quadratic complexity, but as BinDiff (IMHO) is still the best code-based binary similarity tool, it's at least a workaround to generate some ground-truth (about 4GB in text files with details about all matches).
And BinDiffDeluxe.exe is probably not widely known and didn't receive as much love in the past as it actually deserves. ;)

3) Additional Notes

Will I release the raw files/data? Maybe, I have to think about it and consider some factors regarding this, this will most likely happen in the context of Malpedia.

For now, this is just a dump of raw results.
I may have a closer look into some of the versions seemingly being outliers and try to explain what's happening there.
For example, for VM Zeus 2.0.0.0+, I assume this is related to the introduced API call obfuscation scheme.

I am aware that there are more families and versions that could be included here.
The following are on my list for future updates of this experiment:

ZLoader / Zeus Terdot - I simply have not enough coverage insight into this family.
Gameover P2P: It should be possible to find more of this one to cover more of its development over time...
@mak said I should test Diaphora and Kam1n0 vs. BinDiff. Sounds like a lot of work but interesting as well?
Chthonic: This doesn't really fit the picture, it's possible that only few parts of the Zeus code are used?

If you have further ideas or comments, let me know!

4) Credits

I would like to thank Maciej Kotowicz for several discussions we had over this.
I would also like to thank Dennis Schwarz for his many contributions of versioned samples that eased compilation of the data and increased the coverage.

5) Changelog

2018-05-22 Version 2: 99 versions of 14 zeus-like families. I added many versions of PandaBanker, Murofet (predecessor of Gameover), some samples to document the jump from VMZeus to Panda, and Zeus Sphinx.
2018-01-14 Initial release: 52 versions of 10 zeus-like families compared.

	>0.90 BinDiff similarity
	>0.80 BinDiff similarity
	>0.70 BinDiff similarity
	>0.60 BinDiff similarity
	>0.50 BinDiff similarity
	>0.40 BinDiff similarity
	>0.30 BinDiff similarity
	>0.20 BinDiff similarity
	>0.10 BinDiff similarity
	>=0.0 BinDiff similarity