Mitigating false positives.

[mudlord]

This is starting to turn into a serious problem for me.

 

I have been packing my demo prods with beroexepacker. This works out well since it was optimized for demoscene productions and thus gives a very good compression ratio. However, I do get false positives, coming to the point that now Google blocks my demo downloads in their browser. I tried explaining to Google that the files are all false positives but no action is taken.

 

I noticed a lot of false positives with BEP and so this is turning into quite a problem, especially with AV vendors are blocking download URLs to my demos, as part of thier site screening services. I feel this is a losing battle, possibly due to BEP being used by malware devs too. I noticed similar problems with kkrunchy sadly.

 

I am seriously considering deploying my own executable packer but I am wondering on the best practises to mitigate false positives in future. Should I go with the taggant scheme for my own personal packer, and then even digitally signing all downloads I make that use the packer as well? Is that even effective? Or is it a fundamental problem due to how packers are designed,and must I continue doing the current method of advising companies of the packer I use in every single case, with the insistance that they are false positives? Or is this a general problem with the AV industry in general with packers.

 

I thought the taggant scheme was to mitigate these problems. Or am I wrong? Am I forced to spend money just for a completely non profit hobby?

[kao]

Few years ago anti-malware companies were losing battle with custom cryptors and packers. They were losing so badly that they all decided to ban packers that are not used in commercial and widely available software. So, to name a few, FSG, PESpin, Telock, MEW, UPACK and MPRESS got blacklisted. I guess kkrunchy and ber0 also got banned. Now 99.999% of ordinary user's computers are safer and nobody cares about the very small demoscene and others caught in the middle. So, yeah, it's a problem for you but not for the AV company and majority of its customers.

 

 

#1 - Taggant started as a good idea but ended up reinventing digital signatures. Nobody really gives a crap about it. You might as well just digitally sign your EXE, it is easier and faster than going through the taggant bureaucracy. And even then getting your signature on the whitelist might be harder than you expected (as authors of ProtectionID will tell you...).

 

Pretty much every single AV company has a way to report false positives. They also usually have a way to submit your files in an automated mode so that they can analyze/whitelist them before any harm is done. For example, all the issues I've reported to Bitdefender via http://www.bitdefender.com/site/Main/automaticSampleUploader/ have been fixed in matter of hours. Yes, it sucks - but that's the only real way for a small software dev who's not doing the mainstream UPX way.. 

 

The best bet would be to try and get contact with guys who are actually in charge of black-/whitelisting - as "customer care teams" in India are bunch of trained monkeys that can only read from a script in front of them. If you're ever in some RE-related conference, just make new friends like a madman..

 

#2 - As for hosting files, easiest solution I can think of would be password-protected RAR files with encrypted filenames - if bot/scanner can't unpack it, it can't blacklist it. Your site will be removed from the "shit filter" URL lists eventually, but it might take some time. Google Webmaster tools could help a lot (remove detected files, request review of site, so they can verify it's clean atm). However, the domain name might never get back to 100% clean reputation.

 

Other option I can think of, is placing downloads on Dropbox/Google Drive/whatever cloud hosting and making sure the antivirus they're using doesn't have False Positives. Not ideal, but at least you won't taint your domains reputation is something goes wrong.

 

 

obligatory disclaimer - all views represented in this post are my own and in no way represent the views of my employer and/or my colleagues.

[mudlord]

Thanks kao for responding.

 

To sum up the problem with reporting to every single AV company, here is the most recent VirusTotal scan of a file that got flagged using BEP.

 

https://www.virustotal.com/en/file/28e8fbe9f596fd76e82b6f9a21c3824f962c94cc107023b9da63c42dad26d825/analysis/

 

I am not sure if I have the time to contact 27 companies each time a demo prod or any other prod of mine is done. This is getting insane that anything other than ASPack/UPX/PECompact is banned, and those packers are unsuitable for democoding. I shudder to think that the devs of Crinkler have to put up with such garbage too.

 

 

The best bet would be to try and get contact with guys who are actually in charge of black-/whitelisting - as "customer care teams" in India are bunch of trained monkeys that can only read from a script in front of them. If you're ever in some RE-related conference, just make new friends like a madman..

 

Yeah, seems to be the best approach. As I said, Google's Webmaster tools were completely hopeless in my case.

 

 

Few years ago anti-malware companies were losing battle with custom cryptors and packers. They were losing so badly that they all decided to ban packers that are not used in commercial and widely available software. So, to name a few, FSG, PESpin, Telock, MEW, UPACK and MPRESS got blacklisted. I guess kkrunchy and ber0 also got banned.

 

Wow, just wow. So this means than any other packer thats not ASPack/UPX/PECompact will get banned, thats just...wow.

 

 

1 - Taggant started as a good idea but ended up reinventing digital signatures. Nobody really gives a crap about it.

 

Wow again. I wonder then why the IEEE thought it was such a good idea in the first place.

 

 

You might as well just digitally sign your EXE, it is easier and faster than going through the taggant bureaucracy. And even then getting your signature on the whitelist might be harder than you expected (as authors of ProtectionID will tell you...).

 

So even plain code signing certs don't work? Damn, there goes that idea, even if it would expand the size of exes where 200 bytes or so matters.

 

 

#2 - As for hosting files, easiest solution I can think of would be password-protected RAR files with encrypted filenames - if bot/scanner can't unpack it, it can't blacklist it.

 

The biggest problem is thats frowned upon in the demoscene. All packages must be easily unpacked, and there is a unwritten rule of files being unpassworded ZIPs. Scene.org is a good host for demo files, and hosts releases from parties anyway. Maybe in future I should refrain from mirroring demostuff on my own domain. Or get dedicated demoscene hosting like untergrund.net or Planet-D, which is dedicated for demosceners.

Edited August 18, 2015 by mudlord

[kao]

I feel your pain. But as I said, you're a minority. Unfortunately these days everyone cares only about the majority of users, not the edge cases.

 

As for your points: 

* Taggant - it started as a good idea but got morphed by theoretically philosophically bureacratic organization of IEEE. It still is a decent replacement for custom watermarks used by commercial protectors (Themida, VMProtect) but pretty useless in your case.

 

* Code signing certs work. But they are also used by malware authors for exactly same reasons. So, you need to convince AV companies that you're trustworthy and not going to sign malware with your cert. ProtectionID is using self-signed certificate, so it's even harder for them.

 

* It's actually not 27 companies. Many of the are using same engine, so if the engine provider fixes the False Positive, it should get fixed automatically for all their customers too. Here's the earlier scan of the same file, getting detection ratio 20/57:

Engine                 Signature                             Version                Update

Ad-Aware               Gen:Trojan.Heur.RP.dqX@aSQ8GOli       12.0.163.0             20150607

Arcabit                Trojan.Heur.RP.E25E53                 1.0.0.425              20150607

AVG                    Win32/Heur                            15.0.0.4355            20150607

Avira                  TR/Crypt.XPACK.Gen                    8.3.1.6                20150607

BitDefender            Gen:Trojan.Heur.RP.dqX@aSQ8GOli       7.2                    20150607

Bkav                   W32.HfsAutoB.6467                     1.3.0.6379             20150606

CAT-QuickHeal          (Suspicious) - DNAScan                14.00                  20150606

Comodo                 TrojWare.Win32.Trojan.NSPM.~gen       22370                  20150607

Emsisoft               Gen:Trojan.Heur.RP.dqX@aSQ8GOli (   3.5.0.636              20150607

F-Secure               Gen:Trojan.Heur.RP.dqX@aSQ8GOli       11.0.19100.45          20150607

GData                  Gen:Trojan.Heur.RP.dqX@aSQ8GOli       25                     20150607

Jiangmin               TrojanDownloader.Cabby.deq            16.0.100               20150606

Malwarebytes           Packer.Suspicious                     2.1.1.1115             20150607

MicroWorld-eScan       Gen:Trojan.Heur.RP.dqX@aSQ8GOli       12.0.250.0             20150607

Symantec               Suspicious.Emit                       20141.2.0.56           20150607

Tencent                Trojan.Win32.YY.Gen.7                 1.0.0.1                20150607

TheHacker              W32/Bagle.gen@MM                      6.8.0.5.575            20150607

TrendMicro             Mal_Bero                              9.740.0.1012           20150607

TrendMicro-HouseCall   Mal_Bero                              9.700.0.1001           20150607

VBA32                  Malware-Cryptor.General.6             3.12.26.4              20150605

As you can see, most of these detections are the same, and coming from BitDefender engine.

MalwareBytes, AVG, Avira, Symantec and VBA32 all have heuristic detections - most likely due to very non-standard PE file layout.

Based on detection name, I'm guessing that Trend Micro has blacklisted Bero packer itself, not your file specifically.

 

If you deal with those companies, the rest should not cause you much trouble. Yeah, it's still a problem but not as horrible as it looked before.

 

* As for file hosting, if scene has specific rules, there's not much you can do. Just shift the problems to someone else by hosting your files on Planet-D..

 

and the obligatory disclaimer again - all views represented in this post are my own and in no way represent the views of my employer and/or my colleagues.

[Nemo]

it's annoying, you try to get smallest file possible by using other packers and end up with something that is reported as malware.. I didn't know it happened with kkrunchy though..

[mudlord]

Yep, happens with kkrunchy too. Seems anything not endorsed by the AV conglomerates is flagged as malware, as kao said.

[Struppigel]

Hi mudlord. That problem s*cks, but it is just the way it goes. If too much malware is packed with any of these packers of if the packers are not widely used, they will get flagged. But the more False Positives you report for certain packers, the more likely they won't be seen as suspicious anymore.

Often it is also not the packer that is flagged, but certain properties in the file that are just too common for viruses will make it heuristically suspicious. Packers that are made to keep files very small and use lots of tricks to do so are even more prone to be seen as malcious because of all the anomalies in the format.

 

You might want to try PEStudio to get some hints to the things that look suspicious to AV companies. Especially if you intent to create your own packer.

 

 

And yes, you don't have to contact all Antivirus Companies. Bitdefender's engine is used by a lot of other companies and most of the others have a set of rules to flag or not flag files if certain other vendors detect them or not. So do it one by one, starting with Bitdefender, then go to the big ones like Kaspersky, and other detections will likely drop too.

 

Edit: Bitdefender is already not detecting it anymore.

Edited August 23, 2015 by Struppigel

[mudlord]

Thanks very much for the assistance! I will keep PeStudio in mind in future when packing executables.

 

I tried out PeStudio on my own packer and heres what I get.

 

 

Already a great improvement over bero's packer, though the compression ratio is not as good. I should fix the things PeStudio points out, like the invalid checksum and things.

[simple]

That problem s*cks, but it is just the way it goes.

 

ask u permission / pay $ to work on my own clients machines? dont know about u brother but where I come from theres a word for that...

 

Here's another way it can go

 

https://github.com/gentilkiwi/mimikatz/tree/master/mimidrv

 

BEFORE:

 

 

AFTER:

 

Edited August 22, 2015 by simple

[simple]

Hi simple.I think we are going off-topic in this thread, so I would rather PM you.What did you mean by that sentence:"ask u permission / pay $ to work on my own clients machines? dont know about u brother but where I come from theres a word for that..."And do you mind telling me details (e.g. code) how you disabled GData protection?Best regardsStruppigel

We put legal jargon in eulas that say "remove/disable ur AV or I'll do it for u". I'm not asking ur permission to work on my own clients/customers. If u can't understand why there's no point explaining.

Already posted the code on how I did it brother. It's not hard. Read up on kernel coding and u will see what I did cannot be stopped & is very well known. Good luck.

 

[Struppigel]

If u can't understand why there's no point explaining.

 

I am not a native speaker. It would already help me a great deal to understand you if you just wrote in normal English.

I might have I understood the sentence now, but I am not sure if I did it the way it was meant.

 

By the way, private messages are called private for reason.

[metr0]

User simple may have meant that he deems AV software intrusive as it has some sort of control over the applications a customer runs on his own computer (i.e., see mudlord's initial problem). There may have been other means to bring this point across.