Jump to content
Tuts 4 You

x86 Code Obfuscation Techniques


ghandi

Recommended Posts

Posted (edited)

Hi everybody,

I was playing around with a few binary files on my computer today which employ code obfuscation at some point in their execution. The techniques used differ and yet the end result is very similar, code which causes OllyDbg or IDA to disassemble it incorrectly. The 2 ways i thought of using to see the true code is to either log the execution and then remove redundant operations from the log, leaving only the true instruction or to deobfuscate the code prior to execution.

This second option is faster for sure, but it is fraught with danger also because it increases the margin for error. If the deobfuscation should incorrectly remove or change a valid opcode then the subsequent execution might not do what it was intended to do or worse, crash the thread entirely.

In a nutshell, if done correctly, code obfuscation can be a real pain in the rectum yet doing a Google search on the subject doesn't yield much in the way of educational material. After reading through the wikibooks page its refreshing to click through page after page of advertisement for Bart's PELock, Code Virtualizer/Themida, Dyamar Code Obfuscator, only to find that there isn't a great deal of information regarding the how-to's and why-for's of x86 code obfuscation and its implementation in the different coding languages.

I was sort of hoping that any forum members could share their experiences or knowledge regarding this matter? Possibly even some real life examples? SunBeam has already made a great start in his CodeDoctor thread, i wanted a more generalized thread and didn't want to hijack his so started this one.


007AC000 <ModuleEntry> /EB 02 JMP SHORT 007AC004
007AC002 |72 2E JB SHORT 007AC032
007AC004 \50 PUSH EAX
007AC005 EB 02 JMP SHORT 007AC009
007AC007 C191 E8190000 00 RCL DWORD PTR [ECX+19E8],0
007AC00E EB 04 JMP SHORT 007AC014
007AC010 E5 A1 IN EAX,0A1
007AC012 73 5E JNB SHORT 007AC072
007AC014 EB 01 JMP SHORT 007AC017
007AC016 0133 ADD DWORD PTR [EBX],ESI
007AC018 C0EB 04 SHR BL,4
007AC01B 12B43C 2D713AEB ADC DH,BYTE PTR [ESP+EDI+EB3A712D]
007AC022 04 D9 ADD AL,0D9
007AC024 886D BC MOV BYTE PTR [EBP-44],CH
007AC027 EB 02 JMP SHORT 007AC02B
007AC029 A7 CMPS DWORD PTR [ESI],DWORD PTR ES:[EDI]
007AC02A A0 33C0EB04 MOV AL,BYTE PTR [4EBC033]
007AC02F 41 INC ECX
007AC030 9C PUSHFD
007AC031 CF IRETD
007AC032 AC LODS BYTE PTR [ESI]
007AC033 64:FF30 PUSH DWORD PTR FS:[EAX]
007AC036 EB 01 JMP SHORT 007AC039
007AC038 E0 64 LOOPDNE SHORT 007AC09E
007AC03A 8920 MOV DWORD PTR [EAX],ESP
007AC03C EB 03 JMP SHORT 007AC041
007AC03E 09F3 OR EBX,ESI
007AC040 C9 LEAVE
007AC041 EB 03 JMP SHORT 007AC046
007AC043 6E OUTS DX,BYTE PTR ES:[EDI]
007AC044 17 POP SS
007AC045 3C 8B CMP AL,8B
007AC047 10EB ADC BL,CH
007AC049 04 FA ADD AL,0FA
007AC04B 37 AAA
007AC04C - 66:75 58 JNZ SHORT 0000C0A7
007AC04F EB 03 JMP SHORT 007AC054
007AC051 D328 SHR DWORD PTR [EAX],CL
007AC053 05 C3EB0402 ADD EAX,204EBC3
007AC058 D85C54 EB FCOMP DWORD PTR [ESP+EDX*2-15]
007AC05C 0100 ADD DWORD PTR [EAX],EAX
007AC05E 8B5424 0C MOV EDX,DWORD PTR [ESP+C]
007AC062 EB 04 JMP SHORT 007AC068
007AC064 F8 CLC
007AC065 186A E4 SBB BYTE PTR [EDX-1C],CH
007AC068 EB 01 JMP SHORT 007AC06B
007AC06A 0C EB OR AL,0EB
007AC06C 04 4F ADD AL,4F
007AC06E EE OUT DX,AL
007AC06F A4 MOVS BYTE PTR ES:[EDI],BYTE PTR [ESI]
007AC070 CC INT3
007AC071 B9 FCFFFFFF MOV ECX,-4
007AC076 EB 03 JMP SHORT 007AC07B
007AC078 41 INC ECX
007AC079 D9D2 FST EDX ; Illegal use of register
007AC07B 8182 B8000000 8D00>ADD DWORD PTR [EDX+B8],8D
007AC085 EB 04 JMP SHORT 007AC08B
007AC087 281D B064EB02 SUB BYTE PTR [2EB64B0],BL
007AC08D ^ 70 81 JO SHORT 007AC010
007AC08F 8942 04 MOV DWORD PTR [EDX+4],EAX
007AC092 8942 10 MOV DWORD PTR [EDX+10],EAX
007AC095 EB 04 JMP SHORT 007AC09B
007AC097 4E DEC ESI
007AC098 7F 55 JG SHORT 007AC0EF
007AC09A 34 EB XOR AL,0EB
007AC09C 02E6 ADD AH,DH
007AC09E A6 CMPS BYTE PTR [ESI],BYTE PTR ES:[EDI]
007AC09F EB 03 JMP SHORT 007AC0A4
007AC0A1 3F AAS
007AC0A2 80EE 89 SUB DH,89
007AC0A5 42 INC EDX
007AC0A6 1889 420CEB04 SBB BYTE PTR [ECX+4EB0C42],CL
007AC0AC 09AD 275CEB04 OR DWORD PTR [EBP+4EB5C27],EBP
007AC0B2 304F C4 XOR BYTE PTR [EDI-3C],CL
007AC0B5 F7EB IMUL EBX
007AC0B7 01DA ADD EDX,EBX
007AC0B9 8942 08 MOV DWORD PTR [EDX+8],EAX
007AC0BC EB 02 JMP SHORT 007AC0C0
007AC0BE 8806 MOV BYTE PTR [ESI],AL
007AC0C0 EB 01 JMP SHORT 007AC0C3
007AC0C2 AE SCAS BYTE PTR ES:[EDI]
007AC0C3 2BE1 SUB ESP,ECX
007AC0C5 EB 02 JMP SHORT 007AC0C9
007AC0C7 A6 CMPS BYTE PTR [ESI],BYTE PTR ES:[EDI]
007AC0C8 A7 CMPS DWORD PTR [ESI],DWORD PTR ES:[EDI]
007AC0C9 FF240C JMP DWORD PTR [ESP+ECX]
007AC0CC EB 01 JMP SHORT 007AC0CF
007AC0CE FE ??? ; Unknown command
007AC0CF EB 02 JMP SHORT 007AC0D3
007AC0D1 9E SAHF
007AC0D2 12EB ADC CH,BL

This example uses short jumps and byte sequences which will make Olly's disassembler analyze the code incorrectly. Embedded in this is a SEH installation, memory access violation and in the handler it clears the DR0-DR3/DR7 registers after adjusting the EIP (adding 0x8D to the EIP).

Here is the routine without obfuscation, i've simply stepped through it and noted the 'real' instructions:


SEH installation::
007AC004 50 PUSH EAX ; Save EAX007AC009 E8 19000000 CALL 007AC027 ; Set address of SEH in stack, same as PUSH 007AC00E
007AC02B 33C0 XOR EAX,EAX ; Zero EAX
007AC033 64:FF30 PUSH DWORD PTR FS:[EAX] ; Save old first handler
007AC039 64:8920 MOV DWORD PTR FS:[EAX],ESP ; Place SEH into chain as new first handler
007AC046 8B10 MOV EDX,DWORD PTR [EAX] ; Trigger memory access violationStructured Exception Handler:
007AC017 33C0 XOR EAX,EAX ; Zero EAX
007AC05E 8B5424 0C MOV EDX,DWORD PTR [ESP+C] ; Load ptr to provided CONTEXT structure
007AC071 B9 FCFFFFFF MOV ECX,-4 ; Set ECX to -4
007AC07B 8182 B8000000 8D00>ADD DWORD PTR [EDX+B8],8D ; Adjust EIP by +0x8D
007AC08F 8942 04 MOV DWORD PTR [EDX+4],EAX ; Zero Dr0
007AC092 8942 10 MOV DWORD PTR [EDX+10],EAX ; Zero Dr3
007AC0A4 8942 18 MOV DWORD PTR [EDX+18],EAX ; Zero Dr7
007AC0A7 8942 0C MOV DWORD PTR [EDX+C],EAX ; Zero Dr2
007AC0B9 8942 08 MOV DWORD PTR [EDX+8],EAX ; Zero Dr1
007AC0C3 2BE1 SUB ESP,ECX ; Adjust ESP by ECX (add esp,4)
007AC0C9 FF240C JMP DWORD PTR [ESP+ECX] ; Jump to address at [ESP-4]

We can reduce this code further by removing the MOV ECX, -4 and replacing the last 2 lines with a simple RET 4. With all of the junk removed, the code simply installs the SEH, triggers it and then clears the debug registers before continuing execution from an adjusted EIP.

the example doesn't show the call destination because it immediately jumps but the next line is correct:


007AC009 E8 19000000 CALL 007AC027 ; Set address of SEH in stack, same as PUSH 007AC00E
007AC02B 33C0 XOR EAX,EAX ; Zero EAX

The first line of the SEH is also a jump, which is why i haven't shown it, instead only showing the first valid opcode i encountered within the jumps.


007AC009 E8 19000000 CALL 007AC027 ; Set address of SEH in stack, same as PUSH 007AC00E
007AC00E /EB 04 JMP SHORT 007AC014 ; SEH first line
007AC014 /EB 01 JMP SHORT 007AC017 ; SEH second line
007AC017 33C0 XOR EAX,EAX ; First valid opcode
007AC019 /EB 04 JMP SHORT 007AC01F ; Off into more junk

Now lets look at the actual obfuscation technique used here:


007AC000 <ModuleEntry> /EB 02 JMP SHORT 007AC004
007AC002 |72 2E JB SHORT 007AC032
007AC004 \50 PUSH EAX

Olly has recognized where the jump will land, thus was able to disassemble it correctly. Note that the 'filler' bytes between the jump and its destination were disassembled into a conditional jump, jump if below.


007AC005 /EB 02 JMP SHORT 007AC009
007AC007 |C191 E8190000 00 RCL DWORD PTR [ECX+19E8],0

With the next jump though, Olly's disassembler chokes. Even though we can see clearly that the jump length is still only 2 bytes away, the 0xC1 byte has confused Olly's flow analysis and it has given us totally different instructions than what will actually be executed. This is just one of the many operands which can cause this to happen and it is something which is employed by many protectors.


007AC014 /EB 01 JMP SHORT 007AC017
007AC016 |0133 ADD DWORD PTR [EBX],ESI007AC019 /EB 04 JMP SHORT 007AC01F
007AC01B |12B43C 2D713AEB ADC DH,BYTE PTR [ESP+EDI+EB3A712D]007AC027 /EB 02 JMP SHORT 007AC02B
007AC029 |A7 CMPS DWORD PTR [ESI],DWORD PTR ES:[EDI]
007AC02A |A0 33C0EB04 MOV AL,BYTE PTR [4EBC033]007AC036 /EB 01 JMP SHORT 007AC039
007AC038 |E0 64 LOOPDNE SHORT 007AC09E007AC041 /EB 03 JMP SHORT 007AC046
007AC043 |6E OUTS DX,BYTE PTR ES:[EDI]
007AC044 |17 POP SS
007AC045 |3C 8B CMP AL,8B007AC048 /EB 04 JMP SHORT 007AC04E
007AC04A |FA CLI
007AC04B |37 AAA
007AC04C -|66:75 58 JNZ SHORT 0000C0A7007AC04F /EB 03 JMP SHORT 007AC054
007AC051 |D328 SHR DWORD PTR [EAX],CL
007AC053 |05 C3EB0402 ADD EAX,204EBC3
007AC058 D85C54 EB FCOMP DWORD PTR [ESP+EDX*2-15]007AC05B /EB 01 JMP SHORT 007AC05E
007AC05D |008B 54240CEB ADD BYTE PTR [EBX+EB0C2454],CL007AC068 /EB 01 JMP SHORT 007AC06B
007AC06A |0C EB OR AL,0EB

As we can see in each of these examples, the jump length is known and the jump is unconditional, yet OllyDbg's disassembler engine can't correctly decipher the instructions due to the code structuring and use of such bytes to throw it off. This doesn't use any self-modifiying code at all, simply jumps and junk bytes to make things a little more difficult.

HR,

Ghandi

Edited by ghandi
Posted (edited)

I've written deobfuscation engines for themida and vmprotect.

Themida uses a few standard obfu techniques which it reuses is different cycles. My engine did the same only backwards.

Was purely pattern driven. But did a reasonably good job, well until the code (of the engine that is) started to look crap and I couldn't make heads or tails from it afterwards. So well discontinued it.

And by then I knew the VM and packer code so well I didn't need deobfuscation.

VMprotect only uses non-functional opcodes and jumps/calls and simple stack obfu. That deobfuscator makes a few errors but for interpretation of routines works fine.

Mostly BTC/BT and AAS etc opcodes. stack obfu is just push random stuff and then lea esp, [esp+20] or so.

I found the themida obfu quite good, but most obfu is not very advanced in preventing interpretation.

Edited by quosego
Posted

Thanks for your input Quosego, :D

Is there any chance you could show a few examples of the obfuscation used by VMP and Themida with a brief explanation how they actually work, please?

This time i'll show some of the obfuscation used by Armadillo in its stub:


00685007 50 PUSH EAX
00685008 51 PUSH ECX
00685009 0FCA BSWAP EDX
0068500B F7D2 NOT EDX
0068500D 9C PUSHFD
0068500E F7D2 NOT EDX
00685010 0FCA BSWAP EDX
00685012 EB 0F JMP SHORT 00685023
00685014 B9 EB0FB8EB MOV ECX,EBB80FEB
00685019 07 POP ES
0068501A B9 EB0F90EB MOV ECX,EB900FEB
0068501F 08FD OR CH,BH
00685021 EB 0B JMP SHORT 0068502E
00685023 F2: PREFIX REPNE:
00685024 ^ EB F5 JMP SHORT 0068501B
00685026 ^ EB F6 JMP SHORT 0068501E
00685028 F2: PREFIX REPNE:
00685029 EB 08 JMP SHORT 00685033
0068502B FD STD
0068502C ^ EB E9 JMP SHORT 00685017
0068502E F3: PREFIX REP:
0068502F ^ EB E4 JMP SHORT 00685015
00685031 FC CLD
00685032 - E9 9D0FC98B JMP 8C315FD4
00685037 CA F7D1 RETF 0D1F7
0068503A 59 POP ECX
0068503B 58 POP EAX

This section of code accomplishes absolutely nothing. If you were to set a breakpoint on the line after 0068503B and press F9, the registers will not have changed at all and tracing through it with F7, the only thing worth commenting on is the prefix 'trick' used here.

I'll reference a page on CrackZ over at Woodmanns: http://www.woodmann.com/crackz/Tutorials/Protect.htm

21. Anti-tracing trick. More intelligent tracer applications would most likely implement an instruction decoder in order to follow the execution path, of course special handlers will be needed for conditional/unconditional jump instructions as well as others. Another possibility which could fool a tracer is a JMP instruction with a REPE or REPNE prefix. Here is how it works :

REPE (0xF2) <-- prefix only valid with CMPS/SCAS

JMP SHORT LOC_1 (0xEBxx) <-- JMP length is 2

SOME_JUNK_BYTES_HERE

LOC_1:

RESUME_CODE

When the instruction decoder reaches the REPE it correctly decodes the instruction as REPE JMP with an instruction length of 3 bytes. If a special handler routine isn't called the tracer will assume the next instruction to be executed is in the junk bytes and not the JMP to LOC_1. This trick is used by portions of the Armadillo loading code but will only be effective against single step tracers, not those using the trap flag. Armadillo uses a SEH handler to detect if there is a tracer using the trap flag.

I'm not sure what version of Armadillo they were talking about, but the example i've shown doesn't do anything at all. If you are tracing and you reach the PREFIX instructions, pressing F7 or F8 will not land you on the following line, instead the CPU will also execute the next instruction and you will land at the jump destination instead.

But most other examples you find on the net will incorporate a SEH and an exception which will be caught by the SEH if no debugger is present, else the debugger 'consumes' the exception. I'm not sure what other debuggers are affected, but VS2008 and OllyDbg 1.10 were.


006850A6 60 PUSHAD
006850A7 33C9 XOR ECX,ECX
006850A9 75 02 JNZ SHORT 006850AD
006850AB EB 15 JMP SHORT 006850C2
006850AD EB 33 JMP SHORT 006850E2
006850AF C9 LEAVE
006850B0 75 18 JNZ SHORT 006850CA
006850B2 7A 0C JPE SHORT 006850C0
006850B4 70 0E JO SHORT 006850C4
006850B6 EB 0D JMP SHORT 006850C5
006850B8 E8 720E79F1 CALL F1E15F2F
006850BD FF15 00790974 CALL DWORD PTR [74097900]
006850C3 F0:EB 87 LOCK JMP SHORT 0068504D ; LOCK prefix is not allowed
006850C6 DB7A F0 FSTP TBYTE PTR [EDX-10]
006850C9 A0 33615051 MOV AL,BYTE PTR [51506133]

We have a PUSHAD which saves registers to the stack, then more junk code is executed which achieves nothing, finally landing on the 0x61 on that last line, which is a POPAD. This is very similar to the example code i showed in my first post in the sense that it is using a code structure and bytes which confuse the disassembler engine of OllyDbg.

These kinds of obfuscation are easily bypassed, because Armadillo have not changed these 'obfuscation' bytes for many versions. An Olly script is simple to create which scans the code section for instances of these and replaces them with NOPs instead with no risk of removing valid code.

Finally i'll show a simple form of self-modifying code which Armadillo uses:


00685101 60 PUSHAD
00685102 9C PUSHFD
00685103 33C0 XOR EAX,EAX
00685105 E8 09000000 CALL 00685113
0068510A E8 E8230000 CALL 006874F7
0068510F 007A 23 ADD BYTE PTR [EDX+23],BH
00685112 A0 8B0424EB MOV AL,BYTE PTR [EB24048B]
00685117 037A 29 ADD EDI,DWORD PTR [EDX+29]

Tracing along, we could be forgiven for thinking that the call at 0068510A will be executed, but stepping into the call at 00685105 we see:


00685113 8B0424 MOV EAX,DWORD PTR [ESP]
00685116 /EB 03 JMP SHORT 0068511B
0x7A, 0x29, 0xE9 ; Junk bytes to confuse disassembly
0068511B C600 90 MOV BYTE PTR [EAX],90
0068511E C3 RETand our code at the return has now changed to:[code]
00685105 E8 09000000 CALL 00685113
0068510A 90 NOP
0068510B E8 23000000 CALL 00685133

The 0xE8 was replaced with a 0x90 and now it is a NOP, leaving the real call below. Tracing into that call though, it is simply a RET. If we were to try using F8 to step over the call, OllyDbg doesn't catch the return and instead lands in ntdll with an access violation error which is part of the Armadillo shell (This occurs even using hardware breakpoints to single step).

HR,

Ghandi

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...