Python Pyarmor + My Protector

[CRoot]

On 5/18/2021 at 2:34 AM, uchiha_indra said:

Hi! I followed your guide, and edited the file in the following way:

...
/* Start of code */

    /* push frame */
    if (Py_EnterRecursiveCall(""))
        return NULL;

    tstate->frame = f;
    FILE *dump_file = NULL;
    dump_file = fopen("./dump.log", "ab");
    PyMarshal_WriteObjectToFile((PyObject *) f, dump_file, 2);
    fclose(dump_file);

    if (tstate->use_tracing) {
...

 

However, when I execute the code and read the dump file, there are only '?' in them:

????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

Am I missing something?

Could you open this file with HexEdit?

----------------------------------------------------------20210519---------------

Yes, I have a same problem, and I use HexEdit opened too.

Edited May 19, 2021 by CRoot

[Extreme Coders]

On 5/18/2021 at 12:04 AM, uchiha_indra said:

However, when I execute the code and read the dump file, there are only '?' in them:

Am I missing something?

You're dumping the PyFrameObject. Instead you need to dump the PyCodeObject of the execution frame.

Check the parameters passed to PyMarshal_WriteObjectToFile.

[IDRALOU]

In my version of python i have ceval.h so the code is not the same, what can i do ?

[Newby User Learning]

Does this work in super mode + restrict 0 of pyarmor?

[svenskithesource]

https://github.com/Svenskithesource/cpython

For the lazy people, please do read what the solution is and dont just blindly download it.

[greenasitget]

On 3/29/2020 at 1:50 PM, Extreme Coders said:

Not necessary to unpack to get the key.

Key:

  Hide contents

GENERATE-KEY-0X72GOD-UNPACKME

Converted it to run on Linux because it's easier to compile CPython on Linux and also because I don't have Visual Studio installed in the Windows VM.
This also explains why "title" and "cls" commands were not found.

Steps :

  Hide contents

 

1. Use pyinstxtractor.py to extract the executable in Python 3.7

2. Using the extracted files, create the following directory structure

.
|-- martisor.pyc
`-- pytransform
    |-- __init__.py
    |-- _pytransform.dll
    |-- license.lic
    `-- pytransform.key

1 directory, 5 files

For running on Linux, you need _pytransform.so downloadable from https://pyarmor.dashingsoft.com/platforms.html

3. Install psutil using pip (Required for pyarmor). From now on, you can just run python3.7 martisor.pyc instead of the unpackme executable.

4. pyarmor encrypts the code objects on disk and they are only decrypted at runtime just before they are executed. The entire logic is implemented in _pytransform.dll. There are anti-debugging/timing checks to prevent us from using a debugger to dump code objects from memory. But there's no need to use a debugger at all when CPython itself is open source.

5. Compile Python 3.7 from source. Modify the _PyEval_EvalFrameDefault function such that it dumps the code object to disk. By doing so we do not need to bother about all the anti-debugging and encrypted stuff. This is because pyarmor decrypts the code object in memory before it hands it to the Python VM for execution.

6. Run strings on the dumped code  object. We get many base64 strings. Like this one: CkdFTkVSQVRFLUtFWS0wWDcyR09ELVVOUEFDS01FCg==

7. Base64 decode and profit!

 

 

Sorry, for the stupid question - since pyarmor release 6.3.0, the pytransform.key is no longer included. It only contains pytransform.pyd. does the solution still work when we substitute pytransform.key with pytransform.pyd.  

[Extreme Coders]

9 hours ago, greenasitget said:

since pyarmor release 6.3.0, the pytransform.key is no longer included. It only contains pytransform.pyd. does the solution still work when we substitute pytransform.key with pytransform.pyd

In this case, you won't need pytransform.key as pyarmor would read it from the pyd/dll.

Replacing the python dll is a generic solution that'll work in most cases. However it won't if you're trying to unpack a VM or super mode protected script.
The reason for this is in either of these modes pyarmor uses it's own private implementation of PyEval_EvalFrameDefault to execute obfuscated python bytecode. Hence the modified PyEval_EvalFrameDefault function in the custom python dll wont even be run in the first place to execute the protected code. It will be run though for non-protected code but dumping non-protected code is not what you want.

[xcfrg]

Does someone know of a easy way to tell which version of pyarmor was used? (as well as the different modes, advanced, super, etc).

[Extreme Coders]

On 9/5/2021 at 8:01 AM, xcfrg said:

Does someone know of a easy way to tell which version of pyarmor was used? (as well as the different modes, advanced, super, etc).

This requires knowledge of git internals.

All versions of pyamor ever released can be found on their GitHub repo: https://github.com/dashingsoft/pyarmor-core/

Essentially what you need to do is hash search (md5/sha etc) your target pyarmor dll/pyd in that repo to find the file and thus the commit. However there's another point to keep note of.

As mentioned in this thread, pyarmor now bundles the license data within the dll/pyd. Hence the license data would led to a different hash in-spite of the rest of the dll/pyd contents being the same.

To solve this problem, instead of hashing the whole file you can hash only a part (say the last 10KiB of the target dll/pyd which excludes the license data) and search all blobs in the repo which have the same hash for the last 10 KiB bytes.  You can use a library like gitdb for searching.

Using this you should be able to pinpoint the exact commit and the corresponding file on the repo.

As for the other question, the mode use can be deciphered from the numerical prefix.

0 => NONE (dll)
7 => JIT, ANTI-DEBUG, ADV (dll)
11 => JIT, ANTI-DEBUG, SUPER (pyd)
21 => VM, ANTI-DEBUG, ADV (dll)
25 => VM, ANTI-DEBUG, SUPER (pyd)

For example, windows.x86_64.25.py39 implies VM + ANTI-DEBUG + ADV modes using the dll pyd.

Edited September 16, 2021 by Extreme Coders
Change DLL to PYD (25 is pyd)

[xcfrg]

On 9/6/2021 at 1:25 PM, Extreme Coders said:

This requires knowledge of git internals.

All versions of pyamor ever released can be found on their GitHub repo: https://github.com/dashingsoft/pyarmor-core/

Essentially what you need to do is hash search (md5/sha etc) your target pyarmor dll/pyd in that repo to find the file and thus the commit. However there's another point to keep note of.

As mentioned in this thread, pyarmor now bundles the license data within the dll/pyd. Hence the license data would led to a different hash in-spite of the rest of the dll/pyd contents being the same.

To solve this problem, instead of hashing the whole file you can hash only a part (say the last 10KiB of the target dll/pyd which excludes the license data) and search all blobs in the repo which have the same hash for the last 10 KiB bytes.  You can use a library like gitdb for searching.

Using this you should be able to pinpoint the exact commit and the corresponding file on the repo.

As for the other question, the mode use can be deciphered from the numerical prefix.

0 => NONE (dll)
7 => JIT, ANTI-DEBUG, ADV (dll)
11 => JIT, ANTI-DEBUG, SUPER (pyd)
21 => VM, ANTI-DEBUG, ADV (dll)
25 => VM, ANTI-DEBUG, SUPER (pyd)

For example, windows.x86_64.25.py39 implies VM + ANTI-DEBUG + ADV modes using the dll.

Thank you!
I don't see any file with the format of  "windows.x86_64.25.py39" in the directory and after extracting, where did you find this?

[Extreme Coders]

13 hours ago, xcfrg said:

I don't see any file with the format of  "windows.x86_64.25.py39" in the directory and after extracting, where did you find this?

There won't be any file with that exact name.
Instead there should be pytransform.pyd or pytransform.dll and you need to figure what protection and mode it's using judging from the file size and searching on the git repo.

VM mode dlls and pyd has a size around 4 MiB.
JIT is around 1.2 MiB.
NONE is about 700 KiB.

This is just a rule of thumb though.

[goldensec]

any steps for unpack 

[mmo4122]

On 7/30/2021 at 3:24 PM, svenskithesource said:

https://github.com/Svenskithesource/cpython

For the lazy people, please do read what the solution is and dont just blindly download it.

i add this into ceval.c: strstr(PyUnicode_AsUTF8(co->co_filename), "dumped.py")

also add #include <string.h> in the header

but i got this error when compile

[svenskithesource]

18 hours ago, mmo4122 said:

i add this into ceval.c: strstr(PyUnicode_AsUTF8(co->co_filename), "dumped.py")

also add #include <string.h> in the header

but i got this error when compile

Does it compile even if you don't make any changes? Also while this method can still be useful, it might be easier to use https://github.com/Svenskithesource/PyArmor-Unpacker

[Napcaz]

On 10/3/2022 at 5:51 PM, svenskithesource said:

Does it compile even if you don't make any changes? Also while this method can still be useful, it might be easier to use https://github.com/Svenskithesource/PyArmor-Unpacker

There is a problem that I think is related to python 3.11. The cpython in this repo throws an error when compiling: Cannot open include file: 'ffi.h': No such file or directory

I thought about switching to an old python version, 3.10, and overwriting and compiling the same injection myself, but this time I get the following error Assertion failed: p->str != NULL, file C:\Users\user\Desktop\cpython-3.10\ cpython-3.10\Python\marshal.c, line 115 when python run

 

magically I was able to compile in the final version. I added the marshal.h and stdio.h you added and injected the code and succeeded. I saw the dump file, it gave a 41kb output for a 2-line python code,I was able to see my code even though it looks a bit corrupted with the text editor and the beginning of dumped.txt was like this: Standard "encodings" Package Standard Python encoding modules are stored in this package directory.

What exactly does this file mean, how can I use it? I will be grateful if you could help me.

Edited November 20, 2022 by Napcaz
I think it was but new question comes

[Extreme Coders]

8 hours ago, Napcaz said:

The cpython in this repo throws an error when compiling: Cannot open include file: 'ffi.h': No such file or directory

When compiling from source, get_externals.bat sometimes pulls in the wrong libffi.

The workaround is to delete the externals/libffi directory and download the correct libffi (should be libff-3.4.2 for Py 3.11) from https://github.com/python/cpython-bin-deps/archive/refs/tags/libffi-3.4.2.zip 
and extract the zip to externals/libffi.

8 hours ago, Napcaz said:

beginning of dumped.txt was like this: Standard "encodings" Package Standard Python encoding modules are stored in this package directory.

Won't be able to help fully with the other questions though as I don't use the injection technique myself to deobfuscate.
But this usually means you have dumped the wrong file. The line Standard "encodings" Package Standard Python encoding modules are stored in this package directory.  is a part of the encodings/__init__.py file.

[Napcaz]

On 11/21/2022 at 9:03 AM, Extreme Coders said:

Won't be able to help fully with the other questions though as I don't use the injection technique myself to deobfuscate.

I think I misunderstood myself when I said inject. I used your method, edited the _PyEval_EvalFrameDefault function to dump me the data and compiled the cpython. Just while doing this, I compiled the cpython from svenskithesource's github address that I quoted above, and when I did that, I got a lot of errors about libraries. Then I tried to compile the codes that svenskithesource uses for _PyEval_EvalFrameDefault manually by adding them to the stable versions of cpython, and it worked without any problems

On 11/21/2022 at 9:03 AM, Extreme Coders said:

But this usually means you have dumped the wrong file. The line Standard "encodings" Package Standard Python encoding modules are stored in this package directory.  is a part of the encodings/__init__.py file.

I can see the parts of the python code I'm running among the data I dump, but it is very complex. I wonder if the data I see is what they call the bytecode that the working python code turns into?

I'm also learning more about python and obfuscating thanks to you and svenskithesource, thank you very much for that. Now I'm trying to understand this address https://github.com/Svenskithesource/PyArmor-Unpacker

[Extreme Coders]

11 hours ago, Napcaz said:

I can see the parts of the python code I'm running among the data I dump, but it is very complex. I wonder if the data I see is what they call the bytecode that the working python code turns into?

That is the marshalled code object. Can be referred to as bytecode for that matter. The instructions that Python executes are a part of the code object.

 

11 hours ago, Napcaz said:

I'm also learning more about python and obfuscating thanks to you and svenskithesource, thank you very much for that. Now I'm trying to understand this address https://github.com/Svenskithesource/PyArmor-Unpacker

The last challenge of this year's FlareOn featured a pyarmor challenge. More research about pyarmor is now public than before. Here are some links:

1. https://github.com/levanvn/FLARE-ON9-Chal11_Unpacking-Pyarmor/ : This discusses about patching PyEval_EvalFrameDefault and also restoring the mapped opcodes.
    
2. https://nesrak1.github.io/2022/11/13/flareon09-11 : The author did a fantastic job about reversing the "JIT protection". Pyarmor does use GNU lightning to calculate the decryption keys from license data. Extending this approach it is possible to develop a static unpacker for pyarmor. The downside is this needs to be redone if pyarmor dev change the algorithm. In this regard a better solution can be to emulate the JIT code to calculate the decryption keys.
    
3. https://re-dojo.github.io/post/2022-11-13-FlareOn-9-part-4/#challenge-11---the-challenge-that-shall-not-be-named : This discusses about pyarmor internals a bit.
    
4. https://github.com/binref/refinery/blob/master/tutorials/tbr-files.v0x05.flare.on.9.ipynb  (Scroll down to challenge 11) : Again a nice read. The author discusses about the ciphers used to encrypt the code object and instructions. Pyarmor changes the cipher across versions as I'm aware of. Older versions used Triple DES. Now it's using AES in CTR mode. Combined with the information in (1) developing a static unpacker is possible.

[ZeN]

On 5/18/2020 at 2:20 PM, Extreme Coders said:

You can simply run strings on the dumped file.

What does "run strings on the dumped file" mean? I have a file like this after using the function:

co = f->f_code;

if (strstr(PyUnicode_AsUTF8(co->co_filename), ".py")){
    FILE *file;
    file = fopen("./dumped.txt", "ab");
    PyMarshal_WriteObjectToFile(co, file, 2);
    fclose(file);
}

but I can't figure out how to run it

[Extreme Coders]

12 hours ago, ZeN said:

What does "run strings on the dumped file" mean?

A python code object has multiple sub parts- the instructions themselves (co_code), constant list (co_consts) which includes strings and other numbers used, variable names (co_names) and so on. During protection, pyarmor will encrypt the instructions (co_code) followed by marshalling the code object (marshal.dumps). Marshalling is just a way to get a byte array representation of an in-memory object. The array of bytes is then re-encrypted and stored in the final obfuscated file.

Conversely on importing an obfuscated file, pyarmor will decrypt the bytes array followed by unmarshalling it (marshal.loads). However the co_code is not decrypted yet. As a result when at this stage the code object is dumped to a file the strings will be readable (as co_consts has already been decrypted). The instructions (co_code) is still encrypted and hence they will be undecipherable. This is the answer to your question.

12 hours ago, ZeN said:

but I can't figure out how to run it

As explained the output file is not intended to be run. This technique shown is  a way to retrieve strings used in the file. If you need to get runnable output it entails more work including decrypting the co_code.

[devil]

Nothing show