Decoding xmm instruction ??? distorm

[CodeExplorer]

Decoding xmm instruction ??? like:
https://c9x.me/x86/html/file_module_x86_id_204.html

I've investigated a bit distorm:
https://github.com/gdabah/distorm/tree/master/src

https://github.com/gdabah/distorm/blob/master/src/instructions.c
static _InstInfo* inst_vex_lookup(_CodeInfo* ci, _PrefixState* ps)
{
...
    /* start can be either 1 (0x0f), 2 (0x0f, 0x038) or 3 (0x0f, 0x3a), otherwise it's illegal. */
    switch (start)
    {
        case 1: in = Table_0F; break;
        case 2: in = Table_0F_38; break;
        case 3: in = Table_0F_3A; break;
        default: return NULL;
    }

in = InstructionsTree[INST_NODE_INDEX(in) + *ci->code];

Table_0F is defined in https://github.com/gdabah/distorm/blob/master/src/insts.c
_InstNode Table_0F = 256;
_InstNode Table_0F_0F = 1440;
_InstNode Table_0F_38 = 1896;
_InstNode Table_0F_3A = 2152;

https://github.com/gdabah/distorm/blob/master/src/instructions.h
/* Instruction node is treated as { int index:13;  int type:3; } */
typedef uint16_t _InstNode;

Last 3 bits are type right? are in all those 4 cases 000
https://github.com/gdabah/distorm/blob/master/src/mnemonics.c
so how instruction are decoded?
Weird stuff, actually all decompiller are in this way except https://www.ollydbg.de/disasm.zip
which doesn't' support xmm instructions.
 

[jackyjask]

Olly v2.01 indeed supports XMM and commands , eg:

but it was not released as open source package... so maybe we give it a try?

[kao]

The ultimate reference will always be "Intel® 64 and IA-32 Architectures Software Developer’s Manual". Get Volume 2D, and check "APPENDIX A OPCODE MAP" and "APPENDIX B INSTRUCTION FORMATS AND ENCODINGS" - they give a great overview, even though it might seem slightly complicated at first.

 

[CodeExplorer]

I was able to compile distorm in Visual Studio.
The above code is not used at all, it uses instead InstructionsTree and _MNEMONICS

unsigned char rawData2[] = { 0xF2,0x0F,0x10,0x83,0xE4,0xF8,0x81,0xEC};

    /* Walk first byte in InstructionsTree root. */
    in = InstructionsTree[tmpIndex0];

insts.c:
_InstNode InstructionsTree[5688] = {
/* 0 - _00 */  0x2000,
...
/* f - _0F */  0xa100,

#define INST_NODE_TYPE(n) ((n) >> 13)

    /* Try 2 bytes long instruction (doesn't include ModRM byte). */
    if (instType == INT_LIST_FULL) {
        in = InstructionsTree[INST_NODE_INDEX(in) + tmpIndex1];
        if (in == INT_NOTEXISTS) return NULL;
        instType = INST_NODE_TYPE(in);

        /* This is where we check if we just read two escape bytes in a row, which means it is a 3DNow! instruction. */
        if ((tmpIndex0 == _3DNOW_ESCAPE_BYTE) && (tmpIndex1 == _3DNOW_ESCAPE_BYTE)) return &II_3DNOW;

        /* 2 bytes instruction (OCST_2BYTES). */
        if (instType < INT_INFOS)
            return instType == INT_INFO ? &InstInfos[INST_NODE_INDEX(in)] : (_InstInfo*)&InstInfosEx[INST_NODE_INDEX(in)];

        /*
         * 2 bytes + mandatory prefix.
         * Mandatory prefixes can be anywhere in the prefixes.
         * There cannot be more than one mandatory prefix, unless it's a normal operand size prefix.
         */
        if (instType == INT_LIST_PREFIXED) return inst_lookup_prefixed(in, ps);
    }

opcode = 0x00000860
_MNEMONICS[opcode]:
const unsigned char _MNEMONICS[] =
...

Not helpful at all, I need Streaming SIMD Extensions separated.

I was searching Streaming SIMD Extensions complete instruction list,
the best result was https://www.officedaytime.com/simd512e/

 

[boot]

On 8/3/2023 at 2:28 AM, CodeExplorer said:

Decoding xmm instruction

You can refer to Zydis or Capstone, which can get the source code from GitHub.