A basic way using RtlAdjustPrivilege to detect the debugger (OllyDbg and IDA demo 6.6)As usually but not (enabled by default) for all debugger, the Debugger must acquiring debug privilege to work with its complete capacity.
The snippet is simple and probably already used but I write it as simple as possible to get a clear ASM code inside the debugger.RtlAdjustPrivilege: Enables or disables a privilege from the calling thread or process.

NTSTATUS RtlAdjustPrivilege (  ULONG    Privilege,     //[In]    Privilege index to change.  BOOLEAN  Enable,        //[In]    If TRUE, then enable the privilege otherwise disable.  BOOLEAN  CurrentThread, //[In]    If TRUE, then enable in calling thread, otherwise process.  PBOOLEAN Enabled        //[Out]    Whether privilege was previously enabled or disabled. )  

RtlAdjustPrivilege store the previous status into boolean variable.Our work is to read the contents of this variable after calling RtlAdjustPrivilege with SE_DEBUG_PRIVILEGE as parameter, and of course if a status is already enabled then we have a likely debugging situation.

/* * ------------------------------------------------- * Using RtlAdjustPrivilege to detect debugger. * Tested on (OllyDbg and IDA demo 6.6) * Released 03/2015. * [by Insid3Code from I3CT] * -------------------------------------------------*/#include <windows.h>#include <ntdll.h>#ifdef _WIN64#define captionMsg L"Application 64-bit"#else#define captionMsg L"Application 32-bit"#endifint WINAPI iWinMain() {    //Boolean to check after calling RtlAdjustPrivilege.    BOOLEAN bPreviousPrivilegeStatus;     RtlAdjustPrivilege(        SE_DEBUG_PRIVILEGE,        FALSE, // avoid to adjust privilege (DISABLE IT).        FALSE,        &bPreviousPrivilegeStatus);// check if SE_DEBUG_PRIVILEGE was already acquired then voluntary crash the application,// by calling memset with invalid pointer as parameter.            if (bPreviousPrivilegeStatus)         memset(NULL, 0, 1); //<-- BOOM! PADA BOOM!!!    MessageBoxW(        NULL,        L"Nothing!",        captionMsg,        MB_ICONINFORMATION);    return 0;}  

Attached: Source, screenshots and binary (32bit/64bit)

RtlAdjustPrivilege.rar

doesnt seem to work, if i debug it in olly, i still get the 'nothing' messagebox.. olly 1.10 unmodified and windows 7 x64

doesnt seem to work, if i debug it in olly, i still get the 'nothing' messagebox.. olly 1.10 unmodified and windows 7 x64

 

Thanks for reporting...

Already tested under Windows 8.1 X64 + OllyDbg 1.10 (Virgin and with Plugins)

 

Can you provide your debugging steps ?

 

See attached flash demo...

demo.rar

RtlAdjustPriv() moves around and isn't always in ntdll, and should return a "driver not loaded" error when it isn't (like on many modern kernels). This kind of stuff is a bit frowned upon, behavior is unpredictable.

Seems to work as-described in Olly 1.10+Win7 x64.

To prevent debugger being detected, you can use ScyllaHide and check "Remove Debug Privileges".

On x86 & x64 7601.17514 it wont work. I think at least 2 other builds it wont work on either, in both win 7 and win 8 (although may or may not be due to ntdll)

 

Anyway, detecting a debugger by checking for debug privileges is really bad. It is very unreliable on Vista+, because of UAC!

Yes, it seems that the desired result is uncontrollable, and some conditions which must be fulfilled, such Run as administrator (UAC) and debug privilege which must already acquired by the Debugger...

As mentioned by Archer (Exetools forum) there are similarity with detecting the debugger by trying to open "csrss.exe" process with PROCESS_ALL_ACCESS as parameter (debug privilege needed) also limited by the same conditions mentioned above.

#include <windows.h>#include <ntdll.h>#ifdef _WIN64#define captionMsg L"64-bit Application"#else#define captionMsg L"32-bit Application"#endifint WINAPI iWinMain() {    HANDLE ProcessHandle = NULL;    OBJECT_ATTRIBUTES ObjectAttributes;    CLIENT_ID ClientId;    ObjectAttributes.Length = sizeof(OBJECT_ATTRIBUTES);    ObjectAttributes.RootDirectory = 0;    ObjectAttributes.ObjectName = NULL;    ObjectAttributes.Attributes = OBJ_CASE_INSENSITIVE;    ObjectAttributes.SecurityDescriptor = NULL;    ObjectAttributes.SecurityQualityOfService = NULL;    ClientId.UniqueProcess = CsrGetProcessId(); // getting "csrss.exe" ProcessId.    ClientId.UniqueThread = 0;    NtOpenProcess(        &ProcessHandle,        PROCESS_ALL_ACCESS, // This parameter need SeDebugPrivilege.        &ObjectAttributes,        &ClientId);    if (ProcessHandle != NULL)        memset(NULL, 0, 1); //<-- BOOM! PADA BOOM!!!    MessageBoxW(        NULL,        L"Nothing!",        captionMsg,        MB_ICONINFORMATION);    return 0;}

Then to complete this topic (Debug Privilege), attached second sample based on "csrss.exe" process handling.Regards

csrssDBG.rar

"The kernel32 OpenProcess() function (or the ntdll NtOpenProcess() function) has at times been claimed to detect the presence of a debugger when used on the "csrss.exe" process. This is incorrect. While it is true that the function call will succeed in the presence of some debuggers, this is due to a side-effect of the debugger's behaviour (specifically, acquiring the debug privilege), and not due to the debugger itself (this should be obvious since the function call does not succeed when used with certain debuggers). All it reveals is that the user account for the process is a member of the administrators group and it has the debug privilege."

Required reading (from 2011!): http://pferrie.host22.com/papers/antidebug.pdf

In this case, section 7.B.i. The acquisition of privileges is discussed there, too.