Intravision/reactos
1
0
Fork 0
mirror of https://github.com/reactos/reactos.git synced 2024-07-06 04:35:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
reactos/ntoskrnl/include/internal/i386/trap_x.h

422 lines
12 KiB
C
Raw Normal View History

Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/*
* PROJECT: ReactOS Kernel
* LICENSE: BSD - See COPYING.ARM in the top level directory
[NTOS] Improve the FILE header section. Brought to you by Adam Stachowicz. CORE-10114 svn path=/trunk/; revision=69439
2015-10-04 11:54:25 +00:00
* FILE: ntoskrnl/include/internal/i386/trap_x.h
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
* PURPOSE: Internal Inlined Functions for the Trap Handling Code
* PROGRAMMERS: ReactOS Portable Systems Group
*/
- Move from using include guards to pragma once. - Whilst this is non-standard and potentially makes the code less portable, it's supported by both gcc and msvc and should bringg increased compilation speed due to its optimized behaviour (doesn't need to invoke the preprocessor) - Patch by Amine Khaldi [amine.khaldi@reactos.org] svn path=/trunk/; revision=45685
2010-02-26 11:43:19 +00:00
#pragma once
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
Stop using ATTRIB_NORETURN in hal and kernel, replace with the UNREACHABLE macro that both compilers can understand. Also fix up the UNREACHABLE def a bit. svn path=/trunk/; revision=69220
2015-09-13 20:37:28 +00:00
#if defined(_MSC_VER)
#define UNREACHABLE __assume(0)
#define __builtin_expect(a,b) (a)
#elif defined(__GNUC__)
#define UNREACHABLE __builtin_unreachable()
#else
#error
Comment out DECLSPEC_NORETURN for now. svn path=/trunk/; revision=45230
2010-01-24 02:25:20 +00:00
#endif
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
//
// Helper Code
//
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
BOOLEAN
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
KiUserTrap(IN PKTRAP_FRAME TrapFrame)
{
/* Anything else but Ring 0 is Ring 3 */
[NTOSKRNL] - Fix incorrect KiUserTrap which returned 3 for UserMode traps. Kind-of spotted by Jerome, but fixed in the wrong place...ReactOS-style, as always. -Patch by Alex Ionescu svn path=/trunk/; revision=52366
2011-06-19 13:55:28 +00:00
return !!(TrapFrame->SegCs & MODE_MASK);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
}
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
//
// Debug Macros
//
[FREELDR]: Jump to a standard 1-parameter STDCALL kernel entrypoint instead of a FASTCALL double-parameter entrypoint. [NTOS]: Make KiSystemStartup the real C entrypoint of the kernel, and move the "Am I being booted by FreeLDR" logic inside it -- it will then call KiRosPrepareForSystemStartup as earlier. [NTOS]: Move the Double Fault and Boot Stack declaration in C code, with the proper alignment attribute. [NTOS]: Although the concern that KiSystemStartup cannot be 100% C since it modifies ESP is real (Thomas' original fix of Alex's code), we don't need that much of it in assembly. Instead, write a simple trampoline (KiSwitchToBootStack) inline which switches stacks and jumps to a second-stage C function. [NTOS]: Completely remove boot.S as it isn't needed anymore, ReactOS startup is back to being (nearly) 100% C. svn path=/trunk/; revision=45152
2010-01-19 18:27:24 +00:00
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiDumpTrapFrame(IN PKTRAP_FRAME TrapFrame)
{
/* Dump the whole thing */
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("DbgEbp: %x\n", TrapFrame->DbgEbp);
DbgPrint("DbgEip: %x\n", TrapFrame->DbgEip);
DbgPrint("DbgArgMark: %x\n", TrapFrame->DbgArgMark);
DbgPrint("DbgArgPointer: %x\n", TrapFrame->DbgArgPointer);
DbgPrint("TempSegCs: %x\n", TrapFrame->TempSegCs);
DbgPrint("TempEsp: %x\n", TrapFrame->TempEsp);
DbgPrint("Dr0: %x\n", TrapFrame->Dr0);
DbgPrint("Dr1: %x\n", TrapFrame->Dr1);
DbgPrint("Dr2: %x\n", TrapFrame->Dr2);
DbgPrint("Dr3: %x\n", TrapFrame->Dr3);
DbgPrint("Dr6: %x\n", TrapFrame->Dr6);
DbgPrint("Dr7: %x\n", TrapFrame->Dr7);
DbgPrint("SegGs: %x\n", TrapFrame->SegGs);
DbgPrint("SegEs: %x\n", TrapFrame->SegEs);
DbgPrint("SegDs: %x\n", TrapFrame->SegDs);
DbgPrint("Edx: %x\n", TrapFrame->Edx);
DbgPrint("Ecx: %x\n", TrapFrame->Ecx);
DbgPrint("Eax: %x\n", TrapFrame->Eax);
DbgPrint("PreviousPreviousMode: %x\n", TrapFrame->PreviousPreviousMode);
[NTOSKRNL] - Fix some DPRINTs. Spotted by VS11. svn path=/trunk/; revision=55161
2012-01-25 00:09:43 +00:00
DbgPrint("ExceptionList: %p\n", TrapFrame->ExceptionList);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("SegFs: %x\n", TrapFrame->SegFs);
DbgPrint("Edi: %x\n", TrapFrame->Edi);
DbgPrint("Esi: %x\n", TrapFrame->Esi);
DbgPrint("Ebx: %x\n", TrapFrame->Ebx);
DbgPrint("Ebp: %x\n", TrapFrame->Ebp);
DbgPrint("ErrCode: %x\n", TrapFrame->ErrCode);
DbgPrint("Eip: %x\n", TrapFrame->Eip);
DbgPrint("SegCs: %x\n", TrapFrame->SegCs);
DbgPrint("EFlags: %x\n", TrapFrame->EFlags);
DbgPrint("HardwareEsp: %x\n", TrapFrame->HardwareEsp);
DbgPrint("HardwareSegSs: %x\n", TrapFrame->HardwareSegSs);
DbgPrint("V86Es: %x\n", TrapFrame->V86Es);
DbgPrint("V86Ds: %x\n", TrapFrame->V86Ds);
DbgPrint("V86Fs: %x\n", TrapFrame->V86Fs);
DbgPrint("V86Gs: %x\n", TrapFrame->V86Gs);
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
- Implement super-simple KeSweepICache that always flushes the entire instruction cache. Use it in KD64 after modifying memory to make sure CPU gets the correct code to execute, and in NtFlushInstructionCache. May improve breakpoints somewhat. - Move NtFlushInstructionCache from sysinfo.c to virtual.c where it fits better. Likewise, move it from kefuncs to mmfuncs in NDK, and fix function arguments (ULONG -> SIZE_T). - Re-enable TRAP_DEBUG, adding back critical checks in the trap code. Checks can be improved but it is better than potentially silently messing up system state. - Move remaining RtlPrefetchMemoryNonTemporal code into kernel. Stubbed for non-x86. - By Hermes suggestion, override ASSERT to NT_ASSERT only for MSVC builds as that is where the main benefit is. svn path=/trunk/; revision=68907
2015-09-01 23:45:48 +00:00
#if DBG
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
- Implement super-simple KeSweepICache that always flushes the entire instruction cache. Use it in KD64 after modifying memory to make sure CPU gets the correct code to execute, and in NtFlushInstructionCache. May improve breakpoints somewhat. - Move NtFlushInstructionCache from sysinfo.c to virtual.c where it fits better. Likewise, move it from kefuncs to mmfuncs in NDK, and fix function arguments (ULONG -> SIZE_T). - Re-enable TRAP_DEBUG, adding back critical checks in the trap code. Checks can be improved but it is better than potentially silently messing up system state. - Move remaining RtlPrefetchMemoryNonTemporal code into kernel. Stubbed for non-x86. - By Hermes suggestion, override ASSERT to NT_ASSERT only for MSVC builds as that is where the main benefit is. svn path=/trunk/; revision=68907
2015-09-01 23:45:48 +00:00
VOID
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiFillTrapFrameDebug(IN PKTRAP_FRAME TrapFrame)
{
/* Set the debug information */
TrapFrame->DbgArgPointer = TrapFrame->Edx;
TrapFrame->DbgArgMark = 0xBADB0D00;
TrapFrame->DbgEip = TrapFrame->Eip;
[NTOSKRNL] Move x86 specific file trap_x.h into include/internal/i386 svn path=/trunk/; revision=53699
2011-09-13 12:53:50 +00:00
TrapFrame->DbgEbp = TrapFrame->Ebp;
[NTOS:KE] - Make decoding of the error code more obvious in KiTrap0EHandler - Avoid or make explicit some signed/unsigned conversions svn path=/trunk/; revision=74304
2017-04-14 10:39:07 +00:00
TrapFrame->PreviousPreviousMode = (ULONG)-1;
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
#define DR7_RESERVED_READ_AS_1 0x400
#define CheckDr(DrNumner, ExpectedValue) \
{ \
ULONG DrValue = __readdr(DrNumner); \
if (DrValue != (ExpectedValue)) \
{ \
DbgPrint("Dr%ld: expected %.8lx, got %.8lx\n", \
DrNumner, ExpectedValue, DrValue); \
__debugbreak(); \
} \
}
extern BOOLEAN StopChecking;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiExitTrapDebugChecks(IN PKTRAP_FRAME TrapFrame,
[NTOSKRNL] - Make Trap debugging usable again. Suggested by Stefan. svn path=/trunk/; revision=55701
2012-02-19 08:38:09 +00:00
IN BOOLEAN SkipPreviousMode)
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Don't check recursively */
if (StopChecking) return;
StopChecking = TRUE;
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* Make sure interrupts are disabled */
if (__readeflags() & EFLAGS_INTERRUPT_MASK)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with interrupts enabled: %lx\n", __readeflags());
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* Make sure this is a real trap frame */
if (TrapFrame->DbgArgMark != 0xBADB0D00)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with an invalid trap frame? (No MAGIC in trap frame)\n");
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiDumpTrapFrame(TrapFrame);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* Make sure we're not in user-mode or something */
if (Ke386GetFs() != KGDT_R0_PCR)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with an invalid FS: %lx\n", Ke386GetFs());
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* Make sure we have a valid SEH chain */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
if (KeGetPcr()->NtTib.ExceptionList == 0)
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
DbgPrint("Exiting with NULL exception chain: %p\n", KeGetPcr()->NtTib.ExceptionList);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* Make sure we're restoring a valid SEH chain */
if (TrapFrame->ExceptionList == 0)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Entered a trap with a NULL exception chain: %p\n", TrapFrame->ExceptionList);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/* If we're ignoring previous mode, make sure caller doesn't actually want it */
[NTOSKRNL] - Make Trap debugging usable again. Suggested by Stefan. svn path=/trunk/; revision=55701
2012-02-19 08:38:09 +00:00
if (SkipPreviousMode && (TrapFrame->PreviousPreviousMode != -1))
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
DbgPrint("Exiting a trap witout restoring previous mode, yet previous mode seems valid: %lx\n", TrapFrame->PreviousPreviousMode);
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
Completely disable the debug register assertions in the trap exit code for KDBG as it sets them improperly. These checks should not have been added for KDBG to begin with and would have made any usage of address breakpoints inside KDBG end up in a trap exit breakpoint. Whoever knows KDBG better than I do please investigate. However, enable the DR7 debug check for KD as it handles the debug registers just fine. The likelihood of loading a driver that modifies a debug register (which would be the only other thing triggering this) is pretty slim. svn path=/trunk/; revision=69244
2015-09-15 17:55:14 +00:00
/* FIXME: KDBG messes around with these improperly */
#if !defined(KDBG)
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Check DR values */
[NTOSKRNL] Fix a typo I introduced in TRAP_DEBUG only code, spotted by Thomas svn path=/trunk/; revision=60702
2013-10-19 11:40:00 +00:00
if (KiUserTrap(TrapFrame))
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
{
/* Check for active debugging */
if (KeGetCurrentThread()->Header.DebugActive)
{
if ((TrapFrame->Dr7 & ~DR7_RESERVED_MASK) == 0) __debugbreak();
CheckDr(0, TrapFrame->Dr0);
CheckDr(1, TrapFrame->Dr1);
CheckDr(2, TrapFrame->Dr2);
CheckDr(3, TrapFrame->Dr3);
CheckDr(7, TrapFrame->Dr7 | DR7_RESERVED_READ_AS_1);
}
}
else
{
PKPRCB Prcb = KeGetCurrentPrcb();
CheckDr(0, Prcb->ProcessorState.SpecialRegisters.KernelDr0);
CheckDr(1, Prcb->ProcessorState.SpecialRegisters.KernelDr1);
CheckDr(2, Prcb->ProcessorState.SpecialRegisters.KernelDr2);
CheckDr(3, Prcb->ProcessorState.SpecialRegisters.KernelDr3);
[NTOS] Disable again the DR7 check until we correctly understand why bit 10 of DR7 (reserved) is almost always set (this lead to MSVC-built ReactOS crashing when being starting in non-debugging mode). See also r69244. CORE-10165 #comment DR7 check again disabled in r70418. svn path=/trunk/; revision=70418
2015-12-26 00:29:07 +00:00
// CheckDr(7, Prcb->ProcessorState.SpecialRegisters.KernelDr7); // Disabled, see CORE-10165 for more details.
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
}
Completely disable the debug register assertions in the trap exit code for KDBG as it sets them improperly. These checks should not have been added for KDBG to begin with and would have made any usage of address breakpoints inside KDBG end up in a trap exit breakpoint. Whoever knows KDBG better than I do please investigate. However, enable the DR7 debug check for KD as it handles the debug registers just fine. The likelihood of loading a driver that modifies a debug register (which would be the only other thing triggering this) is pretty slim. svn path=/trunk/; revision=69244
2015-09-15 17:55:14 +00:00
#endif
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
StopChecking = FALSE;
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
[NTOS:KE] - Unconditionally perform APC and IRQL checks on user call exit svn path=/trunk/; revision=63143
2014-05-04 08:42:02 +00:00
#else
#define KiExitTrapDebugChecks(x, y)
#define KiFillTrapFrameDebug(x)
#endif
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
KiExitSystemCallDebugChecks(IN ULONG SystemCall,
IN PKTRAP_FRAME TrapFrame)
{
KIRQL OldIrql;
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
/* Check if this was a user call */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
if (KiUserTrap(TrapFrame))
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
{
/* Make sure we are not returning with elevated IRQL */
OldIrql = KeGetCurrentIrql();
if (OldIrql != PASSIVE_LEVEL)
{
/* Forcibly put us in a sane state */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
KeGetPcr()->Irql = PASSIVE_LEVEL;
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
_disable();
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
/* Fail */
KeBugCheckEx(IRQL_GT_ZERO_AT_SYSTEM_SERVICE,
SystemCall,
OldIrql,
0,
0);
}
[NTOS] Re-enable the APC debug check in the system call exit code that somehow got removed in 46247. [NTOS] Use an inline for emitting the iret instruction in C code for portability. [NTOS] Simplify the MSC assembly in KiSwitchToBootStack. svn path=/trunk/; revision=47487
2010-05-31 12:52:16 +00:00
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
/* Make sure we're not attached and that APCs are not disabled */
[NTOSKRNL]: When you rewrite my ASM code, get it right, dumbasses. Thanks Stefan. svn path=/trunk/; revision=55690
2012-02-19 00:08:50 +00:00
if ((KeGetCurrentThread()->ApcStateIndex != OriginalApcEnvironment) ||
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
(KeGetCurrentThread()->CombinedApcDisable != 0))
{
/* Fail */
KeBugCheckEx(APC_INDEX_MISMATCH,
SystemCall,
KeGetCurrentThread()->ApcStateIndex,
KeGetCurrentThread()->CombinedApcDisable,
0);
}
}
}
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Generic Exit Routine
//
[NTOS] Fix compilation with MSVC. Dedicated to Stefan. svn path=/trunk/; revision=46264
2010-03-18 16:30:54 +00:00
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallSysExitReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiEditedTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiTrapReturnNoSegments(IN PKTRAP_FRAME TrapFrame);
[NTOSKRNL] - Introduce a new and faster way to return to kernel mode from traps by using a ret 8 instruction instead of an iret. - Make use of KiUserTrap where appropriate - Remove some pointless toplevel volatile svn path=/trunk/; revision=60701
2013-10-19 11:33:34 +00:00
DECLSPEC_NORETURN VOID FASTCALL KiTrapReturnNoSegmentsRet8(IN PKTRAP_FRAME TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
typedef
VOID
[NTOS] Re-enable the APC debug check in the system call exit code that somehow got removed in 46247. [NTOS] Use an inline for emitting the iret instruction in C code for portability. [NTOS] Simplify the MSC assembly in KiSwitchToBootStack. svn path=/trunk/; revision=47487
2010-05-31 12:52:16 +00:00
(FASTCALL *PFAST_SYSTEM_CALL_EXIT)(
IN PKTRAP_FRAME TrapFrame
);
[NTOS]: Rework the trap macros again. This time we needed some added complexity because of the need for runtime patching at boot. We got away with it nicely and basically implement a simple "if" in ASM, which gives us a deterministic set of instructions (vs the compiler). We then patch if if needed (see next bullet). [NTOS]: Support systems without SYSENTER. Nothing was actually disabling the SYSEXIT mechanism recently enabled. Now IRET is always used unless a SYSENTER machine is discovered, in which case the jmp to the IRET code is patched with a jmp to SYSEXIT code. [PERF]: Set WP0 bit in CR0 later, thus allowing the existing runtime patches (and this new one) to function without requiring MmSetPageProtect. Saves TLB flushes and page table walking/mapping on boot. Left in some debug prints to see what buildbot reports...will remove if it survives. svn path=/trunk/; revision=45280
2010-01-27 03:05:10 +00:00
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
extern PFAST_SYSTEM_CALL_EXIT KiFastCallExitHandler;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
//
// Save user mode debug registers and restore kernel values
//
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
KiHandleDebugRegistersOnTrapEntry(
IN PKTRAP_FRAME TrapFrame)
{
PKPRCB Prcb = KeGetCurrentPrcb();
/* Save all debug registers in the trap frame */
TrapFrame->Dr0 = __readdr(0);
TrapFrame->Dr1 = __readdr(1);
TrapFrame->Dr2 = __readdr(2);
TrapFrame->Dr3 = __readdr(3);
TrapFrame->Dr6 = __readdr(6);
TrapFrame->Dr7 = __readdr(7);
/* Disable all active debugging */
__writedr(7, 0);
/* Restore kernel values */
__writedr(0, Prcb->ProcessorState.SpecialRegisters.KernelDr0);
__writedr(1, Prcb->ProcessorState.SpecialRegisters.KernelDr1);
__writedr(2, Prcb->ProcessorState.SpecialRegisters.KernelDr2);
__writedr(3, Prcb->ProcessorState.SpecialRegisters.KernelDr3);
__writedr(6, Prcb->ProcessorState.SpecialRegisters.KernelDr6);
__writedr(7, Prcb->ProcessorState.SpecialRegisters.KernelDr7);
}
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
KiHandleDebugRegistersOnTrapExit(
PKTRAP_FRAME TrapFrame)
{
/* Disable all active debugging */
__writedr(7, 0);
/* Load all debug registers from the trap frame */
__writedr(0, TrapFrame->Dr0);
__writedr(1, TrapFrame->Dr1);
__writedr(2, TrapFrame->Dr2);
__writedr(3, TrapFrame->Dr3);
__writedr(6, TrapFrame->Dr6);
__writedr(7, TrapFrame->Dr7);
}
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Virtual 8086 Mode Optimized Trap Exit
//
FORCEINLINE
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
DECLSPEC_NORETURN
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiExitV86Trap(IN PKTRAP_FRAME TrapFrame)
{
PKTHREAD Thread;
KIRQL OldIrql;
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Get the thread */
Thread = KeGetCurrentThread();
while (TRUE)
{
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
/* Return if this isn't V86 mode anymore */
[NTOS] Re-enable the APC debug check in the system call exit code that somehow got removed in 46247. [NTOS] Use an inline for emitting the iret instruction in C code for portability. [NTOS] Simplify the MSC assembly in KiSwitchToBootStack. svn path=/trunk/; revision=47487
2010-05-31 12:52:16 +00:00
if (!(TrapFrame->EFlags & EFLAGS_V86_MASK)) KiEoiHelper(TrapFrame);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Turn off the alerted state for kernel mode */
Thread->Alerted[KernelMode] = FALSE;
/* Are there pending user APCs? */
if (__builtin_expect(!Thread->ApcState.UserApcPending, 1)) break;
/* Raise to APC level and enable interrupts */
OldIrql = KfRaiseIrql(APC_LEVEL);
_enable();
/* Deliver APCs */
KiDeliverApc(UserMode, NULL, TrapFrame);
/* Restore IRQL and disable interrupts once again */
KfLowerIrql(OldIrql);
_disable();
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* If we got here, we're still in a valid V8086 context, so quit it */
if (__builtin_expect(TrapFrame->Dr7 & ~DR7_RESERVED_MASK, 0))
{
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Restore debug registers from the trap frame */
KiHandleDebugRegistersOnTrapExit(TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Return from interrupt */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
KiTrapReturnNoSegments(TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
//
// Virtual 8086 Mode Optimized Trap Entry
//
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiEnterV86Trap(IN PKTRAP_FRAME TrapFrame)
{
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
/* Save exception list */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Save DR7 and check for debugging */
TrapFrame->Dr7 = __readdr(7);
if (__builtin_expect(TrapFrame->Dr7 & ~DR7_RESERVED_MASK, 0))
{
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Handle debug registers */
KiHandleDebugRegistersOnTrapEntry(TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
}
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Interrupt Trap Entry
//
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiEnterInterruptTrap(IN PKTRAP_FRAME TrapFrame)
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Save exception list and terminate it */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
KeGetPcr()->NtTib.ExceptionList = EXCEPTION_CHAIN_END;
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Default to debugging disabled */
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
TrapFrame->Dr7 = 0;
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Check if the frame was from user mode or v86 mode */
[NTOSKRNL] - Introduce a new and faster way to return to kernel mode from traps by using a ret 8 instruction instead of an iret. - Make use of KiUserTrap where appropriate - Remove some pointless toplevel volatile svn path=/trunk/; revision=60701
2013-10-19 11:33:34 +00:00
if (KiUserTrap(TrapFrame) ||
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
(TrapFrame->EFlags & EFLAGS_V86_MASK))
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
{
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Check for active debugging */
if (KeGetCurrentThread()->Header.DebugActive & 0xFF)
{
/* Handle debug registers */
KiHandleDebugRegistersOnTrapEntry(TrapFrame);
}
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Set debug header */
KiFillTrapFrameDebug(TrapFrame);
}
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Generic Trap Entry
//
FORCEINLINE
[NTOS] - FORCEINLINE belongs before the return type (-Wold-style-declaration) svn path=/trunk/; revision=61098
2013-11-26 13:45:33 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiEnterTrap(IN PKTRAP_FRAME TrapFrame)
{
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
/* Save exception list */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Default to debugging disabled */
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
TrapFrame->Dr7 = 0;
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Check if the frame was from user mode or v86 mode */
[NTOSKRNL] - Introduce a new and faster way to return to kernel mode from traps by using a ret 8 instruction instead of an iret. - Make use of KiUserTrap where appropriate - Remove some pointless toplevel volatile svn path=/trunk/; revision=60701
2013-10-19 11:33:34 +00:00
if (KiUserTrap(TrapFrame) ||
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
(TrapFrame->EFlags & EFLAGS_V86_MASK))
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
{
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
/* Check for active debugging */
if (KeGetCurrentThread()->Header.DebugActive & 0xFF)
{
/* Handle debug registers */
KiHandleDebugRegistersOnTrapEntry(TrapFrame);
}
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
[NTOSKRNL] - Implement saving / restoring debug registers on traps - Replace the loop in KeContextToTrapFrame with something less ridiculous - fixes a number of ntdd exception winetests svn path=/trunk/; revision=56357
2012-04-18 13:39:19 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Set debug header */
KiFillTrapFrameDebug(TrapFrame);
[FREELDR]: Jump to a standard 1-parameter STDCALL kernel entrypoint instead of a FASTCALL double-parameter entrypoint. [NTOS]: Make KiSystemStartup the real C entrypoint of the kernel, and move the "Am I being booted by FreeLDR" logic inside it -- it will then call KiRosPrepareForSystemStartup as earlier. [NTOS]: Move the Double Fault and Boot Stack declaration in C code, with the proper alignment attribute. [NTOS]: Although the concern that KiSystemStartup cannot be 100% C since it modifies ESP is real (Thomas' original fix of Alex's code), we don't need that much of it in assembly. Instead, write a simple trampoline (KiSwitchToBootStack) inline which switches stacks and jumps to a second-stage C function. [NTOS]: Completely remove boot.S as it isn't needed anymore, ReactOS startup is back to being (nearly) 100% C. svn path=/trunk/; revision=45152
2010-01-19 18:27:24 +00:00
}
Reference in a new issue Copy permalink