Intravision/reactos
1
0
Fork 0
mirror of https://github.com/reactos/reactos.git synced 2024-10-30 19:41:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
reactos/ntoskrnl/include/internal/trap_x.h

320 lines
9.1 KiB
C
Raw Normal View History

Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
/*
* PROJECT: ReactOS Kernel
* LICENSE: BSD - See COPYING.ARM in the top level directory
* FILE: ntoskrnl/include/trap_x.h
* PURPOSE: Internal Inlined Functions for the Trap Handling Code
* PROGRAMMERS: ReactOS Portable Systems Group
*/
- Move from using include guards to pragma once. - Whilst this is non-standard and potentially makes the code less portable, it's supported by both gcc and msvc and should bringg increased compilation speed due to its optimized behaviour (doesn't need to invoke the preprocessor) - Patch by Amine Khaldi [amine.khaldi@reactos.org] svn path=/trunk/; revision=45685
2010-02-26 11:43:19 +00:00
#pragma once
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
//#define TRAP_DEBUG 1
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
- Use __builtin_trap for old GCC, too. - KiEoiHelper, KiDispatchExceptionFromTrapFrame: mark noreturn. - Return DECLSPEC_NORETURNs. svn path=/trunk/; revision=45232
2010-01-24 15:18:50 +00:00
// Unreachable code hint for GCC 4.5.x, older GCC versions, and MSVC
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
- Use __builtin_trap for old GCC, too. - KiEoiHelper, KiDispatchExceptionFromTrapFrame: mark noreturn. - Return DECLSPEC_NORETURNs. svn path=/trunk/; revision=45232
2010-01-24 15:18:50 +00:00
#ifdef __GNUC__
Trying to fix build... svn path=/trunk/; revision=45225
2010-01-24 01:35:25 +00:00
#if __GNUC__ * 100 + __GNUC_MINOR__ >= 405
#define UNREACHABLE __builtin_unreachable()
- Use __builtin_trap for old GCC, too. - KiEoiHelper, KiDispatchExceptionFromTrapFrame: mark noreturn. - Return DECLSPEC_NORETURNs. svn path=/trunk/; revision=45232
2010-01-24 15:18:50 +00:00
#else
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
#define UNREACHABLE __builtin_trap()
- Use __builtin_trap for old GCC, too. - KiEoiHelper, KiDispatchExceptionFromTrapFrame: mark noreturn. - Return DECLSPEC_NORETURNs. svn path=/trunk/; revision=45232
2010-01-24 15:18:50 +00:00
#endif
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
#elif _MSC_VER
#define UNREACHABLE __assume(0)
Trying r45222 hack again... svn path=/trunk/; revision=45228
2010-01-24 01:58:20 +00:00
#else
Comment out DECLSPEC_NORETURN for now. svn path=/trunk/; revision=45230
2010-01-24 02:25:20 +00:00
#define UNREACHABLE
#endif
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
//
// Helper Code
//
BOOLEAN
FORCEINLINE
KiUserTrap(IN PKTRAP_FRAME TrapFrame)
{
/* Anything else but Ring 0 is Ring 3 */
return (TrapFrame->SegCs & MODE_MASK);
}
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
//
// Debug Macros
//
VOID
[FREELDR]: Jump to a standard 1-parameter STDCALL kernel entrypoint instead of a FASTCALL double-parameter entrypoint. [NTOS]: Make KiSystemStartup the real C entrypoint of the kernel, and move the "Am I being booted by FreeLDR" logic inside it -- it will then call KiRosPrepareForSystemStartup as earlier. [NTOS]: Move the Double Fault and Boot Stack declaration in C code, with the proper alignment attribute. [NTOS]: Although the concern that KiSystemStartup cannot be 100% C since it modifies ESP is real (Thomas' original fix of Alex's code), we don't need that much of it in assembly. Instead, write a simple trampoline (KiSwitchToBootStack) inline which switches stacks and jumps to a second-stage C function. [NTOS]: Completely remove boot.S as it isn't needed anymore, ReactOS startup is back to being (nearly) 100% C. svn path=/trunk/; revision=45152
2010-01-19 18:27:24 +00:00
FORCEINLINE
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiDumpTrapFrame(IN PKTRAP_FRAME TrapFrame)
{
/* Dump the whole thing */
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("DbgEbp: %x\n", TrapFrame->DbgEbp);
DbgPrint("DbgEip: %x\n", TrapFrame->DbgEip);
DbgPrint("DbgArgMark: %x\n", TrapFrame->DbgArgMark);
DbgPrint("DbgArgPointer: %x\n", TrapFrame->DbgArgPointer);
DbgPrint("TempSegCs: %x\n", TrapFrame->TempSegCs);
DbgPrint("TempEsp: %x\n", TrapFrame->TempEsp);
DbgPrint("Dr0: %x\n", TrapFrame->Dr0);
DbgPrint("Dr1: %x\n", TrapFrame->Dr1);
DbgPrint("Dr2: %x\n", TrapFrame->Dr2);
DbgPrint("Dr3: %x\n", TrapFrame->Dr3);
DbgPrint("Dr6: %x\n", TrapFrame->Dr6);
DbgPrint("Dr7: %x\n", TrapFrame->Dr7);
DbgPrint("SegGs: %x\n", TrapFrame->SegGs);
DbgPrint("SegEs: %x\n", TrapFrame->SegEs);
DbgPrint("SegDs: %x\n", TrapFrame->SegDs);
DbgPrint("Edx: %x\n", TrapFrame->Edx);
DbgPrint("Ecx: %x\n", TrapFrame->Ecx);
DbgPrint("Eax: %x\n", TrapFrame->Eax);
DbgPrint("PreviousPreviousMode: %x\n", TrapFrame->PreviousPreviousMode);
DbgPrint("ExceptionList: %x\n", TrapFrame->ExceptionList);
DbgPrint("SegFs: %x\n", TrapFrame->SegFs);
DbgPrint("Edi: %x\n", TrapFrame->Edi);
DbgPrint("Esi: %x\n", TrapFrame->Esi);
DbgPrint("Ebx: %x\n", TrapFrame->Ebx);
DbgPrint("Ebp: %x\n", TrapFrame->Ebp);
DbgPrint("ErrCode: %x\n", TrapFrame->ErrCode);
DbgPrint("Eip: %x\n", TrapFrame->Eip);
DbgPrint("SegCs: %x\n", TrapFrame->SegCs);
DbgPrint("EFlags: %x\n", TrapFrame->EFlags);
DbgPrint("HardwareEsp: %x\n", TrapFrame->HardwareEsp);
DbgPrint("HardwareSegSs: %x\n", TrapFrame->HardwareSegSs);
DbgPrint("V86Es: %x\n", TrapFrame->V86Es);
DbgPrint("V86Ds: %x\n", TrapFrame->V86Ds);
DbgPrint("V86Fs: %x\n", TrapFrame->V86Fs);
DbgPrint("V86Gs: %x\n", TrapFrame->V86Gs);
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
#ifdef TRAP_DEBUG
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiFillTrapFrameDebug(IN PKTRAP_FRAME TrapFrame)
{
/* Set the debug information */
TrapFrame->DbgArgPointer = TrapFrame->Edx;
TrapFrame->DbgArgMark = 0xBADB0D00;
TrapFrame->DbgEip = TrapFrame->Eip;
TrapFrame->DbgEbp = TrapFrame->Ebp;
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
TrapFrame->PreviousPreviousMode = -1;
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiExitTrapDebugChecks(IN PKTRAP_FRAME TrapFrame,
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
IN KTRAP_EXIT_SKIP_BITS SkipBits)
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
/* Make sure interrupts are disabled */
if (__readeflags() & EFLAGS_INTERRUPT_MASK)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with interrupts enabled: %lx\n", __readeflags());
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
/* Make sure this is a real trap frame */
if (TrapFrame->DbgArgMark != 0xBADB0D00)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with an invalid trap frame? (No MAGIC in trap frame)\n");
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
KiDumpTrapFrame(TrapFrame);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
/* Make sure we're not in user-mode or something */
if (Ke386GetFs() != KGDT_R0_PCR)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Exiting with an invalid FS: %lx\n", Ke386GetFs());
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
/* Make sure we have a valid SEH chain */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
if (KeGetPcr()->NtTib.ExceptionList == 0)
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
DbgPrint("Exiting with NULL exception chain: %p\n", KeGetPcr()->NtTib.ExceptionList);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
/* Make sure we're restoring a valid SEH chain */
if (TrapFrame->ExceptionList == 0)
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
DbgPrint("Entered a trap with a NULL exception chain: %p\n", TrapFrame->ExceptionList);
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
/* If we're ignoring previous mode, make sure caller doesn't actually want it */
[NTOS]: Simplify trap exit code. [NTOS]: Move some stuff in appropriate headers. [NTOS]: Write V86 fast entry/exit traps, needed for later. svn path=/trunk/; revision=45028
2010-01-10 15:00:44 +00:00
if ((SkipBits.SkipPreviousMode) && (TrapFrame->PreviousPreviousMode != -1))
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
{
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
DbgPrint("Exiting a trap witout restoring previous mode, yet previous mode seems valid: %lx\n", TrapFrame->PreviousPreviousMode);
__debugbreak();
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
}
}
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
VOID
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
FORCEINLINE
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
KiExitSystemCallDebugChecks(IN ULONG SystemCall,
IN PKTRAP_FRAME TrapFrame)
{
KIRQL OldIrql;
/* Check if this was a user call */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
if (KiUserTrap(TrapFrame))
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
{
/* Make sure we are not returning with elevated IRQL */
OldIrql = KeGetCurrentIrql();
if (OldIrql != PASSIVE_LEVEL)
{
/* Forcibly put us in a sane state */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
KeGetPcr()->Irql = PASSIVE_LEVEL;
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
_disable();
/* Fail */
KeBugCheckEx(IRQL_GT_ZERO_AT_SYSTEM_SERVICE,
SystemCall,
OldIrql,
0,
0);
}
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
#if 0
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
/* Make sure we're not attached and that APCs are not disabled */
if ((KeGetCurrentThread()->ApcStateIndex != CurrentApcEnvironment) ||
(KeGetCurrentThread()->CombinedApcDisable != 0))
{
/* Fail */
KeBugCheckEx(APC_INDEX_MISMATCH,
SystemCall,
KeGetCurrentThread()->ApcStateIndex,
KeGetCurrentThread()->CombinedApcDisable,
0);
}
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
#endif
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
}
}
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
#else
#define KiExitTrapDebugChecks(x, y)
#define KiFillTrapFrameDebug(x)
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
#define KiExitSystemCallDebugChecks(x, y)
Trap Handlers in C Patch 1 of X (Patch by Sir_Richard <ros.arm@reactos.org>): [NTOS]: The kernel normally does not save FPU state during Ring 0 transitions since the FPU should not be used. The one exception is when a kernel debugger is attached. Unfortunately, the latter check in ReactOS results in even "print on the serial line" to count as "debugger attached", and thus FPU state was almost always saved, slowing down traps significantly. [NTOS]: The kernel also does not typically save DRx (debug) registers unless they were in use. During an exception dispatch, they are zeroed out, and later during trap exit, if any debug register is set, DR7 is updated to enable that hardware breakpoint. Unfortunately, the code to clear the debug registers had a bug: DR2 was never cleared. Because DR2 ended up being a random stack value during trap frame generation, this caused a bogus address to be added to DR2, and DR7 would then enable the 2nd hardware breakpoint. This caused the kernel to always save DRx state, which is slow, and worse, could cause random hardware breakpoints to fire. [NTOS]: Start implementing trap handling in C. ASM trap handlers will now only be 5 lines of assembly including a function call to a C handler. All C handling code uses maximum two arguments and is all FASTCALL for efficiency. [NTOS]: Implement C versions of TRAP_PROLOG and TRAP_EPILOG. Implement C version of Ki386EoiHelper. Implement C version of CommonDispatchException (and helper) and KiFatalSystemException. Implement C version of CHECK_FOR_APC_DELIVER. Implement trap debugging checks as a separate entity instead of always doing them. [NTOS]: Add missing intrinsics for DS/ES/GS segment query. The kernel is now ready for some trap handling to be done in C. Due to the FPU/Debug fixes and relaxation of paranoid debug checks, the C code will likely be faster than the original assembly. svn path=/trunk/; revision=45000
2010-01-08 15:04:19 +00:00
#endif
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Generic Exit Routine
//
[NTOS] Fix compilation with MSVC. Dedicated to Stefan. svn path=/trunk/; revision=46264
2010-03-18 16:30:54 +00:00
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallSysExitReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiSystemCallTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiEditedTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiTrapReturn(IN PKTRAP_FRAME TrapFrame);
DECLSPEC_NORETURN VOID FASTCALL KiTrapReturnNoSegments(IN PKTRAP_FRAME TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
typedef
VOID
(FASTCALL
[NTOS] Fix compilation with MSVC. Dedicated to Stefan. svn path=/trunk/; revision=46264
2010-03-18 16:30:54 +00:00
*PFAST_SYSTEM_CALL_EXIT)(IN PKTRAP_FRAME TrapFrame);
[NTOS]: Rework the trap macros again. This time we needed some added complexity because of the need for runtime patching at boot. We got away with it nicely and basically implement a simple "if" in ASM, which gives us a deterministic set of instructions (vs the compiler). We then patch if if needed (see next bullet). [NTOS]: Support systems without SYSENTER. Nothing was actually disabling the SYSEXIT mechanism recently enabled. Now IRET is always used unless a SYSENTER machine is discovered, in which case the jmp to the IRET code is patched with a jmp to SYSEXIT code. [PERF]: Set WP0 bit in CR0 later, thus allowing the existing runtime patches (and this new one) to function without requiring MmSetPageProtect. Saves TLB flushes and page table walking/mapping on boot. Left in some debug prints to see what buildbot reports...will remove if it survives. svn path=/trunk/; revision=45280
2010-01-27 03:05:10 +00:00
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
extern PFAST_SYSTEM_CALL_EXIT KiFastCallExitHandler;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Virtual 8086 Mode Optimized Trap Exit
//
VOID
FORCEINLINE
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
DECLSPEC_NORETURN
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiExitV86Trap(IN PKTRAP_FRAME TrapFrame)
{
PKTHREAD Thread;
KIRQL OldIrql;
/* Get the thread */
Thread = KeGetCurrentThread();
while (TRUE)
{
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
/* Return if this isn't V86 mode anymore */
if (!(TrapFrame->EFlags & EFLAGS_V86_MASK)) KiEoiHelper(TrapFrame);;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Turn off the alerted state for kernel mode */
Thread->Alerted[KernelMode] = FALSE;
/* Are there pending user APCs? */
if (__builtin_expect(!Thread->ApcState.UserApcPending, 1)) break;
/* Raise to APC level and enable interrupts */
OldIrql = KfRaiseIrql(APC_LEVEL);
_enable();
/* Deliver APCs */
KiDeliverApc(UserMode, NULL, TrapFrame);
/* Restore IRQL and disable interrupts once again */
KfLowerIrql(OldIrql);
_disable();
}
/* If we got here, we're still in a valid V8086 context, so quit it */
if (__builtin_expect(TrapFrame->Dr7 & ~DR7_RESERVED_MASK, 0))
{
/* Not handled yet */
DbgPrint("Need Hardware Breakpoint Support!\n");
while (TRUE);
}
/* Return from interrupt */
[NTOS] Rewrite Trap exits stubs in raw assembly (2nd try) and remove inline assembly. Bugcheck in KiTrap0DHandler, when the fault was not handled. Replace code patching for sysexit vs iret with a function pointer. Slightly refactor KiSystemServiceHandler/KiFastCallEntryHanlder. Unroll the loop in the systemcall handler and use volatile keyword when reloading TrapFrame and DescriptorTable from the new stack after converting to gui thread to prevent the compiler from optimizing it away (or moving it out of the loop). Add an asm macro KiCallHandler, which expands to call on debug builds to make sure backtraces work as expected and to jmp on release builds for improved performance. Modify KiExitV86Trap to always exit and add DECLSPEC_NORETURN. Use __debugbreak() instead of while(TRUE) on errors in KiExitTrapDebugChecks. The old code hat 2 issues: one was restoring segments in KiExitV86Trap when they shouldn't be, leading to a bugcheck. And the other was a long hang (5 mintes or more) in 3rd stage on qemu when selecting RosDbg, caused by the KiFastCallExitHandler function pointer being initialized with a pointer to the iret handler. Initializing it in code solved the issue. To figure out why is left as an exercise to the reader. svn path=/trunk/; revision=46247
2010-03-17 13:12:46 +00:00
KiTrapReturnNoSegments(TrapFrame);
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
}
//
// Virtual 8086 Mode Optimized Trap Entry
//
VOID
FORCEINLINE
KiEnterV86Trap(IN PKTRAP_FRAME TrapFrame)
{
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
/* Save exception list */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Save DR7 and check for debugging */
TrapFrame->Dr7 = __readdr(7);
if (__builtin_expect(TrapFrame->Dr7 & ~DR7_RESERVED_MASK, 0))
{
DbgPrint("Need Hardware Breakpoint Support!\n");
while (TRUE);
}
[NTOS]: Convert system call handling to C. Only kernel system calls are done this way for now, not SYSENTER calls from user-mode. A small ASM trampoline is used inline for the call itself. svn path=/trunk/; revision=45147
2010-01-19 09:20:40 +00:00
}
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Interrupt Trap Entry
//
VOID
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
FORCEINLINE
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
KiEnterInterruptTrap(IN PKTRAP_FRAME TrapFrame)
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
{
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Save exception list and terminate it */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
KeGetPcr()->NtTib.ExceptionList = EXCEPTION_CHAIN_END;
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Flush DR7 and check for debugging */
TrapFrame->Dr7 = 0;
if (__builtin_expect(KeGetCurrentThread()->DispatcherHeader.DebugActive & 0xFF, 0))
{
DbgPrint("Need Hardware Breakpoint Support!\n");
while (TRUE);
}
/* Set debug header */
KiFillTrapFrameDebug(TrapFrame);
}
[NTOS]: Implement GUI thread promotion during the first GUI system call in C. This is tricky due to EBP, and actually requires some tiny inline ASM magic to make it work right. [NTOS]: Implement SYSENTER system calls in C as well. All system calls are now handled in C. This code will be further optimized/refined soon. svn path=/trunk/; revision=45148
2010-01-19 09:45:30 +00:00
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
//
// Generic Trap Entry
//
VOID
FORCEINLINE
KiEnterTrap(IN PKTRAP_FRAME TrapFrame)
{
[PERF]: Omit setting certain useless trap frame fields. [NTOS]: Fix KiEnterInterruptFrame. In the case of a V86 trap, the segments were overriden because this wasn't a kernel trap (missing if/elseif). svn path=/trunk/; revision=45353
2010-01-30 18:25:30 +00:00
/* Save exception list */
[NTOS] - KPCR::Tib -> NtTib, KPCR::Self -> SelfPcr - Rename ExInitializePushLock inline function to _ExInitializePushLock and use a define, as in newer versions of windows the function exists as export and is in the WDK svn path=/branches/header-work/; revision=46136
2010-03-12 16:28:04 +00:00
TrapFrame->ExceptionList = KeGetPcr()->NtTib.ExceptionList;
[PERF]: Make the trap entry/exit macros inlined, just as they used to be before in their pure-assembly forms. This should boost performance quite a bit by avoiding 2-3 extra calls during traps and system calls. [NTOS]: Cleanup the traphdlr.c vs. trap_x.h mess into something much more structured. trap_x.h is now all inlined C functions which pretty much replace the asmmacros.S file. It is meant to be used outside the kernel as well (such as by the HAL, which requires trap entry/exit too). [PERF]: Give UNREACHABLE another try, this time with GCC 4.5+, 4.4+, 4.4- and MSVC support. [PERF]: Remove ABIOS checks since ReactOS does not support this, and since it doesn't look like the target kernel actually does either, we can remove this vestigal/unsupported code, saving some extra branches and checks. svn path=/trunk/; revision=45231
2010-01-24 05:40:04 +00:00
/* Flush DR7 and check for debugging */
TrapFrame->Dr7 = 0;
if (__builtin_expect(KeGetCurrentThread()->DispatcherHeader.DebugActive & 0xFF, 0))
{
DbgPrint("Need Hardware Breakpoint Support!\n");
while (TRUE);
}
/* Set debug header */
KiFillTrapFrameDebug(TrapFrame);
[FREELDR]: Jump to a standard 1-parameter STDCALL kernel entrypoint instead of a FASTCALL double-parameter entrypoint. [NTOS]: Make KiSystemStartup the real C entrypoint of the kernel, and move the "Am I being booted by FreeLDR" logic inside it -- it will then call KiRosPrepareForSystemStartup as earlier. [NTOS]: Move the Double Fault and Boot Stack declaration in C code, with the proper alignment attribute. [NTOS]: Although the concern that KiSystemStartup cannot be 100% C since it modifies ESP is real (Thomas' original fix of Alex's code), we don't need that much of it in assembly. Instead, write a simple trampoline (KiSwitchToBootStack) inline which switches stacks and jumps to a second-stage C function. [NTOS]: Completely remove boot.S as it isn't needed anymore, ReactOS startup is back to being (nearly) 100% C. svn path=/trunk/; revision=45152
2010-01-19 18:27:24 +00:00
}
Reference in a new issue Copy permalink