- Acquire the appropriate working set lock when calling MmLocateMemoryAreaByAddress
- Do not access MemoryArea without holding the lock (otherwise it can be pulled away under our feet)
- Fix range check for paged pool
These faults are handled by ARM³ and we don't need to check for a memory area. They can be recursive faults (e.g. from MiDeleteSystemPageableVm), so we might be holding the WS lock already. Passing it straight to ARM³ allows to acquire the WS lock below to look up the memory area.
Addendum to commit b3c55b9e6 (PR #4399).
Passing &CapturedObjectName as pointer to be probed and captured would
fail if e.g. PreviousMode == UserMode, since that pointer is always in
kernel space. Instead, pass the original user-mode pointer.
Bug caught by Timo Kreuzer ;)
This is a hack, because the kernel mode path can incur a recursive page fault with the AddressCreationLock acquired, which would lead to a recursive acquisition, once we do proper locking in MmAccessFault.
To properly fix this the PDE must be made valid, similar to the user mode path, but that is not that simple...
They can be spammy. Also clarify these debug prints, because some people
think that "failed to grant access rights" means there's something wrong
in the core access check functions.
Temporarily add the local group to the system token so that Virtualbox
GA services can properly set up network drives for shared folders.
What happens is that a security descriptor has a DACL with only one ACE
that grants access to Local SID (presumably coming from Vbox?)
but the client token is that of the service which is a SYSTEM token.
Perhaps we are not impersonating the right user or whatever else.
This is only a temporary placebo, until a proper solution is found.
CORE-18250
Certain apps such as AIM installer passes an empty generic mapping (this can
be understood with their generic masks set to 0) and our code tries to map
the access right from an ACE with the mapping provided by AccessCheck.
This can lead to a bug where we would not be able to decode the generic right
from an ACE as we need a proper generic mapping in order to do so. A mask
right that is not decoded it cannot be used to mask out the remaining rights,
further resulting into a denied access right.
What Windows does instead is they are mapping the ACE's rights in another place,
presumably when setting security data to an object, and they are using the
generic mapping passed by the kernel.
What we can do for the time being is to temporarily grant access to the client,
but only if they are an administrator.
CORE-18576
During an open or create procedure of a registry key, the registry parser grabs
a key control block (KCB) from the parser object and uses its information to do the
necessary work in order to obtain a pointer to the newly created or opened registry key.
However, the registry parsers faces several issues. First, we don't do subkey cache cleaning
information against gathered KCBs so whenever we do a registry parse we end up with KCBs
that have cache inconsistencies. Moreover we don't do any locking of whatever KCB we
are grabing during a parse procedure.
=== PROPOSED CHANGES ===
* Implement CmpComputeHashValue and CmpLookInCache functions. With CmpComputeHashValue we can
compute the convkey hashes of each subkey in the path name of a key so we can lock them
with CmpBuildAndLockKcbArray. CmpLookInCache is a function that searches for the suitable
KCB in the cache. The factors that determine if a KCB is "suitable" are:
-- the currently found KCB in the hash list has the same levels as that of the
given KCB from the parse object;
-- The key names from the computed hash values match with the block name of
the KCB;
-- The currently found KCB is not deleted.
The KCB will be changed if the key path name points to a partial match name in
the cache. The KCB from the parse object will be used if we have a full match
of remaining levels.
* Add missing CMP_LOCK_HASHES_FOR_KCB flags on CmpCreateKeyControlBlock calls
that create KCBs during a parse procedure. Such lock has to be preserved until
we're done with the registry parsing.
* On CmpDoCreateChild, preserve the exclusive lock of the KCB when we are
enlisting the key body.
* On CmpDoCreate, make sure that the passed parent KCB is locked exclusively and
lock the hiver flusher as we don't want the flusher to kick in during a key
creation on the given hive. Cleanup the subkey info when we're creating a key
object. Also implement missing cleanup path codes. Furthermore, avoid key
object creation if the parent KCB is protected with a read-only switch.
* Soft rewrite the CmpDoOpen function, namely how we manage a direct open vs
create KCB on open scenario. When a KCB is found in cache avoid touching
the key node. If the symbolic link has been resolved (aka found) then lock
exclusively the symbolic KCB. Otherwise just give the cached KCB to the caller.
If it were for the caller to request a KCB creation, we must check the passed
KCB from the parser object is locked exclusively, unlike on the case above
the caller doesn't want to create a KCB because there's already one in the cache.
We don't want anybody to touch our KCB while we are still toying with it during
its birth. Furthermore, enlist the key body but mind the kind of lock it's been
used.
* On CmpCreateLinkNode, avoid creating a key object if the parent KCB is protected
with a read-only switch. In addition, add missing hive flusher locks for both
the target hive and its child. Cleanup the subkey information of the KCB when
creating a link node, this ensures our cached KCB data remains consistent.
* Do a direct open on CmpParseKey if no remaining subkey levels have been found
during hash computation and cache lookup, in this case the given KCB is the
block that points to the exact key. This happens when for example someone tried
to call RegOpenKeyExW but submitting NULL to the lpSubKey argument parameter.
CORE-10581
ROSTESTS-198
CmpSecurityMethod is a method used by the Object Manager and called by this
subsystem whenever a security operation has to be done against a key object.
As CmpSecurityMethod is a specific OB construct we should not make any direct
call attempts to CmpSecurityMethod, only OB is responsible for that. This fixes
a deadlock where CmpSecurityMethod acquires a push lock for exclusive access
even though such lock is already acquired by the same calling thread in
CmpDoCreateChild.
This prevents a deadlock in DelistKeyBodyFromKCB when we delete a key
object because of an access check failure during a open procedure of a
registry key, as we are already holding a lock against the target KCB of
the key body.
Whenever a security request is invoked into a key object, such as when requesting
information from its security descriptor, the Object Manager will execute
the CmpSecurityMethod method to do the job.
The problem is that CmpSecurityMethod is not aware if the key control block
of the key body already has a lock acquired which means the function will attempt
to acquire a lock again, leading to a deadlock. This happens if the same
calling thread locks the KCB but it also wants to acquire security information
with ObCheckObjectAccess in CmpDoOpen.
Windows has a hack in CmpSecurityMethod where the passed KCB pointer is ORed
with a bitfield mask to avoid locking in all cases. This is ugly because it negates
every thread to acquire a lock if at least one has it.
The CmpUnLockKcbArray, CmpLockKcbArray and CmpBuildAndLockKcbArray routines
help us to lock KCBs within array so that information remains consistent when
we are doing a cache lookup during a parse procedure of the registry database.
Implement CmpBuildAndLockKcbArray and CmpUnLockKcbArray prototypes, we'll gonna need these
to do the locking/unlocking of KCBs stacked up in an array. In addition implement some CM
constructs specifically for cache lookup implementation (more at documentation remarks).
=== DOCUMENTATION REMARKS ===
CMP_SUBKEY_LEVELS_DEPTH_LIMIT -- This is the limit of up to 32 subkey levels
that the registry can permit. This is used in CmpComputeHashValue to ensure
that we don't compute more than the limit of subkeys we're allowed to.
CMP_KCBS_IN_ARRAY_LIMIT -- This is equal to CMP_SUBKEY_LEVELS_DEPTH_LIMIT
plus the addition by 2. This construct is used as a limit of KCB elements
the array can hold. 2 serves as an additional space for the array (one for
the root object and another one as extra space so we don't blow up the stack
array).
CMP_LOCK_KCB_ARRAY_EXCLUSIVE & CMP_LOCK_KCB_ARRAY_SHARED -- These flags are used exclusively
for CmpBuildAndLockKcbArray and CmpLockKcbArray. Their meaning are obvious.
CM_HASH_CACHE_STACK -- A structure used to store the hashes of KCBs for locking. It is named
"stack" because the way we store the hashes of KCBs is within an auxilliary "outer stack array".
CmpAcquireKcbLockSharedByKey can come in handy for use to lock KCBs by their convkey with a shared lock, specifically we would need this for cache lookup stuff.
- RtlpQuerySecurityDescriptor: Change argument type of first parameter from PISECURITY_DESCRIPTOR to PSECURITY_DESCRIPTOR, since it handles both absolute and self-relative SDs.
- RtlMakeSelfRelativeSD: rename first parameter from AbsoluteSD to SecurityDescriptor, since it handles both absolute and self-relative SDs.
- SepGetGroupFromDescriptor/SepGetOwnerFromDescriptor/SepGetDaclFromDescriptor/SepGetSaclFromDescriptor: Change parameter type from PVOID to PSECURITY_DESCRIPTOR for clarity.
The code was passing 0 instead of SECTION_INHERIT::ViewUnmap (2). 0 isn't even a proper constant to be used here. It worked, because MmMapViewOfSection only compares against ViewShare (1) and treats everything else as ViewUnmap.
The function set CtxSwitchFrame->ApcBypass to FALSE, preventing APCs (like when user mode sets the context while the thread is suspended) from being delivered as soon as the thread lowers IRQL to PASSIVE_LEVEL. This resulted in the SetContext APC to be delivered only after the user mode APC was initialized, overwriting the user mode APC context in the trap frame. This caused kernel32_winetest process to break.
Now that the Memory Management is a bit more under control again,
and branching releases/0.4.15 is near,
do mute some frequent log-spam that got introduced during 0.4.15-dev'ing
regarding lazy-flushes and MM balancing.
It frequently logged even while being idle.
Slightly improve the headers of the two touched files.
No rocket-science.
- They notify, via the "\\Callback\\SetSystemTime" callback, components
of a change of system time (for example, Win32k).
Note, that our Win32k currently does not handle power callouts, so
it isn't affected by these changes (yet).
- NtSetSystemTime(NULL, ...) means "update system time using the current
time-zone information", which is something we don't implement yet.
(And, nothing was previously protecting this call from a NULL parameter...)
Add redirections for KdSave/KdRestore and KdD0Transition/KdD3Transition.
Both KDBG and KD(TERM) need those since they will become external
transport DLLs later.
Split KdSendPacket and KdReceivePacket into those that manipulate the
KDBG state proper (reside in kdbg/kdbg.c), and those that deal only with
debug input/output that will reside in a KDTERM "KD Terminal Driver" DLL.
Based on some previous preparatory work by Hervé Poussineau in PR #4600.
(Equivalents of commits 5162bf106 and partly e9bcf7275.)
As it turns out, those three functions were duplicating the same code
between each other. Reimplement these in terms of a common helper,
RtlFindExportedRoutineByName().
Indeed: MiFindExportedRoutineByName() was just MiLocateExportName()
but taking a PANSI_STRING instead of a NULL-terminated string.
A similar state of affairs also existed in Windows <= 2003, and the
MS guys also noticed it. Both routines have been then merged and renamed
to MiFindExportedRoutineByName() on Windows 8 (taking a PCSTR instead),
and finally renamed and exported as RtlFindExportedRoutineByName()
on Windows 10.
It was implemented in psmgr.c but in a recursive way. That implementation
is replaced, in the NameToOrdinal() helper, by the better non-recursive one
found in the MiLocateExportName() and MiFindExportedRoutineByName() functions.
This NameToOrdinal() helper is then called in lieu of the duplicated code
in MiLocateExportName() and MiFindExportedRoutineByName(). In addition,
one block of code in MiSnapThunk() is simplified in a similar manner.
ACCESS_DENIED_ACE_TYPE, ACCESS_ALLOWED_ACE_TYPE, SYSTEM_AUDIT_ACE_TYPE and
SYSTEM_ALARM_ACE_TYPE belong to the same commonly internal ACE type, aka KNOWN_ACE,
as each of these ACEs have the same structure field offsets.
The only difference are ACCESS_DENIED_OBJECT_ACE_TYPE and ACCESS_ALLOWED_OBJECT_ACE_TYPE
as they have their own internal ACE type variant, the KNOWN_OBJECT_ACE structure.
The general guideline is that public ACE structure variants have to be used elsehwere
such as in UM whilst the kernel has to use the internal known ACE type variants when possible.
- Implement SepDenyAccessObjectTypeResultList, SepAllowAccessObjectTypeResultList,
SepDenyAccessObjectTypeList and SepAllowAccessObjectTypeList. These routines will
be used to grant or deny access to sub-objects of an object in the list.
- Refactor SepAnalyzeAcesFromDacl and SepAccessCheck to accomodate the newly
implemented access check by type mechanism.
- SepAccessCheck will now be SepAccessCheckWorker, a worker helper function that further
abstracts the access check mechanism in the kernel. Whereas the SepAccessCheck name will be
used as a centralized function used by the access check NT system calls.
- Deprecate SepGetSDOwner and SepGetSDGroup in favor of SepGetOwnerFromDescriptor and
SepGetGroupFromDescriptor. The former functions were buggy as they might potentially
return garbage data if either the owner or group were passed as NULL to a security
descriptor, hence a second chance exception fault. This was caught when writing tests
for NtAccessCheckByType.
- Shorten the debug prints by removing the name of the functions, the person who reads
the debugger output has to look at the source code anyway.
This implements various private kernel routines for object type list management
needed for access check code infrastructure. In addition, update the code documentation
and add missing comments.
This function will dump all the access status and granted access rights
of each object list of a list whenever an access check by type (or by type
result list) fails. This is for debugging purposes.
OBJECT_TYPE_LIST_INTERNAL will serve as an internal kernel data structure
to hold validated object type contents that are copied from UM.
The difference between the public and the internal one is that the internal structure has
an additional member for access check rights that have been granted on each
object element in the list.
I intend to port back the combined work of Thomas Faber and Serge Gautherie in context of CORE 14271.
Both developers fixed wrong retval evaluations for SeSinglePrivilegeCheck() and RtlCreateUnicodeString().
Both functions do return a BOOLEAN, and therefore using NTSTATUS() on them is wrong.
Those bugs have been fixed at multiple places. That is long gone.
But Serge fixed his locations a bit more elegantly, without the need for additional variables.
Therefore this addendum adapts a few of Thomas locations to the improved Serge-ified style.
Yes: I intentionally used a space instead of a minus after the mentioned CORE 14271,
as I don't want that pure stylistic addendum to be linked with the initial ticket anymore.
That would be overkill.
The function updates the entry in the section page table and updates the section association rmaps for it. In the page-in path, when the new section association is set before the entry is updated, a concurrent attempt to unmap the page would find an inconsistent entry, where there is an rmap, but the section page table entry is still an MM_WAIT_ENTRY.
MmGetSectionAssociation races with _MmSetPageEntrySectionSegment without sharing a lock. So we need to hold the PFN lock, until we have referenced the section segment found in the RMAP. This prevents that a section segment, which still has associated RMAPs from being deleted behind our back.
These are used in the paging path, when the page is currently in the process of being read from or written to the disk. While YieldProcessor() provides the chance to switch context to the other paging thread, it only does so, once the current thread's quantum has expired. On a single CPU system this effectively leads to busy waiting for the rest of the quantum. On SMP systems this could succeed earlier, thus reducing latency, but it would still contribute to high CPU usage, while waiting for the IO operation to complete, which is not what we want.
Using KeDelayExecutionThread() will instantly allow another thread to run, providing enough time to complete the IO operation.
This should be performed early enough before CM initialization,
but after the TSC has been initialized and calibrated by HAL.
Based on existing i386 kiinit code. CORE-17971 CORE-14922
Implement IoConnectInterruptEx() for CONNECT_FULLY_SPECIFIED.
This gives ability to load various modern drivers that use IoConnectInterruptEx.
Various drivers work after this change, such as serial.sys MS sample driver when compiled with the reactos tree and many more KMDF drivers from later Windows versions.
Co-authored-by: Victor Perevertkin <victor@perevertkin.ru>
KiGetFeatureBits() is now being called in the early boot phase 0
when the Kernel Debugger is not yet initialized, so debug prints
are not available here. Move the debug prints into a new function
and call it at the right time. CORE-18023
The function should return the kernel time for the idle thread in the
first argument, and kernel time + user time for the current thread in
the second argument.
Also retrieve the processor number from the cached PRCB instead of
calling KeGetCurrentProcessorNumber() which retrieves the PRCB again
since the processor could switch in-between those calls.
NdisGetCurrentProcessorCounts() function follows the same prototype
which is the correct one.
Besides creating the PDO and device node for it, it has to set up the
necessary registry keys, and register PDO at PnP root driver properly.
CORE-18989
The root device object is in fact a PDO and a FDO at the same time. Thus
there is no need in creating two device objects here, one is enough.
This commit also removes the explicit device extension for the root DO,
because the only reason it existed is to distinguish the root driver's
FDO from its PDOs. This can easily be done by comparing with
IopRootDeviceNode.
Also collect some unused garbage while we are here.
Handling PnP root driver power IRPs requires that a device object must come up
with a device extension to determine whether it is a function driver and if so,
handle the IRP accordingly.
CORE-18989
- Add missing ExAllocatePool NULL checks.
- Fix order of KeBugCheckEx parameters for PNP_DETECTED_FATAL_ERROR.
- The Controller and Peripheral numbers are zero-based, so if the caller
wants to inspect controller (or peripheral) zero, let it be so!
The original code was treating controller number zero for enumerating
controllers of a given class within the different buses, which is
wrong. See the diff'ed trace below.
Tested with Windows' videoprt.sys VideoPortGetDeviceData().
```diff
IoQueryDeviceDescription()
BusType: 0xB093C224 (0)
BusNumber: 0xB093C228 (0)
ControllerType: 0xF9D01030 (19)
ControllerNumber: 0xF9D01038 (0)
PeripheralType: 0x00000000 (4294967295)
PeripheralNumber: 0x00000000 (4294967295)
CalloutRoutine: 0xF9CF74E4
Context: 0xF9D5A340
--> Query: 0xF9D5A22C
IopQueryBusDescription(Query: 0xF9D5A22C)
RootKey: '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM'
RootKeyHandle: 0x00000598
KeyIsRoot: TRUE
Bus: 0xF9D5A290 (4294967295)
Seen: 'CentralProcessor'
Seen: 'FloatingPointProcessor'
Seen: 'MultifunctionAdapter'
SubRootRegName: '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter'
IopQueryBusDescription(Query: 0xF9D5A22C)
RootKey: '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter'
RootKeyHandle: 0x00000590
KeyIsRoot: FALSE
Bus: 0xF9D5A290 (4294967295)
Seen: '0'
SubRootRegName: '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter\0'
Getting bus value: 'Identifier'
Getting bus value: 'Configuration Data'
Getting bus value: 'Component Information'
--> Getting device on Bus #0 : '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter\0'
IopQueryDeviceDescription(Query: 0xF9D5A22C)
RootKey: '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter\0'
RootKeyHandle: 0x00000590
Bus: 0
- Enumerating controllers in '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter\0\DisplayController'...
+ Getting controller #0
+ Retrieving controller '\REGISTRY\MACHINE\HARDWARE\DESCRIPTION\SYSTEM\MultifunctionAdapter\0\DisplayController\0'
```
Based on a commit by Vadim Galyant:
5ef5c11e7f
Also fix a minor type conversion warning. CORE-18963 CORE-17977
Co-authored-by: Vadim Galyant <vgal@rambler.ru>
We should compare against DeviceObject as DeviceInstance is never NULL.
Fix a resource leak as well. The bug CORE-18983 seems to lay somewhere
else though, I just stumbled upon this one while researching it.
Note there is a BSOD in the PnP manager on reboot after the driver
installation failure, but it seems it was uncovered by the fix
as opposed to caused by it.
- Refactor most of the code, since there's quite some stuff that don't make much sense.
For instance ImpersonationLevel is basically the requested impersonation level a
server asks for. PsImpersonateClient doesn't explicitly say that SecurityAnonymous
and SecurityIdentification are not allowed. If the server was to give such levels
it simply means it doesn't want to impersonate the client.
Another thing that doesn't make much sense is that we check if the client is
associated with an anonymous token, then avoid impersonating regular anonymous
tokens that weren't created by the system. Only system can create such tokens
and an anonymous token basically means a token with hidden security info.
- Check that the server is within the same client logon session.
- If the server is granted the SeImpersonatePrivilege privilege, allow impersonation
regardless of the conditions we want to check for.
- Update the documentation and code comments.
As it currently stands the PsImpersonateClient routine does the following approach.
If impersonation couldn't be granted to a client the routine will make a copy
of the client's access token. As it makes a copy of the said token PsImpersonateClient
will reference the copied token after impersonation info have been filled out.
In the same code path we are assigning the desired level for impersonation to thread
impersonation info.
This is wrong for two reasons:
- On a copy situation the SeCopyClientToken routine holds a reference as the object
has been created. Referencing it at the bottom of the PsImpersonateClient routine
will make it that the token is referenced twice and whenever a server stops
impersonation the token still has an extra reference count which keeps the token
still alive in object database and memory space.
- If client impersonation is not possible the thread impersonation info should
have been assigned SecurityIdentification level to further indicate that the
actual impersonation of the thread is not currently in force but instead we
are assigning the impersonation level that is supplied by the caller. For instance
if the requested level is SecurityDelegation but impersonation is not possible
the level will be assigned that of SecurityDelegation yet the token has an
impersonation level of SecurityIdentification. This could lead to erratic behaviors
as well as potential impersonation escalation.
Fix the aforementioned issues by avoiding a double reference and properly assign
the impersonation level to SecurityIdentification if the server is not able to
impersonate the target client.
- Add the missing privileges to the SYSTEM privileges which might be needed,
notably SeUndockPrivilege, SeManageVolumePrivilege, SeCreateGlobalPrivilege and
SeImpersonatePrivilege.
Specifically SeImpersonatePrivilege is important here because with it we
allow system components of the core OS to perform certain system tasks.
- Declare the Groups array with a maximum of 3 elements in SepCreateSystemProcessToken
and 1 element in SepCreateSystemAnonymousLogonToken respectively, because previously
this array was oversized with most of free space left as a waste.
- Avoid hardcoding the size value of the Privilege array, instead initialize it
by hand and compute the exact number of elements with RTL_NUMBER_OF.
- Fix whitespace; add SAL annotations, doxygen documentation...
- Deduplicate the array of description strings corresponding to
IO_QUERY_DEVICE_DATA_FORMAT.
- Unhardcode the "[3]" into 'IoQueryDeviceMaxData': the maximum number
of device data queried.
- Wrap most of the code into a new private routine, SepOpenThreadToken.
And properly fail gracefully if we fail to open a thread's token instead of just keeping going.
- Do not use the same thread object that we have referenced in NtOpenThreadTokenEx
to do a copy of the access token in case we can't open it directly.
Instead we must reference a new object with full access, solely used for
the purpose to do our required operations.
- Add debug prints
CORE-18986
Removing any disabled privileges or groups in the middle of token dynamic
part allocation can pose problems. During the operation of making an access
token as effective, we are toying with the privileges and groups arrays
of the token.
After that we are allocating the dynamic part and set EndMem (the end tail
of the memory part) to that dynamic part, previously it was set to the
variable part. As a matter of fact we are making the token effective in
the middle where EndMem still points to VariablePart, thus DynamicPart
will end up with memory pool blocks butchered in the pool list.
Another problem, albeit not related to the DynamicPart corruption, is that
the code starts iterating over the UserAndGroups array from 0, which is
the actual user. One cannot simply remove the user from the array, so we
have to start looping right from the groups.
Move the token effective code part at the end of the SepDuplicateToken
function, which fixes the random pool corruptions caused by the butchered
DynamicPart.
CORE-18986
CORE-18962
- Deduplicate a while-loop by adding one more recursive call.
- Add IopMapDetectedDeviceId() helper function with a structure
in order to reduce hardcoded constants and checks.
- Do not allocate a new stack, if the thread already has a large one. This prevents the function from freeing a large stack as a normal stack and subsequently leaking system PTEs.
- Fix the check for failure of PsConvertToGuiThread (test eax, not rax, for being negative, because by default rax is zero extended from eax, not sign extended). This fixes an infinite loop on failure.
The data has to be written into ObjectTypeInfo based on the return length,
not only what is provided by the input buffer length. Fix suggested by
Hermès.
On a x86 system aligning the return length pointer to a 4-byte boundary
works best since pointers in general are 4-byte aligned for x86 systems.
However, what happens on a AMD64 system is that we still align this pointer
to 4-byte, ObjectTypeInfo is a 8-byte pointer and we might write into
the return length past the 4-byte boundary.
If one were to allocate a pool of memory with that length and query all
the object types info and free the said pool of memory thereafter, the
system will crash with BAD_POOL_HEADER because ObQueryTypeInfo overwrote
the return length past the 4-byte boundary length therefore leading up with
corrupted memory blocks in the pool header.
This symptom of BAD_POOL_HEADER happens exactly the same in Windows Server
2003 x64 Edition. Newer versions of Windows like 10 aren't affected.
But, Windows has another bug where they are using MaximumLength for the
calculation of the needed length to be returned to caller. MaximumLength
does not guarantee you that it includes the NULL-terminator in the length
and that potentially leads to a buffer overrun.
Also annotate the ObQueryTypeInfo function with SAL2.
https://processhacker.sourceforge.io/doc/object_8c_source.html (read the
comment in KphObjectTypeInformation).
Second parameter is optional, so mark it as such and check whether it was passed. Fixes a sporadic 0x24 bugcheck caused by access violation when running ReactOS on NTFS volume with WinXP ntfs.sys.
Finally handlers are - unlike except blocks - not part of the function they are in, but separate functions, which are called during unwind. PSEH implements them on GCC using nested functions. While "return" from a finally handler is allowed with native SEH, it's handled by the compiler through an extra unwinding operation using _local_unwind, WHICH IS NOT SUPPORTED BY PSEH! With PSEH, returning from a finally handler does not return from the function, instead it will only return from the finally handler and the function will continue below the finally handler as if there was no return at all. To fix this, the return is removed and an additional success check is added.
Also use _SEH_VOLATILE to make sure the variable assignment is not optimized away by the compiler and add zero out the result parameters on error.
... that would otherwise cause a debugger re-entry.
Also use KdbPuts/Printf instead of KdpDprintf that won't be available
once KDBG is moved out of it.
... since the original ones are internal to the kernel and won't be
available once KDBG is moved out of it.
Use these functions in the pager/prompt support.
The built string can be:
°°Kernel Debugger: Serial port found: COM1 (Port 0x000003F8) BaudRate 115200°°°°
(with ° representing the \r and \n in the message)
and you can verify that this is more than 80 characters in total.
CORE-17627
When closing a file, fastfat zeroes it out from ValidDataLength up to the end of the file.
The ValidDataLength field is updated when the file content is actually written to disk.
There is currently a race between the file-close path and the page out path, leading to potential file corruptions when the zeroing happens after the memory has been flushed to disk.
Fix this by actually flushing the file to disk when unmapping files, with file lock acquired. This way, the FS driver cannot zero out the tail of the file while we're actually flushing it to disk.
This one was more subtle because the prompt (KdIoReadLine) functionality
makes a call-back to KDBG own command history getter function KdbGetHistoryEntry.
It is planned for this to become a registered optional callback pointer.
This is done in preparation for moving all this functionality in a
separate KDTERM "KD Terminal Driver" DLL.
Additionally:
- Flush the terminal input before sending ANSI escape sequences.
- In KDBG pager, always use the correct reading-key function (the
same used also for reading keys for a line of user input), and not
the simplistic two-call KdbpGetCharSerial + KdbpTryGetCharSerial
that would split the \r \n across calls.
- Call KdbpGetCommandLineSettings() in KdbInitialize() at BootPhase 0,
which is indirectly called by KdDebuggerInitialize0(). And fix its
command-line parsing too.
Rename KdbpReadCommand as KdIoReadLine. Extract the last-command
repetition functionality out of KdIoReadLine and put it where it
belongs: only in the KDBG command main loop KdbpCliMainLoop.
Use this function instead of KdpDprintf(), otherwise, we send them to
**ALL** the display providers, including for example dmesg. Replaying
the listing with dmesg would then cause the terminal to misbehave later.
For example, it would send the answer of a "Query Device Attributes"
command, as the response to a query for terminal size...
Addendum to commit 84e32e4e.
Explain more accurately what's going on regarding the returned string
and the inaccurate claims made in the official DbgPrompt documentation
in MSDN. (Has been verified by looking through the traffic in WinDbg
debugging of Windows and ReactOS.)
Of course, now that we **correctly** set the LoadSymbools setting,
we attempt loading symbols at BootPhase 0 and everything goes awry!
So introduce that hack to fallback to our old behaviour.
A proper fix (and removal of the hack) will be done in future commits.
Addendum to commit de892d5b.
The boot options get stripped of their optional command switch '/'
(and replaced by whitspace separation) by the NT loader. Also, forbid
the presence of space between the optional '=' character following
(NO)LOADSYMBOLS.
In addition, fix the default initialization of LoadSymbols in KdbSymInit():
we cannot rely on MmNumberOfPhysicalPages in BootPhase 0 since at this point,
the Memory Manager hasn't been initialized and this variable is not yet set.
(We are called by KdInitSystem(0) -> KdDebuggerInitialize0 at kernel init.)
It gets initialized later on between BootPhase 0 and 1.
Also display a nice KDBG signon showing the status of symbols loading.
LoadSymbols was reset to its default value whenever KdbSymInit() was
called, thus we would e.g. load symbols even if /NOLOADSYMBOLS or
/LOADSYMBOLS=NO were specified at the command line.
CORE-17470
+ KdpDebugLogInit: Add resources cleanup in failure code paths.
Fix, in an NT-compatible manner, how (and when) the KD/KDBG BootPhase >=2
initialization steps are performed.
These are necessary for any functionality KDBG needs, that would depend
on the NT I/O Manager and the storage and filesystem stacks to be running.
This includes, creating the debug log file, and for KDBG, loading its
KDBinit initialization file.
As a result, file debug logging is fixed.
The old ReactOS-specific (NT-incompatible) callback we did in the middle
of IoInitSystem() is removed, in favor of a runtime mechanism that should
work on Windows as well.
The idea for this new mechanism is loosely inspired by the TDL4 rootkit,
see http://blog.w4kfu.com/public/tdl4_article/draft_tdl4article.html
but contrary to it, a specific hook is used instead, as well as the
technique of driver reinitialization:
https://web.archive.org/web/20211021050515/https://driverentry.com.br/en/blog/?p=261
Its rationale is as follows:
We want to be able to perform I/O-related initialization (starting a
logger thread for file log debugging, loading KDBinit file for KDBG,
etc.). A good place for this would be as early as possible, once the
I/O Manager has started the storage and the boot filesystem drivers.
Here is an overview of the initialization steps of the NT Kernel and
Executive:
----
KiSystemStartup(KeLoaderBlock)
if (Cpu == 0) KdInitSystem(0, KeLoaderBlock);
KiSwitchToBootStack() -> KiSystemStartupBootStack()
-> KiInitializeKernel() -> ExpInitializeExecutive(Cpu, KeLoaderBlock)
(NOTE: Any unexpected debugger break will call KdInitSystem(0, NULL); )
KdInitSystem(0, LoaderBlock) -> KdDebuggerInitialize0(LoaderBlock);
ExpInitializeExecutive(Cpu == 0): ExpInitializationPhase = 0;
HalInitSystem(0, KeLoaderBlock); <-- Sets HalInitPnpDriver callback.
...
PsInitSystem(LoaderBlock)
PsCreateSystemThread(Phase1Initialization)
Phase1Initialization(Discard): ExpInitializationPhase = 1;
HalInitSystem(1, KeLoaderBlock);
...
Early initialization of Ob, Ex, Ke.
KdInitSystem(1, KeLoaderBlock);
...
KdDebuggerInitialize1(LoaderBlock);
...
IoInitSystem(LoaderBlock);
...
----
As we can see, KdDebuggerInitialize1() is the last KD initialization
routine the kernel calls, and is called *before* the I/O Manager starts.
Thus, direct Nt/ZwCreateFile ... calls done there would fail. Also,
we want to do the I/O initialization as soon as possible. There does
not seem to be any exported way to be notified about the I/O manager
initialization steps... that is, unless we somehow become a driver and
insert ourselves in the flow!
Since we are not a regular driver, we need to invoke IoCreateDriver()
to create one. However, remember that we are currently running *before*
IoInitSystem(), the I/O subsystem is not initialized yet. Due to this,
calling IoCreateDriver(), much like any other IO functions, would lead
to a crash, because it calls
ObCreateObject(..., IoDriverObjectType, ...), and IoDriverObjectType
is non-initialized yet (it's NULL).
The chosen solution is to hook a "known" exported callback: namely, the
HalInitPnpDriver() callback (it initializes the "HAL Root Bus Driver").
It is set very early on by the HAL via the HalInitSystem(0, ...) call,
and is called early on by IoInitSystem() before any driver is loaded,
but after the I/O Manager has been minimally set up so that new drivers
can be created.
When the hook: KdpInitDriver() is called, we create our driver with
IoCreateDriver(), specifying its entrypoint KdpDriverEntry(), then
restore and call the original HalInitPnpDriver() callback.
Another possible unexplored alternative, could be to insert ourselves
in the KeLoaderBlock->LoadOrderListHead boot modules list, or in the
KeLoaderBlock->BootDriverListHead boot-driver list. (Note that while
we may be able to do this, because boot-drivers are resident in memory,
much like we are, we cannot insert ourselves in the system-driver list
however, since those drivers are expected to come from PE image files.)
Once the KdpDriverEntry() driver entrypoint is called, we register
KdpDriverReinit() for re-initialization with the I/O Manager, in order
to provide more initialization points. KdpDriverReinit() calls the KD
providers at BootPhase >= 2, and schedules further reinitializations
(at most 3 more) if any of the providers request so.
CORE-10749
The dmesg command is now available even if screen output is disabled.
Co-authored-by: Hermès Bélusca-Maïto <hermes.belusca-maito@reactos.org>
- KdbSymInit() in kdb_symbols.c only initializes symbols implementation
support.
- The rest of KdbInitialize gets moved into kdb_cli.c and initializes
the KDBG debugger itself.
- Move KdbDebugPrint to kdb_cli.c as well.
Access check is an expensive operation, that is, whenever an access to an
object is performed an access check has to be done to ensure the access
can be allowed to the calling thread who attempts to access such object.
Currently SepAnalyzeAcesFromDacl allocates a block of pool memory for
access check rights, nagging the Memory Manager like a desperate naughty
creep. So instead initialize the access rights as a simple variable in
SepAccessCheck and pass it out as an address to SepAnalyzeAcesFromDacl so
that the function will fill it up with access rights. This helps with
performance, avoiding wasting a few bits of memory just to hold these
access rights.
In addition to that, add a few asserts and fix the copyright header on
both se.h and accesschk.c, to reflect the Coding Style rules.
This mutes a lot of debug spam that fills up the debugger when an access
check fails because a requestor doesn't have enough privileges to access
an object.
MmLoadSystemImage has a PUNICODE_STRING NamePrefix parameter which is
currently unused in ReactOS. When the kernel loads the crash dump
storage stack drivers, the drivers will be loaded with MmLoadSystemImage
with a "dump_" or "hiber_" (for hibernation, which uses crash dump
stack too) prefix. This change adds in the prefix support, and is
supposed to push crash dump support forward.
CORE-376
- Use SAL2 annotations.
- KdSendPacket(): Validate DEBUG_IO API call.
- KdReceivePacket(): Take the LengthOfStringRead into account; use
KdbpReadCommand() to read the input, so that correct line edition
is available (backspace, etc.)
- Don't read anything and return immediately if the buffer size is zero.
- Allow the controlling keys (up/down arrows, backspace) even if the
buffer is full! (especially backspace if too much has been written).
- Return the number of characters stored, not counting the NULL terminator.
Currently the failure code path doesn't do any kind of cleanup against
the hive that was being linked to master. The cleanup is pretty
straightforward as you just simply close the hive file handles and free
the registry kernel structures.
CORE-5772
CORE-17263
CORE-13559
Log debug output of the lazy flusher as much information as possible so that we can examine how does the lazy flusher behave, since it didn't work for almost a decade. Verbose debugging will be disabled once we're confident enough the registry implementation in ReactOS is rock solid.
Whenever ReactOS finishes its operations onto the registry and unlocks it, a lazy flush is invoked to do an eventual flushing of the registry to the backing storage of the system. Except that... lazy flushing never comes into place.
This is because whenever CmpLazyFlush is called that sets up a timer which needs to expire in order to trigger the lazy flusher engine worker. However, registry locking/unlocking is a frequent occurrence, mainly when on desktop. Therefore as a matter of fact, CmpLazyFlush keeps removing and inserting the timer and the lazy flusher will never kick in that way.
Ironically the lazy flusher actually does the flushing when on USETUP installation phase because during text-mode setup installation in ReactOS the frequency of registry operations is actually less so the timer has the opportunity to expire and fire up the flusher.
In addition to that, we must queue a lazy flush when marking cells as dirty because such dirty data has to be flushed down to the media storage of the system. Of course, the real place where lazy flushing operation is done should be in a subset helper like HvMarkDirty that marks parts of a hive as dirty but since we do not have that, we'll be lazy flushing the registry during cells dirty marking instead for now.
CORE-18303
Addendum to commit de81021ba.
Otherwise, we get the following build error:
\ntoskrnl\kd64\kddata.c(532,5): error: initializer element is not a compile-time constant
PtrToUL64(RtlpBreakWithStatusInstruction),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
\ntoskrnl\kd64\kddata.c(526,26): note: expanded from macro 'PtrToUL64'
#define PtrToUL64(x) ((ULPTR64)(x))
^~~~~~~~~~~~
If you ask why there are two sets of functions that do the same, it's
because this file (and the kdmain.c) will very soon some day be moved to
a transport dll, outside the kernel, and it will need these functions.
See this command's documentation:
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/-dbgprint
and the section "DbgPrint buffer and the debugger"
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/reading-and-filtering-debugging-messages#dbgprint-buffer-and-the-debugger
for more details.
- Loosely implement the function, based on our existing circular printout
buffers in kdio.c.
- Enable its usage in the KdpPrint() and KdpPrompt() functions.
Notice that this function will *only* capture the strings being sent **to**
the debugger, and not the strings the debugger itself produce. (This means
that we cannot use the KdPrintCircularBuffer as a replacement for our
KDBG dmesg one, for example...)
How to test:
Run ReactOS under WinDbg, and use the !dbgprint command to view the
buffer. You can also use the Memory Window, place yourself at the
address pointed by KdPrintCircularBuffer and KdPrintWritePointer, and
read its contents.
What you should observe:
Prior notice: The circular buffer in debug builds of ReactOS and Windows
is 0x8000 bytes large. In release builds, its size is down to 0x1000.
1- When you start e.g. the 2nd-stage GUI installation of ReactOS, going
past the initial "devices installation" and letting it stabilize on
the Welcome page, break into WinDbg and run the !dbgprint command. You
should notice that the end of its output is weirdly truncated, compared
to what has been actually emitted to the debug output. Comparing this
with the actual contents of the circular buffer (via Memory Window),
shows that the buffer contents is actually correct.
2- Copy all the text that has been output by the !dbgprint command and
paste it in an editor; count the number of all characters appearing +
newlines (only CR or LF), and observe that this number is "mysteriously"
equal to 16384 == 0x4000.
3- Continue running ReactOS installation for a little while, breaking back
back into WinDbg and looking at !dbgprint again. Its output seems to be
still stopping at the same place as before (but the actual buffer memory
contents shows otherwise). Continue running ROS installation, and break
into the debugger when ROS is about to restart. You should now observe
that the dbgprint buffer rolled over:
dd nt!KdPrintRolloverCount shows 1.
Carefully analysing the output of !dbgprint, however, you will notice
that it looks a bit garbage-y: the first part of the output is actually
truncated after 16384 characters, then you get a second part of the
buffer showing what ReactOS was printing while shutting down. Then
you get again what was shown at the top of the !dbgprint output.
(Of course, comparing with the actual contents of the circular buffer
in memory shows that its contents are fine...)
The reason of these strange observations, is because there is an intrinsic
bug in the !dbgprint command implementation (in kdexts.dll). Essentially,
it displays the contents of the circular buffer in two single dprintf()
calls: one for the "older" (bottom) part of the buffer:
[WritePointer, EndOfBuffer]
and one for the "newer" (upper) part of the buffer:
[CircularBuffer, WritePointer[ .
The first aspect of the bug (causing observation 3), is that those two
parts are not necessarily NULL-terminated strings (especially after
rollover), so for example, displaying the upper part of the buffer, will
potentially also display part of the buffer's bottom part.
The second aspect of the bug (explaining observations 1 and 2), is due
to the implementation of the dprintf() function (callback in dbgenv.dll).
There, it uses a fixed-sized buffer of size 0x4000 == 16384 characters.
Since the output of the circular buffer is not done by little chunks,
but by the two large parts, if any of those are larger than 0x4000 they
get truncated on display.
(This last observation is confirmed in a completely different context by
https://community.osr.com/discussion/112439/dprintf-s-max-string-length .)
But the underlying GCC stupidity is still there (15 years later).
However, enable it only in 32-bit GCC builds, not in 64-bits nor with MSVC.
See commit b9cd3f2d9 (r25845) for some details.
GCC is indeed still incapable of casting 32-bit pointers up to 64-bits,
when static-initializing arrays (**outside** a function) without emitting
the error:
"error: initializer element is not constant"
(which might somehow indicate it actually tries to generate executable
code for casting the pointers, instead of doing it at compile-time).
Going down the rabbit hole, other stupidities show up:
Our PVOID64 type and the related POINTER_64 (in 32-bit archs), or the
PVOID32 and POINTER_32 (in 64-bit archs), are all silently broken in
GCC builds, because the pointer size attributes __ptr64 and __ptr32,
which are originally MSVC-specific, are defined to nothing in _mingw.h.
(And similarly for the __uptr and __sptr sign-extension attributes.)
Clang and other sane ompilers has since then implemented those (enabled
with -fms-extensions), but not GCC. The closest thing that could exist
for GCC is to do:
#define __ptr64 __attribute__((mode(DI)))
in order to get a 64-bit-sized pointer type with
typedef void* __ptr64 PVOID64;
but even this does not work, with the error:
"error: invalid pointer mode 'DI'"
Choose the correct element of the KiUnexpectedRange array,
depending on the interrupt vector, the same way as here:
a2c6af0da4/ntoskrnl/ke/amd64/except.c (L77)
And guard KeConnectInterrupt() execution with dispatcher lock.
CORE-14922
- Line-wrapping is enabled with 'ESC[?7h' (the '?' was forgotten).
Notice that the following reference also shows it wrong:
https://www.cse.psu.edu/~kxc104/class/cse472/09s/hw/hw7/vt100ansi.htm
- Terminal type is actually queried with 'ESC Z' (VT52-compatible), or
with 'ESC[c' (VT100-compatible). The antediluvian CTRL-E ('\x05')
control code gives instead a user-configurable (usually empty) string,
so it's not reliable.
Also, we don't really care about the returned result, we just need to
know that one has been sent.
Cross-checked with online documentation:
* "Digital VT100 User Guide" (EK-VT100-UG-001) (1st edition, August 1978,
reviewed November 1978).
* https://www.real-world-systems.com/docs/ANSIcode.html
* https://geoffg.net/Downloads/Terminal/TerminalEscapeCodes.pdf
* https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
* https://en.wikipedia.org/wiki/Enquiry_character
- Retrieve the size of the *controlling terminal* with escape sequences
only when it's a serial one: serial output is enabled *and* KDSERIAL
is set (i.e. user input through serial). See code for the actual logic
(the corresponding truth table is left as an exercise for the reader).
- Fix also a "Buffer" vs. "InBuffer" mismatch, that caused the whole
code to fail.
- For fallback terminal size values, use meaningful ones when SCREEN
is instead the controlling "terminal" (based on full-size BOOTVID
values).
- When echoing read characters during line-cooking, do direct output by
using KdpDprintf() instead of going through the heavy KdbpPrint() function.
This fixes some input artifacts like: 1. extra slowdowns due to
querying terminal size at each keypress, and 2. getting obnoxious
"--- Press q to abort ..." prompts in the middle of typing a long
comamnd because you are at the very bottom of the screen.
- Remove KdbInit() macro and directly use KdbpCliInit() (since the place
where it was used was already within an #ifdef KDBG block).
- Declare KdpKdbgInit() only when KDBG is defined, move its definition
into kdio.c and remove the legacy wrappers/kdbg.c file.
And in KdbInitialize(), set KdpInitRoutine directly to the former,
instead of using the KdpKdbgInit indirection.
- Don't reset KdComPortInUse in KdpDebugLogInit().
- Minor refactorings: KdpSerialDebugPrint -> KdpSerialPrint and make it
static; argument name "Message" -> "String", "StringLength" -> "Length".
What we have:
- Maximum number of pagefiles: 16
- Minimum pagefile size: 256 pages (1 MB when page size = 4096 bytes)
- Maximum pagefile size:
* 32-bit platforms: (1024 * 1024 - 1) pages (~ 4095 MB)
* x86 with PAE support: same size as for AMD x64
* x64 platform: (4 * 1024 * 1024 * 1024 - 1) pages (~ 16 TB)
* IA64 platform: (8 * 1024 * 1024 * 1024 - 1) pages (~ 32 TB)
Those are the values as supported and verified by the NT kernel.
Now, user-mode programs (including SMSS.EXE) have different opinions
on these, namely, they consider estimates directly in MB, respectively:
4095 MB, (16 * 1024 * 1024) MB, and (32 * 1024 * 1024) MB
(verified on Win2k3 and Win7 32 and 64 bits).
Also here, the minimum pagefile size is set to 2 MB.
Starting Windows 8+ (and 10), those values change slightly, and are
still not fully synchronized between NTOS:MM and SMSS. Finally, while
(x86 PAE and) AMD64 and ARM64 seem to share the maximum pagefile
size limit, 32-bit ARMv7 appears to use different limits than regular
x86 (2 GB instead of 4).
Please keep those values as they are for NT compatibility!
See the following references:
https://www.geoffchappell.com/studies/windows/km/ntoskrnl/api/mm/modwrite/create.htmhttps://techcommunity.microsoft.com/t5/ask-the-performance-team/what-is-the-page-file-for-anyway/ba-p/372608
+ Manual extraction of the values from different NT 6.2,6.3,10 builds.
[SMSS] Fill out in particular the x86-specific case for PAE.
[NTOS:MM] Some cleanup in the NtCreatePagingFile() code, namely:
- Clarify some comments;
- Validate the lower and upper bounds of the Minimum and Maximum sizes
(based on Windows behaviour as explained by Geoff + manual tests).
- Open the pagefile in case-insensitive;
- Simplify the loop that finds an existing matching pagefile;
- Simplify some failure exit paths;
- Add a "Missing validation steps TODO" comment block explaining the
existing code-hole.
The "failed to grant access rights" message isn't enough to understand what kind of access rights haven't been granted and why. Dumping information of the captured security descriptor, the ACL and its ACEs with mask rights and token SIDs should be enough to understand the reason of the failure in question.
