Implement the correct start-stop sequence for resource rebalancing
without the actual rebalancing. Also move IoInvalidateDeviceState
processing into the enumeration thread as it should be.
CORE-17519
Implement initial token debug code. For now debug information that is being tracked are: process image file name, process and thread client IDs and token creation method. More specific debug code can be added later only if needed.
As for the token creation method, this follows the same principle as on Windows where the creation method is defined by a value denoting the first letter of the said method of creation. That is, 0xC is for token creation, 0xD is for token duplication and 0xF is for token filtering. The debug field names are taken from Windows PDB symbols for WinDBG debug extension support purposes. The names must not be changed!
- Add a new cmboot.h header to isolate the boot-support definitions
shared with the NT/ReactOS bootloader.
- Move CmpFreeDriverList() to cmboot.c so that we can use it for
cleanup paths in the NT/ReactOS bootloader.
- CmpFindControlSet(): Directly build the control set name in UNICODE,
instead of doing an ANSI->UNICODE conversion.
- Directly assign the CurrentControlSet\Services constant string,
instead of going the route of init-empty-string + append-string.
This is possible since that string is not modified later.
- Remove ASSERT(FALSE), replacing them with correct failure handling.
- Add cleanup paths in CmpAddDriverToList().
- Simplify and fix CmpFreeDriverList(): it's the full DriverNode
that needs to be freed; not the LIST_ENTRY pointer.
- Add other validity checks:
* Registry value types and data sizes;
* For multi-strings, verify that they are NULL-terminated.
* For (multi-)strings, check whether they are NULL-terminated before
optionally removing their trailing NULL character from the count.
Check also whether they are of zero-length and take appropriate
action where necessary.
- Add CmpIsDriverInList() for future usage in CMBOOT compiled in
bootloader mode.
- Add SAL annotations and Doxygen documentation.
- Add debug traces.
- Formatting / code style fixes.
** TODO: Fix SafeBoot support **
- inbv.c now only contains the Inbv-specific API and nothing else.
- It will make easier for people to write their own boot themes & animations,
by just copying/adapting the bootanim.c file (and the resources).
- Add SAL annotations.
- All INBV progress bar functions (except for InbvIndicateProgress())
should not be INIT-only functions, since they can be (not yet in ROS)
used at later times -- namely, for feedback during hibernation.
In particular, the progress percentage specified to InbvUpdateProgressBar(),
or the progress feedback made with InbvIndicateProgress() calls, is
**relative** to the progress sub-range specified with a previous call to
InbvSetProgressBarSubset() (by default, the range is 0...100%).
This functionality is used e.g. when the number of progress steps is
unknown prior, for example when loading drivers: in this case progress
is made within a given percentage range.
This bug has always been with us since 2010.
-- Rewrite PspChargeProcessQuotaSpecifiedPool and PspReturnProcessQuotaSpecifiedPool private kernel routines, with the goal to implement the algorithms necessary to manage the fields of quota blocks (Usage, Return, Limit and Peak).
-- Invoke the Mm if quota limit raising or returning has to be done
-- When destroying a quota block, make sure that we're giving back all the rest of non-returned quotas to Memory Mm
-- Crash the system with QUOTA_UNDERFLOW if someone is returning way too much quota than it was previously charged
-- When a process exits, ensure that it doesn't hold up any charged quotas in QuotaUsage field of the process object, that way we're enforcing proper kernel consistency
-- Implement PsChargeSharedPoolQuota and PsChargeProcessPageFileQuota functions, used exclusively by the Object Manager. These routines are used to charge or return amount of quotas of a newly created object.
-- On PspInheritQuota, when assigning to process the default quota block if no parent process is given, we must increment the reference counts as we're using it
-- Handle the ProcessCount reference field, as it wasn't used
-- Annotate the functions with SAL
-- Document the code
=== REMARKS ===
Windows LogOn (Winlogon) is responsible for setting up a different quota block for all the processes within an interactive session, which is what we don't do. What we're currently doing instead is we're using the default block, PspDefaultQuotaBlock, for all the processes
across the system. The default block contains the default limits of -1 (which would imply no limits). By definition, the kernel won't ever return STATUS_QUOTA_EXCEEDED as we literally don't set up a definite limit for regular processes. This situation has to be tackled
in the future.
=== TODO FOR FUTURE ===
Most of the code in PspChargeProcessQuotaSpecifiedPool and PspReturnProcessQuotaSpecifiedPool private routines must be refactored in order to reduce the usage of the quota spin lock, possibly wrapping such code in a loop and whatnot.
CORE-17784
OBP_SYSTEM_PROCESS_QUOTA is a macro that'll be used as a way to assign a dummy quota block to system processes, as we mustn't do anything to those in case the Object Manager is charging or returning pool quotas.
Declare PsReturnSharedPoolQuota and PsChargeSharedPoolQuota prototypes and annotate the functions. Furthermore, add two definitions related to quota pool limits threshold -- PSP_NON_PAGED_POOL_QUOTA_THRESHOLD and PSP_PAGED_POOL_QUOTA_THRESHOLD. For further details, see the commit description of "[NTOS:MM] Add the pool quota prototypes and some definitions".
Declare the MmRaisePoolQuota and MmReturnPoolQuota prototypes in the header and add some definitions related to pool quotas, namely MmTotalNonPagedPoolQuota and MmTotalPagedPoolQuota. These variables are used internally by the kernel as sort of "containers" (for the lack of a better term)
which uphold the amount of quotas that the Process Manager is requesting the Memory Manager to raise or return the pool quota limit. In addition to that, add some definitions needed for both of these functions.
The definitions, MI_CHARGE_PAGED_POOL_QUOTA and MI_CHARGE_NON_PAGED_POOL_QUOTA respectively, bear some interesting aspect. Seemingly the 0x80000 and 0x10000 values (that would denote to 524288 and 65536 specifically) are used as quota "limits" or in other words, thresholds that the kernel
uses. So for example if one would want to raise the quota limit charge, MmRaisePoolQuota will raise it so based on this formula -- NewMaxQuota = CurrentQuota + LIMIT_VALUE. LIMIT_VALUE can be either MI_CHARGE_PAGED_POOL_QUOTA or MI_CHARGE_NON_PAGED_POOL_QUOTA, depending a per quota pool basis.
What's more interesting is that these values are pervasive in Process Manager even. This is when quotas are to be returned back and trim the limit of the quota block if needed, the kernel would either take the amount provided by the caller of quotas to return or the threshold (paged or not paged)
if the amount to return exceeds the said threshold in question.
We have a special file, tag.h, which serves as a place to store whatever kernel pool allocation tag yet we still have some tags sparse over the kernel code... So just re-group them in one unique place.
SIDs are variadic by nature which means their lengths can vary in a given amount of time and certain factors that allow for this happen. This also especially can lead to issues when capturing SIDs and attributes because SeCaptureSidAndAttributesArray might end up overwriting the buffer during the time it's been called.
Therefore when we're copying the SIDs, validate their lengths. In addition to that, update the documentation header accordingly and add some debug prints in code.
- Disentangle the usage of ObpAcquireDirectoryLockExclusive() when it's
used only for accessing a directory structure, or as part of a lookup
operation.
The Obp*DirectoryLock*() -- both shared and exclusive -- functions
are only for locking an OB directory, for reading or writing its
structure members.
When performing lookup operations (insertions/deletions of entries
within a directory), use a ObpAcquireLookupContextLock() function that
exclusively locks the directory and saves extra lock state, that can
be used by ObpReleaseLookupContextObject() for cleanup.
- Add documentation for these functions.
* Quality of service kernel stuff bears nothing with security descriptors in anyway, so just have a file specifically for it
* Annotate the function arguments parameters with SAL
* Document the functions
KD64: Raise to HIGH_LEVEL when entering trap
KDBG: lower to DISPATCH_LEVEL when applying IRQL hack & use a worker thread to load symbols
KD&KDBG: Actually unload symbols when required
ProcessUserModeIOPL, ProcessWow64Information and ThreadZeroTlsCell classes fail on AMD64 build because of wrong IQS values assigned to them. Also explicitly tell the compiler that ProcessUserModeIOPL is strictly for x86 as user mode I/O privilege level is 32-bit stuff.
In addition to that, implement IQS_NO_TYPE_LENGTH macro which it'll be used for classes such as ProcessUserModeIOPL where type length is not required and that it should be 0. With that said, we indirectly fix a size length mismatch issue with ProcessUserModeIOPL on 32-bit of ReactOS as well.
In addition to that, here are some stuff done in this commit whilst testing:
- ICIF_QUERY_SIZE_VARIABLE and friends were badly misused, they should be used only when an information class whose information length size is dyanmic and not fixed. By removing such flags from erroneous classes, this fixes the STATUS_INFO_LENGTH_MISMATCH testcases.
- Use CHAR instead of UCHAR for classes that do not need alignment probing, as every other class in the table do, for the sake of consistency.
- ProcessEnableAlignmentFaultFixup uses BOOLEAN as type size, not CHAR. This fixes a testcase failure on ROS.
- Check for information length size before proceeding further on querying the process' cookie information.
- ProcessHandleTracing wants an alignment of a ULONG, not CHAR.
- Move PROCESS_LDT_INFORMATION and PROCESS_LDT_SIZE outside of NTOS_MODE_USER macro case. This fixes a compilation issue when enabling the alignment probing. My mistake of having them inside NTOS_MODE_USER case, sorry.
- On functions like NtQueryInformationThread and the Process equivalent, complete probing is not done at the beginning of the function, complete probing including if the buffer is writable alongside with datatype misalignment check that is. Instead such check is done on each information class case basis. With that said, we have to explicitly tell DefaultQueryInfoBufferCheck if we want a complete probing or not initially.
A few of these classes have fixed size lengths, the rest are arbitrary. Also the TokenAuditPolicy class hasn't a size length type specified in the table, which is wrong (and move the corresponding TOKEN_AUDIT_POLICY_INFORMATION structure into the private header).
As of now the Object Manager private service, ObpCloseHandleTableEntry, looks for OBJ_PROTECT_CLOSE attribute if a handle should not be closed. However, in ObDuplicateObject if an attribute of OBJ_PROTECT_CLOSE is found as it's been filled to the caller (see L2466) this attribute is removed from the attributes list of the new handle and ObpAccessProtectCloseBit access is granted to the newly duplicated object handle.
With that being said ObpCloseHandleTableEntry indiscriminately closes the object handle albeit it shouldn't do so. As a matter of fact in Windows Server 2003 SP2 this service indeed checks for ObpAccessProtectCloseBit flag bit and if the condition is met then it returns STATUS_HANDLE_NOT_CLOSABLE as it should. Therefore we should do the same.
Now NtClose can properly warn the calling thread the object handle can't be closed which fixes a testcase failure within NtDuplicateObject NTDLL APITEST where this function gives handle close protection bit as requested by the caller.
Properly handle PDE refcounting
Clean-up of the internal API
Enforce attaching to the process when modifying its memory layout, instead of
making circonvoluted mappings which always end up being broken.
- Do not lock the section segment when we are serving a fault for a process private page.
- Do not keep the process address space lock while writing to pagefile.
- Do not wait for an event that might never be set.
These private functions are needed to set up two different kinds of system's anonymous logon tokens: one that includes everyone in the group and the other that doesn't. These functions are needed as next step closer to the
implementation of NtImpersonateAnonymousToken system call.
for manually reported devices, as it is required by the newdev.dll
for installing drivers from INF files
CORE-17212 CORE-17398
Co-authored-by: Stanislav Motylkov <x86corez@gmail.com>
This control class is triggered when a driver is being installed for a
non-critical device. The driver info should already be in the registry
so we just need to push the device through the state graph
Meanwhile, combine the code for similar control classes into
PiControlSyncDeviceAction routine
CORE-17463 CORE-17490
In Windows Server 2003 the lock is initialised on a per-token basis, that is, the lock resource is created in SepDuplicateToken() and SepCreateToken() functions. This ensures that the lock initialisation is done locally for the specific token thus avoiding the need of a global lock.
Do not ditch the pages as soon as the section are unmapped
Improve MmBalancer "algorithm" (or whatever you call that)
Various needed fixes to get this going.
Previously, when creating a file section, Mm requested Cc to cache the file, then Cc would request pages from Mm, then Mm would request them back to serve its file-mapping role
Now, Mm does it all by itself. If file cahcing is requested by the FS driver, then Cc creates a file mapping and uses that to serve its purpose.
This is a rewrite of Cc
During the boot process, it makes possible to initalize the driver's
devices right after the driver is loaded. Moreover, this way one can be
sure that all critical devices are initialized before the
IopMarkBootPartition call (because we explicitly call the driver's
AddDevice routine now, after each driver is loaded)
CORE-7826
- Use DeviceNode->State field and its values, instead of
DeviceNode->Flags for tracking current node state
- Change DNF_* flags to the ones compatible with Windows XP+
- Simplify state changes for device nodes and encapsulate all the logic
inside the PiDevNodeStateMachine routine. This makes the ground for
future improvements in the device removal sequence and
resource management
- Now values inside DeviceNode->State and ->Flags are compatible with
the windbg !devnode macro and can be tracked using it
- BUGFIX: fixed cases where IRP_MN_START_DEVICE or
IRP_MN_QUERY_DEVICE_RELATIONS may be sent to a device after a
IRP_MN_REMOVE_DEVICE
CORE-7826
- Move the driver's name obtaining logic into the IopGetDriverNames
function
- Create a new PiCallDriverAddDevice instead of PipCallDriverAddDevice
and move it to pnpmgr/devaction.c file. Move around all its internal
helpers too
- Support a proper Windows-compatible driver loading order for a PDO
(lower filters, main service, upper filters, etc.)
- Set a correct Problem for the DeviceNode, in case of an error during
driver loading
- Check the Start Type for all drivers before loading
- Do not try to load drivers during the early boot stage when there is
no disk subsystem initialized
- Do not hold the IopDriverLoadResource while trying to reference a
driver object (but still acquire it when we actually need to load a
driver)
- Change IopLoadDriver and IopInitializeDriverModule to use registry
handle instead of a service name string and/or full registry path
- Do not try to reference a driver object inside IopLoadDriver. It's
supposed to be done before the function call
- Split IopLoadUnloadDriver into IopLoadDriver and calling DriverUnload
- Schedule the worker for (un)loading driver in a separate routine
(IopDoLoadUnloadDriver) this allows IopLoadDriver to be called
separately (if we are sure that we're in the system process)
- Remove IopCreateDriver and put its code into IoCreateDriver and
IopInitializeDriverModule. It's hard to extract a meaningful common
part from it
- Refactor IopInitializeDriverModule. Extend and put the DriverName
generation logic into it. Now this function frees the ModuleObject in
case of failure and returns STATUS_FAILED_DRIVER_ENTRY in case of
DriverInit failure (will be used later)
- Convert PARTITION_TABLE_OFFSET to the number of bytes instead of
(number of bytes) / 2. This avoids many confusing casts
- Use a cache aligned buffer for MBR
- BUGFIX: do not call IoGetRelatedTargetDevice while guarded mutex is acquired
(the function issues an APC, but they are disabled inside a critical section)
- BUGFIX: only the beginning of a structure for GUID_PNP_CUSTOM_NOTIFICATION was copied and queued.
Just pass it as-is to a subscriber, without copying
- Don't convert event GUID to string, store and compare GUID struct itself
- Split IopNotifyPlugPlayNotification into 3 functions for each type of notification
(less stack usage and for future changes)
- Move initialization code for notifications into a separate routine
- Use separate lists and locks for every type of notification
- Put "TargetDeviceChange" notifications into their place inside DEVICE_NODE
- Change INIT_FUNCTION and INIT_SECTION to CODE_SEG("INIT") and DATA_SEG("INIT") respectively
- Remove INIT_FUNCTION from function prototypes
- Remove alloc_text pragma calls as they are not needed anymore
Introduce the PiPerformSyncDeviceAction routine for queuing
synchronous device actions
Change all kernel code to use PiPerformSyncDeviceAction and
PiQueueDeviceAction for device enumeration
CORE-10456
