Add support for configuring the CM lazy-flush and delay-close variables:
`CmpLazyFlushIntervalInSeconds`, `CmpLazyFlushHiveCount`,
and `CmpDelayedCloseSize`,
using REG_DWORD values named respectively:
`RegistryLazyFlushInterval`, `RegistryLazyFlushHiveCount`,
and `DelayCloseSize`,
in the registry key
`HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Configuration Manager` .
Extra observations:
- While delay-close support exists in Windows 2003, configuring
the delay-close size is possible only in Windows Vista and later.
- The possibility of configuring the lazy-flush hive count has been
removed in Windows 8+.
See the comparison tables at:
https://redplait.blogspot.com/2011/07/cmcontrolvector.htmlhttps://redplait.blogspot.com/2012/06/cmcontrolvector-for-w8.htmlhttps://redplait.blogspot.com/2016/03/cmcontrolvector-from-windows-10-build.html
In addition:
Remove `CmpDelayedCloseIndex` from cm.h as it is not used anymore in our code.
Its support is controlled with a REG_DWORD value named
`VolatileBoot` inside the registry key
`HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Configuration Manager` .
Its implementation is very trivial: when enabled, CmpShareSystemHives
is set to TRUE and doesn't change state during runtime.
In a sense this is similar to what happens in WinPE-boot, except that
ALL hives without exception (system hives and others) will be loaded
in shared mode.
This is controlled with a REG_DWORD value named
`SelfHealingEnabled` inside the registry key
`HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Configuration Manager` .
The `CmSelfHeal` variable is used to retrieve the data,
hence it needs to be a ULONG.
The REG_DWORD value `AutoReboot` stored in the registry key
`HKLM\SYSTEM\CurrentControlSet\Control\CrashControl`
(used as a boolean value), controls whether to automatically reboot
the operating system at the end of the crash dump, after a bugcheck.
ReactOS doesn't currently implement crash dumps, so only the auto-reboot
is done. (The reason of the apparent redundant `Reboot` variable in
KeBugCheckWithTf() is because that variable would be updated on return
by the not-yet-existing crash-dump helper routines called from there.)
The PR #6649 which fixed an issue with orphaned KCBs leaking in memory which also pointed to unloaded registry hives, it also brought a problem.
In CmpEnumerateOpenSubKeys there is a risk of getting hit by a deadlock as we enumerate the cache table to remove empty cache entries.
Fundamentally CmpEnumerateOpenSubKeys locks down a KCB from cache for exclusive use in order to tear down its contents from memory but it doesn't address the fact a KCB might have already been locked in the same calling thread, leading to a recursion.
This leads to random hangs when unloading a hive during system startup (tipically on a clean install).
The solution here is to simply lock the whole registry when we unload a hive so that we don't have to worry the KCBs are getting tampered by anybody else. This also simplifies the code.
Although locking the entire registry while other apps are doing registry related operations to other hives can cause overhead. If this turns out to be bad then we have to rethink the locking mechanism here.
CORE-19539
In case of failure, ObInsertObject itself dereferences the object, so calling ObDereferenceObject one more time in the failure handler is redundant.
- Remove ObDereferenceObject call from CmpCreateRegistryRoot in case ObInsertObject fails.
- Add missing RootKey dereference if CmpCreateRegistryRoot failed to get the key cell (KeyCell).
CORE-17904
A hive whose KCBs have a reference count of 0, meaning nobody is using these keys anymore, will not get removed from the cache table.
As a result during a normal hive unloading operation you will get orphaned KCBs which results in an unload failure.
This is wrong, because this is what a normal hive unloading is supposed to do. What it cannot do of course is that it cannot
scramble the references of opened keys by the users who use the Registry, as it is the job of force unloading mechanism to do that.
Also remove a misleading debug print. Force unloading works as intended by scrambling the references of keys and marking the KCB for deletion,
which is what how a force unload works. Namely Windows does exactly that.
CORE-10705
- Annotate the CmpEnumerateOpenSubKeys function with SAL2
- When removing an orphaned cached KCB, ensure that it is locked before clearing it from cache table entries
+ Improve related comments.
Registry hives are opened in shared read access when NT is loaded in PE
mode (MININT) or from network (the hives residing on a network share).
This is true in particular for the main system hives (SYSTEM, SOFTWARE,
DEFAULT, ...).
However, in PE mode, we can allow other hives, e.g. those loaded by the
user (with NtLoadKey) to be loaded with full read/write access, since we
boot from a local computer.
In particular remove some extra-parentheses around single code tokens,
and replace few "DPRINT1 + while (TRUE);" by UNIMPLEMENTED_DBGBREAK.
+ Improve some comments.
Sometimes repairing a broken hive with a hive log does not always guarantee the hive
in question has fully recovered. In worst cases it could happen the LOG itself is even
corrupt too and that would certainly lead to a total unbootable system. This is most likely
if the victim hive is the SYSTEM hive.
This can be anyhow solved by the help of a mirror hive, or also called an "alternate hive".
Alternate hives serve the purpose as backup hives for primary hives of which there is still
a risk that is not worth taking. For now only the SYSTEM hive is granted the right to have
a backup alternate hive.
=== NOTE ===
Currently the SYSTEM hive can only base upon the alternate SYSTEM.ALT hive, which means the
corresponding LOG file never gets updated. When time comes the existing code must be adapted
to allow the possibility to use .ALT and .LOG hives simultaneously.
If FreeLdr performed recovery against the SYSTEM hive with a log, all of its data is only present in volatile memory thus dirty. So the kernel is responsible to flush all the data that's been recovered within the SYSTEM hive into the backing storage.
In addition to that, in some functions like CmFlushKey, CmSaveKey and CmSaveMergedKeys we must validate the underlying hives as a matter of precaution that everything is alright and we don't fuck all the shit up.
CmCheckRegistry is a function that provides the necessary validation checks for a registry hive. This function usually comes into action when logs have been replayed for example, or when a registry hive internals have changed such as when saving a key, loading a key, etc.
This commit implements the whole Check Registry infrastructure (cmcheck.c) in CMLIB library for ease of usage and wide accessibility across parts of the OS. In addition, two more functions for registry checks are also implemented -- HvValidateHive and HvValidateBin.
Instead of having the CmCheckRegistry implementation in the kernel, it's better to have it in the Configuration Manager library instead (aka CMLIB). The benefits of having it in the library are the following:
- CmCheckRegistry can be used in FreeLdr to fix the SYSTEM hive
- It can be used on-demand in the kernel
- It can be used for offline registry repair tools
- It makes the underlying CmCheckRegistry implementation code debug-able in user mode
CORE-9195
CORE-6762
During a I/O failure of whatever kind the upper-level driver, namely a FSD, can raise a hard error and a deadlock can occur. We wouldn't want that to happen for particular files like hives or logs so in such cases we must disable hard errors before toying with hives until we're done.
In addition to that, annotate the CmpFileSetSize function's parameters with SAL.
When shutting down the registry of the system we don't want that the registry in question gets poked again, such as flushing the hives or syncing the hives and respective logs for example. The reasoning behind this is very simple, during a complete shutdown the system does final check-ups and stuff until the computer
shuts down.
Any writing operations done to the registry can lead to erratic behaviors. CmShutdownSystem call already invokes a final flushing of all the hives on the backing storage which is more than enough to ensure consistency of the last session configuration. So after that final flushing, mark HvShutdownComplete as TRUE indicating
that any eventual flushing or syncying (in the case where HvSyncHive gets called) request is outright ignored.
During an open or create procedure of a registry key, the registry parser grabs
a key control block (KCB) from the parser object and uses its information to do the
necessary work in order to obtain a pointer to the newly created or opened registry key.
However, the registry parsers faces several issues. First, we don't do subkey cache cleaning
information against gathered KCBs so whenever we do a registry parse we end up with KCBs
that have cache inconsistencies. Moreover we don't do any locking of whatever KCB we
are grabing during a parse procedure.
=== PROPOSED CHANGES ===
* Implement CmpComputeHashValue and CmpLookInCache functions. With CmpComputeHashValue we can
compute the convkey hashes of each subkey in the path name of a key so we can lock them
with CmpBuildAndLockKcbArray. CmpLookInCache is a function that searches for the suitable
KCB in the cache. The factors that determine if a KCB is "suitable" are:
-- the currently found KCB in the hash list has the same levels as that of the
given KCB from the parse object;
-- The key names from the computed hash values match with the block name of
the KCB;
-- The currently found KCB is not deleted.
The KCB will be changed if the key path name points to a partial match name in
the cache. The KCB from the parse object will be used if we have a full match
of remaining levels.
* Add missing CMP_LOCK_HASHES_FOR_KCB flags on CmpCreateKeyControlBlock calls
that create KCBs during a parse procedure. Such lock has to be preserved until
we're done with the registry parsing.
* On CmpDoCreateChild, preserve the exclusive lock of the KCB when we are
enlisting the key body.
* On CmpDoCreate, make sure that the passed parent KCB is locked exclusively and
lock the hiver flusher as we don't want the flusher to kick in during a key
creation on the given hive. Cleanup the subkey info when we're creating a key
object. Also implement missing cleanup path codes. Furthermore, avoid key
object creation if the parent KCB is protected with a read-only switch.
* Soft rewrite the CmpDoOpen function, namely how we manage a direct open vs
create KCB on open scenario. When a KCB is found in cache avoid touching
the key node. If the symbolic link has been resolved (aka found) then lock
exclusively the symbolic KCB. Otherwise just give the cached KCB to the caller.
If it were for the caller to request a KCB creation, we must check the passed
KCB from the parser object is locked exclusively, unlike on the case above
the caller doesn't want to create a KCB because there's already one in the cache.
We don't want anybody to touch our KCB while we are still toying with it during
its birth. Furthermore, enlist the key body but mind the kind of lock it's been
used.
* On CmpCreateLinkNode, avoid creating a key object if the parent KCB is protected
with a read-only switch. In addition, add missing hive flusher locks for both
the target hive and its child. Cleanup the subkey information of the KCB when
creating a link node, this ensures our cached KCB data remains consistent.
* Do a direct open on CmpParseKey if no remaining subkey levels have been found
during hash computation and cache lookup, in this case the given KCB is the
block that points to the exact key. This happens when for example someone tried
to call RegOpenKeyExW but submitting NULL to the lpSubKey argument parameter.
CORE-10581
ROSTESTS-198
CmpSecurityMethod is a method used by the Object Manager and called by this
subsystem whenever a security operation has to be done against a key object.
As CmpSecurityMethod is a specific OB construct we should not make any direct
call attempts to CmpSecurityMethod, only OB is responsible for that. This fixes
a deadlock where CmpSecurityMethod acquires a push lock for exclusive access
even though such lock is already acquired by the same calling thread in
CmpDoCreateChild.
This prevents a deadlock in DelistKeyBodyFromKCB when we delete a key
object because of an access check failure during a open procedure of a
registry key, as we are already holding a lock against the target KCB of
the key body.
Whenever a security request is invoked into a key object, such as when requesting
information from its security descriptor, the Object Manager will execute
the CmpSecurityMethod method to do the job.
The problem is that CmpSecurityMethod is not aware if the key control block
of the key body already has a lock acquired which means the function will attempt
to acquire a lock again, leading to a deadlock. This happens if the same
calling thread locks the KCB but it also wants to acquire security information
with ObCheckObjectAccess in CmpDoOpen.
Windows has a hack in CmpSecurityMethod where the passed KCB pointer is ORed
with a bitfield mask to avoid locking in all cases. This is ugly because it negates
every thread to acquire a lock if at least one has it.
The CmpUnLockKcbArray, CmpLockKcbArray and CmpBuildAndLockKcbArray routines
help us to lock KCBs within array so that information remains consistent when
we are doing a cache lookup during a parse procedure of the registry database.
Now that the Memory Management is a bit more under control again,
and branching releases/0.4.15 is near,
do mute some frequent log-spam that got introduced during 0.4.15-dev'ing
regarding lazy-flushes and MM balancing.
It frequently logged even while being idle.
Slightly improve the headers of the two touched files.
No rocket-science.
I intend to port back the combined work of Thomas Faber and Serge Gautherie in context of CORE 14271.
Both developers fixed wrong retval evaluations for SeSinglePrivilegeCheck() and RtlCreateUnicodeString().
Both functions do return a BOOLEAN, and therefore using NTSTATUS() on them is wrong.
Those bugs have been fixed at multiple places. That is long gone.
But Serge fixed his locations a bit more elegantly, without the need for additional variables.
Therefore this addendum adapts a few of Thomas locations to the improved Serge-ified style.
Yes: I intentionally used a space instead of a minus after the mentioned CORE 14271,
as I don't want that pure stylistic addendum to be linked with the initial ticket anymore.
That would be overkill.
Currently the failure code path doesn't do any kind of cleanup against
the hive that was being linked to master. The cleanup is pretty
straightforward as you just simply close the hive file handles and free
the registry kernel structures.
CORE-5772
CORE-17263
CORE-13559
Log debug output of the lazy flusher as much information as possible so that we can examine how does the lazy flusher behave, since it didn't work for almost a decade. Verbose debugging will be disabled once we're confident enough the registry implementation in ReactOS is rock solid.
Whenever ReactOS finishes its operations onto the registry and unlocks it, a lazy flush is invoked to do an eventual flushing of the registry to the backing storage of the system. Except that... lazy flushing never comes into place.
This is because whenever CmpLazyFlush is called that sets up a timer which needs to expire in order to trigger the lazy flusher engine worker. However, registry locking/unlocking is a frequent occurrence, mainly when on desktop. Therefore as a matter of fact, CmpLazyFlush keeps removing and inserting the timer and the lazy flusher will never kick in that way.
Ironically the lazy flusher actually does the flushing when on USETUP installation phase because during text-mode setup installation in ReactOS the frequency of registry operations is actually less so the timer has the opportunity to expire and fire up the flusher.
In addition to that, we must queue a lazy flush when marking cells as dirty because such dirty data has to be flushed down to the media storage of the system. Of course, the real place where lazy flushing operation is done should be in a subset helper like HvMarkDirty that marks parts of a hive as dirty but since we do not have that, we'll be lazy flushing the registry during cells dirty marking instead for now.
CORE-18303
- Add a new cmboot.h header to isolate the boot-support definitions
shared with the NT/ReactOS bootloader.
- Move CmpFreeDriverList() to cmboot.c so that we can use it for
cleanup paths in the NT/ReactOS bootloader.
- CmpFindControlSet(): Directly build the control set name in UNICODE,
instead of doing an ANSI->UNICODE conversion.
- Directly assign the CurrentControlSet\Services constant string,
instead of going the route of init-empty-string + append-string.
This is possible since that string is not modified later.
- Remove ASSERT(FALSE), replacing them with correct failure handling.
- Add cleanup paths in CmpAddDriverToList().
- Simplify and fix CmpFreeDriverList(): it's the full DriverNode
that needs to be freed; not the LIST_ENTRY pointer.
- Add other validity checks:
* Registry value types and data sizes;
* For multi-strings, verify that they are NULL-terminated.
* For (multi-)strings, check whether they are NULL-terminated before
optionally removing their trailing NULL character from the count.
Check also whether they are of zero-length and take appropriate
action where necessary.
- Add CmpIsDriverInList() for future usage in CMBOOT compiled in
bootloader mode.
- Add SAL annotations and Doxygen documentation.
- Add debug traces.
- Formatting / code style fixes.
** TODO: Fix SafeBoot support **
Also generate processor identifier properly based on this value
on the Configuration Manager machine-dependent initialization.
Update processor driver INF file accordingly.
CORE-17970 CORE-14922
If the CPU does support CPUID instruction but does not have the Brand String,
the legacy name detection should be performed either in CIM_Processor (WMI),
in the Processor device driver, or in the application itself.
The deleted Intel CPU name detection code can go to the WMI at some point.
In fact Windows XP and 2003 kernels do not have any strings of that kind.
This effectively reverts the hack committed 6 years ago in 7a985425 (r63798)
that was intended to workaround CORE-7952.
CORE-17413
