The registers that are saved/restored in the trap / exception frame need to be consistent between all entry/exit points as well as the functions that convert between trap/exception frame and context. The trap frame contains only the non-volatile registers and rbp, the rest is saved in the exception frame.
The previous code didn't save rbp in the syscall handler, which led to it being clobbered when exiting though KiServiceExit2 rather than returning back to the syscall exit path. Also KeContextToTrapFrame would use rbx, rsi and rdi from the trap frame, which wouldn't be saved there by the syscall handler.
This fixes invalid contexts being passed to RtlDispatchException. Also update the ExceptionAddress field in the EXCEPTION_FRAME, to match the Rip value in the CONTEXT, which is required for proper unwinding.
- For NT6.0+, NTDLL exports those hash APIs that ADVAPI32 thunks forward to
- Fix undocumented context structure layout and content
- Add apitests to verify both of hash result and structure content
[MMIXER] Implement hadling WAVE_FORMAT_EXTENSIBLE audio format
Fix opening audio device error when trying to open it with WAVE_FORMAT_EXTENSIBLE format tag set in wFormatTag member of WAVEFORMATEX:
- Pass additional data size from WAVEFORMATEX.cbSize to pin data format.
- Append it to the whole size of pin data format (KSDATAFORMAT.FormatSize).
- Set additional fields in WAVEFORMATEXTENSIBLE structure (data format, BPS and channel mask) when WAVE_FORMAT_EXTENSIBLE is used. They are used by our inbuilt Intel AC97 miniport driver at least. It simply fails when these members are not set.
- Fix pin connect allocation size by appending an additional data size from WAVEFORMATEX.cbSize to KSPIN_CONNECT + KSDATAFORMAT + WAVEFORMATEX. This allows to proerly initialize additional extensible data and avoids kernel memory leakage when using extensible audio format.
- Remove format tag check which allowed WAVE_FORMAT_PCM to be opened correctly. So now all possible audio formats can be opened properly at least (although it does not mean they may work correctly).
This fixes the audio playback for all apps those are supporting extensible audio data and use it by default (e. g. AIMP 5.30, QMMP 0.12.17, all Chrome/Chromium-based browsers, GameDev Tycoon Demo game etc.).
CORE-10907, CORE-14783
* [KS] Allow passing additional extensible audio data when extensible audio format is used
- Append additional data size from WAVEFORMATEX.cbSize to pin connect size passed to KsCreatePin. If the tag is WAVE_FORMAT_PCM, then this member should always be zero. So in that case, no any additional data is passed to creation request, and the passed data size is correct for PCM too (KSDATAFORMAT + WAVEFORMATEX).
This fixes audio playback in several apps those are supporting extensibble audio and use it by default (e. g. AIMP 5.30, QMMP 0.12.17, all Chrome/Chromium based browsers, GameDev Tycoon Demo game etc.).
CORE-10907, CORE-14783.
* [WDMAUD.DRV] Pass the correct additional data size to I/O control request
Store a correct size of additional data in WAVEFORMATEX.cbSize when performing open of audio device, when WAVE_FORMAT_EXTENSIBLE audio format is used.
It allows to properly open audio device with Legacy APIs enabled too.
This fixes audio playback in several apps those are using extensible audio data (e. g. AIMP 5.30, QMMP 0.12.17, all Chrome/Chromium based browsers, GameDevTycoon Demo game etc.).
CORE-10907, CORE-14783
This can fail on optimized builds, where functions can end after a noreturn function call without an epilog / ret, but with an int 3. We simply fail gracefully, which is the right thing to do.
This also fixes delivering APCs from the system call handler, which previously would have clobbered rax. Also don't use the thread's TrapFrame member, which is not always set, when returning.
- Implement `GetFirmwareType` and improve existing API test for it
- Move (Get/Set)FirmwareEnvironmentVariableEx(A/W) to kernel32_vista_static and export them when NT version >= 6.2
Addendum to 4c8a2a8815. CORE-11954
Simplify code and reduce binary size.
JIRA issue: CORE-19363
- Use shlwapi!QISearch for QueryInterface
implementation.
- Add delay link to shlwapi.
- Define QITABENT macro in <shlwapi.h>.
Follow-up to #6624 (fd96666). ReactOS got rekt in that PR
because kernel32.dll got so larger.
Fix base addresses of DLLs to boot successfully. Sorry.
(dll/ntdll/ldr/ldrutils.c:1423) Illegal DLL relocation!
C:\ReactOS\System32\ntdll.dll overlaps kernel32.dll
*** Fatal System Error: 0xc0000269
Proposed changes
After latest build:
python gen_baseaddress.py C:\Users\katahiromz\reactos-1\output-VS-i386
python gen_baseaddress.py C:\Users\katahiromz\reactos-1\output-MinGW-i386
shell32!SHRestricted is necessary to make shell working well.
JIRA issue: CORE-11515
- Add dll/win32/shell32/wine/PolicyData.h to define the shell policy data.
- Define POLICYDATA structure and SHELL_NO_POLICY constant
in <shlwapi_undoc.h>.
- Fix shell32!SHRestricted, shell32!SHSettingsChanged,
shlwapi!SHGetRestriction, and shlwapi!SHRestrictionLookup functions.
The definitions in the NDK inline_ntcurrentteb.h and psfuncs.h files are
synced togther with those of winnt.h (in the winnt_old.h generator file).
See commits c1929445d (r49627) and 925c587c8 (r56318).
IMPORTANT NOTE: The ARM64 definition does not exist yet in our headers!
The official MS DDK/WDK does not expose KeGetPcr() for x86, so do
not expose it there as well. Use instead the private NDK definition.
If it were exposed in the DDK/WDK, it would have to be a
multiprocessor-compatible definition.
Note that the broken definition was working only in single-processor
mode, using the PCR static memory pointer value.
The official MS DDK/WDK exposes KeGetPcr() as an alias to such a
PCR value only for IA64, MIPS and PPC, which is of course not great.
Unify the private IMM32 macro
definitions. Refactoring...
JIRA issue: CORE-19361
- Define IS_SPECIAL_HKL macro
in <imm32_undoc.h>.
- Use IS_IME_HKL and
IS_SPECIAL_HKL macros of
<imm32_undoc.h>,
in kbswitch and input modules.
Implementing the back-end of
the Language bar...
JIRA issue: CORE-19361
- Add mlng.cpp and mlng.h.
- Modify msctf.spec.
- Implement MLNGINFO
structure and CStaticIconList
class.
- Implement TF_InitMlngInfo,
TF_MlngInfoCount,
TF_InatExtractIcon,
TF_GetMlngIconIndex, and
TF_GetMlngHKL functions.
This was a MinGW-specific, non-MS-DDK/WDK-compatible define, that was
used to mark NTOS kernel/hal exports, instead of NTSYSAPI etc.
We have since fixed that, and changed the way Freeldr (and rossym)
manages these, see commits:
186c8b72d (r16028), 51f0dfd30 (r17651) and 526efd2ee (r24359)
Used in `ATL::CStringW` and `ATL::CStringA`, for example:
```
ATL::CStringW cswItemText = "";
cswItemText.Preallocate(64);
SendDlgItemMessageW(pdis->CtlID, LB_GETTEXT, pdis->itemID, reinterpret_cast<LPARAM>(cswItemText.GetBuffer()));
cswItemText.ReleaseBuffer();
```
This public function is used to allocate memory for the string via private PrepareWrite, but it's missing somehow. Now it shouldn't be.
Supporting the Language bar...
JIRA issue: CORE-19363
- Delete TFInitLib and TFUninitLib
calls from ctfmon.exe.
- Delete TFUninitLib from Cicero
library.
- Implement InitDisplayAttrbuteLib
in msctfime.ime.
- Improve CIC_LIBTHREAD structure.
Refactoring and reduce binary size.
JIRA issue: CORE-19268
- Add cicero static library in sdk/lib/cicero folder.
- Delete sdk/include/reactos/cicero folder.
- Adapt the dependencies to these changes.
- Make ctfmon, msutb, and msctf modules UNICODE.
Improve header compatibility and
code quality.
JIRA issue: CORE-19268
- Improve <imm.h> and <immdev.h>
compatibility by correctly choosing
the items.
- Use <immdev.h> instead of
<ddk/immdev.h>.
- Move INPUTCONTEXTDX, IMEINFOEX,
IMEDPI, and CLIENTIMC into
<imm32_undoc.h>.
- Adapt to these changes.
Use the MS PSDK-compatible name _INC_MALLOC, instead of the MinGW one
_MALLOC_H_, so that code that may depend on this to determine whether
malloc.h functions are defined, can compile without problems.
Supporting Language Bar...
JIRA issue: CORE-19363
- Add many global variables
to add UTB settings.
- Add IsSkipRedrawHKL, IsBiDiLocalizedSystem,
InitSkipRedrawHKLArray helper functions.
- Add InitFromReg helper function to
load the settings from registry.
- Add CicRegKey::EnumValue method.
+ Improve related comments.
Registry hives are opened in shared read access when NT is loaded in PE
mode (MININT) or from network (the hives residing on a network share).
This is true in particular for the main system hives (SYSTEM, SOFTWARE,
DEFAULT, ...).
However, in PE mode, we can allow other hives, e.g. those loaded by the
user (with NtLoadKey) to be loaded with full read/write access, since we
boot from a local computer.
In particular remove some extra-parentheses around single code tokens,
and replace few "DPRINT1 + while (TRUE);" by UNIMPLEMENTED_DBGBREAK.
+ Improve some comments.
Add a basic IOpenControlPanel implementation that supports Vista canonical registry names.
Implements `control.exe /name company.name [/page id]` and `IOpenControlPanel`
handling of Vista-style canonical registry names.
The documented `Microsoft.*` names don't work because they are simply not
in our registry but "[Executable Control Panel Items](https://learn.microsoft.com/en-us/windows/win32/shell/how-to-register-an-executable-control-panel-item-registration-)" registered by 3rd-party ISVs
will function correctly in control.exe and the COM API.
Notes:
- `IOpenControlPanel` is implemented in CControlPanelFolder.cpp because
it is supposed to have tighter integration with that shell folder than
it does in this PR.
- `IOpenControlPanel` is also supposed to handle .cpl files with canonical
names registered under [`Extended Properties`](https://learn.microsoft.com/en-us/windows/win32/shell/how-to-register-dll-control-panel-item-registration-#step-3) but the control panel folder
does not implement `IShellFolder2::GetDetailsEx` yet, so it will have to wait.
- These "Executable Control Panel Items" are also supposed to be displayed
in the control panel itself but this PR does not address that. The
`ITEMIDLIST` format for those needs investigation...
- The Wow64 handling is perhaps not correct but it does not matter,
`ShellExecuteEx` gets to deal with whatever is in the `...\shell\open\command` key.
`CControlPanelFolder` would have to take more care when it starts
reading those keys so it knows when to append "(32-bit)" to the display name.
- `%s%s` because .cpl canonical names don't have the `::` prefix according
to Geoff Chappell.
- Always returns `CPVIEW_CLASSIC` because our `CControlPanelFolder` does
not support the category view.
GCC 8.4 and Clang 13.0.1:
`cc1: note: unrecognized command-line option ‘-Wno-deprecated-non-prototype’ may have been intended to silence earlier diagnostics`
Addendum to commit 4e3bf252d
* [SDK][VIRTIO][NETKVM] Make VirtIO a separate library
This is to avoid code duplication when more VirtIO drivers are brought in. This will also be used on development of a VirtIO XDDM GPU Driver.
* [VIRTIO] Sync with upstream
This is needed, because Wine code expects RtlUnwind to restore the non-volatile registers, before returning, but ours / the native one doesn't do that.
Should fix CORE-19392 and CORE-19397
Supporting TIPs...
JIRA issue: CORE-19360
- Fix some mistakes of
CUIFButton::OnMouseIn and
CUIFButton::OnMouseOut.
- Fix some mistakes of CUIFObject
and CUIFWindow.
Improve readability and reduce code.
JIRA issue: CORE-19268
- Delete cicSetLayout function and
use gdi32!SetLayout function directly.
- Don't use ::SetRect and ::SetRectEmpty.
- s/CUIFButton::m_dwUnknown10/CUIFButton::m_bPressed/.
- Reduce lines.
Supporting TIPs...
JIRA issue: CORE-19360
- Implement cicInitUIFLib and cicDoneUIFLib functions.
- Use them in msctfime and msutb.
- Add CUIFSystemInfo class in <cicero/cicuif.h>.
Supporting TIPs... UIComposition class requires a caret class.
JIRA issue: CORE-19360
- Add <cicero/ciccaret.h> header.
- Add CicCaret class for displaying a caret.
Use new(cicNoThrow) instead of plain operator new.
JIRA issue: CORE-19360
- Define CicNoThrow structure and
cicNoThrow macro in <cicero/cicbase.h>.
- Use new(cicNoThrow) instead of
plain operator new.
This is actually a Vista+ function, but we need it enable umpnpmgr.dll to notify services of device events. Up until WinXP it was possible to make direct calls to the service manager by umpnpmgr.dll because umpnpmgr.dll was loaded into the service manager process. From Vista onwards umpnpmgr.dll is run as a separate service host process. And since ReactOS always ran umpnpmgr.dll as a separate process, we have to use the Vista RPC functions to notify services.
They are renamed to 'msi_custom' and 'msi_selfreg', respectively.
This is to avoid any future build target name collisions with other
tests that would also use similarly-named targets. (For example, a
future setupapi wine-synced commit.)
Supporting TIPs and Language Bar...
JIRA issue: CORE-19361
- Implement InitLangChangeHotKey function.
- Add some global variables about hot-keys.
- Fix CicRegKey in <cicero/cicreg.h>.
Cicero interface is not Unicode (W)
but ANSI (A).
- ctfmon.exe is ANSI, not Unicode.
- msutb.dll is ANSI, not Unicode.
- Apply generic text mapping to the
cicero headers.
- Include <tchar.h> to use generic
text mapping.
CORE-19361, CORE-19362, CORE-19363
msutb.dll is the GUI back-end
of Language Bar (Tipbar).
- Add msutb.dll module at
dll/win32/msutb/.
- The implementation of
msutb.dll is currently stub.
- Modify msctf.spec and msctf.idl.
- Add <cicero/cicutb.h>.
- Adapt ctfmon.exe to these changes.
CORE-19362, CORE-19363
Make LdrUnlockLoaderLock Cookie type consistent with LdrLockLoaderLock
and LdrpMakeCookie functions.
In addition:
* Adjustment of the formatting string for ULONG_PTR;
* ntdll.spec: specify ptr for 2nd parameter of LdrUnlockLoaderLock:
This is a parameter whose length is pointer-like, platform-dependent.
Co-authored-by: Hermès BÉLUSCA - MAÏTO <hermes.belusca-maito@reactos.org>
- Rename _IMCCLock as IMCCLOCK.
- Rename InternalIMCCLock as IMCCLock.
- Rename _IMCLock as IMCLOCK.
- Add get() pointer accessor of IMCCLock and IMCLOCK.
- Protect the pointer of IMCCLOCK and IMCLOCK.
CORE-19360
- Add CTFIMECONTEXT structure.
- Add CicInputContext class.
- Rename INPUTCONTEXTDX.dwUnknown5 as
hCtfImeContext and retype it as HIMCC.
- Implement CtfImeGetGuidAtom by using them.
CORE-19360
- Add link to imm32.dll.
- Add <cicero/imclock.h>.
- Add INIT_GUIDMAP constant to <immdev.h>.
- Implement CtfImeIsGuidMapEnable by using them.
CORE-19360
- Simplify patch directory usage;
- Fix the path shown in the warning message.
The staging patch path in the warning message didn't show the correct
sub-directory where the patch resides.
This fixes in particular problems when reverting created new files (i.e.
they are deleted). I suspect this is due to an endline-format "expectation"
from the git apply command.
This now correctly sets the commit author as 'winesync' (before it was
the committer's developer name) for the following types of commits:
`[WINESYNC]: revert wine-staging patchset for <MODULE_NAME>`
and
`[WINESYNC]: <MODULE_NAME> is now in sync with wine-staging <WINE_TAG>`
Expand on Timo's commit 4b5a55516.
We may encounter cases where either the 'directories' or 'files' lists
in the .cfg YAML files are empty, and we don't want the script to throw
an exception in that case.
Furthermore, explicitly check for such empty lists when calling PyGit2
index.add_all(...) function, because if it's called on a None or empty
list, _all_ untracked files in the selected git repository get added,
which is not what we want there.
Wine-Staging switched to staging/patchinstall.py , removing the
deprecated patches/patchinstall.sh in Feb.16, 2023 commit
c1b4af92f7
just before the v8.2 release.
In order to maintain interoperability between older and newer
Wine-Staging versions, try to run first the new script; if it fails,
fall back to the older script.
Whenever using git, be it on Windows CMD, git-bash on Windows, or in
*nix systems, git appears to always use posix-like paths internally.
So, enforce them with posixpath when building/concatenating paths that
are going to be passed to pygit2 functions.
Otherwise, exceptions like the following one happens:
```
File "./winesync.py", line 296, in revert_staged_patchset
self.reactos_index.remove(os.path.join(self.staged_patch_dir, patch_file_name))
[... skipped ...]
OSError: index does not contain sdk\tools\winesync\setupapi_staging\0002-wine-staging-4.0-setupapi_winetest.patch at stage 0
```
(The git index actually contains the specified path, but in "posix" form
with slash-separators.)
On *nix platforms, these changes should not matter.
ctfmon.exe will be a replacement of our
kbswitch.exe in the future. That is the
front-end of Language Bar. It is needed
to support TIPs.
- Add ctfmon.exe at base/applications/ctfmon.
- Add <cicero/cicbase.h>,
<cicero/CModulePath.h>, and
<cicero/osinfo.h> headers and use them.
CORE-19362
## Overview
1. msctfime.ime is an IME file interface
for new-style IMEs a.k.a. "Text Input
Processors" (TIPs).
2. msctfime.ime is loaded as old-style
IME file at ImmLoadLayout in specific
condition.
3. msctfime.ime communicates with
the current TIP (This feature is not
implemented yet).
## Proposed changes
- Add msctfime module at dll/ime/msctfime.
- The functions in this module are currently
stub.
- Move IME file interface declarations from
<imm.h> to <ddk/immdev.h>.
- Modify ImmNotifyIME, NotifyIME, and
ImeProcessKey prototypes for x64
compliance.
CORE-19360
Complete CTF IMM.
- Add CtfImeProcessCicHotkey and CtfImeSetActiveContextAlways
to access CTF IMEs.
- Check whether Cicero is started in the current thread in ImmProcessKey.
- Call CtfImeProcessCicHotkey if necessary in ImmProcessKey.
- Modify <CtfImeTable.h>.
- Add CI_CICERO_STARTED flag to "ntuser.h".
- Fix Imm32JCloseOpen.
CORE-19268
- Add IMM32!GetKeyboardLayoutCP function.
- Add it to <imm32_undoc.h>.
- Delete ImmDisableLegacyIME and
ImmSendMessageToActiveDefImeWndW functions
(2k3/xp IMM32 doesn't have such functions).
- Modify imm32.spec.
- Refer ntdll!RtlDllShutdownInProgress function (that is WinXP+).
CORE-19268
- Add Imm32InitTLS, Imm32AllocateTLS, Imm32GetTLS,
Imm32GetCoInitCountSkip, Imm32IncCoInitCountSkip, and
Imm32DecCoInitCountSkip helper functions to
control the TLS data.
- Introduce "CoInitialize Spy" (ISPY) to manage COM
initialization status.
- Implement CtfImmCoInitialize and CtfImmCoUninitialize.
- Implement CtfImmEnterCoInitCountSkipMode and
CtfImmLeaveCoInitCountSkipMode.
- Implement CtfImmLastEnabledWndDestroy,
ImmDisableTextFrameService, and CtfImmTIMActivate.
CORE-19268
Implement PathIsEqualOrSubFolder function.
- Add it to <undocshell.h>.
- Add PathIsEqualOrSubFolder testcase.
- Add SHGetPathCchFromIDListW as an
extension of SHGetPathFromIDListW.
CORE-19278
CORE-12686
Isolate PathCch* functions from Wine implementation in kernelbase
(which are mixed with a ton of unrelated stuff).
These functions are compiled into a pathcch.lib library, similarly
to the one in the official MS PSDK. Excepting that here, their actual
implementation is in the library. This contrasts with the one in the
MS PSDK, which is an import library to an apiset DLL.
The pathcch.h header is an original one, that contains SAL annotations
and descriptive parameter names, based on the MinGW and MS PSDK headers.
Wine's header was not used as it is poor and incomplete.
Co-authored-by: Katayama Hirofumi MZ <katayama.hirofumi.mz@gmail.com>
Implementing advanced text service...
- Add CtfImmGenerateMessage function.
- Modify imm32.spec.
- Move TRANSMSG, TRANSMSGLIST etc. in
win32ss/include/ntuser.h to <immdev.h>.
- Move win32ss/include/imetable.h to
sdk/include/reactos/imetable.h.
- Move dll/win32/imm32/CtfImeTable.h to
sdk/include/reactos/CtfImeTable.h.
- Fix build failure of imm32_winetest due
to TRANSMSG redefinition.
CORE-19268
- Rename version.rc as imm32.rc.
- Add resource.h and t1keys.h.
- Add some resource bitmaps.
- Modify <immdev.h>.
Test:
Press Ctrl+Alt+comma on FreeCJ2004.
NOTE: There's a visual bug in PatBlt with negative values.
CORE-19268
Revert "[PSDK][AFD][VGADDI] Further build and MS PSDK compatibility fixes."
This reverts commit 99efc2ae50.
Revert "[PSDK] Sync winresrc.h with wine-8.20"
This reverts commit 3e83562aa7.
Revert "[OSK][PROGMAN] Fix resource file build."
This reverts commit 84e4ad0a82.
Revert "[PSDK] Use the new .rh files in winresrc.h"
This reverts commit f6fb7c48c9.
- windows.h:
* should define _WINDOWS_ and _INC_WINDOWS instead of _WINDOWS_H
* include winresrc.h if RC_INVOKED and not NOWINRES (Wine also agrees
with that)
* also, some .h included from there should _not_ be included if their
respective NO*** flags are defined.
- batclass.h: Use correct _WINDOWS_ guard.
- winsnmp.h:
* should define _INC_WINSNMP instead of _WINSNMP_H
* use _INC_WINDOWS instead of _WINDOWS_H
- afd.h, vgaddi.h: Use the correct guards.
- winnt.rh: Define the (SUB)LANG_* exactly as they are in the winnt.h,
otherwise we get macro redefinition compile errors.
NOTE: Ideally these .rh files should be auto-generated from their
corresponding .h files.
- afxres.h: Do not include the whole windows.h, but instead, only
the winres.h file, and only if RC_INVOKED is defined.
NOTE: Both afxres.h and winres.h do not really belong to the PSDK,
but belong instead to MFC. The reason why we have them here is twofold:
1. Wine used to have also winres.h (from where we got ours). This is
because the .rc resource files of some non-MFC modules were generated
with Visual Studio, which always includes afxres.h for these, and the
solution was to use a slightly less MFC-specific header: winres.h
(Wine commit cb08c82244673f26842e7a0766de90f091b5a493).
However, this winres.h has been since removed from Wine tree
(Wine commit 197f4059ab2af5f13f9c56faa26e3b4af902f869).
2. Few of our modules either use afxres.h or winres.h in their resource
files, and we still want to be able to compile them.
It is also compatible with MS PSDK file.
NOTE: I have observed that at least one .rh file (commctrl.rh) we have
from Wine, is not yet fully compatible with its MS PSDK counterpart
(for example it misses the TBSTYLE_* defines).
Note that this file is already in sync with wine-8.20
NOWINMESSAGES, NOWINSTYLES, NOCLIPBOARD, NOMENUS, NOSYSCOMMANDS,
OEMRESOURCE and NOICONS #ifndef guards should not be present in the
.rh file, contrary to the .h file.
However, #ifndef NONCMESSAGES guard should be present.
Finally, remove the useless SC_SIZE redefinition
("At least HP-UX defines it in /usr/include/sys/signal.h" -- like,
what the hell, we are developing ReactOS...)
Keep compatibility with MS PSDK ddraw.h file by using WINNT instead of
_WINNT_ in the preprocessor conditional test.
Incidentally this allows also removing those #define NT_BUILD_ENVIRONMENT
in the win32ss modules.
See commit 5fcfaf2e1 (r42346).
Co-authored-by: Victor Perevertkin <victor.perevertkin@reactos.org>
Introduce the initial changes needed to get other processors up and into kernel mode.
This only supports x86 as of now but is the first real step towards using other system processors.
Sometimes repairing a broken hive with a hive log does not always guarantee the hive
in question has fully recovered. In worst cases it could happen the LOG itself is even
corrupt too and that would certainly lead to a total unbootable system. This is most likely
if the victim hive is the SYSTEM hive.
This can be anyhow solved by the help of a mirror hive, or also called an "alternate hive".
Alternate hives serve the purpose as backup hives for primary hives of which there is still
a risk that is not worth taking. For now only the SYSTEM hive is granted the right to have
a backup alternate hive.
=== NOTE ===
Currently the SYSTEM hive can only base upon the alternate SYSTEM.ALT hive, which means the
corresponding LOG file never gets updated. When time comes the existing code must be adapted
to allow the possibility to use .ALT and .LOG hives simultaneously.
As we iterate over the chunk hive data pointer for hive bins that we are going
to enlist, we might encounter one or several bins that would get corrupted
during a premature abortion of a registry writing operation such as due to
a power outage of the system, hardware malfunction, etc.
Corruption at the level of hive bins is nasty because they contain actual cell
data of registry information such as keys, values etc. Assuming a bin is corrupt
in part we can fix it by recovering some of the bin properties that, theoretically,
could be fixed -- namely the signature, size and offset.
For size and offset we are more or less safe because a bin typically has a size
of a block, and the offset is the coordinate index of where a hive bin should lay at.
The newly implemented code for registry recovery makes the FreeLdr binary to grow
in size, to the point that it would BSOD because the PE image is too big.
For now we have to temporarily disable any of the newly added code, until
either FreeLdr is split into a basic PE bootloader image itself and a
"FreeLdrlib" that is used by the PE image to access various bootloader APIs
or another proper solution is found.
Thanks to CmCheckRegistry, the function can perform volatile data purging upon boot which this removes old hacky CmPrepareHive code. This also slightly refactors HvInitialize making it more proper.
=== DOCUMENTATION REMARKS ===
This implements (also enables some parts of code been decayed for years) the transacted writing of the registry. Transacted writing (or writing into registry in a transactional way) is an operation that ensures the successfulness can be achieved by monitoring two main points.
In CMLIB, such points are what we internally call them the primary and secondary sequences. A sequence is a numeric field that is incremented each time a writing operation (namely done with the FileWrite function and such) has successfully completed.
The primary sequence is incremented to suggest that the initial work of syncing the registry is in progress. During this phase, the base block header is written into the primary hive file and registry data is being written to said file in form of blocks. Afterwards the seconady sequence
is increment to report completion of the transactional writing of the registry. This operation occurs in HvpWriteHive function (invoked by HvSyncHive for syncing). If the transactional writing fails or if the lazy flushing of the registry fails, LOG files come into play.
Like HvpWriteHive, LOGs are updated by the HvpWriteLog which writes dirty data (base block header included) to the LOG themselves. These files serve for recovery and emergency purposes in case the primary machine hive has been damaged due to previous forced interruption of writing stuff into
the registry hive. With specific recovery algorithms, the data that's been gathered from a LOG will be applied to the primary hive, salvaging it. But if a LOG file is corrupt as well, then the system will perform resuscitation techniques by reconstructing the base block header to reasonable values,
reset the registry signature and whatnot.
This work is an inspiration from PR #3932 by mrmks04 (aka Max Korostil). I have continued his work by doing some more tweaks and whatnot. In addition to that, the whole transaction writing code is documented.
=== IMPORTANT NOTES ===
HvpWriteLog -- Currently this function lacks the ability to grow the log file size since we pretty much lack the necessary code that deals with hive shrinking and log shrinking/growing as well. This part is not super critical for us so this shall be left as a TODO for future.
HvLoadHive -- Currently there's a hack that prevents us from refactoring this function in a proper way. That is, we should not be reading the whole and prepare the hive storage using HvpInitializeMemoryHive which is strictly used for HINIT_MEMORY but rather we must read the hive file block by block
and deconstruct the read buffer from the file so that we can get the bins that we read from the file. With the hive bins we got the hive storage will be prepared based on such bins. If one of the bins is corrupt, self healing is applied in such scenario.
For this matter, if in any case the hive we'll be reading is corrupt we could potentially read corrupt data and lead the system into failure. So we have to perform header and data recovery as well before reading the whole hive.
CmCheckRegistry is a function that provides the necessary validation checks for a registry hive. This function usually comes into action when logs have been replayed for example, or when a registry hive internals have changed such as when saving a key, loading a key, etc.
This commit implements the whole Check Registry infrastructure (cmcheck.c) in CMLIB library for ease of usage and wide accessibility across parts of the OS. In addition, two more functions for registry checks are also implemented -- HvValidateHive and HvValidateBin.
Instead of having the CmCheckRegistry implementation in the kernel, it's better to have it in the Configuration Manager library instead (aka CMLIB). The benefits of having it in the library are the following:
- CmCheckRegistry can be used in FreeLdr to fix the SYSTEM hive
- It can be used on-demand in the kernel
- It can be used for offline registry repair tools
- It makes the underlying CmCheckRegistry implementation code debug-able in user mode
CORE-9195
CORE-6762
This implements cmheal.c file which provides the basic registry self-heal infrastructure needed by the public CmCheckRegistry function. The infrastructure provides a range of various self-heal helpers for the hive, such as subkey, class, values and node healing functions.
Add these NTSTATUS codes in the CMLIB library. STATUS_INVALID_PARAMETER will be used mostly for HvInitialize function, STATUS_REGISTRY_IO_FAILED for whatever routines that deal with reading or writing into a hive file.
=== DOCUMENTATION REMARKS ===
HBOOT_TYPE_REGULAR and HBOOT_TYPE_SELF_HEAL are boot type values set up by the CMLIB library (for the BootType field respectively). HBOOT_TYPE_REGULAR indicates a normal system boot whereas HBOOT_TYPE_SELF_HEAL indicates the system boot is assisted within self healing mode.
Whether the former or the latter value is set it's governed by both the kernel and the bootloader. The bootloader and the kernel negotiate together to determine if any of the registry properties (the hive, the base block, the registry base, etc) are so severed from corruption or not. In extreme cases where
registry healing is possible, the specific base block of the damaged hive will have its flags marked with HBOOT_TYPE_SELF_HEAL. At this point the boot phase procedure is orchestrated since the boot phase no longer goes on the default path but it's assisted, as I have already said above.
HBOOT_NO_BOOT_RECOVER, HBOOT_BOOT_RECOVERED_BY_HIVE_LOG and HBOOT_BOOT_RECOVERED_BY_ALTERNATE_HIVE on the other hand are identifiers for the BootRecover field of the BASE_BLOCK header structure. These are used exclusively by FreeLdr to tell the kernel if the bootloader recovered the SYSTEM hive or not. In case where the bootloader did recover the SYSTEM hive,
the kernel will perform a flush request on the dirty data down to disk. The (almost) worse case FreeLdr could not repair the main hive by applying log data, it will load the alternate mirror version of the hive.
In addition to that, declare other miscellaneous CMLIB identifiers for log transaction writes purposes.
Reason: VS solution builds pass the compiler options to the assembler. These are usually ignored and result in a warning only, but ML of the latest VS supports /Gy (function level linking), but that requires all functions to be declared with "PROC" and otherwise results in an error.
- Uniform function declarations in .c and .h with fixes and SAL2 improvements.
- Use LDR_[UN]LOCK_LOADER_LOCK_FLAG_XXX correctly.
- Fix incorrect UsedSize calculation in LdrQueryProcessModuleInformationEx (caught by ThFabba).
- Whatever the callback request stop the enumeration or not in LdrEnumerateLoadedModules, the following operations should be the same.
- Fix 2 incorrect DPRINT1 that printed incorrect parameter.
- Return error if RtlAllocateHeap failed in LdrpGetProcedureAddress, and add comments about NT6.2 new changes.
- Adds support for registry controlled menu separators and the documented
values to turn off verbs.
- Adds support for CMF_OPTIMIZEFORINVOKE, CMF_NODEFAULT, CMF_DONOTPICKDEFAULT,
CMF_EXPLORE and CMF_DISABLEDVERBS.
Bugs fixed:
- A verb with "Extended" set in the registry could cause the menu to invoke
the incorrect command! This happened because skipping InsertMenuItemW
caused InvokeCommand to use the wrong index with m_StaticEntries.
- Uses IS_INTRESOURCE instead of HIWORD to check if something is a string
(only matters on 64-bit).
- TryToBrowse leaking a PIDL when calling ILCombine.
Notes:
- This PR introduces the RosGetProcessEffectiveVersion() helper function
discussed in chat.
- Relaxed FAILED_UNEXPECTEDLY to FAILED in two places because IContextMenu
cannot assume that it has a site that leads to IShellBrowser.
This prevents the code from linking to __imp_*, which would pull in the symbol from the import library (e.g. kernel32), which creates problems with the GNU linker, which doesn't like to have the import and a static definition at the same time.
because George is having an open Draft PR since July 2022,
which might also touch this file on master in some years.
And it ofc is easier for me to revert my work now, then for him to
go through the great lengths of merging his work then.
bdasup: Addendum to 40c15ec (r46632).
kmixer: Addendum to 3e489bf (r42143).
mmixer: Addendum to c42d9f2 (r44872).
stream: Addendum to 4a0debf (r41662).
Serge plans a follow up that will remove all other remaining YDEBUG in the source tree.
The ones covered here he considered to be the most trivial ones.
Define _STLP_THREADS to make stl thread safe.
Also remove defined(__BUILDING_STLPORT) from the preprocessor conditions for which STLPInterlockedExchangePointer is defined, because that function is used in _threads.h, which can get included by apps using the stl through stl headers.
