MmLoadSystemImage has a PUNICODE_STRING NamePrefix parameter which is
currently unused in ReactOS. When the kernel loads the crash dump
storage stack drivers, the drivers will be loaded with MmLoadSystemImage
with a "dump_" or "hiber_" (for hibernation, which uses crash dump
stack too) prefix. This change adds in the prefix support, and is
supposed to push crash dump support forward.
CORE-376
- Use SAL2 annotations.
- KdSendPacket(): Validate DEBUG_IO API call.
- KdReceivePacket(): Take the LengthOfStringRead into account; use
KdbpReadCommand() to read the input, so that correct line edition
is available (backspace, etc.)
- Don't read anything and return immediately if the buffer size is zero.
- Allow the controlling keys (up/down arrows, backspace) even if the
buffer is full! (especially backspace if too much has been written).
- Return the number of characters stored, not counting the NULL terminator.
Currently the failure code path doesn't do any kind of cleanup against
the hive that was being linked to master. The cleanup is pretty
straightforward as you just simply close the hive file handles and free
the registry kernel structures.
CORE-5772
CORE-17263
CORE-13559
Log debug output of the lazy flusher as much information as possible so that we can examine how does the lazy flusher behave, since it didn't work for almost a decade. Verbose debugging will be disabled once we're confident enough the registry implementation in ReactOS is rock solid.
Whenever ReactOS finishes its operations onto the registry and unlocks it, a lazy flush is invoked to do an eventual flushing of the registry to the backing storage of the system. Except that... lazy flushing never comes into place.
This is because whenever CmpLazyFlush is called that sets up a timer which needs to expire in order to trigger the lazy flusher engine worker. However, registry locking/unlocking is a frequent occurrence, mainly when on desktop. Therefore as a matter of fact, CmpLazyFlush keeps removing and inserting the timer and the lazy flusher will never kick in that way.
Ironically the lazy flusher actually does the flushing when on USETUP installation phase because during text-mode setup installation in ReactOS the frequency of registry operations is actually less so the timer has the opportunity to expire and fire up the flusher.
In addition to that, we must queue a lazy flush when marking cells as dirty because such dirty data has to be flushed down to the media storage of the system. Of course, the real place where lazy flushing operation is done should be in a subset helper like HvMarkDirty that marks parts of a hive as dirty but since we do not have that, we'll be lazy flushing the registry during cells dirty marking instead for now.
CORE-18303
Addendum to commit de81021ba.
Otherwise, we get the following build error:
\ntoskrnl\kd64\kddata.c(532,5): error: initializer element is not a compile-time constant
PtrToUL64(RtlpBreakWithStatusInstruction),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
\ntoskrnl\kd64\kddata.c(526,26): note: expanded from macro 'PtrToUL64'
#define PtrToUL64(x) ((ULPTR64)(x))
^~~~~~~~~~~~
If you ask why there are two sets of functions that do the same, it's
because this file (and the kdmain.c) will very soon some day be moved to
a transport dll, outside the kernel, and it will need these functions.
See this command's documentation:
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/-dbgprint
and the section "DbgPrint buffer and the debugger"
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/reading-and-filtering-debugging-messages#dbgprint-buffer-and-the-debugger
for more details.
- Loosely implement the function, based on our existing circular printout
buffers in kdio.c.
- Enable its usage in the KdpPrint() and KdpPrompt() functions.
Notice that this function will *only* capture the strings being sent **to**
the debugger, and not the strings the debugger itself produce. (This means
that we cannot use the KdPrintCircularBuffer as a replacement for our
KDBG dmesg one, for example...)
How to test:
Run ReactOS under WinDbg, and use the !dbgprint command to view the
buffer. You can also use the Memory Window, place yourself at the
address pointed by KdPrintCircularBuffer and KdPrintWritePointer, and
read its contents.
What you should observe:
Prior notice: The circular buffer in debug builds of ReactOS and Windows
is 0x8000 bytes large. In release builds, its size is down to 0x1000.
1- When you start e.g. the 2nd-stage GUI installation of ReactOS, going
past the initial "devices installation" and letting it stabilize on
the Welcome page, break into WinDbg and run the !dbgprint command. You
should notice that the end of its output is weirdly truncated, compared
to what has been actually emitted to the debug output. Comparing this
with the actual contents of the circular buffer (via Memory Window),
shows that the buffer contents is actually correct.
2- Copy all the text that has been output by the !dbgprint command and
paste it in an editor; count the number of all characters appearing +
newlines (only CR or LF), and observe that this number is "mysteriously"
equal to 16384 == 0x4000.
3- Continue running ReactOS installation for a little while, breaking back
back into WinDbg and looking at !dbgprint again. Its output seems to be
still stopping at the same place as before (but the actual buffer memory
contents shows otherwise). Continue running ROS installation, and break
into the debugger when ROS is about to restart. You should now observe
that the dbgprint buffer rolled over:
dd nt!KdPrintRolloverCount shows 1.
Carefully analysing the output of !dbgprint, however, you will notice
that it looks a bit garbage-y: the first part of the output is actually
truncated after 16384 characters, then you get a second part of the
buffer showing what ReactOS was printing while shutting down. Then
you get again what was shown at the top of the !dbgprint output.
(Of course, comparing with the actual contents of the circular buffer
in memory shows that its contents are fine...)
The reason of these strange observations, is because there is an intrinsic
bug in the !dbgprint command implementation (in kdexts.dll). Essentially,
it displays the contents of the circular buffer in two single dprintf()
calls: one for the "older" (bottom) part of the buffer:
[WritePointer, EndOfBuffer]
and one for the "newer" (upper) part of the buffer:
[CircularBuffer, WritePointer[ .
The first aspect of the bug (causing observation 3), is that those two
parts are not necessarily NULL-terminated strings (especially after
rollover), so for example, displaying the upper part of the buffer, will
potentially also display part of the buffer's bottom part.
The second aspect of the bug (explaining observations 1 and 2), is due
to the implementation of the dprintf() function (callback in dbgenv.dll).
There, it uses a fixed-sized buffer of size 0x4000 == 16384 characters.
Since the output of the circular buffer is not done by little chunks,
but by the two large parts, if any of those are larger than 0x4000 they
get truncated on display.
(This last observation is confirmed in a completely different context by
https://community.osr.com/discussion/112439/dprintf-s-max-string-length .)
But the underlying GCC stupidity is still there (15 years later).
However, enable it only in 32-bit GCC builds, not in 64-bits nor with MSVC.
See commit b9cd3f2d9 (r25845) for some details.
GCC is indeed still incapable of casting 32-bit pointers up to 64-bits,
when static-initializing arrays (**outside** a function) without emitting
the error:
"error: initializer element is not constant"
(which might somehow indicate it actually tries to generate executable
code for casting the pointers, instead of doing it at compile-time).
Going down the rabbit hole, other stupidities show up:
Our PVOID64 type and the related POINTER_64 (in 32-bit archs), or the
PVOID32 and POINTER_32 (in 64-bit archs), are all silently broken in
GCC builds, because the pointer size attributes __ptr64 and __ptr32,
which are originally MSVC-specific, are defined to nothing in _mingw.h.
(And similarly for the __uptr and __sptr sign-extension attributes.)
Clang and other sane ompilers has since then implemented those (enabled
with -fms-extensions), but not GCC. The closest thing that could exist
for GCC is to do:
#define __ptr64 __attribute__((mode(DI)))
in order to get a 64-bit-sized pointer type with
typedef void* __ptr64 PVOID64;
but even this does not work, with the error:
"error: invalid pointer mode 'DI'"
Choose the correct element of the KiUnexpectedRange array,
depending on the interrupt vector, the same way as here:
a2c6af0da4/ntoskrnl/ke/amd64/except.c (L77)
And guard KeConnectInterrupt() execution with dispatcher lock.
CORE-14922
- Line-wrapping is enabled with 'ESC[?7h' (the '?' was forgotten).
Notice that the following reference also shows it wrong:
https://www.cse.psu.edu/~kxc104/class/cse472/09s/hw/hw7/vt100ansi.htm
- Terminal type is actually queried with 'ESC Z' (VT52-compatible), or
with 'ESC[c' (VT100-compatible). The antediluvian CTRL-E ('\x05')
control code gives instead a user-configurable (usually empty) string,
so it's not reliable.
Also, we don't really care about the returned result, we just need to
know that one has been sent.
Cross-checked with online documentation:
* "Digital VT100 User Guide" (EK-VT100-UG-001) (1st edition, August 1978,
reviewed November 1978).
* https://www.real-world-systems.com/docs/ANSIcode.html
* https://geoffg.net/Downloads/Terminal/TerminalEscapeCodes.pdf
* https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
* https://en.wikipedia.org/wiki/Enquiry_character
- Retrieve the size of the *controlling terminal* with escape sequences
only when it's a serial one: serial output is enabled *and* KDSERIAL
is set (i.e. user input through serial). See code for the actual logic
(the corresponding truth table is left as an exercise for the reader).
- Fix also a "Buffer" vs. "InBuffer" mismatch, that caused the whole
code to fail.
- For fallback terminal size values, use meaningful ones when SCREEN
is instead the controlling "terminal" (based on full-size BOOTVID
values).
- When echoing read characters during line-cooking, do direct output by
using KdpDprintf() instead of going through the heavy KdbpPrint() function.
This fixes some input artifacts like: 1. extra slowdowns due to
querying terminal size at each keypress, and 2. getting obnoxious
"--- Press q to abort ..." prompts in the middle of typing a long
comamnd because you are at the very bottom of the screen.
- Remove KdbInit() macro and directly use KdbpCliInit() (since the place
where it was used was already within an #ifdef KDBG block).
- Declare KdpKdbgInit() only when KDBG is defined, move its definition
into kdio.c and remove the legacy wrappers/kdbg.c file.
And in KdbInitialize(), set KdpInitRoutine directly to the former,
instead of using the KdpKdbgInit indirection.
- Don't reset KdComPortInUse in KdpDebugLogInit().
- Minor refactorings: KdpSerialDebugPrint -> KdpSerialPrint and make it
static; argument name "Message" -> "String", "StringLength" -> "Length".
What we have:
- Maximum number of pagefiles: 16
- Minimum pagefile size: 256 pages (1 MB when page size = 4096 bytes)
- Maximum pagefile size:
* 32-bit platforms: (1024 * 1024 - 1) pages (~ 4095 MB)
* x86 with PAE support: same size as for AMD x64
* x64 platform: (4 * 1024 * 1024 * 1024 - 1) pages (~ 16 TB)
* IA64 platform: (8 * 1024 * 1024 * 1024 - 1) pages (~ 32 TB)
Those are the values as supported and verified by the NT kernel.
Now, user-mode programs (including SMSS.EXE) have different opinions
on these, namely, they consider estimates directly in MB, respectively:
4095 MB, (16 * 1024 * 1024) MB, and (32 * 1024 * 1024) MB
(verified on Win2k3 and Win7 32 and 64 bits).
Also here, the minimum pagefile size is set to 2 MB.
Starting Windows 8+ (and 10), those values change slightly, and are
still not fully synchronized between NTOS:MM and SMSS. Finally, while
(x86 PAE and) AMD64 and ARM64 seem to share the maximum pagefile
size limit, 32-bit ARMv7 appears to use different limits than regular
x86 (2 GB instead of 4).
Please keep those values as they are for NT compatibility!
See the following references:
https://www.geoffchappell.com/studies/windows/km/ntoskrnl/api/mm/modwrite/create.htmhttps://techcommunity.microsoft.com/t5/ask-the-performance-team/what-is-the-page-file-for-anyway/ba-p/372608
+ Manual extraction of the values from different NT 6.2,6.3,10 builds.
[SMSS] Fill out in particular the x86-specific case for PAE.
[NTOS:MM] Some cleanup in the NtCreatePagingFile() code, namely:
- Clarify some comments;
- Validate the lower and upper bounds of the Minimum and Maximum sizes
(based on Windows behaviour as explained by Geoff + manual tests).
- Open the pagefile in case-insensitive;
- Simplify the loop that finds an existing matching pagefile;
- Simplify some failure exit paths;
- Add a "Missing validation steps TODO" comment block explaining the
existing code-hole.
The "failed to grant access rights" message isn't enough to understand what kind of access rights haven't been granted and why. Dumping information of the captured security descriptor, the ACL and its ACEs with mask rights and token SIDs should be enough to understand the reason of the failure in question.
