- Un-hardcode the RPL_MASK value.
- s/KiUnexpectedInterrupt&Number/KiUnexpectedInterrupt&Vector/
- Use C-style comments in traphdlr.c as everywhere else in the file.
- Update the URLs for the MSDN "FPO" macro documentation.
- Windows requires 16 bytes of response data.
- Add the PLUGPLAY_CONTROL_USER_RESPONSE_DATA type.
- Usetup and Umpnpmgr must fail if NtPlugPlayControl(PlugPlayControlUserResponse) does not return STATUS_SUCCESS.
So that they handle LUID mappings and process device maps.
Get rid of the ObpParseSymbolicLinkToIoDeviceObject helper and introduce a
new helper ObpProcessDosDeviceSymbolicLink that will do the same things
but also handle name creation/deletion as well as device map handling.
All this is based on previous code (hence the same comments :-)).
What's left to do now is to add support for device maps in ObpLookupObjectName
It is now able to set the newly created device map to any
process and will default to current process if none is provided.
It also sets system device map if no process is specified.
It also deferences existing device map in the process if needed.
Finaly, it will make the directory object permanant.
- Always include kd64.h
- Change KdpPrompt() prototype to be compatible between KDBG and _WINDK_
- Rename KdComponentTable to KdpComponentTable to prevent a conflict
- Add some functions stubs and global variables
This could happen if BCB was marked dirty previously.
Marking VACB dirty on unpin could lead to a double write of
the VACB, even if clean.
Indeed, now that setting BCB dirty leads to marking VACB
dirty, the VACB can be flushed in between by the lazy-writer.
The BCB state is not reset on VACB flush, contrary to the VACB state.
Thus, on unpin even if the VACB was already flushed, we were
setting back the dirty state, leading the VACB to be flushed again.
This could bring a small performance downgrade. Though it remains
limited since this is mostly used for FS metadata.
Possibly it could lead to metadata corruption, but this is likely
less possible.
CORE-15954
- Add and initialize a global power capabilities variable.
- Return the global power capabilities via NtPowerInformation.SystemPowerCapabilities.
- Report the presence of power button, sleep button and lid.
It better captures the intent now in FsRtlIsNameInExpressionPrivate and
fixes a slight overallotion by 4 bytes in FsRtlIsDbcsInExpression.
While at it, use the ANSI_DOS_DOT macro in the Dbcs version.
CORE-15902
This allows querying volume information without issuing an IRP to the owner device.
The kernel is supposed to already have all the required information to return
to the caller.
Side effect: this allows querying volume information for devices not implementing
IRP_MJ_QUERY_VOLUME_INFORMATION such as null.sys
This fixes opening null device in Python.
Fix based on debugging by Maxim Smirnov in PR #1442
CORE-14551
"Release" builds should not BSoD on 'Freeing pool - invalid tag specified' case,
this is compatible with Windows behaviour.
Nothing changes otherwise concerning the "Debug" builds, where pool-tag mismatch
triggers a BSoD as expected.
CORE-15446
- Correctly probe the FileName user pointer before capturing its contents.
- A paging file name with zero length is invalid too.
- Remember (TODO!) in the code that the lower bounds of the Safe*Size values
need to be checked too!
* The previous version was overcomplicated and broken and therefore disabled.
* The new version also enforces NX protection on x64.
* Now that protecting works, also protect the boot loaded images.
This allows avoiding one of the previous implementation limits:
leaked IRP not queued to a thread are now totally visible since
we look directly in the memory pool.
For now, it allows searching for pool allocations in
both paged and non paged pool.
It is based on Andreas Schuster work to identify POOL_HEADER
structures.
* [WIN32K] Fix handle calculation in DbgGdiHTIntegrityCheck
* [NOTEPAD] Fix MSVC warnings
* [PSDK] Simplify *PROC definitions in windef.h
* [VIDEOPRT] Don't try to use NtVdmControl on x64
* [FREELDR] Fix some macros
* [CRT] Make qsort 64 bit compatible
* [NTOS] Use #ifndef _WIN64 instead of #ifdef _M_IX86 around C_ASSERTs
* [FAST486] Fix 64 bit warnings and change DWORD to ULONG, so it can be used in kernel mode
* [APPHELP_APITEST] Fix 64 bit issue
This is far from perfect, and totally doesn't match the
WinDBG way of doing it. Instead of browsing pool to find
matching 'IRP' tags, we just browse all the processes
to find the queued IRP. This requires the IRPs to be queued,
obviously, and will make us miss the leaked IRPs, for instance.
Proper way to do it would be to implement !poolfind and then
rely on its implementation to find our IRPs.
Perhaps later ;-)
This may look strange, since this buffer is originally allocated using
the TAG_IO_NAME tag. However, it happens that file-system drivers are
allowed to re-allocate this buffer: this is what the MS' open-sourced
CDFS driver does, see e.g. CdCommonCreate() and CdNormalizeFileNames()
in cdfs/create.c .
This fixes a pool tag mismatch 'mNoI' != 'nFdC' BSOD when resources
are freed when closing a file that has been opened with a relative name
on a CDFS-mounted volume.
* Add an NDK header to define INIT_FUNCTION/INIT_SECTION globally
* Use _declspec(allocate(x)) and _declspec(code_seg(x)) on MSVC versions that support it
* Use INIT_FUNCTION on functions only and INIT_SECTION on data only (required by MSVC)
* Place INIT_FUNCTION before the return type (required by MSVC)
* Make sure declarations and implementations share the same modifiers (required by MSVC)
* Add a global linker option to suppress warnings about defined but unused INIT section
* Merge INIT section into .text in freeldr
For now, this is just a split between scan and flush that
were both done during lazy scan previously.
Lazy scan shouldn't perform any write operation, but only
queue a write behind operation.
Our implementation is far from the original, as it seems
our lazy scan should queue a write behind operation per
shared cache map. Right now, we only perform global
operation.
This will avoid corruption when a file size is little grown and read afterwards.
Up to now, FSD where reading 0es instead of expected data, causing corruption.
This fixes MS FastFAT not being able to mount a FAT volume in ReactOS, corrupting
the FAT.
This also fixes the CcSetFileSizes kmtest tests.
This is based on a patch by Thomas Faber.
CORE-11819
This incorrect behavior was leading to a call at too high IRQL for paged code.
This was triggered by MS FastFAT.
ReleaseFromLazyWrite call was already correctly called to that regard.
CORE-11819
Also, probe the service name when unloading a driver if called from
user-mode. This will avoid that userland applications can trigger an
invalid read in the kernel (and thus, a BSOD).
CORE-15468
- Require the user to have TCB privilege for using this function.
- Probe and capture the user-provided string (and avoid usermode-triggered BSODS ;-)
- Allocate the OEM-converted string in *NonPagedPool* because we are
going to transmit the buffer to BOOTVID.
This avoids performing a double-free (even though that's hidden by the
fact we use lookaside allocations for VACB), and it avoids freeing
a memory address at an uninitialized address.
We don't care about references here, the VACB was just allocated, never
linked and we're its only user.
CORE-15413
- Simplify the volume-deletion code in RawCheckForDismount().
- Fixes the OpenCount check in RawClose(): the VCB mutex must be
released when the volume has not been dismounted, either because
OpenCount != 0 or because RawCheckForDismount() returned FALSE.
- Explicitly use VCB_STATE_LOCKED instead of hardcoding its value.
- In IRP_MN_VERIFY_VOLUME handling, lock the volume before playing
with it, and again let the volume be dismounted only if OpenCount == 0
(and the IoDeleteDevice() call is done by RawCheckForDismount()).
Now, we make sure that we update ref count and BCB list membership
with the BCB lock held, in a row.
This will avoid race conditions where the BCB was removed from the
list, then referenced again, leading to inconsistencies in memory
and crashes later on.
This could notably be triggered while building ReactOS on ReactOS
(one would call this a regression).
CORE-15235
- Rework CmpSetSystemValues() and remove its 1st-stage text-mode setup hack, since a real registry hive will be used for 1st-stage either.
- Lock, then unlock the registry in NtInitializeRegistry when initializing the hives & flusher.
- Call CmpInitializeHiveList() (i.e., initialize the other hives like \Software, \User, \.Default) only when we are not in setup-mode.
svn path=/branches/setup_improvements/; revision=74747
This reverts a7c26408 (r53255) and ff75ae1b (r53694), and a hack from 6075ae9a (r46690).
svn path=/branches/setup_improvements/; revision=74745
svn path=/branches/setup_improvements/; revision=74746
- Improve the capture of OBJECT_ATTRIBUTES parameters that are passed
(by pointer) to the Cm* helper functions, and the capture of
UNICODE_STRINGs.
- Correctly differentiate user-mode vs. kernel-mode root directory handles
(in OBJECT_ATTRIBUTES): note that most of the Cm* APIs expect their
parameters to be kernel-mode (pointers, handles...).
CORE-13448
- Validate the information class parameter in NtQueryValueKey().
- Call the post-callback in NtSetValueKey() only if the callback
has been registered and the CmSetValueKey() call is executed.
We won't reuse a BCB created for mapping, we will now have
our own dedicated BCB.
This allows having a bit more cleaner implementation of CcPinMappedData()
We now handle race conditions when creating BCB to avoid
having duplicated BCB per shared maps.
Also, we already specify whether the memory will be pinned
when creating the BCB, to avoid potential duplications or
BCB misuse.
If so, return such BCB instead of creating a new one. This will
allow (at some point) to be more consistent in case of concurrent
mapping.
This fixes a few CcMapData tests.
Given current ReactOS implementation, a VACB can be pinned
several times, with different BCB. In next commits, a single
BCB will be able to be pinned several times. That would
lead to severe inconsistencies in counting and thus corruption.
This quick check based on bits and operation is for 2^ based
sector sizes (most of the cases) and will perform faster than
the modulo operation which is still used in fallback in case
the sector size wouldn't be a power of 2.
IopCloseFile can be called by IopDeleteFile. In that situation, it
doesn't set any process as first parameter. Furthermore, we are in a
situation where it's not required to lock the file object (see the
assert before the call).
- SeIsTokenChild(): Correctly check whether a caller-provided token
is a child from the current process' primary token by looking at
its ParentTokenId member.
- Add a SeIsTokenSibling() helper to determine whether a caller-provided
token and the current process' primary token are siblings, by comparing
their ParentTokenId's and AuthenticationId's.
NOTE: Children tokens are created through CreateRestrictedToken();
sibling tokens are created through DuplicateToken() (amongst others).
See slide 49 of https://www.slideshare.net/Shakacon/social-engineering-the-windows-kernel-by-james-forshaw
or https://googleprojectzero.blogspot.com/2016/01/raising-dead.html
for some details.
This could be triggered when attempting to read/write to really big
files. It was causing an attempt to read 0 bytes in Cc, leading to
asserts failure in the kernel (and corrupted file).
CORE-15067
Currently, our CcMapData() behavior (same goes for CcPinRead()) is broken
and is the total opposite of what Windows kernel does. By default, the later
will let you map a view in memory without even attempting to bring its
data in memory. On first access, there will be a fault and memory will
be read from the hardware and brought to memory. If you want to force read
on mapping/pinning, you have to set the MAP_NO_READ (or PIN_NO_READ) flag
where kernel will fault on your behalf (hence the need for MAP_WAIT/PIN_WAIT).
On ReactOS, by default, on mapping (and thus pinning), we will force a view
read so that data is in memory. The way our cache memory is managed at the
moment seems not to allow to fault on invalid access and if we don't force
read, the memory content will just be zeroed.
So trying to match Windows behavior, by default, now CcMapData() will enforce
the MAP_NO_READ flag and warn once about this behavior change.
It's based on the code that was in CcPinRead() implementation. This
made no sense to have CcPinMappedData() doing nothing while implementing
everything in CcPinRead(). Indeed, drivers (starting with MS drivers)
can map data first and pin it afterwards with CcPinMappedData(). It was
leading to incorrect behavior with our previous noop implementation.
Short: The code was suffering from an off-by-one bug (inconsistency between inclusive end exclusive end address), which could lead to freeing one page above the initialization code. This led to freeing part of the kernel import section on x64. Now it is consistently using the aligned/exclusive end address.
Long:
* Initialization sections are freed both for the boot loaded images as well as for drivers that are loaded later. Obviously the second mechanism needs to be able to run at any time, so it is not initialization code itself. For some reason someone decided though, it would be a smart idea to implement the code twice, once for the boot loaded images, once for drivers and concluding that the former was initialization code itself and had to be freed.
* Since freeing the code that frees the initialization sections, while it is doing that is not possible, it uses a "smart trick", initially skipping that range, returning its start and end to the caller and have the caller free it afterwards.
* The code was using the end address in an inconsistent way, partly aligning it to the start of the following section, sometimes pointing to the last byte that should be freed. The function that freed each chunk was assuming the latter (i.e. that the end was included in the range) and thus freed the page that contained the end address. The end address for the range that was returned to the caller was aligned to the start of the next section, and the caller used it to free the range including the following page. On x64 this was the start of the import section of ntoskrnl. How that worked on x86 I don't even want to know.
The PROCESS_DEVICEMAP_INFORMATION union has 2 fields, one is a handle, the other one is a structure of 36 bytes (independent of architecture). The handle forces 64 bit alignment on 64 bit builds, making the structure 4 bytes bigger than on 32 bit builds. The site is checked in NtQueryInformationProcess (case ProcessDeviceMap). The expected size on x64 is the size of the Query structure without alignment. autocheck correctly passes the site of the Query union member, while smss passes the full size of PROCESS_DEVICEMAP_INFORMATION. Packing the structure is not an option, since it is defined in public headers without packing. Using the original headers sizeof(PROCESS_DEVICEMAP_INFORMATION) is 0x28, sizeof(PROCESS_DEVICEMAP_INFORMATION::Query) is 0x24.
- Rename ObDirectoryType to ObpDirectoryObjectType and remove it from NDK (this is not exported!)
- Rename ObSymbolicLinkType to ObpSymbolicLinkObjectType
- Remove duplicated ObpTypeObjectType from ob.h
Kernel stacks that re freed, can be placed on an SLIST for quick reuse. The old code was using a member of the PFN of the last stack page as the SLIST_ENTRY. This relies on the following (non-portable) assumptions:
- A stack always has a PTE associated with it.
- This PTE has a PFN associated with it.
- The PFN has an empty field that can be re-used as an SLIST_ENTRY.
- The PFN has another field that points back to the PTE, which then can be used to get the stack base.
Specifically: On x64 the PFN field is not 16 bytes aligned, so it cannot be used as an SLIST_ENTRY. (In a "usermode kernel" the other assumptions are also invalid).
The new code does what Windows does (and which seems absolutely obvious to do): Place the SLIST_ENTRY directly on the stack.
It's hardly understandable and doesn't really makes sense.
Furthermore, it breaks compatibility with 3rd party FSD that
don't implement such FSCTL.
Obviously, Windows doesn't do this.
For user mode, when probing output buffer, if it's null, length
will also be set to 0.
This avoids user mode applications being able to trigger various
asserts in ReactOS (and thus BSOD when no debugger is plugged ;-)).
NDK: Define PLUGPLAY_CONTROL_PROPERTY_DATA.Properties and PLUGPLAY_CONTROL_DEVICE_RELATIONS_DATA.Relations values.
NTOSKRNL: Map PLUGPLAY_CONTROL_PROPERTY_DATA.Properties values to IoGetDeviceProperty properties and add (dummy) code for unsupported cases.
UMPNPMGR: Use PLUGPLAY_CONTROL_PROPERTY_DATA.Properties values in PNP_GetDeviceRegProp.
- Overhaul SepCreateToken() and SepDuplicateToken() so that they
implement the "variable information area" of the token, where
immutable lists of user & groups and privileges reside, and the
"dynamic information area" (allocated separately in paged pool),
where mutable data such as the token's default DACL is stored.
Perform the necessary adaptations in SepDeleteToken() and in
NtSetInformationToken().
- Actually dereference the token's logon session, when needed, in the
'TokenSessionReference' case in NtSetInformationToken().
- Overhaul SepFindPrimaryGroupAndDefaultOwner() so that it returns
the indices of candidate primary group and default owner within the
token's user & groups array. This allows for fixing the 'TokenOwner'
and 'TokenPrimaryGroup' cases of NtSetInformationToken(), since the
owner or primary group being set *MUST* already exist in the token's
user & groups array (as a by-product, memory corruptions that existed
before due to the broken way of setting these properties disappear too).
- Lock tokens every time operations are performed on them (NOTE: we
still use a global token lock!).
- Touch the ModifiedId LUID member of tokens everytime a write operation
(property change, etc...) is made on them.
- Fix some group attributes in the SYSTEM process token, SepCreateSystemProcessToken().
- Make the SeCreateTokenPrivilege mandatory when calling NtCreateToken().
- Update the token pool tags.
- Explicitly use the Ex*ResourceLite() versions of the locking functions
in the token locking macros.
- Use TRUE/FALSE instead of 1/0 for booleans.
- Use NULL instead of 0 for null pointers.
- Print 0x prefix for hex values in DPRINTs.
- Use new annotations for SepCreateToken() and SepDuplicateToken().
Caught while debugging, in the case the ImpersonationLevel value was
uninitialized, due to the fact it was left untouched on purpose by
PsReferenceEffectiveToken().
kmtest:NtCreateSection calls CcInitializeCacheMap with a
NULL value for SectionObjectPointers. This will cause an exception when
trying to access it, which in Windows can be handled gracefully.
However accessing it while holding ViewLock means the lock will not be
released, leading to an APC_INDEX_MISMATCH bugcheck.
This solves the problem by allocating SharedCacheMap outside the lock,
then freeing it again under lock if another thread has updated SharedCacheMap
in the mean time. This is also What Windows Does(TM).
This avoids a really nasty race condition in our cache controler where
two concurrents could try to initialize cache on the same file.
This had two nasty effects: first shared map was purely leaked and erased
by the second one. And the private cache map, allocated on the first shared
cache map couldn't be freed and was leading to Mm BSOD (free in a middle of
a block).
This was often triggered while building ReactOS on ReactOS (with multi threads).
With that patch, I cannot crash anylonger while building ReactOS.
CORE-14634
Oneliner of the day... This typo just prevented the
whole feature to work properly. Because any allocated
work item would miserably fail to be freed.
This will obviously help real world FSD relying on
StackOverflow worker from FsRtl to work better!
CORE-14611
In the lazy writer run, first post items that are queued for this.
Only then, start executing deferred writes if any.
If there were any, reschedule immediately a lazy writer run, to keep
Cc warm and to make it unqueue write faster in case of high IOs situation.
To make second lazy writer run happen faster, we keep our state active to
use short delay (1s) instead of standard idle (3s).
Recent changes seem to show that it's not
required to be exclusive on VACB to be able
to flush it.
This commit goes with f2c44aa and fixes the
last issues going with copying huge files.
There are no longer BSODs (be it in Mm or Cc).
I could, with 750MB RAM extract a 2GB file from
a 53MB archive and copy a 2,5GB file from a VBox
share to the disk. Note that writes are often
deferred, so if copy works, it's not that fast for now.
Note that it also brings some beloved behavior from
Windows: copy times are totally unreliable now when
writes are deferred. Little remaining times when
actively copying, high remaining times when deferred
writes in action. And goes between both... Sorry! ;-)
https://xkcd.com/612/
CORE-9696
CORE-11175
Same mechanism exists in Windows (even their Cc
is way different from ours...) where when Cc is
out of memory (in their case, out of VACB), we
will start scavenge old & unused VACB to free
some of the memory.
It's useful in case we're operating we big files
operations, we may run out of memory where to map
VACB for them, so start to scavenge VACB to free
some of that memory.
With this, I am able to install Qt 4.8.6 with 2,5GB of RAM,
scavenging acting when needed!
CORE-12081
CORE-14582
Adjusting refcount and enabling lazy-write for pinned
VACB makes it actually more efficient, often purging
data to disk, reducing memory stress for the system.
This is required for defering writes.
This commit unfortunately (?) reverts a previous revert.
CORE-12081
CORE-14582
CORE-14313
When no name is set in the file object, try to read the name
from the FCB. We only support FastFAT (ours) FCB for now.
This is clearly a hack, but for a kdbg command, so ;-)
