Commit graph

133 commits

Author SHA1 Message Date
Pierre Schweitzer
ee7b3b315d
[NTOSKRNL] Save system size on init in CcCapturedSystemSize 2018-02-07 21:37:17 +01:00
Pierre Schweitzer
d56a249089
[NTOSKRNL] Define the Mm global variables related to Cc on Mm init.
This means that MmSystemCacheStart, MmSystemCacheEnd, MmSizeOfSystemCacheInPages
have now a valid value.
System cache is not used atm the moment though. MmMapViewInSystemCache() is to
be implemented, and Cc is to be made aware of this.

CORE-14259
2018-02-06 13:41:23 +01:00
Timo Kreuzer
dae88fb828 [NTOS:MM] Make x64 address space layout more like Vista+
- Change MM_SYSTEM_SPACE_START to 0xFFFFF88000000000
- Move MI_DEBUG_MAPPING to the end of the system PTE range
- Add MI_SYSTEM_CACHE_START and MI_SYSTEM_CACHE_END, which is in the range that Vista uses as dynamic VA space for cache and other allocations
- Wrap x86 specific code that makes now invalid assumptions about the address space layout in #ifdef _M_IX86
2018-02-05 01:09:32 +01:00
Pierre Schweitzer
41792c517e
[NTOSKRNL] Yet another ASSERT...
CORE-14285
2018-02-04 22:02:27 +01:00
Pierre Schweitzer
8dc4ac3692
[NTOSKRNL] Add an assert against VACB reference count overflow.
Very likely to be hit...

CORE-14285
2018-02-04 21:46:08 +01:00
Pierre Schweitzer
3f8788d6e5
[NTOSKRNL] Add an helper for marking a VACB clean (ie, not dirty).
CORE-14263
2018-02-01 08:36:17 +01:00
Pierre Schweitzer
f8d58bba38
[NTOSKRNL] Don't let CcWriteVirtualAddress() deal with VACB dirty status
CORE-14263
2018-01-31 21:47:20 +01:00
David Quintana
9b89cd1ef2
[NTOSKRNL] Fix CcIdleDelay initializer for old msvc versions (#339)
[NTOSKRNL] Fix initialization of CcIdleDelay for msvc builds (fixes boot).
* Introduce a macro to initialize LARGE_INTEGERs in a consistent way.
2018-01-31 18:12:57 +01:00
Thomas Faber
6e750d189a
[NTOS:CM] Correctly interpret RtlCreateUnicodeString return value. CORE-14271 2018-01-31 10:37:56 +01:00
Pierre Schweitzer
b49a2d6356
[NTOSKRNL] Drop ROS_DEFERRED_WRITE_CONTEXT in favor of DEFERRED_WRITE
that was introduced in d3e0eb2.

CORE-14235
2018-01-28 11:55:40 +01:00
Pierre Schweitzer
52287be9a9
[NTOSKRNL] Misc fixes to Cc:
- CcUnpinDataForThread() only release VACB when the last BCB reference is gone. This avoids having a valid BCB with an invalid VACB
- CcRosMarkDirtyVacb() will only accept non-dirty VACB now. This avoids a major bug where a an already dirty VACB was over-dereferenced
- Thanks to previous point, simplify CcRosUnmapVacb(), CcRosReleaseVacb() implementation
- And only set VACB dirty once in CcSetDirtyPinnedData()
- Add a few sanity checks

With that I can again install ReactOS with 128MB RAM :-).

CORE-14263
CORE-14268
2018-01-28 11:14:15 +01:00
Thomas Faber
d3a29816ce
[NTOS:MM] Assert MmLocateMemoryAreaByAddress return value to satisfy Coverity. CID 701289 2018-01-28 10:48:27 +01:00
Thomas Faber
5c52ded05f
[NTOS:PS] Check for rundown success in PsReferenceProcessFilePointer. CID 514553 2018-01-28 10:48:19 +01:00
Pierre Schweitzer
f032b77c51
[NTOSKRNL] Fix MSVC build? 2018-01-27 22:02:01 +01:00
Pierre Schweitzer
2d4a54dcf2
[NTOSKRNL] Hello CcIdleDelay :-)
(Good link for the interested reader: https://msdn.microsoft.com/en-us/library/bb742613.aspx)
2018-01-27 21:52:19 +01:00
Pierre Schweitzer
78664ebe15
[NTOSKRNL] On file cache deletion, only manipulate VACB once locked.
Should help avoiding race conditions with lazy write.

CORE-14263
2018-01-27 18:37:34 +01:00
Pierre Schweitzer
a3d78067ab
[NTOSKRNL] When marking a VACB dirty on unmap/unpin, use CcRosMarkDirtyVacb().
This should help reducing race conditions with lazy writer, and thus reduce random BSODs.

CORE-14263
2018-01-27 10:59:27 +01:00
Pierre Schweitzer
dc85171674
[NTOSKRNL] Simplify (and speedup!) CcCanIWrite() using the dirty pages count in cache map. 2018-01-26 10:50:13 +01:00
Pierre Schweitzer
5cc0668b21
[NTOSKRNL] Store the total dirty pages per shared cache map in it. 2018-01-26 10:42:09 +01:00
Pierre Schweitzer
2362e0faae
[NTOSKRNL] Allow pinned dirty VACB to be lazy written.
This is an addition to 07e6e9c.
Also, fix a cppcheck warning (so minor!).

CORE-14249
2018-01-25 22:00:20 +01:00
Pierre Schweitzer
9c27193a43
[NTOSKRNL] Contrary to WinDBG !filecache, we don't display CONTROL_AREA but shared cache map 2018-01-25 13:47:00 +01:00
Pierre Schweitzer
5c52ae6695
[NTOSKRNL] Fix MSVC build? 2018-01-24 22:03:23 +01:00
Pierre Schweitzer
cb52c82125
[NTOSKRNL] Implement (it's a bit raw for now!) the !filecache command in KDBG 2018-01-24 21:46:16 +01:00
Pierre Schweitzer
9d1e16663a
[NTOSKRNL] Link all the shared cache map together. 2018-01-24 21:46:16 +01:00
Pierre Schweitzer
4a910a12cc
[NTOSKRNL] Make lazy writer thread run with higher priority.
CORE-14248
2018-01-24 10:09:19 +01:00
Pierre Schweitzer
f93acd806a
[NTOSKRNL] Implement per-file dirty page threshold.
Namely, implement CcSetDirtyPageThreshold() and add support for it
in CcCanIWrite().

Also added my name in the headers of the few files I touched tonight.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer
389f683026
[NTOSKRNL] Unexpected cleanup 2018-01-23 23:25:26 +01:00
Pierre Schweitzer
9a07c71eef
[NTOSKRNL] Implement support for deferred writes in Cc.
Namely, implement CcCanIWrite() (very basic, and likely wrong).
And implement CcDeferWrite() which will queue the write operation.

In CciLazyWriter() (which may be renamed CcWorkerThread() ;-)),
handle the queued write operations one by one. This is likely
not to be accurate, but, given we have only on FS supporting
this for now (NFS / RDBSS / Shares), this is OK.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer
07e6e9c9c1
[NTOSKRNL] When marking a BCB dirty, also mark the underlying VACB dirty.
Experiment and MSDN tend to show that a dirty BCB is queued for lazy write.
This will do the job here!

Also, renamed CcRosMarkDirtyFile() which is more accurate, and added a new
function CcRosMarkDirtyVacb() which just takes a VACB as arg (expected locked)
and marks it dirty (using previous implementation). Make CcRosMarkDirtyFile()
use it.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer
6ba67b8152
[NTOSKRNL] Bug fix: lazy write more often.
CcDirtyPageThreshold is not here to compute when you have to write,
but to know where you have to deny writes.
More commits to come in that direction!

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer
c7ad200f8b
[NTOSKRNL] Reimplement the lazy writer in Cc and remove the "basic" one in Mm.
This removes the "modified page writer" thread in Mm that was regularly blindly
attempting to flush dirty pages to the disk.
Instead, this commit introduces a lazy writer that will monitor dirty pages count
and will flush them to disk when this count is above a threshold. The threshold is
computed on Cc init.
Compared to what was done previously, this lazy writer will only write down files
that are not marked as temporary.
The mechanisms involved in this lazy writer worker are well described in Windows
Internals 4th editions (constants are coming from it ;-)).
Also fixed a bad (and old!) bug in CcRosFlushDirtyPages() where target count could
be overflow and the function would spin forever while holding the VACBs lock. This is
mandatory as now lazy writer will call it with "random" values.
This also allows implementing CcWaitForCurrentLazyWriterActivity() :-).
Also renamed DirtyPageCount to its MS equivalent.

CORE-14235
2018-01-23 19:33:59 +01:00
Pierre Schweitzer
2abb99faa9
[NTOSKRNL] In FsRtlAddToTunnelCache() allocate memory from PagedPool when required.
Also, if allocating from lookaside list, reattempt a cold allocation.
2018-01-20 21:21:00 +01:00
Thomas Faber
fda24da872
[NTOS] Remove unnecessary NULL checks and add a missing one. CIDs 1411958, 716670, 1322186 2018-01-20 15:59:35 +01:00
Thomas Faber
2ae756a4e5 [NTOS:MM] Fix broken ASSERT. CID 701285 2018-01-16 22:29:19 +01:00
Pierre Schweitzer
c779dbd4c5
[NTOSKRNL] Implement CcIsThereDirtyData() 2018-01-15 23:20:53 +01:00
Timo Kreuzer
2dade10d54 [NTOS] Fix 64 bit warnings 2018-01-14 13:54:14 +01:00
Pierre Schweitzer
371aba1abe
[NTOSKRNL] When mapping data in CcMapData(), don't truncate offset to ULONG.
This would affect reads/writes on large volumes where offset is higher than what a ULONG can hold.

This really nasty bug was hitting CcMapData() but also CcPinRead() (due to the nature of its implementation)
and both were returning garbage data under certain circumstances with Ext2Fsd.

This should (I hope!) help some other FSDs to work better in ROS.

CORE-12456
2018-01-13 21:30:37 +01:00
Mark Jansen
7b398514f1 [NTOS] Limit the spam from NtNotifyChangeMultipleKeys.
CORE-13125
2018-01-08 22:24:54 +01:00
Timo Kreuzer
1014d50a4b [NTOS:MM] Pass FaultCode to MiDispatchFault 2018-01-06 18:20:24 +01:00
Timo Kreuzer
3f426aabde [NTOS:MM] Mark local functions with static 2018-01-06 18:20:24 +01:00
Timo Kreuzer
a00378a1a1 [NTOS:MM] Add support for NX page-faults 2018-01-06 18:20:24 +01:00
Timo Kreuzer
4d043aa05e [NTOS:MM] Make use of FaultCode and MI_IS_NOT_PRESENT_FAULT and MI_IS_WRITE_ACCESS macros in MmArmAccessFault. 2018-01-06 18:20:24 +01:00
Timo Kreuzer
d5c71429d7 [NTOS:MM] Pass The FaultCode to MmArmAcceessFault as well and translate it there to what was declared as "StoreInstruction"
No functional changes.
2018-01-06 18:20:24 +01:00
Timo Kreuzer
3021c2d571 [NTOS:MM] Pass page fault code to MmAccessFault
Note: before we had a BOOLEAN parameter called StoreInstruction, but in reality it was not specifying whether the fault was from a store store instruction, but whether it was an access violation rather than a page-not-present fault. On x86 without PAE there are only 2 kinds of access violations: (1) Access of a kernel mode page from user mode, which is handled early and (2) access of a read-only (or COW) page with a writing instruction. Therefore we could get away with this, even though it relied on the wrong assumption that a fault, which was not a page-not-present-fault, was automatically a write access. This commit only changes one thing: we pass the full fault-code to MmAccessFault and handle the rest from there in exactly the same way as before. More changes are coming to make things clear.
2018-01-06 18:20:24 +01:00
Colin Finck
f5d366b200
[NTOS:CM] Improve code in cmsysini.c (#216)
Based on an original patch by Timo Kreuzer, with modifications by me to adapt it to latest HEAD and use a single exit path through the Cleanup label. This reliably frees all allocated handles.

The original code returns STATUS_SUCCESS for many cases. This has been preserved.
In the future, it should be checked though whether returning success is appropriate for all these cases.

CORE-6844
2018-01-06 13:27:41 +01:00
Timo Kreuzer
f555c102b7 [NTOS:EX] Don't dereference ExpCritSecOutOfMemoryEvent when it was never referenced. 2018-01-04 21:05:10 +01:00
Katayama Hirofumi MZ
a8bd06be98
[NTOS:KDBG] Fix DPRINT format specifier in KdPortInitializeEx. CORE-14174 2018-01-04 11:37:50 +01:00
Timo Kreuzer
d17812f933 [NTOS:MM] Use MiResolveDemandZeroFault instead of MiDispatchFault 2018-01-03 23:07:19 +01:00
Timo Kreuzer
d4765fe366 [NTOS:MM] Implement resolving PXE/PPE page table demand zero faults 2018-01-03 23:07:19 +01:00
Timo Kreuzer
2eff510074 [NTOS:MM] Increment Prcb->MmDemandZeroCount while holding the Pfn lock 2018-01-03 23:07:19 +01:00