Commit graph

167 commits

Author SHA1 Message Date
Pierre Schweitzer
dd392b9d6c
[NTOSKRNL] Fix mismatching spinlock release in CcPerformReadAhead() 2018-02-18 19:32:08 +01:00
Pierre Schweitzer
e319f85e67
[NTOSKRNL] Don't make CcScheduleReadAhead() too noisy 2018-02-18 11:31:14 +01:00
Pierre Schweitzer
bdae0cce66
[NTOSKRNL] Don't call OOM dumper in case we were trying to allocate more than 100 pages.
In such case, we'll consider the caller is bogus and we'll fail in silence.
Dedicated to Mark (again!)
2018-02-17 15:09:38 +01:00
Pierre Schweitzer
01fcb02880
[NTOSKRNL] Allow more chars in ExpTagAllowPrint().
Dedicated to Mark
2018-02-17 15:06:29 +01:00
Pierre Schweitzer
bd70987909
[NTOSKRNL] In debug builds, print more information about the top level IRP in the Cc worker thread
CORE-14315
2018-02-10 23:50:26 +01:00
Pierre Schweitzer
a6e080bd3d
[NTOSKRNL] Avoid private cache map allocation for the first handle
Standard shared cache map provides space for a private cache map, do the same
and make it available for the first handle. It avoids two allocations in a row.
2018-02-09 21:52:41 +01:00
Pierre Schweitzer
5d93941d31
[NTOSKRNL] Modified CcCanIWrite to handle Mm throttling limits 2018-02-09 14:22:00 +01:00
Pierre Schweitzer
6b0cb54fe5
[NTOSKRNL] Define the throttling limits for Cc in Mm: MmThrottleTop, MmThrottleBottom.
Modified !defwrites to reflect that change
2018-02-09 14:21:14 +01:00
Pierre Schweitzer
945ff8ea2e
[NTOSKRNL] Rewrite CcCanIWrite() to make it more accurate and handle specific callers 2018-02-09 13:56:16 +01:00
Pierre Schweitzer
f51b74ca61
[NTOSKRNL] Don't defer write through write operations 2018-02-09 12:25:58 +01:00
Pierre Schweitzer
d35243d4e0
[NTOSKRNL] Quickly implement the !defwrites in KDBG 2018-02-09 12:16:29 +01:00
Pierre Schweitzer
0518444217
[NTOSKRNL] Drop the iSharedCacheMapLock in favor of the master lock 2018-02-09 11:13:53 +01:00
Pierre Schweitzer
8119e0ad2b
[NTOSKRNL] Silence lazy writer now we know it works 2018-02-09 11:05:44 +01:00
Pierre Schweitzer
fc4744da5f
[NTOSKRNL] Use better locking for private cache map in CcRosReleaseFileCache().
Suggested by Thomas
2018-02-09 11:05:15 +01:00
Pierre Schweitzer
c5139563db
[NTOSKRNL] Bring an initial (and not perfect ;-)) implementation of read ahead to our Cc!
This halfplements CcScheduleReadAhead() which is responsible for finding the next reads
to perform given last read and previous reads. I made it very basic for now, at least
to test the whole process.
This also introduces the CcExpressWorkQueue in the lazy writer which is responsible
for dealing with read ahead items and which is dealt with before the regular queue.
In CcCopyData(), if read was fine, schedule read ahead so that it can happen in background
without the FSD to notice it! Also, update the read history so that scheduling as a
bit of data.
Implement (à la "old Cc" ;-)) CcPerformReadAhead() which is responsible for performing
the read. It's only to be called by the worker thread.

Side note on the modifications done in CcRosReleaseFileCache(). Private cache map
is tied to a handle. If it goes away, private cache map gets deleted. Read ahead
can run after the handle was closed (and thus, private cache map deleted), so
it is mandatory to always lock the master lock before accessing the structure in
read ahead or before deleting it in CcRosReleaseFileCache(). Otherwise, you'll
just break everything. You've been warned!

This commit also partly reverts f8b5d27.

CORE-14312
2018-02-09 10:14:11 +01:00
Pierre Schweitzer
f0c7f862d4
[NTOSKRNL] Fix a ****ing bug where private cache map was deleted in CcUninitializeCacheMap()
before the call to CcRosReleaseFileCache() which expects to have it to properly clean the file.
So, move deletion code to CcRosReleaseFileCache() so that he's the only one to handle private map.
Should hopefully fix all the recent buildbots issues (and the universe perhaps, who knows?)
2018-02-08 14:15:02 +01:00
Pierre Schweitzer
f8b5d27807
[NDK] Update the PRIVATE_CACHE_MAP structure to match W2K3 one.
Also update the spin lock on allocation.
2018-02-08 14:02:42 +01:00
Pierre Schweitzer
de897cbe3d
[NTOSKRNL] Revert 2362e0f and 07e6e9c (partly) they seem to have caused high instability in Cc.
This reverts BCB being lazy written when marked dirty.

We'll go back to this behavior when this part will have been reworked and stabilized.

CORE-14263
CORE-14279
CORE-14285
2018-02-08 13:40:54 +01:00
Pierre Schweitzer
e26e4445b3
[NTOSKRNL] Revert 6ece4e9fc2 for third tools compatibility 2018-02-08 11:50:45 +01:00
Pierre Schweitzer
df5839db5b
[NTOSKRNL] Implement CcSetAdditionalCacheAttributes() 2018-02-08 11:49:22 +01:00
Pierre Schweitzer
f8c3f0c9c4
[NTOSKRNL] Misc cosmetic changes 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
9c87925197
[NTOSKRNL] Implement CcSetReadAheadGranularity() 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
c19c7c7674
[NTOSKRNL] Magic value-- 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
074e7e97fe
[NTOSKRNL] When initializing file cache, also allocate a private cache map if needed.
Not used yet.
2018-02-07 23:18:54 +01:00
Pierre Schweitzer
0e932c7bef
[NTOSKRNL] Drop the CcTryToInitializeFileCache() which is actually doing nothing relevant 2018-02-07 22:23:49 +01:00
Serge Gautherie
6ece4e9fc2 [NTOS:KE] Replace "Fatal System Error" with "BugCheck" 2018-02-07 22:22:28 +01:00
Pierre Schweitzer
323f71a551
[NTOSKRNL] Use the LockQueueMasterLock where appropriate 2018-02-07 21:37:17 +01:00
Pierre Schweitzer
19c822341b
[NTOSKRNL] Handle FSRTL_FLAG_LIMIT_MODIFIED_PAGES flag in CcDeferWrite().
Util now, CcDeferWrite() was ignoring limits per file object.
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
c11e947f61
[NTOSKRNL] Add wait support in CcCanIWrite() 2018-02-07 21:37:17 +01:00
Pierre Schweitzer
92e64a6c9b
[NTOSKRNL] Implement CcPostDeferredWrites() that executes deferred writes.
Make use of it in the lazy writer
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
f4e67aa837
[NTOSKRNL] Properly implement lazy writer activity watching.
We get rid of the old iLazyWriterNotify event in favor of work items
that contain an event that lazy writer will set once its done.
To implement this, we rely on the newly introduced CcPostTickWorkQueue work queue
that will contain work items that are to be queued once lazy writer is done.

Move the CcWaitForCurrentLazyWriterActivity() implementation to the
lazy writer file, and reimplemented it using the new support mechanisms
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
7e550edb26
[NTOSKRNL] Drop the always running thread for lazy writer.
Instead move to a threading model like the Windows one.
We'll queue several work items to be executed in a system thread (Cc worker)
when there are VACB that have been marked as dirty. Furthermore, some delay
will be observed before action to avoid killing the system with IOs.
This new threading model opens way for read ahead and write behind implementation.

Also, moved the initialization of the lazy writer to CcInitializeCacheManager()
it has nothing to do with views and shouldn't be initialized there.

Also, moved the lazy writer implementation to its own file.

Modified CcDeferWrite() and CcRosMarkDirtyVacb() to take into account the new threading model.

Introduced new functions:
- CcPostWorkQueue(): post an item to be handled by Cc worker and spawn a worker if required
- CcScanDpc(): called after some time (not to have lazy writer always running) to queue a lazy scan
- CcLazyWriteScan(): the lazy writer we used to have
- CcScheduleLazyWriteScan(): function to call when you want to start a lazy writer run. It will make a DPC after some time and queue execution
- CcWorkerThread(): the worker thread that will handle lazy write, read ahead, and so on
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
3b147adafb
[NTOSKRNL] On Cc init, also define CcNumberWorkerThreads which will be used later on
Also make a "default" case (really limited) in case SystemSize doesn't match.
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
853f04e6fc
[NTOSKRNL] Rename ExpCriticalWorkerThreads and ExpDelayedWorkerThreads (no p)
to match Windows names and make them available for the whole kernel.
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
ee7b3b315d
[NTOSKRNL] Save system size on init in CcCapturedSystemSize 2018-02-07 21:37:17 +01:00
Pierre Schweitzer
d56a249089
[NTOSKRNL] Define the Mm global variables related to Cc on Mm init.
This means that MmSystemCacheStart, MmSystemCacheEnd, MmSizeOfSystemCacheInPages
have now a valid value.
System cache is not used atm the moment though. MmMapViewInSystemCache() is to
be implemented, and Cc is to be made aware of this.

CORE-14259
2018-02-06 13:41:23 +01:00
Timo Kreuzer
dae88fb828 [NTOS:MM] Make x64 address space layout more like Vista+
- Change MM_SYSTEM_SPACE_START to 0xFFFFF88000000000
- Move MI_DEBUG_MAPPING to the end of the system PTE range
- Add MI_SYSTEM_CACHE_START and MI_SYSTEM_CACHE_END, which is in the range that Vista uses as dynamic VA space for cache and other allocations
- Wrap x86 specific code that makes now invalid assumptions about the address space layout in #ifdef _M_IX86
2018-02-05 01:09:32 +01:00
Pierre Schweitzer
41792c517e
[NTOSKRNL] Yet another ASSERT...
CORE-14285
2018-02-04 22:02:27 +01:00
Pierre Schweitzer
8dc4ac3692
[NTOSKRNL] Add an assert against VACB reference count overflow.
Very likely to be hit...

CORE-14285
2018-02-04 21:46:08 +01:00
Pierre Schweitzer
3f8788d6e5
[NTOSKRNL] Add an helper for marking a VACB clean (ie, not dirty).
CORE-14263
2018-02-01 08:36:17 +01:00
Pierre Schweitzer
f8d58bba38
[NTOSKRNL] Don't let CcWriteVirtualAddress() deal with VACB dirty status
CORE-14263
2018-01-31 21:47:20 +01:00
David Quintana
9b89cd1ef2
[NTOSKRNL] Fix CcIdleDelay initializer for old msvc versions (#339)
[NTOSKRNL] Fix initialization of CcIdleDelay for msvc builds (fixes boot).
* Introduce a macro to initialize LARGE_INTEGERs in a consistent way.
2018-01-31 18:12:57 +01:00
Thomas Faber
6e750d189a
[NTOS:CM] Correctly interpret RtlCreateUnicodeString return value. CORE-14271 2018-01-31 10:37:56 +01:00
Pierre Schweitzer
b49a2d6356
[NTOSKRNL] Drop ROS_DEFERRED_WRITE_CONTEXT in favor of DEFERRED_WRITE
that was introduced in d3e0eb2.

CORE-14235
2018-01-28 11:55:40 +01:00
Pierre Schweitzer
52287be9a9
[NTOSKRNL] Misc fixes to Cc:
- CcUnpinDataForThread() only release VACB when the last BCB reference is gone. This avoids having a valid BCB with an invalid VACB
- CcRosMarkDirtyVacb() will only accept non-dirty VACB now. This avoids a major bug where a an already dirty VACB was over-dereferenced
- Thanks to previous point, simplify CcRosUnmapVacb(), CcRosReleaseVacb() implementation
- And only set VACB dirty once in CcSetDirtyPinnedData()
- Add a few sanity checks

With that I can again install ReactOS with 128MB RAM :-).

CORE-14263
CORE-14268
2018-01-28 11:14:15 +01:00
Thomas Faber
d3a29816ce
[NTOS:MM] Assert MmLocateMemoryAreaByAddress return value to satisfy Coverity. CID 701289 2018-01-28 10:48:27 +01:00
Thomas Faber
5c52ded05f
[NTOS:PS] Check for rundown success in PsReferenceProcessFilePointer. CID 514553 2018-01-28 10:48:19 +01:00
Pierre Schweitzer
f032b77c51
[NTOSKRNL] Fix MSVC build? 2018-01-27 22:02:01 +01:00
Pierre Schweitzer
2d4a54dcf2
[NTOSKRNL] Hello CcIdleDelay :-)
(Good link for the interested reader: https://msdn.microsoft.com/en-us/library/bb742613.aspx)
2018-01-27 21:52:19 +01:00
Pierre Schweitzer
78664ebe15
[NTOSKRNL] On file cache deletion, only manipulate VACB once locked.
Should help avoiding race conditions with lazy write.

CORE-14263
2018-01-27 18:37:34 +01:00