Commit graph

58 commits

Author SHA1 Message Date
Pierre Schweitzer 43836b0fbb
[NTOSKRNL] In !filecache, try to display FCB name
When no name is set in the file object, try to read the name
from the FCB. We only support FastFAT (ours) FCB for now.

This is clearly a hack, but for a kdbg command, so ;-)
2018-04-27 18:57:30 +02:00
Pierre Schweitzer 579a784e04
[NTOSKNRL] In case we leak a VACB, debug as much information as possible.
CORE-14578
2018-04-27 14:14:56 +02:00
Pierre Schweitzer fcf83315dc
[NTOSKRNL] Noisily dereference mapped VACB on cache release.
It seems that on process killing, some VACB may be deleted while
still mapped. With current reference counting, they will actually
not be deleted, but leaked, and an ASSERT will be triggered.

CORE-14578
2018-04-27 10:23:06 +02:00
Pierre Schweitzer fd3a6c1089 [NTOSKRNL] Properly reset VACB on free
CID 1434271
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 953dc72dad [NTOSKRNL] Drop the VACB lock.
This has have several benefits for ReactOS Cc:
- It helps reducing potential deadlocks situations in Cc
- It speeds up ReactOS by reducing locks
- It gets us a bit closer to Windows VACB

CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 40017a54f9 [NTOSKRNL] Use interlocked operations when dealing with map count.
CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 1b672981e2 [NTOSKRNL] Map the VACB in kernel space before inserting it in lists.
The avoids race conditions where attempts to read from disk to
not fully initialized VACB were performed.
Also, added more debug prints in such situations.

CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 42df4683d7 [NTOSKRNL] Add extra sanity checks for VACB lists.
We now always initialize list members from the VACB
and make sure the list entry has properly been removed
from the list before free.

CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer ffd524275e
[NTOSKRNL] Properly delete VACB in CcRosCreateVacb() when mapping fails.
Spotted by Thomas.

CORE-14478
CORE-14502
2018-03-25 18:27:19 +02:00
Pierre Schweitzer 14b05e65ff
[NTOSKRNL] Use interlocked operations for VACB reference counting.
CORE-14480
CORE-14285
2018-03-24 19:15:58 +01:00
Pierre Schweitzer dea9c291ab
[NTOSKRNL] Add a few asserts when mapping a VACB in kernel space
Also, reset VACB content when returning it to the lookaside list

CORE-14478
2018-03-24 19:15:58 +01:00
Pierre Schweitzer 2b6df67f0a
[NTOSKRNL] More asserts regarding reference count
CORE-14285
CORE-14480
2018-03-24 11:59:45 +01:00
Pierre Schweitzer 1e579843bc
[NTOSKNRL] Always reference a newly created VACB
This allows being consistent between newly created and looked up
so that VACB can always safely be released.

Should really help with reference issues.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 18:16:55 +01:00
Pierre Schweitzer 2a0e996c9d
[NTOSKRNL] In CcRosInternalFreeVacb(), in case of invalid free, also print file name.
CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer 2fbba22789
[NTOSKRNL] In CcFlushCache(), release the VACB using CcRosReleaseVacb()
Instead of reimplementing it partially and wrongly.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer 13b57fb5b5
[NTOSKRNL] Misc fixes to VACB reference counting
This fixes various bugs linked to VACB counting:
- VACB not released when it should be
- Reference count expectations not being accurate

For the record, VACB should always have at least a reference
count of 1, unless you are to free it and removed it from
any linked list.

This commit also adds a bunch of asserts that should
help triggering invalid reference counting.

It should also fix numerous ASSERT currently triggered and
may help fixing random behaviours in Cc.

CORE-14285
CORE-14401
CORE-14293
2018-03-17 11:56:25 +01:00
Pierre Schweitzer a6e080bd3d
[NTOSKRNL] Avoid private cache map allocation for the first handle
Standard shared cache map provides space for a private cache map, do the same
and make it available for the first handle. It avoids two allocations in a row.
2018-02-09 21:52:41 +01:00
Pierre Schweitzer 6b0cb54fe5
[NTOSKRNL] Define the throttling limits for Cc in Mm: MmThrottleTop, MmThrottleBottom.
Modified !defwrites to reflect that change
2018-02-09 14:21:14 +01:00
Pierre Schweitzer d35243d4e0
[NTOSKRNL] Quickly implement the !defwrites in KDBG 2018-02-09 12:16:29 +01:00
Pierre Schweitzer 0518444217
[NTOSKRNL] Drop the iSharedCacheMapLock in favor of the master lock 2018-02-09 11:13:53 +01:00
Pierre Schweitzer fc4744da5f
[NTOSKRNL] Use better locking for private cache map in CcRosReleaseFileCache().
Suggested by Thomas
2018-02-09 11:05:15 +01:00
Pierre Schweitzer c5139563db
[NTOSKRNL] Bring an initial (and not perfect ;-)) implementation of read ahead to our Cc!
This halfplements CcScheduleReadAhead() which is responsible for finding the next reads
to perform given last read and previous reads. I made it very basic for now, at least
to test the whole process.
This also introduces the CcExpressWorkQueue in the lazy writer which is responsible
for dealing with read ahead items and which is dealt with before the regular queue.
In CcCopyData(), if read was fine, schedule read ahead so that it can happen in background
without the FSD to notice it! Also, update the read history so that scheduling as a
bit of data.
Implement (à la "old Cc" ;-)) CcPerformReadAhead() which is responsible for performing
the read. It's only to be called by the worker thread.

Side note on the modifications done in CcRosReleaseFileCache(). Private cache map
is tied to a handle. If it goes away, private cache map gets deleted. Read ahead
can run after the handle was closed (and thus, private cache map deleted), so
it is mandatory to always lock the master lock before accessing the structure in
read ahead or before deleting it in CcRosReleaseFileCache(). Otherwise, you'll
just break everything. You've been warned!

This commit also partly reverts f8b5d27.

CORE-14312
2018-02-09 10:14:11 +01:00
Pierre Schweitzer f0c7f862d4
[NTOSKRNL] Fix a ****ing bug where private cache map was deleted in CcUninitializeCacheMap()
before the call to CcRosReleaseFileCache() which expects to have it to properly clean the file.
So, move deletion code to CcRosReleaseFileCache() so that he's the only one to handle private map.
Should hopefully fix all the recent buildbots issues (and the universe perhaps, who knows?)
2018-02-08 14:15:02 +01:00
Pierre Schweitzer f8b5d27807
[NDK] Update the PRIVATE_CACHE_MAP structure to match W2K3 one.
Also update the spin lock on allocation.
2018-02-08 14:02:42 +01:00
Pierre Schweitzer de897cbe3d
[NTOSKRNL] Revert 2362e0f and 07e6e9c (partly) they seem to have caused high instability in Cc.
This reverts BCB being lazy written when marked dirty.

We'll go back to this behavior when this part will have been reworked and stabilized.

CORE-14263
CORE-14279
CORE-14285
2018-02-08 13:40:54 +01:00
Pierre Schweitzer f8c3f0c9c4
[NTOSKRNL] Misc cosmetic changes 2018-02-08 11:02:26 +01:00
Pierre Schweitzer c19c7c7674
[NTOSKRNL] Magic value-- 2018-02-08 11:02:26 +01:00
Pierre Schweitzer 074e7e97fe
[NTOSKRNL] When initializing file cache, also allocate a private cache map if needed.
Not used yet.
2018-02-07 23:18:54 +01:00
Pierre Schweitzer 0e932c7bef
[NTOSKRNL] Drop the CcTryToInitializeFileCache() which is actually doing nothing relevant 2018-02-07 22:23:49 +01:00
Pierre Schweitzer 323f71a551
[NTOSKRNL] Use the LockQueueMasterLock where appropriate 2018-02-07 21:37:17 +01:00
Pierre Schweitzer 7e550edb26
[NTOSKRNL] Drop the always running thread for lazy writer.
Instead move to a threading model like the Windows one.
We'll queue several work items to be executed in a system thread (Cc worker)
when there are VACB that have been marked as dirty. Furthermore, some delay
will be observed before action to avoid killing the system with IOs.
This new threading model opens way for read ahead and write behind implementation.

Also, moved the initialization of the lazy writer to CcInitializeCacheManager()
it has nothing to do with views and shouldn't be initialized there.

Also, moved the lazy writer implementation to its own file.

Modified CcDeferWrite() and CcRosMarkDirtyVacb() to take into account the new threading model.

Introduced new functions:
- CcPostWorkQueue(): post an item to be handled by Cc worker and spawn a worker if required
- CcScanDpc(): called after some time (not to have lazy writer always running) to queue a lazy scan
- CcLazyWriteScan(): the lazy writer we used to have
- CcScheduleLazyWriteScan(): function to call when you want to start a lazy writer run. It will make a DPC after some time and queue execution
- CcWorkerThread(): the worker thread that will handle lazy write, read ahead, and so on
2018-02-07 21:37:17 +01:00
Pierre Schweitzer 3b147adafb
[NTOSKRNL] On Cc init, also define CcNumberWorkerThreads which will be used later on
Also make a "default" case (really limited) in case SystemSize doesn't match.
2018-02-07 21:37:17 +01:00
Pierre Schweitzer ee7b3b315d
[NTOSKRNL] Save system size on init in CcCapturedSystemSize 2018-02-07 21:37:17 +01:00
Pierre Schweitzer 41792c517e
[NTOSKRNL] Yet another ASSERT...
CORE-14285
2018-02-04 22:02:27 +01:00
Pierre Schweitzer 8dc4ac3692
[NTOSKRNL] Add an assert against VACB reference count overflow.
Very likely to be hit...

CORE-14285
2018-02-04 21:46:08 +01:00
Pierre Schweitzer 3f8788d6e5
[NTOSKRNL] Add an helper for marking a VACB clean (ie, not dirty).
CORE-14263
2018-02-01 08:36:17 +01:00
David Quintana 9b89cd1ef2
[NTOSKRNL] Fix CcIdleDelay initializer for old msvc versions (#339)
[NTOSKRNL] Fix initialization of CcIdleDelay for msvc builds (fixes boot).
* Introduce a macro to initialize LARGE_INTEGERs in a consistent way.
2018-01-31 18:12:57 +01:00
Pierre Schweitzer b49a2d6356
[NTOSKRNL] Drop ROS_DEFERRED_WRITE_CONTEXT in favor of DEFERRED_WRITE
that was introduced in d3e0eb2.

CORE-14235
2018-01-28 11:55:40 +01:00
Pierre Schweitzer 52287be9a9
[NTOSKRNL] Misc fixes to Cc:
- CcUnpinDataForThread() only release VACB when the last BCB reference is gone. This avoids having a valid BCB with an invalid VACB
- CcRosMarkDirtyVacb() will only accept non-dirty VACB now. This avoids a major bug where a an already dirty VACB was over-dereferenced
- Thanks to previous point, simplify CcRosUnmapVacb(), CcRosReleaseVacb() implementation
- And only set VACB dirty once in CcSetDirtyPinnedData()
- Add a few sanity checks

With that I can again install ReactOS with 128MB RAM :-).

CORE-14263
CORE-14268
2018-01-28 11:14:15 +01:00
Pierre Schweitzer f032b77c51
[NTOSKRNL] Fix MSVC build? 2018-01-27 22:02:01 +01:00
Pierre Schweitzer 2d4a54dcf2
[NTOSKRNL] Hello CcIdleDelay :-)
(Good link for the interested reader: https://msdn.microsoft.com/en-us/library/bb742613.aspx)
2018-01-27 21:52:19 +01:00
Pierre Schweitzer 78664ebe15
[NTOSKRNL] On file cache deletion, only manipulate VACB once locked.
Should help avoiding race conditions with lazy write.

CORE-14263
2018-01-27 18:37:34 +01:00
Pierre Schweitzer a3d78067ab
[NTOSKRNL] When marking a VACB dirty on unmap/unpin, use CcRosMarkDirtyVacb().
This should help reducing race conditions with lazy writer, and thus reduce random BSODs.

CORE-14263
2018-01-27 10:59:27 +01:00
Pierre Schweitzer dc85171674
[NTOSKRNL] Simplify (and speedup!) CcCanIWrite() using the dirty pages count in cache map. 2018-01-26 10:50:13 +01:00
Pierre Schweitzer 5cc0668b21
[NTOSKRNL] Store the total dirty pages per shared cache map in it. 2018-01-26 10:42:09 +01:00
Pierre Schweitzer 2362e0faae
[NTOSKRNL] Allow pinned dirty VACB to be lazy written.
This is an addition to 07e6e9c.
Also, fix a cppcheck warning (so minor!).

CORE-14249
2018-01-25 22:00:20 +01:00
Pierre Schweitzer 9c27193a43
[NTOSKRNL] Contrary to WinDBG !filecache, we don't display CONTROL_AREA but shared cache map 2018-01-25 13:47:00 +01:00
Pierre Schweitzer 5c52ae6695
[NTOSKRNL] Fix MSVC build? 2018-01-24 22:03:23 +01:00
Pierre Schweitzer cb52c82125
[NTOSKRNL] Implement (it's a bit raw for now!) the !filecache command in KDBG 2018-01-24 21:46:16 +01:00
Pierre Schweitzer 9d1e16663a
[NTOSKRNL] Link all the shared cache map together. 2018-01-24 21:46:16 +01:00