This fixes various bugs linked to VACB counting:
- VACB not released when it should be
- Reference count expectations not being accurate
For the record, VACB should always have at least a reference
count of 1, unless you are to free it and removed it from
any linked list.
This commit also adds a bunch of asserts that should
help triggering invalid reference counting.
It should also fix numerous ASSERT currently triggered and
may help fixing random behaviours in Cc.
CORE-14285
CORE-14401
CORE-14293
This avoids locking Cc for too long by trying to read-ahead data which
is already in cache.
We now will only schedule a read ahead if next read should bring us
to a new VACB (perhaps not in cache).
This notably fixes Inkscape setup which was slown down by read-ahead
due to continous 1 byte reads.
Thanks to Thomas for his help on this issue.
CORE-14395
Standard shared cache map provides space for a private cache map, do the same
and make it available for the first handle. It avoids two allocations in a row.
This halfplements CcScheduleReadAhead() which is responsible for finding the next reads
to perform given last read and previous reads. I made it very basic for now, at least
to test the whole process.
This also introduces the CcExpressWorkQueue in the lazy writer which is responsible
for dealing with read ahead items and which is dealt with before the regular queue.
In CcCopyData(), if read was fine, schedule read ahead so that it can happen in background
without the FSD to notice it! Also, update the read history so that scheduling as a
bit of data.
Implement (à la "old Cc" ;-)) CcPerformReadAhead() which is responsible for performing
the read. It's only to be called by the worker thread.
Side note on the modifications done in CcRosReleaseFileCache(). Private cache map
is tied to a handle. If it goes away, private cache map gets deleted. Read ahead
can run after the handle was closed (and thus, private cache map deleted), so
it is mandatory to always lock the master lock before accessing the structure in
read ahead or before deleting it in CcRosReleaseFileCache(). Otherwise, you'll
just break everything. You've been warned!
This commit also partly reverts f8b5d27.
CORE-14312
before the call to CcRosReleaseFileCache() which expects to have it to properly clean the file.
So, move deletion code to CcRosReleaseFileCache() so that he's the only one to handle private map.
Should hopefully fix all the recent buildbots issues (and the universe perhaps, who knows?)
This reverts BCB being lazy written when marked dirty.
We'll go back to this behavior when this part will have been reworked and stabilized.
CORE-14263
CORE-14279
CORE-14285
We get rid of the old iLazyWriterNotify event in favor of work items
that contain an event that lazy writer will set once its done.
To implement this, we rely on the newly introduced CcPostTickWorkQueue work queue
that will contain work items that are to be queued once lazy writer is done.
Move the CcWaitForCurrentLazyWriterActivity() implementation to the
lazy writer file, and reimplemented it using the new support mechanisms
Instead move to a threading model like the Windows one.
We'll queue several work items to be executed in a system thread (Cc worker)
when there are VACB that have been marked as dirty. Furthermore, some delay
will be observed before action to avoid killing the system with IOs.
This new threading model opens way for read ahead and write behind implementation.
Also, moved the initialization of the lazy writer to CcInitializeCacheManager()
it has nothing to do with views and shouldn't be initialized there.
Also, moved the lazy writer implementation to its own file.
Modified CcDeferWrite() and CcRosMarkDirtyVacb() to take into account the new threading model.
Introduced new functions:
- CcPostWorkQueue(): post an item to be handled by Cc worker and spawn a worker if required
- CcScanDpc(): called after some time (not to have lazy writer always running) to queue a lazy scan
- CcLazyWriteScan(): the lazy writer we used to have
- CcScheduleLazyWriteScan(): function to call when you want to start a lazy writer run. It will make a DPC after some time and queue execution
- CcWorkerThread(): the worker thread that will handle lazy write, read ahead, and so on
- CcUnpinDataForThread() only release VACB when the last BCB reference is gone. This avoids having a valid BCB with an invalid VACB
- CcRosMarkDirtyVacb() will only accept non-dirty VACB now. This avoids a major bug where a an already dirty VACB was over-dereferenced
- Thanks to previous point, simplify CcRosUnmapVacb(), CcRosReleaseVacb() implementation
- And only set VACB dirty once in CcSetDirtyPinnedData()
- Add a few sanity checks
With that I can again install ReactOS with 128MB RAM :-).
CORE-14263
CORE-14268
Namely, implement CcSetDirtyPageThreshold() and add support for it
in CcCanIWrite().
Also added my name in the headers of the few files I touched tonight.
CORE-14235
Namely, implement CcCanIWrite() (very basic, and likely wrong).
And implement CcDeferWrite() which will queue the write operation.
In CciLazyWriter() (which may be renamed CcWorkerThread() ;-)),
handle the queued write operations one by one. This is likely
not to be accurate, but, given we have only on FS supporting
this for now (NFS / RDBSS / Shares), this is OK.
CORE-14235
Experiment and MSDN tend to show that a dirty BCB is queued for lazy write.
This will do the job here!
Also, renamed CcRosMarkDirtyFile() which is more accurate, and added a new
function CcRosMarkDirtyVacb() which just takes a VACB as arg (expected locked)
and marks it dirty (using previous implementation). Make CcRosMarkDirtyFile()
use it.
CORE-14235
CcDirtyPageThreshold is not here to compute when you have to write,
but to know where you have to deny writes.
More commits to come in that direction!
CORE-14235
This removes the "modified page writer" thread in Mm that was regularly blindly
attempting to flush dirty pages to the disk.
Instead, this commit introduces a lazy writer that will monitor dirty pages count
and will flush them to disk when this count is above a threshold. The threshold is
computed on Cc init.
Compared to what was done previously, this lazy writer will only write down files
that are not marked as temporary.
The mechanisms involved in this lazy writer worker are well described in Windows
Internals 4th editions (constants are coming from it ;-)).
Also fixed a bad (and old!) bug in CcRosFlushDirtyPages() where target count could
be overflow and the function would spin forever while holding the VACBs lock. This is
mandatory as now lazy writer will call it with "random" values.
This also allows implementing CcWaitForCurrentLazyWriterActivity() :-).
Also renamed DirtyPageCount to its MS equivalent.
CORE-14235
This would affect reads/writes on large volumes where offset is higher than what a ULONG can hold.
This really nasty bug was hitting CcMapData() but also CcPinRead() (due to the nature of its implementation)
and both were returning garbage data under certain circumstances with Ext2Fsd.
This should (I hope!) help some other FSDs to work better in ROS.
CORE-12456
