Commit graph

188 commits

Author SHA1 Message Date
Pierre Schweitzer
ffd524275e
[NTOSKRNL] Properly delete VACB in CcRosCreateVacb() when mapping fails.
Spotted by Thomas.

CORE-14478
CORE-14502
2018-03-25 18:27:19 +02:00
Pierre Schweitzer
14b05e65ff
[NTOSKRNL] Use interlocked operations for VACB reference counting.
CORE-14480
CORE-14285
2018-03-24 19:15:58 +01:00
Pierre Schweitzer
dea9c291ab
[NTOSKRNL] Add a few asserts when mapping a VACB in kernel space
Also, reset VACB content when returning it to the lookaside list

CORE-14478
2018-03-24 19:15:58 +01:00
Pierre Schweitzer
2b6df67f0a
[NTOSKRNL] More asserts regarding reference count
CORE-14285
CORE-14480
2018-03-24 11:59:45 +01:00
Vadim Galyant
111e19f8a4 [NTOSKRNL] Adding IopValidateID() to test characters in PnP IDs (IRP_MN_QUERY_ID). (#341) 2018-03-19 14:04:17 +01:00
Thomas Faber
a17bf6f669
[NTOS:MM] Improve debug prints in MmCreateVirtualMappingUnsafe. CORE-14478 2018-03-18 23:49:48 +01:00
Pierre Schweitzer
1e579843bc
[NTOSKNRL] Always reference a newly created VACB
This allows being consistent between newly created and looked up
so that VACB can always safely be released.

Should really help with reference issues.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 18:16:55 +01:00
Pierre Schweitzer
2a0e996c9d
[NTOSKRNL] In CcRosInternalFreeVacb(), in case of invalid free, also print file name.
CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer
2fbba22789
[NTOSKRNL] In CcFlushCache(), release the VACB using CcRosReleaseVacb()
Instead of reimplementing it partially and wrongly.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer
13b57fb5b5
[NTOSKRNL] Misc fixes to VACB reference counting
This fixes various bugs linked to VACB counting:
- VACB not released when it should be
- Reference count expectations not being accurate

For the record, VACB should always have at least a reference
count of 1, unless you are to free it and removed it from
any linked list.

This commit also adds a bunch of asserts that should
help triggering invalid reference counting.

It should also fix numerous ASSERT currently triggered and
may help fixing random behaviours in Cc.

CORE-14285
CORE-14401
CORE-14293
2018-03-17 11:56:25 +01:00
Thomas Faber
2b8f4f6e24
[NTOS:EX] Worker threads must not exit with a critical region held. CORE-13734 2018-03-07 13:18:09 +01:00
Thomas Faber
6b78ff036f
[NTOS:KE] Don't loop indefinitely trying to figure out the CPU frequency. CORE-14419
Previously, we would keep sampling the CPU frequency until two subsequent
samples differed by at most 1 MHz. This could take several seconds, and would
unnecessarily delay boot.
Instead, if sampling is too unreliable, just give up and calculate the average
frequency from 10 samples. This is no worse than picking the frequency that
just happened to be returned twice in a row.

The fact that this method of sampling fails could indicate that there's a
problem with our performance counter implementation or timer interrupt,
but that's a separate issue...
2018-03-01 14:26:44 +01:00
Pierre Schweitzer
c4f58bbfd8
[NTOKSRNL] Don't blindly schedule read-ahead on CcCopyRead() call.
This avoids locking Cc for too long by trying to read-ahead data which
is already in cache.
We now will only schedule a read ahead if next read should bring us
to a new VACB (perhaps not in cache).

This notably fixes Inkscape setup which was slown down by read-ahead
due to continous 1 byte reads.

Thanks to Thomas for his help on this issue.

CORE-14395
2018-02-28 20:58:36 +01:00
Pierre Schweitzer
62facb7a49
[NTOSKRNL] Implement ThreadHideFromDebugger in NtSetInformationThread() 2018-02-24 22:07:12 +01:00
Pierre Schweitzer
a3c58ca76f
[NTOSKRNL] Implement the support for reserve IRP in IO
The reserve IRP is an IRP which is allocated on system boot and kept during
the whole system life. Its purpose is to allow page reads in case of
low-memory situations where the system doesn't have enough memory left
to allocate an IRP to read from the page file (would be catastrophic situation).
2018-02-24 14:52:04 +01:00
Pierre Schweitzer
9ac2e9855a
[NTOSKRNL] Add the CcDataFlushes and CcDataPages counters 2018-02-24 14:52:04 +01:00
Pierre Schweitzer
0fbdf31709
[NTOSKRNL] Add the CcPinReadWait and CcPinReadNoWait counters 2018-02-24 14:52:04 +01:00
Pierre Schweitzer
45964099f3
[NTOSKRNL] Return some Cc counters in SystemPerformanceInformation 2018-02-24 13:36:26 +01:00
Pierre Schweitzer
227c4321c2
[NTOSKRNL] Add the CcMapDataWait and CcMapDataNoWait counters 2018-02-24 13:36:26 +01:00
Thomas Faber
a2f77ee3fb
[NTOS:CC] Don't read past the end of the file in CcPerformReadAhead. 2018-02-22 14:03:05 +01:00
Thomas Faber
56e2bf2f92
[NTOS:CC] Avoid some magic numbers. 2018-02-22 14:03:03 +01:00
Pierre Schweitzer
dd392b9d6c
[NTOSKRNL] Fix mismatching spinlock release in CcPerformReadAhead() 2018-02-18 19:32:08 +01:00
Pierre Schweitzer
e319f85e67
[NTOSKRNL] Don't make CcScheduleReadAhead() too noisy 2018-02-18 11:31:14 +01:00
Pierre Schweitzer
bdae0cce66
[NTOSKRNL] Don't call OOM dumper in case we were trying to allocate more than 100 pages.
In such case, we'll consider the caller is bogus and we'll fail in silence.
Dedicated to Mark (again!)
2018-02-17 15:09:38 +01:00
Pierre Schweitzer
01fcb02880
[NTOSKRNL] Allow more chars in ExpTagAllowPrint().
Dedicated to Mark
2018-02-17 15:06:29 +01:00
Pierre Schweitzer
bd70987909
[NTOSKRNL] In debug builds, print more information about the top level IRP in the Cc worker thread
CORE-14315
2018-02-10 23:50:26 +01:00
Pierre Schweitzer
a6e080bd3d
[NTOSKRNL] Avoid private cache map allocation for the first handle
Standard shared cache map provides space for a private cache map, do the same
and make it available for the first handle. It avoids two allocations in a row.
2018-02-09 21:52:41 +01:00
Pierre Schweitzer
5d93941d31
[NTOSKRNL] Modified CcCanIWrite to handle Mm throttling limits 2018-02-09 14:22:00 +01:00
Pierre Schweitzer
6b0cb54fe5
[NTOSKRNL] Define the throttling limits for Cc in Mm: MmThrottleTop, MmThrottleBottom.
Modified !defwrites to reflect that change
2018-02-09 14:21:14 +01:00
Pierre Schweitzer
945ff8ea2e
[NTOSKRNL] Rewrite CcCanIWrite() to make it more accurate and handle specific callers 2018-02-09 13:56:16 +01:00
Pierre Schweitzer
f51b74ca61
[NTOSKRNL] Don't defer write through write operations 2018-02-09 12:25:58 +01:00
Pierre Schweitzer
d35243d4e0
[NTOSKRNL] Quickly implement the !defwrites in KDBG 2018-02-09 12:16:29 +01:00
Pierre Schweitzer
0518444217
[NTOSKRNL] Drop the iSharedCacheMapLock in favor of the master lock 2018-02-09 11:13:53 +01:00
Pierre Schweitzer
8119e0ad2b
[NTOSKRNL] Silence lazy writer now we know it works 2018-02-09 11:05:44 +01:00
Pierre Schweitzer
fc4744da5f
[NTOSKRNL] Use better locking for private cache map in CcRosReleaseFileCache().
Suggested by Thomas
2018-02-09 11:05:15 +01:00
Pierre Schweitzer
c5139563db
[NTOSKRNL] Bring an initial (and not perfect ;-)) implementation of read ahead to our Cc!
This halfplements CcScheduleReadAhead() which is responsible for finding the next reads
to perform given last read and previous reads. I made it very basic for now, at least
to test the whole process.
This also introduces the CcExpressWorkQueue in the lazy writer which is responsible
for dealing with read ahead items and which is dealt with before the regular queue.
In CcCopyData(), if read was fine, schedule read ahead so that it can happen in background
without the FSD to notice it! Also, update the read history so that scheduling as a
bit of data.
Implement (à la "old Cc" ;-)) CcPerformReadAhead() which is responsible for performing
the read. It's only to be called by the worker thread.

Side note on the modifications done in CcRosReleaseFileCache(). Private cache map
is tied to a handle. If it goes away, private cache map gets deleted. Read ahead
can run after the handle was closed (and thus, private cache map deleted), so
it is mandatory to always lock the master lock before accessing the structure in
read ahead or before deleting it in CcRosReleaseFileCache(). Otherwise, you'll
just break everything. You've been warned!

This commit also partly reverts f8b5d27.

CORE-14312
2018-02-09 10:14:11 +01:00
Pierre Schweitzer
f0c7f862d4
[NTOSKRNL] Fix a ****ing bug where private cache map was deleted in CcUninitializeCacheMap()
before the call to CcRosReleaseFileCache() which expects to have it to properly clean the file.
So, move deletion code to CcRosReleaseFileCache() so that he's the only one to handle private map.
Should hopefully fix all the recent buildbots issues (and the universe perhaps, who knows?)
2018-02-08 14:15:02 +01:00
Pierre Schweitzer
f8b5d27807
[NDK] Update the PRIVATE_CACHE_MAP structure to match W2K3 one.
Also update the spin lock on allocation.
2018-02-08 14:02:42 +01:00
Pierre Schweitzer
de897cbe3d
[NTOSKRNL] Revert 2362e0f and 07e6e9c (partly) they seem to have caused high instability in Cc.
This reverts BCB being lazy written when marked dirty.

We'll go back to this behavior when this part will have been reworked and stabilized.

CORE-14263
CORE-14279
CORE-14285
2018-02-08 13:40:54 +01:00
Pierre Schweitzer
e26e4445b3
[NTOSKRNL] Revert 6ece4e9fc2 for third tools compatibility 2018-02-08 11:50:45 +01:00
Pierre Schweitzer
df5839db5b
[NTOSKRNL] Implement CcSetAdditionalCacheAttributes() 2018-02-08 11:49:22 +01:00
Pierre Schweitzer
f8c3f0c9c4
[NTOSKRNL] Misc cosmetic changes 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
9c87925197
[NTOSKRNL] Implement CcSetReadAheadGranularity() 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
c19c7c7674
[NTOSKRNL] Magic value-- 2018-02-08 11:02:26 +01:00
Pierre Schweitzer
074e7e97fe
[NTOSKRNL] When initializing file cache, also allocate a private cache map if needed.
Not used yet.
2018-02-07 23:18:54 +01:00
Pierre Schweitzer
0e932c7bef
[NTOSKRNL] Drop the CcTryToInitializeFileCache() which is actually doing nothing relevant 2018-02-07 22:23:49 +01:00
Serge Gautherie
6ece4e9fc2 [NTOS:KE] Replace "Fatal System Error" with "BugCheck" 2018-02-07 22:22:28 +01:00
Pierre Schweitzer
323f71a551
[NTOSKRNL] Use the LockQueueMasterLock where appropriate 2018-02-07 21:37:17 +01:00
Pierre Schweitzer
19c822341b
[NTOSKRNL] Handle FSRTL_FLAG_LIMIT_MODIFIED_PAGES flag in CcDeferWrite().
Util now, CcDeferWrite() was ignoring limits per file object.
2018-02-07 21:37:17 +01:00
Pierre Schweitzer
c11e947f61
[NTOSKRNL] Add wait support in CcCanIWrite() 2018-02-07 21:37:17 +01:00