Commit graph

178 commits

Author SHA1 Message Date
Pierre Schweitzer 3f8788d6e5
[NTOSKRNL] Add an helper for marking a VACB clean (ie, not dirty).
CORE-14263
2018-02-01 08:36:17 +01:00
Pierre Schweitzer f8d58bba38
[NTOSKRNL] Don't let CcWriteVirtualAddress() deal with VACB dirty status
CORE-14263
2018-01-31 21:47:20 +01:00
David Quintana 9b89cd1ef2
[NTOSKRNL] Fix CcIdleDelay initializer for old msvc versions (#339)
[NTOSKRNL] Fix initialization of CcIdleDelay for msvc builds (fixes boot).
* Introduce a macro to initialize LARGE_INTEGERs in a consistent way.
2018-01-31 18:12:57 +01:00
Thomas Faber 6e750d189a
[NTOS:CM] Correctly interpret RtlCreateUnicodeString return value. CORE-14271 2018-01-31 10:37:56 +01:00
Pierre Schweitzer b49a2d6356
[NTOSKRNL] Drop ROS_DEFERRED_WRITE_CONTEXT in favor of DEFERRED_WRITE
that was introduced in d3e0eb2.

CORE-14235
2018-01-28 11:55:40 +01:00
Pierre Schweitzer 52287be9a9
[NTOSKRNL] Misc fixes to Cc:
- CcUnpinDataForThread() only release VACB when the last BCB reference is gone. This avoids having a valid BCB with an invalid VACB
- CcRosMarkDirtyVacb() will only accept non-dirty VACB now. This avoids a major bug where a an already dirty VACB was over-dereferenced
- Thanks to previous point, simplify CcRosUnmapVacb(), CcRosReleaseVacb() implementation
- And only set VACB dirty once in CcSetDirtyPinnedData()
- Add a few sanity checks

With that I can again install ReactOS with 128MB RAM :-).

CORE-14263
CORE-14268
2018-01-28 11:14:15 +01:00
Thomas Faber d3a29816ce
[NTOS:MM] Assert MmLocateMemoryAreaByAddress return value to satisfy Coverity. CID 701289 2018-01-28 10:48:27 +01:00
Thomas Faber 5c52ded05f
[NTOS:PS] Check for rundown success in PsReferenceProcessFilePointer. CID 514553 2018-01-28 10:48:19 +01:00
Pierre Schweitzer f032b77c51
[NTOSKRNL] Fix MSVC build? 2018-01-27 22:02:01 +01:00
Pierre Schweitzer 2d4a54dcf2
[NTOSKRNL] Hello CcIdleDelay :-)
(Good link for the interested reader: https://msdn.microsoft.com/en-us/library/bb742613.aspx)
2018-01-27 21:52:19 +01:00
Pierre Schweitzer 78664ebe15
[NTOSKRNL] On file cache deletion, only manipulate VACB once locked.
Should help avoiding race conditions with lazy write.

CORE-14263
2018-01-27 18:37:34 +01:00
Pierre Schweitzer a3d78067ab
[NTOSKRNL] When marking a VACB dirty on unmap/unpin, use CcRosMarkDirtyVacb().
This should help reducing race conditions with lazy writer, and thus reduce random BSODs.

CORE-14263
2018-01-27 10:59:27 +01:00
Pierre Schweitzer dc85171674
[NTOSKRNL] Simplify (and speedup!) CcCanIWrite() using the dirty pages count in cache map. 2018-01-26 10:50:13 +01:00
Pierre Schweitzer 5cc0668b21
[NTOSKRNL] Store the total dirty pages per shared cache map in it. 2018-01-26 10:42:09 +01:00
Pierre Schweitzer 2362e0faae
[NTOSKRNL] Allow pinned dirty VACB to be lazy written.
This is an addition to 07e6e9c.
Also, fix a cppcheck warning (so minor!).

CORE-14249
2018-01-25 22:00:20 +01:00
Pierre Schweitzer 9c27193a43
[NTOSKRNL] Contrary to WinDBG !filecache, we don't display CONTROL_AREA but shared cache map 2018-01-25 13:47:00 +01:00
Pierre Schweitzer 5c52ae6695
[NTOSKRNL] Fix MSVC build? 2018-01-24 22:03:23 +01:00
Pierre Schweitzer cb52c82125
[NTOSKRNL] Implement (it's a bit raw for now!) the !filecache command in KDBG 2018-01-24 21:46:16 +01:00
Pierre Schweitzer 9d1e16663a
[NTOSKRNL] Link all the shared cache map together. 2018-01-24 21:46:16 +01:00
Pierre Schweitzer 4a910a12cc
[NTOSKRNL] Make lazy writer thread run with higher priority.
CORE-14248
2018-01-24 10:09:19 +01:00
Pierre Schweitzer f93acd806a
[NTOSKRNL] Implement per-file dirty page threshold.
Namely, implement CcSetDirtyPageThreshold() and add support for it
in CcCanIWrite().

Also added my name in the headers of the few files I touched tonight.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer 389f683026
[NTOSKRNL] Unexpected cleanup 2018-01-23 23:25:26 +01:00
Pierre Schweitzer 9a07c71eef
[NTOSKRNL] Implement support for deferred writes in Cc.
Namely, implement CcCanIWrite() (very basic, and likely wrong).
And implement CcDeferWrite() which will queue the write operation.

In CciLazyWriter() (which may be renamed CcWorkerThread() ;-)),
handle the queued write operations one by one. This is likely
not to be accurate, but, given we have only on FS supporting
this for now (NFS / RDBSS / Shares), this is OK.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer 07e6e9c9c1
[NTOSKRNL] When marking a BCB dirty, also mark the underlying VACB dirty.
Experiment and MSDN tend to show that a dirty BCB is queued for lazy write.
This will do the job here!

Also, renamed CcRosMarkDirtyFile() which is more accurate, and added a new
function CcRosMarkDirtyVacb() which just takes a VACB as arg (expected locked)
and marks it dirty (using previous implementation). Make CcRosMarkDirtyFile()
use it.

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer 6ba67b8152
[NTOSKRNL] Bug fix: lazy write more often.
CcDirtyPageThreshold is not here to compute when you have to write,
but to know where you have to deny writes.
More commits to come in that direction!

CORE-14235
2018-01-23 23:25:26 +01:00
Pierre Schweitzer c7ad200f8b
[NTOSKRNL] Reimplement the lazy writer in Cc and remove the "basic" one in Mm.
This removes the "modified page writer" thread in Mm that was regularly blindly
attempting to flush dirty pages to the disk.
Instead, this commit introduces a lazy writer that will monitor dirty pages count
and will flush them to disk when this count is above a threshold. The threshold is
computed on Cc init.
Compared to what was done previously, this lazy writer will only write down files
that are not marked as temporary.
The mechanisms involved in this lazy writer worker are well described in Windows
Internals 4th editions (constants are coming from it ;-)).
Also fixed a bad (and old!) bug in CcRosFlushDirtyPages() where target count could
be overflow and the function would spin forever while holding the VACBs lock. This is
mandatory as now lazy writer will call it with "random" values.
This also allows implementing CcWaitForCurrentLazyWriterActivity() :-).
Also renamed DirtyPageCount to its MS equivalent.

CORE-14235
2018-01-23 19:33:59 +01:00
Pierre Schweitzer 2abb99faa9
[NTOSKRNL] In FsRtlAddToTunnelCache() allocate memory from PagedPool when required.
Also, if allocating from lookaside list, reattempt a cold allocation.
2018-01-20 21:21:00 +01:00
Thomas Faber fda24da872
[NTOS] Remove unnecessary NULL checks and add a missing one. CIDs 1411958, 716670, 1322186 2018-01-20 15:59:35 +01:00
Thomas Faber 2ae756a4e5 [NTOS:MM] Fix broken ASSERT. CID 701285 2018-01-16 22:29:19 +01:00
Pierre Schweitzer c779dbd4c5
[NTOSKRNL] Implement CcIsThereDirtyData() 2018-01-15 23:20:53 +01:00
Timo Kreuzer 2dade10d54 [NTOS] Fix 64 bit warnings 2018-01-14 13:54:14 +01:00
Pierre Schweitzer 371aba1abe
[NTOSKRNL] When mapping data in CcMapData(), don't truncate offset to ULONG.
This would affect reads/writes on large volumes where offset is higher than what a ULONG can hold.

This really nasty bug was hitting CcMapData() but also CcPinRead() (due to the nature of its implementation)
and both were returning garbage data under certain circumstances with Ext2Fsd.

This should (I hope!) help some other FSDs to work better in ROS.

CORE-12456
2018-01-13 21:30:37 +01:00
Mark Jansen 7b398514f1 [NTOS] Limit the spam from NtNotifyChangeMultipleKeys.
CORE-13125
2018-01-08 22:24:54 +01:00
Timo Kreuzer 1014d50a4b [NTOS:MM] Pass FaultCode to MiDispatchFault 2018-01-06 18:20:24 +01:00
Timo Kreuzer 3f426aabde [NTOS:MM] Mark local functions with static 2018-01-06 18:20:24 +01:00
Timo Kreuzer a00378a1a1 [NTOS:MM] Add support for NX page-faults 2018-01-06 18:20:24 +01:00
Timo Kreuzer 4d043aa05e [NTOS:MM] Make use of FaultCode and MI_IS_NOT_PRESENT_FAULT and MI_IS_WRITE_ACCESS macros in MmArmAccessFault. 2018-01-06 18:20:24 +01:00
Timo Kreuzer d5c71429d7 [NTOS:MM] Pass The FaultCode to MmArmAcceessFault as well and translate it there to what was declared as "StoreInstruction"
No functional changes.
2018-01-06 18:20:24 +01:00
Timo Kreuzer 3021c2d571 [NTOS:MM] Pass page fault code to MmAccessFault
Note: before we had a BOOLEAN parameter called StoreInstruction, but in reality it was not specifying whether the fault was from a store store instruction, but whether it was an access violation rather than a page-not-present fault. On x86 without PAE there are only 2 kinds of access violations: (1) Access of a kernel mode page from user mode, which is handled early and (2) access of a read-only (or COW) page with a writing instruction. Therefore we could get away with this, even though it relied on the wrong assumption that a fault, which was not a page-not-present-fault, was automatically a write access. This commit only changes one thing: we pass the full fault-code to MmAccessFault and handle the rest from there in exactly the same way as before. More changes are coming to make things clear.
2018-01-06 18:20:24 +01:00
Colin Finck f5d366b200
[NTOS:CM] Improve code in cmsysini.c (#216)
Based on an original patch by Timo Kreuzer, with modifications by me to adapt it to latest HEAD and use a single exit path through the Cleanup label. This reliably frees all allocated handles.

The original code returns STATUS_SUCCESS for many cases. This has been preserved.
In the future, it should be checked though whether returning success is appropriate for all these cases.

CORE-6844
2018-01-06 13:27:41 +01:00
Timo Kreuzer f555c102b7 [NTOS:EX] Don't dereference ExpCritSecOutOfMemoryEvent when it was never referenced. 2018-01-04 21:05:10 +01:00
Katayama Hirofumi MZ a8bd06be98
[NTOS:KDBG] Fix DPRINT format specifier in KdPortInitializeEx. CORE-14174 2018-01-04 11:37:50 +01:00
Timo Kreuzer d17812f933 [NTOS:MM] Use MiResolveDemandZeroFault instead of MiDispatchFault 2018-01-03 23:07:19 +01:00
Timo Kreuzer d4765fe366 [NTOS:MM] Implement resolving PXE/PPE page table demand zero faults 2018-01-03 23:07:19 +01:00
Timo Kreuzer 2eff510074 [NTOS:MM] Increment Prcb->MmDemandZeroCount while holding the Pfn lock 2018-01-03 23:07:19 +01:00
Timo Kreuzer 1240ed9e18 [NTOS/MM] Add Protection parameter to MiResolveDemandZeroFault 2018-01-03 23:07:19 +01:00
Hermes Belusca-Maito b2be558e61 [NTOS:INBV] Simplify a little bit the BootLogoFadeIn() function
- Use KeDelayExecutionThread() instead of KeStallExecutionProcessor().
- Fix magic values and add comments.
- Fix structure name.
2018-01-03 02:57:28 +01:00
Stanislav Motylkov dd71fb5ec8 [NTOS:INBV] Implement rotation bar for boot screen
CORE-10327 #resolve
2018-01-03 02:57:28 +01:00
Timo Kreuzer c4d6c2c3f1 [NTOS:KE] Fix pushing of fake error code in KiDoubleFaultAbort (amd64) 2018-01-01 12:09:38 +01:00
Timo Kreuzer a2f7de7ee8
Amd64/misc 64bit fixes (#236)
[REACTOS] Misc 64 bit fixes

* [NTOS:MM] Allow MEM_DOS_LIM in NtMapViewOfSection on x64 as well
* [NTOS:MM] Implement x64 version of MmIsDisabledPage
* [HAL] Remove obsolete code
* [NTOS:KE] Fix amd64 version of KeContextToTrapFrame and KeTrapFrameToContext
* [XDK] Fix CONTEXT_XSTATE definition
* [PCNET] Convert physical address types from pointers to PHYSICAL_ADDRESS
2018-01-01 12:07:50 +01:00
Hermès Bélusca-Maïto 9fa693e59b
[REACTOS] Switch to year 2018...
... and Happy New Year 2018 to everyone!!

☆.¸¸.•´☆.¸¸.•´☆.¸¸.•´☆.¸¸.•´☆

  ⋱★⋰⋱☆⋰⋱★⋰⋱☆⋰⋱☆⋰⋱★⋰⋱☆⋰⋱★⋰
░░░░░░█░█ █▀█ █▀█ █▀█ █░█░░░░░░
░░░░░░█▀█ █▀█ █▀▀ █▀▀ ▀█▀░░░░░░
░░░░░░▀░▀ ▀░▀ ▀░░ ▀░░ ░▀░░░░░░░
█▄░█ █▀▀ █░█░█░░█░█ █▀▀ █▀█ █▀█
█░██ █▀▀ █░█░█░░▀█▀ █▀▀ █▀█ ██▀
▀░░▀ ▀▀▀ ░▀▀▀░░░░▀░ ▀▀▀ ▀░▀ ▀░▀
  ⋱☆⋰⋱★⋰⋱☆⋰⋱★⋰⋱★⋰⋱☆⋰⋱★⋰⋱☆⋰
░░░▄████▄░▄███▄░░▄██░▄████▄░░░
░░░▀▀░▄██░██░██░████░██▄▄██░░░
░░░░▄██▀░░██░██░░░██░██▀▀██░░░
░░░██████░▀███▀░░░██░▀████▀░░░

☆.¸¸.•´☆.¸¸.•´☆.¸¸.•´☆.¸¸.•´☆
2018-01-01 00:00:00 +01:00
Thomas Faber 75850abcbb
[NTOS] Use KeClearEvent instead of KeResetEvent where the previous state is not needed. 2017-12-30 11:45:03 +01:00
Pierre Schweitzer 7fecde4a20
[NTOSKRNL] Print tags which contains numerics in the memory dumper.
This fixes i8042prt.sys allocations not properly being rendered.
2017-12-29 20:37:43 +01:00
Pierre Schweitzer 4663d02cd7
[NTOSKRNL] In the memory dumper, print in the direct order to make !poolused TAG usage easier 2017-12-29 20:37:43 +01:00
Pierre Schweitzer ca3143b9ab
[NTOSKRNL] Add support for verbose output in the !poolused command 2017-12-29 20:37:43 +01:00
Timo Kreuzer cbc4cfeed6 [NTOS:EX] Fix handle table code for x64.
Based on patch by Ivan Labutin. See PR #115
2017-12-29 19:44:09 +01:00
Pierre Schweitzer 2daf2391a6
[NTOSKNRL] Addendum to 9a633f7 2017-12-29 18:22:37 +01:00
Pierre Schweitzer 9a633f7b48
[NTOSKRNL] Fix MSVC build 2017-12-29 18:03:33 +01:00
Pierre Schweitzer 4e34bf2a0b
[NTOSKRNL] Add support for the ? wildcard in the !poolused command 2017-12-29 17:23:36 +01:00
Pierre Schweitzer 454e8738f2
[NTOSKRNL] Allow filtering !poolused output using a tag 2017-12-29 17:23:36 +01:00
Pierre Schweitzer 879d8f2104
[NTOSKRNL] Now that the memory dumper handles paged pool, make use of it in any situation 2017-12-29 17:23:36 +01:00
Pierre Schweitzer 1433ade827
[NTOSKRNL] Add support for the paged pool in the memory dumper 2017-12-29 17:23:36 +01:00
Pierre Schweitzer bb63841b57
[NTOSKRNL] Prettify the memory dumper output and add number of allocs 2017-12-29 17:23:36 +01:00
Pierre Schweitzer 78b55550bb
[NTOSKRNL] Make the memory dumper available as a kdbg command: !poolused 2017-12-29 17:23:36 +01:00
Amine Khaldi 007131b5de [NTOS] Fix Shim AVL prototypes to exactly match the types expected by RtlInitializeGenericTableAvl. #179 2017-12-25 22:34:57 +01:00
Amine Khaldi 4d4e4bb68a [NTOS] Deduplicate MiSynchronizeSystemPde(). #179 2017-12-25 20:14:46 +01:00
Pierre Schweitzer fc5a61d8b7
[NTOSKRNL] Revert 4ef0887: experiments show that our FastFAT is not ready yet to live on its own.
So, bring back the infamous IopParseDevice() hack. Dismounting is to be fixed in FastFAT.
Even though it might not be the last remaining bug...

CORE-14124
CORE-14126
CORE-14133
2017-12-25 11:24:13 +01:00
Hermès Bélusca-Maïto ff8d8a8162
Revert 57c0678 (PR #194) that has been erroneously committed too early. 2017-12-24 16:54:48 +01:00
Stanislav Motylkov 57c0678818
[NTOS:INBV] Implement rotation bar for boot screen 2017-12-24 16:47:02 +01:00
Samuel Serapion a59d4674de [NTOS] Remove forgotten assert, see CORE-14128. 2017-12-20 21:00:52 +01:00
Timo Kreuzer 12e2c48a25 [NTOS:MM] Fix a bug in MiBuildPfnDatabase
MxPfnAllocation is in pages, not in bytes!
2017-12-20 06:56:09 +01:00
Timo Kreuzer 18cd763160 [NTOS:MM] Make debug code in MmArmInitSystem portable 2017-12-20 06:56:09 +01:00
Timo Kreuzer 11c78f701d [NTOS:MM] Implement x64 version of MI_MAKE_SUBSECTION_PTE 2017-12-20 06:56:09 +01:00
Ivan Labutin 8723be733c [NTOS:PS] Skip backtrace on non-x86 2017-12-20 06:56:09 +01:00
Ivan Labutin 0d01b00871 [NTOS:FSRTL] Fixed incorrect comparison 2017-12-20 06:56:09 +01:00
Ivan Labutin d6792047f3 [NTOS][NDK][RTL] A bunch of 'wrong size' fixes 2017-12-20 06:56:09 +01:00
Ivan Labutin 4d35d59fb9 [NTOS:MM] Quick fix: use SIZE_T instead of ULONG, because ULONG is 32-bit and on 64-bit truncates values 2017-12-20 06:56:09 +01:00
Ivan Labutin ede3f4d449 [NTOS:KE] Improve readability of CPU flag checks and fix error checking wrong register for HT support 2017-12-20 06:56:09 +01:00
Ivan Labutin 5857733996 [NTOS:KE] Remove duplicate NX enabling 2017-12-20 06:56:09 +01:00
Ivan Labutin 39dd47cebb [NTOS:MM] Comment out debug code for x86 that should not run on x64 2017-12-20 06:56:09 +01:00
Ivan Labutin 1aa70ef6d1 [NTOS:KE] Move RtlpUse16ByteSLists initialization into KiInitializeKernel 2017-12-20 06:56:09 +01:00
Ivan Labutin b9f592c054 [NTOS:KE] EFER constants fix 2017-12-20 06:56:09 +01:00
Timo Kreuzer 92b8d327b9 [NTOSKRNL] Fix return of uninitialized variable in PspSetQuotaLimits
CID-1322247
2017-12-20 05:23:06 +01:00
Pierre Schweitzer 4ef08871ee
[NTOSKRNL] Make again an attempt at killing the IopParseDevice() hack.
For the record, the only place it was remaining was 1st stage, notably because
FSD (FastAT <3) was missing a few features. As this features weren't triggered
in 3rd stage (unless forced), it was not needed there any longer.

Now that they are implemented, and seem working out of the box, this hack might
not be longer anymore.

This is my ... ?! pfff attempt at killing it. It might not be the last, so just
disabling the hack for now. If there are no regressions (who can really believe that?)
we'll be clear for dropping it once for all.
2017-12-17 23:30:58 +01:00
Serge Gautherie 614ace03fa
[NTOS:MM] Fix NtFreeVirtualMemory() FreeType parameter checks. CORE-13126 2017-12-15 11:56:35 +01:00
Thomas Faber 7e8de26164
[NTOS:KD] Pass PreviousMode down to KdpPrintString, since ExGetPreviousMode is not accurate. CORE-14103 2017-12-14 11:46:18 +01:00
Thomas Faber 7b95fcf93d
[NTOS:KD] Avoid _alloca inside SEH, as it's apparently incompatible with PSEH. CORE-14103 2017-12-14 11:46:16 +01:00
Thomas Faber a07b569b25
[NTOS:KD] Don't assume null termination in KdpSerialDebugPrint and KdpScreenPrint. CORE-14057 CORE-14103 2017-12-14 11:46:14 +01:00
Amine Khaldi 631a14ff64 [ASM][NDK][NTOS] Rename Self to SelfPcr in the KIPCR structure. 2017-12-13 13:49:00 +01:00
Amine Khaldi 24f35397c6 [NTOS] Add some header guards. 2017-12-12 12:42:13 +01:00
Thomas Faber 68ebcf16b8 [NTOS:KD] Protect against invalid user arguments for BREAKPOINT_LOAD_SYMBOLS. CORE-14057 2017-12-10 15:26:18 +01:00
Thomas Faber 34ccecbce8 [NTOS:KD] Protect against invalid user arguments in KdpPrintString. CORE-14057 2017-12-10 15:26:18 +01:00
Thomas Faber 1a38c76266 [NTOS:KD] Protect against invalid user arguments in KdpPrompt. CORE-14057 2017-12-10 15:26:18 +01:00
Thomas Faber 81532227b6
[NTOS:MM] Implement and use MiAcquirePfnLockAtDpcLevel/MiReleasePfnLockFromDpcLevel. 2017-12-08 10:32:22 +01:00
Thomas Faber da5949280c
[NTOS:MM] Use inline functions to acquire/release the PFN lock. 2017-12-08 10:32:08 +01:00
Pierre Schweitzer c473b1a4d6
[NTOSKRNL] Only activate dumper in DBG builds 2017-12-05 23:28:04 +01:00
Pierre Schweitzer 149e1a2610
[NTOSKRNL] Add a non paged memory dumper in case of low memory situation.
Its purpose is to dump the non paged consumption, tag by tag,
to allow tracking potential faulting driver in case ReactOS starts lacking memory.
This will look like what !poolused outputs, even though it doesn't deal with paged pool.

Thanks to Thomas for his kind review and improvement suggestions.

CORE-14048
2017-12-05 23:19:11 +01:00
Pierre Schweitzer 9ecbbe2a33
[NTOSKRNL] Use ExAllocatePoolWithQuotaTag() when allocating SystemBuffer for the IRP
That way, in case the system lacks memory, an exception is thrown and IRP isn't sent
to the device with NULL SystemBuffer.

CORE-14048
2017-11-26 14:03:30 +01:00
Pierre Schweitzer d01184b164
[NTOSKRNL] Use cache aligned buffer for devioctrl 2017-11-26 14:03:30 +01:00
Amine Khaldi 09c06a2f45 [CLANG-CL] Initial commit that allows us to compile ReactOS with clang-cl. 2017-11-23 14:09:57 +01:00