Commit graph

224 commits

Author SHA1 Message Date
Hermès Bélusca-Maïto de87a936ce
[NTOS] Demote an ASSERT to a mere DPRINT since we only support registry hives with cluster size == 1 in any case so far (other sizes are UNIMPLEMENTED!) 2018-05-19 22:07:06 +02:00
Pierre Schweitzer 7eca874824
[NTOSKRNL] Fix FsRtlStackOverflow work item free.
Oneliner of the day... This typo just prevented the
whole feature to work properly. Because any allocated
work item would miserably fail to be freed.

This will obviously help real world FSD relying on
StackOverflow worker from FsRtl to work better!

CORE-14611
2018-05-10 10:49:17 +02:00
Pierre Schweitzer 65e29b4b1f
[NTOSKRNL] Optimize a bit deferred writes.
In the lazy writer run, first post items that are queued for this.
Only then, start executing deferred writes if any.
If there were any, reschedule immediately a lazy writer run, to keep
Cc warm and to make it unqueue write faster in case of high IOs situation.
To make second lazy writer run happen faster, we keep our state active to
use short delay (1s) instead of standard idle (3s).
2018-05-02 23:33:45 +02:00
Pierre Schweitzer 54c049bd6e
[NTOKSNRL] Always flush dirty VACB.
Recent changes seem to show that it's not
required to be exclusive on VACB to be able
to flush it.

This commit goes with f2c44aa and fixes the
last issues going with copying huge files.
There are no longer BSODs (be it in Mm or Cc).
I could, with 750MB RAM extract a 2GB file from
a 53MB archive and copy a 2,5GB file from a VBox
share to the disk. Note that writes are often
deferred, so if copy works, it's not that fast for now.

Note that it also brings some beloved behavior from
Windows: copy times are totally unreliable now when
writes are deferred. Little remaining times when
actively copying, high remaining times when deferred
writes in action. And goes between both... Sorry! ;-)

https://xkcd.com/612/

CORE-9696
CORE-11175
2018-04-30 22:24:30 +02:00
Pierre Schweitzer 74c5d8b6bd
[NTOSKRNL] Free unused VACB when required.
Same mechanism exists in Windows (even their Cc
is way different from ours...) where when Cc is
out of memory (in their case, out of VACB), we
will start scavenge old & unused VACB to free
some of the memory.

It's useful in case we're operating we big files
operations, we may run out of memory where to map
VACB for them, so start to scavenge VACB to free
some of that memory.

With this, I am able to install Qt 4.8.6 with 2,5GB of RAM,
scavenging acting when needed!

CORE-12081
CORE-14582
2018-04-30 12:10:24 +02:00
Pierre Schweitzer cc54e51495
[NTOSKRNL] Unmark dirty first, and then write.
This will avoid trying to flush twice a dirty VACB under
high IOs pressure.

CORE-14584
2018-04-30 10:36:19 +02:00
Pierre Schweitzer f2c44aa483
[NTOSKRNL] Fix lazy writer for in-use VACB.
Adjusting refcount and enabling lazy-write for pinned
VACB makes it actually more efficient, often purging
data to disk, reducing memory stress for the system.

This is required for defering writes.

This commit unfortunately (?) reverts a previous revert.

CORE-12081
CORE-14582
CORE-14313
2018-04-29 20:42:53 +02:00
Pierre Schweitzer 2ea6de8a42
[NTOSKRNL] Also try to extract name from FCB when leaking VACB 2018-04-27 19:01:35 +02:00
Pierre Schweitzer 43836b0fbb
[NTOSKRNL] In !filecache, try to display FCB name
When no name is set in the file object, try to read the name
from the FCB. We only support FastFAT (ours) FCB for now.

This is clearly a hack, but for a kdbg command, so ;-)
2018-04-27 18:57:30 +02:00
Pierre Schweitzer 579a784e04
[NTOSKNRL] In case we leak a VACB, debug as much information as possible.
CORE-14578
2018-04-27 14:14:56 +02:00
Pierre Schweitzer fcf83315dc
[NTOSKRNL] Noisily dereference mapped VACB on cache release.
It seems that on process killing, some VACB may be deleted while
still mapped. With current reference counting, they will actually
not be deleted, but leaked, and an ASSERT will be triggered.

CORE-14578
2018-04-27 10:23:06 +02:00
Jason Johnson 054c11a02c Correctly bugcheck if we aren't returning a usermode thread (#506)
Cause a corresponding bug check to occur for the reason of the user context flag being zero (a system thread), instead of using DbgBreakPoint.
2018-04-24 23:07:02 +01:00
Hermès Bélusca-Maïto e616a456c0
[NTOSKRNL] Minor PnP enhancements.
- For non-PnP devices reported to the PnP manager through the
  IoReportDetectedDevice() function, store the corresponding
  service/driver name and (non-)legacy information inside their
  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\Root\ entries.
- Drivers flagged as "DRVO_BUILTIN_DRIVER" (basically, only those
  created via a IoCreateDriver() call) have their "Service" name that
  contain "\Driver\", which should be stripped before being used in
  building e.g. the corresponding "DETECTEDxxx" PnP compatible IDs.
  CORE-14247

- Use explicit REG_OPTION_NON_VOLATILE flag where needed in the
  IopCreateDeviceKeyPath() calls.
- Save NULL-terminated REG-SZ string properties in the enumeration tree
  for each device enumerated inside \Enum\Root\.
- Always use upcased key name for the "LEGACY_***" elements in \Enum\Root\.
- Add a default "ConfigFlags" value for the legacy elements.
- Simplify few parts of code.
2018-04-22 16:52:45 +02:00
Hermès Bélusca-Maïto fa103197ad
[NTOSKRNL] Whitespace fixes only. 2018-04-22 16:52:40 +02:00
Hermès Bélusca-Maïto 45012aa486
[NTOSKRNL] Set the DRVO_BUILTIN_DRIVER DriverObject flag when a built-in driver is created using IoCreateDriver(). 2018-04-22 16:52:38 +02:00
Hermès Bélusca-Maïto 635775d249
[NTOSKRNL] Set flags the standard way. 2018-04-22 16:52:36 +02:00
Timo Kreuzer e49ef251b7 [NTOS:MM] Fix ViewSize parameter passed to MiInsertVadEx() from MiCreatePebOrTeb()
The size is in bytes, not in pages! On x86 we got away with it, since PEB and TEB require only a single page and the 1 passed to MiInsertVadEx() was aligned up to PAGE_SIZE. On x64 this doesn't work, since the size is 2 pages.
2018-04-22 15:06:19 +02:00
Thomas Faber d82ccaa2a2
[NTOS:PNP] Use pool tagging for DEVICE_ACTION_DATA. CORE-10456 2018-04-18 16:13:14 +02:00
Thomas Faber 121146e651
[NTOS:PNP] Rename IopDeviceRelations* to IopDeviceAction*. CORE-10456 2018-04-18 16:12:56 +02:00
Pierre Schweitzer fd3a6c1089 [NTOSKRNL] Properly reset VACB on free
CID 1434271
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 953dc72dad [NTOSKRNL] Drop the VACB lock.
This has have several benefits for ReactOS Cc:
- It helps reducing potential deadlocks situations in Cc
- It speeds up ReactOS by reducing locks
- It gets us a bit closer to Windows VACB

CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 40017a54f9 [NTOSKRNL] Use interlocked operations when dealing with map count.
CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 1b672981e2 [NTOSKRNL] Map the VACB in kernel space before inserting it in lists.
The avoids race conditions where attempts to read from disk to
not fully initialized VACB were performed.
Also, added more debug prints in such situations.

CORE-14349
2018-04-15 22:52:53 +02:00
Pierre Schweitzer 42df4683d7 [NTOSKRNL] Add extra sanity checks for VACB lists.
We now always initialize list members from the VACB
and make sure the list entry has properly been removed
from the list before free.

CORE-14349
2018-04-15 22:52:53 +02:00
Thomas Faber b54e5c689c
[NTOS:MM] Do not map two pages into hyperspace in MiCopyFromUserPage. CORE-14548
Doing this is not only wrong because it acquires the same spinlock twice,
it also completely breaks the TLB flushing logic in MiMapPageInHyperSpace.
If the PTE with Offset 1 is still valid when a wrap-around to 0 happens,
the TLB flush on wrap-around will not clear the entry for this previous page.
After another loop around all hyperspace pages, page 1 is re-used but its
TLB entry has not been flushed, which may result into incorrect translation.
2018-04-15 20:07:21 +02:00
Serge Gautherie 6618a2fd2c [NTOS:CC] Use UNIMPLEMENTED_ONCE instead of custom code
- Rewrite e319f85e67.
2018-04-07 12:00:10 +02:00
Mark Jansen a5f12908df
[NTOSKRNL] Fix IopValidateID using uninitialized var, reset SeparatorsCount in the MultiSz case. 2018-04-05 22:06:31 +02:00
Timo Kreuzer 6b1ca75899
[REACTOS] Fix 64 bit build (#465)
* [HAL] Simplify HalpReboot() and make it portable
* [NTOS:MM] Cast constant to PVOID
* [BINPATCH] Fix 64 bit build
* [VFDDRV] Fix 64 bit build and buffer overruns
* [USBOHCI] Fix structure alignment issues
* [ATL_APITEST] Fix 64 bit build
* [XDK] Update unwind structures in winnt.h
* [NTDLL_APITEST] Fix 64 bit build
* [NTDLL_WINETEST] Fix 64 bit build
* [TFTPD] Fix x64 build
* [USBPORT] Fix a C_ASSERT
* [DSOUND] Fix x64 build
* [HAL] Remove obsolete GetPteAddress() macro
2018-04-03 15:13:17 -06:00
Hermès Bélusca-Maïto c54aaacc8e
[NTOSKRNL] Formatting fix only. 2018-04-01 22:51:24 +02:00
Hermès Bélusca-Maïto 6cefd1242d
[NTOSKRNL] Simplify NtRaiseHardError() by merging the terminating blocks.
- Return the status codes provided by the Ex(p)RaiseHardError() calls.
- Fix the return values in case of failure.
2018-04-01 22:50:56 +02:00
Hermès Bélusca-Maïto ea390c2b3f
[NTOSKRNL] ExRaiseHardError(): Protect strings copy to user-mode space inside a SEH block. 2018-04-01 22:50:44 +02:00
Hermès Bélusca-Maïto 842605e3ef
[NTOSKRNL] Don't emit hard errors for the calling thread if hard errors have been disabled for this thread on user-mode side. 2018-04-01 22:50:33 +02:00
Hermès Bélusca-Maïto 3a0ecb3dc8
[NTOSKRNL] Don't hardcode flag values for DefaultHardErrorProcessing. 2018-04-01 22:50:16 +02:00
Hermès Bélusca-Maïto 1fadfea053
[NTOSKRNL] Update Doxygen descriptions for NtRaiseHardError, ExRaiseHardError, ExpRaiseHardError and NtSetDefaultHardErrorPort. 2018-04-01 22:39:32 +02:00
Hermès Bélusca-Maïto f0729b30bb
[NTOSKRNL] Forbid processes without the Tcb prvilege to perform a user-mode hard-error BSOD. 2018-04-01 22:39:31 +02:00
Hermès Bélusca-Maïto ed06b843fb
[NTOSKRNL] In addition to the hard-error port, reference also the process that handles the hard errors so that it doesn't disappear behind our back. On shutdown both the hard-error port and process are dereferenced. 2018-04-01 22:39:30 +02:00
Pierre Schweitzer ffd524275e
[NTOSKRNL] Properly delete VACB in CcRosCreateVacb() when mapping fails.
Spotted by Thomas.

CORE-14478
CORE-14502
2018-03-25 18:27:19 +02:00
Pierre Schweitzer 14b05e65ff
[NTOSKRNL] Use interlocked operations for VACB reference counting.
CORE-14480
CORE-14285
2018-03-24 19:15:58 +01:00
Pierre Schweitzer dea9c291ab
[NTOSKRNL] Add a few asserts when mapping a VACB in kernel space
Also, reset VACB content when returning it to the lookaside list

CORE-14478
2018-03-24 19:15:58 +01:00
Pierre Schweitzer 2b6df67f0a
[NTOSKRNL] More asserts regarding reference count
CORE-14285
CORE-14480
2018-03-24 11:59:45 +01:00
Vadim Galyant 111e19f8a4 [NTOSKRNL] Adding IopValidateID() to test characters in PnP IDs (IRP_MN_QUERY_ID). (#341) 2018-03-19 14:04:17 +01:00
Thomas Faber a17bf6f669
[NTOS:MM] Improve debug prints in MmCreateVirtualMappingUnsafe. CORE-14478 2018-03-18 23:49:48 +01:00
Pierre Schweitzer 1e579843bc
[NTOSKNRL] Always reference a newly created VACB
This allows being consistent between newly created and looked up
so that VACB can always safely be released.

Should really help with reference issues.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 18:16:55 +01:00
Pierre Schweitzer 2a0e996c9d
[NTOSKRNL] In CcRosInternalFreeVacb(), in case of invalid free, also print file name.
CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer 2fbba22789
[NTOSKRNL] In CcFlushCache(), release the VACB using CcRosReleaseVacb()
Instead of reimplementing it partially and wrongly.

CORE-14481
CORE-14480
CORE-14482
2018-03-18 13:21:54 +01:00
Pierre Schweitzer 13b57fb5b5
[NTOSKRNL] Misc fixes to VACB reference counting
This fixes various bugs linked to VACB counting:
- VACB not released when it should be
- Reference count expectations not being accurate

For the record, VACB should always have at least a reference
count of 1, unless you are to free it and removed it from
any linked list.

This commit also adds a bunch of asserts that should
help triggering invalid reference counting.

It should also fix numerous ASSERT currently triggered and
may help fixing random behaviours in Cc.

CORE-14285
CORE-14401
CORE-14293
2018-03-17 11:56:25 +01:00
Thomas Faber 2b8f4f6e24
[NTOS:EX] Worker threads must not exit with a critical region held. CORE-13734 2018-03-07 13:18:09 +01:00
Thomas Faber 6b78ff036f
[NTOS:KE] Don't loop indefinitely trying to figure out the CPU frequency. CORE-14419
Previously, we would keep sampling the CPU frequency until two subsequent
samples differed by at most 1 MHz. This could take several seconds, and would
unnecessarily delay boot.
Instead, if sampling is too unreliable, just give up and calculate the average
frequency from 10 samples. This is no worse than picking the frequency that
just happened to be returned twice in a row.

The fact that this method of sampling fails could indicate that there's a
problem with our performance counter implementation or timer interrupt,
but that's a separate issue...
2018-03-01 14:26:44 +01:00
Pierre Schweitzer c4f58bbfd8
[NTOKSRNL] Don't blindly schedule read-ahead on CcCopyRead() call.
This avoids locking Cc for too long by trying to read-ahead data which
is already in cache.
We now will only schedule a read ahead if next read should bring us
to a new VACB (perhaps not in cache).

This notably fixes Inkscape setup which was slown down by read-ahead
due to continous 1 byte reads.

Thanks to Thomas for his help on this issue.

CORE-14395
2018-02-28 20:58:36 +01:00
Pierre Schweitzer 62facb7a49
[NTOSKRNL] Implement ThreadHideFromDebugger in NtSetInformationThread() 2018-02-24 22:07:12 +01:00