Commit graph

31 commits

Author SHA1 Message Date
Sir Richard 14f8621042 [NTOS]: Add MiRemoveZeroPageSafe helper function, when a zero page is required, but the inline zeroing of MiRemoveZeroPage is not. This function will only try grabbing a zero page if one exists, otherwise a free page will be grabbed and zeroed with custom code of the caller's choosing.
[NTOS]: Add concept of process color and system color. Compute correct color to use whenever requesting a page.
[NTOS]: Uncondtionally enable the color code when inserting/removing pages.
For now, when requesting a page, colors are still ignored, and the global PFN lists are scanned instead. If there are no regressions, we are one patch away from that.

svn path=/trunk/; revision=48927
2010-09-29 01:10:28 +00:00
Sir Richard edfb344c53 [NTOS]: Remove useless variables in kernel code that were set, but never actually used (dead code, tests, copy/pasters). If a variable was set but not used because of missing/#if'ed out code, a note was added instead.
[NTOS]: In the process, fix bugs in the Event dispatcher code that used Win32 EVENT_TYPE instead of NT KOBJECTS enumeration.
[NTOS]: Fix a bug in ObpInsertHandleCount, where the object access check was being done with the previous mode, instead of honoring the probe mode, which is defined by OBJ_FORCE_ACCESS_CHECK.
[NTOS]: Fix a bug in a section function which was always returning STATUS_SUCCESS, now it returns the result of the previous Status = function assignment. If this isn't desired, then don't check for the Status anymore.
[NTOS]: Note that MDL code does not support SkipBytes argument. If it is used, MDL could be invalid.
[NTOS]: Add checks for VerifierAllocation and set it when needed (WIP).
[NTOS]: Clarify what _WORKING_LINKER_ is, and the legal risks in continuing to use a linker that builds non-Microsoft drivers when used with headers whose EULA specify that they can only be used for Microsoft drivers.

svn path=/trunk/; revision=48692
2010-09-04 08:17:17 +00:00
Sir Richard c574f50663 [NTOS]: Add an extra layer of protection for freed nonpaged pool: write a 4-byte signature on freed blocks, and assert its valid on checked builds. Use a slightly less egocentric ASCII value than on Windows (name of the developer who wrote the first memory manager).
svn path=/trunk/; revision=48651
2010-08-29 19:32:25 +00:00
Sir Richard b85ab20f13 [NTOS]: Missed a bunch of codepaths, protected pool "should" work now.
svn path=/trunk/; revision=48650
2010-08-29 19:27:58 +00:00
Sir Richard cf28a01e5e [NTOS]: Add DRIVER_CAUGHT_MODIFYING_FREED_POOL bugcheck code.
[NTOS]: Add support for protected freed nonpaged pool. This is controlled through MmProtectFreedNonPagedPool, which is initialized based on a registry value (see cmdata.c). This is not "Special Pool", but a useful debugging feature Windows implements that we now have too, since I noticed a lot of mj's work was with freed pool access.
NB. It's 3AM and I have not tested this, it should be off in trunk by default, you'll need to try turning it on and testing it. Hope it helps.
--This line, and those low, will be ignored--

M    ntoskrnl/mm/ARM3/pagfault.c
M    ntoskrnl/mm/ARM3/pool.c
M    include/reactos/mc/bugcodes.mc

svn path=/trunk/; revision=48649
2010-08-29 19:13:08 +00:00
Sir Richard a9fb81cbae [NTOS]: Another long-lost-promised straggler for Timo. There may be more of these needed, this is what I had done on my disk.
svn path=/trunk/; revision=48288
2010-07-26 21:45:42 +00:00
Timo Kreuzer ef44812048 [NTOSKRNL]
#if out another broken C_ASSERT on amd64

svn path=/trunk/; revision=48084
2010-07-16 16:30:05 +00:00
Stefan Ginsberg 622c4a0919 [NTOS] Inline writing to PTEs through MI_WRITE_VALID/INVALID_PTE.
svn path=/trunk/; revision=47633
2010-06-06 18:45:46 +00:00
Sir Richard 570567b87e [NTOS]: Kill some debug spew.
svn path=/trunk/; revision=47604
2010-06-06 00:49:26 +00:00
Sir Richard f4f8ee78d1 [NTOS]: Implement MiDeleteSystemPageableVm.
[NTOS]: The paged pool free code was behaving incorrectly, assuming that paged pool was "locked down" and never paged out/reused (a valid NT operation mode), while the allocation code was assuming paged pool was a volatile, reusable, pageable resource (normal NT operation mode). The free code now assumes normal operation mode, and actually frees the freed paged pool pages, by using MiDeleteSystemPageableVm.
I have a feeling this will make ARM3 paged pool work.

svn path=/trunk/; revision=47582
2010-06-05 04:16:46 +00:00
Sir Richard 25bf23bfc1 [NTOS]: When expanding paged pool, use MiRemoveAnyPage, not MmAllocPage.
[NTOS]: When expanding paged pool, initialize the PFN entry for the allocated page. Note we might be in arbitrary process space, so the PTE is not necessary valid for the process causing the expansion.
[NTOS]: Implement MiInitializePfnForOtherProcess to handle the case above.
[NTOS]: Change two static ASSERTs into C_ASSERTs. Might break non-x86 builds for a bit (vs breaking them at boot, which is worse).
Paged pool should start working soon.

svn path=/trunk/; revision=47579
2010-06-04 22:08:40 +00:00
Sir Richard 63ce635b0f [NTOS]: Build paged pool demand-zero PTE with MI_MAKE_SOFTWARE_PTE macro.
[NTOS]: Handle paged pool demand-zero fault fulfillment with MI_MAKE_HARDWARE_PTE macro.
[NTOS]: Use MiRemoveAnyPage instead of MmAllocPage, in paged pool demand-zero fault fulfillment.
These changes affect code paths that are not currently in-use.

svn path=/trunk/; revision=47575
2010-06-04 20:18:27 +00:00
Sir Richard 1c28c16dfe [NTOS]: Allocate non-paged pool pages with MiRemoveAnyPage instead of MmAllocPage.
svn path=/trunk/; revision=47570
2010-06-04 17:40:11 +00:00
Sir Richard df33b38ed0 [NTOS]: Fix definition of unused MI_MAKE_SOFTWARE_PTE macro.
[NTOS]: Correctly setup the PFN entries for freshly allocated paged pool pages. Fixes a problem where the page could've still had stale/garbage data.
[NTOS]: Add some extra assertions in the code to catch memory corruption and detect invalid logic.
[NTOS]: Fix some typos in the code (comments/whitespace).
[NTOS]: Make the dreaded page fault message that breaks paged pool on some systems more verbose for future debugging.

svn path=/trunk/; revision=47189
2010-05-12 22:47:46 +00:00
Sir Richard 11453cf565 [NTOS]: Add assertions to the paged pool and demand zero page faults, to catch possible errors and corruptions. These paths are not yet taken in today's builds, so they do not affect any runtime code.
[NTOS]: Add assertions regarding the portability of certain code, which will need changes on ARM/x64. These should probably be C_ASSERT's but I don't want to break Timo's build.
[NTOS]: Define MM_NOIRQL (found in assertions) instead of magical -1.
[NTOS]: Add MI_IS_SESSION_PTE macro.
[NTOS]: Export the MiXxxPte variables.
[NTOS]: Fix some typos in comments.


svn path=/trunk/; revision=47183
2010-05-12 20:48:15 +00:00
Sir Richard f90e9cc903 [NTOS]: Read almost all the Memory Management variables into the system configuration vector. These includes pool limits, percentages, debugging flags, behavioral changes, and others.
[NTOS]: Minimal stubbed support for some large page functionality in case it gets implemented later and these things would've been ignored. One critical change is that now the MmProcessList is initialized, and the system process is added to it. Other processes should be added later.
[NTOS]: Add stubbed support for parsing the registry list of drivers that should have large pages enabled.
[NTOS]: Initialize the loaded module list before working set work.
[NTOS]: Reload the boot drivers much earlier, as part of ARM3 initialization.
[NTOS]: Start doing some minor MmResidentAvailablePages accounting during bootup.
[NTOS]: Start doing the math required for computing the appropriate MmSystemCacheWsMinimum on the booting system (based on RAM).
[NTOS]: Now that we use the HAL heap, add the code to scan it for I/O mappings. These are dangerous because we need to sync up caching attributes (not yet used in ReactOS, so not a problem for now).
[NTOS]: Add more header definitions, and helper definitions such as PDE_MAPPED_VA instead of doing the math by hand.
[NTOS]: Move MmPageEntireDriver and MmResetDriverPaging to sysldr.c since drvmgmt.c is more for Driver Verifier-style functionality.

svn path=/trunk/; revision=46971
2010-04-20 22:47:51 +00:00
Sir Richard 23323a7250 [NTOS]: Make MM init read MmProductType to determine what SKU of ReactOS this is, instead of assuming Server. If you want to go back to the old behavior, you need to change "WinNT" to "ServerNT" in the hivesys under Product Type.
[NTOS]: Configure the MmSystemSize variable properly based on SKU and RAM. Previously, ReactOS told all drivers and applications you were running on a system with < 13MB RAM.
[NTOS]: Initialize thresholds for low and high memory (in pages), low and high paged pool memory, and low and high nonpaged pool memory. These are described in the source.
[NTOS]: Initialize events for each of those thresholds, and populate the \KernelObject\xxxCondition events that are documented in MSDN for driver and app developers.
[NTOS]: Define some internal thresholds to use later, representing the minimum number of free pages under we go berserk, and the minimum number of free pages that we consider "plenty".
[NTOS]: Rename MiRemoveFromList to MiUnlinkFreeOrZeroedPage (Windows name). Make the function handle MmAvailablePages decrement, instead of having the caller do it.
[NTOS]: Remove run-time initialization of the PFN lists, just initialize them statically (also fixes the fact we forgot to initialize their names).
[NTOS]: Move some more initialization code to ARM3 instead of mm.
[NTOS]: Read ProductType from registry into MmProductType instead of dummy value. Remove duplicate "Mirroring" variable read.

svn path=/trunk/; revision=45638
2010-02-20 21:48:36 +00:00
Sir Richard 60f00192eb [NTOS]: Remove SwapEntry parameter from MmAllocPage since it was always zero.
[NTOS]: Store SwapEntry as WsIndex. We can do this now since Flink is only used when the page is in a list (meaning it's not active, ie. free or zero), and non-active pages don't have a swap entry.

svn path=/trunk/; revision=45635
2010-02-20 14:47:23 +00:00
Dmitry Gorbachev b7b948cff5 Do not crash, use valid kernel PTE.
svn path=/trunk/; revision=45555
2010-02-10 13:56:54 +00:00
ReactOS Portable Systems Group b69d9da41a Implement paged pool expansion. We are now ready to switch to ARM pool 100%.
svn path=/trunk/; revision=44884
2010-01-02 16:10:11 +00:00
Stefan Ginsberg 82bd7858d1 - Add 'simple' implementation of MmDbgCopyMemory to read/write virtual memory in a somewhat safe way (still no support for physical memory). Properly implement KdpCopyMemoryChunks as a wrapper around MmDbgCopyMemory and make most of the remaining unsafe copies use it instead of RtlCopyMemory. This fixes most of the remaining crashes during KD debugging as WinDbg/KD relies on the kernel support to handle bad addresses.
- Stub out the remaining missing global variables for the debugger data block -- fixes some cases of WinDbg failures and gives it a chance to handle errors instead of failing on a NULL read. Several of these variables are for functionality we don't yet implement, so I tried to put them where they are "least wrong". Everything besides the MmLoadedUserImageList variable is left unitialized -- KD should mostly be able to handle this properly.
- Define correctly sized KDDEBUGGER_DATA64 for our kernel (needs to be done in a better way).

svn path=/trunk/; revision=43863
2009-10-31 01:02:35 +00:00
ReactOS Portable Systems Group d37d315a7e - Move some unimplemented Ex* Pool routines into ARM3's expool.c.
- Move some unimplemented Mm* Pool routines into ARM3's pool.c.


svn path=/trunk/; revision=43508
2009-10-16 00:28:33 +00:00
ReactOS Portable Systems Group 5ac42f871f - Add some simple code to handle paged pool page allocation and freeing in MiAllocatePoolPages and MiFreePoolPages.
- Allocation does not support paged pool expansion yet, so we're limited to the initial 4MB that are initialized on startup.
- This only tests the raw page allocation routines so touching memory will fail as we don't handle paged pool faults yet.
- Nothing excercises this code path other than some simple tests while booting.


svn path=/trunk/; revision=43503
2009-10-15 21:23:24 +00:00
ReactOS Portable Systems Group b20ae45a3f - Fix calculations error in the setup of the paged pool bitmap.
- Initialize the paged pool guarded mutex.
- Add helper routines ExLock/UnlockPool to either acquire the NPP QSL or the PP GM depending on the pool descriptor, instead of hardcoding the NPP QSL.
- Implement InitializePool for the PagedPool case.
- Now call InitializePool for PagedPool as well.


svn path=/trunk/; revision=43482
2009-10-15 17:23:21 +00:00
Stefan Ginsberg b42aaf88bb - Fix recursive spinlock acquisition in Mm caused by locking inconsistency between ARM3 and the old ReactOS Mm. The old Mm calls certain routines to modify PFN entries (lock, unlock, reference, dereference, etc) and acquires/releases the PFN lock inside those functions (which is extremely inefficient as you can't, for example, have to acquire/release the PFN lock twice to reference and lock the same page), while ARM3 synchronizes differently and holds the lock while calling those routines, resulting in a recursive lock attempt on MP (which works on UP because spinlocks are just IRQL raise/lower there). Move out locking from MmAllocPage, MmReference/DereferencePage and MmLock/UnlockPage to the callers to be consistent with ARM3.
- Add missing PFN locking to MmFreePagesFromMdl and MiAllocatePoolPages.
- Get rid of MmLockPageUnsafe and MmReferencePageUnsafe. The "safe" routines just forwarded to the unsafe versions -- call them directly instead. Remove unused MmAcquirePageListLock/MmReleasePageListLock.

svn path=/trunk/; revision=43240
2009-09-30 18:24:00 +00:00
ReactOS Portable Systems Group 916f5b12a6 - Initialize the value of MmBootImageSize in ARM3 now.
- Also fix its value such that it's PDE aligned -- this makes sure that we don't step on any of the boot loader's PDE mappings and can blow everything away later.
- Initialize the MmSystem/User/Probe Addresses in ARM3 as well (no functional change).
- Print out a lot more of the VA ranges in ARM3's Phase 2 initialization. Most of the VA space is now dumped out.
- Write out the code to initialize session space VA ranges
  - Image space, view space, working set space and pool space values are all calculated properly.
    - NT default sizes are used, without support for registry overrides (yet).
  - Also system view space is initialized and sized.
  - Code is heavily commented and explained for inquisitive minds.
- Define the paged pool start address, minimum/default size, and add some extra pool header asserts/definitions.
- Define MmPagedPoolInfo to keep track of all paged pool related information (start/end PTEs, VA ranges, allocation/free bitmaps, etc).
- Fixed a lot of comments and added some new ones to provide extra clarity.
- Implement MiBuildPagedPool. It has two jobs:
  - Build and create the shadow system page directory, which double-maps the System process' PDE.
    - More explenations are in the comments.
  - Define the paged pool region and size, and initialize MmPagedPoolInfo accordingly.
  - Create and setup the paged pool allocation and free bitmaps (again explained in the comments).
- There shouldn't be any real functional change yet due to this commit.
  - We need to create memory areas for session space and system view space otherwise the VA regions could get used by ReactOS instead.


svn path=/trunk/; revision=42148
2009-07-22 22:46:29 +00:00
ReactOS Portable Systems Group 1f4bf9180e - Get rid of debug spew used during testing.
svn path=/trunk/; revision=42124
2009-07-21 20:04:17 +00:00
ReactOS Portable Systems Group 1279c44e6c - Implement the rest of the nonpaged pool allocator, now with support for allocating pages in the nonpaged pool expansion area.
- This uses System PTEs, so if you're still not sick of the same old mantra -- optimizations to the former will help the latter.
    - Additionally, we should eventually implement a single-page SLIST for nonpaged pool pages, which will greately improve allocate/free of 1 page.
- As a reminder, this code isn't being used yet.

svn path=/trunk/; revision=41933
2009-07-12 13:02:05 +00:00
ReactOS Portable Systems Group 2aabdcc5fd - Implement a simple nonpaged pool page allocator and deallocator:
- Not actually used yet!
  - No support for expansion nonpaged pool yet.
  - No major optimizations.
    - We simply do the sane thing of trying to combine the next free allocation with the one we're about to make, on free.
      - If possible, we'll also check if the previous allocation was a free block, and we'll merge ourselves (and in the best case, whoever is following us) into that block instead.
  - No debug checks/validation.
- Write a couple lines of code to allocate and free up to 4096 pages (it will stop when it runs out).
  - This is to properly excercise this code on everyone's machines to make sure it works fine (it adds a little delay to booting, but only temporary).

svn path=/trunk/; revision=41859
2009-07-11 06:46:39 +00:00
ReactOS Portable Systems Group e5cc98dbc4 - Initialize nonpaged pool expansion system PTEs (MiInitializeSystemPtes):
- Just supports expansion space for now, not system space.
  - Very basic initialization of the accounting structures required.
  - Sets up the first two system PTE clusters for the space (the first contains the linkage, the second contains the sizage).
  - Called from MiInitializeArmPool.


svn path=/trunk/; revision=41528
2009-06-22 08:51:28 +00:00
ReactOS Portable Systems Group e980bd998f - Initialize the ARM pool (MiInitializeArmPool):
- Do some additional accounting to keep track of initial nonpaged pool range and size.
  - Create and initialize the free page lists, and free page entries.
  - Validate that the initial nonpaged pool address space was properly mapped.
  - Validate that the expansion nonpaged pool address space is unmapped, and prepare to map it.


svn path=/trunk/; revision=41527
2009-06-22 08:22:41 +00:00