Complete rewrite reserving and releasing of System PTEs.
The previous algorithm, in a nutshell, worked as follows:
- PTE clusters are in a singly linked list, ordered by their base address.
- All PTEs in the clusters are zeroed (except for cluster list bookkeeping).
- Upon reservation: Walk the list to get the first cluster that's large enough, cut the requested amount of PTEs off its tail and return them.
- Upon release: Create a new cluster using the PTEs to release, and merge it together with possible adjacent clusters.
Problems with the previous algorithm:
- While the idea is that all PTEs in clusters are zeroed, which requesters rely on, cluster bookkeeping isn't zeroed on merges.
The side effect of this was that PTEs that weren't really zeroed were randomly delivered to requesters.
- 99% of the time, allocations are serviced using the first cluster in the list, which is virtually always the first suitable cluster.
This is so because the ordering is based on the base address of the clusters, and allocations are serviced using the cluster tail.
Because the first cluster starts out as the whole pool, and the pool is quite sizable, it can deal with virtually allocations.. for a while.
- A corollary of the previous point is *massive fragmentation* because: as long as an allocation isn't released back into the pool,
the space of previous allocations that have been released isn't reused because the first cluster can't suck them up, and enough allocations remain in use.
- The combined effect of the previous two points: a first cluster that effectively shrinks mostly, with small clusters forming behind it.
Once the first cluster has shrunk far enough (which of course takes a long time), 90% of the space may still be free, scattered in mostly small clusters.
This would make decent sized allocations fail because of the heavy fragmentation.
- An implementation detail that caused the head of the list to be treated as a genuine cluster when the first cluster in the list was too small.
The algorithm (as explained above) made this case quite unlikely until your system has been running for a while, after which it could happily
corrupt list heads of other pools, depending on where the list head is with respect to its own pool.
Empirically obtained data revealed that after just *booting to the desktop*, the pool for System Pte Space entries
contained roughly 70 (unusable) clusters, blocking 15 to 20% of the pool. These figures increased to roughly 100
clusters and 30 to 35% after opening a foxy browser and using it to visit a mathematically inspired search engine.
The same data also showed that over 95% of allocations requested just a single PTE, and a noticable allocation spike
also occured in the range of 65-128 PTEs. It should be clear optimizing for small allocations is a good idea,
and preferably encourage reuse the same PTEs for such allocations.
And the new algorithm was born:
- PTE clusters are in a singly linked list, ordered by increasing cluster size.
- All PTEs in the clusters are zeroed (except for cluster list bookkeeping) .. really this time!
- Upon reservation: Walk the list to get the first cluster that's large enough, cut the requested amount of PTEs off its tail and return them.
- Upon release: Create a new cluster using the PTEs to release, and merge it together with possible adjacent clusters.
- Both in the reservation and release actions, insertions into the list preserve the increasing cluster size order.
Empirically obtained data now revealed that after just booting to the desktop, the pool for System Pte Space entries
contained exactly 2 clusters. This increased to 10 clusters after some minor internet browsing and watching a 5 minute video using a media player.
svn path=/trunk/; revision=50347
In GreCreateBitmapEx handle allocation failure in the rle hack path and set LastError, when failed to allocate bitmap bits.
svn path=/trunk/; revision=50344
- In BitmapFormat, allow intermediate bpp values, use ULONG as parameter type, instead of WORD and DWORD
- In NtGdiCreateBitmap get the real bpp value from the gajBitsPerFormat array
- Add back check of too large nWidth (needed to make sure, cjWidthBytes didn't overflow)
- Merge all parameter checks
- Check cPlanes and cBitsPixel paramters explicitly
- Use GreCreateBitmapEx
- Remove BITMAP_GetRealBitsPixel
svn path=/trunk/; revision=50343
Improve NtGdiStretchDIBitsInternal, use _SEH2_YIELT instead of saving an NTSTATUS and handle the fast path in place instead of setting a BOOL variable. Fixes warnings about uninitialized variables.
svn path=/trunk/; revision=50342
- Move EFLOAT handling in seperate file, its x86 specific
- Implement CombineTransform fully in usermode instead of forwarding to NtGdiCombineTransform
- Implement MatrixToXForm
- Implement GdiTransformPoints
svn path=/trunk/; revision=50330
- Use if, instead of switch to handle flags in XFORMOBJ_bXformFixPoints
- Define XFORMOBJ to EXFORMOBJ, a stack object that links to the matrix
- There is no spoon.
svn path=/trunk/; revision=50307
- EhciDefferedRoutine: Only free Mdl if one was created when the request was added to the queue.
- Dont assume that three transfer descriptors need to be freed, use a while moving to each linked and free instead.
- Yabba Dabba Doo added to increase size of message. :)
svn path=/trunk/; revision=50304
- Implement URB_FUNCTION_SELECT_INTERFACE.
- Add missing IoMarkIrpPending before calling SubmitControlRequest.
- Now use BuildSetupPacketFromURB for creating the CtrlSetup data structure.
- When selecting configuration, make the ConfigurationHandle from the related UsbDevice instead of always the RootHub device.
- Fix typo that caused all pipes to be marked as interrupt type.
- Change CompletePendingURBRequest to only complete one SCE request Irp.
svn path=/trunk/; revision=50303
- Fix bug in InitializeUsbDevice that caused endpoints to be duplicated in internal config structure.
- Fix implementation of UsbRemoveDevice.
- Implement SetDeviceHandleData and RestoreUsbDevice.
- SubmitControlTransfer: Not all control request need data, in which case dont create a MDL and a PID_CODE_IN_TOKEN TD Descriptor.
- Fix bug in ReleaseMemory that caused memory not to be marked as free.
- Fix incorrect memory header size that resulted in memory corruption.
svn path=/trunk/; revision=50302
- Simplify error checks, by reordering code in NtGdiScaleViewportExtEx
- Simplify exception handling, use _SEH2_YIELD
- Change function order
- Minor code improvements
svn path=/trunk/; revision=50283
- Access the DC member directly instead of using IntGdiGetDCOrg
- Remove unused IntGdiGetDCOrg, GdiSetDCOrg, GdiGetDCOrgEx
svn path=/trunk/; revision=50281
Improve new printf implementation:
- remove duplicated code and implement a generic function for all (v)s(w)(n)printf
- don't call _flsbuf for string streams, while this works on windows, it doesn't work correctly on reactos (bug!)
- Fix return error codes
svn path=/trunk/; revision=50273
