Commit graph

46 commits

Author SHA1 Message Date
Pierre Schweitzer ea76ef8477 [0.4.9] cherry-pick [FASTFAT] Properly check for buffer size on dir enumeration.
This avoids improper returns when multiple entries are requested
and, in rare circumstances, entries not being seen by umode.

CORE-14699

(cherry picked from commit 01e2877161)
2018-06-07 20:57:47 +02:00
Pierre Schweitzer 98e87bcafd [0.4.9] cherry-pick [FASTFAT] Lock DirResource on flush in case we're updating entry
CORE-14660

(cherry picked from commit 9f3169b295)
2018-05-28 19:44:27 +02:00
Pierre Schweitzer 367cb1fff3 [0.4.9] cherry-pick [FASTFAT] Lock DirResource when modifying an entry on disk.
Likely not optimal, but fixes some races conditions where
the directory is uninit in the middle of the write.

(cherry picked from commit fc788cf2fd)
2018-05-27 12:58:00 +02:00
Pierre Schweitzer c71d198fd8 [0.4.9] cherry-pick [FASTFAT] Initialize cache if neded for changing volume label.
CORE-14654

(cherry picked from commit 47b9db9d02)
2018-05-26 14:13:17 +02:00
Pierre Schweitzer bf2d787a94 [0.4.9] cherry-pick [FASTFAT] Properly handle IRPs that can wait and these that cannot.
CORE-14634

(cherry picked from commit b4363068d1)
2018-05-26 14:12:36 +02:00
Pierre Schweitzer 07a13ca67c [0.4.9] cherry-pick [FASTFAT] Properly queue pending write IRPs.
Addendum to 30b836b

(cherry picked from commit e9e14be661)
2018-05-26 14:12:29 +02:00
Pierre Schweitzer 4af12f40ca [0.4.9] cherry-pick [FASTFAT] When extending a file, also lock the DirResource.
This avoids race conditions under high IOs and thus corruption on
the FS, or assertions failures in the kernel.
Easily triggered by building ReactOS on ReactOS ;-).

(cherry picked from commit 30b836bf3f)
2018-05-26 14:12:22 +02:00
Pierre Schweitzer a10f6c7a0c
[FASTFAT] Fail if allocating the stream FO fails. 2018-05-20 13:53:43 +02:00
Pierre Schweitzer 7283f3cfd9
[FASTFAT] Addendum to d69f318 2018-05-19 13:46:34 +02:00
Pierre Schweitzer d69f31848e
[FASTFAT] Misc. fixes spotted by Thomas. 2018-05-19 10:07:14 +02:00
Pierre Schweitzer 7c01587680
[FASTFAT] Completely rewrite support for dirty volumes.
Until now, our support for dirty volumes was totally broken
to a point where, on FAT32 volume, the dirty couldn't even
be written nor read from the disk.

This commit totally rewrites its handling, for both FAT16 and FAT32
so that it's now fully functionnal. Furthermore, it also gets
totally compatible with our vfatlib, and thus, autochk.
Now, on mount, FastFAT will check if the volume is dirty or not, and
autochk will be able to ask for a repair if dirty. vfatlib will
repair the volume and remove the dirty bit. So that, on next
reboot, the volume will be mounted clean.

As a reminder, the dirty bit is set immediately after mounting
the volume, so that, if you crash or have a powercut, autochk
will always attempt to repair your volume (with more or less,
that's FAT!).

If you want to experience without breaking your FAT volume,
just boot, open a cmd prompt and type: fsutil dirty set c:
and reboot!

CORE-13758
CORE-13760
CORE-13759
2018-05-18 23:05:05 +02:00
Pierre Schweitzer efa75dd5b2
[FASTFAT] Allow locking system volume on boot.
This is a hack, and totally not the default behavior.
But it will allow autochk locking the boot volume in
order to verify it on boot, in case it would have
been improperly dismounted.

CORE-13759
2018-05-18 23:05:05 +02:00
Pierre Schweitzer a837694518
[FASTFAT] Uninit directory cache on last handle close.
CORE-14629
CORE-14557
2018-05-18 14:51:24 +02:00
Pierre Schweitzer 3c3ebe3320
[FASTFAT] Only initialize directory cache on use.
This avoids initializing cache directly on directory
open/create.
The advantage is we reduce the load on cache manager
and on memory manager by avoiding creating everytime
a stream file object, and initializing cache for it.

This will avoid initializing cache for started
applications 'current directory' which is just opened
for having a valid handle but no read/write is performed
in it, by default.

This is a step forward for autochk.

CORE-14629
2018-05-18 14:09:30 +02:00
Pierre Schweitzer 5d743b7bd7
[FASTFAT] When attaching our FCB to a FO, also set the VPB.
Spotted by Alex.
2018-05-18 10:30:52 +02:00
Pierre Schweitzer d37280efaa
[FASTFAT] Deny dismounting system volume or a volume with a pagefile 2018-05-16 21:45:35 +02:00
Pierre Schweitzer 38078335b9
[FASTFAT] Implement support for stack overflow in read operations.
Before performing a read operation, FastFAT driver will
attempt to compute whether it would run out of stack
during the operation. If so, instead of attempting the
operation in the current thread, it will post the read
request to the overflow thread.

This should help with the regressions brought in by
94ead99e0c.

CORE-14601
2018-05-12 12:03:18 +02:00
Pierre Schweitzer 2a7d16727a
[FASTFAT] Implement write IOs defering.
Before any write operation that would involve caching, ask
the cache controler whether writing would make it exceed its memory
consumption. If so, queue the write operation for later execution.

In case the write operation can wait, then, the FSD operation will be
halted until the write is allowed.

I could test it successfully by copying huge files from a FAT volume to
another. The write is halted until some portions of the file is written
to the disk.
I could also properly install Qt (SDK) on ReactOS with this and less than 1GB RAM:
- https://www.heisspiter.net/~Pierre/rostests/Qt_OS.png
- https://www.heisspiter.net/~Pierre/rostests/Qt_OS2.png

CORE-12081
CORE-14582
CORE-14313
2018-04-29 20:42:53 +02:00
Pierre Schweitzer 00e843b1ed
[FASTFAT] Reduce the number of local declaration. 2018-04-29 12:33:53 +02:00
Pierre Schweitzer fd8197645f
[FASTFAT] Use KdRosRegisterCliCallback() to register in KDBG. 2018-04-29 12:23:18 +02:00
Pierre Schweitzer c5a35ecd37 [FASTFAT] Introduce a KDBG extension.
This is a PoC of what it's possible to realize thanks to an
already existing hack in ntoskrnl :-).
With this extension, on the kdb:> prompt, you're able to type
in commands that will be handled by the FastFAT driver and not
by the kernel, allowing internal debug, not possible otherwise.

So far, three commands exist:
- ?fat.vols: lists all the mounted volumes by FastFAT
- ?fat.files: lists all the files on a specific volume (with their attributes)
- ?fat.setdbgfile: allows watching on specifics files lifetime

This is obviously only the begin and could be greatly improved.

For instance, this is what allowed to debug CORE-14557
2018-04-29 12:15:11 +02:00
Pierre Schweitzer 94ead99e0c [FASTFAT] Don't leak directories FILE_OBJECT, FCB and cache entries.
Once a directory is crossed (opened or a child is opened), associated
FCB structure is created in FastFAT, but also a stream FO for caching.
Up to now, due to an extra reference taken by the stream file object,
even when the directory was no longer used, the directory was kept in
memory: the FCB was never deleted, the file object was never dereferenced,
and the cache never released.

The immediate effect of this bug is that our FAT driver was leaking every
directory that was used affecting the whole OS situation. In case of
directories intensive operation (like extraction the ReactOS source code
in ReactOS ;-)), we were just killin the whole OS RAM without any way to
release it and recover.

The other side effects: IOs were faster as half of the FS was always
permanant in RAM.

This commit fixes the issue by forcing the FSD to release the FO,
and the cache when a directory is no longer used, leading to its
destruction in RAM.
Downside: on IO intensive operation, expect slowdowns, obviously,
there's less caching now. But more efficient!

CORE-14557
2018-04-28 18:33:14 +02:00
Pierre Schweitzer 315f9a13d4
[FASTFAT] When allocating a FCB, set its node type code.
We set the same value as MS FastFAT driver for consistency.
2018-04-27 18:56:31 +02:00
Pierre Schweitzer 69f51d1533
[FASTFAT] Drop read ahead locking routines in favor of lazy writer routines.
This avoids code duplication while being consistent with MS FastFAT.
2018-02-17 13:49:15 +01:00
Serge Gautherie d6fab4158e [FASTFAT] Improve code style in addition to 8294118174. 2018-01-07 19:29:23 +01:00
Pierre Schweitzer f0a1a5c773
[FASTFAT] Notify on file size change in FileAllocationInformation/FileEndOfFileInformation 2018-01-07 15:29:33 +01:00
Pierre Schweitzer 8294118174
[FASTFAT] Add a wrapper around FsRtlNotifyFullReportChange 2018-01-07 14:16:31 +01:00
Pierre Schweitzer 2a36b632ef
[FASTFAT] When a file gets overwritten, notify about the changes 2018-01-07 11:44:25 +01:00
Pierre Schweitzer ffeb3f070b
[FASTFAT] When overwriting a file, also update its write time 2018-01-07 11:34:41 +01:00
Pierre Schweitzer 0eacb0b1ef
[FASTFAT] When dealing with attributes on IRP_MJ_CREATE, filter user input to ensure we have only attributes we can handle.
This applies to files newly created, or for files overwritten where attributes are reset/modified.
2018-01-06 22:02:10 +01:00
Pierre Schweitzer 2b4d5c5cff
[FASTFAT] When overwriting a file with FILE_OVERWRITE*, don't erase previous file attributes, only update them.
Keep erasing them in case the file gets overwritten with FILE_SUPERSEDE.

CORE-14158
2018-01-01 14:29:15 +01:00
Pierre Schweitzer f3fecf92c1
[FASTFAT] On file overwrite, deny it if file as HIDDEN/SYSTEM flag and caller doesn't set them.
CORE-14158
2018-01-01 12:13:46 +01:00
Pierre Schweitzer 453e42beb5
[FASTFAT] Revert bd1e7bf: our VPB swapout implementation in FastFAT is buggy and
leaves volumes in half-(dis)mounted state and thus they get remounted while not
completely dismounted.
This can lead to major breakage and data corruption.
This requires deeper fixes (let's just drop that driver!).

CORE-14124
CORE-14126
CORE-14133
2017-12-25 11:24:13 +01:00
Pierre Schweitzer bd1e7bf85e
[FASFAT] Enable FAT volume dismount using VPB swapout 2017-12-17 23:29:33 +01:00
Pierre Schweitzer dcd95c1a85
[FASTFAT] Add a comment (and an ASSERT) in VfatCreateFile() that shows that my stats code is broken... 2017-12-17 18:25:43 +01:00
Pierre Schweitzer 64bc96558e
[FASTFAT] While closing FCBs on dismount, release from tail to head and not the contrary.
It fixes assertion failure in vfatDestroyFCB() where we would have release parent before child.
This is still not perfect, but less bug prone...

With this commits (and ENABLE_SWAPOUT defined), ReactOS seems to unmount FAT volumes quite nice! :-)
(Tried with fsutil volume dismount X:)
2017-12-17 18:24:01 +01:00
Pierre Schweitzer de03686148
[FASTFAT] Fix a handle count leak on volume close. This can prevent locking a volume! 2017-12-17 18:24:01 +01:00
Pierre Schweitzer 27773dbb3f
[FASTFAT] When not able to lock a volume due to open handles, print open handles 2017-12-17 18:24:01 +01:00
Pierre Schweitzer 18d6584da4
[FASTFAT] Fix FastFAT not returning short name for FAT volumes in FileBothDirectoryInformation case
This is likely due to a copy paste error where long name was copied twice and short never.
Fun fact: this was not affecting FATX volumes
Fun fact2: this was defeating a buffer overflow check and thus was allowing buffer overflow!

CORE-14088
2017-12-09 21:22:55 +01:00
Pierre Schweitzer bddb0db469
[FASTFAT] Finally drop the FileNameInformation class for directories.
It makes no sense for them and shouldn't be implemented.
It's an addendum to 9f3c801.
2017-12-09 14:37:07 +01:00
Pierre Schweitzer 28343c6c0c
[FASTFAT] Fix size checking in VfatGetFileNameInformation() 2017-12-09 12:45:16 +01:00
Pierre Schweitzer 9f3c80193e
[FASTFAT] Don't mix FileNameInformation and FileNamesInformation (and support the later).
This commit fixes weird behavior in our FastFAT implementation. It was mixing two classes:
FileNameInformation and FileNamesInformation. It was handling FileNameInformation like
FileNamesInformation and was filling buffer with FILE_NAMES_INFORMATION structure instead
of FILE_NAME_INFORMATION structure (how many things did that break?!).
Also, it wasn't implementing the FileNamesInformation class at all. This is required by
ntdll_winetest:directory which doesn't expect it to fail and thus, attempts to read
never filled in memory.

This commit fixes the winetest crash, and may fix other weird FS behavior.

CORE-13367
2017-12-09 12:37:34 +01:00
Pierre Schweitzer e95751ab78
[FASTFAT] Calculate more accurately buffer length in VfatGetAllInformation(). 2017-12-09 10:59:37 +01:00
Pierre Schweitzer 52f0726817
[FASTFAT] Allow partial returns on directory info query for first entry.
This mimics what MS FastFAT does and fixes (a bit) ntdll_winetest:directory.
It still crashes, but go farther.

CORE-13367
2017-12-09 10:58:19 +01:00
Pierre Schweitzer 8503842309
[FASTFAT] More FS statistics 2017-11-21 22:18:11 +01:00
Colin Finck c2c66aff7d Git conversion: Make reactos the root directory, move rosapps, rostests, wallpapers into modules, and delete rossubsys. 2017-10-03 07:45:34 +00:00