Problem:
When fs_open() is called with FS_O_TRUNC, the FS backend
opens the underlying file via mp->fs->open() before the
truncate is attempted. If mp->fs->truncate() then fails,
the previous code cleared zfp->mp and returned right away,
which causes two issues:
- the backend's close hook is never invoked, so the
resources allocated during open (file slab entries,
internal caches, backend-specific structures such as
lfs_file, etc.) stay permanently allocated;
- a follow-up fs_close(zfp) cannot recover them either,
because zfp->mp has already been NULL'd and fs_close()
returns early.
The leak is reproducible on every backend (LittleFS, FAT,
...) and accumulates one slot per failed call until the
file slab is exhausted.
Solution:
Close the backend file explicitly in the truncate failure
path, before clearing zfp->mp, so the FS-specific close
hook can release everything it allocated during open().
If close itself fails, log the secondary error but still
return the original truncate error code, since that is
the root cause callers diagnose against.
Signed-off-by: Shuai Ma <malin719426@gmail.com>
- Fix to_write calculation to account for already written bytes
- Advance offset in loop to fetch correct block on each iteration
Signed-off-by: DEVER Emiel <emiel.dever@psicontrol.com>
ext2_fetch_direntry() trusted the on-disk de_rec_len and de_name_len,
and the lookup and readdir paths advanced traversal by an unvalidated
de_rec_len. A crafted ext2 image could trigger an out-of-bounds read
past the directory block buffer or a zero-progress loop in any path
that walks a directory.
Validate rec_len and name_len in the parser, and reject entries whose
header does not fit in the remaining block or whose rec_len would
cross the block boundary in each caller.
Signed-off-by: Flavio Ceolin <flavio@hubblenetwork.com>
Implement Native Simulator host filesystem mounting under Zephyr. This file
system allows to access host files and directories inside Zephyr running
under Native Simulator.
There is new Native Simulator executable parameter supported '-volume',
which can be used to mount any host directory in Zephyr (following Docker
volume mount syntax):
-volume=/host/dir:/dir
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
Since commit 37717b229f ("sys: util: rename Z_MIN Z_MAX Z_CLAMP to min
max and clamp"), <zephyr/sys/util.h> unconditionally defines function-
like macros named `min`, `max`, and `clamp` in the global namespace (in
C mode). util.h gets pulled in transitively by very broad headers,
including the POSIX layer's <pthread.h>, so any third-party C code that
uses these names as ordinary identifiers (e.g. XNNPACK's static `clamp`
helper and its public `clamp` struct field) fails to build as soon as
<pthread.h> is included.
Following the approach used by Linux, move the lowercase `min`, `max`,
`min3`, `max3`, and `clamp` macros (and their helpers) into a new
<zephyr/sys/minmax.h> header that has to be included explicitly by
source files that want them. util.h keeps the uppercase MIN/MAX/CLAMP,
so most code is unaffected; only the (much smaller) set of files that
actually use the lowercase variants needs to pick up the new include.
Fixes#107853.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
ffa_create_top() was opening files with FS_O_CREATE | FS_O_WRITE
only, which caused host applications like 'cat' to fail with
Permission denied when trying to read the file through FUSE. This
is because FAT filesystem explicitly checks the READ flag before
allowing read access.
Fix by replacing FS_O_WRITE with FS_O_RDWR so the file is opened
with both read and write access.
Signed-off-by: Surya Prakash T <suryat@aerlync.com>
Introduces K_MEM_SLAB_DEFINE_TYPE() and K_MEM_SLAB_DEFINE_STATIC_TYPE()
helpers to allow the user to declare slabs for types without having to
manually ensure the alignment is correct.
Manual slab alignment was very error-prone and this change fixes several
instances of misalignment that would be trapped by the undefined
behavior sanitizer when running on 64-bit targets.
Signed-off-by: Egill Sigurdur <egill@egill.xyz>
This commit fixes the bug by ensuring `to_write` is properly bounded.
It is now calculated as the minimum of the remaining requested bytes
(`nbytes - written`) and the remaining space in the current block
(`block_size - block_off`).
Fixes#106276
Signed-off-by: Ashirwad Paswan <ashi06712@gmail.com>
Increase the maximum allowed repeat count for read and write/erase
test commands from 10 to 10000. This allows for more accurate
measurements over longer test durations.
Signed-off-by: Tim Pambor <tim.pambor@codewrights.de>
Replaces usage of these deprecated macros with ones that support
fixed and mapped partition compatibles. Also includes an update to
hal_espressif which also (rightly or wrongly) has zephyr specific
code in it
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Add an early check whether FAT fs is already mounted similar to
littlefs mount command. This fixes a null pointer access if the mount
command is used twice.
Fixes#104777
Signed-off-by: Jonas Spinner <jonas.spinner@burkert.com>
Support for using v3 of the FUSE library instead of v2 was added in
b86e52bd6f
As time has been passing more distros are removing v2 support, and at
this point probably all include the v3 in their package repos.
So, let's switch the default to v3.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Let's avoid using off_t in the interface between the embedded and the
host side of the FUSE driver.
off_t is a C library defined type and as such can have a different sizes
between the host C library and the embedded C library.
While at it, let's do also the same with size_t. Even though size_t is
meant to be defined in stddef.h which is provided by the compiler, it is
possible to override it in the embedded side.
We just replace them with a 64 bit type for each (signed and unsigned
respectively matching the signedness of off_t and size_t).
There was a bug in the queue type which had the read/write second
parameter as off_t while it should have been size_t, which is now fixed
to be uint64_t.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Use SHELL_HELP macro for help strings to ensure consistency across
various shell modules.
This also documents the usage for each command.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Rather than manually checking the number of arguments and printing an
error message, let the shell subsytem handle the validation for us by
indicating all arguments as required.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
for checking the first two chars of a string
we don't need to get the full lenght, we can simply
just check the 2 chars, especially as we already check
the first one already.
in `fs_opendir` we only need to check the first char.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
NVS writes an entry by first programming the data area and then writing
the corresponding ATE. If a power loss occurs after the data has been
written but while the ATE is being programmed, and the remaining space
in the sector is only large enough for a single ATE, the ATE may end up
partially written.
1) Power loss while writing the last data ATE, leaving only space for
a delete ATE but no space for additional data entries:
[ data ][ data ][ erase ][ ATE ][ ATE ][ erase ] (sector end)
^^^
In this cases, nvs_startup() scans the sector and fails to find any
erased space usable for data writes, causing failed to mount.
[ data ][ data ][ erase ][ ATE (Invalid) ][ ATE ][ erase ]
^ ^
ate_wra
data_wra
The sector is logically exhausted andshould be closed and garbage
collected.
Detect this condition during startup. If no erased space exists after
the last ATE write, explicitly close the sector and trigger garbage
collection to restore a writable sector.
This completes the missing recovery path for power-loss scenarios
where a sector becomes effectively full due to ATE writes.
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
NVS ATE validation currently relies on CRC8 and coarse boundary checks
against the sector size. Because CRC8 has a non-negligible collision
probability (1/256), a corrupted ATE may occasionally pass CRC
verification while still containing invalid offset or length fields.
Instead of only validating that the referenced data fits within the
sector, this change additionally verifies that the end of the data
region (offset + len) does not exceed the physical position of the ATE
itself.
Due to the append-only write order of NVS, data is always written before
its corresponding ATE. Therefore, any ATE referencing data beyond its
own location is structurally invalid and must be rejected.
This tighter consistency check significantly reduces the likelihood of
accepting corrupted ATEs without introducing additional flash reads or
changing the on-flash format.
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
If fs_stat() is done on a mount point treat
it like it is a directory, instead of leaving it to
the implementation of the filesystem.
The fatfs for example will fail.
That is also the reason `fs cd` from the fs shell
fails, when trying it on a mountpoint with fatfs.
This also makes the behavior of fs_stat() similar to the one
of fs_readdir(). Especially as both functions use the
`struct fs_dirent` to return the informations.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
filter automount durring build.
Also add the automaticly enabled Kconfig option
CONFIG_FS_LITTLEFS_FSTAB_AUTOMOUNT, similar
to the fat fs and ext2 fs.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
If automount is enabled in the devicetree for
a fatfs, then enable the Kconfig option by default.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
In NVS, allocation table entries (ATEs) are written backwards within
each sector. Under delete-only or delete-heavy workloads, a sector may
contain only delete ATEs, causing the ATE write pointer to approach the
sector boundary.
Without an explicit boundary check, ATE writes may occur at offset 0 of
the current sector, allowing the write pointer to underflow into the
previous sector and corrupt unrelated data or metadata.
Fix this by disallowing ATE writes when the write pointer is at the
sector boundary. This ensures that ATE writes remain confined to the
current sector and prevents pointer underflow across sectors.
Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
Replace some manually-defined DT_COMPAT_<> Kconfig macro variables with
their automatically generated counterparts. In most cases, this is
straightforward as the manually defined macro is named identically to the
one generated by the build system.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
The initial implementation had a couple of flaws, including:
* Generally overcounting the free space by ATE_SIZE bytes per sector.
* Counting multiple ATEs of the same ID if placed in the same sector.
The correct behavior is to only count the most recent entry.
* When a sector was mostly or completely filled with small data sizes,
there was a correction term applied in the wrong location.
* The same correction term would be mistakenly applied when a sector
was filled with only delete ATEs, which resulted in undercounting
the free space in that sector.
This is addressed by rewriting the API according to a simple principle:
the total free space in a filesystem should equal the sum of free space
in every GC'd sector (minus 1 sector reserved for GC itself).
This should work because during garbage collection, ATEs are not moved
between sectors arbitrarily. Only one sector gets GC'd at a time and
every entry in sector N gets either removed or copied to sector N-1.
This means that any two valid entries that occupy a single sector would
still occupy a (different) single sector before and after the operation.
This property lets us simplify the total free space calculation by
considering one sector at a time and reapplying the calculation which
was just introduced for `zms_active_sector_free_space()`.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
The initial implementation had a flaw in it: when an active sector was
almost full, the naïve calculation could easily underflow and return a
negative value, which would be misinterpreted as an errno.
Rewrite this API to satisfy the following, reasonable expectations:
* Always return a non-negative value, provided ZMS is initialized.
* Always return 0 if no more data can be written into the sector
without triggering garbage collection.
* Never return a value less than the actual number of bytes that can
still be written into the sector.
This requires awareness of a few special cases in ZMS. These are now
captured in a helper function, which will be reused later.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Remove duplicated #include directives within the same
preprocessor scope across the Zephyr tree.
Duplicates inside different #ifdef branches are preserved
as they may be intentional.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
Avoid polluting every build that includes this Kconfig with EXT2
logging config unless the ext2 filesystem is actually enabled.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
For devices that need an erase before a write, ZMS do not verify that
the next available location in the open sector is filled with the
erase_value.
Fix this by adding a check at init.
Signed-off-by: Riadh Ghaddab <rghaddab@baylibre.com>
Lfs provide lfs_fs_gc function for some time now.
Function allow offloading of expensive block allocation scan.
Introduce lfs_fs_gc via api call fc_gc.
Signed-off-by: Maciej Zagrabski <mzi@trackunit.com>
Adds cast to int when ret is assigned error code returned by
functions that return int64_t. The cast has been added to indicate
that assignment with truncation is here intentional.
Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
REQUEST_QUEUE was passed as queue_num to virtio_init_virtqueues
where REQUEST_QUEUE+1 was required
Signed-off-by: Jakub Michalski <jmichalski@antmicro.com>
This reverts commit 77f19eb1b5.
This dependency was misplaced. See the parent commit.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Allow the ZMS API to optionally accept 64 bit IDs. A typedef `zms_id_t`
is added, so that the maximum ID width can be controlled using Kconfig.
The current ATE structure is already large enough that it is possible to
reserve 64 bits for IDs without increasing its total size (128 bits).
This makes the feature a natural, low footprint alternative to Settings,
for cases where the supported key namespace must be larger than 32 bit
but not arbitrarily large.
The ATE format does have to be altered to accommodate larger IDs, but
the default "32 bit" format is left as is. Now, the `struct zms_ate`
describes one of two supported formats, selected by an `#if` condition.
In the future, it may be possible to support multiple ATE formats at
runtime, in which case the structure can be turned into a union.
In the new, "64 bit" ATEs, the `offset` and `metadata` fields are moved
into a union, because they are found to be mutually exclusive. With the
old format, the same fields are in different locations, but one of them
always gets filled with a dummy value, depending on the given ATE type.
To cover both cases, a `memset` is used, which should be optimized away
by the compiler when appropriate.
The only limitation is that the new ATE format has no room for data CRC,
but an alternative integrity check can be implemented by the caller.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Support using FUSE v3, instead of FUSE v2.
FUSE v3 has been out for a couple of years now, so most distributions
have it.
But note the FUSE library (and its v3 branch) is currently unmaintained,
and older distributions do not ship it.
Some Linux distros have transitioned their packages away from fuse2.
Which results in less atention in these distros to fuse2, and therefore
a higher likelyhood that there would be distribution issues with the
corresponding packages.
There is also some likelihood the fuse2 packages may be eventually
droped by some of these.
So let's support either version, with a kconfig adapting to either API
and their quirks.
Note that both the fuse2 and fuse3 library code have quite a few
sideeffects on the process that uses it.
By now we continue defaulting to fuse2 as it is the most common.
But users can select to use the v3 if they have an issue w v2 or want
to start trying out using v3.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Avoid a possible race between the FUSE thread and the Zephyr threads
during init.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
FUSE_INCLUDE_DIRS should be passed to the native_simulator build.
If there is several fuse libraries installed, we need to ensure the
bottom side of the driver is built with the correct include paths.
(These includes are irrelevant for the Zephyr side build)
Support having a list of include paths instead of single one.
Support having a list of libraries to link to instead of a single one.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Fix deleting a key that doesn't exist from writing a new entry to the
filesystem when `CONFIG_ZMS_NO_DOUBLE_WRITE` and
`CONFIG_ZMS_LOOKUP_CACHE` are enabled.
Signed-off-by: Jordan Yates <jordan@embeint.com>
List the file size helps in developer debugging experience. Provide
a config to disable new behavior in case any users depended on
command output to be a list.
Signed-off-by: Utsav Munendra <utsavm@meta.com>
