Shadow Stack is one of the capabilities provided by Intel Control-flow
Enforcement Technology (CET), aimed at defending against Return Oriented
Programming.
This patch enables it for x86_64 (32-bit support coming in future
patches):
- Add relevant Kconfigs;
- Shadow stacks should live in specially defined memory pages, so
gen_mmu.py was updated to allow that;
- A new macro, Z_X86_SHADOW_STACK_DEFINE, added to define the area
for a shadow stack;
- A new function, z_x86_thread_attach_shadow_stack(), added to
attach a shadow stack to a never started thread;
- locore.S changed to enable/disable shadow stack when a thread
using it comes in/out of execution.
As not all threads are currently shadow stack capable, threads that do
not use it will still run with shadow stack disabled. Ideally, at some
point in the future, all threads would use the shadow stack, so no need
to disable it at all.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Add code to enable it and sprinkle `endbr64` on asm code, where needed.
Namely, IRQs and excepts entrypoints.
Finally, tests added to ensure IBT behaves sanely.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Indirect Branch Tracking (IBT) is one of the capabilities provided by
Intel Control-flow Enforcement Technology (CET), aimed at defending
against Jump/Call Oriented Programming.
This patch enables it for x86 (32-bit, 64-bit support coming in future
patches):
- Add relevant Kconfigs (everything is behind X86_CET);
- Code to enable it;
- Enable compiler flags to enable it;
- Add `endbr32` instructions to asm code, where needed.
Points in the code where an indirect branch is expected to land need
special instructions that tell the CPU they are valid indirect branch
targets. Those are added by the compiler, so toolchain support is
necessary. Note that any code added to the final ELF also need those
markers, such as libc or libgcc.
Finally, tests added to ensure IBT behaves sanely.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
This adds exception handling of control protection exception
in fatal code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Fix an issue where 1 vector is being requested when MSI-X is
enabled. The previous logic always assumed the PCIE device has
only fixed or single MSI when we are requesting 1 vector, which
is not entirely correct. So if there is no vector allocated
already, try to allocate one.
Fixes#93319
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_pcie_msi_vectors_allocate() has a return type of uint8_t.
One of the error path returns -1 which would result in 255
being returned. So fix that by returning 0 instead, as there is
no vector being allocated anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
GCC 14.3 will happily delete any code that appears before
__builtin_unreachable that isn't separated with an obvious branch. That
includes __asm__ statements, even those which generate traps.
The failure case that I debugged was on x86 in
z_check_stack_sentinel. There is a store to restore the sentinel to the
correct value just before the ARCH_EXCEPT, and that macro emits 'int $32'
followed by CODE_UNREACHABLE. Because the compiler didn't understand that
ARCH_EXCEPT was changing execution flow, it decided that the sentinel
restoring store 'couldn't' be reached and elided it.
I added the "memory" clobber to the asm statement in ARCH_EXCEPT before
CODE_UNREACHABLE to enforce that all pending store operations be performed
before the asm statement occurs. This ensures that they are not deleted by
the compiler.
I think this might be a GCC bug. The GCC documentation explicitly documents
that asm statements which change the flow of control should be followed by
__builtin_unreachable.
Signed-off-by: Keith Packard <keithp@keithp.com>
The intent of arch_coprocessors_disable() is to replace
arch_float_disable() in halt_thread() for the FPU will not
always be the only coprocessor that will need to be disabled.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
instead of depending on logging for exceptions and fatal messages, use
macro and dump exceptions if printk is enable with no logging.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is an X86 specific option and should not appear as generic debug
option.
Fixeszephyrproject-rtos/zephyr#52929
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Instead of both checking CONFIG_BOARD_QEMU_X86 and
CONFIG_BOARD_QEMU_X86_64, simply check for CONFIG_QEMU_TARGET.
This also helps qemu_x86_tiny, qemu_x86_lakemont and any
future/downstream QEMU targets to keep things simple.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Stack walking on x86_64 can run with compiler optimization and
does not need CONFIG_NO_OPTIMIZATIONS as long as frame pointers
are not omitted. So remove the "depends on" for x86_64
from CONFIG_ARCH_HAS_STACKWALK.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This ASSERT fails because the comparison is made between the physical
APIC ID and the virtual CPU's LAPIC ID. If CPU-1 is assigned to ACRN,
it considers that CPU as its CPU-0.
Signed-off-by: Anisetti Avinash Krishna <anisetti.avinash.krishna@intel.com>
There is no need for kernel/internal/smp.h as SOF does not call
z_sched_ipi(). Actually... git log over there has no mention of
z_sched_ipi() anywhere, just arch_sched_ipi().
And include <ksched.h> for source using z_sched_ipi() since
they are using scheduling functions, and would be the correct
file to include.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The current_fp field in the z_kernel structure is only used
by 32-bit x86 (which does not support SMP). As such, it should
reside in the arch specific of section of _kernel.cpus[0].
This also changes the name of 'current_fp' to 'fpu_owner' to
be more consistent with other architectures.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
CONFIG_DYNAMIC_BOOTARGS currently depends on CONFIG_MULTIBOOT_INFO
and CONFIG_BUILD_OUTPUT_EFI which are x86 specific.
This change removes those dependencies so that another architecture
can use CONFIG_DYNAMIC_BOOTARGS without the x86 specifc features.
Signed-off-by: William Tambe <williamt@cadence.com>
Adds support for directed IPIs to x86/intel64. Use of direct IPIs
can further reduce the number of schedule IPIs sent and processed
in a system. Fewer IPI related ISRs mean that ...
1. Application code is interrupted less frequently
2. Lower likelihood of scheduler spinlock contention
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Enable support for warm reboot for x86 arch as per the Reset Control
Register description.
Signed-off-by: Anisetti Avinash Krishna <anisetti.avinash.krishna@intel.com>
This change initializes was_valid_access in the case that
CONFIG_X86_KPTI is selected. Previously, the variable was
causing an uninitialized variable warning that would be
escalated to an error when run with twister.
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
`CONFIG_MP_NUM_CPUS` has been deprecated for more than 2
releases, it's time to remove it.
Updated all usage of `CONFIG_MP_NUM_CPUS` to
`CONFIG_MP_MAX_NUM_CPUS`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Do not use SYS_INIT for initializing irq_offload when enabled, instead
using a new interface that is called during the boot process for all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a new call for initializing cache on architectures that need that.
Avoid using SYS_INIT for this and instead call the hook in a fixed place
and run if implemented.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
On 32-bit x86, it was supposed to print the first argument to
the function during stack trace. However, it only works when
code optimizations are totally disabled (i.e. -O0). As such,
printing the args is not meaningful to aid with debugging.
So change it to simply print the function address, the same
as x86 64-bit.
Also, since unwind_stack() has exactly one caller, make it
ALWAYS_INLINE to skip a function call.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce soc and board hooks to replace arch specific code
and replace usages of SYS_INIT for platform initialization.
include/zephyr/platform/hooks.h introduces the hooks to be implemented
by boards and SoCs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add bootargs support to zefi. This implements
get_bootargs() when both efi and bootargs are
selected in config.
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Add bootargs support for multiboot. This
implements get_bootargs() when multiboot and
bootargs are selected in config.
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This implements arch_thread_priv_stack_space_get() so this can
be used to figure out how much privileged stack space is used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to initialize the privileged stack for
each thread during thread initialization. This prevents
information leaking if the thread stack is reused, and
also aids in calculating stack space usage during system
calls.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Only set the privileged stack pointer for thread stacks, but
nullify the pointer for kernel-only stacks, as these stacks
do not have the reserved space. The psp pointer may point to
arbitrary memory in this case if stack is not big enough.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Currently it supports `esf` based unwinding only.
Then, update the exception stack unwinding to use
`arch_stack_walk()`, and update the Kconfigs & testcase
accordingly.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Relocate stack unwind backends from `arch/` to perf's
`backends/` folder, just like logging/shell/..
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement stack trace function for x86_32 arch, that get required
thread register values and unwind stack with it.
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
Implement stack trace function for x86_64 arch, that get required
thread register values and unwind stack with it.
Originally-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
custom arch_cpu_idle and arch_cpu_atomic_idle implementation was done
differently on different architectures. riscv implemented those as weak
symbols, xtensa used a kconfig and all other architectures did not
really care, but this was a global kconfig that should apply to all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When CONFIG_X86_EXCEPTION_STACK_TRACE is enabled, also forcibly
enable CONFIG_THREAD_STACK_INFO. Without the thread stack info,
it is possible the stack unwinding would go out of the thread
stack and into unknown memory, resulting in hard fault.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series of move memory management related
stuff out of Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Also any demand paging and page frame related bits are
renamed.
This is part of a series to move memory management related
stuff out of the Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is part of a series to move memory management related
stuff from Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
