Fix the way we read the current l1 page table set in the mmu.
We use it check if the current page table is different from the
running thread.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
We use ptevaddr_get to know the address the page table is set.
It happens that for this use, we should just use the ptebase field
of ptevaddr register.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The ring field in the pte mapping a memory partition should
be based in the partition attribute and not in the domain
asid that is used only to set the ASID (in RASID) position for
user ring.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add support for relocating local symbols, as specified in the
.rela.dyn section.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Andy Ross re-implementation of MMU layer with some subtle changes,
like re-using existent macros, fix page table cache property when
direct mapping it in TLB.
From Andy's original commit message:
This is a reworked MMU layer, sitting cleanly below the page table
handling in the OS. Notable differences from the original work:
+ Significantly smaller code and simpler API (just three functions to
be called from the OS/userspace/ptable layer).
+ Big README-MMU document containing my learnings over the process, so
hopefully fewer people need to go through this in the future.
+ No TLB flushing needed. Clean separation of ASIDs, just requires
that the upper levels match the ASID to the L1 page table page
consistently.
+ Vector mapping is done with a 4k page and not a 4M page, leading to
much more flexibility with hardware memory layout. The original
scheme required that the 4M region containing vecbase be mapped
virtually to a location other than the hardware address, which makes
confusing linkage with call0 and difficult initialization
constraints where the exception vectors run at different addresses
before and after MMU setup (effectively forcing them to be PIC
code).
+ More provably correct initialization, all MMU changes happen in a
single asm block with no memory accesses which would generate a
refill.
Signed-off-by: Andy Ross <andyross@google.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This file doesn't need the asm2 header, and the preprocessor logic
around whether to include the backtrace header is needless (all it
does is declare functions).
Signed-off-by: Andy Ross <andyross@google.com>
Add a Kconfig option (and build warning) alerting about the problem
of the kernel spilling register in behave of the userspace.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Use thread local storage to check whether or not a thread is running
in user mode. This allows to use threadptr to properly support tls.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Only clear the user stack to 0xAA if CONFIG_INIT_STACKS is
enabled. Otherwise, write 0x00 as if the stack is in BSS.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
During arch_kernel_init(), the interrupt stack is being
initialized. However, if the current in-use stack is
the interrupt stack, it would wipe all the data up to
that point in stack, and might result in crash. So skip
initializing the interrupt stack if the current stack
pointer is within the boundary of interrupt stack.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We need to use the mmu spin lock when invalidating the cache during
tlb shootdown, otherwise it is possible that this happens when another
thread is updating the page tables.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When the target has a cache way size (cache size / cache wasy) bigger
than the page size we have cache aliasing, since the number of bits
required by the cache index is bigger than the number of bits in the page
offset.
To avoid this problem we flush the whole cache on context switch or when
the current page table is changed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Swap page tables at exit of exception handler if we are going to
be restored to another thread context. Or else we would be using
the outgoing thread's page tables which is not going to work
correctly due to mapping and permissions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When adding a thread to a memory domain, we need to also update
the mapped page table if it is the current running thread on
the same CPU. If it's not on the same CPU, we need to notify
the other CPUs in case the thread is running in one of them.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
After changing content of page table(s), it is needed to notify
the other CPUs that the page table(s) have been changed so they
can do the necessary steps to use the updated version. Note that
the actual way to send IPI is SoC specific as Xtensa does not
have a common way to do this at the moment.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When kernel OOPS is raised, we need to actually go through
the process of terminating the offending thread, instead of
simply printing the stack and continue running. This change
employs similar mechanism to xtensa_arch_except() to use
illegal instruction to raise hardware exception, and going
through the fatal exception path.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Trigger exception on Xtensa requires kernel privileges. Add
a new syscall that is used when ARCH_EXCEPT is invoked from userspace.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This extracts the printing of fatal exception information into
its own function to declutter xtensa_excint1_c().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
There are known exceptions which are not fatal, and we need to
handle them properly by returning to the fixup addresses as
indicated. This adds the code necessary in the exception
handler for this situation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
When MMU is enabled, we need some scratch registers to preload
page table entries. So update gen_zsr.py to that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This changes the TLB misses handling back to the assembly
in user exception, and any page faults during TLB misses to be
handled in double exception handler. This should speed up
simple TLB miss handling as we don't have to go all the way to
the C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Userspace support for Xtensa architecture using Xtensa MMU.
Some considerations:
- Syscalls are not inline functions like in other architectures because
some compiler issues when using multiple registers to pass parameters
to the syscall. So here we have a function call so we can use
registers as we need.
- TLS is not supported by xcc in xtensa and reading PS register is
a privileged instruction. So, we have to use threadptr to know if a
thread is an user mode thread.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Simplify the logic around xtensa_mmu_init.
- Do not have a different path to init part of kernel
- Call xtensa_mmu_init from C
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Replace all autorefill helpers with only one that invalidates both,
DTLB and ITLB, since that is what is really needed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This register alias was originally introduced to allow A0 to be used
as a scratch register when handling exceptions from MOVSP
instructions. (It replaced some upstream code from Cadence that
hard-coded EXCSAVE1). Now the MMU code is now using too, and for
exactly the same purpose.
Calling it "ALLOCA" is only confusing. Rename it to make it clear
what it's doing.
Signed-off-by: Andy Ross <andyross@google.com>
Some workarounds were introduced for intel cavs2.5 platform bring up.
It is not general so move them to platform code.
Signed-off-by: Rander Wang <rander.wang@intel.com>
This moves the k_* memory management functions from sys/ into
kernel/ includes, as there are kernel public APIs. The z_*
functions are further separated into the kernel internal
header directory.
Also made a quick change to doxygen to group sys_mem_* into
the OS Memory Management group so they will appear in doc.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
z_mp_entry has been removed from Xtensa architecture.
So there is no need for a function declaration. Remove it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This provides custom memory range check functions as
it gets a bit complicated with cached/uncached regions.
These functions are marked as __weak so SoC or board
can override these if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds support for using coredump with Xtensa DC233C core,
which are being used by qemu_xtensa and qemu_xtensa_mmu.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_switch() is basically an alias to xtensa_switch() so
we can mark arch_switch() as ALWAYS_INLINE to avoid another
function call, especially when no optimization is used when
debugging.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a function arch_xtensa_mmu_post_init() which can
be implemented on the SoC layer to perform additional MMU
initialization steps if necessary.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...VECBASE during MMU initialization. This is to make sure
that we can use the TLB miss handling in the exception
vector after we have moved back the VECBASE during MMU
initialization. Or else we would be forever stuck in ITLB
miss because the exception vectors are not in TLB and we
cannot populate the TLB because those vectors are not in
TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
...and move it to xtensa_mmu_priv.h.
This would allow the SoC layer to use the RING number if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds arch_reserved_pages_update() which is called in
k_mem_manage_init() to reserve some physical pages so they
are not re-mapped. This is due to Zephyr's linker scripts
for Xtensa which often puts something before z_mapped_start
(aka .text, for example, vecbase). That space needs to be
reserved or else k_mem_map() would be mapping those that
could result in faults.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of only handling data TLB misses for VECBASE, change it
to handle all data TLB misses in the double exception handler.
It is because we may encounter data TLB misses when trying to
preload page table entries inside user exception handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
If CONFIG_XTENSA_RPO_CACHE is not enabled, it can be assumed
that memory is not double mapped in hardware for cached and
uncached access. So we can specify those regions to have
cache via TLB.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Simply using __data_start and __data_end is not enough as
it leaves out kobject regions which is supposed to be
near .data section. So use _image_ram_start and
_image_ram_end instead to enclose data, bss and various
kobject regions (among others).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
During MMU initialization, we clear TLB way 6 to remove all
identity mapping. Depending on CPU configuration, there are
certain number of entries per way. So use the number from
core-isa.h to clear enough entries instead of hard-coded
number 8. Specifying an entry number outside of permitted
range may result in CPU reacting in weird way so better to
avoid that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the identity map of the first 512MB in TLB way 6.
Or else it would interfere with mapped entries resulting in
double mapped TLB exception.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
