michaelh/zephyr
1
0
Fork 0
Code Releases Activity

tests: Port Thread-Metric benchmark from ThreadX

Browse source 
Ports the Thread-Metric suite of benchmarks from ThreadX to Zephyr.
This makes it easier for others to run these benchmarks with the
best set of configuration options for their board so that
they can better compare Zephyr performance to another RTOS.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
This commit is contained in:
Peter Mitsis 2024-09-27 10:32:25 -07:00 • committed by Anas Nashif
commit 95a97fd287
16 changed files with 2353 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

16
tests/benchmarks/thread_metric/CMakeLists.txt Normal file
View file

@ -0,0 +1,16 @@
# SPDX-License-Identifier: Apache-2.0
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(thread_metric)
FILE(GLOB app_sources src/tm_porting_layer_zephyr.c)
target_sources(app PRIVATE ${app_sources})
target_sources_ifdef(CONFIG_TM_BASIC app PRIVATE src/tm_basic_processing_test.c)
target_sources_ifdef(CONFIG_TM_COOPERATIVE app PRIVATE src/tm_cooperative_scheduling_test.c)
target_sources_ifdef(CONFIG_TM_INTERRUPT app PRIVATE src/tm_interrupt_processing_test.c)
target_sources_ifdef(CONFIG_TM_INTERRUPT_PREEMPTION app PRIVATE src/tm_interrupt_preemption_processing_test.c)
target_sources_ifdef(CONFIG_TM_MEMORY_ALLOCATION app PRIVATE src/tm_memory_allocation_test.c)
target_sources_ifdef(CONFIG_TM_MESSAGE app PRIVATE src/tm_message_processing_test.c)
target_sources_ifdef(CONFIG_TM_PREEMPTIVE app PRIVATE src/tm_preemptive_scheduling_test.c)
target_sources_ifdef(CONFIG_TM_SYNCHRONIZATION app PRIVATE src/tm_synchronization_processing_test.c)

83
tests/benchmarks/thread_metric/Kconfig Normal file
View file

@ -0,0 +1,83 @@
# Copyright (c) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
mainmenu "Eclipse ThreadX Thread-Metric RTOS Test Suite"
source "Kconfig.zephyr"
choice TM_TEST
prompt "Select a Thread-Metric test to execute"
default TM_PREEMPTIVE
help
The Thread-Metric benchmark suite consists of eight RTOS tests.
These tests measure the total number of RTOS events that can be
processed during a 30 second time interval.
config TM_BASIC
bool "Baseline basic benchmark"
help
The baseline basic benchmark consists of a single thread that counts
the number of times it performs a set of calculations. This number
is reported every 30 seconds.
config TM_COOPERATIVE
bool "Cooperative context switching"
help
The cooperative context switching benchmark spawns five (5) threads
of equal priority that yield to each other and increment counters
on each context switch. The sum total of the counters is reported
every 30 seconds.
config TM_INTERRUPT
bool "Interrupt processing"
select TEST
select IRQ_OFFLOAD
select IRQ_OFFLOAD_NESTED
help
The interrupt processing benchmark has a single thread that causes
an interrupt which results in its ISR incrementing a counter and then
posting a semaphore. The thread then increments its own counter and
takes that semaphore. The sum total of the counters is reported
every 30 seconds.
config TM_INTERRUPT_PREEMPTION
bool "Interrupt processing preemption"
select TEST
select IRQ_OFFLOAD
select IRQ_OFFLOAD_NESTED
help
The interrupt preemption benchmark counts the number of times that
an ISR from a software generated interrupt results in the preemption
of a thread. The total number of context switches is reported every
30 seconds.
config TM_MEMORY_ALLOCATION
bool "Memory allocation"
help
The memory allocation benchmark counts the number of times a thread
is able to allocate and then release a 128-byte block. This number
is reported every 30 seconds.
config TM_MESSAGE
bool "Message processing"
help
The message processing benchmark counts the number of times that a
thread can send and receive a 16-byte message from a message queue.
This number is reported every 30 seconds.
config TM_PREEMPTIVE
bool "Preemptive context switching"
help
The preemptive context switching benchmark creates five (5) threads
of different priorities that suspend and resume each other in a
cyclical pattern. The total number of context switches is reported
every 30 seconds.
config TM_SYNCHRONIZATION
bool "Synchronization"
help
The synchronization benchmark counts the number of times that a
thread can give and take a semaphore without blocking. This number
is reported every 30 seconds.
endchoice

25
tests/benchmarks/thread_metric/prj.conf Normal file
View file

@ -0,0 +1,25 @@
# Default base configuration file
# Use a tickless kernel to minimize the number of timer interrupts
CONFIG_TICKLESS_KERNEL=y
CONFIG_SYS_CLOCK_TICKS_PER_SEC=100
# Optimize for speed
CONFIG_SPEED_OPTIMIZATIONS=y
# Disable time slicing
CONFIG_TIMESLICING=n
# Test is only designed for a single CPU
CONFIG_MP_MAX_NUM_CPUS=1
# Disabling hardware stack protection can greatly
# improve system performance.
CONFIG_HW_STACK_PROTECTION=n
# Picolibc is faster than Zephyr's minimal libc memcpy
CONFIG_PICOLIBC_SPEED_OPTIMIZATIONS=y
CONFIG_PICOLIBC_USE_MODULE=y
# Disable Thread Local Storage for better context switching times
CONFIG_THREAD_LOCAL_STORAGE=n

103
tests/benchmarks/thread_metric/src/tm_api.h Normal file
View file

@ -0,0 +1,103 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Application Interface (API) */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* APPLICATION INTERFACE DEFINITION RELEASE */
/* */
/* tm_api.h PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the basic Application Interface (API) */
/* implementation source code for the Thread-Metrics performance */
/* test suite. All service prototypes and data structure definitions */
/* are defined in this file. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#ifndef TM_API_H
#define TM_API_H
#include "tm_porting_layer.h"
/*
* Determine if a C++ compiler is being used. If so, ensure that standard
* C is used to process the API information.
*/
#ifdef __cplusplus
/* Yes, C++ compiler is present. Use standard C. */
extern "C" {
#endif
/* Define API constants. */
#define TM_SUCCESS 0
#define TM_ERROR 1
/* Define the time interval in seconds. This can be changed with a -D compiler option. */
#ifndef TM_TEST_DURATION
#define TM_TEST_DURATION 30
#endif
/*
* Define RTOS Neutral APIs. RTOS vendors should fill in the guts of the following
* API. Once this is done the Thread-Metric tests can be successfully run.
*/
void tm_initialize(void (*test_initialization_function)(void));
int tm_thread_create(int thread_id, int priority, void (*entry_function)(void *, void *, void *));
int tm_thread_resume(int thread_id);
int tm_thread_suspend(int thread_id);
void tm_thread_relinquish(void);
void tm_thread_sleep(int seconds);
int tm_queue_create(int queue_id);
int tm_queue_send(int queue_id, unsigned long *message_ptr);
int tm_queue_receive(int queue_id, unsigned long *message_ptr);
int tm_semaphore_create(int semaphore_id);
int tm_semaphore_get(int semaphore_id);
int tm_semaphore_put(int semaphore_id);
int tm_memory_pool_create(int pool_id);
int tm_memory_pool_allocate(int pool_id, unsigned char **memory_ptr);
int tm_memory_pool_deallocate(int pool_id, unsigned char *memory_ptr);
/*
* Determine if a C++ compiler is being used. If so, complete the standard
* C conditional started above.
*/
#ifdef __cplusplus
}
#endif
#endif

169
tests/benchmarks/thread_metric/src/tm_basic_processing_test.c Normal file
View file

@ -0,0 +1,169 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Basic Processing Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_basic_processing_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the basic test for determining board processing */
/* capabilities */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
volatile unsigned long tm_basic_processing_counter;
/*
* Test array. We will just do a series of calculations on the
* test array to eat up processing bandwidth. The idea is that
* all RTOSes should produce the same metric here if everything
* else is equal, e.g. processor speed, memory speed, etc.
*/
volatile unsigned long tm_basic_processing_array[1024];
/* Define the test thread prototypes. */
void tm_basic_processing_thread_0_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_basic_processing_thread_report(void);
/* Define the initialization prototype. */
void tm_basic_processing_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_basic_processing_initialize);
return 0;
}
/* Define the basic processing test initialization. */
void tm_basic_processing_initialize(void)
{
/* Create thread 0 at priority 10. */
tm_thread_create(0, 10, tm_basic_processing_thread_0_entry);
/* Resume thread 0. */
tm_thread_resume(0);
tm_basic_processing_thread_report();
}
/* Define the basic processing thread. */
void tm_basic_processing_thread_0_entry(void *p1, void *p2, void *p3)
{
int i;
(void)p1;
(void)p2;
(void)p3;
/* Initialize the test array. */
for (i = 0; i < 1024; i++) {
/* Clear the basic processing array. */
tm_basic_processing_array[i] = 0;
}
while (1) {
/*
* Loop through the basic processing array, add the previous
* contents with the contents of the tm_basic_processing_counter
* and xor the result with the previous value... just to eat
* up some time.
*/
for (i = 0; i < 1024; i++) {
/* Update each array entry. */
tm_basic_processing_array[i] =
(tm_basic_processing_array[i] + tm_basic_processing_counter) ^
tm_basic_processing_array[i];
}
/* Increment the basic processing counter. */
tm_basic_processing_counter++;
}
}
/* Define the basic processing reporting function. */
void tm_basic_processing_thread_report(void)
{
unsigned long last_counter;
unsigned long relative_time;
/* Initialize the last counter. */
last_counter = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Basic Single Thread Processing Test **** Relative Time: "
"%lu\n",
relative_time);
/* See if there are any errors. */
if (tm_basic_processing_counter == last_counter) {
printf("ERROR: Invalid counter value(s). Basic processing thread died!\n");
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n", tm_basic_processing_counter - last_counter);
/* Save the last counter. */
last_counter = tm_basic_processing_counter;
}
}

250
tests/benchmarks/thread_metric/src/tm_cooperative_scheduling_test.c Normal file
View file

@ -0,0 +1,250 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Cooperative Scheduling Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_cooperative_scheduling_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the cooperative scheduling test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_cooperative_thread_0_counter;
unsigned long tm_cooperative_thread_1_counter;
unsigned long tm_cooperative_thread_2_counter;
unsigned long tm_cooperative_thread_3_counter;
unsigned long tm_cooperative_thread_4_counter;
/* Define the test thread prototypes. */
void tm_cooperative_thread_0_entry(void *p1, void *p2, void *p3);
void tm_cooperative_thread_1_entry(void *p1, void *p2, void *p3);
void tm_cooperative_thread_2_entry(void *p1, void *p2, void *p3);
void tm_cooperative_thread_3_entry(void *p1, void *p2, void *p3);
void tm_cooperative_thread_4_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_cooperative_thread_report(void);
/* Define the initialization prototype. */
void tm_cooperative_scheduling_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_cooperative_scheduling_initialize);
return 0;
}
/* Define the cooperative scheduling test initialization. */
void tm_cooperative_scheduling_initialize(void)
{
/* Create all 5 threads at priority 3. */
tm_thread_create(0, 3, tm_cooperative_thread_0_entry);
tm_thread_create(1, 3, tm_cooperative_thread_1_entry);
tm_thread_create(2, 3, tm_cooperative_thread_2_entry);
tm_thread_create(3, 3, tm_cooperative_thread_3_entry);
tm_thread_create(4, 3, tm_cooperative_thread_4_entry);
/* Resume all 5 threads. */
tm_thread_resume(0);
tm_thread_resume(1);
tm_thread_resume(2);
tm_thread_resume(3);
tm_thread_resume(4);
tm_cooperative_thread_report();
}
/* Define the first cooperative thread. */
void tm_cooperative_thread_0_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Relinquish to all other threads at same priority. */
tm_thread_relinquish();
/* Increment this thread's counter. */
tm_cooperative_thread_0_counter++;
}
}
/* Define the second cooperative thread. */
void tm_cooperative_thread_1_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Relinquish to all other threads at same priority. */
tm_thread_relinquish();
/* Increment this thread's counter. */
tm_cooperative_thread_1_counter++;
}
}
/* Define the third cooperative thread. */
void tm_cooperative_thread_2_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Relinquish to all other threads at same priority. */
tm_thread_relinquish();
/* Increment this thread's counter. */
tm_cooperative_thread_2_counter++;
}
}
/* Define the fourth cooperative thread. */
void tm_cooperative_thread_3_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Relinquish to all other threads at same priority. */
tm_thread_relinquish();
/* Increment this thread's counter. */
tm_cooperative_thread_3_counter++;
}
}
/* Define the fifth cooperative thread. */
void tm_cooperative_thread_4_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Relinquish to all other threads at same priority. */
tm_thread_relinquish();
/* Increment this thread's counter. */
tm_cooperative_thread_4_counter++;
}
}
/* Define the cooperative test reporting function. */
void tm_cooperative_thread_report(void)
{
unsigned long total;
unsigned long relative_time;
unsigned long last_total;
unsigned long average;
/* Initialize the last total. */
last_total = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Cooperative Scheduling Test **** Relative Time: %lu\n",
relative_time);
/* Calculate the total of all the counters. */
total = tm_cooperative_thread_0_counter + tm_cooperative_thread_1_counter +
tm_cooperative_thread_2_counter + tm_cooperative_thread_3_counter +
tm_cooperative_thread_4_counter;
/* Calculate the average of all the counters. */
average = total / 5;
/* WCC - integrity check */
printf("tm_cooperative_thread_0_counter: %lu\n", tm_cooperative_thread_0_counter);
printf("tm_cooperative_thread_1_counter: %lu\n", tm_cooperative_thread_1_counter);
printf("tm_cooperative_thread_2_counter: %lu\n", tm_cooperative_thread_2_counter);
printf("tm_cooperative_thread_3_counter: %lu\n", tm_cooperative_thread_3_counter);
printf("tm_cooperative_thread_4_counter: %lu\n", tm_cooperative_thread_4_counter);
/* See if there are any errors. */
if ((tm_cooperative_thread_0_counter < (average - 1)) ||
(tm_cooperative_thread_0_counter > (average + 1)) ||
(tm_cooperative_thread_1_counter < (average - 1)) ||
(tm_cooperative_thread_1_counter > (average + 1)) ||
(tm_cooperative_thread_2_counter < (average - 1)) ||
(tm_cooperative_thread_2_counter > (average + 1)) ||
(tm_cooperative_thread_3_counter < (average - 1)) ||
(tm_cooperative_thread_3_counter > (average + 1)) ||
(tm_cooperative_thread_4_counter < (average - 1)) ||
(tm_cooperative_thread_4_counter > (average + 1))) {
printf("ERROR: Invalid counter value(s). Cooperative counters should not "
"be more that 1 different than the average!\n");
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n", total - last_total);
/* Save the last total. */
last_total = total;
}
}

218
tests/benchmarks/thread_metric/src/tm_interrupt_preemption_processing_test.c Normal file
View file

@ -0,0 +1,218 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Interrupt Preemption Processing Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_interrupt_preemption_processing_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the preemptive scheduling test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_interrupt_preemption_thread_0_counter;
unsigned long tm_interrupt_preemption_thread_1_counter;
unsigned long tm_interrupt_preemption_handler_counter;
/* Define the test thread prototypes. */
void tm_interrupt_preemption_thread_0_entry(void *p1, void *p2, void *p3);
void tm_interrupt_preemption_thread_1_entry(void *p1, void *p2, void *p3);
/* Define the interrupt handler. This must be called from the RTOS. */
void tm_interrupt_handler(void);
/* Define the initialization prototype. */
void tm_interrupt_preemption_processing_initialize(void);
/* Define the reporting function */
void tm_interrupt_preemption_thread_report(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_interrupt_preemption_processing_initialize);
return 0;
}
/* Define the interrupt processing test initialization. */
void tm_interrupt_preemption_processing_initialize(void)
{
/* Create interrupt thread at priority 3. */
tm_thread_create(0, 3, tm_interrupt_preemption_thread_0_entry);
/* Create thread that generates the interrupt at priority 10. */
tm_thread_create(1, 10, tm_interrupt_preemption_thread_1_entry);
/* Resume just thread 1. */
tm_thread_resume(1);
tm_interrupt_preemption_thread_report();
}
/*
* Define the interrupt thread. This thread is resumed from the
* interrupt handler. It runs and suspends.
*/
void tm_interrupt_preemption_thread_0_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Increment this thread's counter. */
tm_interrupt_preemption_thread_0_counter++;
/*
* Suspend. This will allow the thread generating the
* interrupt to run again.
*/
tm_thread_suspend(0);
}
}
/* Define the thread that generates the interrupt. */
void tm_interrupt_preemption_thread_1_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/*
* Force an interrupt. The underlying RTOS must see that the
* the interrupt handler is called from the appropriate software
* interrupt or trap.
*/
TM_CAUSE_INTERRUPT;
/*
* We won't get back here until the interrupt processing is complete,
* including the execution of the higher priority thread made ready
* by the interrupt.
*/
/* Increment this thread's counter. */
tm_interrupt_preemption_thread_1_counter++;
}
}
/*
* Define the interrupt handler. This must be called from the RTOS trap handler.
* To be fair, it must behave just like a processor interrupt, i.e. it must save
* the full context of the interrupted thread during the preemption processing.
*/
void tm_interrupt_handler(void)
{
/* Increment the interrupt count. */
tm_interrupt_preemption_handler_counter++;
/* Resume the higher priority thread from the ISR. */
tm_thread_resume(0);
}
/* Define the interrupt test reporting function. */
void tm_interrupt_preemption_thread_report(void)
{
unsigned long total;
unsigned long relative_time;
unsigned long last_total;
unsigned long average;
/* Initialize the last total. */
last_total = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Interrupt Preemption Processing Test **** Relative "
"Time: %lu\n",
relative_time);
/* Calculate the total of all the counters. */
total = tm_interrupt_preemption_thread_0_counter +
tm_interrupt_preemption_thread_1_counter +
tm_interrupt_preemption_handler_counter;
/* Calculate the average of all the counters. */
average = total / 3;
/* See if there are any errors. */
if ((tm_interrupt_preemption_thread_0_counter < (average - 1)) ||
(tm_interrupt_preemption_thread_0_counter > (average + 1)) ||
(tm_interrupt_preemption_thread_1_counter < (average - 1)) ||
(tm_interrupt_preemption_thread_1_counter > (average + 1)) ||
(tm_interrupt_preemption_handler_counter < (average - 1)) ||
(tm_interrupt_preemption_handler_counter > (average + 1))) {
printf("ERROR: Invalid counter value(s). Interrupt processing test has "
"failed!\n");
}
/* Show the total interrupts for the time period. */
printf("Time Period Total: %lu\n\n",
tm_interrupt_preemption_handler_counter - last_total);
/* Save the last total number of interrupts. */
last_total = tm_interrupt_preemption_handler_counter;
}
}

202
tests/benchmarks/thread_metric/src/tm_interrupt_processing_test.c Normal file
View file

@ -0,0 +1,202 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Interrupt Processing Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_interrupt_processing_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the No-preemption interrupt processing test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_interrupt_thread_0_counter;
unsigned long tm_interrupt_handler_counter;
/* Define the test thread prototypes. */
void tm_interrupt_thread_0_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_interrupt_thread_report(void);
/* Define the interrupt handler. This must be called from the RTOS. */
void tm_interrupt_handler(void);
/* Define the initialization prototype. */
void tm_interrupt_processing_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_interrupt_processing_initialize);
return 0;
}
/* Define the interrupt processing test initialization. */
void tm_interrupt_processing_initialize(void)
{
/* Create thread that generates the interrupt at priority 10. */
tm_thread_create(0, 10, tm_interrupt_thread_0_entry);
/*
* Create a semaphore that will be posted from the interrupt
* handler.
*/
tm_semaphore_create(0);
/* Resume just thread 0. */
tm_thread_resume(0);
tm_interrupt_thread_report();
}
/* Define the thread that generates the interrupt. */
void tm_interrupt_thread_0_entry(void *p1, void *p2, void *p3)
{
int status;
/* Pickup the semaphore since it is initialized to 1 by default. */
status = tm_semaphore_get(0);
/* Check for good status. */
if (status != TM_SUCCESS) {
return;
}
while (1) {
/*
* Force an interrupt. The underlying RTOS must see that the
* the interrupt handler is called from the appropriate software
* interrupt or trap.
*/
TM_CAUSE_INTERRUPT;
/*
* We won't get back here until the interrupt processing is
* complete, including the setting of the semaphore from the
* interrupt handler.
*/
/* Pickup the semaphore set by the interrupt handler. */
status = tm_semaphore_get(0);
/* Check for good status. */
if (status != TM_SUCCESS) {
return;
}
/* Increment this thread's counter. */
tm_interrupt_thread_0_counter++;
}
}
/*
* Define the interrupt handler. This must be called from the RTOS trap handler.
* To be fair, it must behave just like a processor interrupt, i.e. it must save
* the full context of the interrupted thread during the preemption processing.
*/
void tm_interrupt_handler(void)
{
/* Increment the interrupt count. */
tm_interrupt_handler_counter++;
/* Put the semaphore from the interrupt handler. */
tm_semaphore_put(0);
}
/* Define the interrupt test reporting function. */
void tm_interrupt_thread_report(void)
{
unsigned long total;
unsigned long last_total;
unsigned long relative_time;
unsigned long average;
/* Initialize the last total. */
last_total = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Interrupt Processing Test **** Relative Time: %lu\n",
relative_time);
/* Calculate the total of all the counters. */
total = tm_interrupt_thread_0_counter + tm_interrupt_handler_counter;
/* Calculate the average of all the counters. */
average = total / 2;
/* See if there are any errors. */
if ((tm_interrupt_thread_0_counter < (average - 1)) ||
(tm_interrupt_thread_0_counter > (average + 1)) ||
(tm_interrupt_handler_counter < (average - 1)) ||
(tm_interrupt_handler_counter > (average + 1))) {
printf("ERROR: Invalid counter value(s). Interrupt processing test has "
"failed!\n");
}
/* Show the total interrupts for the time period. */
printf("Time Period Total: %lu\n\n", tm_interrupt_handler_counter - last_total);
/* Save the last total number of interrupts. */
last_total = tm_interrupt_handler_counter;
}
}

150
tests/benchmarks/thread_metric/src/tm_memory_allocation_test.c Normal file
View file

@ -0,0 +1,150 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Memory Allocation Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_memory_allocation_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the Message exchange processing test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_memory_allocation_counter;
/* Define the test thread prototypes. */
void tm_memory_allocation_thread_0_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_memory_allocation_thread_report(void);
/* Define the initialization prototype. */
void tm_memory_allocation_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_memory_allocation_initialize);
return 0;
}
/* Define the memory allocation processing test initialization. */
void tm_memory_allocation_initialize(void)
{
/* Create a memory pool. */
tm_memory_pool_create(0);
/* Create thread 0 at priority 10. */
tm_thread_create(0, 10, tm_memory_allocation_thread_0_entry);
/* Resume thread 0. */
tm_thread_resume(0);
tm_memory_allocation_thread_report();
}
/* Define the memory allocation processing thread. */
void tm_memory_allocation_thread_0_entry(void *p1, void *p2, void *p3)
{
int status;
unsigned char *memory_ptr;
while (1) {
/* Allocate memory from pool. */
tm_memory_pool_allocate(0, &memory_ptr);
/* Release the memory back to the pool. */
status = tm_memory_pool_deallocate(0, memory_ptr);
/* Check for invalid memory allocation/deallocation. */
if (status != TM_SUCCESS) {
break;
}
/* Increment the number of memory allocations sent and received. */
tm_memory_allocation_counter++;
}
}
/* Define the memory allocation test reporting function. */
void tm_memory_allocation_thread_report(void)
{
unsigned long last_counter;
unsigned long relative_time;
/* Initialize the last counter. */
last_counter = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Memory Allocation Test **** Relative Time: %lu\n",
relative_time);
/* See if there are any errors. */
if (tm_memory_allocation_counter == last_counter) {
printf("ERROR: Invalid counter value(s). Error allocating/deallocating "
"memory!\n");
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n", tm_memory_allocation_counter - last_counter);
/* Save the last counter. */
last_counter = tm_memory_allocation_counter;
}
}

161
tests/benchmarks/thread_metric/src/tm_message_processing_test.c Normal file
View file

@ -0,0 +1,161 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Message Processing Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_message_processing_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* Basic test for message exchange processing. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_message_processing_counter;
unsigned long tm_message_sent[4];
unsigned long tm_message_received[4];
/* Define the test thread prototypes. */
void tm_message_processing_thread_0_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_message_processing_thread_report(void);
/* Define the initialization prototype. */
void tm_message_processing_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_message_processing_initialize);
return 0;
}
/* Define the message processing test initialization. */
void tm_message_processing_initialize(void)
{
/* Create thread 0 at priority 10. */
tm_thread_create(0, 10, tm_message_processing_thread_0_entry);
/* Resume thread 0. */
tm_thread_resume(0);
/* Create a queue for the message passing. */
tm_queue_create(0);
tm_message_processing_thread_report();
}
/* Define the message processing thread. */
void tm_message_processing_thread_0_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
/* Initialize the source message. */
tm_message_sent[0] = 0x11112222;
tm_message_sent[1] = 0x33334444;
tm_message_sent[2] = 0x55556666;
tm_message_sent[3] = 0x77778888;
while (1) {
/* Send a message to the queue. */
tm_queue_send(0, tm_message_sent);
/* Receive a message from the queue. */
tm_queue_receive(0, tm_message_received);
/* Check for invalid message. */
if (tm_message_received[3] != tm_message_sent[3]) {
break;
}
/* Increment the last word of the 16-byte message. */
tm_message_sent[3]++;
/* Increment the number of messages sent and received. */
tm_message_processing_counter++;
}
}
/* Define the message test reporting function. */
void tm_message_processing_thread_report(void)
{
unsigned long last_counter;
unsigned long relative_time;
/* Initialize the last counter. */
last_counter = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Message Processing Test **** Relative Time: %lu\n",
relative_time);
/* See if there are any errors. */
if (tm_message_processing_counter == last_counter) {
printf("ERROR: Invalid counter value(s). Error sending/receiving "
"messages!\n");
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n", tm_message_processing_counter - last_counter);
/* Save the last counter. */
last_counter = tm_message_processing_counter;
}
}

20
tests/benchmarks/thread_metric/src/tm_porting_layer.h Normal file
View file

@ -0,0 +1,20 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
#ifndef TM_PORTING_LAYER_H
#define TM_PORTING_LAYER_H
#include <stdio.h>
extern void tm_cause_interrupt(void);
#define TM_CAUSE_INTERRUPT tm_cause_interrupt()
#endif

223
tests/benchmarks/thread_metric/src/tm_porting_layer_zephyr.c Normal file
View file

@ -0,0 +1,223 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
* Copyright (c) 2024 Intel Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Porting Layer (Must be completed with RTOS specifics) */
/** */
/**************************************************************************/
/**************************************************************************/
/* Include necessary files. */
#include "tm_api.h"
#include <zephyr/kernel.h>
#define TM_TEST_NUM_THREADS 10
#define TM_TEST_STACK_SIZE 1024
#define TM_TEST_NUM_SEMAPHORES 4
#define TM_TEST_NUM_MESSAGE_QUEUES 4
#define TM_TEST_NUM_SLABS 4
#if (CONFIG_MP_MAX_NUM_CPUS > 1)
#error "*** Tests are only designed for single processor systems! ***"
#endif
static struct k_thread test_thread[TM_TEST_NUM_THREADS];
static K_THREAD_STACK_ARRAY_DEFINE(test_stack, TM_TEST_NUM_THREADS, TM_TEST_STACK_SIZE);
static struct k_sem test_sem[TM_TEST_NUM_SEMAPHORES];
static struct k_msgq test_msgq[TM_TEST_NUM_MESSAGE_QUEUES];
static char test_msgq_buffer[TM_TEST_NUM_MESSAGE_QUEUES][8][16];
static struct k_mem_slab test_slab[TM_TEST_NUM_SLABS];
static char __aligned(4) test_slab_buffer[TM_TEST_NUM_SLABS][8 * 128];
/*
* This function called from main performs basic RTOS initialization,
* calls the test initialization function, and then starts the RTOS function.
*/
void tm_initialize(void (*test_initialization_function)(void))
{
test_initialization_function();
}
/*
* This function takes a thread ID and priority and attempts to create the
* file in the underlying RTOS. Valid priorities range from 1 through 31,
* where 1 is the highest priority and 31 is the lowest. If successful,
* the function should return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_thread_create(int thread_id, int priority, void (*entry_function)(void *, void *, void *))
{
k_tid_t tid;
tid = k_thread_create(&test_thread[thread_id], test_stack[thread_id],
TM_TEST_STACK_SIZE, entry_function,
NULL, NULL, NULL, priority, 0, K_FOREVER);
return (tid == &test_thread[thread_id]) ? TM_SUCCESS : TM_ERROR;
}
/*
* This function resumes the specified thread. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_thread_resume(int thread_id)
{
if (test_thread[thread_id].base.thread_state & _THREAD_PRESTART) {
k_thread_start(&test_thread[thread_id]);
} else {
k_thread_resume(&test_thread[thread_id]);
}
return TM_SUCCESS;
}
/*
* This function suspends the specified thread. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_thread_suspend(int thread_id)
{
k_thread_suspend(&test_thread[thread_id]);
return TM_SUCCESS;
}
/*
* This function relinquishes to other ready threads at the same
* priority.
*/
void tm_thread_relinquish(void)
{
k_yield();
}
/*
* This function suspends the specified thread for the specified number
* of seconds.
*/
void tm_thread_sleep(int seconds)
{
k_sleep(K_SECONDS(seconds));
}
/*
* This function creates the specified queue. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_queue_create(int queue_id)
{
k_msgq_init(&test_msgq[queue_id], &test_msgq_buffer[queue_id][0][0], 16, 8);
return TM_SUCCESS;
}
/*
* This function sends a 16-byte message to the specified queue. If successful,
* the function should return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_queue_send(int queue_id, unsigned long *message_ptr)
{
return k_msgq_put(&test_msgq[queue_id], message_ptr, K_FOREVER);
}
/*
* This function receives a 16-byte message from the specified queue. If successful,
* the function should return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_queue_receive(int queue_id, unsigned long *message_ptr)
{
return k_msgq_get(&test_msgq[queue_id], message_ptr, K_FOREVER);
}
/*
* This function creates the specified semaphore. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_semaphore_create(int semaphore_id)
{
/* Create an available semaphore with max count of 1 */
return k_sem_init(&test_sem[semaphore_id], 1, 1);
}
/*
* This function gets the specified semaphore. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_semaphore_get(int semaphore_id)
{
return k_sem_take(&test_sem[semaphore_id], K_NO_WAIT);
}
/*
* This function puts the specified semaphore. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_semaphore_put(int semaphore_id)
{
k_sem_give(&test_sem[semaphore_id]);
return TM_SUCCESS;
}
/* This function is defined by the benchmark. */
extern void tm_interrupt_handler(const void *);
void tm_cause_interrupt(void)
{
irq_offload(tm_interrupt_handler, NULL);
}
/*
* This function creates the specified memory pool that can support one or more
* allocations of 128 bytes. If successful, the function should
* return TM_SUCCESS. Otherwise, TM_ERROR should be returned.
*/
int tm_memory_pool_create(int pool_id)
{
int status;
status = k_mem_slab_init(&test_slab[pool_id], &test_slab_buffer[pool_id][0], 128, 8);
return (status == 0) ? TM_SUCCESS : TM_ERROR;
}
/*
* This function allocates a 128 byte block from the specified memory pool.
* If successful, the function should return TM_SUCCESS. Otherwise, TM_ERROR
* should be returned.
*/
int tm_memory_pool_allocate(int pool_id, unsigned char **memory_ptr)
{
int status;
status = k_mem_slab_alloc(&test_slab[pool_id], (void **)memory_ptr, K_NO_WAIT);
return (status == 0) ? TM_SUCCESS : TM_ERROR;
}
/*
* This function releases a previously allocated 128 byte block from the specified
* memory pool. If successful, the function should return TM_SUCCESS. Otherwise, TM_ERROR
* should be returned.
*/
int tm_memory_pool_deallocate(int pool_id, unsigned char *memory_ptr)
{
k_mem_slab_free(&test_slab[pool_id], (void *)memory_ptr);
return TM_SUCCESS;
}

281
tests/benchmarks/thread_metric/src/tm_preemptive_scheduling_test.c Normal file
View file

@ -0,0 +1,281 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Preemptive Scheduling Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_preemptive_scheduling_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the preemptive scheduling test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_preemptive_thread_0_counter;
unsigned long tm_preemptive_thread_1_counter;
unsigned long tm_preemptive_thread_2_counter;
unsigned long tm_preemptive_thread_3_counter;
unsigned long tm_preemptive_thread_4_counter;
/* Define the test thread prototypes. */
void tm_preemptive_thread_0_entry(void *p1, void *p2, void *p3);
void tm_preemptive_thread_1_entry(void *p1, void *p2, void *p3);
void tm_preemptive_thread_2_entry(void *p1, void *p2, void *p3);
void tm_preemptive_thread_3_entry(void *p1, void *p2, void *p3);
void tm_preemptive_thread_4_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_preemptive_thread_report(void);
/* Define the initialization prototype. */
void tm_preemptive_scheduling_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_preemptive_scheduling_initialize);
return 0;
}
/* Define the preemptive scheduling test initialization. */
void tm_preemptive_scheduling_initialize(void)
{
/* Create thread 0 at priority 10. */
tm_thread_create(0, 10, tm_preemptive_thread_0_entry);
/* Create thread 1 at priority 9. */
tm_thread_create(1, 9, tm_preemptive_thread_1_entry);
/* Create thread 2 at priority 8. */
tm_thread_create(2, 8, tm_preemptive_thread_2_entry);
/* Create thread 3 at priority 7. */
tm_thread_create(3, 7, tm_preemptive_thread_3_entry);
/* Create thread 4 at priority 6. */
tm_thread_create(4, 6, tm_preemptive_thread_4_entry);
/* Resume just thread 0. */
tm_thread_resume(0);
tm_preemptive_thread_report();
}
/* Define the first preemptive thread. */
void tm_preemptive_thread_0_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Resume thread 1. */
tm_thread_resume(1);
/*
* We won't get back here until threads 1, 2, 3, and 4 all execute and
* self-suspend.
*/
/* Increment this thread's counter. */
tm_preemptive_thread_0_counter++;
}
}
/* Define the second preemptive thread. */
void tm_preemptive_thread_1_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Resume thread 2. */
tm_thread_resume(2);
/*
* We won't get back here until threads 2, 3, and 4 all execute and
* self-suspend.
*/
/* Increment this thread's counter. */
tm_preemptive_thread_1_counter++;
/* Suspend self! */
tm_thread_suspend(1);
}
}
/* Define the third preemptive thread. */
void tm_preemptive_thread_2_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Resume thread 3. */
tm_thread_resume(3);
/*
* We won't get back here until threads 3 and 4 execute and
* self-suspend.
*/
/* Increment this thread's counter. */
tm_preemptive_thread_2_counter++;
/* Suspend self! */
tm_thread_suspend(2);
}
}
/* Define the fourth preemptive thread. */
void tm_preemptive_thread_3_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Resume thread 4. */
tm_thread_resume(4);
/*
* We won't get back here until thread 4 executes and
* self-suspends.
*/
/* Increment this thread's counter. */
tm_preemptive_thread_3_counter++;
/* Suspend self! */
tm_thread_suspend(3);
}
}
/* Define the fifth preemptive thread. */
void tm_preemptive_thread_4_entry(void *p1, void *p2, void *p3)
{
(void)p1;
(void)p2;
(void)p3;
while (1) {
/* Increment this thread's counter. */
tm_preemptive_thread_4_counter++;
/* Self suspend thread 4. */
tm_thread_suspend(4);
}
}
/* Define the preemptive test reporting function. */
void tm_preemptive_thread_report(void)
{
unsigned long total;
unsigned long relative_time;
unsigned long last_total;
unsigned long average;
/* Initialize the last total. */
last_total = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Preemptive Scheduling Test **** Relative Time: %lu\n",
relative_time);
/* Calculate the total of all the counters. */
total = tm_preemptive_thread_0_counter + tm_preemptive_thread_1_counter +
tm_preemptive_thread_2_counter + tm_preemptive_thread_3_counter +
tm_preemptive_thread_4_counter;
/* Calculate the average of all the counters. */
average = total / 5;
/* See if there are any errors. */
if ((tm_preemptive_thread_0_counter < (average - 1)) ||
(tm_preemptive_thread_0_counter > (average + 1)) ||
(tm_preemptive_thread_1_counter < (average - 1)) ||
(tm_preemptive_thread_1_counter > (average + 1)) ||
(tm_preemptive_thread_2_counter < (average - 1)) ||
(tm_preemptive_thread_2_counter > (average + 1)) ||
(tm_preemptive_thread_3_counter < (average - 1)) ||
(tm_preemptive_thread_3_counter > (average + 1)) ||
(tm_preemptive_thread_4_counter < (average - 1)) ||
(tm_preemptive_thread_4_counter > (average + 1))) {
printf("ERROR: Invalid counter value(s). Preemptive counters should not be "
"more that 1 different than the average!\n");
printf(" Average: %lu, 0: %lu, 1: %lu, 2: %lu, 3: %lu, 4: %lu\n",
average, tm_preemptive_thread_0_counter,
tm_preemptive_thread_1_counter,
tm_preemptive_thread_2_counter,
tm_preemptive_thread_3_counter,
tm_preemptive_thread_4_counter);
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n", total - last_total);
/* Save the last total. */
last_total = total;
}
}

152
tests/benchmarks/thread_metric/src/tm_synchronization_processing_test.c Normal file
View file

@ -0,0 +1,152 @@
/***************************************************************************
* Copyright (c) 2024 Microsoft Corporation
*
* This program and the accompanying materials are made available under the
* terms of the MIT License which is available at
* https://opensource.org/licenses/MIT.
*
* SPDX-License-Identifier: MIT
**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/** */
/** Thread-Metric Component */
/** */
/** Synchronization Processing Test */
/** */
/**************************************************************************/
/**************************************************************************/
/**************************************************************************/
/* */
/* FUNCTION RELEASE */
/* */
/* tm_synchronization_processing_test PORTABLE C */
/* 6.1.7 */
/* AUTHOR */
/* */
/* William E. Lamie, Microsoft Corporation */
/* */
/* DESCRIPTION */
/* */
/* This file defines the Semaphore get/put processing test. */
/* */
/* RELEASE HISTORY */
/* */
/* DATE NAME DESCRIPTION */
/* */
/* 10-15-2021 William E. Lamie Initial Version 6.1.7 */
/* */
/**************************************************************************/
#include "tm_api.h"
/* Define the counters used in the demo application... */
unsigned long tm_synchronization_processing_counter;
/* Define the test thread prototypes. */
void tm_synchronization_processing_thread_0_entry(void *p1, void *p2, void *p3);
/* Define the reporting function prototype. */
void tm_synchronization_processing_thread_report(void);
/* Define the initialization prototype. */
void tm_synchronization_processing_initialize(void);
/* Define main entry point. */
int main(void)
{
/* Initialize the test. */
tm_initialize(tm_synchronization_processing_initialize);
return 0;
}
/* Define the synchronization processing test initialization. */
void tm_synchronization_processing_initialize(void)
{
/* Create thread 0 at priority 10. */
tm_thread_create(0, 10, tm_synchronization_processing_thread_0_entry);
/* Resume thread 0. */
tm_thread_resume(0);
/* Create a semaphore for the test. */
tm_semaphore_create(0);
tm_synchronization_processing_thread_report();
}
/* Define the synchronization processing thread. */
void tm_synchronization_processing_thread_0_entry(void *p1, void *p2, void *p3)
{
int status;
while (1) {
/* Get the semaphore. */
tm_semaphore_get(0);
/* Release the semaphore. */
status = tm_semaphore_put(0);
/* Check for semaphore put error. */
if (status != TM_SUCCESS) {
break;
}
/* Increment the number of semaphore get/puts. */
tm_synchronization_processing_counter++;
}
}
/* Define the synchronization test reporting function. */
void tm_synchronization_processing_thread_report(void)
{
unsigned long last_counter;
unsigned long relative_time;
/* Initialize the last counter. */
last_counter = 0;
/* Initialize the relative time. */
relative_time = 0;
while (1) {
/* Sleep to allow the test to run. */
tm_thread_sleep(TM_TEST_DURATION);
/* Increment the relative time. */
relative_time = relative_time + TM_TEST_DURATION;
/* Print results to the stdio window. */
printf("**** Thread-Metric Synchronization Processing Test **** Relative Time: "
"%lu\n",
relative_time);
/* See if there are any errors. */
if (tm_synchronization_processing_counter == last_counter) {
printf("ERROR: Invalid counter value(s). Error getting/putting "
"semaphore!\n");
}
/* Show the time period total. */
printf("Time Period Total: %lu\n\n",
tm_synchronization_processing_counter - last_counter);
/* Save the last counter. */
last_counter = tm_synchronization_processing_counter;
}
}

52
tests/benchmarks/thread_metric/testcase.yaml Normal file
View file

@ -0,0 +1,52 @@
common:
tags:
- kernel
- benchmark
# Native platforms excluded as timer interrupts may not be detected
# qemu_nios2 excluded as it is slow
platform_exclude:
- native_posix
- native_sim
- qemu_nios2
integration_platforms:
- qemu_x86
- qemu_cortex_a53
timeout: 300
harness: console
harness_config:
type: one_line
regex:
- "(.*) Relative Time: (.*)"
tests:
benchmark.thread_metric.basic:
extra_configs:
- CONFIG_TM_BASIC=y
benchmark.thread_metric.cooperative:
extra_configs:
- CONFIG_TM_COOPERATIVE=y
benchmark.thread_metric.interrupt:
extra_configs:
- CONFIG_TM_INTERRUPT=y
benchmark.thread_metric.interrupt_preemption:
extra_configs:
- CONFIG_TM_INTERRUPT_PREEMPTION=y
benchmark.thread_metric.memory_allocation:
extra_configs:
- CONFIG_TM_MEMORY_ALLOCATION=y
benchmark.thread_metric.message:
extra_configs:
- CONFIG_TM_MESSAGE=y
benchmark.thread_metric.preemptive:
extra_configs:
- CONFIG_TM_PREEMPTIVE=y
benchmark.thread_metric.synchronization:
extra_configs:
- CONFIG_TM_SYNCHRONIZATION=y

248
tests/benchmarks/thread_metric/thread_metric_readme.txt Normal file
View file

@ -0,0 +1,248 @@
Thread-Metric RTOS Test Suite
1. Thread-Metric Test Suite
The Thread-Metric test suite consists of 8 distinct RTOS
tests that are designed to highlight commonly used aspects
of an RTOS. The test measures the total number of RTOS events
that can be processed during a specific timer interval. A 30
second time interval is recommended.
1.1. Basic Processing Test
This is the baseline test consisting of a single thread. This
should execute the same on every operating system. Test values
from testing with different RTOS products should be scaled
relative to the difference between the values of this test.
1.2. Cooperative Scheduling Test
This test consists of 5 threads created at the same priority that
voluntarily release control to each other in a round-robin fashion.
Each thread will increment its run counter and then relinquish to
the next thread. At the end of the test the counters will be verified
to make sure they are valid (should all be within 1 of the same
value). If valid, the numbers will be summed and presented as the
result of the cooperative scheduling test.
1.3. Preemptive Scheduling Test
This test consists of 5 threads that each have a unique priority.
In this test, all threads except the lowest priority thread are
left in a suspended state. The lowest priority thread will resume
the next highest priority thread. That thread will resume the
next highest priority thread and so on until the highest priority
thread executes. Each thread will increment its run count and then
call thread suspend. Eventually the processing will return to the
lowest priority thread, which is still in the middle of the thread
resume call. Once processing returns to the lowest priority thread,
it will increment its run counter and once again resume the next
highest priority thread - starting the whole process over once again.
1.4. Interrupt Processing Test
This test consists of a single thread. The thread will cause an
interrupt (typically implemented as a trap), which will result in
a call to the interrupt handler. The interrupt handler will
increment a counter and then post to a semaphore. After the
interrupt handler completes, processing returns to the test
thread that initiated the interrupt. The thread then retrieves
the semaphore set by the interrupt handler, increments a counter
and then generates another interrupt.
1.5. Interrupt Preemption Processing Test
This test is similar to the previous interrupt test. The big
difference is the interrupt handler in this test resumes a
higher priority thread, which causes thread preemption.
1.6. Message Processing Test
This test consists of a thread sending a 16 byte message to a
queue and retrieving the same 16 byte message from the queue.
After the send/receive sequence is complete, the thread will
increment its run counter.
1.7. Synchronization Processing Test
This test consists of a thread getting a semaphore and then
immediately releasing it. After the get/put cycle completes,
the thread will increment its run counter.
1.8. RTOS Memory allocation
This test consists of a thread allocating a 128-byte block and
releasing the same block. After the block is released, the thread
will increment its run counter.
2. Zephyr Modifications
A few minor modifications have been made to the Thread-Metric source
code to resolve some minor issues found during porting.
2.1. tm_main() -> main()
Zephyr's version of this benchmark has modified the original tm_main()
to become main().
2.2. Thread entry points
Thread entry points used by Zephyr have a different function signature
than that used by the original Thread-Metric code. These functions
have been updated to match Zephyr's.
2.3. Reporting thread
Zephyr's version does not spawn a reporting thread. Instead it calls
the reporting function directly. This helps ensure that the test
operates correctly on QEMU platorms.
2.4. Directory structure
Each test has been converted to its own project. This has necessitated
some minor changes to the directory structure as compared to the
original version of this benchmark.
The source code to the Thread-Metric test suite is organized into
the following files:
File Meaning
tm_basic_processing_test.c Basic test for determining board
processing capabilities
tm_cooperative_scheduling_test.c Cooperative scheduling test
tm_preemptive_scheduling_test.c Preemptive scheduling test
tm_interrupt_processing_test.c No-preemption interrupt processing
test
tm_interrupt_preemption_processing_test.c Interrupt preemption processing
test
tm_message_processing_test.c Message exchange processing test
tm_synchronization_processing_test.c Semaphore get/put processing test
tm_memory_allocation_test.c Basic memory allocation test
tm_porting_layer_zephyr.c Specific porting layer source
code for Zephyr
3 Porting
3.1 Porting Layer
As for the porting layer defined in tm_porting_layer_template.c, this file contain
shell services of the generic RTOS services used by the actual tests. The
shell services provide the mapping between the tests and the underlying RTOS.
The following generic API's are required to map any RTOS to the performance
measurement tests:
void tm_initialize(void (*test_initialization_function)(void));
This function is typically called by the application from its
main() function. It is responsible for providing all the RTOS
initialization, calling the test initialization function as
specified, and then starting the RTOS.
int tm_thread_create(int thread_id, int priority, void (*entry_function)(void));
This function creates a thread of the specified priority where 1 is
the highest and 16 is the lowest. If successful, TM_SUCCESS
returned. If an error occurs, TM_ERROR is returned. The created thread
is not started.
int tm_thread_resume(int thread_id);
This function resumes the previously created thread specified by
thread_id. If successful, a TM_SUCCESS is returned.
int tm_thread_suspend(int thread_id);
This function suspend the previously created thread specified by
thread_id. If successful, a TM_SUCCESS is returned.
void tm_thread_relinquish(void);
This function lets all other threads of same priority execute
before the calling thread runs again.
void tm_thread_sleep(int seconds);
This function suspends the calling thread for the specified
number of seconds.
int tm_queue_create(int queue_id);
This function creates a queue with a capacity to hold at least
one 16-byte message. If successful, a TM_SUCCESS is returned.
int tm_queue_send(int queue_id, unsigned long *message_ptr);
This function sends a message to the previously created queue.
If successful, a TM_SUCCESS is returned.
int tm_queue_receive(int queue_id, unsigned long *message_ptr);
This function receives a message from the previously created
queue. If successful, a TM_SUCCESS is returned.
int tm_semaphore_create(int semaphore_id);
This function creates a binary semaphore. If successful, a
TM_SUCCESS is returned.
int tm_semaphore_get(int semaphore_id);
This function gets the previously created binary semaphore.
If successful, a TM_SUCCESS is returned.
int tm_semaphore_put(int semaphore_id);
This function puts the previously created binary semaphore.
If successful, a TM_SUCCESS is returned.
int tm_memory_pool_create(int pool_id);
This function creates a memory pool able to satisfy at least one
128-byte block of memory. If successful, a TM_SUCCESS is returned.
int tm_memory_pool_allocate(int pool_id, unsigned char **memory_ptr);
This function allocates a 128-byte block of memory from the
previously created memory pool. If successful, a TM_SUCCESS
is returned along with the pointer to the allocated memory
in the "memory_ptr" variable.
int tm_memory_pool_deallocate(int pool_id, unsigned char *memory_ptr);
This function releases the previously allocated 128-byte block
of memory. If successful, a TM_SUCCESS is returned.
2.2 Porting Requirements Checklist
The following requirements are made in order to ensure fair benchmarks
are achieved on each RTOS performing the test:
1. Time period should be 30 seconds. This will ensure the printf
processing in the reporting thread is insignificant.
* Zephyr : Requirement met.
2. The porting layer services are implemented inside of
tm_porting_layer_[RTOS].c and NOT as macros.
* Zephyr : Requirements met.
3. The tm_thread_sleep service is implemented by a 10ms RTOS
periodic interrupt source.
* Zephyr : Requirement met. System tick rate = 100 Hz.
4. Locking regions of the tests and/or the RTOS in cache is
not allowed.
* Zephyr : Requirement met.
5. The Interrupt Processing and Interrupt Preemption Processing tests
require an instruction that generates an interrupt. Please refer
to tm_porting_layer.h for an example implementation.
* Zephyr : Requirement met. See irq_offload().