shell: Fix uart_rx_handle byte processing when ring buffer full

In case when shell ring buffer gets full the data may still be taken
from the UART fifo, byte by byte, and accepted by the SMP back-end,
if the back-end has been enabled.

Unfortunately the uart_rx_handle failed to check if it has been
successfull in reading a byte from the fifo, before passing it to
the SMP for processing, which could lead to processing random stack data
as a part of an SMP frame.

Additinally, when the SMP would have accepted the byte,
the uart_rx_handle would fail to signal shell_thread, that the SMP has
internally buffered data that could be processed.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
This commit is contained in:
Dominik Ermel 2020-08-18 21:26:48 +00:00 committed by Ioannis Glaropoulos
commit d8c7fc8be6

View file

@ -78,10 +78,13 @@ static void uart_rx_handle(struct device *dev,
rd_len = uart_fifo_read(dev, &dummy, 1);
#ifdef CONFIG_MCUMGR_SMP_SHELL
/* Divert this byte from shell handling if it
* is part of an mcumgr frame.
/* If successful in getting byte from the fifo, try
* feeding it to SMP as a part of mcumgr frame.
*/
smp_shell_rx_byte(&sh_uart->ctrl_blk->smp, dummy);
if (rd_len != 0 &&
smp_shell_rx_byte(&sh_uart->ctrl_blk->smp, dummy)) {
new_data = true;
}
#endif /* CONFIG_MCUMGR_SMP_SHELL */
}
} while (rd_len && (rd_len == len));