Change the http timeout mechanism to use poll instead of shutdown.
This should fix a problem where the shutdown will be called in a
different thread context which can lead to deadlocks on certain
driver implementations like offloaded modem drivers.
Fixes#53967
Signed-off-by: Wouter Cappelle <wouter.cappelle@crodeon.com>
Some minor housekeeping prior to adding an http server
implementation. There are already a number of http headers
and that number will likely increase with subsequent work.
Moving them into a common directory cleans up the
`include/net` directory a bit.
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
Logging v1 has been removed and log_strdup wrapper function is no
longer needed. Removing the function and its use in the tree.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
In order to bring consistency in-tree, migrate all subsystems code to
the new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
http_client_req() was supposed to return the number of bytes sent as a
HTTP request. The return value was not riht however due to some bugs in
helper functions:
* http_send_data() returned the current buffer position istead of the
number of bytes actually sent. This could result in counting the same
data into the total request size several times. A helper variable was
added to track how many bytes were actually sent to the network.
* http_flush_data() forwarded the return value of sendall() helper
function. That function however did not return number of bytes sent,
but 0 or a negative error code.
Additionally, change the return type of sendall() function - according
to standard the ssize_t type is only capable of holding -1 negative
value, but the function could return the full range of negative errno
values. Use int instead.
Signed-off-by: Robert Lubos <robert.lubos@nordicsemi.no>
Following #42026, the body_start pointer now points to the
start of the body fragment in the recv_buffer as long as there
is body in it, either entirely or partially.
Rename the body_start to body_frag_start to better reflect
what it represents.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Adds an official behavior in response to null response from HTTP
endpoint.
Fixes#42988
Signed-off-by: Georges Oates_Larsen <georges.larsen@nordicsemi.no>
So far close() was called on underlying socket when timeout has expired.
This is wrong in several ways. First of all POSIX specification of
close() does not specify what should happen on blocking recv() call and
different systems have different behaviors (e.g. Linux does not wake up
recv() caller if there was no new incoming data, while other systems
might wakeup recv() caller immediately). Another (and much more severe)
problem is that HTTP client user does not know whether underlying socket
was already closed or not after HTTP request has finished. As a result
it was not clear whether close() should be called by HTTP client user.
Use shutdown(..., SHUT_RD) in internal HTTP client implementation, so
that recv() is woken up immediately with 0 as result (which means EOF).
This will allow to gracefully handle timeouts and make it clear that it
is application responsibility to always call close() after HTTP
request (successful or not).
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
A common pattern here was to take the work item as the subfield of a
containing object. But the contained field is not a k_work, it's a
k_work_delayable.
Things were working only because the work field was first, so the
pointers had the same value. Do things right and fix things to
produce correct code if/when that field ever moves within delayable.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Set `body_start` pointer regardless of the body position in the recv
buffer. In result, the pointer shall indicate correctly position of the
body for each fragment, it's also explicit now that if the pointer is
not set for a fragment, there's no body in that particular fragment.
Signed-off-by: Robert Lubos <robert.lubos@nordicsemi.no>
HTTP_DATA_FINAL was incorrectly notified in case Content Length field
was present in the HTTP respone - in such case it was set for every
response fragment, not only the last one.
Fix this by relying on `message_complete` flag instead of
`http_should_keep_alive()` function to determine whether to notify
HTTP_DATA_FINAL or not. As the HTTP parser calls the
`on_message_complete()` callback in either case (response is chunked or
not), this seems to be a more reasonable apporach to determine whether
the fragment is final or not.
Additinally, instead of calling response callback for
`on_body`/`on_message_complete` separately, call it directly from
`http_wait_data()` function, after the parsing round. This fixes the
case when headers were not reported correctly when the provided buffer
was smaller than the total headers length, resulting in corrupted data
being reported to the user.
Signed-off-by: Robert Lubos <robert.lubos@nordicsemi.no>
Using zsock_ in http_client instead of the POSIX API versions of the
functions allows the usage of http_client in combination with
CONFIG_POSIX_API.
Signed-off-by: Benedikt Schmidt <benedikt.schmidt@embedded-solutions.at>
Call on_status if the Reason-Phrase is not provided.
This allows for the numeric status code to be set.
Also, ensure the numeric status code is always set
in on_status, not just if the specific callback is set.
Signed-off-by: Justin Morton <justin.morton@nordicsemi.no>
Add numeric http status code to the response struct to allow for
easier processing by the caller. Textual status already exists.
Signed-off-by: Justin Morton <justin.morton@nordicsemi.no>
Replace all existing deprecated API with the recommended alternative.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Bug fix and improved `payload` handling in `http_client_req`.
Changes to `http_client_req` behaviour:
If the user provides `payload_len` it is used to generate the
`Content-Length` header. This is done even if `payload_cb` is used to
provide the actual data. If no `payload_len` is specified then no
`Content-Length` is generated.
If `payload_cb` is provided it is called to send the payload data.
Otherwise `payload` is used as the payload buffer and sent. If
`payload_len` is not zero, it is used as the size of `payload`.
Otherwise `payload` is assumed to be a string and `strlen` is used to
determine its size. This is to maintain current behaviour and not break
existing samples.
Fixes#24431
Signed-off-by: Arvin Farahmand <arvinf@ip-logix.com>
Use k_timeout_t internally, no change to user API.
Clarify the documentation of the timeout parameter that it is
in milliseconds.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The old legacy APIs use net-app library and as that is being
removed, then the dependencies need to be removed also.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Remove network specific default and max log level setting
and start to use the zephyr logging values for those.
Remove LOG_MODULE_REGISTER() from net_core.h and place the
calls into .c files. This is done in order to avoid weird
compiler errors in some cases and to make the code look similar
as other subsystems.
Fixes#11343Fixes#11659
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
As the debugging print calls are async, all the strings that might
be overwritten must use log_strdup() which will create a copy
of the printable string.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The return of memset is never checked. This patch explicitly ignore
the return to avoid MISRA-C violations.
The only directory excluded directory was ext/* since it contains
only imported code.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
net_app_ctx maintains multiple net contexts(net_ctx). But when http
api's wants to reply or send some data, its always choose the first
net_context in the array, which is not correct always.
net_app_get_net_pkt_with_dst() api will select proper context
based on destination address. So with the help of new api in
net_app, http can select proper context and send packets. To
achieve this, desination address is provided in http_recv_cb_t
and http_connect_cb_t callbacks. Also chaged relevant API's to
provide destination address in http message preparation methods.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@linux.intel.com>
Logic for sending chunks of data is incompatible with adding
Content-Length: header.
Per https://tools.ietf.org/html/rfc7230#section-3.3.1:
"A sender MUST NOT send a Content-Length header field in any
message that contains a Transfer-Encoding header field."
Going a bit further in my mind: also don't send Transfer-Encoded
chunked data either when the Content-Length header is present.
In general, there will be problems if the http client library
makes payload changes without the user code knowing about it.
This patch removes the use of http_send_chunk() from the new
HTTP client code and instead sends the payload directly to
http_prepare_and_send()
This fixes an issue where every available buffer would be allocated
with repeating payload data because the for loop in http_request()
wasn't ending until we ran out of memory.
NOTE: This patch was previously applied but was lost when
commit d1675bf3e6 ("net: http: Remove the old legacy API")
moved code around.
Signed-off-by: Michael Scott <michael@opensourcefoundries.com>
We should not use the user suppied timeout setting in
http_client_send_req() for the connection timeout. In the
previous API the call to tcp_connect() used
CONFIG_HTTP_CLIENT_NETWORK_TIMEOUT as the timeout setting.
Let's do that here too.
This fixes -ETIMEDOUT error generation when using K_NO_WAIT
for http_client_send_req().
NOTE: This patch was previously applied but was lost when
commit d1675bf3e6 ("net: http: Remove the old legacy API")
moved code around.
Signed-off-by: Michael Scott <michael@opensourcefoundries.com>
In previous version of the HTTP API, commit 8ebaf29927 ("net: http:
dont timeout on HTTP requests w/o body") fixed handling of HTTP
responses without body content.
For the new API, let's add a specific fix for when PUT/POST requests
are responded to with just the status code.
Signed-off-by: Michael Scott <michael@opensourcefoundries.com>
Recent commit fb7f6cfa97 ("net: lib: http: Fix invalid pointer
body_start") introduced logic to reset the response body_start pointer
when the response buffer was reused.
This check needs to be fixed so that it doesn't arbitrarily change
body_start when not needed.
The problem with the current check can be demonstrated by not setting
a response callback for request which generates a large response
spanning multiple packets.
In this case body_start is still valid (not reusing the response buffer
because there is no callback set), but it will be changed when the 2nd
packet is received and the "at" marker is located at the head of the
new packet (!= response_buffer).
Signed-off-by: Michael Scott <michael.scott@linaro.org>
The body_start field at http_client_ctx.rsp is used to check if this
fragment contains (a part of) headers or not.
If the device recived more than one fragment in one http response,
may cause re-use of the result buffer in function on_body().
Once the device re-use the result buffer, the body_start that point
to this buffer address will no longer be valid.
Signed-off-by: Ding Tao <miyatsu@qq.com>
Currently this is defined as a k_thread_stack_t pointer.
However this isn't correct, stacks are defined as arrays. Extern
references to k_thread_stack_t doesn't work properly as the compiler
treats it as a pointer to the stack array and not the array itself.
Declaring as an unsized array of k_thread_stack_t doesn't work
well either. The least amount of confusion is to leave out the
pointer/array status completely, use pointers for function prototypes,
and define K_THREAD_STACK_EXTERN() to properly create an extern
reference.
The definitions for all functions and struct that use
k_thread_stack_t need to be updated, but code that uses them should
be unchanged.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
If the caller of http_client_send_req() sets the timeout to
K_NO_WAIT, then the function would still wait for a while before
returning to the caller.
Jira: ZEP-2624
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The original commit 8ebaf29927 ("net: http: dont timeout
on HTTP requests w/o body") was intended to handle a case
where an HTTP response had been retrieved from the server but
the HTTP parser couldn't meet the criteria for calling
"on_message_complete". For example, a POST to a REST API
where the server doesn't return anything but an HTTP
status code.
It was a really bad idea to check a semaphore count. There
is a lot of kernel logic built into semaphores and how the
count is adjusted. The assumption that the value is 0
after the k_sem_give() is incorrect. It's STILL 0 if
something is pending with a k_sem_take(). By the time
k_sem_give() is done executing the other thread has now
been kicked and the count is back to 0.
This caused the original check to always pass and in turn
breakage was noticed in the http_client sample.
Let's do this the right way by setting a flag when
on_message_complete is called and if that flag is not set
by the time we reach recv_cb, let's give back the semaphore
to avoid a timeout.
Jira: ZEP-2561
Signed-off-by: Michael Scott <michael.scott@linaro.org>
POSIX requires struct sockaddr's field to be named "sa_family"
(not just "family"):
http://pubs.opengroup.org/onlinepubs/009696699/basedefs/sys/socket.h.html
This change allows to port POSIX apps easier (including writing
portable apps using BSD Sockets compatible API).
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Currently, the HTTP_NETWORK_TIMEOUT setting is hard-coded as 20 seconds.
Not every application may want to wait that long, so let's change this
to a CONFIG option: CONFIG_HTTP_CLIENT_NETWORK_TIMEOUT
NOTE: This also removes HTTP_NETWORK_TIMEOUT from the public http.h
include file. It was not being used externally to HTTP client sources.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
A TCP FIN message is passed on to user apps as a tcp_received_callback
with a NULL pkt parameter. This means the connection is closing and
the app should do whatever cleanup it needs as there will be no further
callbacks for the current TCP connection.
Currently, if a HTTP client request doesn't receive a "body" which
the HTTP parser can use to trigger on_message_complete, then the request
will end up timing out and most apps will think an error has occurred.
Instead, let's handle the TCP FIN message and return the waiting
semaphore, leaving the app to deal with whatever has been set in the
current HTTP context response data (IE: http_status).
This fixes using HTTP client to send POST data to servers which
only respond with HTTP_OK status and no body.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
RFC-7230 "HTTP/1.1 Message Syntax and Routing" Section 5.4
describes the "Host" header formatting. If Zephyr user
specifies a host string as a part of the HTTP client request
structure, we end up sending an incorrect HTTP header due
to a missing "Host :" text.
Fix this by prepending "Host: " to the header data before
the user supplied host string.
Signed-off-by: Michael Scott <michael.scott@linaro.org>
Historically, stacks were just character buffers and could be treated
as such if the user wanted to look inside the stack data, and also
declared as an array of the desired stack size.
This is no longer the case. Certain architectures will create a memory
region much larger to account for MPU/MMU guard pages. Unfortunately,
the kernel interfaces treat both the declared stack, and the valid
stack buffer within it as the same char * data type, even though these
absolutely cannot be used interchangeably.
We introduce an opaque k_thread_stack_t which gets instantiated by
K_THREAD_STACK_DECLARE(), this is no longer treated by the compiler
as a character pointer, even though it really is.
To access the real stack buffer within, the result of
K_THREAD_STACK_BUFFER() can be used, which will return a char * type.
This should catch a bunch of programming mistakes at build time:
- Declaring a character array outside of K_THREAD_STACK_DECLARE() and
passing it to K_THREAD_CREATE
- Directly examining the stack created by K_THREAD_STACK_DECLARE()
which is not actually the memory desired and may trigger a CPU
exception
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit adds http_client_set_net_pkt_pool() function that allows
caller to define net_buf pool that is used when sending a TCP packet.
This is needed for those technologies like Bluetooth or 802.15.4 which
compress the IPv6 header during send.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Coverity reported false positives, add comment about these in
the code.
Jira: ZEP-2344
Jira: ZEP-2345
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
The global mbedtls heap is set automatically now so no need to
set it individually in the http library.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Add HTTPS support into http-client library. The init of the
HTTPS client connection is different compared to HTTP client,
but the actual HTTP request sending is using the same API as
HTTP client.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
