2506081045

Committed 07 May 2026 02:30AM UTC coverage: 81.442%. Remained the same

Build # 2506081045

Build Type

push

gitlab-ci

Committed by

lhoward

Commit Message

fix: common/error: include <ostream> for operator<<(ostream&, string&)

src/common/error.cpp defines

    std::ostream &operator<<(std::ostream &os, const Error &err) {
        os << err.String();
        ...
    }

err.String() returns std::string, so this relies on the free
function operator<<(std::ostream&, const std::string&) being a
complete definition at the call site, not just a forward declaration.

error.cpp only includes <common/error.hpp>, which in turn includes
<string> and <system_error>. Under libstdc++ that chain happens to
drag in the full body of the string ostream operator, so the call
gets inlined and the link succeeds. Under libc++, <string> only
provides a forward declaration (marked _LIBCPP_HIDE_FROM_ABI); the
body lives in <ostream>/<__ostream/basic_ostream.h>. Without that
include, the compiler emits an out-of-line call to a hidden-
visibility symbol that never gets defined and the link fails:

  undefined reference to 'std::operator<<[abi:...]<char, ...>(
    basic_ostream<char,...>&, const basic_string<char,...>&)'

Include <ostream> explicitly so the definition is reachable regardless
of which C++ standard library is in use.

Changelog: none
Signed-off-by: Luke Howard <lukeh@padl.com>

Coverage Stats

9185 of 11278 relevant lines covered (81.44%)

20238.44 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

80.57

/src/mender-update/update_module/v3/update_module_download.cpp

// Copyright 2023 Northern.tech AS
//
//    Licensed under the Apache License, Version 2.0 (the "License");
//    you may not use this file except in compliance with the License.
//    You may obtain a copy of the License at
//
//        http://www.apache.org/licenses/LICENSE-2.0
//
//    Unless required by applicable law or agreed to in writing, software
//    distributed under the License is distributed on an "AS IS" BASIS,
//    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
//    See the License for the specific language governing permissions and
//    limitations under the License.

#include <mender-update/update_module/v3/update_module.hpp>

#include <mender-update/progress_reader/progress_reader.hpp>

#include <common/events.hpp>
#include <common/events_io.hpp>
#include <common/log.hpp>
#include <common/path.hpp>
#include <common/processes.hpp>

namespace mender {
namespace update {
namespace update_module {
namespace v3 {

namespace log = mender::common::log;
namespace path = mender::common::path;
namespace processes = mender::common::processes;
namespace progress = mender::update::progress;

/* A note on how streams tree works in the download process:

According to Update Module protocol documentation, Update Modules need to do one full read of the
stream-next pipe, then a full read of a file that is mentioned in this pipe (payload pipe), and so
on, until they get a zero-length read from stream-next. This raises the question: is is possible
that stream-next pipe or the payload pipe are not  ready or even not exist when an Update Module
tries to read from it? How do we design against that?

The answer is: with current implementation it is not possible for such race conditions to occur.
The full download flow is as follows:

----------------------------------------------------------------------------------------------------
1. Before calling an Update Module with Download command, mender (meaning mender-orchestrator) first
creates a named pipe "stream-next", but doesn't open it for writing,
(UpdateModule::PrepareStreamNextPipe).

2. Now we have two possibilities:

2a. Mender is faster

mender opens the pipe for writing (UpdateModule::OpenStreamNextPipe), but there's no reader yet
(Update Module did not connect to the pipe): mender blocks until it can write to the pipe. As soon
as the Update Module connects, mender writes data to the pipe.

2b. Update Module is faster

Update Module tries to read from the pipe, but mender did not connect to the pipe for writing yet.
Because named pipes are blocking by default, Update Module will block, waiting for data to come from
the pipe. As soon as mender connects to the pipe, it will immediately start writing to it
(UpdateModule::OpenStreamNextPipe) and Update Module will begin reading from it.

There's no possibility of an Update Module getting an error saying that the pipe does not exist
instead, as it was created before the Update Module was called with a Download command. The Update
Module will always block on a pipe that exists but does not have a writer yet.

3. Update Module has read stream-next, receiving the name of the payload pipe.

When mender has written everything to stream-next, it does not close the pipe for writing yet. This
causes the Update Module to wait for additional data from the pipe indefinitely. Instead, mender
first calls UpdateModule::PrepareAndOpenStreamPipe where it creates the payload pipe and connects to
it.

Only after the payload pipe exists, mender closes the stream-next pipe, causing the Update Module to
unblock. Notice that thanks to this, there is no possibility of the Update Module being so fast that
it tries to read from the payload pipe before mender has a chance to create it - there's no
possibility of "file does not exist" happening. Mender is now free to start writing to the payload
pipe anytime
- it doesn't matter if the Update Module tries to read from it before mender is ready, as it will
just block on the pipe, waiting for mender to connect to it.

4. Update Module finished reading from the payload pipe. Tries to read again from stream-next.

Mender closes the payload pipe.
Because mender only closed stream-next pipe, but did not delete it, the Update Module won't get
"file does not exist" under any circumstances. Instead, it blocks on stream-next until mender is
ready to write to it. Mender is now free to open the stream-next for writing anytime, and the Update
Module will wait for it. As soon as mender connects to the pipe for writing, the Update Module
ublocks and starts reading from it.

5. Steps 2 - 4 repeat as many times as there are payloads and thus payload pipes.
Mender marks the download as completed internally (UpdateModule::StreamNextOpenHandler).

6. As soon as the Update Module tries to read from the stream-next pipe again, mender does not write
anything to it, and instead just connects to the pipe and immediately closes its connection. This
causes the Update Module to unblock and get a zero-length read from stream-next.

Update Module knows that there are no more payloads. Mender closes the stream-next pipe
(UpdateModule::StreamNextOpenHandler) and is free to continue to the next command.
----------------------------------------------------------------------------------------------------
Summary:
There are no places where it is possible for a race condition to occur. If the Update Module is
faster, it will wait for mender. If mender is faster, it will wait for the Update Module. As the
named pipes are created beforehand, there's no possibility of the Update Module getting "file does
not exist" error. This all assumes that a given Update Module implements the API correctly. If one
were to use non-blocking reads from the pipes in an Update Module, then it could break. API
explicitly says that "full reads" are required so if an Update Module doesn't block while there's a
writer on the other end of the pipe - it does not implement the API correctly.

Note:
What happens if an Update Module is broken, for example it does not read from stream-next the last
time, where it would receive a zero-length read? As soon as the Update Module returns after being
called with Download command, mender calls UpdateModule::ProcessEndedHandler where, if it sees that
not all reads were completed, it marks the download process as failed and closes all pipes
unblocking itself.
*/

void UpdateModule::StartDownloadProcess() {
        string download_command = "Download";
        if (download_->downloading_with_sizes_) {
                download_command = "DownloadWithFileSizes";
        }
        log::Debug(
                "Calling Update Module with command `" + update_module_path_ + " " + download_command + " "
                + update_module_workdir_ + "`.");
        download_->proc_ = make_shared<procs::Process>(
                vector<string> {update_module_path_, download_command, update_module_workdir_});

        download_->proc_->SetWorkDir(update_module_workdir_);

        auto err = PrepareStreamNextPipe();
        if (err != error::NoError) {
                DownloadErrorHandler(err);
                return;
        }

        processes::OutputHandler stdout_handler {"Update Module output (stdout): "};
        processes::OutputHandler stderr_handler {"Update Module output (stderr): "};

        err = download_->proc_->Start(stdout_handler, stderr_handler);
        if (err != error::NoError) {
                DownloadErrorHandler(GetProcessError(err));
                return;
        }

        err = download_->proc_->AsyncWait(
                download_->event_loop_,
                [this](error::Error err) {
                        if (err.code == make_error_condition(errc::timed_out)) {
                                DownloadTimeoutHandler();
                        } else {
                                ProcessEndedHandler(err);
                        }
                },
                chrono::seconds(ctx_.GetConfig().module_timeout_seconds));
        if (err != error::NoError) {
                DownloadErrorHandler(err);
                return;
        }

        DownloadErrorHandler(OpenStreamNextPipe(
                [this](io::ExpectedAsyncWriterPtr writer) { StreamNextOpenHandler(writer); }));
}

void UpdateModule::StreamNextOpenHandler(io::ExpectedAsyncWriterPtr writer) {
        if (!writer) {
                DownloadErrorHandler(writer.error());
                return;
        }
        download_->stream_next_writer_ = writer.value();

        download_->module_has_started_download_ = true;

        auto reader = download_->payload_.Next();
        if (!reader) {
                if (reader.error().code
                        == artifact::parser_error::MakeError(
                                   artifact::parser_error::NoMorePayloadFilesError, "")
                                   .code) {
                        download_->module_has_finished_download_ = true;
                        log::Debug("Update Module finished all downloads");
                        EndStreamNext();
                } else {
                        DownloadErrorHandler(reader.error());
                }
                return;
        }
        auto payload_reader = make_shared<artifact::Reader>(std::move(reader.value()));

        auto progress_reader = make_shared<progress::Reader>(payload_reader, payload_reader->Size());

        download_->current_payload_reader_ =
                make_shared<events::io::AsyncReaderFromReader>(download_->event_loop_, progress_reader);
        download_->current_payload_name_ = payload_reader->Name();
        download_->current_payload_size_ = payload_reader->Size();

        auto stream_path =
                path::Join(update_module_workdir_, string("streams"), download_->current_payload_name_);
        auto exp_path_is_safe =
                path::IsWithinOrEqual(stream_path, path::Join(update_module_workdir_, string("streams")));
        if (!exp_path_is_safe.has_value()) {
                DownloadErrorHandler(exp_path_is_safe.error().WithContext(
                        "Error checking if path is equal to or within directory"));
                return;
        }
        if (!exp_path_is_safe.value()) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::invalid_argument),
                        "Error downloading payload: Provided payload file (" + download_->current_payload_name_
                                + ") would point outside work directory when extracted."));
                return;
        }

        auto err = PrepareAndOpenStreamPipe(
                stream_path, [this](io::ExpectedAsyncWriterPtr writer) { StreamOpenHandler(writer); });
        if (err != error::NoError) {
                DownloadErrorHandler(err);
                return;
        }

        string stream_next_string;
        if (download_->downloading_with_sizes_) {
                stream_next_string = path::Join("streams", download_->current_payload_name_) + " "
                                                         + to_string(download_->current_payload_size_);
        } else {
                stream_next_string = path::Join("streams", download_->current_payload_name_);
        }
        size_t entry_size = stream_next_string.size() + 1;
        if (entry_size > download_->buffer_.size()) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::no_buffer_space), "Payload name is too large for buffer"));
                return;
        }
        copy(stream_next_string.begin(), stream_next_string.end(), download_->buffer_.begin());
        download_->buffer_[entry_size - 1] = '\n';
        DownloadErrorHandler(download_->stream_next_writer_->AsyncWrite(
                download_->buffer_.begin(),
                download_->buffer_.begin() + entry_size,
                [this, entry_size](io::ExpectedSize result) {
                        StreamNextWriteHandler(entry_size, result);
                }));
}

void UpdateModule::StreamOpenHandler(io::ExpectedAsyncWriterPtr writer) {
        if (!writer) {
                DownloadErrorHandler(writer.error());
                return;
        }
        download_->current_stream_writer_ = writer.value();

        DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(
                download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {
                        PayloadReadHandler(result);
                }));
}

void UpdateModule::StreamNextWriteHandler(size_t expected_n, io::ExpectedSize result) {
        // Close stream-next writer.
        download_->stream_next_writer_.reset();
        if (!result) {
                DownloadErrorHandler(result.error());
        } else if (expected_n != result.value()) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::io_error),
                        "Unexpected number of written bytes to stream-next"));
        }
}

void UpdateModule::PayloadReadHandler(io::ExpectedSize result) {
        if (!result) {
                // Close streams.
                download_->current_stream_writer_.reset();
                download_->current_payload_reader_.reset();
                DownloadErrorHandler(result.error());
        } else if (result.value() > 0) {
                DownloadErrorHandler(download_->current_stream_writer_->AsyncWrite(
                        download_->buffer_.begin(),
                        download_->buffer_.begin() + result.value(),
                        [this, result](io::ExpectedSize write_result) {
                                StreamWriteHandler(0, result.value(), write_result);
                        }));
        } else {
                // Close streams.
                download_->current_stream_writer_.reset();
                download_->current_payload_reader_.reset();

                if (download_->downloading_to_files_) {
                        StartDownloadToFile();
                } else {
                        DownloadErrorHandler(OpenStreamNextPipe(
                                [this](io::ExpectedAsyncWriterPtr writer) { StreamNextOpenHandler(writer); }));
                }
        }
}

void UpdateModule::StreamWriteHandler(size_t offset, size_t expected_n, io::ExpectedSize result) {
        if (!result) {
                DownloadErrorHandler(result.error());
        } else if (result.value() == 0 || result.value() > expected_n) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::io_error),
                        "Unexpected number of written bytes to download stream"));
        } else if (result.value() < expected_n) {
                auto new_offset = offset + result.value();
                auto new_expected = expected_n - result.value();
                DownloadErrorHandler(download_->current_stream_writer_->AsyncWrite(
                        download_->buffer_.begin() + new_offset,
                        download_->buffer_.begin() + new_offset + new_expected,
                        [this, new_offset, new_expected](io::ExpectedSize write_result) {
                                StreamWriteHandler(new_offset, new_expected, write_result);
                        }));
        } else {
                download_->written_ += result.value();
                log::Trace("Wrote " + to_string(download_->written_) + " bytes to Update Module");
                DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(
                        download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {
                                PayloadReadHandler(result);
                        }));
        }
}

void UpdateModule::EndStreamNext() {
        // Empty write.
        DownloadErrorHandler(download_->stream_next_writer_->AsyncWrite(
                download_->buffer_.begin(), download_->buffer_.begin(), [this](io::ExpectedSize result) {
                        if (!result) {
                                DownloadErrorHandler(result.error());
                        } else {
                                DownloadErrorHandler(error::NoError);
                        }
                        // Close writer.
                        download_->stream_next_writer_.reset();
                        // No further action necessary. Now we just need to wait for the process to finish.
                }));
}

void UpdateModule::DownloadErrorHandler(const error::Error &err) {
        if (err != error::NoError) {
                EndDownloadLoop(err);
        }
}

void UpdateModule::EndDownloadLoop(const error::Error &err) {
        download_->download_finished_handler_(err);
}

void UpdateModule::DownloadTimeoutHandler() {
        download_->proc_->EnsureTerminated();
        EndDownloadLoop(error::Error(
                make_error_condition(errc::timed_out), "Update Module Download process timed out"));
}

void UpdateModule::ProcessEndedHandler(error::Error err) {
        if (err != error::NoError) {
                err = GetProcessError(err);
                DownloadErrorHandler(error::Error(
                        err.code, "Download: Update Module returned non-zero status: " + err.message));
        } else if (download_->module_has_finished_download_) {
                EndDownloadLoop(error::NoError);
        } else if (download_->module_has_started_download_) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::broken_pipe),
                        "Update Module started downloading, but did not finish"));
        } else {
                download_->downloading_to_files_ = true;
                download_->stream_next_opener_.reset();
                download_->current_stream_opener_.reset();
                err = DeleteStreamsFiles();
                if (err != error::NoError) {
                        DownloadErrorHandler(err);
                } else {
                        StartDownloadToFile();
                }
        }
}

void UpdateModule::StartDownloadToFile() {
        auto reader = download_->payload_.Next();
        if (!reader) {
                if (reader.error().code
                        == artifact::parser_error::MakeError(
                                   artifact::parser_error::NoMorePayloadFilesError, "")
                                   .code) {
                        log::Debug("Downloaded all files to `files` directory.");
                        EndDownloadLoop(error::NoError);
                } else {
                        DownloadErrorHandler(reader.error());
                }
                return;
        }
        auto payload_reader = make_shared<artifact::Reader>(std::move(reader.value()));
        download_->current_payload_reader_ =
                make_shared<events::io::AsyncReaderFromReader>(download_->event_loop_, payload_reader);
        download_->current_payload_name_ = payload_reader->Name();

        auto stream_path = path::Join(update_module_workdir_, string("files"));
        auto err = PrepareDownloadDirectory(stream_path);
        if (err != error::NoError) {
                DownloadErrorHandler(err);
                return;
        }

        stream_path = path::Join(stream_path, download_->current_payload_name_);
        auto exp_path_is_safe =
                path::IsWithinOrEqual(stream_path, path::Join(update_module_workdir_, string("files")));
        if (!exp_path_is_safe.has_value()) {
                DownloadErrorHandler(exp_path_is_safe.error().WithContext(
                        "Error checking if path is equal to or within directory"));
                return;
        }
        if (!exp_path_is_safe.value()) {
                DownloadErrorHandler(error::Error(
                        make_error_condition(errc::invalid_argument),
                        "Error downloading payload: Provided payload file (" + download_->current_payload_name_
                                + ") would point outside work directory when extracted."));
                return;
        }

        auto current_stream_writer =
                make_shared<events::io::AsyncFileDescriptorWriter>(download_->event_loop_);
        err = current_stream_writer->Open(stream_path);
        if (err != error::NoError) {
                DownloadErrorHandler(err);
                return;
        }
        download_->current_stream_writer_ = current_stream_writer;

        DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(
                download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {
                        PayloadReadHandler(result);
                }));
}

} // namespace v3
} // namespace update_module
} // namespace update
} // namespace mender

1	// Copyright 2023 Northern.tech AS
2	//
3	// Licensed under the Apache License, Version 2.0 (the "License");
4	// you may not use this file except in compliance with the License.
5	// You may obtain a copy of the License at
6	//
7	// http://www.apache.org/licenses/LICENSE-2.0
8	//
9	// Unless required by applicable law or agreed to in writing, software
10	// distributed under the License is distributed on an "AS IS" BASIS,
11	// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12	// See the License for the specific language governing permissions and
13	// limitations under the License.
14
15	#include <mender-update/update_module/v3/update_module.hpp>
16
17	#include <mender-update/progress_reader/progress_reader.hpp>
18
19	#include <common/events.hpp>
20	#include <common/events_io.hpp>
21	#include <common/log.hpp>
22	#include <common/path.hpp>
23	#include <common/processes.hpp>
24
25	namespace mender {
26	namespace update {
27	namespace update_module {
28	namespace v3 {
29
30	namespace log = mender::common::log;
31	namespace path = mender::common::path;
32	namespace processes = mender::common::processes;
33	namespace progress = mender::update::progress;
34
35	/* A note on how streams tree works in the download process:
36
37	According to Update Module protocol documentation, Update Modules need to do one full read of the
38	stream-next pipe, then a full read of a file that is mentioned in this pipe (payload pipe), and so
39	on, until they get a zero-length read from stream-next. This raises the question: is is possible
40	that stream-next pipe or the payload pipe are not ready or even not exist when an Update Module
41	tries to read from it? How do we design against that?
42
43	The answer is: with current implementation it is not possible for such race conditions to occur.
44	The full download flow is as follows:
45
46	----------------------------------------------------------------------------------------------------
47	1. Before calling an Update Module with Download command, mender (meaning mender-orchestrator) first
48	creates a named pipe "stream-next", but doesn't open it for writing,
49	(UpdateModule::PrepareStreamNextPipe).
50
51	2. Now we have two possibilities:
52
53	2a. Mender is faster
54
55	mender opens the pipe for writing (UpdateModule::OpenStreamNextPipe), but there's no reader yet
56	(Update Module did not connect to the pipe): mender blocks until it can write to the pipe. As soon
57	as the Update Module connects, mender writes data to the pipe.
58
59	2b. Update Module is faster
60
61	Update Module tries to read from the pipe, but mender did not connect to the pipe for writing yet.
62	Because named pipes are blocking by default, Update Module will block, waiting for data to come from
63	the pipe. As soon as mender connects to the pipe, it will immediately start writing to it
64	(UpdateModule::OpenStreamNextPipe) and Update Module will begin reading from it.
65
66	There's no possibility of an Update Module getting an error saying that the pipe does not exist
67	instead, as it was created before the Update Module was called with a Download command. The Update
68	Module will always block on a pipe that exists but does not have a writer yet.
69
70	3. Update Module has read stream-next, receiving the name of the payload pipe.
71
72	When mender has written everything to stream-next, it does not close the pipe for writing yet. This
73	causes the Update Module to wait for additional data from the pipe indefinitely. Instead, mender
74	first calls UpdateModule::PrepareAndOpenStreamPipe where it creates the payload pipe and connects to
75	it.
76
77	Only after the payload pipe exists, mender closes the stream-next pipe, causing the Update Module to
78	unblock. Notice that thanks to this, there is no possibility of the Update Module being so fast that
79	it tries to read from the payload pipe before mender has a chance to create it - there's no
80	possibility of "file does not exist" happening. Mender is now free to start writing to the payload
81	pipe anytime
82	- it doesn't matter if the Update Module tries to read from it before mender is ready, as it will
83	just block on the pipe, waiting for mender to connect to it.
84
85	4. Update Module finished reading from the payload pipe. Tries to read again from stream-next.
86
87	Mender closes the payload pipe.
88	Because mender only closed stream-next pipe, but did not delete it, the Update Module won't get
89	"file does not exist" under any circumstances. Instead, it blocks on stream-next until mender is
90	ready to write to it. Mender is now free to open the stream-next for writing anytime, and the Update
91	Module will wait for it. As soon as mender connects to the pipe for writing, the Update Module
92	ublocks and starts reading from it.
93
94	5. Steps 2 - 4 repeat as many times as there are payloads and thus payload pipes.
95	Mender marks the download as completed internally (UpdateModule::StreamNextOpenHandler).
96
97	6. As soon as the Update Module tries to read from the stream-next pipe again, mender does not write
98	anything to it, and instead just connects to the pipe and immediately closes its connection. This
99	causes the Update Module to unblock and get a zero-length read from stream-next.
100
101	Update Module knows that there are no more payloads. Mender closes the stream-next pipe
102	(UpdateModule::StreamNextOpenHandler) and is free to continue to the next command.
103	----------------------------------------------------------------------------------------------------
104	Summary:
105	There are no places where it is possible for a race condition to occur. If the Update Module is
106	faster, it will wait for mender. If mender is faster, it will wait for the Update Module. As the
107	named pipes are created beforehand, there's no possibility of the Update Module getting "file does
108	not exist" error. This all assumes that a given Update Module implements the API correctly. If one
109	were to use non-blocking reads from the pipes in an Update Module, then it could break. API
110	explicitly says that "full reads" are required so if an Update Module doesn't block while there's a
111	writer on the other end of the pipe - it does not implement the API correctly.
112
113	Note:
114	What happens if an Update Module is broken, for example it does not read from stream-next the last
115	time, where it would receive a zero-length read? As soon as the Update Module returns after being
116	called with Download command, mender calls UpdateModule::ProcessEndedHandler where, if it sees that
117	not all reads were completed, it marks the download process as failed and closes all pipes
118	unblocking itself.
119	*/
120
121	void UpdateModule::StartDownloadProcess() {	136✔
122	string download_command = "Download";	136✔
123	if (download_->downloading_with_sizes_) {	136✔
124	download_command = "DownloadWithFileSizes";	3✔
125	}
126	log::Debug(	136✔
127	"Calling Update Module with command `" + update_module_path_ + " " + download_command + " "	136✔
128	+ update_module_workdir_ + "`.");	272✔
129	download_->proc_ = make_shared<procs::Process>(	136✔
130	vector<string> {update_module_path_, download_command, update_module_workdir_});	680✔
131
132	download_->proc_->SetWorkDir(update_module_workdir_);
133
134	auto err = PrepareStreamNextPipe();	136✔
135	if (err != error::NoError) {	136✔
136	DownloadErrorHandler(err);	×
137	return;
138	}
139
140	processes::OutputHandler stdout_handler {"Update Module output (stdout): "};	136✔
141	processes::OutputHandler stderr_handler {"Update Module output (stderr): "};	136✔
142
143	err = download_->proc_->Start(stdout_handler, stderr_handler);	272✔
144	if (err != error::NoError) {	136✔
145	DownloadErrorHandler(GetProcessError(err));	×
146	return;	×
147	}
148
149	err = download_->proc_->AsyncWait(	136✔
150	download_->event_loop_,
151	[this](error::Error err) {	136✔
152	if (err.code == make_error_condition(errc::timed_out)) {	135✔
153	DownloadTimeoutHandler();	2✔
154	} else {
155	ProcessEndedHandler(err);	266✔
156	}
157	},	135✔
158	chrono::seconds(ctx_.GetConfig().module_timeout_seconds));	272✔
159	if (err != error::NoError) {	136✔
160	DownloadErrorHandler(err);	×
161	return;
162	}
163
164	DownloadErrorHandler(OpenStreamNextPipe(	136✔
165	[this](io::ExpectedAsyncWriterPtr writer) { StreamNextOpenHandler(writer); }));	290✔
166	}	136✔
167
168	void UpdateModule::StreamNextOpenHandler(io::ExpectedAsyncWriterPtr writer) {	13✔
169	if (!writer) {	13✔
170	DownloadErrorHandler(writer.error());	×
171	return;	3✔
172	}
173	download_->stream_next_writer_ = writer.value();	13✔
174
175	download_->module_has_started_download_ = true;	13✔
176
177	auto reader = download_->payload_.Next();	13✔
178	if (!reader) {	13✔
179	if (reader.error().code	3✔
180	== artifact::parser_error::MakeError(	6✔
181	artifact::parser_error::NoMorePayloadFilesError, "")
182	.code) {
183	download_->module_has_finished_download_ = true;	3✔
184	log::Debug("Update Module finished all downloads");	3✔
185	EndStreamNext();	3✔
186	} else {
187	DownloadErrorHandler(reader.error());	×
188	}
189	return;	3✔
190	}
191	auto payload_reader = make_shared<artifact::Reader>(std::move(reader.value()));	10✔
192
193	auto progress_reader = make_shared<progress::Reader>(payload_reader, payload_reader->Size());	10✔
194
195	download_->current_payload_reader_ =
196	make_shared<events::io::AsyncReaderFromReader>(download_->event_loop_, progress_reader);	20✔
197	download_->current_payload_name_ = payload_reader->Name();	20✔
198	download_->current_payload_size_ = payload_reader->Size();	10✔
199
200	auto stream_path =
201	path::Join(update_module_workdir_, string("streams"), download_->current_payload_name_);	10✔
202	auto exp_path_is_safe =
203	path::IsWithinOrEqual(stream_path, path::Join(update_module_workdir_, string("streams")));	20✔
204	if (!exp_path_is_safe.has_value()) {	10✔
205	DownloadErrorHandler(exp_path_is_safe.error().WithContext(	×
206	"Error checking if path is equal to or within directory"));
207	return;	×
208	}
209	if (!exp_path_is_safe.value()) {	10✔
210	DownloadErrorHandler(error::Error(	×
211	make_error_condition(errc::invalid_argument),	×
212	"Error downloading payload: Provided payload file (" + download_->current_payload_name_	×
213	+ ") would point outside work directory when extracted."));	×
214	return;	×
215	}
216
217	auto err = PrepareAndOpenStreamPipe(	20✔
218	stream_path, [this](io::ExpectedAsyncWriterPtr writer) { StreamOpenHandler(writer); });	22✔
219	if (err != error::NoError) {	10✔
220	DownloadErrorHandler(err);	×
221	return;
222	}
223
224	string stream_next_string;
225	if (download_->downloading_with_sizes_) {	10✔
226	stream_next_string = path::Join("streams", download_->current_payload_name_) + " "	2✔
227	+ to_string(download_->current_payload_size_);	3✔
228	} else {
229	stream_next_string = path::Join("streams", download_->current_payload_name_);	18✔
230	}
231	size_t entry_size = stream_next_string.size() + 1;	10✔
232	if (entry_size > download_->buffer_.size()) {	10✔
233	DownloadErrorHandler(error::Error(	×
234	make_error_condition(errc::no_buffer_space), "Payload name is too large for buffer"));	×
235	return;
236	}
237	copy(stream_next_string.begin(), stream_next_string.end(), download_->buffer_.begin());	10✔
238	download_->buffer_[entry_size - 1] = '\n';	10✔
239	DownloadErrorHandler(download_->stream_next_writer_->AsyncWrite(	20✔
240	download_->buffer_.begin(),
241	download_->buffer_.begin() + entry_size,
242	[this, entry_size](io::ExpectedSize result) {	9✔
243	StreamNextWriteHandler(entry_size, result);	9✔
244	}));	9✔
245	}
246
247	void UpdateModule::StreamOpenHandler(io::ExpectedAsyncWriterPtr writer) {	6✔
248	if (!writer) {	6✔
249	DownloadErrorHandler(writer.error());	×
250	return;	×
251	}
252	download_->current_stream_writer_ = writer.value();	6✔
253
254	DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(	6✔
255	download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {	12✔
256	PayloadReadHandler(result);	6✔
257	}));	6✔
258	}
259
260	void UpdateModule::StreamNextWriteHandler(size_t expected_n, io::ExpectedSize result) {	9✔
261	// Close stream-next writer.
262	download_->stream_next_writer_.reset();	9✔
263	if (!result) {	9✔
264	DownloadErrorHandler(result.error());	×
265	} else if (expected_n != result.value()) {	9✔
266	DownloadErrorHandler(error::Error(	×
267	make_error_condition(errc::io_error),	×
268	"Unexpected number of written bytes to stream-next"));	×
269	}
270	}	9✔
271
272	void UpdateModule::PayloadReadHandler(io::ExpectedSize result) {	758✔
273	if (!result) {	758✔
274	// Close streams.
275	download_->current_stream_writer_.reset();	3✔
276	download_->current_payload_reader_.reset();	3✔
277	DownloadErrorHandler(result.error());	3✔
278	} else if (result.value() > 0) {	755✔
279	DownloadErrorHandler(download_->current_stream_writer_->AsyncWrite(	1,896✔
280	download_->buffer_.begin(),
281	download_->buffer_.begin() + result.value(),	632✔
282	[this, result](io::ExpectedSize write_result) {	2,528✔
283	StreamWriteHandler(0, result.value(), write_result);	632✔
284	}));	632✔
285	} else {
286	// Close streams.
287	download_->current_stream_writer_.reset();	123✔
288	download_->current_payload_reader_.reset();	123✔
289
290	if (download_->downloading_to_files_) {	123✔
291	StartDownloadToFile();	118✔
292	} else {
293	DownloadErrorHandler(OpenStreamNextPipe(	5✔
294	[this](io::ExpectedAsyncWriterPtr writer) { StreamNextOpenHandler(writer); }));	18✔
295	}
296	}
297	}	758✔
298
299	void UpdateModule::StreamWriteHandler(size_t offset, size_t expected_n, io::ExpectedSize result) {	632✔
300	if (!result) {	632✔
301	DownloadErrorHandler(result.error());	1✔
302	} else if (result.value() == 0 \|\| result.value() > expected_n) {	631✔
303	DownloadErrorHandler(error::Error(	×
304	make_error_condition(errc::io_error),	×
305	"Unexpected number of written bytes to download stream"));	×
306	} else if (result.value() < expected_n) {	631✔
307	auto new_offset = offset + result.value();	×
308	auto new_expected = expected_n - result.value();	×
309	DownloadErrorHandler(download_->current_stream_writer_->AsyncWrite(	×
310	download_->buffer_.begin() + new_offset,
311	download_->buffer_.begin() + new_offset + new_expected,
312	[this, new_offset, new_expected](io::ExpectedSize write_result) {	×
313	StreamWriteHandler(new_offset, new_expected, write_result);	×
314	}));	×
315	} else {
316	download_->written_ += result.value();	631✔
317	log::Trace("Wrote " + to_string(download_->written_) + " bytes to Update Module");	1,262✔
318	DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(	631✔
319	download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {	1,262✔
320	PayloadReadHandler(result);	631✔
321	}));	631✔
322	}
323	}	632✔
324
325	void UpdateModule::EndStreamNext() {	3✔
326	// Empty write.
327	DownloadErrorHandler(download_->stream_next_writer_->AsyncWrite(	6✔
328	download_->buffer_.begin(), download_->buffer_.begin(), [this](io::ExpectedSize result) {	6✔
329	if (!result) {	3✔
330	DownloadErrorHandler(result.error());	×
331	} else {
332	DownloadErrorHandler(error::NoError);	3✔
333	}
334	// Close writer.
335	download_->stream_next_writer_.reset();	3✔
336	// No further action necessary. Now we just need to wait for the process to finish.
337	}));	3✔
338	}	3✔
339
340	void UpdateModule::DownloadErrorHandler(const error::Error &err) {	1,564✔
341	if (err != error::NoError) {	1,564✔
342	EndDownloadLoop(err);	17✔
343	}
344	}	1,564✔
345
346	void UpdateModule::EndDownloadLoop(const error::Error &err) {	136✔
347	download_->download_finished_handler_(err);	136✔
348	}	136✔
349
350	void UpdateModule::DownloadTimeoutHandler() {	2✔
351	download_->proc_->EnsureTerminated();	2✔
352	EndDownloadLoop(error::Error(	2✔
353	make_error_condition(errc::timed_out), "Update Module Download process timed out"));	4✔
354	}	2✔
355
356	void UpdateModule::ProcessEndedHandler(error::Error err) {	133✔
357	if (err != error::NoError) {	133✔
358	err = GetProcessError(err);	5✔
359	DownloadErrorHandler(error::Error(	5✔
360	err.code, "Download: Update Module returned non-zero status: " + err.message));	10✔
361	} else if (download_->module_has_finished_download_) {	128✔
362	EndDownloadLoop(error::NoError);	3✔
363	} else if (download_->module_has_started_download_) {	125✔
364	DownloadErrorHandler(error::Error(	3✔
365	make_error_condition(errc::broken_pipe),	6✔
366	"Update Module started downloading, but did not finish"));	6✔
367	} else {
368	download_->downloading_to_files_ = true;	122✔
369	download_->stream_next_opener_.reset();	122✔
370	download_->current_stream_opener_.reset();	122✔
371	err = DeleteStreamsFiles();	122✔
372	if (err != error::NoError) {	122✔
373	DownloadErrorHandler(err);	×
374	} else {
375	StartDownloadToFile();	122✔
376	}
377	}
378	}	133✔
379
380	void UpdateModule::StartDownloadToFile() {	240✔
381	auto reader = download_->payload_.Next();	240✔
382	if (!reader) {	240✔
383	if (reader.error().code	118✔
384	== artifact::parser_error::MakeError(	236✔
385	artifact::parser_error::NoMorePayloadFilesError, "")
386	.code) {
387	log::Debug("Downloaded all files to `files` directory.");	114✔
388	EndDownloadLoop(error::NoError);	114✔
389	} else {
390	DownloadErrorHandler(reader.error());	4✔
391	}
392	return;	118✔
393	}
394	auto payload_reader = make_shared<artifact::Reader>(std::move(reader.value()));	122✔
395	download_->current_payload_reader_ =
396	make_shared<events::io::AsyncReaderFromReader>(download_->event_loop_, payload_reader);	244✔
397	download_->current_payload_name_ = payload_reader->Name();	122✔
398
399	auto stream_path = path::Join(update_module_workdir_, string("files"));	122✔
400	auto err = PrepareDownloadDirectory(stream_path);	122✔
401	if (err != error::NoError) {	122✔
402	DownloadErrorHandler(err);	×
403	return;
404	}
405
406	stream_path = path::Join(stream_path, download_->current_payload_name_);	122✔
407	auto exp_path_is_safe =
408	path::IsWithinOrEqual(stream_path, path::Join(update_module_workdir_, string("files")));	244✔
409	if (!exp_path_is_safe.has_value()) {	122✔
410	DownloadErrorHandler(exp_path_is_safe.error().WithContext(	×
411	"Error checking if path is equal to or within directory"));
412	return;	×
413	}
414	if (!exp_path_is_safe.value()) {	122✔
415	DownloadErrorHandler(error::Error(	×
416	make_error_condition(errc::invalid_argument),	×
417	"Error downloading payload: Provided payload file (" + download_->current_payload_name_	×
418	+ ") would point outside work directory when extracted."));	×
419	return;	×
420	}
421
422	auto current_stream_writer =
423	make_shared<events::io::AsyncFileDescriptorWriter>(download_->event_loop_);	122✔
424	err = current_stream_writer->Open(stream_path);	122✔
425	if (err != error::NoError) {	122✔
426	DownloadErrorHandler(err);	1✔
427	return;
428	}
429	download_->current_stream_writer_ = current_stream_writer;
430
431	DownloadErrorHandler(download_->current_payload_reader_->AsyncRead(	121✔
432	download_->buffer_.begin(), download_->buffer_.end(), [this](io::ExpectedSize result) {	242✔
433	PayloadReadHandler(result);	121✔
434	}));	121✔
435	}
436
437	} // namespace v3
438	} // namespace update_module
439	} // namespace update
440	} // namespace mender

mendersoftware / mender / 2506081045

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous