This function returns an iterator with all the available rooms that match the behaviour previously in `Client::get_dm`. `Client::get_dm` now just calls this function and returns the first item.
I believe that it is a footgun to cache the data indefinitely by default
so this copies the same behavior as for the supported versions in the
ClientCaches: it sets an expiry duration of 1 day and refreshes the data
in the background when it has expired.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This API is no longer in use so we can remove it. We also rename
`into_data_unchecked()` to `into_data()`.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
The previous behavior was making the data unavailable as soon as it is
expired, forcing the user to make a request to access fresh data. This
change keeps the data after it is expired, allowing the function to
return quickly, and triggers a background task to refresh it so that
later calls will be able to use the updated data.
This is a step towards #6090. While it's not clear in the issue what
should be the trigger for the background refreshes, this implements at
least the background refresh while keeping the expiry time as a trigger.
The next steps are to apply this to other cached values and shorten the
expiry time to something like 1 day, which should be more helpful.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Its is now available as `matrix_sdk_common::ttl_cache::TtlValue`,
allowing to use it outside of a store.
It also supports deserializing data without a timestamp, to allow to
add an expiration time to data that was already persisted.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Since the boolean is read only once when subscribing the event cache,
let's make it clear when the FFI function should be called (based on
most prominent types used at the FFI layer).
This patch fixes a critical infinite loop when loading pinned events
from the storage if and only if there is more than one chunk for the
pinned events.
The previous implementation was broken in two ways:
1. if there was no previous chunk, then `prev` was set to `None`, but
`previous` was never updated, so it was loop forever,
2. `load_previous_chunk` was called with the `last_chunk.previous` chunk
identifier, which is incorrect: it must be called with the last chunk
identifier, not the previous': otherwise it skips one chunk every time.
This patch adds a test to ensure everything works as expected.
Fixes#5902
## What this PR does
Keeps the existing `Client::send()` demo but extends it with:
- A doc comment on `get_profile` explaining the spec behaviour and the 401
condition on (Synapse's `require_auth_for_profile_requests`)
- A `get_profile_authenticated()` fallback using `account().fetch_user_profile()`
which internally uses `force_auth()`
The current implementation assumes that if the `m.profile_fields`
capability is missing, it means that the capability is not enabled, but
this is not what the spec says. However the spec says that it depends on
the Matrix versions advertised by the homeserver. So this adds a method
that properly computes the capability depending on the supported
versions too.
Similarly, for the display name and avatar URL fields the implementation
assumes that if the `m.profile_fields` capability is present but
disabled, we should fallback to the legacy capabilities. This behavior
is not present in the spec, so the code is changed to always use the new
capability when it is present, whether it is enabled or not. The
legacy capabilities are only used if the new capability is missing and
the homeserver doesn't advertise support for extended profile fields.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This test forgot to subscribe to the Event Cache, hence the result
was partially okay. It was working by luck before. With the previous
commits, it was not possible to work without the Event Cache being
listening to the sync.
This patch adjust tests since the Event Cache ignores events from the
Send Queue if the cache is empty. It mostly impacts the tests as this
scenario is pretty rare in real world use cases.
This patch fixes a bug where inserting an event from the Send Queue in
an empty Event Cache will break the back-pagination logic. Indeed, the
detection of the start of the timeline during the back-pagination is
conditioned to the emptiness of the cache:
- if there is no gap, and if the cache is not empty, then we consider
the start of the timeline has been reached.
However, if an event from the Send Queue has been inserted, with no
previous batch token (because none can be computed at this step), no gap
is present and the cache won't be empty, so… this is wrongly assumed to
be the start of the timeline.
The solution to this problem is to insert the event from the Send Queue
if the cache is not empty.
This patch updates the logic to update a `LatestEventValue`. We still
can't compare two `LatestEventValue` because the type doesn't implement
`PartialEq`. However, we can use the `EventId` in some case.
It fixes https://github.com/matrix-org/matrix-rust-sdk/issues/6381.
This patch fixes the SpaceRoom::compare_rooms method to be transitive
(If A ≤ B and B ≤ C then A ≤ C). The transitive property of a comparison
function is required by the sorting functions we are using.
If the property doesn't hold, sorting will panic.
The previous logic violated strict weak ordering by falling back towards
room ID comparison as soon as one of the values doesn't have a
SpaceRoomChildState.
This introduced inconsistent ordering paths where:
- A and B would be compared using the SpaceRoomChildState
- B and C would be compared using only the room ID
- A and C would be compared using only the room ID
Leading to the case where:
- A < B due to the state
- B < C due to the room ID
- and finally A > C due to the room ID.
As a result, transitivity could be broken (A < B, B < C, but C < A),
leading to a panic during the sorting.
Several new functions implicitly rely on the sqlite config.
sqlite might not be present if you are using the indexedb store, in which case
these functions are likely irrelevant.
This change conditionalizes their compilation.
matrix_sdk_ffi::ruma::TagName and matrix_sdk_ffi::event::TimelineEventType enums
that are used as HashMap keys by matrix_sdk_ffi::ruma::Tag and
matrix_sdk_ffi::room::power_levels::RoomPowerLevels respectively should probably
export Eq and Hash traits, so that we can generate bindings for languages that
implement uniffi's record/HashMap type using a hash table (e.g.
std::unordered_map in C++)
Signed-off-by: Stanislav Skobelkin <stanislav@skobelk.in>
This allows people to get a secrets bundle out of band or out of a database and import it
after logging in a new client.
Mainly targeted to support the Element Classic -> Element X migration.
Signed-off-by: Damir Jelić <poljar@termina.org.uk>
Co-authored-by: Benoit Marty <benoitm@matrix.org>
Previously there wasn't a way to determine whether a given sub-space or
room within a given space had been marked as "suggested".
This was one of the things missing in support for spaces (#227) even
though it wasn't explicitly mentioned.
I tested that this fix does work properly in
[Robrix](https://github.com/project-robius/robrix).
Signed-off-by: Kevin Boos kevinaboos@gmail.com
This is a first step towards
https://github.com/matrix-org/matrix-rust-sdk/issues/4162 and adds a way
to send redactions (including their local echoes) via the send queue.
I had to introduce new variants for `SentRequestKey` and
`LocalEchoContent` because in some room versions the redacted event ID
sits at the top-level of the event rather than in `content`.
At the timeline level redactions are handled via a new boolean flag in
`AggregationKind::Redaction`. Local echoes of redactions merely set a
flag on the timeline event whereas remote echoes of redactions lead to
actual redactions as before.
The FFI bindings will be updated in a follow-up PR.
Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
Note that the rustls-tls feature flag is temporarily left in place
to keep other crates from breaking, but it becomes a trivial flag.
It will be removed altogether in a later commit.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
This allows for notifications to use the intent for deciding how to
render the message (e.g. whether to call it a "call" or a "video call").
Signed-off-by: Bryant Mairs <bryant@mai.rs>
The previous warning could trigger with abnormal numbers, since there
could be related events applying to a pinned event, the `loaded_events`
list could be bigger than the list of `pinned_event_ids`, due to the
eager flattening. This patch makes it display the correct number, by
postponing the flattening after the warning.
For non-live timelines, the recurring pattern was the following:
- first, fill the timeline with initial events, with a first call to
some form of subscribe() method,
- then, subscribe to updates in a second time later, using a second call
to the same subscribe() method.
Unfortunately, this is wrong, and opens a window for a very small race:
- first call to subscribe() to get the initial events
- new updates are emitted in the receiver, that lead to new events
- second call to subscribe() to get the receiver, and subscribe to it
In this case, the new updates would be lost by an observer.
The patch consists in refactoring timeline initialization, such that the
initial events and receiver are created at the same time; if there are
some updates happening after the now single subscribe() call, the
updates are accumulated and will be observable by external users.
This also has the nice benefit of tidying up the number of background
task handles for non-live timelines: there should be at most one such
focus task, which is now cleanly reflected in the timeline drop handles.
The issue is that when the most recent message was an live location event that started as live and later transitioned to a stopped state, the room list summary would ignore it entirely, even though the timeline still showed it as the latest item.
That behaviour feels incorrect. At most, the client should update the displayed description based on whether isLive is true or not.
This patch prevents the stopped live location events from being filtered out in the latest events.
This extracts the `/capabilities` logic to its own `HomeserverCapabilities` component in the SDK that can be manually asked to fetch, cache locally and return these capabilities.
### Expose sync v2 API through FFI via `Client.sync()` and
`Client.sync_once()` #6348
Mobile clients can now sync using the traditional `/sync` v2 endpoint
through the FFI bindings, without requiring Sliding Sync (MSC4186)
support on the homeserver.
This PR introduces a new API (non-breaking change).
#### Implementation notes
I chose to expose only the **list of room IDs** (invited, joined, left,
knocked) in the `SyncResponse`, rather than forwarding the full per-room
event payloads. This keeps the FFI surface simple. Clients that need
room details can query them through the existing `Room` APIs after being
notified of changes via the listener.
Two entry points are provided:
- **`Client.sync(settings, listener)`** starts a continuous sync loop,
returning a `TaskHandle` for cancellation. The `SyncListener` callback
fires after each successful sync response.
- **`Client.sync_once(settings)`** performs a single sync call, useful
for initial sync or one-off use cases.
`SyncSettings` exposes `timeout_ms` and `full_state`.
Signed-off-by: Philippe Bertin <pbertin@teladochealth.com>
This avoids profile updates when a user decides to leave on their own (in which case their profile is kept as is), and when a user is banned (in which case their profile is removed).
Signed-off-by: JoFrost <20685007+JoFrost@users.noreply.github.com>
This means that when some caller is subscribing to the `EventCache`,
even if the event cache drops in the background, the task will remain
active until the end of the subscriber.
There wasn't good reason to use an async lock, as this lock is always
super short-lived, it can be sync, which avoids complications in the
subsequent commit when calling sync init code.
- Change the example config file to not make it appear as though it
processes environment variables
- Remove the `ar` setting from the example config file, as that setting
is deprecated and unused (see
https://doc.rust-lang.org/cargo/reference/config.html#targettriplear)
- Replace references of `.cargo/config` to `.cargo/config.toml`, as the
former is deprecated
- Add example of how to set the linker through an environment variable
- Add instruction to include the NDK binary tools directory to PATH,
because builds may fail without it
(see https://github.com/matrix-org/matrix-rust-sdk/issues/4042)
When ThreadListService is created it now immediately spawns a background
task (via the client's TaskMonitor) that subscribes to the room event
cache and listens for RoomEventCacheUpdate::UpdateTimelineEvents.
For every incoming event that carries an m.thread relation pointing to a
root we are already tracking, the service rebuilds a ThreadListItemEvent via the
existing build_latest_event helper and replaces latest_event on the matching
ThreadListItem and increments num_replies by 1
Introduce ThreadListItemEvent as a shared struct for both the thread
root and the latest reply, replacing the previous flat fields on
ThreadListItem
The `latest_event` is populated from the bundled thread summary returned by the
server alongside the root event, as is the number of replies.
We think this might be a cause for intermittent failures of the
`room_unread_count` test on CI, as the `room_info` might be outdated
after we decided to process it.
More generally, the room info is a critical resources that should be
protected in better ways, as it's inner mutable state that can be
modified by all code claling `set_room_info` in any place, but that'll
be a problem for later.
`ThreadListService` is the FFI-facing wrapper around
[`matrix_sdk_ui::timeline::thread_list_service::ThreadListService`]. It
maintains an observable list of [`ThreadListItem`]s and exposes a
pagination state publisher, making it straightforward to build reactive UIs
on top of the thread list.
This patch updates the fuzzy filter from using _smart case_ to _ignore
case_. Most of the time, the keyboard might activate an upper case for
the first character pressed in the search bar. Some users aren't used
to smart case and will believe there is a bug. Let's disable smart case
here.
This will skip using the custom workaround for Android and the latest rustls, but since that was to fix an issue with the verifier and the verifier is not used anymore, it should be fine
There was a hack added that the unread count of a room shouldn't
decrease, if the room's latest active receipt hasn't changed.
Unfortunately, this heuristic doesn't hold: if an event was counted
because it was encrypted, and it turns out that after decrypting it as a
UTD, it should now be uncounted, then the read count would in fact
legitimately decrease.
Let's remove this hack, and in the meantime keep on working on automatic
backpagination, which should help fix this kind of issues.
<!-- description of the changes in this PR -->
Depends on: https://github.com/ruma/ruma/pull/2414
Partity with web/js-sdk.
Allows to get the consus intent for the current call using the
membership `m.call.intent`
EW use it to change the icon in the room list
<img width="298" height="117" alt="image"
src="https://github.com/user-attachments/assets/21e59f69-e099-40a6-ae27-d9246df35b64"
/>
- [x] I've documented the public API Changes in the appropriate
`CHANGELOG.md` files.
- [ ] This PR was made with the help of AI.
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by:
The test shows a situation where an in-thread message would be counted
as an implicit receipt, while, when we're in the `main` timeline mode,
we're counting unreads only for the main timeline. In this case, the
test would fail because the unread counts would be set to 0, since
select_best_receipt would have selected $3.
The patch is to *not* select an implicit receipt, in that mode, when
it's in a thread.
Related to https://github.com/element-hq/element-x-android/pull/6420
Part of https://github.com/element-hq/element-x-android/issues/5075
I noticed when investigating a bug about resetting your identity when
using a MAS login that we poll the server checking whether the user has
given us permission with no limit on how fast we poll, and with no
ability to give up if it's not working.
This change causes us to retry only twice per second, and give up after
2 minutes. These are guesses as to the right values and I am open to
discussion.
This adds back the `webpki` verifier and sets it to a custom rustls instance created only for Android, instead of using the platform verifier that results in false positives with Let's Encrypt certs (and from other CAs).
See https://github.com/matrix-org/matrix-rust-sdk/issues/6319
My only guess for this semantic change, is that the pagination status
update and the end of the pagination now happen at different times, or
close enough that they're regrouped in the same stream update. This
doesn't fundamentally change the semantics, so we'll see if this holds
on slower machines (e.g. CI).
- use the modern MatrixMockServer facilities
- provide a previous-batch token so as not to wait for the initial batch
token
- lower the delay for the /messages responses to 1 sec
This makes it possible to common out the implementations of functions
that should be available to both kinds.
Also, moves a bit of code that could easily live in the
`RoomEventCacheState` impl block, instead of being free functions.
There was a subtle bug that a receipt would be considered active, and
then on the subsequent call to `select_best_receipt`, it could be
forgotten in favor of an older receipt. The regression test shows one
such case, where before this patch, the count would incorrectly say 3,
not 2, because the active read receipt moved backwards to the implicit
receipt.
The solution is to stop looking for a better receipt, if we run into the
latest active read receipt. Having `found` set to `None` in this case
means we hadn't found any better read receipt anyways.
It turns out on Android, rustls needs [a custom setup](https://github.com/rustls/rustls-platform-verifier#android) and adding the `rustls-platform-verifier-android` library that's [not available on Maven](https://github.com/rustls/rustls-platform-verifier/issues/115).
Then, from the Android clients we'd need to call some exposed JNI function so we can provide a JVM context from where Rust can take the `Application` component and access its contents to read its credentials storage. Thanks to some tricks we can use `libloading` to simulate this call from Rust itself and properly initialise the platform verifier.
Note self-signed certificates will no longer work with these changes on Android, and providing them in `ClientBuilder::add_root_certificates` will make most requests fail. This can be handled separately.
This method is now only used by tests, so I opted to lock it
behind the test configuration to appease CI.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This patch removes the `enable_latest_event_sorter` flag in
`RoomList::entry_with_dynamic_adapters_with`. This sorter is now stable
enough, we can always enable it.
The method
`RoomEventCacheStateLockWriteGuard::load_more_events_backwards` is used
only in one-place: in `RoomPagination::load_more_events_backwards`. This
patch inlines this method as it aims at living in `RoomPagination`, not
somewhere else.
It's purely code move. Nothing else has changed.
This patch also updates a test that was accessing
`load_more_events_backwards` directly. Now it runs it via
`RoomPagination`.
This patch updates `ThreadPagination` to hold a `ThreadEventCacheInner`
instead of a `RoomEventCacheInner`! It makes more sense and it
splits/isolates the types even more.
`RoomEventCache::thread_pagination` is now async and returns a
`Result<ThreadPagination>` because it needs to load its state to fetch
a `ThreadEventCache`. Later, accessing a thread wouldn't happen in
`RoomEventCache` but in `Caches`, one step at a time.
This patch adds the `weak_room: WeakRoom` field to
`ThreadEventCacheInner`. This is a prerequisite to have
`ThreadPagination` uses `ThreadEventCacheInner` instead of
`RoomEventCacheInner`!
This patch creates the `ThreadEventCacheState` type. It uses
`caches::lock::StateLock`, just like `RoomEventCacheState`. It allows
to have the `read()` and `write()` method to access the state, and to
reload it when necessary, see the `caches::lock::Store` implementation.
This patch thus creates `ThreadEventCacheStateLockReadGuard` and
`ThreadEventCacheStateLockWriteGuard`. The methods touching the state in
`ThreadEventCacheInner` are moved to these lock types.
They are purely code moves (plus changes to reach the correct data): no
change in the semantics.
This patch creates `ThreadEventCacheInner` so that `ThreadEventCache`
can be shallow cloned (which will be useful for `ThreadPagination`).
That's also the first step to introduce `ThreadEventCacheState`!
The new integration tests for the event cache cover the same situations
that were tested by the previous tests on compute_unread_count_legacy,
so we're fine here.
A gappy sync may cause a linked chunk to shrink, waiting for callers to
lazy-reload it again in the future. But, because the unread counts
computation rely on the in-memory linked chunk, this means that the
values computed for the unread count may be incorrect (and decrease).
Fortunately, this situation is rather easy to detect, because the latest
active read receipt doesn't change in this case, so we can first check
that, and then manually readjust the unread counts, if they've
decreased.
Future work should trigger back-pagination in those cases, so the unread
counts keeps on being precise, despite the gappy sync.
This test exhibits an edge case: when a room event cache is shrunk
(because of a gappy/limited sync), then the unread count might decrease,
without the latest active read receipt changing.
`test_room_notification_count` started to intermittently fail on main,
because the computation of unread counts has moved from the sliding sync
processing to the event cache. As a result, new irrelevant RoomInfo
updates (related to the unread counts) can happen, and they might happen
quickly enough that the server reponse for sending an event happens
after 2 seconds (remember, we need to factor in the time to do the e2ee
key exchange, and so on and so forth).
Bumping the time between two RoomInfo updates should be sufficient to
avoid the intermittent failure.
This patch fixes a bug in `xtask log sync` which can miss a `sync_once`
log when the `pos` field is absent. It happens when there is no `pos`!
Example where `pos` is absent before `timeout`. Note the double space
before `timeout`:
```
… > sync_once{conn_id="room-list" timeout=0} > send{request_id="REQ-15" …
```
While when the `pos` is present, it's:
```
… > sync_once{conn_id="room-list" pos="0/m67590980…" timeout=30000} > send{request_id="REQ-23" …
```
This is a basic implementation that works, but it should unlock
improvements already (getting the unread count updated whenever a UTD
has been resolved) and it will pave the way for future improvements
(notably with respect to performance).
This is almost only code motion in this commit.
At this point, some tests don't pass, as the support for using the read
receipt code in the event cache isn't plugged in to the event cache
itself.
This patch makes `RoomEventCacheInner` more private, from `event_cache`
to `event_cache::caches`.
Consequently, `PinnedEventCacheState`, `RoomPagination::new` and
`ThreadPagination::new` follows the same restriction.
This patch makes `RoomEventCache::new`,
`RoomEventCache::handle_joined_room_update` and
`RoomEventCache::handle_left_room_update` more private. They are no more
accessible from `event_cache` but only from `event_cache::caches`.
This patch moves `test_uniq_read_marker` from `event_cache` to
`event-cache::caches::room`.
This patch is a prerequisite to the next patch where the
`RoomEventCache::handle_joined_room_update` will become more private.
Moving this test is necessary.
This patch moves `EventLocation` from `event_cache::caches::room` to
`event_cache::caches`.
This type is used by the redecryptor, and can possibly be used by other
event caches. So let's move it in the `caches` module.
This patch puts the `EventCache::subscribe_thread_subscriber_updates`
method behind `#[cfg(feature = "testing")]` along with the
`EventCacheInner::thread_subscriber_receiver` field.
This data is only used for tests, as the documentation tells so. Let
make it clear. Also, it reduces the size `EventCacheInner` by 16 bytes
for non-testing builds.
This patch updates `EventCacheInner::all_caches_for_rooms` by returning
an `OwnedRwLockReadGuard` instead of `Caches`. `Caches` no longer
implement `Clone`, which prevents cloning all the event caches per room
when we need only one for example.
`Caches` has two new methods: `handle_joined_room_update` and
`handle_left_room_update`. This type acts more like a dispatcher now,
which was the initial idea. That way, we don't need to dispatch manually
on all event caches: `Caches` is responsible for that.
We need to be a bit careful now since `all_caches_for_rooms` returns an
owned read-lock over `EventCacheInner::by_room`.
This patch introduces the `Caches::prepare_to_reset` method, which
returns a `ResetCaches` type. This new type is responsible to lock all
the event caches related to a room, and to reset their in-memory state
with `ResetCaches::reset_all`. This patch fixes 2 bugs (see below).
This type is acquiring exclusive locks over all the event caches
managed by `Caches`. Once dropped, all the locks are released. Note that
`ResetCaches::reset_all` takes `self`, not `&self` or `&mut self`, i.e.
it consumes `ResetCaches`, ensuring the locks are _always_ released.
This patch also fixes a possible bug when acquiring the exclusive
locks could have failed, but the error wasn't stopping the execution;
the database would have been reset anyway. Now, `try_join_all` is used
before and after resetting the database (prior to this patch, it was
using `join_all` before resetting the database).
This patch also ensures that only one reset per `Caches` can
happen at a time, by making `Caches::prepare_to_reset` taking a
`&mut self`. This was partially supported by the write lock over
`EventCacheInner::by_room`, but it would have still been possible to
reset the same `Caches` concurrently.
This patch is the first step for the `Caches` type. It creates it, along
with the `new` constructor. More changes are required, but this is a
first step.
The pattern `Caches { room }` is used everywhere where a
`RoomEventCache` was read previously, so that we are sure the type
system will complain when we will add more fields to `Caches` (like for
threads and pinned events).
This isn't strictly necessary, but the lack of these events was
causing spurious CI timeouts when tested with the logic that
assumed a stricter history visibility than the spec required.
This commit additionally flips the order of a few assertions to
meet developer expectations, i.e. errors are reported as diffs
from expected.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
- https://github.com/matrix-org/matrix-spec-proposals/blob/matthew/location/proposals/3488-location.md
- `If m.asset is missing from the location's content the client should render it as m.self as that will be the most common use case. Otherwise, if it's not missing but the type is invalid or unknown the client should attempt to render it as a generic location. Clients should be able to distinguish between m.self and explicit assets for this feature to be correctly implemented as interpreting everything as m.self is unwanted.`
- this aligns the behavior with the newly introduced live location asset type handling
# Overview
There are scenarios in which it is sensible to have an event exist in
the same room more than once. Notably, this is true in the context of a
thread, where an event exists in the main timeline of a room, as well as
in a thread of that same room.
Support for this behavior has been implemented in the
`SQLiteEventCacheStore` in #6065; however, this was never implemented
for the `IndexeddbEventCacheStore` or the `MemoryStore`. This pull
request extends this behavior to both of those stores.
# Changes
## Integration Tests
First, `test_event_chunks_allows_same_event_in_room_and_thread` was
moved from `matrix_sdk_sqlite::event_cache_store` to
`matrix_sdk_base::event_cache::store::integration_tests`. Then, a few
additional integration tests were added to ensure that behavior is
consistent across implementations of `EventCacheStore`.
## `IndexeddbEventCacheStore`
In order to accommodate the behavioral changes specified by the
integration tests, it was necessary to modify the schema in the
IndexedDB implementation of `EventCacheStore`. Namely, the events object
store was cleared and removed and then replaced with a nearly identical
one, the only difference being the removal of a uniqueness constraint on
one of the indices.
The remaining changes mostly involved updating the behavior of top-level
`EventCacheStore` functions - e.g., filtering out events where they were
duplicated or removing positioning information where it was not
relevant.
## `MemoryStore`
The changes to `MemoryStore` mostly involved updating the behavior of
top-level `EventCacheStore` function - e.g., filtering out events where
they were duplicated or removing positioning information where it was
not relevant.
That being said, it also involved some breaking changes to
`RelationalLinkedChunk`.
1. `RelationalLinkedChunk::items` - this function returned an `Iterator`
that did not contain information about the `LinkedChunkId`, so this
information was added to the items in the `Iterator`.
2. `RelationalLinkedChunk::save_item` - this function did not update the
item in all linked chunks of the provided `Room`. It now does this, but
requires that the provided `Item` be `Clone`.
(1) could probably have been a new function, but I thought a nicer
interface was worth the breaking change. (2) could probably be prevented
by re-organizing `RelationalLinkedChunk`'s internal data structures to
remove the `Clone` requirement, but that seemed like it could turn into
a large refactoring project, so I opted for something simpler albeit
somewhat crude.
In both cases, I'm open to suggestions and would be happy to revisit if
something else is preferred.
---
Closes#6094.
- [x] I've documented the public API Changes in the appropriate
`CHANGELOG.md` files.
- [x] I've read [the `CONTRIBUTING.md`
file](https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md),
notably the sections about Pull requests, Commit message format, and AI
policy.
- [ ] This PR was made with the help of AI.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
---------
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
**Note:** _this pull request has a companion pull request in the
[`complement-crypto`](https://github.com/matrix-org/complement-crypto/pull/229)
repository, which must be merged in conjunction with this one._
_Before merging, this should be tested in conjunction with the Element X
iOS client to ensure that TLS v1.3 is working properly._ @stefanceriu
has agreed to work on this.
## Overview
The primary change in this pull request upgrades the `reqwest`
dependency to its latest version, which defaults to using `rustls` with
support for `rustls-platform-verifier` instead of `native-tls` (see
[`reqwest@0.13.0`](https://github.com/seanmonstar/reqwest/releases/tag/v0.13.0)).
The benefit here is that `rustls` supports TLS v1.3 on all platforms,
whereas [`native-tls` does
not](https://github.com/sfackler/rust-native-tls/pull/278).
Additionally, this pull request makes `rustls` the default TLS
implementation in all the crates in this repository.
This will be particularly helpful with element-hq/element-x-ios#786.
## Changes
- `reqwest` bumped to `0.13.1`
- The API for adding/replacing certificates has changed a bit, so this
required some updating in `HttpSettings::make_client`
- `oauth2-reqwest` added in favor of `oauth2/reqwest`
- This is required in order to be compatible with `reqwest^0.13`
- _**`oauth2-reqwest` is currently in alpha release, so it probably
makes sense to let this stabilize a bit.**_ For details, see
https://github.com/ramosbugs/oauth2-rs/issues/333#issuecomment-3906712203.
- `getrandom` bumped to `0.3.4`
- This is required in order to be compatible with `oauth2@5.1.0`
- `proptest` bumped to `1.9.0`
- This is required in order to be compatible with `getrandom@0.3.4`
- Make `rustls` the default TLS implementation
## Questions
### Mirror feature flag names?
A number of feature flags have been replaced in the dependencies above.
1. _**`reqwest/rustls-tls` => `reqwest/rustls`**_ - this is simply a
name change, but is semantically identical (see
[`reqwest@0.13.0`](https://github.com/seanmonstar/reqwest/releases/tag/v0.13.0)).
2. _**`getrandom/js` => `getrandom/wasm_js`**_ - the semantics here have
changed slightly, but it seems to just make it easier to enable the
`wasm_js` backend (see
[`getrandom@0.3.4`](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md#major-change-to-wasm_js-backend)).
At any rate, I have updated references to these flags in each of the
various `Cargo.toml` files, but have not changed the names of our
exposed features to mimic those in the dependencies.
Any thoughts or preferences on whether to mirror those names? That
would, of course, result in a breaking change.
### Default to using `rustls`? Deprecate `native-tls`?
Now that the dependencies have all been bumped, we can use `rustls` on
all platforms. Should this be the new default given that `native-tls`
will very likely never support TLS v1.3 on Apple devices? And should
`native-tls` be deprecated as a result?
**UPDATE:** _The consensus here seems to be that we should default to
using `rustls`, but that `native-tls` should still be available._
---
Fixes#5800.
- [ ] Public API changes documented in changelogs (optional)
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
---------
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
Turns out, the `m.read` event was invalid, because it should've been
part of the ephemeral events in the sync response (and not part of the
room's timeline response), so the event was dismissed. The test already
passed while behaving this way, so let's make it reflect what it did
indeed.
The test was aborted just a bit too early, in that if you introduce
arbitrary sleep statements, it would fail because the stream of notable
reasons updates wouldn't be empty, and include two `LATEST_EVENT`
updates instead.
It's not clear why we get the second update, but this isn't critical to
fix at the moment, so I'll leave this as an exercise to the reader.
This will allow to reuse it outside of the `oauth` module.
It can now also be converted from a `QueryString`, for improved
compatibility with `LocalServerRedirectHandle`.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
The log message can contain HTML data. It happens notably when the
homeserver is broken, and an HTML document is returned in some errors.
We don't want to parse the fields in this case, because HTML breaks
everything.
We rely on the EventFactory to generate a hash of the events for the
event IDs, and we use the appropriate length for room IDs.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
We rely on the EventFactory to generate a hash of the events for the
event IDs, and we use the appropriate length for room IDs.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
We rely on the EventFactory to generate a hash of the events for the
event IDs, and we use the appropriate length for room IDs.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
We rely on the EventFactory to generate a hash of the events for the
event IDs, and we use the appropriate length for room IDs.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
As a fallback when the ID is not provided when constructing the event.
It allows to work with data that looks like what we would get in
production, which is important for benchmarks.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This allows passing the `Transaction` by mutable reference, instead of
passing it by ownership and requiring the callback to pass it back,
which is slightly better in terms of ergonomics. This was hard to
achieve without async closures, but now that we have them, this is
trivial.
Many of our tests make use of `assert_let` for checking that some value
comes out of a stream, while they could use `assert_let_timeout`, which
provides better ergonomics when the expected value doesn't arrive
immediately, by failing quickly.
This converts a few instances, making those tests easier to debug in the
future, would they fail again.
Removed a few TODOs that were not applicable anymore, because they were
either very low value (in timeline test code) or already done (in event
cache, with respect to the cross-process locking).
Also removed my nick from some TODOs and comments, as code comments
aren't the best way to store assignees for issues.
This patch logs errors from `EventLinkedChunk` and send them to Sentry
(if Sentry is enabled).
The trick is to use [`#[instrument(err)]`][0] for logging errors.
Quoting the documentation:
> If the function returns a `Result<T, E>` and `E` implements
> `std::fmt::Display`, adding `err` or `err(Display)` will emit error
> events when the function returns `Err`:
>
> ```rust
> #[instrument(err)]
> fn my_function(arg: usize) -> Result<(), std::io::Error> {
> Ok(())
> }
> ```
>
> The level of the error value event defaults to `ERROR`.
It sounds exactly what we need.
[0]: https://docs.rs/tracing-attributes/0.1.31/tracing_attributes/attr.instrument.html#examples-2
They are error prone because they need to be bumped for every migration
otherwise the new migration will not happen because we exit early.
So instead we get rid of the early returns and log each individual
upgrade separately. It makes more noise when creating a new database,
but since it is logged at the DEBUG level it is not much of a problem.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
When the event cache decides to deduplicate items, it will remove the
duplicate items, then push them back at a position further down the
line.
This can lead to spurious re-creation of items, which may show up, in
embeddings, as different items as they don't share the same internal id.
This patch makes it so that if the same transaction includes a move of
an event (i.e. it's removed then reinserted elsewhere), then the
internal ID will be reused in this case.
There seems to be a race condition where the
`RoomSendQueueUpdate::SentEvent` update arrives after the remote echo
has already been processed which leads to the latest event value getting
stuck at `LocalHasBeenSent`. This fixes the issue by trying to retrieve
the event from the cache first.
- [x] I've documented the public API Changes in the appropriate
`CHANGELOG.md` files.
- [x] I've read [the `CONTRIBUTING.md`
file](https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md),
notably the sections about Pull requests, Commit message format, and AI
policy.
- [ ] This PR was made with the help of AI.
---------
Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
This is equivalent to what happens in `init_focus` when the timeline is
initialized from scratch.
Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
The PR template includes an item for a warning that is automatically
shown once (and once and for all) by the Github UI. In the previous
review of the changes to the pull request template, Ivan agreed that
this was redundant, so it seems there's no point in keeping it in the
pull request template itself.
Instead encourage users to use the ones available on
`AuthorizationServerMetadata` because they support both the stable and
unstable actions.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
The method `RoomEventCacheStateLockWriteGuard::handle_backpagination`
is used in a single place. This patch inlines it in…
`RoomEventCachePagination`, where it's supposed to be
declared! More precisely, inside the `PaginatedCache`
implementation for the `RoomEventCachePagination`, in the
`conclude_backwards_pagination_from_network`. The `state` module is
lighter with this change, and the code lives in the correct place.
This patch also renames `EventLinkedChunk::finish_back_pagination` to
`push_backwards_pagination_events`. The naming follows other names, like
`push_live_events`. Moreover, it removes the entire concept of “this
is part of a flow of methods”, it's just a single standalone method. On
this `EventLinkedChunk` alone, it is absolutely stateless.
This patch is purely code move, nothing changes.
This patch creates the new `RoomEventCacheUpdateSender` type to group
both the room update sender, and the room generic update sender. It
simplifies a couple of constructor and makes the code more robust by
isolating this logic in a single type instead of two types.
Firstly, the idea is to avoid accessing `RoomEventCache::inner`. Second,
by having a `send_updates` method, we increase the chances to forget
about one update (like `RoomEventcacheGenericUpdate` as it was the case
in the past).
This patch simplifies a lot of `pub(in super::…)` to just `pub`. The
visibility is now defined by the type itself, onto which the methods
are implemented.
Check if an Event is redacted before trying to decrypt it.
Use a new enum `NotificationStatus.EventRedacted` when trying to resolve
an event for a notification.
PR done with the help of AI: GitHub copilot chat in VisualStudio.
I confirm that the fix is working in EXA with the code in
https://github.com/element-hq/element-x-android/pull/6241Fixes#5796
<!-- description of the changes in this PR -->
- [x] I've documented the public API Changes in the appropriate
`CHANGELOG.md` files.
- [x] I've read [the `CONTRIBUTING.md`
file](https://github.com/matrix-org/matrix-rust-sdk/blob/main/CONTRIBUTING.md),
notably the sections about Pull requests, Commit message format, and AI
policy.
- [x] This PR was made with the help of AI.
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by: benoitm@element.io
Previously, invite acceptance details were cleared in `RoomInfo::set_state`. We
can't do that any more, because (a) that method is synchronous (b) it doesn't
have access to the crypto store (c) it would be a bit of a layering violation
even if it did.
Instead, we pull the logic up to higher-level methods which do have access to
the crypto store (though, for now, we don't use it).
Step two in a series of refactoring: move and rename
`matrix_sdk_base::InviteAcceptanceDetails` to
`matrix_sdk_crypto::store::types::RoomPendingKeyBundleDetails`.
We're going to store this in the crypto store instead, so need to move it
down. I also want to rename it to better reflect how we interpret it.
This field is only ever used when encryption is enabled, and we want to move it
to the crypto crate. So, for a starting point, gate it behind the
e2e-encryption feature flag.
`matrix_sdk_base::response_processors::e2ee::E2EE` exists to pass E2EE context to the
logic within `response_processors`. Currently we pass it around by value in
many places, and hence have repetitive code for building the context, as well
as lots of calls to `.clone`.
Cloning of this type is cheap, but we can do better, by passing it around by
reference instead.
This patch adds the `waited_for_initial_prev_token` field directly
inside `LoadMoreEventsBackwardsOutcome::Gap`. It removes the need
to manage the `state_guard` manually (which can be error-prone). It
also removes one access to the `state_guard` in the pagination module.
Finally, it paves the road for a shared `Pagination<C>` type for
`RoomEventCache` and `ThreadEventCache`.
This patch renames the `RoomPaginationStatus` enum to `PaginationStatus`
because it won't be restricted to the `RoomEventCache` only: indeed, the
`ThreadEventCache` will soon be able to yield it too. Let's do the
renaming now.
The code was dereferencing to the borrowed type, only to convert it
back to an owned type. We can just use `.clone()` for this.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Similarly to #5382 and #5744, this patch introduces a write-only
connection in `SqliteCryptoStore`. The idea is to get many read-only
connections, and a single write-only connection behind a lock, so that
there is a single writer at a time.
This patch renames the `acquire` method to `read`, and it introduces a
new `write` connection.
This patch adds a new checkbox to ensure the user has read the `CONTRIBUTING.md` file, notably the Pull requests, Commit message format, and AI policy Sections.
Signed-off-by: Ivan Enderlin <ivan@mnt.io>
This avoids the sleep statement by only listening to the room's event
cache, instead of listening to the background send task and then wait
for it to save the sent event in the room cache. The test is more
resilient this way.
Change JoinedRoomBuilder::add_account_data() to accept any type that
implements Into<Raw<AnyRoomAccountDataEvent>>, preparing for migration
away from RoomAccountDataTestEvent enum.
Add support for room account data events in EventFactory:
- Add RoomAccountData format to EventFormat enum
- Add From implementations for Raw<AnyRoomAccountDataEvent>
- Add room_account_data() generic method
- Add fully_read() helper for m.fully_read events
- Add marked_unread() helper for m.marked_unread events
Replace all usages of ENCRYPTION_CONTENT and
ENCRYPTION_WITH_ENCRYPTED_STATE_EVENTS_CONTENT static values
with EventFactory::room_encryption().into_content() calls.
This uses the new into_content() method to get just the event
content for HTTP response mocking.
Add a generic method to EventBuilder that returns just the event
content as a serde_json::Value. This is useful when mocking HTTP
responses that return event content rather than full events.
There's one case where we want to create a custom join rule, and thus
must parse directly from JSON, but the rest can reuse helpers that
already exist.
An invite room is only constitued of stripped-state events. These events
do not have an `origin_server_ts` field. It means we cannot compute
the timestamp of the `LatestEventValue`. To workaround this, we set
the timestamp to `now()`. See `Builder::new_remote_for_invite` to learn
more. If an invite room receives a new event, its `LatestEventValue`'s
timestamp can be updated to `now()` again, which will make the room
bumps to the top of the room list for example. This is not an acceptable
behaviour because it can be an “attack vector”, i.e. a way to annoy
people with spammy invites. That's why, once a `RemoteInvite` has been
computed, we do not refresh it.
The fix is 2 instructions. The rest are comments and tests.
This new file writer supports both total size and age based rolling, and
will take care of a whole class of edge cases that make rageshake
uploading fail.
* It will rotate log files when the configured time period elapses
(e.g., hourly, daily).
* When the total size of all log files exceeds the configured limit, the
oldest files are removed.
* Logs older than the configured max age are automatically removed
during cleanup.
* Only files matching the configured prefix and suffix are managed by
any given writer instance.
The pinned event timeline only cares about updates to the vector of
events, not the other types of updates, at the moment. As such, it
doesn't need to use the fully-fledged `RoomEventCacheUpdate`, but a
simpler version, like the one that is being used for threads. Hence, we
can reuse `TimelineVectorUpdate` here.
Without this, the `Account::set_avatar_url()` function does not work
on homeservers that don't advertise support for the new endpoints
for setting profile fields.
This patch adds the `open` attribute to the `<details>` for displaying
leaf nodes of the tree. My experience shows that it's nice to always
have them open. The “occurences” list is still folded, so it shouldn't
take to much space on the screen most of the time (one line per level of
logs in this node).
This patch adds the `log overview` task that generates a standalone HTML
report representing the logs as a tree where each node is a target, and
each leaf is a log location with occurrences, spans and fields.
Each node displays the sum of errors and warnings for this node.
It helps to quickly spot the problematic targets, and it guides to
exploration of the node without taking the reader's hand by trying to
draw conclusions. We don't want to guide the reader to a mistake: we
just want to guide the reader to draw its own conclusions.
Each line can be highlighted. When a node is folded/closed, it is also
highlighted if at least one of its child is highlighted.
The occurrences of logs are displayed on a timeline.
This patch changes `RoomEventCacheState::waited_for_initial_prev_token`
from `Arc<AtomicBool>` to `bool`. First off, the `Arc` wasn't used in
any useful way (never cloned for example). Second, the `AtomicBool` was
always used as a regular bool, no atomicity was really used. Lastly,
this patch adds the `assume_has_waited_for_initial_prev_token` method to
replace the `= true` operation to make code a bit more readable.
This patch removes the write lock acquisition in
`RoomEventCache::subscribe_to_pinned_events` to replace it by read lock
acquisition. Nothing requires a `&mut self` at any point in this flow,
since `OnceLock::get_or_init` needs a `&self`.
The `RoomEventCacheStateLockWriteGuard::subscribe_to_pinned_events`
method is moved onto `RoomEventCacheStateLockReadGuard`.
`LinkedChunk::push_items_back` expects an `IntoIter<Item = Event>`.
`EventLinkedChunk::push_live_events` has an `events` of kind `&[Event]`.
Using `events.iter().cloned()` instead of `events.to_vec()` not only
removes one `Vec` allocation, but it allows the compiler to apply more
optimisation.
Checking on godbolt.org, I see twice fewer LLVM IR lines, and 2.8x times
less ASM code.
This patch relies on `MembershipChange` to decide when a `m.room.member`
represents a `LatestEvent` candidate. It was a mistale to rely on the
`membership` strictly, because the `prev_content` must be taken into
account.
Thus, this patch adds the following cases to `Knocked`, `Joined` and
`Invited`: `InvitationAccepted` and `KnockedAccepted`. Moreover, this
patch excludes other cases, including `ProfileChanged`, which was a bug
previously! When the user had a new display name, it was considered as a
`LatestEvent` candidate.
This patch changes an `info!` log to a `trace!`. Latest Events are
pretty stable now, and we don't get an info for each new computation
except when we want a proper trace.
This patch changes an `error!` log to a `info!`. Indeed, this is not an
error to compute a Latest Event that doesn't exist yet. The system is
lazy purposely.
Small bug fix that makes `remove_child_from_parent` behave like
`add_child_to_parent`.
Additionally introduces a new Error case so that clients can decide to
ignore any failures updating the child → parent relationship.
## Problem
When a room goes through:
join -> leave/kick -> re-invite
Sliding Sync responses still include the room, but the SDK does not emit
an
Invited/Knocked update. Because of this, RoomListService is never
notified and
the room disappears from the room list until the application restarts.
## Root cause
"update_any_room()" only returned a RoomUpdateKind when one was
explicitly
produced. For re-invites it could return "None", causing the room to be
skipped.
## Fix
Always emit a default:
- "RoomUpdateKind::Invited"
- "RoomUpdateKind::Knocked"
when the room state is Invited/Knocked but no update kind was generated.
## Tests
Added a regression test covering:
join -> leave -> re-invite to ensure the room is surfaced again in the
invited list.
---------
Signed-off-by: Lakshya Nayak <89520692+VEL0C1TY22@users.noreply.github.com>
This patch fixes a design issue. The
`BaseClient::room_info_notable_update_sender` is moved inside
`BaseStateStore` so that, when creating a new `BaseStateStore`, the
updates are not shared with other state stores. Updates are isolated to
the state store.
This bug has surfaced in `BaseClient::clone_with_in_memory_state_store`,
where a new `BaseStateStore` is created, but the
`room_info_notable_update_sender` was _cloned_, and that is a bug! We
could have re-created a new channel from scratch, but it would have
been hacky. Semantically, this channel should be part of the state store
itself. One proof is how it simplifies many call-sites, functions,
methods and structs: the `room_info_notable_update_sender` was passed
to multiple methods on `BaseStateStore`.
There were two issues:
- first, `load_or_fetch_event_with_relations()` allowed to pass a
filter, but the filter wasn't taken into account when fetching relations
from the network. This would cause the initial load of pinned events to
also include thread responses, which we don't want.
- similarly, when adding related events from sync, we'd only look if an
event had a `m.relates_to` field; but it could be a thread response
being added in live.
The two issues are fixed similarly, by using a new `extract_relation`
serde helper that gives both the related_to event and the relation type.
That way, we can apply a manual filter in
`load_or_fetch_event_with_relations` after fetching relations from
network, and we can filter out live events based on the relation type.
We usually don't care if the event was redacted or not, we usually want
to no whether a field is set or not, so we don't need `Original` and
`Redacted` variants.
This simplifies several parts of the code since we don't have to handle
the intermediate enum to access the content now. Due to new APIs in
Ruma we can also just convert original and redacted event contents to
possibly redacted event contents.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch updates the `cargo xtask log sync` command to extract the
`pos` and `timeout` fields so that we can display them. The `timeout`
is displayed in its own column, while the `pos` is displayed in the
line summary.
There was nothing called `RoomEventCacheState` anymore, and the `Inner`
suffix is dubious, at best. Also, we can get rid of the `Lock`
component, since indeed it's locked, but it's a detail from the point of
view of the `RoomEventCacheState` itself. This makes for a shorter and
nicer name.
It allows having way smaller binaries while still being able to have proper backtraces: `reldbg` is great for iOS because it allows inline debugging using LLDB in Xcode, but it produces enormous binaries, while for Android we can't use that properly and we'd only be interested in having symbolicated backtraces, which this profile achieves with binaries an order of magnitude smaller.
This is intended for reducing the binary size of the SDK distributed in Android/iOS bindings.
Its optimization level is 'binary size', it contains LTO optimizations and by default removes part of the debug info - the rest can be removed later if needed.
The method is kept on the pinned loader trait at the moment, because
it's too inconvenient to remove it quite yet. This will happen in a
subsequent PR.
The tests removed are either covered by other tests or
ensure properties that shouldn't exist - e.g., that
new chunks can link to chunks that haven't been put
into the store yet.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
This will come in handy once we attempt to support MSC4388 as this MSC
changes how the client-server API operates. The rendezvous channel will
in the future operate on JSON messages where we'll include our sealed
ciphertext as a base64 encoded string.
This patch adds the `LatestEventValue::RemoteInvite` variant. The goal
of this is to be able to compute a `LatestEventValue` for an invite to
a room. Using `LatestEventValue::Remote` isn't possible because it's
usually built from the `RoomEventCache`. However, the `EventCache`
doesn't handle invites for one reason: invites only manipulate stripped
state-events, whist the `EventCache` manipulates non-stripped (state)
events.
The `LatestEvents` API receives a stream of `RoomInfoNotableUpdate`. It
reacts to update from the `RoomInfo`. It filters out all reasons except
`MEMBERSHIP`. When the `MEMBERSHIP` is updated, and the room' state is
`Invited`, then a `RemoteInvite` is computed.
The `Invite` type is updated to include the `inviter_id` in case the
`inviter` is missing. Indeed, we always know the user ID of the inviter,
this information isn't optional.
This patch replaces the `interact(…).unwrap()` by a proper error.
So far, `interact()` was only returning `InteractError::Panic`
despites `InteractError::Aborted` exists. With
https://github.com/deadpool-rs/deadpool/pull/461, we now get
`InteractError::Aborted` when the SDK is shutdown, sometimes. This
results in hitting the `unwrap` and having a panic again. This patch
solves the problem by changing the `unwrap` to a proper error. Note: in
case of `InteractError::Panic`, we continue to panic.
This patch makes sense with or without the merge of the PR on
`deadpool`.
This patch introduces a new family of commands: `xtask log`. The goal
is to manipulate logs, to extract the right amount of data we need to
solve specific problems. The first member of this family is `sync` to
visualise logs about the sync process. It presents the sync requests and
responses in a table, with a "timeline" _à la_ network profiler graph.
The code is rather simple, on purpose. The generated HTML reports are
lightweight, and fully standalone: no JavaScript, pure HTML and CSS, no
external resources. These reports can be shared or archived super
easily.
Features:
- requests/responses are grouped by connection ID
- permalink to specific request ID
- status have colours
- time is displayed in a human form
- duration is calculated from the log timestamps
- view syncs in a "tree-like" flavor, a "time graph", super quick to
spot long requests
- each line can be "opened" to see details, so far only log line numbers
to get more context manually
MSC4388 won't have the full rendezvous URL encoded in the QR code,
instead it'll have a rendezvous ID.
So let's remove this accessors and use the `intent_data()` getter
instead.
MSC4388: https://github.com/matrix-org/matrix-spec-proposals/pull/4388
Since the intent is encoded differently in MSC4108 and MSC4388 it
doesn't make sense to publicly expose the binary constants for a
specific MSC in the public API.
This patch removes access to the binary constants and you only access to
a generic enum.
This patch modifies the QrLoginData, it now hides all its public fields
and appropriate getters have been created for it instead.
This is necessary to hide the MSC specific parts of the data type thus
allowing support of multiple versions of the data type.
Our current implementation of this QR code data type corresponds to the
data type defined in MSC4108. The data format has been updated a bit in
MSC4833 and thus we'll need to support both formats for a while.
This moves al the MSC4108-specific parts into a separate MSC-specific
submodule.
MSC4108: https://github.com/matrix-org/matrix-spec-proposals/pull/4108
Instead of going through `serde_json::to_value()` and then converting it
to canonical JSON to serialize it.
Currently `to_canonical_value()` has the same behavior internally as
here, but an upcoming change in Ruma makes it use its own `Serializer`
so it is directly serialized as a `CanonicalJsonValue` which should be
somewhat more efficient.
This upcoming change also removes the `CanonicalJsonError::SerDe`
variant, so it is not as straightforward to propagate
`serde_json::Error`s.
This commit also removes outdated `#[allow(clippy::…)]` attributes.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Sometimes, running the wasm tests with a given runner will fail with
obscure, undecipherable reasons. As a result, it's convenient to be able
to locally run the wasm-pack tests with only a single runner, which this
commit allows.
There is an error in how JNA performs the API checksums that results in incorrect checks that make the app fail as soon as any SDK method is called, since initializing the SDK performs these checks
This test was setting a client-wide retry limit of 3 attempts for every
single network request. It happens that it was using the login method,
which unconditionally overrides this retry limit to 3 anyways, in
`LoginBuilder::send()`, so it worked only because the two retry limits
were accidentally in sync. Changing the retry limit in the test to 4
would make it thus fail; the test has been changed so it tries to use
the /whoami endpoint instead of login, as the former doesn't override
the retry limit.
The `default_timeout` is a timeout value provided by `backon`, and
that's a suggestion of what the timeout value should be for the next
request (based on the backoff method used under the hood — in our case,
the exponential backoff). Since we have another concept of a default
timeout (the one present in the RequestConfig), it seems better to call
the timeout suggested by backon in a different manner, that's more
explicit in the given context.
The sync lock was acquired too late in `LatestEvent::store`, which could lead to a race condition where we read some room info, that room info was modified and saved in parallel somewhere else, and then we modified the copy of the room info and overwrote that saved data with it, resulting in data loss.
In the clients, this was experienced as notifications sometimes lacking the room display name
Use case: Display membership changes (join, leave, etc.) in the timeline but suppress profile changes (display name or avatar URL). This is currently not possible with `TimelineEventTypeFilter` because both types of changes have the same event type (`m.room.member`).
This pull request introduces a new `TimelineEventFilter` for filtering on either the event type or parts of its content. Content filters are only added for membership and profile changes but more enum variants can be added in future.
While https://github.com/matrix-org/matrix-rust-sdk/pull/6017 is mostly
functional, there are two issues:
- I did not process `changes.room_key_bundles_fully_downloaded` in
`matrix-sdk-sqlite`, meaning any updates made via `Changes` would not be
persisted;
- I used a non-encrypting `JsValue` serialisation for the same field in
`matrix-sdk-indexeddb`, which causes errors when passed to the
decryption-enabled deserializer.
Solutions:
- Process the aforementioned changes such that keys are added to SQLite;
- Use a non-encrypting deserialiser, since this is effectively a
hash-set, and the contents aren't sensitive.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
If the homeserver provides a state event to a client, it means that it
considers the event to be valid. If a state event is valid, it always
updates the state map of the room. So ignoring events that fail to
deserialize means that the local state map is different than the one
from the server.
In some cases the Matrix spec even explicitly says that if a required
field is missing from the content of a state event, it should be treated
as if the event is missing from the state map. And if a required field
is missing, the event will fail to deserialize.
So this handles state events very closely to how a server would we only
deserialize the event type and the state key first to make sure that a
valid state event always updates the local state map. Then we only
deserialize the events lazily when we encounter an event type that
updates the `RoomInfo`. Because we deserialize the event lazily and some
methods might edit parts of an event before passing it to `RoomInfo`,
the (possibly edited) deserialized event is cached alongside the raw
event and its keys to be able to pass it further down the chain.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch replaces `HashMap` by `BTreeMap` in `EventCacheInner` as
we don't need any ordering. It is then faster to get (or insert) a new
`RoomEventCache`:
- an insert is O(1) for `HashMap` vs. O(log(n)) for `BTreeMap`,
- a get is O(1) for `HashMap` vs. O(log(n)) for `BTreeMap`.
This patch fix an off-by-one check for `Error::InvalidItemIndex` in
`LinkedChunk::remove_item_at`.
This patch updates the `test_remove_item_at` test to cover this bug.
Before: after uploading a media and a thumbnail with the send queue, the
thumbnail would be cached only as a "file", and not as a thumbnail. This
is wasteful, if the embedder is interested in getting a thumbnail of the
exact same dimensions, for the file they've updated; there's no good
reason to wait for the server to return it back.
However, there were good reasons to store it as a file in the past. So,
we're choosing here to duplicate the thumbnail in the media store:
- it's saved as a file for its own MXC URI (which preserves the previous
behavior)
- it's also saved as a thumbnail for the media MXC URI (which implements
the desired behavior).
Tests are updated to reflect this.
This is half of the work: this will load the threaded receipt for each
thread, every time we add/update a timeline item for an event that had a
thread summary. Since we don't know which of the private or the public
receipt is the most advanced, we simply pass both, to start with; it's
expected that this code dies later, when we fold it in into the event
cache.
The second half will consist in updating the thread summaries when a new
read receipt event happens.
This patch adds a new state event candidate for `LatestEventValue`:
`m.room.member` when the `membership` is `join` and the `state_key`
is the current user ID. Put differently: when the current user joins a
room, we are able to compute a `LatestEventValue`.
The `SpaceFilter`s API provides a simple interface that can be used in conjuncture with the `RoomList` to filter down the hierarchy to a particular space or its descendants.
Per design, the first level `SpaceFilter`s will only contain direct descendants while the second level ones will contain the rest of the hierarchy recursively.
The full feature is defined in https://github.com/element-hq/element-meta/issues/2966
This patch ensures that a `LatestEventValue` is erased when a room has
been emptied.
If we are computing a value from the Event Cache, it's because we
have received an update from the Event Cache. This update falls in two
categories: either an event has been added or updated, or the room has
been emptied. We consider the room has been emptied by default. If we
are able to scan at least one in-memory event, we consider the room has
not been emptied.
This patch adds one specific, and updates other tests that were using
an empty Event Cache (which now produces a different result in this
situation).
This patch splits the `latest_events/latest_event.rs`
module into `latest_events/latest_event/mod.rs` and
`latest_events/latest_event/builder.rs`. The file was too big and asked
for a diet. The `LatestEventValueBuilder` type has been renamed to
`Builder`, and the `LatestEventVAluesForLocalEvents` has been renamed
to `BufferOfValuesForLocalEvents` for the sake of clarity and shorter
names.
This fixes an issue that prevented default values coming from the SDK from being uploaded in the create room request, which could mean rooms created with the wrong power levels if the default values in the homeserver didn't match those of the SDK
This patch introduces the `PollTimeout` type to represent
either no timeout with `PollTimeout::None`, some timeout
with `PollTimeout::Some(_)`, or a default timeout with
`PollTimeout::Default`. It's finer than the previous `bool` that
was used, where `false` meant `PollTimeout::None`, and `true` meant
`PollTimeout::Default`. It's now possible to pass a precise timeout
value.
Hello!
As the `Not` filter is not available on the FFI SDK due to UniFFI
constraints, this PR adds a NonFavorite filter to RoomList, implemented
as the negation of the existing Favorite filter. This was made to
address the issue #5978.
- [x] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by:
---------
Signed-off-by: JoFrost <20685007+JoFrost@users.noreply.github.com>
In the send queue, failures to send can be classified into two
categories:
- permanent failures (e.g. invalid parameters)
- recoverable failures (e.g. network is down; server responded with a
transient error code)
The latest event system would classify all the failures as "cannot be
sent", which is slightly incorrect if the failure was recoverable. In
this case, we should still consider the local event as being sent, as
the system should try to send it some time soon.
https://github.com/matrix-org/matrix-rust-sdk/pull/4568 merged together
`SyncTimelineEvent` and `TimelineEvent`, but removed some of the useful
documentation on `TimelineEvent`, and left behind some confusing
references. This change fixes it up.
This patch changes `subscribe_to_rooms` in `RoomListService` to replace
the old subscriptions. It avoids accumulating subscriptions forever and
is closer to the old `visible_rooms` sliding sync list behaviour.
This patch adds two methods on `SlidingSync`: `unsubscribe_to_rooms`
and `clear_and_subscribe_to_rooms` to respectively remove many room
subscriptions, and to replace room subscriptions by new ones.
Our information were wrong. `room_subscriptions` is definitely not
sticky, we must send them for each request.
This patch removes `RoomSubscriptionState`. Subscriptions are not marked
as “applied” anymore, they are always sent.
For some reason, the automatic WAL checkpoints don't seem to be working as expected. Since we should periodically run VACUUM operations, we might as well add checkpoints before vacuuming (so the WAL size is reset and can grow to fit the whole DB) and after (so we clean up after that).
This patch restores a couple of assertions from a recently removed
test where it is asserted that `Request::extensions::to_device::since`
is set from the Olm machine.
This patch extracts `SlidingSyncListStickyParameters::filters` to
no longer make it sticky. We are dropping sticky parameters as it's not
part of the last MSC.
THis patch extracts `SlidingSyncListStickyParameters::required_state` to
no longer make it sticky. We are dropping sticky parameters as it's not
part of the last MSC.
This patch removes `SlidingSyncStickyManager` as it only contains
the logic for a single request field: `room_subscriptions`. Also,
previously, `room_subscriptions` was considered sent based on the
transaction ID. This is useless as it's always sticky per MSC4186. The
logic from `SlidingSyncStickyManager` is sent inlined, allowing to
effectively remove this type.
This patch extracts `SlidingSyncStickyParameters::extensions` to no
longer make it sticky. We are dropping sticky parameters as it's not
part of the last MSC.
The issue is related to the encrypt_store_dir fn where, when creating
the key file, it doesn't ensure that the parent directory exists first.
It might be not optimal for the user of the crate to ensure in an non
hacky manner, as the sdk iterates through most of its directories
internally. Have a test to verify it, which can be removed later (if
being merged)
Needs a review, might not be the optimal solution as this is my first pr
with the crate and am not that familiar with it (although do use it in
one of my apps).
We can have 3 different states for the same aggregation in
related_events, in chronological order:
1. The local echo with a transaction ID.
2. The local echo with the event ID returned by the server after
sending the event.
3. The remote echo received via sync.
The transition from states 1 to 2 was already handled in
`mark_aggregation_as_sent()`.
But the transition from states 2 to 3 was never handled and we ended up
with both the local echo and the remote echo in the related events.
This resulted in the local echo being chosen over the remote echo when
computing the latest edit only because it was first in the list, even
though it didn't contain the raw JSON of the edit.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Remove unused `ShieldStateCode::SentInClear`
`VerificationState::to_shield_state_{strict,lax}` return a type `ShieldState`, whose inner
type `ShieldStateCode` currently includes a variant `SentInClear` for "unencrypted event".
This is very misleading, because those functions never actually set that variant.
Rather, there is a separate method in `matrix-sdk-ui`, `EventTimelineItem::get_shield`,
which uses the same type, but *does* set that variant where appropriate.
As a user of matrix-sdk-common, without the matrix-sdk-ui layer, this is dangerously
misleading: it gives the impression that we are checking for unencrypted events, when in
fact we are not.
The solution seems to be to use different types for the different levels of the stack.
While we're at it, we fix up some of the confusion of methods that return an `Option` of an
enum type which itself has a `None` variant.
feat: Add `forwarder: ForwarderInfo` to `EncryptionInfo`.
Introduces `ForwarderInfo` which which exposes information about the forwarder of the keys with which an event was encrypted if they were shared as part of an [MSC4268](https://github.com/matrix-org/matrix-spec-proposals/pull/4268) room key bundle.
<!-- description of the changes in this PR -->
- Introduces a new enum `ForwarderData` as a wrapper for valid variants
of `SenderData` for which we can accept key bundles under MSC4268.
- Converts `forwarder_data` in `InboundGroupSession` and
`PickledInboundGroupSession` to use `Option<ForwarderData>` over
`Option<SenderData>`.
- [x] Public API changes documented in changelogs (optional)
---------
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This patch implements a new feature: when a `m.room.redaction` targets
the current [`LatestEventValue`], this one must be erased by the new
computed `LatestEventValue`.
This patch spreads the current value's event ID in
`LatestEventValueBuilder`.
This patch also changes `LatestEventValueBuilder::new_remote` to return
an `Option`, similarly to `new_local`. The `must_overwrite_existing`
variable is set to `true` to keep the existing behaviour, but it's going
to change in the next patch.
This patch is purely a refactoring with no feature change. Most of the
changes are in test to keep track of the _previous value_ so that the
current value's event ID can be calculated instead of hardcoded.
This patch renames a variable. Since `rfind_map_event_id_memory_by`
returns the previous event instead of the previous event ID, this
variable must have been renamed.
Separate the shield types between common and UI, so that we can change common
without breaking UI.
The new type does not include a `message` field: since it cannot be localised,
clients should not be using it.
What's more, this is saved into the event cache and sometimes it overrides another instance of the same event that actually contains the right info. This results in unresolvables UTDs.
This change tries to fetch the session id from the existing event content. It's fixed these kind of UTDs when tested in a real client.
This time we're testing the redecryption of pinned events that were not
part of the main timeline, more importantly we never backpaginated
enough for them to be part of the main timeline and thus never got put
into the event cache.
This test expectedly fails for now.
We can't use `LatestEventValue::None` as an optional value anymore,
since it erases the previous `LatestEventValue`. This patch updates
`LatestEventValueBuilder::new_local` to return an `Option` to handle all
the cases where a local value cannot be computed.
This patch changes the semantics of the Room List `latest_event`
sorter by changing “is local” to “is remote like”, to include the new
`LatestEventValue::LocalHasBeenSent` variant.
The problem we are trying to solve is the following:
- a local event is being sent,
- the `LatestEventValue` is `LocalIsSending`,
- the local event is finally sent,
- the `LatestEventValue` is still `LocalIsSending` purposely, with the
hope that an update from the Event Cache will replace it.
But sometimes, this update from the Event Cache comes **before** the
update from the Send Queue. Why is it problem? Because updates from the
Event Cache are ignored until the buffer of local `LatestEventValue`s
aren't empty, which means that if an update from the Event Cache is
received before `RoomSendQueueUpdate::SentEvent`, it is ignored, and the
`LatestEventValue` stays in the `LocalIsSending` state. That's annoying.
The idea is to introduce a new state: `LocalHasBeenSent` which mimics
`Remote`, but for a local event. It clarifies the state of a sent event,
without relying on the Event Cache.
This patch simplifies the code after the recent refactorings.
It uses `LatestEventValue::is_none()` to replace a `matches!`, and it
replaces the last use of `new_remote` by `new_remote_with_power_levels`
to finally rename this latter to `new_remote`.
The `LatestEventValue` can be `None` (the default value) but the Event
Cache contains enough data to compute a `Remote(_)` one. The system
lazily triggers a `LatestEventQueueUpdate` to achieve that.
Previously, `LatestEventValue` was always initialized from the
`RoomEventCache`. Now, it is restored from the `RoomInfo`. First off,
this is something we wanted to do since a long time. Second, it is
more performant. Third, it allows the system to be lazier. Indeed,
it's possible that when a `LatestEvent` is created, its correspond
`RoomEventCache` doesn't exist yet. It happens during a sync when a room
is new: latest events are registered, but their `RoomEventCache` aren't
yet created. By postponing the use of `RoomEventCache`, the system is
lazier and more solid.
Bonus, less methods are async, which simplifes the workflow.
This patch removes `RoomRegistration` along with the full room
registration mechanism. It's been introduced to remove contention
around the `RegisteredRooms` lock, but it actually creates more async
flows, which makes the `latest_events` logic a bit less predictable.
By removing this room registration mechanism, our hope is to make the
result more predictable and less buggy in appareance. Our real-life
tests have shown that the lock contention isn't problematic, especially
since `RoomLatestEventsReadGuard` and `RoomLatestEventsWriteGuard` have
been introduced.
This test for one reason or the other sporadically panics with an:
> RuntimeError: unreachable
Let's disable this test on Wasm for now since the memory store isn't
that relevant anyways, especially not on Wasm.
This patch abstracts away the cryptographic channel which is used in the
SecureChannel implementation for the QR code login support.
This will allow us to use HPKE alongside of ECIES since MSC4108 recently
proposed the switch to HPKE.
This patch revisits the way redacted and redaction events are handled in
the Latest Event.
Previously, all redacted events were considered suitable candidate. It's
no longer the case.
Redaction and redacted events are no longer considered suitable.
This patch also revisits `rfind_map_event_in_memory_by` to return a
`&TimelineEvent` instead of an `OwnedEventId`, which could be more
performant in the future.
The tests have been updated accordingly.
---
* Fix https://github.com/matrix-org/matrix-rust-sdk/issues/5899
* Address https://github.com/matrix-org/matrix-rust-sdk/issues/4112
Signed-off-by: Stefan Ceriu <stefanc@matrix.org>
Co-authored-by: Stefan Ceriu <stefanc@matrix.org>
This patch increases the capacity of the room registration channel. The
hope is that it can reduce the need to wait on available permits under
heavy load.
This patch removes the use of `Fuse` in the
`listen_to_event_cache_and_send_queue_updates` task. `mpsc::Receiver`
and `broadcast::Received` are cancellation safe.
This patch reduces the lifetime of the write locks in
`RegisteredRooms::room_latest_event` when the `RoomLatestEvents`
doesn't exists. If the `room_registration_sender` channel is full,
it has to wait. When waiting, the write lock is still alive, probably
blocking other operations. The idea is to create the `RoomLatestEvents`,
to downgrade the write lock to a read lock, and then to send the
registration message onto the `room_registration_sender`.
In the implementation of EventCacheStore, there are a number of
places where the upper and lower bounds of an EventId are
constructed. It is important to bypass hashing and encryption
when constructing these bounds, otherwise the values will be
modified and will no longer represent the bounds.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
Note that the encrypted tests were actually being run unencrypted.
Introducing a store cipher causes them to run encrypted, and
furthermore, reveals some bugs which are only visible when running
an encrypted event cache store.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
This path adds events or state events to force the test to execute as
expected since a change in `bump_stamp` alone doesn't trigger a room
list update anymore.
This patch fixes a “bug” in the Room List. It's updated by
`RoomInfoNotableUpdateReasons`. However, 1 reason is creating
unnecessary updates: `RECENCY_STAMP`. The Room List is already updated
by the Latest Event. One usage of the Latest Event is to sort the Room
List by recency. Thus, since the Room List is updated by `LATEST_EVENT`,
we can ignore `RECENCY_STAMP`.
This is not supported by Ruma. The join_rule field, despite being
defined as a pure string, can have associated data to it based on the
join rule variant.
This means that custom and unknown enum variants might lose data when
reserializing.
Let's just skip the serialization of custom join rules in the RoomInfo,
the concrete value is still available in the state store, it's just not
kept at hand in the RoomInfo.
Signed-off-by: Damir Jelić <poljar@termina.org.uk>
Co-authored-by: Ivan Enderlin <ivan@mnt.io>
This method will retrieve the database sizes if available and expose it in the client.
Note: the actual database size measuring is only implemented for the SQLite based stores
The event has been sent to the server and the server has received it.
Yepee! Now, we usually wait on the server to give us back the event via
the sync.
Problem: sometimes the network lags, can be down, or the server may be
slow; well, anything can happen. It results in a weird situation where
the user sees its event being sent, then disappears before it's received
again from the server.
To avoid this situation, this patch eagerly saves the event in the Event
Cache. It's similar to what would happen if the event was echoed back
from the server via the sync, but we avoid any network issues. The Event
Cache is smart enought to deduplicate events based on the event ID, so
it's safe to do that.
This patch removes some logs around the cross-process lock methods. This
is called pretty often by the cross-process lock task, which pollute the
log files.
Making network requests before actually building a client interferes with offline support, especially so in lie-fi situations.
The method is exposed through FFI though and can be used at the final user's discretion (e.g. when submitting a bug report).
We need to handle 2 possible deadlocks for this:
1. We cannot try to refresh an expired access token if this call happens
while we are currently trying to refresh the token. The easiest way
to handle this is to never try to refresh the token when making this
call inside `get_path_builder_input()` so we implement a "failsafe"
mode that disables refreshing the access token in case it expired.
However it attempts the GET /versions again without the token.
2. We cannot access the cached supported versions if we are in the
process of refreshing that cache because the RwLock has a write lock.
So if the access token has expired and we try to refresh it, the
possible calls to `get_path_builder_input()` must not wait for a read
lock to be available. So the solution is to never wait for a read
lock, and skip the cache if a read lock is not available.
This also gets rid of workarounds in other functions.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This will allow to handle automatically whether to send an access token
or not on endpoints that don't require it in contexts were can't refresh
it.
We also don't cache calls to GET /versions that were not authenticated,
because they might lack some features compared to an authenticated
request.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
In theory clients shouldn't make requests to the server-server API. A
way to work around it for this specific case would be to implement
MSC4383.
In the meantime, clients that don't want to use
`Client::server_vendor_info()` won't have to build the extra
dependencies added by ruma-federation-api.
The feature is enabled for the bindings, so it isn't a breaking change
for matrix-sdk-ffi.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Exposes the `is_space` flag to FFI in the `NotificationRoomInfo`, so
that a client can tell through a notification if the room that generated
it, is a space or not.
This patch changes the rule of what is a `LatestEventValue` candidate
in case of an edit. An edit must target/relate to its immediate previous
event to be a candidate. Otherwise it's easy to edit an old message
and create a “broken” `LatestEventValue` because it points to an older
message that the user may not be able to find easily.
The supported versions are necessary for querying almost all endpoints,
but after homeserver auto-discovery the well-known info is only
necessary to get the MatrixRTC foci advertised by the homeserver. So it
shouldn't be necessary to always request both at the same time.
Besides:
- Not all clients support MatrixRTC, so they don't need the well-known
info.
- The well-known info is only supposed to be used for homeserver
auto-discovery before login. In fact, the MatrixRTC MSC was changed to
use a new endpoint for this.
- We don't have access to the server name after restoring the Client, so
the well-known lookup is more likely to fail.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch fixes a bug where a new local `LatestEventValue`
was always created as `LocalIsSending`. It must be created as
`LocalCannotBeSent` if a previous local `LatestEventValue` exists and is
`LocalCannotBeSent`.
This patch adds the companion test too.
This patch improves the Wasm support of the matrix-sdk-ffi crate.
First a uniffi feature needed to be enabled.
Secondly a bunch of methods which don't work under Wasm have been stubbed out.
Signed-off-by: MTRNord <MTRNord@users.noreply.github.com>
Co-authored-by: MTRNord <MTRNord@users.noreply.github.com>
This patch changes the `Semaphore(permit=1)` for a `Mutex`: the
semantics is strictly equivalent, but it removes the need to guarantee
there is a single permit.
This patch updates the reloading of `RoomEventCacheStateLock`
when the cross-process lock over the store is dirty to broadcast
`RoomEventCacheUpdate` and `RoomEventCacheGenericUpdate`. That way the
`Timeline` and other components can react to this reload.
This patch adds the new `test_reset_when_dirty` test, which ensures
the state is correctly reset when the cross-process lock over the store
becomes dirty.
This patch fixes a problem found in a test (not commited yet) where it
was impossible to do multiple calls to `read` if the first guard was
still alive. See the comments to learn more.
This patch extracts fields from `RoomEventCacheState` and move them
into `RoomEventCacheStateLock`. This lock provides 2 methods: `read`
and `write`, respectively to acquire a read-only lock, and a write-only
lock, represented by the `RoomEventCacheStateLockReadGuard` and the
`RoomEventCacheStateLockWriteGuard` types.
All “public” methods on `RoomEventCacheState` now are facade to the read
and write guards.
This refactoring makes the code to compile with the last change in
`EventCacheStore::lock`, which now returns a `EventCacheStoreLockState`.
The next step is to re-load `RoomEventCacheStateLock` when the lock is
dirty! But before doing that, we need this new mechanism to centralise
the management of the store lock.
This patch changes `EventCacheStoreLockState` to own a clone of
the inner store. It helps to remove the `'a` lifetime, and so it
“disconnects” from the lifetime of the store.
This patch replicates the `is_dirty` and `clear_dirty` methods from
`CrossProcessLock` to `CrossProcessLockGuard`. It allows to get an
access to this API from a guard when one doesn't have the cross-process
lock at hand.
This patch updates `EventCacheStoreLock::lock()` to return an
`EventCacheStoreLockState` instead of an `EventCacheStoreLockGuard`, so
that the caller has to handle dirty locks.
This patch adds the `CrossProcessLockState::map` method along with its
companion `MappedCrossProcessLockState` type. The idea is to facilitate
the creation of custom `CrossProcessLockState`-like type in various
usage of the cross-process lock.
These will use `bridge_trace_id` to map an exising client transaction/span to this one so they'll be displayed as a single one in Sentry.
This is done through the `sentry.trace` field, which will be used by `sentry-tracing` to differentiate these kinds of special spans.
The special fields need to be added on the Span creation, that's why we do it in the constructor instead of just using `span.record(...)` later.
This patch adds a `profile: TimelineDetails<Profile>` field to
`LatestEventValue::Local` in `matrix_sdk_ui::timeline` (and the
corresponding `matrix_sdk_ffi` type).
This patch fixes a race condition where events won't get decrypted
because a room key arrives after the initial decryption attempt but
before the UTD has been persisted in the event cache.
The fix is relatively straightforward, we'd need a synchronization point
for the two different tasks, the event cache which adds events and the
redecryptor which listens to room keys to decrypt events.
A lock could have been used, so the storing and redecrypting of events
becomes synchronized via the storage layer. This approach could have
degraded performance since the event cache needs to handle a lot of
events.
The approach that was chosen here is to let the redecryptor listen to
updates coming from the event cache itself. If the event cache tells us
that it persisted a UTD, we will attempt to decrypt. Upon a successful
decryption we will replace the event in the cache as well.
This patch adds a convenience function for the Update enum. If one only
cares about the items contained in the Update, then they can chose to
use this method to extract them out of the enum.
This patch adds a test confirming that the redecryptor has a race
condition.
Namely, events and room keys are received over two different sync
streams from the homeserver. When events are received over the sync, we
first try to decrypt them, this might fail because the room key hasn't
yet arrived over the other sync stream. The event cache will then
persist the event as a UTD.
At the same time, the redecryptor will listen to room keys that arrive
on the other sync stream. Once the redecryptor gets notified about a
room key, it will attempt to fetch the event from the event cache to
decrypt the event and replace it.
Crucially if the key arrives before the event gets persisted but after
the initial decryption attempt we might never attempt to redecrypt such
an event.
We had an instance where a user joined a room on Element X but did not download
the key bundle, so let's add some logging to help figure out what was going on.
This patch uses the newly introduced
`SlidingSyncListBuilder::requires_timeout` to define when the
`RoomListService` must apply a long-polling depending on its state
machine.
This patch adds a new `SlidingSyncListBuilder::requires_timeout` method
that takes a function deciding whether the list requires a timeout, i.e.
if the list should trigger a `http::Request::timeout`, i.e. if it
deserves a long-polling or not.
The default behaviour is kept for compatibility purposes.
This patch updates `SlidingSyncListInner::state` from a
`RwLock<Observable>` to a `SharedObservable`. It is semantically and
programmatically identical, but the API is simpler.
Previously, this used the latest event in the thread as the event to mark as read, while this is not right if we're in a context that hides thread events
This task is still necessary because the redecryptor in the event cache
might miss some room keys.
In this case the timeline can tell the redecryptor which events it
should retry to decrypt.
We're collecting all the UTDs in the timeline and telling the
redecryptor to do its best.
Add an integration test that checks that, when we receive a copy of a megolm
session directly after having previously received it via history sharing, we
get the best bits of both.
... with `merge_received_group_session`.
`merge_received_group_session` expands the logic of `compare_group_session` to
handle the fact that there is more than one axis of "better" or "worse" and we
may need to take the best bits of two copies of the session.
In order to correctly merge sessions, we need more granular comparisons between
two sessions than just "Better" or "Worse", so factor out a method that *just*
looks at the ratchet states.
Expose the room join rules in the `OtherState::RoomJoinRules` event
for the FFI timeline.
It reuses the existing `JoinRules` type from the client module and
converts the event content accordingly. This allows clients to inspect
the room’s current join rule directly from the event. Like `m.federate`,
this field was previously unavailable in the FFI variant of the SDK.
---------
Signed-off-by: JoFrost <20685007+JoFrost@users.noreply.github.com>
The previous message implied that we had received a session for this
message, but that is only one of the several reasons we might encounter
this situation. If redecryption failed, it is more likely we got here
because we'd been asked to attempt redecryption for all UTDs e.g. when
we build a new timeline.
Additionally, having similar wording for the error case and the unable
to decrypt case could also cause confusion, so I adjusted the wording to
make clear which situation is happening.
This patch replaces the `into_guard()` call by a `match` over
`CrossProcessLockKind` so that the `Dirty` case is explicitly handled.
The mid-term idea is to remove the `into_guard()` method because it
is “dangerous” as it hides the `Dirty` case.
This patch undo an optimisation that was initialising the
`RoomListService` at the `SettingUp` state if a `pos` value was
recovered successfully (see bbf9bf2c0b).
The problem is that it starts with a range of 0..99 instead of 0..19,
which can slow things done in particular cases. Whilst a good idea on
paper, it's not in practise. So let's continue to recover the `pos`, but
let's keep starting at the `Init` state.
This patch removes the `connection::Config` type. It was “inspired”
from `deadpool_sqlite`, but we can clearly remove it by using our own
`SqliteStoreConfig` type. It simplifies the way we open a database.
This patch replaces `deadpool-sqlite` by our own implementation in
`crate::connection`. It still uses `deadpool` but the object manager has
a different implementation.
This avoids the scenario where the mock server gets deallocated before
the rendezvous server and thus the rendezvous specific mock guards.
Dropping those in the wrong order will result in a panic.
Hello, I'm writing on behalf of the Citadel product developed by ERCOM.
This PR expose `m.federate` and `history_visibility` in timeline diffs.
These fields are available in the Matrix SDK but were previously omitted
from the FFI variant.
Signed-off-by: JoFrost <20685007+JoFrost@users.noreply.github.com>
This patch changes the signature of `CrossProcessLock::try_lock_once`.
It was returning a:
```rust
Result<CrossProcessLockResult, CrossProcessLockError>
```
Now it returns a:
```rust
Result<Result<CrossProcessLockKind, CrossProcessLockUnobtained>, L::LockError>
```
We will explain these new types in a moment.
This patch also changes the signature of `CrossProcessLock::spin_lock`.
It was returning a:
```rust
Result<CrossProcessLockGuard, CrossProcessLockError>
```
Now it returns a:
```rust
Result<Result<CrossProcessLockKind, CrossProcessLockUnobtained>, L::LockError>
```
First off, we notice that the returned types are now unified. The
`CrossProcessLockResult` type has been renamed `CrossProcessLockKind`
and lives in a `Result::Ok`. The `CrossProcessLockResult::Unobtained`
variant has been removed, but `CrossProcessLockUnobtainedReason`
has been renamed to `CrossProcessLockUnobtained` and lives in a
`Result::Err`.
Second, the `CrossProcessLockError` now is a union type between
`CrossProcessLockUnobtained` and `TryLock::LockError`. It's not used
by `try_lock_once` or `spin_lock`, but only by the code using the
cross-process lock to provide a unified error type.
The ideas behind these changes are:
- it's easy to forward an error from the `TryLock`,
- it's difficult to ignore the `Clean` vs. `Dirty` state of the lock
guard,
- unified API with clearly separated responsibility (the first `Result`
vs. the second `Result`).
Note: the `CrossProcessLockKind::into_guard` method aims at being
removed. It's useful now to maintain compatibility but it's “dangerous”
as it makes trivial to skip `Clean` vs. `Dirty` states. We ultimately
don't want that.
This is necessary because both the timeline and the event cache attempt
to redecrypt events currently.
This will change once only the event cache handles this task.
This patch detects when the cross-process lock has been dirtied.
A new `CrossProcessLockResult` enum is introduced to simplify the
returned value of `try_lock_once` and `spin_lock`. It flattens the
previous `Result<Option<_>>` by providing 3 variants: `Clean`, `Dirty`
and `Unobtained`.
This patch adds `Lease::generation` support in the crypto, media and
event cache stores.
For the crypto store, we add the new `lease_locks` object store/table.
Previously, `Lease` was stored in `core`, but without any prefix, it's
easy to overwrite another records, it's dangerous. The sad thing is
that it's hard to delete the existing leases in `core` because the keys
aren't known. See the comment in the code explaining the tradeoff.
For media and event cache stores, the already existing `leases` object
store/table is cleared so that we can change the format of `Lease`
easily.
This patch adds `CrossProcessLockGeneration`. A lock generation is an
integer incremented each time the lock is taken by another holder. If
the generation changes, it means the lock is _dirtied_. This _dirtying_
aspect is going to be expanded in the next patches. This patch focuses
on the introduction of this _generation_.
The `CrossProcessLock::try_lock_once` method, and
the `TryLock::try_lock` method, both returns a
`Option<CrossProcessLockGeneration>` instead of a `bool`: `true` is
replaced by `Some(_)`, `false` by `None`.
Creating a `RoomMember` takes a lot of store queries, and previously all of them were done sequentially. I've tried to make this process run as much in parallel as I can.
I find a single match statement easier to reason about than one nested in another.
Also, import `UnableToDecryptReason::*`, to shorten the match lines.
I want to be able to test that the correct `UtdCause` is presented for withheld
historical messages. That means we need to use `/sync` rather than `/event` to
obtain the message (since the MSC4115 `membership` field is missing on `/event`
(https://github.com/element-hq/synapse/issues/17486)). So then the most
realistic way to get hold of the actual UtdCause is to use a Timeline.
Of course, the thing I actually want to test doesn't actually work correctly,
so it's left as a FIXME in this commit.
When constructing a key bundle, if we had received a key bundle ourselves, in
which one or more sessions was marked as "history not shared", pass that on to
the new user.
This patch introduces the `sqlite` and `indexeddb` feature flag,
enabling the use of SQLite or IndexedDB for the stores. This patch also
introduces the ability to use non-persistent, in-memory stores.
The new `ClientBuilder::in_memory_store`, `ClientBuilder::sqlite_store`
and `ClientBuilder::indexeddb_store` methods are introduced to
configure the stores. This patch adds new `SqliteStoreBuilder` and
`IndexedDbStoreBuilder` structure.
It will allow us to fetch the latest event id coming from the SDK instead of deciding which one to use in the clients, which could be altered by filters, post-processing, etc.
This method shouldn't be widely used, but it's useful when we want to mark the room as fully read when leaving it and at the same time we have to destroy the room and timeline instances immediately so their in-memory cache is cleared
Tests that a to-device `m.room_key.withheld` event can be
serialized (using JSON), then deserialized as a RoomKeyWithheldEntry.
Ensures compatibility with exisiting store data.
This is an unstable feature but as per
[MSC3230](https://github.com/matrix-org/matrix-spec-proposals/pull/3230)
each space room might have an optional
`m.space_order`/`org.matrix.msc3230.space_order` string field in its
room account data defining the lexicographical order in which the spaces
should be displayed, with spaces missing this field shown at the bottom
and ordered by their room id.
This is important since we want to move the redecryption logic out of
the timeline into the main crate. This in turn means that we don't have
such low level access to the redecryption logic.
Not all tests were rewritten:
- `test_retry_edit_and_more` Is proving to be difficult to rewrite,
may come in a separate commit.
- `test_retry_fetching_encryption_info` Needs verification state
changes. Will be rewritten on the event cache layer
A new batch of breaking changes, allowing to stop providing dummy
`SupportedVersions` where they are not necessary.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch moves the `Client::get_dm_room` helper function and its tests
from `src/encryption/mod.rs` to `src/client/mod.rs`, so it may be used
without the `e2e-encryption` crate feature enabled.
- [x] Public API changes documented in changelogs (optional)
Signed-off-by: Ginger <ginger@gingershaped.computer>
This patch adds the complementary login flow for the already existing QR code login support.
Namely, previously it was only possible for the new device to scan a QR code to log in. Now
it's possible for the new device to create the QR code and let the existing device scan it.
- [x] Public API changes documented in changelogs (optional)
Signed-off-by: Johannes Marbach <n0-0ne+github@mailbox.org>
In the previous version of the thread, the following sequence of events
could happen:
- we subscribe to the thread linked chunk changes
- *then*, the events are being added to the thread linked chunk
This is an edge case where, since we've subscribed to the thread linked
chunk, the thread root will be "known" to be part of a thread, and will
be appended to the thread linked chunk.
If the events happen in the other order (first the events are added to
the thread linked chunk, then we subscribe to the changes), then the
thread root will not be part of the thread linked chunk (because when it
arrived, we didn't know it would be a thread root). As such, the thread
linked chunk state would end up being different in this case.
The solution is to make it so that the thread linked chunk is always
subscribed to *before* any events are added to it. This way, we make
sure that we'll always have the thread root in the thread linked chunk.
In this initial version, a redaction will:
- *remove* the event from the thread chunk, as does Element Web,
- update the thread summary to reflect the new number of messages in the
thread, and let us have a thread summary with 0 replies.
A next commit will adapt the code so that a thread summary with 0
replies is removed.
Brings changes to the requests metadata. It was changed from a struct to a trait, and the authentication scheme is now an associated type.
This allows to forbid at compile time requests that use an unsupported authentication scheme.
When receiving an encrypted to-device message, if the sender device is not in
the store, but the event includes `sender_device_keys`, use
`sender_device_keys` to do the verification checks etc.
Fixes: https://github.com/matrix-org/matrix-rust-sdk/issues/5768
It was incorrect to say that the timeline focus can be changed after the
timeline has been created, since it is *not* the case. Also explained
what the default value is.
Most clients will probably work with values between 0 and 1 and need to
convert it just to send it, so we can move that conversion into the SDK.
This is also more forwards-compatible, because MSC3246 now has a
different max value for the amplitude, so when this becomes stable, the
only change needed will be in the SDK.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Currently, when we receive a room key bundle to-device event, we don't look up
the sender device at all, meaning that the message is then marked as "from
missing device", which means that if you turn on "exclude insecure devices",
the message is dropped.
This patch changes the logic so that room key bundle to-device events are
treated the same way as most other to-device events (except room keys, which
continue to be special).
Fixes: https://github.com/matrix-org/matrix-rust-sdk/issues/5613, although the
integration test now fails because instead we hit https://github.com/matrix-org/matrix-rust-sdk/issues/5768.
`Account::parse_decrypted_to_device_event` is getting a bit big and unwieldy,
so factor out the bit that attempts to find the sending device.
(Also, remove an outdated TODO.)
The legacy mention push rules were removed, and the
`contains_display_name` condition was deprecated.
Some tests check for backwards-compatibility with legacy mentions, so we
need to add them back for those tests.
A test with an encrypted event was relying on the legacy mentions, so
the encrypted event was replaced with another one with an intentional
mention.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
StringEnum now also implements Ord, PartialOrd, Eq and PartialEq so it
is not necessary to derive them. Also the ordering used is comparing the
string representation of the variants.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Extended profile fields were stabilized so the old endpoints are now
deprecated, and there are a few other changes.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Adds `DeviceKeys::has_signed` and `DeviceKeys::check_self_signature`, and
removes `DeviceData::has_signed` and `DeviceData::verify_device_keys`.
I just found this easier to grok, and it means we can avoid needlessly turning
a `DeviceKeys` into a `DeviceData` sometimes.
This should only happen when a room has been forgotten and was a room
DMs before. Ideally, we'd clean up the room DM event data, but since
this is slightly more involved, we don't do that here just quite yet.
With Rust 2024, by default `impl` return types use any generic that is
in scope, so in these cases the lifetime of `self`.
But since the return type is actually owned, the returned impl shouldn't
use any lifetime, which is what `use<>` does.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
With Rust 2024, by default `impl` return types use any generic that is
in scope, so in these cases the lifetime of `self`.
But since the return type is actually owned, the returned impl shouldn't
use any lifetime, which is what `use<>` does.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
We don't really need the poll start event entirely, since we're only
interested in the start block and the fallback text. With this, it'll be
simpler to create embedded polls from poll edit events.
Brings a breaking change with event structs being non-exhaustive now,
so they need to be constructed with methods rather than with a struct
declaration.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
There is a breaking change in Ruma and those types are now
non-exhaustive so they can't be built with the struct declaration
anymore.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
We actually want other event formats in those cases, and in most cases
just using `.into()` is enough to generate the proper format.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
And use the proper fields for these formats. We also add more conversion
implementations for the types associated with these formats.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch moves the creation of the `message` in `TracingTimer` if and
only if the log is enabled. Computing it every time is useless, and can
even slow things down (because of the time calculation).
This patch introduces a write-only connection in `SqliteStateStore`
_à la_ `SqliteEventCacheStore`. The idea is to get many read-only
connections, and a single write-only connections behind a lock, so that
there is a single writer at a time.
This patch renames the `acquire` method to `read`, and it introduces a
new `write` connection.
**NOTE:** _this should not be merged until matrix-org/rust-indexed-db#1
is merged! The `[patch]` in this branch should point to the official
`matrix-org` fork of `rust-indexed_db`, but is currently pointed at my
personal fork._
## Background
This pull request makes updates
[`indexed_db_futures`](https://docs.rs/indexed_db_futures/latest/indexed_db_futures/index.html)
in the `matrix-sdk-indexeddb` crate. The reason we'd like to update this
dependency is because the version currently used does not fully support
the Chrome browser (see #5420).
The latest version of `indexed_db_futures` has significant changes. Many
of these changes can be integrated without issue. There is, however, a
single change which is incompatible with the `matrix-sdk-indexeddb`
crate. Namely, one cannot access the active transaction in the callback
to update the database (for details, see Alorel/rust-indexed-db#66).
### An Updated Proposal
Originally, new migrations were implemented in order to work around this
issue (see #5467). However, the proposal was ultimately rejected (see
@andybalaam's
[comment](https://github.com/matrix-org/matrix-rust-sdk/pull/5467#issuecomment-3149550617)).
For this reason, the dependency has instead been `[patch]`ed in the
top-level `Cargo.toml` with a modified version of `indexed_db_futures`
(see matrix-org/rust-indexed-db#1). Furthermore, these changes have been
proposed to the maintainer and are awaiting feedback (see
Alorel/rust-indexed-db#72).
### Why do we need the active transaction in our migrations?
The `crypto_store` module provides access to the active transaction to
its migrations (see
[here](https://github.com/matrix-org/matrix-rust-sdk/blob/ca89700dfe9f29dcd823bb10861807f9d75e0634/crates/matrix-sdk-indexeddb/src/crypto_store/migrations/mod.rs#L211)).
Furthermore, there is a single migration (`v11_to_v12`) in the
`crypto_store` module which actually makes use of the active transaction
(see
[here](https://github.com/matrix-org/matrix-rust-sdk/blob/ca89700dfe9f29dcd823bb10861807f9d75e0634/crates/matrix-sdk-indexeddb/src/crypto_store/migrations/v11_to_v12.rs#L23)).
For clarity, the reason `v11_to_v12` is problematic in the latest
versions of `indexed_db_futures` is because it is simply adding an index
to an object store which was created in a different migration and this
requires access to the active transaction. All the other migrations
create object stores and indices in the same migration, which does not
suffer from the same issue.
## Changes
- Move `indexed_db_futures` to the workspace `Cargo.toml` and add a
`[patch]` so that it points to a modified version.
- Add `GenericError` type and conversions in order to more easily map
`indexed_db_futures` errors into `matrix-sdk-*` errors.
- Update all IndexedDB interactions so that they use the upgraded
interface provided by `indexed_db_futures`
- Add functionality for running `wasm-pack` tests against Chrome
---
Closes#5420.
---
- [ ] Public API changes documented in changelogs (optional)
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
---------
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
This patch replaces `user_can_kick` by `user_can_kick`: it performs an
extra check to make sure the acting user has at least the same power
level as the target user.
The former has been merge in the latter, and it errors when generating
the docs in a recent version of nightly, like the one used on docs.rs.
This also requires to bump the version of nightly used in CI, otherwise
it would break the docs generation.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch splits `power_levels: &Option<(&UserId, RoomPowerLevels)>`
into 2 variables: `own_user_id: Option<&UserId>` and `power_levels:
Option<&RoomPowerLevels>`. The idea is to be able to get the
`own_user_id` even if the power levels are `None`.
This helps make sure the rooms to be left were actually part of the space graph as they are stored inside the `LeaveRoomHandle` and filtered from there. On the FFI layer on the other hand, we still take plain strings as working around the limitations would've significantly complicated things.
When leaving a space the user should be informed of which rooms are DMs (already part of the `SpaceRoom`)
and in which they might be the last admin, where leaving would prevent anybody else for taking control.
It doesn't make sense to send a formatted caption without a plain text
caption so using TextMessageEventContent forces the latter to be present.
This also allows to use the helpful constructors of
TextMessageEventContent.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Part of the fix to
https://github.com/element-hq/element-x-android/issues/5099
Allows applications to distinguish between errors that occur when
unlocking Secret Storage, or errors that occur when importing a secret,
so that they can display appropriate feedback (or not) to the user.
- [ ] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by:
---------
Signed-off-by: Hubert Chathi <hubertc@matrix.org>
I was wrong in a previous commit: both receipts are on the same event
anyways, so we can safely override the keys in the read receipts map
(overriding would mean both receipts point to the same event, which is
fine, as we're displaying only one of those).
This is a fix, because some other code elsewhere will use both kinds of
receipts whenever they're received over sync. The code that's modified
in this patch is called for the initial load of receipts, that happens
whenever we see a new event. Since the two code paths were not doing the
same thing, this would affect the displayed receipts, depending on
whether we received them during sync, or after loading the timeline for
the first time.
This case is very specific to the room pagination, and will not apply to
the thread pagination; by removing it from the generic pagination logic,
we'll be able to use the generic pagination logic for threads.
… when known to the client and forward updates through the existing
`subscribe_to_space_updates` mechanisms.
This allows clients to listen to updates without having to resort to a
separate room info subscription on their side.
The raw event was being deserialized in a wrong way and it could have a missing room id in some cases, returning an error even when the event was found
For thread permalinks, we start with a /context query that will load the
focused event, and maybe a few other in-thread events. In fact, it can
also include the thread root event, which was excluded before. Instead,
we would get a previous-token for back-paginations, which would be used
in /relations. When the request to /relations returns an empty previous
token, that means we've reached the start of the thread, and in this
case we would manually load the root with /event.
We can do better, if the root event is part of the initial /context
response: skip the back-paginations altogether, and make sure to include
the root event in `init_focus()`.
It's much faster now as it downloads the latest version of the
precompiled binary from Github, and it will use the latest tagged
version of cargo-machete by default.
This patch revisits a feature we have disabled a couple of days ago:
the `recency` sorter was initially only supporting the recency stamp,
then later the recency stamp _and_ the latest event's timestamp. It was
however buggy and we had to revert it. Now it's time to re-introduce it
but with a different approach.
The previous rules were:
1. if two rooms have a latest event, use their latest event's timestamps
as their _scores,
2. if one of room has a latest event, use the recency stamp as their
_scores_ for both rooms.
Rule 2 was buggy because one room was sometimes using its latest
event's timestamp, and sometimes its recency stamp, based on what it was
compared to. It was an error!
The new rules are the following:
1. unchanged
2. if one room has a latest event, use its latest event's timestamp as
its _score_, and use no _score_ for the other room,
3. if two rooms have NO latest event, use the recency stamp as their
_scores_.
It means that a room with no latest event will always be sorted _after_
a room with a latest event. It can feel cruel, but it should be an edge
case. When a room is synchronised, it should receive events, which
should trigger the computation of a latest event.
Note that this patch also renames _rank_ to _score_, as I consider it's
a better vocabulary. It could be confusing to use _rank_ as one can
expect all rooms to be indexed and get a rank, but it's not the case.
_Score_ sounds better.
This allows custom message-like events (created by the `EventContent` macro from ruma) to be added to the timeline if they are explicitly allowed when building the timeline with a custom `event_filter`.
The custom event content is not available directly to the consumer, but it can still fetch it from the matrix-sdk client with its `event_id`, or display a "this type of event is not supported".
Signed-off-by: Itess <me@aloiseau.com>
Fixes#5598.
This patch tries to solve a problem raised by Complement Crypto. In
`compute_latest_events`, in two places, `RegisteredRooms::rooms` was
locked with an exclusive write access. Then, during the updates of
the latest events (via `RoomLatestEvents::update_with_event_cache` and
`RoomLatestEvents::update_with_send_queue`), the state store lock is
acquired to update the state store. At the same time, in the sync, the
state store lock can **already** be taken to store the new updates, and
the function `subscribe_to_room_latest_events` is called, which waits
on the lock around `RegisteredRooms::rooms` to be available. We have a
dead lock:
- `compute_latest_events` waits on the state store lock while the lock
on `RegisteredRooms::rooms` is taken with an exclusive access,
- the sync has acquired the state store lock while it waits on the lock
on `RegisteredRooms::rooms`
This patch introduces a lock inside `RoomLatestEvents`. A new
`RoomLatestEventsState` type is introduced to be the lockable value. A
new `RoomLatestEvents::read()` and `RoomLatestEvents::write()` methods
are introduced to respectively return a `RoomLatestEventsReadGuard` and
a `RoomLatestEventsWriteGuard` type. The idea is to abstract a bit the
owned lock guard and to distribute the methods that were previously on
`RoomLatestEvents` in the new guard types, so that we keep the `&self`
and `&mut self` semantics, plus we take a lock for multiple operations.
The deadlock is fixed because of all the following reasons combined:
- only `RegisteredRooms::room_latest_event`,
`RegisteredRooms::forget_room` and `RegisteredRooms::forget_thread`
take a write lock over `RegisteredRooms::rooms`,
- `compute_latest_events` no longer takes a write lock over
`RegisteredRoooms::rooms`, but it also has a short-lived lock:
its read lock is dropped as soon as the **owned** write lock over
`RoomLatestEvents` is taken.
Now, `subscribe_to_room_latest_events` can acquire a write lock over
`RegisteredRooms::rooms` while `compute_latest_events` is doing the
updates. If the state store lock is acquired here, it will just wait
its availability: the sync flow can finish and release the lock without
being blocked by `subscribe_to_room_latest_events`.
This patch updates `SlidingSyncResponseProcessor::handle_room_response`
to automatically call `LatestEvents::listen_to_room` based on
`http::Response`'s `rooms`.
Why? Because when a sync is received, we want its `LatestEventValue`
to be computed, so that it can trigger a `RoomInfoNotableUpdate`,
which will update the `RoomList`, which will re-sort the
rooms. So far, `LatestEvents::listen_to_room` was called by
`RoomListService::subscribe_to_rooms`, but it's possible to receive a
room update via the sync for a room that is not subscribed, i.e. out
of the viewport of the room list in Matrix clients (it's recommended
to subscribe to rooms that “enter” the viewport). Without this
patch, rooms are receiving updates but the room list is not entirely
refreshed/recalculated.
This way we can retrieve random events in a room and check their properties - this is needed to decide whether a permalink for an event should open in a thread or not
This way we can have the same focus handling both the focused event pagination in the main timeline with the `Paginator` and the focused event pagination in a thread with `EventThreadsLoader`.
The actual paginator is populated in `TimelineController::init_focus` after we call `/context` and can check if the event is part of a thread.
For some reason, running `cargo xtask ci clippy` locally would now fail,
complaining that the recent emoji functions didn't exist, in the FFI
layer. I suspect it's because some of the uniffi derive macro to export
functions incorrectly propagates the `cfg` guards; so a solution is to
move all this code under a new mod, that's enabled if and only if the
feature's enabled.
This patch fixes an issue where the `recency` sorter is based on either
the latest event's timestamp, or the room recency stamp. This cannot
work with a sort algorithm as the position of a particular room can
be different based on what it is compared to (i.e. if the rooms have a
latest event value or not).
This patch updates the `recency` sorter to only use the recency stamp
for now, as the latest event is not yet computed for all rooms.
Some background knowledge: the room notification modes are functions of
the push rules events, and they will only change when the push rules
event changes. As a result, there are only two cases where we need to
recompute them:
- when the push rules event changed, we need to recompute all the room
notification modes, in case one has changed;
- when we run into a new room, we need to compute an initial value for
its room notification modes.
Based on these observations, this improves the code to avoid recomputing
the room notification mode on every single sync response. Instead,
they're computed if and only if the push rules event has changed, or for
new rooms only.
Also, this avoids reconstructing one `NotificationSettings` object per
room, since this would load from the database each time. Instead, a
single object is created (at most), and its `Rules` object is directly
accessed, to avoid repeatedly taking the lock on its internal `rules`
field.
This makes it so that the time spent under this method from tens of
milliseconds to less than 1 millisecond, in testing. See the pull
request initial post for more numbers.
This patch improves throughput by +710% in `room_list_service` sorters
and filters. It introduces a new `room_list_service::Room` type that
derefs to `matrix_sdk::Room`. However, it **caches** some data from
`matrix_sdk::Room`. Why doing so? Because filters, but more specifically
sorters!, are calling methods on `matrix_sdk::Room`, so likely on
`matrix_sdk::RoomInfo`, quite intensively. `RoomInfo` is behind a
`SharedObservable`, which means it's behind a lock. Each time a sorter
sorts 2 rooms, the lock on `RoomInfo` can be called twice or more.
By caching the data, `RoomInfo` is reached once per refresh data, but
not during the filtering nor the sorting. It greatly reduces contention
on the `RoomInfo` lock, which improves the throughput by +710%, and the
time by -87%.
The cached data are refreshed in `merge_stream_and_receiver` when
(i) the stream of `Room` is updated, or when (ii) the stream of
`RoomInfoNotableUpdate` is updated. It's a central place it happens,
which isolates the behaviour.
This patch adds 2 methods on `RoomInfo`: `new_latest_event_timestamp`
and `new_latest_event_is_local`, which respectively returns
`LatestEventValue::timestamp` and `LatestEventValue::is_local`. The goal
is to avoid cloning a `LatestEventValue` when it's useless. For example,
in the room list sorters!
This patch also updates the room list sorters `recency` and
`latest_event` to use these new methods. It improves the speed and
throughput by 18%.
These are meant to be replaced with specific endpoints for each case.
Added the `global_account_data_mock_builder` helper function to make building these mock endpoints a bit easier.
By making it explicit that the async closures return a unit type, we can
get rid of these two allow lines, and prepare for migrating this code to
the 2024 edition.
This patch updates the `recency` room list sorter to no longer
use the `LatestEventValue::timestamp` method. It keeps using the
`Room::recency_stamp` for the moment, as it was the case before. This
patch is a test to try finding the problem in some Matrix clients where
the room list becomes unusable. We suspect it's because of this patch
sorter.
This patch defines a new log target,
`MatrixSdkCommonDeserializedResponses`. It is enabled by the
`SyncProfiling`, `EventCache` or `Timeline` log packs.
This patch also changes the level of the log in
`TimelineEvent::timestamp` from `trace` to `warn`.
This will pave the way for permalink targets which are for events in a
thread, by making it possible to add a future condition in the
`TimelineFocusKind::Event` case (if the pagination used under the hood
is using /relations, then it's a thread).
This patch adds a log in `TimelineEvent::timestamp` when the `timestamp`
has to be extracted. It can be a performance problem depending on when
it's called.
After the merge of
https://github.com/matrix-org/matrix-rust-sdk/pull/5648, we want
all events to get a `TimelineEvent::timestamp` value (extracted from
`origin_server_ts`).
To accomplish that, we are emptying the event cache. New synced events
will be built correctly, with a valid `TimelineEvent::timestamp`,
allowing a clear, stable situation.
It turns out that creating a cross-signing identity *without* the upload
requests is a very common thing to do, especially in tests. We can simplify
some code by factoring it out as a new helper.
This patch adds the `timestamp` field to `TimelineEvent`.
It's a copy of the `origin_server_ts` value, parsed as an
`Option<MilliSecondsSinceUnixEpoch>`. It's `None` if the parsing failed,
or if the `TimelineEvent` was deserialised from a version before this
new field was added.
A new `extract_timestamp` function is added for this purpose. It
protects against malicious `origin_server_ts` where the value can be
set to year 2100 for example. The only protection we are adding here is
to take the `min(origin_server_ts, now())`, so that the event can never
been “in the future”.
It doesn't protect against a malicious value like 0. It's non-trivial to
define a minimum timestamp for an event.
When a `TimelineEvent` is mapped from one kind to another kind, the
`timestamp` is carried over. To achieve that, new `to_decrypted` and
`to_utd` methods are added.
The rest of the code is updated accordingly.
This patch renames the `LockStoreError` enum to `CrossProcessLockError`
to be consistent with the other types in the same module.
The `BackingStoreError` variant is also renamed to `TryLockError`.
This patch renames the `BackingStore` trait to `TryLock`. It also
renames the `CrossProcessLock::store` field to `locker`. It's not
necessarily a store, it can be anything.
This PR is a start to the process of splitting the media store from the
event cache store. #5410
It contains:
* Split `MediaStore` trait from `EventCacheStore`.
* Rename `EventCacheStoreMedia` to `MediaStoreInner`.
* Move relevant tests into `MediaStoreIntegrationTests`.
This will be done over 3 PR's (reviewing 1, 2, 3 then merging 3 into 2
into 1).
A reminder comment for my own sanity:
This PR will not pass tests until after merging.
Current state of this PR:
- [x] Step 1 reviewed #5568
- [x] Step 2 reviewed #5569
- [x] Step 3 reviewed #5571
- [x] Step 3 merged into Step 2
- [x] Step 2 merged into Step 1
- [ ] Add changes to changelog.
- [ ] Ready to merge 🎉
Note, may also want to:
* Re-organize file structure
* Split/refactor benchmarks namely `benchmarks/benches/event_cache.rs`
<!-- description of the changes in this PR -->
- [ ] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by: Shrey Patel shreyp@element.io
---------
Co-authored-by: Shrey Patel <shreyp@element.io>
This PR is part of an onging effort to move responsiblity to the EC app
and out of the EX apps.
4 intends (f.ex `join_existing` `start_new_dm`... ) (as url paramters)
are introduced in recent element call versions. Those intends behave
like defaults. If an intend is set a set of url parameters are
predefined.
Not all params can be covered by the intend (for insteance the
`widget_id` or the `host_url`).
This PR splits the url parameters into configuration (things that can be
configured by the intent) and properties (things that still need to be
passed one by one)
The goal with this change is that EX only needs to configre the intent
once and the EC codebase can update the behavior in those 4 specific
scenarios in case new features come along (auto hangup when other
participants leave, send call ring notification...)
Signed-off-by: Timo K <toger5@hotmail.de>
<!-- description of the changes in this PR -->
- [ ] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by:
---------
Signed-off-by: Timo K <toger5@hotmail.de>
This patch adds the notion of _rank_ in the `recency` sorter to avoid
confusion around `u64`: is it a timestamp or a recency stamp? It's
purely semantics, but I hope it clarify the code.
This patch renames all the structure `*Matcher` to `*Sorter`.
And their `matches` method become `cmp`. It was copy-pasted from
`room_list_service::filters` probably, but the semantics here are not
_matcher_ but _sorter_. It's more consistent that `cmp` returns an
`Ordering`.
This patch updates the `recency` sorter of the room list to rely on the
`LatestEventValue`'s timestamp, or on the `bump_stamp` returned by the
sync. Using the `LatestEventValue`'s timestamp is more reliable as we
don't rely on the server. However, we must be careful to compare values
of the same nature because the timetamp from the `LatestEventValue` and
the `bump_stamp` doesn't represent the same thing! The `bump_stamp` is
only used when the value for the `LatestEventValue` is `None`.
It's a compromise to get a more accurate listing. Though,
`LatestEventValue::timestamp` returns the `origin_server_ts` value,
which can be forged by a malicious user (then a room could be _sticked_
at the top or at the bottom of the room list). Note that this problem
already existed in the past before the server computed a `bump_stamp`.
Also note that some homeservers use the `origin_server_ts` as the
`bump_stamp` value. Anyway, it's not a security risk as far as I know.
This patch implements the new `latest_event` sorter for the room list
which puts the local latest events before the other kinds (like `Remote`
or `None`).
This patch updates the signature of `TerminationReport`'s
constructors so that it's impossible to create invalid states, like
an `origin` of `TerminationOrigin::RoomList` with an error of type
`encryption_sync_service::Error`. The constructors force the error to
match the origin.
This patch moves `SyncTaskSupervisor::check_if_expired` to
`TerminationReport::has_expired`. Because `TerminationReport` now holds
the error, we can remove the `has_expired` field and get a `has_expired`
method!
This patch changes the `TerminationReport::is_error` field to become
`error: Option<Error>`. This patch also creates new constructor on
`TerminationReport` to simplify the code.
By slightly changing the shape of the function used to process read
receipts, we can make it so that it's trivial to run concurrently, which
gives some nice speedups locally.
Distribution of 6 worst case processing time, initial response:
Before:
0.172524963s
0.216173016s
0.252289760s
0.257619156s
0.275838632s
0.280295891s
After:
0.083094692s
0.117074046s
0.130246646s
0.132577343s
0.138685246s
0.170287945s
This ensures that we're using the same color despite what your color
scheme of your terminal is. Some color schemes might produce unreadable
combinations of foreground color and background color.
Specially important, the one from `ClientBuilder::build` as it avoids a situation where the builder would infinitely try to get a response for this request and never create the `Client` or fail.
This patch zips the `unable_to_decrypt_info` and the `meta` arguments
into a single one. They are strictly related, `meta` has no sense if
`unable_to_decrypt_info` has no sense neither.
The necessary patch was merged[1] now, thanks to Jonas. We still need
the patch section since we don't have a release with the patch.
[1]: https://github.com/tokio-rs/tracing/pull/3000
Our paranoid-android for was configured to use our fork of the tracing
crates.
But paranoid-android will already use our tracing fork since it's
defined in the patch section. The patch section will override the
dependencies for all of our dependencies as well.
This can be seen in the dependency tree using:
$ cargo tree -p matrix-sdk-ffi --target=aarch64-linux-android
This will be used inside the WASM SDK to introduce a similar field to
its EncryptionSettings struct.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This patch updates `LatestEvent::update` to call the new
`LatestEvent::store` method, which will store the new `LatestEventValue`
in the `RoomInfo` struct, and will persist it in the `StateStore`.
This patch also adds the test for this new feature.
This patch adds the new `new_latest_event: LatestEventValue` field in
`RoomInfo`. The `latest_event` is kept for the moment, but it will be
removed once the new API has landed entirely.
This patch moves `LatestEventValue` into the
`matrix_sdk_base::latest_event` module. If we want to store this value
in `RoomInfo`, it must be in this crate.
Because it's not allowed to `impl T` where `T` lives in a different
crate, this patch changes the `impl LatestEventValue` to `struct
LatestEventValueBuilder` + `impl LatestEventValueBuilder`. Luckily, all
methods on `LatestEventValue` are only constructors, so the change is
super straightforward.
This patch replaces `OwnedRoomId` by `WeakRoom` in `LatestEvent`. Apart
from simplifying a couple of method' signatures, it also opens the road
for storing the `LatestEventValue` in `RoomInfo`.
Now that (since https://github.com/matrix-org/matrix-js-sdk/pull/4918) Element
Web uses separate subscribers for each OlmMachine, rather than a single global
one with a separate LevelFilter, EW's logs are very verbose because they
receive all the TRACE logs.
Dropping the TRACE logs on the floor isn't my favourite solution, but
everything else seems to be a bit harder than I have time for right
now:
* Sending TRACE logs up to the JS side in case it wants to print them would (a)
increase overhead and (b) be an annoying breaking change in JsLogger
* Ideally we'd let the application configure its logging more precisely via an
`EnvFilter` or something, but I'm not really sure what the API would look like
for that.
So for now, we take the easy path.
Since the server will reject any duplicate one-time keys forever,
clients which encounter such an error will spam sentry with such
reports.
This patch ensures that we only send the sentry report once.
This patch revisits the `assert_latest_event_content` macro to not take
a `true` or `false` value. It feels a bit weird to read. Instead, `with
|factory| { … }, true` becomes `event |factory| { … } is a candidate`.
Same for the `false`case which becomes `is not a candidate`. No more
comma, it feels a bit more like a sentence.
This patch replaces `LocalLatestEventValue::content` and `…::event_type`
fields by using the existing `SerializableEventContent`. It does exactly
the same thing.
The problem is: `LatestEventContent` cannot be serialized. It's annoying
because it means we can't store a `LatestEventValue` (that wraps a
`LatestEventContent`) in the database.
This patch revisits `LatestEventValue`. Before we got:
```rust
pub enum LatestEventValue {
None,
Remote(LatestEventContent),
LocalIsSending(LatestEventContent),
LocalCannotBeSent(LatestEventContent),
}
pub enum LatestEventContent {
RoomMessage(RoomMessageEventContent),
Sticker(StickerEventContent),
Poll(UnstablePollStartEventContent),
CallInvite(CallInviteEventContent),
CallNotify(CallNotifyEventContent),
KnockedStateEvent(RoomMemberEventContent),
Redacted(AnySyncMessageLikeEvent),
}
```
`LatestEventContent::Redacted` contains an `AnySyncMessageLikeEvent`.
That's the part that is not serializable.
It appears that `LatestEventContent` isn't necessary! The only thing we
need is to _filter_ the events by their type, no need to _find and
map_. The `LatestEventValue` can contain the entry event directly (e.g.
a `TimelineEvent` for the event cache). Okay, let's do that.
```rust
pub enum LatestEventValue {
None,
Remote(RemoteLatestEventValue),
LocalIsSending(???),
LocalCannotBeSent(???),
}
type RemoteLatestEventValue = TimelineEvent;
```
What about the `Local*` variants? We can't use a `TimelineEvent`. We
need a new type for that:
```rust
pub enum LatestEventValue {
None,
Remote(RemoteLatestEventValue),
LocalIsSending(LocalLatestEventValue),
LocalCannotBeSent(LocalLatestEventValue),
}
pub struct LocalLatestEventValue {
pub timestamp: MilliSecondsSinceUnixEpoch,
pub content: Raw<AnyMessageLikeEventContent>,
pub event_type: String,
}
```
We don't need the event ID nor the transaction ID in
`LocalLatestEventValue`.
That's the only change. All the other changes are about the tests.
This will resolve a number of transitive dependency issues when testing
crates that do not enable the `experimental-encrypted-state-events` feature
flag by default.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
Implements support for decryption of state events
- [ ] Introduce a case for `AnySyncStateEvent::RoomEncrypted` to the
`state_events` sync response processor.
- [ ] Introduce modified `Room::decrypt_event` and
`::try_decrypt_room_event`.
- [ ] Introduce testing macro
`assert_let_decrypted_state_event_content`.
- [ ] Add casts and explicit type hints where necessary.
---------
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
Deduplicates common code that would be shared between
SendRawMessageLikeEvent and SendRawStateEvent to a helper method,
`ensure_room_encryption_ready`.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This fixes a small issue encountered while developing the integration test for encrypted state events. If the first sync response received from the server after enabling encryption does not contain the m.room.encryption event, the client will report the encryption state as unknown, even if the next sync response does contain the event.
- [x] Modify Room::enable_encryption_inner to spin on the room encryption state, rather than bailing after the first sync response is received.
Signed-off-by: Skye Elliot
Remove previous code that got updates in `RoomEventCacheState::post_process_new_events`.
Add new task in `EventCache` that subscribes to `linked_chunk_update_sender` and forwards new events to `client::search::SearchIndex` same as before.
Signed-off-by: Shrey Patel shreyp@element.io
The `ruma-common` crate is reexported by `ruma`, which is in our
dependency tree anyways. This change makes it so that qrcode, the only
crate that was making use of `ruma-common`, now depends on `ruma`. It
may move it further down the line in the compilation pipeline, but this
is unlikely to affect compile times in a crazy way. The benefit is that
this avoids the burden of having to specify the ruma commit hash twice
in the top-level Cargo.toml, as well as replacing it twice when
overriding it.
Instead of creating a fresh client every single time, this creates a
single client with a single event cache store, that's cleared between
runs of the benchmark.
So the tradeoff is:
- we don't have to create a new client anymore, which means no more
async setup code, which means we can avoid using `iter_custom`; a
benefit of this is that this can be benchmarked on CI now.
- but we're also measuring the time it takes to clear the database,
which isn't trivial in itself.
The task would stop when the receiver is closed; for the receiver to be
closed, the sender ought to be closed too.
Because the sender lives in `EventCacheInner`, and the task would hold
onto an `EventCacheInner` struct, the sender would never be dropped, so
the full `EventCacheInner` would leak, as a result.
Changes `Room::enable_encryption_with_state` to
`Room::enable_encryption_with_encrypted_state_events`, and updates the
respective unit test and testing utilities.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This patch moves the `test_update_ignores_none_value`
test in a new (correct) test module, and creates the new
`test_local_has_priority_over_remote` test.
This patch changes the order in which the `LatestEventValue` is removed
from the buffer and re-created in case of a `SentEvent`. Previously,
the value from the buffer was removed before re-creating one, now it's
the opposite.
Why this order? Because once the local event is sent, it's not yet
received by the sync and consequently not stored in the event cache. So
once the local event is sent, it won't show up as a `LatestEventValue`
as it will immediately be replaced by an event from the event cache. By
computing the new value before removing it from the buffer, we ensure
the `LatestEventValue` represents the just sent local event.
See the comment in the code to learn more.
This patch renames `LocalIsWedged` to `LocalCannotBeSent` to avoid
confusion with the wedged/unwedged state of the `send_queue`. The
semantics is different in `latest_events`.
This patch handles the case where a local event is replaced by another
local event which isn't suitable for being a latest event value. In this
case, the previous existing latest event value should be removed from
the buffer.
This patch reduces the level of logs when channels have been closed in
`LatestEvents`' tasks from `error` to `warn`. Indeed, when the `Client`
shutdowns, the channels will be closed, but it's not an error at all.
This patch implements `LatestEvent::update_with_send_queue`.
It introduces an intermediate type, for the sake of clarity,
`LatestEventValuesForLocalEvents`.
The difficulty here is to keep a buffer of `LatestEventValue`s requested
by the `SendQueue`. Why? Because we want the latest event value, but we
only receive `RoomSendQueueUpdate`s, we can't iterate over local events
in the `SendQueue` like we do for the `EventCache` to re-compute the
latest event if a local event has been cancelled or updated.
A particular care must also be applied when a local event is wedged:
this local event and all its followings must be marked as wedged too,
so that the `LatestEventValue` is `LocalIsWedged`. Same when the local
event is unwedged.
This patch splits the `LatestEventValue` type into `LatestEventValue`
+ `LatestEventKind`. Basically, all variants in `LatestEventValue` are
moved inside the new `LatestEventKind` enum. `LatestEventValue` keeps
`None`, and see the new `Remote`, `LocalIsSending` and `LocalIsWedged`
variants.
This patch also extracts the message-like handling of `find_and_map`
(now renamed `find_and_map_timeline_event`) into its own function:
`find_and_map_any_message_like_event_content`. This is going to be
handful for the send queue part.
This patch updates `compute_latest_events` to broadcast a
`RoomSendQueueUpdate` onto `LatestEvent`. It introduces the new
`update_with_send_queue` method.
This patch renames the `update` methods to `update_with_event_cache`
in the `latest_events` module. It frees the road to introduce
`update_with_send_queue`.
This patch updates `LatestEvents` to listen to the updates from the
`SendQueue`. The `listen_to_event_cache_and_send_queue_updates` function
contains the important change. A new `LatestEventQueueUpdate` enum is
added to represent either an update from the event cache, or from the
send queue.
So far, `compute_latest_events` does nothing in particular, apart from
panicking with a `todo!()` when a send queue update is met.
Allows `Backups::maybe_download_room_key` to accept any T:
JsonCastable<EncryptedEvent>, which will be required for state events to
trigger fetching the room key.
Implements JsonCastable<EncryptedEvent> for
OriginalSyncStateRoomEncryptedEventContent.
Implements JsonCastable<AnyStateEvent> for RoomEncryptedEventContent.
Modifies `OlmMachine::decrypt_room_event_inner` to call a new method
`OlmMachine::verify_packed_state_key` which, if the event is a state
event, verifies that the original event's state key, when unpacked,
matches the state key and event type in the decrypted event content.
Introduces MegolmError::StateKeyVerificationFailed and
UnableToDecryptReason::StateKeyVerificationFailed which are thrown when
the verification fails.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
This commit also refactors out what would be common code between
::encrypt and ::encrypt_state to a helper ::encrypt_inner.
Signed-off-by: Skye Elliot <actuallyori@gmail.com>
Integrate matrix-sdk-search into matrix-sdk.
When a room is joined, a corresponding index is created.
When a message is received via sync or via a back-pagination, it is
added to the corresponding room's index.
Signed-off-by: Shrey Patel shreyp@element.io
This PR makes some non-domain-specific changes across multiple crates
that are required for proper testing of features implemented for #5397.
* Adds a `experimental-encrypted-state-events` feature flag across the
SDK.
* Introduces a feature set into xtask to ensure feature-gated tests are
run during CI.
* Minor fix to a test that would otherwise fail with the newly
introduced CI.
The semantics of this variant are unclear: sometimes a timeline could be
cleared, which would result in a `UpdateTimeline` (and if we looked at
the vector diffs, it would include a `Clear`), but in this case the
`Clear` variant would not be emitted.
It was only emitted in a few adhoc spots, but it was missing the whole
picture. Also, the current observer was only interested in getting *a*
room update with the room id, and didn't react particularly to clears.
So, there's apparently little reason in having this variant, and as a
result we should get rid of it.
Introduces a new `SyncToken` enum for the `SyncSettings::token` field.
The enum has 3 variants: ReusePrevious (default), NoToken, Specific(String).
Some tests were changed to use the old default (NoToken).
Handle the changes to the Response of the get_profile endpoint. The
content of the response is private and fields must be accessed with
methods.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Handle the previous breaking change that was reverted: `StrippedState`
was removed and `AnyStrippedStateEvent` is used again.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
- [x] Change `DecryptedRoomEvent::event` to `Raw<AnyTimelineEvent>`
- [x] Update usages to pattern match on `AnyTimelineEvent::MessageLike`
where necessary
---------
Signed-off-by: kaylendog <actuallyori@gmail.com>
This patch updates the Timeline Controller's `handle_explicit_read_receipts` method to also consider unthreaded read receipts on **threaded** **main** timelines when calculating timeline items states and adds a test for it.
This picks up from #5442 and fixes https://github.com/matrix-org/matrix-rust-sdk/issues/5440
Add a new `server_vendor_info` method on the `matrix-sdk` `Client` that calls the `/_matrix/federation/v1/version` endpoint to retrieve the server's software name and version information.
Also add it to the bindings + log it when initializing the logs.
This can happen because of an OS-level error, so let's try to handle it
in a slightly cleaner way. At the moment, it will bubble up and only be
logged, but we might try to find a better way to handle this at the
top-level.
The `timeout` setting on the `/sync` endpoint is the maximum allowed
time for the server to send its response, because this is a poll-based
API. It means that if there is no new data to show, the server will wait
until the end of `timeout` before returning a response.
It can be an undesirable behavior when starting a client and informing
the user that we are "catching up" while waiting for the first response.
By not setting a `timeout` on the first request to `/sync`, the
homeserver should reply immediately, whether the response is empty or
not.
---------
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Signed-off-by: Ivan Enderlin <ivan@mnt.io>
Co-authored-by: Ivan Enderlin <ivan@mnt.io>
It is supposed to be an improvement over `state`, since it allows the
server to send updates to the state that might not be reflected in the
timeline.
This is also the same behavior as in Simplified Sliding Sync.
This is MSC4222 that was accepted and is about to get merged in the
spec.
---------
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This brings 2 important bug fixes:
- Make deprecated fields of `m.room.encrypted` optional: it seems that there are events without these fields in the wild.
- Fix deserialization of `RedactedRoomJoinRulesEventContent`. This was found by a bug report in Fractal that caused the same error as #3557 when restoring the client. So maybe we could consider that this bug is fixed? It is still possible that there is another deserialization error.
There is also a breaking change in the format of the `state` field in response to `GET /v3/sync`.
This is a breaking change as spaces are now requested through sliding sync and they need to manually be excluded from the room list by using the newly introduced non-space filter.
We use `cargo ndk -p {package_name}`, where `-p` is short for
`--package`. However, `cargo ndk` has introduced the `-p` option (see
https://github.com/bbqsrc/cargo-ndk/commit/c6b93a89a2723ff0fac99b5ac86ae6636a6cf54a),
short for `--platform`. It creates a confusion and the command line
doesn't execute properly. Let's use the long option `--package` to
clarify everything.
The `m.room.create` event contains at lot of important information for a
room, like its type (i.e. whether it is a space), its version and its
creator(s) (which are important in room version 12). So ignoring the
event completely might break a room.
Instead what we do here is simply ignore the `predecessor` field if it
is considered invalid, allowing us to access the other fields.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Fixes the encrypting part of
https://github.com/matrix-org/matrix-rust-sdk/issues/4147
Probably easiest to review commit-by-commit
<!-- description of the changes in this PR -->
- [x] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by:
This brings in a new breaking change from Ruma, because not all events
are stripped in a room's stripped state. For simplicity, this still
considers the events as stripped during deserialization for now, since
this format is compatible with the other possible formats.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
These tests now fully mock all the end-to-end encryption server
endpoints to test the redecryption and UTD item replacement logic of the
timeline without any manual room key insertions.
We test that the item replacement correctly handles supported event
types as well as unsupported ones.
Instead of having one static method duplicating an underlying
`ClientBuilder` method, we can pass the builder directly to a closure,
that will replace it. Call sites are a bit more verbose, but that would
avoid having to add duplicate `MockClientBuilder` methods for each
`ClientBuilder` method.
As noted in the changelog entry, the event type they send is outdated, and Client is not actually supposed to be able to join MatrixRTC sessions at this time. A MatrixRTC client implementation which actually participates in sessions should be able to send these notifications without the SDK's help.
For clients which integrate Element Call: Currently matrix-rust-sdk is responsible for sending the call notification event when joining MatrixRTC sessions, but this is planned to be changed soon. As of the upcoming Element Call 0.14.0 release, it will request the capability to send call notifications itself, and we should auto-approve this capability.
Client::get_url is there for SDK consumers to be able to use the
existing HTTP stack (configuration and all) however in practice it was a
bit weird:
- Responses came back as strings limiting the types of resource that
could be fetched (as well as requiring the string to be converted back
to data before handed to a JSON decoder).
- HTTP errors weren't being raised and instead you would find the (e.g.
404) error response in the Ok case.
This patch fixes both of these issues.
This patch updates our `Raw` API usage since the newly added `JsonCastable` that disallows Raw casts that are known to fail deserialization.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch changes the query used by
`SqliteEventCacheStore::load_all_chunks_metadata`. It was the cause of
severe slowness. The new query improves the throughput by +1140% and the
time by -91.916%. The benchmark will follow in the next patch.
Metrics for 10'000 events (with 1 gap every 80 events).
- Before:
- throughput: 20.686 Kelem/s,
- time: 483.43 ms.
- After:
- throughput: 253.52 Kelem/s,
- time: 39.478 ms.
This query will visit all chunks of a linked chunk with ID
`hashed_linked_chunk_id`. For each chunk, it collects its ID
(`ChunkIdentifier`), previous chunk, next chunk, and number of
events (`num_events`). If it's a gap, `num_events` is equal to 0,
otherwise it counts the number of events in `event_chunks` where
`event_chunks.chunk_id = linked_chunks.id`.
Why not using a `(LEFT) JOIN` + `COUNT`? Because for gaps, the entire
`event_chunks` will be traversed every time. It's extremely inefficient.
To speed that up, we could use an `INDEX` but it will consume more
storage space. Finally, traversing an `INDEX` boils down to traverse a
B-tree, which is O(log n), whilst this `CASE` approach is O(1). This
solution is nice trade-off and offers great performance.
It uses the statically-known type from the `StaticEventContent`
implementation to call `fetch_account_data()`.
This is the equivalent of `Account::account_data()` but for fetching
from the server. It avoids the need to cast to the proper type after.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Currently Element X can't distinguish the cases where a homeserver only
supports legacy SSO without OIDC (and password login isn't avaliable),
and other server unreachable scenarios.
This patch exposes legacy SSO support infomation so that Element X side
can give a dedicated error message when it encounters a homeserver that
can only support legacy SSO.
Signed-off-by: Yorusaka Miyabi <23130178+ShadowRZ@users.noreply.github.com>
Historic room key bundles are uploaded as an encrypted file to the media
repo and the key to decrypt the file is sent as a to-device message to
the recipient device.
In the nominal case, the invite and this to-device message should arrive
at the same time and accepting the invite would download and import the
bundle.
If the to-device message arrives after the invite has already been
accepted we would never download and import the bundle.
To mitigate this problem, this patch introduces a task that listens for
bundles that arrive. If the bundle is for a room that we have joined we
will consider importing the bundle.
It is going away in MSC4291, which should be accepted next week.
Removing it now makes sure that no one uses it.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
For custom integrations it might be necessary to allow the SDK to
request additional scopes for the OAuth2 authorization code flow.
Currently, only the MSC2967 client API and client device scopes are
requested statically.
Signed-off-by: fl0lli <github@fl0lli.de>
This patch removes the use of `OrderTracker` because the
implementation of `EventCacheStore::load_all_chunks_metadata` for
`SqliteEventCacheStore` is the cause of severe slownesses (up to 100s
for some account).
We are going to undo this patch once the problem has been solved.
Avoids to carry around the event content only to convert it when we
want to use it. Avoids also to carry around the room creator when we
might not need it.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch adds `timer!` logs in each method from `EventCacheStore` for
`SqliteEventCacheStore`. It will help to know the execution duration of
each of these methods.
This patch renames `TracingTiming::new_debug` to `new`. The
documentation claims it sets the log level to `debug` while the `level`
is actually an argument of the constructor. It's then wrong, and the
constructor must be renamed.
This changes the `TracingTimer` message to use the `Debug` impl of
`Duration` instead of displaying it as milliseconds. It can help spotting
seconds without counting all the digits.
Until now, `SqliteEventCacheStore` manages a pool of connections. A
connection is fetched from this pool and operations are executed on it,
regardless whether these are read operations or write operations.
We are seeing more and more _database is busy_ errors. We believe this
is because too many write operations are executed concurrently.
The solution to solve this is to use multiple connections for read
operations, and a single connection for write operations. That way,
concurrent writings are no longer a thing, and we hope it will reduce
the number of _database is busy_ errors to zero. That's our guess.
This patch does that. When the pool of connections is created, a
connection is elected as the `write_connection`. To get a connection for
read operations, one has to use the new `SqliteEventCacheStore::read`
method (it replaces the `acquire` method). To get a connection for
write operations, one has to use the new `SQliteEventCacheStore::write`
method. It returns a `OwnedMutexGuard` from an async `Mutex`. All
callers that want to do write operations on this store have to wait
their turn, this `Mutex` is fair, and the first to wait on the lock is
the first that will take the lock (FIFO). It guarantees the execution
ordering the code expects.
The rest of the patch updates all spots where `acquire` was used and
replaces them by `read()` or `write()`. A particular care was made to
see if other places are using `SqliteEventCacheStore::pool` directly. No
place remains except in `read()` and `write()`.
This patch updates `SqliteStoreConfig::pool_size` to be at least 2. We
need 2 connections: one for write operations, one for read operations.
This behaviour is coming in the next patches.
This patch adds more logs inside `EventCache` around the
`multiple_room_updates_lock` and around the `listen_task`, just to be
sure if everything is listened and works as expected.
The SQLite implementation for the
EventCache::find_event_with_relations() the relation type list isn't
inserted using SQL placeholders.
The relation types are inserted manually using a format!() call. The
usage of the format!() call can lead to SQL injection if a
RelationType::Custom variant is used which contains SQL expressions.
This patch modifies the, query logic which retrieves the related events,
to use two separate queries which use SQL placeholders to insert all
the dynamic variables.
Security-Impact: Moderate
CVE: CVE-2025-53549
GitHub-Advisory: GHSA-275g-g844-73jh
This patch adds the new `RoomListService::new_with_share_pos`
constructor. It decides whether the `share_pos` feature of sliding sync
should be enabled or not.
`SyncServiceBuilder` gains a new `with_share_pos` method to configure
the way the `RoomListService` is built.
The FFI bindings are updated accordingly.
This patch updates the `assert_update` macro in the send queue test
suites to test that `SendQueue::subscribe` provides the same data that
`RoomSendQueue::subscribe`. It ensures `SendQueueUpdate` won't miss
an update.
To get updates from the `SendQueue`, one needs to use
`RoomSendQueue::subscribe`, it emits `RoomSendQueueUpdate`s. However, if
one wants to receive updates for all rooms managed by the `SendQueue`,
instead of subscribing to all `RoomSendQueue` individually, this
patch introduces a new `SendQueue::subscribe` method, which emits
`SendQueueUpdate`s. It's basically a wrapper around
`RoomSendQueueUpdate` by adding an `OwnedRoomId`.
This patch renames variants of `RoomEventCacheGenericUpdate`:
- `TimelineUpdated` becomes `UpdateTimeline`,
- `Cleared` becomes `Clear`.
It matches the rest of the codebase where we use _verb_ + _subject_.
This reorders methods so that they're grouped in "dual" pairs
(encode/decode, serialize/deserialize). Also adds a doc comment to
`encode_key`, as I've wondered in the past what it did.
This patch updates `RoomListService::subscribe_to_rooms` to call
`LatestEvents::listen_to_room` automatically. This method becomes async,
which propagates to a couple of callers.
The idea is that when one is interested by a specific room, a
subscription will be applied. This is an opportunity to also “activate”
the computation of the `LatestEvent` for this specific room, so that the
user doesn't have to do that manually (except if room subscription is
never used).
This patch adds a new log field, named `request_duration` in
`HttpClient::send_request`. It helps to know how much time the request
took to be executed.
This is a bugfix for the `compat-lax-room-create-deser` and
`compat-lax-room-topic-deser` features, which didn't work with events
in the wild.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch removes an optimisation that was considering
`LatestEventValue::None` as a value that should no overwrite the
previous value if any. This is a mistake: if the event cache is cleared,
it won't be possible to calculate a new value, and the previous value
must be overwritten.
This patch adds new variants to `LatestEventValue` that represent
all the possible suitable latest events. It's heavily inspired by
`matrix_sdk_base::latest_event::PossibleLatestEvent`. It uses the
new `find_and_map` method to find a suitable `TimelineEvent` and to
map it into a `LatestEventValue`. This method is heavily inspired by
`matrix_sdk_base::latest_value::is_suitable_for_latest_event`.
To be able to provide the `power_levels` to `find_and_map`, a
`WeakClient` is required by `LatestEvents::new`. It flows up to a
`WeakRoom` in `RoomLatestEvents`, used to create or to update the
`LatestEventValue`.
A particular care is applied to re-compute the `power_levels` only when
necessary and once **per room** (and not per latest event value).
Fetching the `power_levels` requires an access to the storage, it's
not anodyne.
This is more precise than using the event timeline item kind: indeed,
when the event for which we want to add a reaction is a local echo that
has been sent, it is still marked as a local echo, but the send queue
will not know about it, and thus will not be able to add the reaction
immediately, leading to a silent failure.
The `TimelineItemHandle` was made to help with this kind of situation:
in this case, it will return `TimelineItemHandle::Remote`, even though
the item is a local echo that has been sent; that way we can use the
event id, and correctly send a (remote) reaciton event immediately.
The following commit includes a regression test.
Using this type will be mandatory in the next breaking release of Ruma,
that will gain support for recognizing unstable features. Besides, it
allows to cache the supported versions and features in a single
CachedValue, which makes more sense than separately because we always
get both at the same time in production.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
This patch fixes the markup in the `README.md` file. Links were broken because of the mix of HTML and Markdown. Plus, the HTML markup was kind of incorrect. Plus, too much `<br />`, most of them were “collapsed” automatically.
Signed-off-by: Ivan Enderlin <ivan@mnt.io>
Usually tests that don't construct it via MockMatrixServer don't care
about the homeserver URL, so this allows to provide a default URL for
them.
Also we don't force to allocate a string when the inner API actually
uses a borrowed string.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
It seems that some `m.room.topic` events in the wild have the wrong
format for the `m.topic` field. Since this field was stabilized recently
in Ruma, deserializing these events now fails. We added a feature to
ignore this field if its deserialization fails.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
The `EventContent` trait is gone in Ruma so this will ease the upgrade
to the next breaking release.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
With the UI crate now sending threaded read receipts we need to start considering them when computing unread counts. As a first step before the participation model, threaded messages will be ignored when computing room unread counts.
This patch moves away from always sending unthreaded read receipts to checking the timeline's focus and `hide_threaded_events` associated values to see whether `ReceiptThread::Main` or `ReceiptThread::Thread` should be used.
It doesn't make sense to name a thing "Matrix" when it comes from the
core SDK crate.
Also adjust a now slightly-incorrect doc comment for `get_notification`.
These are not included in Element's main privacy policy, and may
constitute PII (because the homeserver may include the name of some
user). We keep the information as separate log lines, so that
rageshakes/manual reports still include those.
The metadata loading shouldn't cause an error to be displayed, when the
linked chunk is empty; this can happen for new rooms we've never
visited.
Spotted while investigating some failures in a rageshake.
This patch introduces a new method named
`RoomEventCache::rfind_event_in_memory_by` to look for an in-memory
event, starting from the most recent event, by applying a predicate
closure on each event.
This patch renames the `RoomEventCache::event_with_relations` method to
`find_event_with_relations` for the sake of clarity, also to match the
other method names, and to match the Rust standard library namings.
The swift bindings aren't getting tested (they don't run) in CI anymore,
so building with the reldbg profile (that's a workaround to make it run
and not crash in production) doesn't provide more value than building in
debug mode, while taking much longer to build.
Let's use the default dev profile for this; we have to specify it
manually, because the default for the xtask command is to use the
`reldbg` profile otherwise.
This requires a fix for the dev profile, that consists in being able to
set the iOS deployment target, and set it to a high value in CI
settings. Production builds *don't* have to set it, though.
These two methods could be only made private now, since they're only
used in `finish_back_pagination` (and in tests). But they only call
inside other methods of the underlying linked chunk without any extra
value, so they can be inlined instead. This reduces the public API and
removes tests for other APIs that were tested some place else.
This patch fixes `EventCache::for_room` to return an error when the room
isn't found. Additionally, it also returns an error when the `Client`
(actually the `WeakClient`) is dropped.
These errors were “hidden” behind the room version logic because of
the fallback. The code has been rewritten in a way it makes it harder to
do it again.
This patch is the first part of the new `LatestEvents` API. It contains
the “framework”, i.e. the structure, tasks, logic to make it work, but
no latest events are computed for the moment.
The Latest Events API provides a lazy, reactive and efficient way to
compute the latest event for a room or a thread.
The latest event represents the last displayable and relevant event
a room or a thread has been received. It is usually displayed in a
_summary_, e.g. below the room title in a room list.
The entry point is `LatestEvents`. It is preferable to get a reference
to it from `Client::latest_events`, which already plugs everything to
build it. `LatestEvents` is using the `EventCache` and the `SendQueue`
to respectively get known remote events (i.e. synced from the server),
or local events (i.e. ones being sent).
\## Laziness
`LatestEvents` is lazy, it means that, despites `LatestEvents`
is listening to all `EventCache` or `SendQueue` updates, it will
only do something if one is expected to get the latest event for a
particular room or a particular thread. Concretely, it means that until
`LatestEvents::listen_to_room` is called for a particular room, no
latest event will ever be computed for that room (and similarly with
`LatestEvents::listen_to_thread`).
If one is no longer interested to get the latest event for a
particular room or thread, the `LatestEvents::forget_room` and
`LatestEvents::forget_thread` methods must be used.
\## Reactive
`LatestEvents` is designed to be reactive. Use
`LatestEvents::listen_and_subscribe_to_room` (same for thread) to get
a `Subscriber`.
This patch moves the `JoinHandleExt` trait and the
`AbortOnDrop` type from `matrix_sdk::sliding_sync::utils` into
`matrix_sdk_common::executor`. This is going to be useful for other
modules.
This patch updates `test_sync_indicator` since the `SyncIndicator`
is shown for the `SettingUp` state. Also, this patch makes this
test non-flaky entirely by changing the `sync::mpsc::channel` to a
`sync::Barrier`.
Since `RoomListService` uses a persistent `pos` for sliding sync, the
`SyncIndicator` no longer shows its face except if the sliding sync
session doesn't exist or has expired.
This patch changes that by extending the `Show` from `Init` to
`SettingUp`.
When we upgrade Ruma, the `EventContent` bound on `EventBuilder` will be
changed to `StaticEventContent`, which is not implemented by the
`Any*EventContent` enums.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
One must now specify the target linked chunk; both callers would filter
out items based on this after the fact, which is a bit overkill since
most items would thus be filtered out on the sinking end.
The one hardship is that lazy-loading updates must NOT affect the order
tracker, otherwise its internal state will be incorrect (disynchronized
from the store) and thus return incorrect values upon shrink/lazy-load.
In this specific case, some updates must be ignored, the same way we do
it for the store using `let _ = store_updates().take()` in a few places.
The author considered that a right place where to flush the pending
updates was at the same time we flushed the updates-as-vector-diffs,
since they would be observable at the same time.
This is a new data structure that will help figuring out a local,
absolute ordering for events in the current linked chunk. It's designed
to work even if the linked chunk is being lazily loaded, and it provides
a few high-level primitives that make it possible to work nicely with
the event cache.
In the next patch, we're going to introduce another user of
`UpdatesToVectorDiff` which doesn't require accumulating the
`VectorDiff` updates; so as to make it optional, let's generalize the
algorithm with a trait, that carries the same semantics.
No changes in functionality.
It doesn't belong there anymore, because it's not used within the event
cache itself. It's used in the timeline, and I think it's nice to keep
it available for external users as well, so it now lives in its own
`matrix-sdk/src/paginators` module. Next step is to move the thread
loader in there as well.
It avoids using different versions in several places for consistency. It
also allows to be able to change it in a single place when needed.
This also bumps the fallback to v11 everywhere, since it is the default
version for new rooms since Matrix 1.14 and it has the sanest redaction
rules.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
Process:
- set style_edition to 2024 in .rustfmt.toml
- run `cargo fmt`
- undo .rustfmt.toml change
- run `cargo fmt` again
- manually rewrap some strings
The `/auth_metadata` endpoint has been supported by Synapse for 6 months
now so there shouldn't be any homeserver that still depend exclusively
on it. This endpoint is also part of Matrix 1.15.
Support for this endpoint has been removed from Ruma so this is
necessary before an upgrade of the dependency.
Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
It turns out that downstream clients can and do call
`Client::join_room_by_id()` rather than `Room::join`, so we need to do
the room key history import in the lower-level method.
---------
Signed-off-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Benjamin Bouvier <benjamin@bouvier.cc>
The previous code would parse the entire bundled event, only to look at
its type indirectly. We can do better than this, and only look at the
type instead. This brings a few benefits:
- it is faster, as we don't have to deserialize the entire event
- while the spec seems to indicate that the latest thread event has a
`room_id`, it seems that, under some circumstances, it does not, as
indicated by some rageshakes. As such, not parsing as `AnyMessageLike`
(which mandates a room id) makes it more robust to the absence of a
room id in there, marking more events as latest events.
The latest mobile designs for Element Call have it displaying the room name in an "app bar". So the Element Call widget will soon start requesting the capability to learn the room name, and the SDK should auto-approve this capability.
This prevents us from having to add type constraints on the dynamic type
and instead only having to specify the type constraint at the call site.
Signed-off-by: Michael Goldenberg <m@mgoldenberg.net>
This can be used to invalidate the persisted state on disk related to
the sliding sync positions. It's useful to do so when clearing up all
the caches, since the sliding sync `pos`itions are stored in the crypto
store (to benefit from the cross-process lock).
The join rule state event can be missing from a room state. In this
case, it's an API footgun to return a default value; instead, we should
return none and let the caller decide what to do with missing
information.
`RoomEventCache::subscribe` is nice to subscribe to every update
happening inside a room in the event cache. However, the returned
`RoomEventCacheSubscriber` has side-effects when dropped (see
auto-shrink to save memory space). In some situation, this is pretty
annoying. for example, if one wants to listen to multiple room updates,
all the room event cache subscribers must be kept in memory, thus
breaking the side-effects. This isn't always the desired output. In
addition, listening to multiple channels/subscribers at the same is
quite complex, as it implies non-trivial async runtime efforts or
complex future types.
To solve this problem, this patch introduces a new
`EventCache::subscribe_to_room_generic_updates` method, which returns a
single `Receiver<RoomEventCacheGenericUpdate>`.
First off, it hides the details of `RoomEventCacheUpdate` (returned by
`RoomEventCacheSubscriber`), which might be desired, but particularly
lighter because events aren't part of the payload.
Second, one no longer needs to subscribe to all rooms. Only one channel
can be listened to get updates for all rooms. It reduces the complexity
on the caller side, plus `Receiver<RoomEventCacheGenericUpdate>` doesn't
have any side-effect.
This patch tests this feature in 4 situations:
1. when a room is created/loaded empty,
2. when a room is loaded and is not empty because data exists in the storage,
3. when a room receives data from the sync,
4. when a room receives data from the pagination.
<!-- description of the changes in this PR -->
Features to configure UserAgent, Proxy, Disabling SSL, and additional
certificates are not available on Wasm platforms. We remove these
configuration options from the FFI layer, while preserving them on
non-Wasm platforms.
- [ ] Public API changes documented in changelogs (optional)
<!-- Sign-off, if not part of the commits -->
<!-- See CONTRIBUTING.md if you don't know what this is -->
Signed-off-by: Daniel Salinas
---------
Co-authored-by: Daniel Salinas <danielsalinas@Daniels-MacBook-Pro-2.local>
Previously, we held a TrustRequirement inside Client, and had to wrap it
in a DecryptionSettings manually a couple of times.
It feels more right to hold on to a DecryptionSettings directly. If
there were ever some additional settings, most likely we would need them
all here.
If I haven't screwed it up, this should not affect behaviour in any way.
The pagination status was only mapped onto the "global" state, that is, the state of the event cache pagination.
Now, consider the following case, where two timeline instances are live:
- the first one could have backpaginated all the items back to the start of the room
- the second one is created later. Because of the initial value for the skip count, it will only return a subset of event items (~20).
However, listening to the pagination status for the second timeline would incorrectly state that the timeline was entirely paginated (because it returned the "global" pagination status). As such, an observer might think that there are no more items in the timeline, while a subsequent pagination would adjust the skip count and return more items.
This fixes it by combining the global pagination state with the local timeline state (aka the skip count value). If the skip count is positive (meaning, we could set it to 0 later and thus returning more events), we pretend we haven't reached the start of the timeline. This way, an observer can call pagination later, which may adjust the skip count and "return" more items.
`RoomPowerLevels` already holds an inner version of the power levels and has access to everything needed to compute the user to level map.
This way the async fetching only happens once and the mapping only on request.
Currently, clients have to make async requests to `Room::get_power_levels` from multiple places throughout the app in order to correctly configure the various UI components and again, on randomly decided events, to keep them up to date.
This patch starts publishing the power levels directly on the `RoomInfo` and allows them to be handled (and updated!) through the normal `subscribe_to_room_info_updates` mechanism.
This patch removes a name ambiguity around _listener_ vs. _subscriber_.
Both terms are used to talk about `RoomEventCacheListener`. We usually
use the term _subscriber_ for the type being returned by a `subscribe`
method. The code refers to this sometimes as listener, sometimes
as subscriber, sometimes both in the same sentence, which can be very
confusing! This patch solves this by using the _subscriber_ term only.
They are only semantically different if a macro of the same name exists,
in which case the foo::{self} import only imports foo-the-module, not
foo-the-macro.
We have a working implementation of the additional forms of QR Code login
of MSC4108. This commit moves the existing code into its own file
to make future updates easier to follow.
Remove the MediaFileHandle concept from the matrix-sdk-ffi crate on Wasm
platforms. File handles are not supported in the browser.
Signed-off-by: Daniel Salinas
The uniffi tool for generating JS/Wasm bindings utilizes rust as its
intermediate language.
As a result, the 'target' uniffi annotated library needs to be marked as
a 'lib' so that the generated rust code can utilize it to generate the
Wasm create + typescript bindings.
Signed-off-by: Daniel Salinas
This patch expands on the already existent `RoomPowerLevels` record (which it renames to `RoomPowerLevelsValues` to work around uniffi not exposing public fields) and nests them inside a new exported object that also provides methods for retrieving the actions that an user can take, methods moved from the room object.
This reduces the amount of async calls the clients need to make, simplifies the API and groups the code better together.
The system of platform targets was already quite messy, and becoming
even worse as we start preparing for Wasm support. Switch to features
instead to make this easier to work with.
UInt::MAX.try_into().unwrap() was causing errors on Wasm platforms, due
to the result being unrepresentable.
This `unwrap_or` was also always being calculated regardless, so I think
using `usize::MAX` is preferable on all platforms.
Signed-off-by: Daniel Salinas <zzorba@users.noreply.github.com>
Some of these Box<dyn Error + Send + Sync> are okay, but a few are
problematic.
Confirmed this still compiles fine on the fully working wasm tree.
Signed-off-by: Daniel Salinas
The main thing left now are the store traits, unsure how to deal with
those. `dynosaur` + `trait_variant` are kind of the modern replacement
for `async_trait`, but (a) `trait_variant` seemed to generate invalid
code here when I tried it and (b) even with that fixed I think the error
type erasure is going to present some extra problems. Maybe it's fine to
just keep the current solution for the store traits for now.
Signed-off-by: Jonas Platte <jplatte+matrix@posteo.de>
This patch switches from tarpaulin to llvm-cov for our coverage reports.
llvm-cov can use cargo-nextest to run the tests which means that we can
tolerate flaky tests for coverage just like do for the rest of our CI
run.
We can also start using JUnit reports to track flaky tests.
Proper support for receiving to-device messages for widgets.
If the widget is in an e2ee room, clear to-device traffic will be excluded. Also filter out internal to-device messages that widgets should not be aware off.
On this date (heh), the date divider reports include the following
information:
- initial timeline items in a shortened format,
- operations to apply,
- final timeline items in the same shortened format,
- errors
The shortened format includes:
- either, for events: "[event id]: server timestamp"
- or for date divider: "--- date divider timestamp"
As such, they don't include any personal information.
The initial timeline items state and set of operations to apply
constitutes a fully enclosed test case, so it's nice to report it to
Sentry, so we can reuse it almost as is (we'd only need to randomize the
event IDs) and fix it in a subsequent commit.
This patch simplifies the selected read receipt rendering by the fact
that we can simply fetch the selected timeline item instead of the event
ID and then do a search for the selected item.
Co-authored-by: Benjamin Bouvier <benjamin@bouvier.cc>
Signed-off-by: Damir Jelić <poljar@termina.org.uk>
This patch changes the strategy of `check_room_upgrades`.
Instead of checking all state events in `StateChanges` and
throwing an error in case of an invalid state, this patch updates
`state_events::sync::dispatch` to filter out invalid state events
(specifically `m.room.create` and `m.room.tombstone`), and log the
error. That way, the sync doesn't stop and the app can continue working
smoothly.
So `check_room_upgrades` is splited into two functions:
`is_create_event_valid` and `is_tombstone_event_valid`. It's no longer
necessary to detect mergers or splitters because those are _soft
errors_, however loops are still critical errors (hence the fact
the state events aren't stored nor saved nor applied, and that a log
is emitted).
Note: `check_room_upgrades` has been reverted in a previous commit,
that's why it doesn't appear in this diff.
This patch adds optional thread root event id parameters to the drafting functions exposed on the room level
allowing unfinished messages to be managed for the main room as well as any inner thread.
Internally it uses the room id or a tuple of the room id and the thread as keys for the various backing stores.
It doesn't make sense to do it after a back-pagination, since a
back-pagination does require a previous-batch token in the first place,
meaning that if we did paginate, then we did wait for a previous-batch
token beforehand.
Development of the SDK is proudly sponsored and maintained by [Element](https://element.io). Element uses the SDK in their next-generation mobile apps Element X on [iOS](https://github.com/element-hq/element-x-ios) and [Android](https://github.com/element-hq/element-x-android) and has plans to introduce it to the web and desktop clients as well.
The SDK is also the basis for multiple Matrix projects and we welcome contributions from all.
</div>
## Purpose
The SDK takes care of the low-level details like encryption,
syncing, and room state, so you can focus on your app's logic and UI. Whether
you're writing a small bot, a desktop client, or something in between, the SDK
is designed to be flexible, async-friendly, and ready to use out of the box.
[Matrix]: https://matrix.org/
[Rust]: https://www.rust-lang.org/
## Project structure
The Matrix Rust SDK is made up of several crates that build on top of each other. Here are the key ones:
The Matrix Rust SDK is made up of several crates that build on top of each
other. The following crates are expected to be usable as direct dependencies:
- [matrix-sdk-ui](https://docs.rs/matrix-sdk-ui/latest/matrix_sdk_ui/) – A high-level client library that makes it easy to build
full-featured UI clients with minimal setup. Check out our reference client,
@@ -45,6 +62,9 @@ The Matrix Rust SDK is made up of several crates that build on top of each other
See the [crypto tutorial](https://docs.rs/matrix-sdk-crypto/latest/matrix_sdk_crypto/tutorial/index.html)
for a step-by-step introduction.
All other crates are effectively internal-only and only structured as crates
for organizational purposes and to improve compilation times. Direct usage of them is discouraged.
## Status
The library is considered production ready and backs multiple client
@@ -54,9 +74,6 @@ implementations such as Element X
[Fractal](https://gitlab.gnome.org/World/fractal) and [iamb](https://github.com/ulyssa/iamb). Client developers should feel
confident to build upon it.
Development of the SDK has been primarily sponsored by Element though accepts
contributions from all.
## Bindings
The higher-level crates of the Matrix Rust SDK can be embedded in other
- Add `NotificationItem::raw_event` to get the raw event content of the event that triggered the notification, which can be useful for debugging and to support clients that want to implement custom handling for certain notifications. ([#6122](https://github.com/matrix-org/matrix-rust-sdk/pull/6122))
- [**breaking**] Extend `TimelineFocus::Event` to allow marking the target
- Add `NonFavorite` filter to the Room List API. ([#5991](https://github.com/matrix-org/matrix-rust-sdk/pull/5991)
- Add `call_intent` (either `RtcCallIntent::Audio` or `RtcCallIntent::Video`) field to `RtcNotification` event content. ([#6207](https://github.com/matrix-org/matrix-rust-sdk/pull/6207))
- Add `RoomInfo::active_room_call_consensus_intent` method to get the call intent for the current call,
- Add `Client::get_store_sizes()` so to query the size of the existing stores, if available. ([#5911](https://github.com/matrix-org/matrix-rust-sdk/pull/5911))
- Expose `is_space` in `NotificationRoomInfo`, allowing clients to determine if the room that triggered the notification is a space.
- Add push actions to `NotificationItem` and replace `SyncNotification` with `NotificationItem`.
- Add `Room::mark_as_fully_read_unchecked` so clients can mark a room as read without needing a `Timeline` instance. Note this method is not recommended as it can potentially cause incorrect read receipts, but it can needed in certain cases.
- Add `Timeline::latest_event_id` to be able to fetch the event id of the latest event of the timeline.
- Add `Room::load_or_fetch_event` so we can get a `TimelineEvent` given its event id ([#5678](https://github.com/matrix-org/matrix-rust-sdk/pull/5678)).
- Add `TimelineEvent::thread_root_event_id` to expose the thread root event id for this type too ([#5678](https://github.com/matrix-org/matrix-rust-sdk/pull/5678)).
- Add `NotificationSettings::get_raw_push_rules` so clients can fetch the raw JSON content of the push rules of the current user and include it in bug reports ([#5706](https://github.com/matrix-org/matrix-rust-sdk/pull/5706)).
- Add new API to decline calls ([MSC4310](https://github.com/matrix-org/matrix-spec-proposals/pull/4310)): `Room::decline_call` and `Room::subscribe_to_call_decline_events`
- Expose `m.federate` in `OtherState::RoomCreate` and `history_visibility` in `OtherState::RoomHistoryVisibility`, allowing clients to know whether a room federates and how its history is shared in the appropriate timeline events.
- Expose `join_rule` in `OtherState::RoomJoinRules`, allowing clients to know the join rules of a room from the appropriate timeline events.
### Changes
- `Timeline::latest_event_id` now uses its `ui::Timeline::latest_event_id` counterpart, instead of getting the latest event from the timeline and then its id.([#5864](https://github.com/matrix-org/matrix-rust-sdk/pull/5864))
- Build Android ARM64 bindings using better default RUSTFLAGS (the same used for iOS ARM64). This should improve performance. [(#5854)](https://github.com/matrix-org/matrix-rust-sdk/pull/5854)
## [0.14.0] - 2025-09-04
### Features:
- Add `LowPriority` and `NonLowPriority` variants to `RoomListEntriesDynamicFilterKind` for filtering
rooms based on their low priority status. These filters allow clients to show only low priority rooms
- `Room::send_call_notification` and `Room::send_call_notification_if_needed` have been removed, since the event type they send is outdated, and `Client` is not actually supposed to be able to join MatrixRTC sessions (yet). In practice, users of these methods probably already rely on another MatrixRTC implementation to participate in sessions, and such an implementation should be capable of sending notifications itself.
- The `GalleryItemInfo` variants now take an `UploadSource` rather than a `String` path to enable uploading
-`Client::send_call_notification_if_needed` now returns `Result<bool>` instead of `Result<()>` so we can check if
- `Client::send_call_notification_if_needed` now returns `Result<bool>` instead of `Result<()>` so we can check if
the event was sent.
- `Client::upload_avatar` and `Timeline::send_attachment` now may fail if a file too large for the homeserver media
config is uploaded.
@@ -25,12 +458,13 @@ Breaking changes:
Additions:
-`Client::subscribe_to_room_info` allows clients to subscribe to room info updates in rooms which may not be known yet.
This is useful when displaying a room preview for an unknown room, so when we receive any membership change for it,
- `Client::subscribe_to_room_info` allows clients to subscribe to room info updates in rooms which may not be known yet.
This is useful when displaying a room preview for an unknown room, so when we receive any membership change for it,
we can automatically update the UI.
- `Client::get_max_media_upload_size` to get the max size of a request sent to the homeserver so we can tweak our media
uploads by compressing/transcoding the media.
- Add `ClientBuilder::enable_share_history_on_invite` to enable experimental support for sharing encrypted room history on invite, per [MSC4268](https://github.com/matrix-org/matrix-spec-proposals/pull/4268).
- Add `ClientBuilder::enable_share_history_on_invite` to enable experimental support for sharing encrypted room history
on invite, per [MSC4268](https://github.com/matrix-org/matrix-spec-proposals/pull/4268).
- Support for adding a Sentry layer to the FFI bindings has been added. Only `tracing` statements with
the field `sentry=true` will be forwarded to Sentry, in addition to default Sentry filters.
@@ -46,7 +480,7 @@ Additions:
Breaking changes:
-`contacts` has been removed from `OidcConfiguration` (it was unused since the switch to OAuth).
- `contacts` has been removed from `OidcConfiguration` (it was unused since the switch to OAuth).
## [0.11.0] - 2025-04-11
@@ -123,7 +557,8 @@ Breaking changes:
- The `dynamic_registrations_file` field of `OidcConfiguration` was removed.
Clients are supposed to re-register with the homeserver for every login.
- `RoomPreview::own_membership_details` is now `RoomPreview::member_with_sender_info`, takes any user id and returns an `Option<RoomMemberWithSenderInfo>`.
- `RoomPreview::own_membership_details` is now `RoomPreview::member_with_sender_info`, takes any user id and returns an
`Option<RoomMemberWithSenderInfo>`.
Additions:
@@ -138,9 +573,11 @@ Additions:
- Add `Timeline::send_thread_reply` for clients that need to start threads
- Add `ClientBuilder::session_pool_max_size`, `::session_cache_size` and `::session_journal_size_limit` to control the stores configuration, especially their memory consumption
- Add `ClientBuilder::session_pool_max_size`, `::session_cache_size` and `::session_journal_size_limit` to control the
stores configuration, especially their memory consumption
- Add `Room::member_with_sender_info` to get both a room member's info and for the user who sent the `m.room.member` event the `RoomMember` is based on.
- Add `Room::member_with_sender_info` to get both a room member's info and for the user who sent the `m.room.member`
This uses [`uniffi`](https://mozilla.github.io/uniffi-rs/Overview.html) to build the matrix bindings for native support and wasm-bindgen for web-browser assembly support. Please refer to the specific section to figure out how to build and use the bindings for your platform.
## Features
Given the number of platforms targeted, we have broken out a number of features
### Functionality
-`sentry`: Enable error monitoring using Sentry, not supports on Wasm platforms.
-`sqlite`: Use SQLite for the session storage.
-`bundled-sqlite`: Use an embedded version of SQLite instead of the system provided one.
-`indexeddb`: Use IndexedDB for the session storage.
### Unstable specs
-`unstable-msc4274`: Adds support for gallery message types, which contain multiple media elements.
## Platforms
Each supported target should use features to build the relevant system. Here are some suggested feature flags for the major platforms:
@@ -41,10 +41,10 @@ impl UnableToDecryptHook for UtdHook {
// UTDs that have been decrypted in the `IGNORE_UTD_PERIOD` are just ignored and
// not considered UTDs.
ifletSome(duration)=&info.time_to_decrypt{
if*duration<IGNORE_UTD_PERIOD{
return;
}
ifletSome(duration)=&info.time_to_decrypt
&&*duration<IGNORE_UTD_PERIOD
{
return;
}
// Report the UTD to the client.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
ganfra