Compare commits

...

138 Commits

Author SHA1 Message Date
RiotRobot 3a55efb476 v30.3.0 2023-12-19 15:47:50 +00:00
RiotRobot dd53ec722f v30.3.0-rc.0 2023-12-12 16:56:48 +00:00
Andy Balaam b03dc6ac43 Move roomList out of MatrixClient, into legacy Crypto (#3944)
* Comment explaining the purpose of RoomList

* Fix incorrect return type declaration on RoomList.getRoomEncryption

* Move RoomList out of MatrixClient, into legacy Crypto

* Initialise RoomList inside Crypto.init to allow us to await it
2023-12-11 10:30:27 +00:00
Valere 13c7e0ebda Element-R: Refactor per-session key backup download (#3929)
* initial commit

* new interation test

* more comments

* fix test, quick refactor on request version

* cleaning and logs

* fix type

* cleaning

* remove delegate stuff

* remove events and use timer mocks

* fix import

* ts ignore in tests

* Quick cleaning

* code review

* Use Errors instead of Results

* cleaning

* review

* remove forceCheck as not useful

* bad naming

* inline pauseLoop

* mark as paused in finally

* code review

* post merge fix

* rename KeyDownloadRateLimit

* use same config in loop and pass along
2023-12-08 14:21:07 +00:00
David Baker 2cd63ca4b9 Fix notifications appearing for old events (#3946)
A method that we use for fetching recursive related events on homeservers
without MSC3981 support injects events into the timeline in timestamp
order using a special method on event-timeline-set. Injecting events using
this method could cause on-screen notifications because it incorrectly set
the 'liveEvent' flag to true if the events were added tio the live timeline.
These events are never live though as the point is that we're fetching them.
2023-12-07 17:03:01 +00:00
Richard van der Hoff 479c4278a6 Element-R: disable sending room key requests (#3939) 2023-12-07 16:17:21 +00:00
David Baker 636fc3daaa Include event & room ID in log line (#3945)
...for when we ignore events that don't appear in the room timeline
2023-12-07 13:41:43 +00:00
Hubert Chathi 1d1309870a Don't back up keys that we got from backup (#3934)
* don't back up keys that we got from backup

* lint

* lint again

* remove key source struct and add function for importing from backup

* apply changes from review

---------

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2023-12-07 11:32:27 +00:00
Jakub Onderka 13b8f01062 Fix upload with empty Content-Type (#3918)
Fixes #3917

Signed-off-by: Jakub Onderka <ahoj@jakubonderka.cz>
Co-authored-by: Florian Duros <florianduros@element.io>
2023-12-06 17:07:19 +00:00
David Baker cd672ec4cf Log if event ID is not foudn in the room (#3943) 2023-12-06 16:30:48 +00:00
David Baker 2363703b64 Prevent phantom notifications from events not in a room's timeline (#3942)
* Test whether an event not in a room's timeline causes notification count increase

Commited separately to demonstrate test failing before.

* Don't fix up notification counts if event isn't in the room

As explained by the comment, hopefully.

* Fix other test
2023-12-06 16:25:10 +00:00
Andy Balaam 1250bb8833 Call scheduleAllGroupSessionsForBackup during resetKeyBackup (#3935)
since its equivalent is done automatically in Rust crypto when we call
resetKeyBackup.
2023-12-06 15:09:31 +00:00
Richard van der Hoff 016ef12c4a Fix "mark_skipped" action again
Yet another go at this. The name is actually coming from an explicit
`github-status-action` action in the called workflows.
2023-12-06 15:00:41 +00:00
Richard van der Hoff 84d193a9a2 Fix "mark_skipped" action again
Turns out that the name we need is the key of the job in the workflow
definition; *not* the `name` property.
2023-12-06 14:53:11 +00:00
Richard van der Hoff 9d5f1bb4fc Fix "mark_skipped" action for end-to-end tests (#3940)
This seems to have been broken by
https://github.com/matrix-org/matrix-js-sdk/pull/3914, which changed the name
of the status check that is updated.
2023-12-06 11:41:50 +00:00
Richard van der Hoff 228131edf3 Bump matrix-sdk-crypto-wasm to 3.4.0 (#3938) 2023-12-05 16:57:38 +00:00
Michael Telatynski 23ad637aad Update cypress.yml 2023-12-05 15:01:08 +00:00
RiotRobot 103617c70e Resetting package fields for development 2023-12-05 13:35:58 +00:00
RiotRobot 8d84621b07 Merge branch 'master' into develop 2023-12-05 13:35:57 +00:00
RiotRobot 6d018826f4 v30.2.0 2023-12-05 13:35:06 +00:00
Richard van der Hoff 41878c7a43 Element-R: await /keys/query during Verification requests (#3932) 2023-12-05 11:18:12 +00:00
Michael Telatynski f31e83fd03 Run matrix-react-sdk playwright tests downstream (#3914)
* Run matrix-react-sdk playwright tests downstream

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update .github/workflows/cypress.yml

Co-authored-by: R Midhun Suresh <hi@midhun.dev>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
Co-authored-by: R Midhun Suresh <hi@midhun.dev>
2023-12-04 10:55:57 +00:00
Richard van der Hoff b515cdbdbb Rust-crypto: fix bootstrapCrossSigning on second call (#3912)
* Rust-crypto: fix `bootstrapCrossSigning` on second call

Currently, `bootstrapCrossSigning` raises an exception if it is called a second
time before secret storage is set up. It is easily fixed by checking that 4S is
set up before trying to export to 4S.

Also a few logging fixes while we're in the area.

* Factor out an `AccountDataAccumulator`

* Another test for bootstrapCrossSigning
2023-12-01 14:39:04 +00:00
Richard van der Hoff f4b6f91ee2 Bump matrix-rust-sdk-crypto-wasm to v3.2.0 (#3933)
* Bump `matrix-rust-sdk-crypto-wasm` to v3.2.0

* Reinstate timeout on `getUserDevices` call

Turns out that this used to have a timeout of 1 second in the wasm
bindings, which it no longer does. Reinstate it here.
2023-12-01 12:05:13 +00:00
Michael Telatynski df4536492c Update Sibz/github-status-action to use node16 to silence warning (#3910)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-12-01 10:08:10 +00:00
Valere 2e98da4224 Signal key backup in cache (#3928)
* Signal key backup in cache

* code review

* quick doc

* code review
2023-11-30 08:15:37 +00:00
Valere 48d9d9b4c9 move get device key API from client to crypto (#3899)
MatrixClient API was exposing two methods that only worked for legacy crypto:
- getDeviceEd25519Key
- getDeviceCurve25519Key

=> These are used in the react-sdk for some functionality (rageshake, sentry, rendez-vous).

I have deprecated those calls from MatrixClient and created a new API in CryptoApi (where it belongs):

getOwnDeviceKeys(): Promise<OwnDeviceKeys>
2023-11-29 17:54:06 +00:00
Richard van der Hoff d90ae11e2b Expose new method CryptoApi.crossSignDevice (#3930) 2023-11-29 14:45:26 +00:00
Valere 3f246c6080 fix uncaught exceptions in Backup Loop for rust sdk (#3907)
* fix uncaught exceptions

* Update src/rust-crypto/backup.ts

Co-authored-by: Florian Duros <florianduros@element.io>

---------

Co-authored-by: Florian Duros <florianduros@element.io>
2023-11-29 09:07:56 +00:00
renovate[bot] 68911520d3 Update babel monorepo to v7.23.4 (#3921)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:37:12 +00:00
renovate[bot] 393a8d0cdb Update dependency @types/node to v18.18.13 (#3923)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:36:35 +00:00
renovate[bot] 51b63092b4 Update all non-major dependencies (#3920)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:36:30 +00:00
renovate[bot] b49c9639b9 Update dependency @types/jest to v29.5.10 (#3922)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:36:25 +00:00
renovate[bot] c588611fc0 Update matrix-org/netlify-pr-preview action to v3 (#3926)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:36:21 +00:00
renovate[bot] 5b34e4beaf Update typedoc (#3925)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-29 08:35:46 +00:00
Johannes Marbach 91f16e5e8e Merge pull request #3927 from matrix-org/midhun/fix-broken-ci 2023-11-29 08:37:33 +01:00
R Midhun Suresh 9cf257da0e Use new commit hash 2023-11-29 12:36:00 +05:30
R Midhun Suresh 188de3c4c8 Use new secret 2023-11-29 11:15:19 +05:30
renovate[bot] 67019a3486 Update matrix-org/matrix-react-sdk digest to e76a37e (#3919)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-28 18:21:56 +00:00
Richard van der Hoff a39b1203f2 Add guards against MatrixClient.stopClient calls (#3913)
If we call methods on `OlmMachine` after `MatrixClient.stopClient` is called,
we will end up with a "use of moved value" error. We can turn these into
something more useful with judicious use of `getOlmMachineOrThrow`.

Alternatively, we can sidestep the issue by bailing out sooner.
2023-11-28 16:30:18 +00:00
RiotRobot df1a6a583a v30.2.0-rc.0 2023-11-28 16:11:15 +00:00
Andy Balaam c49a527e5e Rewrite receipt-handling code (#3901)
* Rewrite receipt-handling code

* Add tests around dangling receipts

* Fix mark as read for some rooms

* Add missing word

---------

Co-authored-by: Florian Duros <florian.duros@ormaz.fr>
Co-authored-by: Florian Duros <florianduros@element.io>
2023-11-28 14:43:48 +00:00
Richard van der Hoff a7496627fc Explicitly free some Rust-side objects (#3911)
* Explicitly `free` stuff returned by `OlmMachine.getIdentity()`

* Explicitly `free` stuff returned by `OlmMachine.getDevice()`

* one more
2023-11-28 13:14:53 +00:00
Richard van der Hoff 8ef2ca681c Update to matrix-sdk-crypto-wasm 3.1.0 (#3909) 2023-11-27 15:15:58 +00:00
Johannes Marbach 0c7342cb20 Set up CI to lint workflows with action-validator (#3905)
* Set up CI to lint workflows with action-validator

* Rename release-action workflow
2023-11-24 14:41:19 +00:00
Will Hunt 429c05ba85 TimestampToEventResponse.origin_server_ts should be a number (#3906) 2023-11-23 16:46:01 +00:00
Andy Balaam af9993a710 Remove 'Ignoring receipt' log line that logs very often' (#3904)
* Remove 'Ignoring receipt' log line that logs very often'

* Fix test expecting the log line I removed
2023-11-22 12:14:42 +00:00
Valere ff501834e6 Only await key query after lazy members resolved (#3902)
* Only await key query after lazy members resolved

* code review

* Update src/rust-crypto/RoomEncryptor.ts

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

---------

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2023-11-22 09:19:13 +00:00
Michael Telatynski ef9157b37a Fix Ingest upstream changes for downstreams with missing sections
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-21 12:31:40 +00:00
Michael Telatynski da0a55cea4 Merge remote-tracking branch 'origin/develop' into develop 2023-11-21 11:46:06 +00:00
Michael Telatynski d644f111ea Fix Ingest upstream changes for element-desktop
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-21 11:45:55 +00:00
RiotRobot b2018ef81b Resetting package fields for development 2023-11-21 11:00:05 +00:00
RiotRobot a4faab6155 Merge branch 'master' into develop 2023-11-21 11:00:04 +00:00
RiotRobot 4ab226e580 v30.1.0 2023-11-21 10:59:16 +00:00
Richard van der Hoff 1889f8dad5 Reduce console log spam (#3896)
* Reduce console log spam

A couple of different things:

 * Increase the `MaxListeners` setting on `MatrixClient` and `Thread`, so that
   we don't get "possible EventEmitter leak" warnings

 * Disable a couple of warnings/info lines that are just part of regular
   operation and are logged in large volumes.

* another noisy log line

* Reinstate warning about receipts for missing events

Apparently this is being worked on
2023-11-20 18:17:06 +00:00
Timo e98ce78027 Better fallback for the event localTimestamp calculation. (#3900)
* better fallback to localTimestamp calculation

Signed-off-by: Timo K <toger5@hotmail.de>

* make `isExpired` impl simpler to read

Signed-off-by: Timo K <toger5@hotmail.de>

* update tests

Signed-off-by: Timo K <toger5@hotmail.de>

* refactor to use localTimestamp in the mocks

Signed-off-by: Timo K <toger5@hotmail.de>

* Update src/models/event.ts

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Update src/models/event.ts

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Update and clarify comments.
So that the localTimestamp and localAge behavior is easier to understand.

Signed-off-by: Timo K <toger5@hotmail.de>

* Replace localTimestamp biding
with binding the whole roomState

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2023-11-20 17:20:04 +00:00
Richard van der Hoff 83ba0fbb49 Improve logging of event encryption in RustCrypto (#3898)
* Improve logging of event encryption in `RustCrypto`

* fix tests
2023-11-19 21:16:41 +00:00
Richard van der Hoff 757c5e1d71 Stop logging decryptions as retries (#3897)
Somebody seems to have decided that `isRetry` needs setting whenever we try to decrypt an event. This is nonsense, and leads to confusing logs.
2023-11-17 14:30:27 +00:00
Johannes Marbach eca651c0c2 Explicitly forward ELEMENT_BOT_TOKEN (#3894)
`inherit` doesn't work across orgs, sadly.
2023-11-16 14:46:56 +00:00
Andy Balaam 2205445a50 Specify the correct environment for the docs builder workflow (#3893) 2023-11-16 13:40:02 +00:00
renovate[bot] f168144c84 Update matrix-org/sonarcloud-workflow-action action to v2.7 (#3892)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-16 10:58:00 +00:00
Michael Telatynski eb288d125f Remove unused dependencies and add transitive dep (#3874)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-16 09:47:30 +00:00
Michael Telatynski 4a72364fe3 Merge remote-tracking branch 'origin/develop' into develop 2023-11-16 08:33:20 +00:00
Michael Telatynski c2fa579fb2 Fix merge-release-notes.js script
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-16 08:33:08 +00:00
renovate[bot] f71735d0c2 Update babel monorepo to v7.23.3 (#3881)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-15 22:26:23 +00:00
renovate[bot] e5ccfa86fe Update actions/github-script action to v7 (#3886)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-15 22:25:24 +00:00
renovate[bot] 97c531aa42 Update all non-major dependencies (#3884)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-15 22:24:58 +00:00
renovate[bot] 44487078fb Update dependency @types/jest to v29.5.8 (#3883)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-15 22:24:49 +00:00
renovate[bot] e3c70a3ee4 Update actions/setup-node action to v4 (#3887)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-15 22:19:02 +00:00
renovate[bot] feb60a54b2 Update definitelyTyped (#3882)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-15 22:10:00 +00:00
David Baker c6e6248cd6 Don't rotate keys if not managing media keys (#3877)
This could have caused weirdness in non per-user key calling mode.
2023-11-15 14:44:58 +00:00
Timo 10cd84a653 Add CallNotifyEvent to support matrixRTC ringing (#3878)
* Add CallNotifyEvent to support matrix rtc ringing

Signed-off-by: Timo K <toger5@hotmail.de>

* test SessionId

Signed-off-by: Timo K <toger5@hotmail.de>

* docs + sessionId->callId

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
2023-11-15 11:20:05 +00:00
Andy Balaam c36166d156 Merge pull request #3869 from matrix-org/andybalaam/unrevert-deletion-move-prs
Unrevert "Move redacted messages out of any thread, into main thread"
2023-11-15 08:23:50 +00:00
renovate[bot] 3a2cf14a68 Update matrix-org/matrix-react-sdk digest to f6ef476 (#3879)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-15 00:58:24 +00:00
Andy Balaam dd94f67a4f Merge branch 'develop' into andybalaam/unrevert-deletion-move-prs 2023-11-14 16:13:56 +00:00
Michael Telatynski 138281c620 Fix RELEASE_ID
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-14 17:12:49 +01:00
RiotRobot f75abecc92 v30.1.0-rc.1 2023-11-14 15:37:50 +00:00
Michael Telatynski 378a91fe10 Fix yarn version call in release-action.yml
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-14 16:31:58 +01:00
Michael Telatynski 300635e3ee Update package.json 2023-11-14 15:31:26 +00:00
Michael Telatynski 37ba169abf Update release-drafter.yml 2023-11-14 14:52:56 +00:00
RiotRobot e6e7798389 v30.1.0-rc.0 2023-11-14 14:29:48 +00:00
Michael Telatynski 48fe267ea7 release-action.yml iterate
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-14 15:26:31 +01:00
Michael Telatynski a11fd8bc86 release-npm.yml ref=staging
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-14 15:22:03 +01:00
Michael Telatynski eb9e557a64 Update release-action.yml 2023-11-14 14:19:52 +00:00
RiotRobot 41c8c40d47 v30.1.0-rc.1 2023-11-14 14:17:37 +00:00
Michael Telatynski b9e684fdc3 Update release-action.yml 2023-11-14 14:07:25 +00:00
RiotRobot 9faff0dbff v30.1.0-rc.0 2023-11-14 14:04:14 +00:00
Michael Telatynski 9044145a7e Update release-action.yml 2023-11-14 14:03:09 +00:00
Michael Telatynski 939def2aa1 Update release-drafter.yml 2023-11-14 13:51:16 +00:00
Michael Telatynski c54f8f6106 Update release-drafter.yml 2023-11-14 13:47:56 +00:00
Michael Telatynski 25a777a0a6 Update release-drafter.yml 2023-11-14 13:41:37 +00:00
Michael Telatynski 7de9b23e59 Add support for ingest-changes to refer to a project without package.json (#3864)
* Tidy reusable release workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add ability to include upstream changes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add ability to upload assets and gpg sign them

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update relative composite actions

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up validating release tarball signature

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Validate release has expected assets

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Paths

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use gpg outputs for email instead of scraping it ourselves

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* v6

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Extract pre-release and post-merge-master scripts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Reuse pre-release and post-merge-master scripts in gha

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Cull unused vars

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove unused variables

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify and fix merge-release-notes script

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Tidy release automation

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update release.sh

* Move environment

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* s/includes/contains/

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate uses syntax

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix action-repo calls

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix RELEASE_NOTES env

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix if check

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix gpg tag signing

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Cull stale params

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix sign-release-tarball paths being outside the workspace

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix gpg validation (of course wget uses `-O` and not `-o`)

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix expected asset assertion

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix release publish mode

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add support for ingest-changes to refer to a project without it being in node_modules

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-13 12:43:18 +00:00
Michael Telatynski d179b8c557 Add automation to advance release blocker labels during the release (#3866)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-13 09:46:11 +00:00
ElementRobot 76f993e7ff Merge branch 'master' into develop 2023-11-13 09:44:12 +00:00
ElementRobot 430e6cae94 v30.0.1 2023-11-13 09:44:06 +00:00
ElementRobot e01a1d533c Prepare changelog for v30.0.1 2023-11-13 09:44:04 +00:00
ElementRobot 46a6a76a41 [Backport staging] Ensure setUserCreator is called when a store is assigned (#3876)
Co-authored-by: R Midhun Suresh <hi@midhun.dev>
2023-11-13 09:11:27 +00:00
Johannes Marbach d2e951738a Automatically add tech-debt issues to the right project (#3872) 2023-11-11 07:21:31 +00:00
R Midhun Suresh 882dc920c3 Ensure setUserCreator is called when a store is assigned (#3867)
* Add method to set store

* Use not null assertion

* Use getter/setter

* No need for check if we use setter
2023-11-11 07:20:36 +00:00
Michael Telatynski 9efc0acb9d Automate checking there is no published release using the version already (#3865)
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-11 06:53:14 +00:00
Michael Telatynski 625753c388 Update tests.yml (#3847) 2023-11-10 20:30:33 +00:00
Richard van der Hoff a28530004a Bump matrix-sdk-crypto-wasm to 3.0.1 (#3849)
* Bump matrix-sdk-crypto-wasm to 3.0.0

... which changes the API of `bootstrapCrossSigning` a bit.

* Fix class names in test

* fix brokenness in bootstrapCrossSigning

* Bump to `matrix-sdk-crypto-wasm` 3.0.1
2023-11-10 16:57:50 +00:00
Richard van der Hoff 437b7ff780 Revert "Better fallback for unavailable event age (#3854)" (#3870)
This reverts commit 84bd8ab81f.
2023-11-10 00:06:48 +00:00
Michael Telatynski 24ed030294 Extend release automation with GPG signing, assets & changelog merging (#3852)
* Tidy reusable release workflow

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add ability to include upstream changes

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add ability to upload assets and gpg sign them

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update relative composite actions

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Wire up validating release tarball signature

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Validate release has expected assets

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Paths

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Use gpg outputs for email instead of scraping it ourselves

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* v6

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Extract pre-release and post-merge-master scripts

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Reuse pre-release and post-merge-master scripts in gha

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Cull unused vars

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove unused variables

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify and fix merge-release-notes script

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Tidy release automation

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Update release.sh

* Move environment

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* s/includes/contains/

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate uses syntax

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix action-repo calls

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix RELEASE_NOTES env

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix if check

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix gpg tag signing

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Cull stale params

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix sign-release-tarball paths being outside the workspace

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix gpg validation (of course wget uses `-O` and not `-o`)

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix expected asset assertion

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix release publish mode

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-09 20:54:29 +00:00
Andy Balaam 5c160d0f45 Shorten TimelineWindow when an event is removed (#3862)
* Shorten TimelineWindow when an event is removed

Needed for the fix for https://github.com/vector-im/element-web/issues/26498

* Declare onTimelineEvent as a standard method to match surrounding code
2023-11-09 16:41:15 +00:00
Andy Balaam 53615c9938 Merge pull request #3855 from matrix-org/rav/cypress-element-r
Cypress workflow: remove redundant `rust-crypto` param
2023-11-09 15:34:49 +00:00
Andy Balaam d8735cf543 Merge pull request #3868 from matrix-org/midhun/update-cypress-workfile
Update workfile to pull in changes from react-sdk
2023-11-09 15:07:40 +00:00
R Midhun Suresh ffb4cae792 Update workfile 2023-11-09 20:00:45 +05:30
Andy Balaam 0261868eb6 Revert "Revert "Move the redaction event to main at the same time we move redacted""
This reverts commit 11755f5a0a1486fa6ad3cb9e4b8959ddc7e1d276.
2023-11-09 14:30:41 +00:00
Andy Balaam 6ba4b35526 Revert "Revert "Don't remove thread info from a thread root when it is redacted""
This reverts commit 4dbff2a837cbc5ba37424c65ccdc833a1843deb2.
2023-11-09 14:30:41 +00:00
Andy Balaam f5ad4d0a73 Revert "Revert "Move all related messages into main timeline on redaction""
This reverts commit 257b40bceb304001c03aaec7b140a1fd05c96d9e.
2023-11-09 14:30:41 +00:00
Andy Balaam 582ea68c31 Revert "Revert "Factor out the code for moving an event to the main timeline""
This reverts commit 272be48a54a45df89603a27fbbe6e26da88b95ba.
2023-11-09 14:30:41 +00:00
Andy Balaam 304c2b12bf Revert "Revert "Factor out utils in redaction tests""
This reverts commit 2525c82049dc1a958446b66cc85656c1b57a5271.
2023-11-09 14:30:41 +00:00
Andy Balaam a3762c8e22 Revert "Revert "Move redaction event tests into their own describe block""
This reverts commit 2e24481df335411ee489ac7046c5514821afa4fa.
2023-11-09 14:30:41 +00:00
Andy Balaam 8b2a334ac4 Revert "Revert "Move redacted messages out of any thread, into main timeline.""
This reverts commit 46114a025c5ec5b2658803c8c86d5e855b55a4fb.
2023-11-09 14:30:41 +00:00
Michael Telatynski 5931a5119c Tidy release automation (#3857) 2023-11-08 12:20:25 +00:00
renovate[bot] 6ae3c208f6 Update all non-major dependencies (#3863)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-08 12:08:04 +00:00
ElementRobot 107e28e114 Resetting package fields for development 2023-11-07 15:11:56 +00:00
ElementRobot 1d1157f546 Merge branch 'master' into develop 2023-11-07 15:11:53 +00:00
Andy Balaam 7813e12eb0 Revert code moving deleted messages to main timeline (#3858)
* Revert "Move the redaction event to main at the same time we move redacted"

This reverts commit 378a776815f63fdd1e4d507af35046c0ba88153c.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Don't remove thread info from a thread root when it is redacted"

This reverts commit 17b61a69c20677e39e4f3b1b4ed5903421eaee6b.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Move all related messages into main timeline on redaction"

This reverts commit d8fc1795f1319b6a77175c5c584ae03be53c457c.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Factor out the code for moving an event to the main timeline"

This reverts commit 942dfcb84b8aef6ea84a419d73e845d3611bd91c.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Factor out utils in redaction tests"

This reverts commit 43a0dc56e130f75ef695b52cd23945e10393119a.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Move redaction event tests into their own describe block"

This reverts commit 9b0ea80f93fe944da03c27df26064cae1765c94d.

Context: https://github.com/vector-im/element-web/issues/26498

* Revert "Move redacted messages out of any thread, into main timeline."

This reverts commit b94d137398.

Context: https://github.com/vector-im/element-web/issues/26498
2023-11-07 13:41:33 +00:00
David Baker 036fd943ac Rotate per-participant keys when a member leaves (#3833)
* WIP refactor for removing m.call events

* Always remember rtcsessions since we need to only have one instance

* Fix tests

* Fix import loop

* Fix more cyclic imports & tests

* Test session joining

* Attempt to make tests happy

* Always leave calls in the tests to clean up

* comment + desperate attempt to work out what's failing

* More test debugging

* Okay, so these ones are fine?

* Stop more timers and hopefully have happy tests

* Test no rejoin

* Test malformed m.call.member events

* Test event emitting

and also move some code to a more sensible place in the file

* Test getActiveFoci()

* Test event emitting (and also fix it)

* Test membership updating & pruning on join

* Test getOldestMembership()

* Test member event renewal

* Don't start the rtc manager until the client has synced

Then we can initialise from the state once it's completed.

* Fix type

* Remove listeners added in constructor

* Stop the client here too

* Stop the client here also also

* ARGH. Disable tests to work out which one is causing the exception

* Disable everything

* Re-jig to avoid setting listeners in the constructor

and re-enable tests

* No need to rename this anymore

* argh, remove the right listener

* Is it this test???

* Re-enable some tests

* Try mocking getRooms to return something valid

* Re-enable other tests

* Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing

* Oops, don't enable the ones that were skipped before

* One more try at the sensible way

* Didn't work, go back to the hack way.

* Log when we manage to send the member event update

* Support `getOpenIdToken()` in embedded mode (#3676)

* Call `sendContentLoaded()` (#3677)

* Start MatrixRTC in embedded mode (#3679)

* Reschedule the membership event check

* Bump widget api version

* Add mock for sendContentLoaded()

* Embeded mode pre-requisites

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Embeded mode E2EE

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Encryption condition

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Embeded mode pre-requisites"

This reverts commit 8cd73702052609c995ad754e31f85d0da0be4aa9.

* Get back event type

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

fds

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Change embedded E2EE implementation

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* More log detail

* Fix tests

and also better assert because the tests were passing undefined which
was considered fine because we were only checking for null.

* Simplify updateCallMembershipEvent a bit

* Split up updateCallMembershipEvent some more

* Use `crypto.getRandomValues()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Rename to `membershipToUserAndDeviceId()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Better error

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add log line

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add comment

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Send call ID in enc events

(also a small refactor)

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert making `joinRoomSession()` async

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `client` `private` again

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Just use `toString()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix `callId` check

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map compare

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix emitting

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Explicit logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Refactor

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` public

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Only update keys based on others

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix call order

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Improve logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Avoid races

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Avoid races"

This reverts commit f65ed72d6e.

* Add try-catch

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` private

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Handle indices and throttling

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix merge mistakes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Mort post-merge fixes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Split out key generation from key sending

And send all keys in a key event (changes the format of the key event)
rather than just the one we just generated.

* Remember and clear the timeout for the send key event

So we don't schedule more key updates if one is already pending.
Also don't update the last sent time when we didn't actually send the
keys.

* Make key event resends more robust

* Attempt to make tests pass

* crypto wasn't defined at all

* Hopefully get interface right

* Fix key format on the wire to base64

* Add comment

* More standard method order

* Rename encryptMedia

The js-sdk doesn't do media and therefore doesn't do media encryption

* Stop logging encryption keys now

* Use regular base64

It's not going in a URL, so no need

* Re-add base64url

randomstring was using it. Also give it a test.

* Add tests for randomstring

* Switch between either browser or node crypto

Let's see if this will work...

* Obviously crypto has already solved this

* Some tests for MatrixRTCSession key stuff

* Test keys object contents

* Change keys event format

To move away from m. keys

* Test key event retries

* Test onCallEncryption

* Test event sending & spam prevention

* Test event cancelation

* Test onCallEncryption called

* Better before/after member comparison

Only trigger for when members actually join, and just generally
make it a bit more understandable.

* Rotate per-participant keys when a member leaves

With a delay borth before making a new key, to try to batch up multiple
people leaving into a single key change, and a delay before actually
using the new key to allow time for it to arrive.

This increasingly feels like storing our own sender key in the same set
is suboptimal because we're starting to have to treat it more & more
specially.

* Some errors didn't have data

* Fix binary key comparison

& add log line

* Fix compare function with undefined values

* Test key rotation

* Test caught a merge bug!

* The missing word was, 'delay'

* More input validation

---------

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
Co-authored-by: Šimon Brandner <simon.bra.ag@gmail.com>
2023-11-07 11:12:11 +00:00
Timo 84bd8ab81f Better fallback for unavailable event age (#3854)
* Age fallback using origin_server_ts instead of 0

Signed-off-by: Timo K <toger5@hotmail.de>

* use getMsUntilExpiry for isExpired

Signed-off-by: Timo K <toger5@hotmail.de>

* fix tests
tests now also rely on localTimestamp. So this need to be mocked as well

Signed-off-by: Timo K <toger5@hotmail.de>

* better fallback for unavailable unsigned

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
2023-11-06 17:12:24 +00:00
Michael Telatynski a25ba7bfd9 Iterate reusable release automation workflows (#3851)
* Clean up unused envvar

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Make the gitflow workflow reusable

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add support for resetting dependencies to develop after merge

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Rename workflow file

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-06 13:25:19 +00:00
Andy Balaam 311494bd44 Ignore receipts pointing at missing or invalid events (#3817)
* Ignore receipts pointing at missing or invalid events

* Remove extra whitespace from log message

* Unit tests for ignoring invalid receipts

* Improve comments around getEventReadUpTo

* Re-instate second param to compareEventOrdering in test

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Further improve comments around getEventReadUpTo

---------

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2023-11-06 09:19:21 +00:00
Richard van der Hoff 89b7e7d792 Cypress workflow: remove redundant rust-crypto param
https://github.com/matrix-org/matrix-react-sdk/pull/11828 removes the
`rust-crypto` input param from the reusable cypress workflow. This gets rid of
it on the calling side.
2023-11-03 15:51:07 +00:00
Valere 7921fee164 Fix members being loaded from server on initial sync (defeating lazy loading) (#3830)
* fix members loaded on intitial sync

* Update test to use KeyResponder

* Use E2EKeyResponder

* code review

* better comment

* fix test

* post merge fix

* fix imports

* refactoring, better names

* code review

* clean tests

* Cleanups per review comments

* fix test

* Apply suggestions from code review

---------

Co-authored-by: Richard van der Hoff <richard@matrix.org>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2023-11-03 14:55:48 +00:00
Richard van der Hoff 5bc132a24c Revert "Age fallback using origin_server_ts instead of 0 (#3839)" (#3853)
This reverts commit 685ef791c8.
2023-11-03 13:32:42 +00:00
Timo 685ef791c8 Age fallback using origin_server_ts instead of 0 (#3839)
* Age fallback using origin_server_ts instead of 0

Signed-off-by: Timo K <toger5@hotmail.de>

* use getMsUntilExpiry for isExpired

Signed-off-by: Timo K <toger5@hotmail.de>

* fix tests
tests now also rely on localTimestamp. So this need to be mocked as well

Signed-off-by: Timo K <toger5@hotmail.de>

* fix another test that now also depends on localTimestamp

Signed-off-by: Timo K <toger5@hotmail.de>

* fix tests and cleanup

Signed-off-by: Timo K <toger5@hotmail.de>

* format

Signed-off-by: Timo K <toger5@hotmail.de>

* make things simpler by calculating localTimestamp
from getLocalAge

Signed-off-by: Timo K <toger5@hotmail.de>

* this test was not covered by the change to mockRTCEvent

Signed-off-by: Timo K <toger5@hotmail.de>

* format

Signed-off-by: Timo K <toger5@hotmail.de>

---------

Signed-off-by: Timo K <toger5@hotmail.de>
2023-11-02 13:02:02 +00:00
Michael Telatynski 4458dcc2a4 Make release automation reusable and add dependency upgrade support (#3848)
* Extract release into reusable action

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add dependency upgrade task to release-action

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Prevent develop dependencies

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Simplify dependency management

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add missing secret declaration

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2023-11-01 23:24:05 +00:00
renovate[bot] 36c958642c Update all non-major dependencies (#3842)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-01 08:34:18 +00:00
renovate[bot] b62e97eb92 Update actions/setup-node action to v4 (#3844)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-11-01 08:31:51 +00:00
Michael Telatynski 448fab9e8a New release automations (#3813) 2023-11-01 08:40:43 +00:00
Hugh Nimmo-Smith e2a2039aa8 Remove deprecated support for unstable MSC3882 (#3755)
* Support for stable MSC3882 get_login_token

* Make changes non-breaking by deprecation

* Remove deprecated exports from MSC3882 stabilisation

* Feat remove support for unstable MSC3882

* Remove bad line from rebase
2023-10-31 17:15:54 +00:00
renovate[bot] 99f70cd048 Update JS-DevTools/npm-publish action to v3 (#3843)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-10-31 16:09:00 +00:00
Šimon Brandner bf81c4bfeb Add E2EE for embedded mode of Element Call (#3667)
* WIP refactor for removing m.call events

* Always remember rtcsessions since we need to only have one instance

* Fix tests

* Fix import loop

* Fix more cyclic imports & tests

* Test session joining

* Attempt to make tests happy

* Always leave calls in the tests to clean up

* comment + desperate attempt to work out what's failing

* More test debugging

* Okay, so these ones are fine?

* Stop more timers and hopefully have happy tests

* Test no rejoin

* Test malformed m.call.member events

* Test event emitting

and also move some code to a more sensible place in the file

* Test getActiveFoci()

* Test event emitting (and also fix it)

* Test membership updating & pruning on join

* Test getOldestMembership()

* Test member event renewal

* Don't start the rtc manager until the client has synced

Then we can initialise from the state once it's completed.

* Fix type

* Remove listeners added in constructor

* Stop the client here too

* Stop the client here also also

* ARGH. Disable tests to work out which one is causing the exception

* Disable everything

* Re-jig to avoid setting listeners in the constructor

and re-enable tests

* No need to rename this anymore

* argh, remove the right listener

* Is it this test???

* Re-enable some tests

* Try mocking getRooms to return something valid

* Re-enable other tests

* Give up trying to get the tests to work sensibly and deal with getRooms() returning nothing

* Oops, don't enable the ones that were skipped before

* One more try at the sensible way

* Didn't work, go back to the hack way.

* Log when we manage to send the member event update

* Support `getOpenIdToken()` in embedded mode (#3676)

* Call `sendContentLoaded()` (#3677)

* Start MatrixRTC in embedded mode (#3679)

* Reschedule the membership event check

* Bump widget api version

* Add mock for sendContentLoaded()

* Embeded mode pre-requisites

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Embeded mode E2EE

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Encryption condition

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Embeded mode pre-requisites"

This reverts commit 8cd73702052609c995ad754e31f85d0da0be4aa9.

* Get back event type

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

fds

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Change embedded E2EE implementation

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* More log detail

* Fix tests

and also better assert because the tests were passing undefined which
was considered fine because we were only checking for null.

* Simplify updateCallMembershipEvent a bit

* Split up updateCallMembershipEvent some more

* Use `crypto.getRandomValues()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Rename to `membershipToUserAndDeviceId()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Better error

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add log line

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add comment

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Send call ID in enc events

(also a small refactor)

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert making `joinRoomSession()` async

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `client` `private` again

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Just use `toString()`

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix `callId` check

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix map compare

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix emitting

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Explicit logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Refactor

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` public

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Only update keys based on others

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix call order

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Improve logging

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Avoid races

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Revert "Avoid races"

This reverts commit f65ed72d6e.

* Add try-catch

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Make `updateEncryptionKeyEvent()` private

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Handle indices and throttling

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Fix merge mistakes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Mort post-merge fixes

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Split out key generation from key sending

And send all keys in a key event (changes the format of the key event)
rather than just the one we just generated.

* Remember and clear the timeout for the send key event

So we don't schedule more key updates if one is already pending.
Also don't update the last sent time when we didn't actually send the
keys.

* Make key event resends more robust

* Attempt to make tests pass

* crypto wasn't defined at all

* Hopefully get interface right

* Fix key format on the wire to base64

* Add comment

* More standard method order

* Rename encryptMedia

The js-sdk doesn't do media and therefore doesn't do media encryption

* Stop logging encryption keys now

* Use regular base64

It's not going in a URL, so no need

* Re-add base64url

randomstring was using it. Also give it a test.

* Add tests for randomstring

* Switch between either browser or node crypto

Let's see if this will work...

* Obviously crypto has already solved this

* Some tests for MatrixRTCSession key stuff

* Test keys object contents

* Change keys event format

To move away from m. keys

* Test key event retries

* Test onCallEncryption

* Test event sending & spam prevention

* Test event cancelation

* Test onCallEncryption called

* Some errors didn't have data

* Fix binary key comparison

& add log line

* Fix compare function with undefined values

* Remove more key logging

* Check content.keys is an array

* Check key index & key

* Better function name

* Tests too

---------

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
Co-authored-by: David Baker <dave@matrix.org>
Co-authored-by: David Baker <dbkr@users.noreply.github.com>
2023-10-31 16:01:46 +00:00
renovate[bot] 370dd6a0eb Update dependency eslint-plugin-unicorn to v49 (#3845)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-10-31 15:59:16 +00:00
renovate[bot] f760ece8b4 Update dependency @types/jest to v29.5.6 (#3841)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-10-31 15:57:27 +00:00
renovate[bot] 93e339affe Update definitelyTyped (#3840)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-10-31 15:57:11 +00:00
91 changed files with 7525 additions and 1887 deletions
@@ -0,0 +1,28 @@
name: Sign Release Tarball
description: Generates signature for release tarball and uploads it as a release asset
inputs:
gpg-fingerprint:
description: Fingerprint of the GPG key to use for signing the tarball.
required: true
upload-url:
description: GitHub release upload URL to upload the signature file to.
required: true
runs:
using: composite
steps:
- name: Generate tarball signature
shell: bash
run: |
git -c tar.tar.gz.command='gzip -cn' archive --format=tar.gz --prefix="${REPO#*/}-${VERSION#v}/" -o "/tmp/${VERSION}.tar.gz" "${VERSION}"
gpg -u "$GPG_FINGERPRINT" --armor --output "${VERSION}.tar.gz.asc" --detach-sig "/tmp/${VERSION}.tar.gz"
rm "/tmp/${VERSION}.tar.gz"
env:
GPG_FINGERPRINT: ${{ inputs.gpg-fingerprint }}
REPO: ${{ github.repository }}
- name: Upload tarball signature
if: ${{ inputs.upload-url }}
uses: shogo82148/actions-upload-release-asset@dccd6d23e64fd6a746dce6814c0bde0a04886085 # v1
with:
upload_url: ${{ inputs.upload-url }}
asset_path: ${{ env.VERSION }}.tar.gz.asc
@@ -0,0 +1,41 @@
name: Upload release assets
description: Uploads assets to an existing release and optionally signs them
inputs:
gpg-fingerprint:
description: Fingerprint of the GPG key to use for signing the assets, if any.
required: false
upload-url:
description: GitHub release upload URL to upload the assets to.
required: true
asset-path:
description: |
The path to the asset you want to upload, if any. You can use glob patterns here.
Will be GPG signed and an `.asc` file included in the release artifacts if `gpg-fingerprint` is set.
required: true
runs:
using: composite
steps:
- name: Sign assets
if: inputs.gpg-fingerprint
shell: bash
run: |
for FILE in $ASSET_PATH
do
gpg -u "$GPG_FINGERPRINT" --armor --output "$FILE".asc --detach-sig "$FILE"
done
env:
GPG_FINGERPRINT: ${{ inputs.gpg-fingerprint }}
ASSET_PATH: ${{ inputs.asset-path }}
- name: Upload asset signatures
if: inputs.gpg-fingerprint
uses: shogo82148/actions-upload-release-asset@dccd6d23e64fd6a746dce6814c0bde0a04886085 # v1
with:
upload_url: ${{ inputs.upload-url }}
asset_path: ${{ inputs.asset-path }}.asc
- name: Upload assets
uses: shogo82148/actions-upload-release-asset@dccd6d23e64fd6a746dce6814c0bde0a04886085 # v1
with:
upload_url: ${{ inputs.upload-url }}
asset_path: ${{ inputs.asset-path }}
+31
View File
@@ -0,0 +1,31 @@
name-template: "v$RESOLVED_VERSION"
tag-template: "v$RESOLVED_VERSION"
change-template: "* $TITLE ([#$NUMBER]($URL)). Contributed by @$AUTHOR."
categories:
- title: "🚨 BREAKING CHANGES"
label: "X-Breaking-Change"
- title: "🦖 Deprecations"
label: "T-Deprecation"
- title: "✨ Features"
label: "T-Enhancement"
- title: "🐛 Bug Fixes"
label: "T-Defect"
- title: "🧰 Maintenance"
label: "Dependencies"
collapse-after: 5
change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.
version-resolver:
major:
labels:
- "X-Breaking-Change"
default: minor
exclude-labels:
- "T-Task"
- "X-Reverted"
exclude-contributors:
- "RiotRobot"
template: |
$CHANGES
prerelease: true
prerelease-identifier: rc
include-pre-releases: false
+38 -7
View File
@@ -1,7 +1,7 @@
# Triggers after the "Downstream artifacts" build has finished, to run the
# cypress tests (with access to repo secrets)
# matrix-react-sdk playwright & cypress tests (with access to repo secrets)
name: matrix-react-sdk Cypress End to End Tests
name: matrix-react-sdk End to End Tests
on:
workflow_run:
workflows: ["Build downstream artifacts"]
@@ -17,10 +17,10 @@ jobs:
name: Cypress
# We only want to run the cypress tests on merge queue to prevent regressions
# from creeping in. They take a long time to run and consume 4 concurrent runners.
# from creeping in. They take a long time to run and consume multiple concurrent runners.
if: github.event.workflow_run.event == 'merge_group'
uses: matrix-org/matrix-react-sdk/.github/workflows/cypress.yaml@v3.83.0-rc.1
uses: matrix-org/matrix-react-sdk/.github/workflows/cypress.yaml@develop
permissions:
actions: read
issues: read
@@ -28,12 +28,27 @@ jobs:
pull-requests: read
secrets:
# secrets are not automatically shared with called workflows, so share the cypress dashboard key, and the Kiwi login details
CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS_RUST: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS_RUST}}
KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS_LEGACY: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS_LEGACY}}
TCMS_USERNAME: ${{ secrets.TCMS_USERNAME }}
TCMS_PASSWORD: ${{ secrets.TCMS_PASSWORD }}
with:
react-sdk-repository: matrix-org/matrix-react-sdk
rust-crypto: true
playwright:
name: Playwright
# We only want to run the playwright tests on merge queue to prevent regressions
# from creeping in. They take a long time to run and consume multiple concurrent runners.
if: github.event.workflow_run.event == 'merge_group'
uses: matrix-org/matrix-react-sdk/.github/workflows/end-to-end-tests.yaml@develop
permissions:
actions: read
issues: read
statuses: write
pull-requests: read
deployments: write
with:
react-sdk-repository: matrix-org/matrix-react-sdk
# We want to make the cypress tests a required check for the merge queue.
#
@@ -50,10 +65,26 @@ jobs:
statuses: write
runs-on: ubuntu-latest
steps:
- uses: Sibz/github-status-action@650dd1a882a76dbbbc4576fb5974b8d22f29847f # v1.1.6
- uses: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e # v1
with:
authToken: "${{ secrets.GITHUB_TOKEN }}"
state: success
description: Cypress skipped
# Keep in step with the `context` that is updated by `Sibz/github-status-action`
# in matrix-org/matrix-react-sdk/.github/workflows/cypress.yaml.
context: "${{ github.workflow }} / cypress"
sha: "${{ github.event.workflow_run.head_sha }}"
- uses: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e # v1
with:
authToken: "${{ secrets.GITHUB_TOKEN }}"
state: success
description: Playwright skipped
# Keep in step with the `context` that is updated by `Sibz/github-status-action`
# in matrix-org/matrix-react-sdk/.github/workflows/end-to-end-tests.yaml.
context: "${{ github.workflow }} / end-to-end-tests"
sha: "${{ github.event.workflow_run.head_sha }}"
+2 -1
View File
@@ -22,7 +22,7 @@ jobs:
path: docs
- name: 📤 Deploy to Netlify
uses: matrix-org/netlify-pr-preview@v2
uses: matrix-org/netlify-pr-preview@v3
with:
path: docs
owner: ${{ github.event.workflow_run.head_repository.owner.login }}
@@ -32,3 +32,4 @@ jobs:
site_id: ${{ secrets.NETLIFY_SITE_ID }}
desc: Documentation preview
deployment_env: PR Documentation Preview
environment: PR Documentation Preview
+1 -1
View File
@@ -20,7 +20,7 @@ concurrency:
jobs:
build-element-web:
name: Build element-web
uses: matrix-org/matrix-react-sdk/.github/workflows/element-web.yaml@v3.82.0
uses: matrix-org/matrix-react-sdk/.github/workflows/element-web.yaml@v3.85.0
with:
matrix-js-sdk-sha: ${{ github.sha }}
react-sdk-repository: matrix-org/matrix-react-sdk
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
pull-requests: read
steps:
- name: Add notice
uses: actions/github-script@v6
uses: actions/github-script@v7
if: contains(github.event.pull_request.labels.*.name, 'X-Blocked')
with:
script: |
@@ -49,7 +49,7 @@ jobs:
- name: Add label
if: ${{ steps.teams.outputs.isTeamMember == 'false' }}
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
script: |
github.rest.issues.addLabels({
@@ -68,7 +68,7 @@ jobs:
github.event.pull_request.head.repo.full_name != github.repository
steps:
- name: Close pull request
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
script: |
github.rest.issues.createComment({
+21
View File
@@ -0,0 +1,21 @@
name: Release Drafter
on:
push:
branches: [staging]
workflow_dispatch:
inputs:
previous-version:
description: What release to use as a base for release note purposes
required: false
type: string
concurrency: ${{ github.workflow }}
jobs:
draft:
runs-on: ubuntu-latest
steps:
- uses: release-drafter/release-drafter@e64b19c4c46173209ed9f2e5a2f4ca7de89a0e86 # v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
disable-autolabeler: true
previous-version: ${{ inputs.previous-version }}
+85
View File
@@ -0,0 +1,85 @@
# Gitflow merge-back master->develop
name: Merge master -> develop
on:
push:
branches: [master]
workflow_call:
secrets:
ELEMENT_BOT_TOKEN:
required: true
inputs:
dependencies:
description: List of dependencies to reset.
type: string
required: false
concurrency: ${{ github.workflow }}
jobs:
merge:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.ELEMENT_BOT_TOKEN }}
fetch-depth: 0
- name: Get actions scripts
uses: actions/checkout@v4
with:
repository: matrix-org/matrix-js-sdk
persist-credentials: false
path: .action-repo
sparse-checkout: |
scripts/release
- uses: actions/setup-node@v4
with:
cache: "yarn"
- name: Install Deps
run: "yarn install --frozen-lockfile"
- name: Set up git
run: |
git config --global user.email "releases@riot.im"
git config --global user.name "RiotRobot"
- name: Merge to develop
run: |
git checkout develop
git merge -X ours master
- name: Run post-merge-master script to revert package.json fields
run: ./.action-repo/scripts/release/post-merge-master.sh
- name: Reset dependencies
if: inputs.dependencies
run: |
while IFS= read -r PACKAGE; do
[ -z "$PACKAGE" ] && continue
CURRENT_VERSION=$(cat package.json | jq -r .dependencies[\"$PACKAGE\"])
echo "Current $PACKAGE version is $CURRENT_VERSION"
if [ "$CURRENT_VERSION" == "null" ]
then
echo "Unable to find $PACKAGE in package.json"
exit 1
fi
if [ "$CURRENT_VERSION" == "develop" ]
then
echo "Not updating dependency $PACKAGE"
continue
fi
echo "Resetting $1 to develop branch..."
yarn add "github:matrix-org/$PACKAGE#develop"
git add -u
git commit -m "Reset $PACKAGE back to develop branch"
done <<< "$DEPENDENCIES"
env:
DEPENDENCIES: ${{ inputs.dependencies }}
FINAL: ${{ inputs.final }}
- name: Push changes
run: git push origin develop
+353
View File
@@ -0,0 +1,353 @@
name: Release Make
on:
workflow_call:
secrets:
ELEMENT_BOT_TOKEN:
required: true
NPM_TOKEN:
required: false
GPG_PASSPHRASE:
required: false
GPG_PRIVATE_KEY:
required: false
inputs:
final:
description: Make final release
required: true
default: false
type: boolean
npm:
description: Publish to npm
type: boolean
default: false
dependencies:
description: |
List of dependencies to update in `npm-dep=version` format.
`version` can be `"current"` to leave it at the current version.
type: string
required: false
include-changes:
description: Project to include changelog entries from in this release.
type: string
required: false
gpg-fingerprint:
description: Fingerprint of the GPG key to use for signing the git tag and assets, if any.
type: string
required: false
asset-path:
description: |
The path to the asset you want to upload, if any. You can use glob patterns here.
Will be GPG signed and an `.asc` file included in the release artifacts if `gpg-fingerprint` is set.
type: string
required: false
expected-asset-count:
description: The number of expected assets, including signatures, excluding generated zip & tarball.
type: number
required: false
jobs:
release:
name: Release
runs-on: ubuntu-latest
environment: Release
steps:
- name: Load GPG key
id: gpg
if: inputs.gpg-fingerprint
uses: crazy-max/ghaction-import-gpg@82a020f1f7f605c65dd2449b392a52c3fcfef7ef # v6
with:
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.GPG_PASSPHRASE }}
fingerprint: ${{ inputs.gpg-fingerprint }}
- name: Get draft release
id: release
uses: cardinalby/git-get-release-action@cedef2faf69cb7c55b285bad07688d04430b7ada # v1
env:
GITHUB_TOKEN: ${{ github.token }}
with:
draft: true
latest: true
- uses: actions/checkout@v4
with:
ref: staging
token: ${{ secrets.ELEMENT_BOT_TOKEN }}
fetch-depth: 0
- name: Get actions scripts
uses: actions/checkout@v4
with:
repository: matrix-org/matrix-js-sdk
persist-credentials: false
path: .action-repo
sparse-checkout: |
.github/actions
scripts/release
- name: Prepare variables
id: prepare
run: |
echo "VERSION=$VERSION" >> $GITHUB_ENV
{
echo "RELEASE_NOTES<<EOF"
echo "$BODY"
echo "EOF"
} >> $GITHUB_ENV
HAS_DIST=0
jq -e .scripts.dist package.json >/dev/null 2>&1 && HAS_DIST=1
echo "has-dist-script=$HAS_DIST" >> $GITHUB_OUTPUT
env:
BODY: ${{ steps.release.outputs.body }}
VERSION: ${{ steps.release.outputs.tag_name }}
- name: Finalise version
if: inputs.final
run: echo "VERSION=$(echo $VERSION | cut -d- -f1)" >> $GITHUB_ENV
- name: Check version number not in use
uses: actions/github-script@v7
with:
script: |
const { VERSION } = process.env;
github.rest.repos.getReleaseByTag({
owner: context.repo.owner,
repo: context.repo.repo,
tag: VERSION,
}).then(() => {
core.setFailed(`Version ${VERSION} already exists`);
}).catch(() => {
// This is fine, we expect there to not be any release with this version yet
});
- name: Set up git
run: |
git config --global user.email "releases@riot.im"
git config --global user.name "RiotRobot"
- uses: actions/setup-node@v4
with:
cache: "yarn"
- name: Install dependencies
run: "yarn install --frozen-lockfile"
- name: Update dependencies
id: update-dependencies
if: inputs.dependencies
run: |
UPDATED=()
while IFS= read -r DEPENDENCY; do
[ -z "$DEPENDENCY" ] && continue
IFS="=" read -r PACKAGE UPDATE_VERSION <<< "$DEPENDENCY"
CURRENT_VERSION=$(cat package.json | jq -r .dependencies[\"$PACKAGE\"])
echo "Current $PACKAGE version is $CURRENT_VERSION"
if [ "$CURRENT_VERSION" == "null" ]
then
echo "Unable to find $PACKAGE in package.json"
exit 1
fi
if [ "$UPDATE_VERSION" == "current" ] || [ "$UPDATE_VERSION" == "$CURRENT_VERSION" ]
then
echo "Not updating dependency $PACKAGE"
continue
fi
echo "Upgrading $PACKAGE to $UPDATE_VERSION..."
yarn upgrade "$PACKAGE@$UPDATE_VERSION" --exact
git add -u
git commit -m "Upgrade $PACKAGE to $UPDATE_VERSION"
UPDATED+=("$PACKAGE")
done <<< "$DEPENDENCIES"
JSON=$(jq --compact-output --null-input '$ARGS.positional' --args -- "${UPDATED[@]}")
echo "updated=$JSON" >> $GITHUB_OUTPUT
env:
DEPENDENCIES: ${{ inputs.dependencies }}
- name: Prevent develop dependencies
if: inputs.dependencies
run: |
ret=0
cat package.json | jq '.dependencies[]' | grep -q '#develop' || ret=$?
if [ "$ret" -eq 0 ]; then
echo "package.json contains develop dependencies. Refusing to release."
exit
fi
- name: Bump package.json version
run: yarn version --no-git-tag-version --new-version "${VERSION#v}"
- name: Ingest upstream changes
if: |
inputs.include-changes &&
(!inputs.dependencies || contains(fromJSON(steps.update-dependencies.outputs.updated), inputs.include-changes))
uses: actions/github-script@v7
env:
RELEASE_ID: ${{ steps.release.outputs.id }}
DEPENDENCY: ${{ inputs.include-changes }}
with:
retries: 3
script: |
const { RELEASE_ID: releaseId, DEPENDENCY, VERSION } = process.env;
const { owner, repo } = context.repo;
const script = require("./.action-repo/scripts/release/merge-release-notes.js");
const notes = await script({
github,
releaseId,
dependencies: [DEPENDENCY.replace("$VERSION", VERSION)],
});
core.exportVariable("RELEASE_NOTES", notes);
- name: Add to CHANGELOG.md
if: inputs.final
run: |
mv CHANGELOG.md CHANGELOG.md.old
HEADER="Changes in [${VERSION#v}](https://github.com/${{ github.repository }}/releases/tag/$VERSION) ($(date '+%Y-%m-%d'))"
{
echo "$HEADER"
printf '=%.0s' $(seq ${#HEADER})
echo ""
echo "$RELEASE_NOTES"
echo ""
} > CHANGELOG.md
cat CHANGELOG.md.old >> CHANGELOG.md
rm CHANGELOG.md.old
git add CHANGELOG.md
- name: Run pre-release script to update package.json fields
run: |
./.action-repo/scripts/release/pre-release.sh
git add package.json
- name: Commit changes
run: git commit -m "$VERSION"
- name: Build assets
if: steps.prepare.outputs.has-dist-script == '1'
run: DIST_VERSION="$VERSION" yarn dist
- name: Upload release assets & signatures
if: inputs.asset-path
uses: ./.action-repo/.github/actions/upload-release-assets
with:
gpg-fingerprint: ${{ inputs.gpg-fingerprint }}
upload-url: ${{ steps.release.outputs.upload_url }}
asset-path: ${{ inputs.asset-path }}
- name: Create signed tag
if: inputs.gpg-fingerprint
run: |
GIT_COMMITTER_EMAIL="$SIGNING_ID" GPG_TTY=$(tty) git tag -u "$SIGNING_ID" -m "Release $VERSION" "$VERSION"
env:
SIGNING_ID: ${{ steps.gpg.outputs.email }}
- name: Generate & upload tarball signature
if: inputs.gpg-fingerprint
uses: ./.action-repo/.github/actions/sign-release-tarball
with:
gpg-fingerprint: ${{ inputs.gpg-fingerprint }}
upload-url: ${{ steps.release.outputs.upload_url }}
# We defer pushing changes until after the release assets are built,
# signed & uploaded to improve the atomicity of this action.
- name: Push changes to staging
run: |
git push origin staging $TAG
git reset --hard
env:
TAG: ${{ inputs.gpg-fingerprint && env.VERSION || '' }}
- name: Validate tarball signature
if: inputs.gpg-fingerprint
run: |
wget https://github.com/$GITHUB_REPOSITORY/archive/refs/tags/$VERSION.tar.gz
gpg --verify "$VERSION.tar.gz.asc" "$VERSION.tar.gz"
- name: Validate release has expected assets
if: inputs.expected-asset-count
uses: actions/github-script@v7
env:
RELEASE_ID: ${{ steps.release.outputs.id }}
EXPECTED_ASSET_COUNT: ${{ inputs.expected-asset-count }}
with:
retries: 3
script: |
const { RELEASE_ID: release_id, EXPECTED_ASSET_COUNT } = process.env;
const { owner, repo } = context.repo;
const { data: release } = await github.rest.repos.getRelease({
owner,
repo,
release_id,
});
if (release.assets.length !== parseInt(EXPECTED_ASSET_COUNT, 10)) {
core.setFailed(`Found ${release.assets.length} assets but expected ${EXPECTED_ASSET_COUNT}`);
}
- name: Merge to master
if: inputs.final
run: |
git checkout master
git merge -X theirs staging
git push origin master
- name: Publish release
uses: actions/github-script@v7
env:
RELEASE_ID: ${{ steps.release.outputs.id }}
FINAL: ${{ inputs.final }}
with:
retries: 3
github-token: ${{ secrets.ELEMENT_BOT_TOKEN }}
script: |
const { RELEASE_ID: release_id, RELEASE_NOTES, VERSION, FINAL } = process.env;
const { owner, repo } = context.repo;
const opts = {
owner,
repo,
release_id,
tag_name: VERSION,
name: VERSION,
draft: false,
body: RELEASE_NOTES,
};
if (FINAL == "true") {
opts.prerelease = false;
opts.make_latest = true;
}
github.rest.repos.updateRelease(opts);
npm:
name: Publish to npm
needs: release
if: inputs.npm
uses: matrix-org/matrix-js-sdk/.github/workflows/release-npm.yml@develop
secrets:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
update-labels:
name: Advance release blocker labels
needs: release
runs-on: ubuntu-latest
steps:
- id: repository
run: echo "REPO=${GITHUB_REPOSITORY#*/}" >> $GITHUB_OUTPUT
- uses: garganshu/github-label-updater@3770d15ebfed2fe2cb06a241047bc340f774a7d1 # v1.0.0
with:
owner: ${{ github.repository_owner }}
repo: ${{ steps.repository.outputs.REPO }}
token: ${{ secrets.GITHUB_TOKEN }}
filter-labels: X-Upcoming-Release-Blocker
remove-labels: X-Upcoming-Release-Blocker
add-labels: X-Release-Blocker
+5 -4
View File
@@ -1,4 +1,3 @@
# Must only be called from `release#published` triggers
name: Publish to npm
on:
workflow_call:
@@ -12,9 +11,11 @@ jobs:
steps:
- name: 🧮 Checkout code
uses: actions/checkout@v4
with:
ref: staging
- name: 🔧 Yarn cache
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
cache: "yarn"
registry-url: "https://registry.npmjs.org"
@@ -24,7 +25,7 @@ jobs:
- name: 🚀 Publish to npm
id: npm-publish
uses: JS-DevTools/npm-publish@fe72237be0920f7a0cafd6a966c9b929c9466e9b # v2.2.2
uses: JS-DevTools/npm-publish@4b07b26a2f6e0a51846e1870223e545bae91c552 # v3.0.1
with:
token: ${{ secrets.NPM_TOKEN }}
access: public
@@ -32,7 +33,7 @@ jobs:
ignore-scripts: false
- name: 🎖️ Add `latest` dist-tag to final releases
if: github.event.release.prerelease == false && steps.npm-publish.outputs.id
if: steps.npm-publish.outputs.id && !contains(steps.npm-publish.outputs.id, '-rc.')
run: npm dist-tag add "$release" latest
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+36 -12
View File
@@ -1,11 +1,38 @@
name: Release Process
on:
release:
types: [published]
concurrency: ${{ github.workflow }}-${{ github.ref }}
workflow_dispatch:
inputs:
mode:
description: What type of release
required: true
default: rc
type: choice
options:
- rc
- final
docs:
description: Publish docs
required: true
type: boolean
default: true
npm:
description: Publish to npm
required: true
type: boolean
default: true
concurrency: ${{ github.workflow }}
jobs:
jsdoc:
release:
uses: matrix-org/matrix-js-sdk/.github/workflows/release-make.yml@develop
secrets: inherit
with:
final: ${{ inputs.mode == 'final' }}
npm: ${{ inputs.npm }}
docs:
name: Publish Documentation
needs: release
if: inputs.docs
runs-on: ubuntu-latest
steps:
- name: 🧮 Checkout code
@@ -18,7 +45,7 @@ jobs:
path: _docs
- name: 🔧 Yarn cache
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
cache: "yarn"
@@ -36,17 +63,14 @@ jobs:
yarn gendoc
symlinks -rc _docs
- name: 🚀 Deploy
- name: 🔨 Set up git
run: |
git config --global user.email "releases@riot.im"
git config --global user.name "RiotRobot"
- name: 🚀 Deploy
run: |
git add . --all
git commit -m "Update docs"
git push
working-directory: _docs
npm:
name: Publish
uses: matrix-org/matrix-js-sdk/.github/workflows/release-npm.yml@develop
secrets:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
+3 -3
View File
@@ -17,7 +17,7 @@ jobs:
steps:
# We create the status here and then update it to success/failure in the `report` stage
# This provides an easy link to this workflow_run from the PR before Cypress is done.
- uses: Sibz/github-status-action@faaa4d96fecf273bd762985e0e7f9f933c774918 # v1
- uses: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e # v1
with:
authToken: ${{ secrets.GITHUB_TOKEN }}
state: pending
@@ -27,7 +27,7 @@ jobs:
- name: "🩻 SonarCloud Scan"
id: sonarcloud
uses: matrix-org/sonarcloud-workflow-action@v2.6
uses: matrix-org/sonarcloud-workflow-action@v2.7
# workflow_run fails report against the develop commit always, we don't want that for PRs
continue-on-error: ${{ github.event.workflow_run.head_branch != 'develop' }}
with:
@@ -42,7 +42,7 @@ jobs:
coverage_extract_path: coverage
extra_args: ${{ inputs.extra_args }}
- uses: Sibz/github-status-action@faaa4d96fecf273bd762985e0e7f9f933c774918 # v1
- uses: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e # v1
if: always()
with:
authToken: ${{ secrets.GITHUB_TOKEN }}
+19 -3
View File
@@ -15,7 +15,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
cache: "yarn"
@@ -41,7 +41,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
cache: "yarn"
@@ -51,13 +51,29 @@ jobs:
- name: Run Linter
run: "yarn run lint:js"
workflow_lint:
name: "Workflow Lint"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
cache: "yarn"
- name: Install Deps
run: "yarn install --frozen-lockfile"
- name: Run Linter
run: "yarn lint:workflows"
docs:
name: "JSDoc Checker"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3
- uses: actions/setup-node@v4
with:
cache: "yarn"
+3 -3
View File
@@ -18,14 +18,14 @@ jobs:
strategy:
matrix:
specs: [integ, unit]
node: [18, "*"]
node: [18, "lts/*", 21]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node
id: setupNode
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
cache: "yarn"
node-version: ${{ matrix.node }}
@@ -82,7 +82,7 @@ jobs:
steps:
- name: Skip SonarCloud on merge queues
if: env.ENABLE_COVERAGE == 'false'
uses: Sibz/github-status-action@faaa4d96fecf273bd762985e0e7f9f933c774918 # v1
uses: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e # v1
with:
authToken: ${{ secrets.GITHUB_TOKEN }}
state: success
+11
View File
@@ -0,0 +1,11 @@
name: Move labelled issues to correct projects
on:
issues:
types: [labeled]
jobs:
call-triage-labelled:
uses: vector-im/element-web/.github/workflows/triage-labelled.yml@develop
secrets:
ELEMENT_BOT_TOKEN: ${{ secrets.ELEMENT_BOT_TOKEN }}
+47
View File
@@ -1,3 +1,50 @@
Changes in [30.3.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v30.3.0) (2023-12-19)
==================================================================================================
## ✨ Features
* Element-R: disable sending room key requests ([#3939](https://github.com/matrix-org/matrix-js-sdk/pull/3939)). Contributed by @richvdh.
## 🐛 Bug Fixes
* Fix notifications appearing for old events ([#3946](https://github.com/matrix-org/matrix-js-sdk/pull/3946)). Contributed by @dbkr.
* Don't back up keys that we got from backup ([#3934](https://github.com/matrix-org/matrix-js-sdk/pull/3934)). Contributed by @uhoreg.
* Fix upload with empty Content-Type ([#3918](https://github.com/matrix-org/matrix-js-sdk/pull/3918)). Contributed by @JakubOnderka.
* Prevent phantom notifications from events not in a room's timeline ([#3942](https://github.com/matrix-org/matrix-js-sdk/pull/3942)). Contributed by @dbkr.
Changes in [30.2.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v30.2.0) (2023-12-05)
==================================================================================================
## ✨ Features
* Only await key query after lazy members resolved ([#3902](https://github.com/matrix-org/matrix-js-sdk/pull/3902)). Contributed by @BillCarsonFr.
## 🐛 Bug Fixes
* Rewrite receipt-handling code ([#3901](https://github.com/matrix-org/matrix-js-sdk/pull/3901)). Contributed by @andybalaam.
* Explicitly free some Rust-side objects ([#3911](https://github.com/matrix-org/matrix-js-sdk/pull/3911)). Contributed by @richvdh.
* Fix type for TimestampToEventResponse.origin\_server\_ts ([#3906](https://github.com/matrix-org/matrix-js-sdk/pull/3906)). Contributed by @Half-Shot.
Changes in [30.1.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v30.1.0) (2023-11-21)
==================================================================================================
## ✨ Features
* Rotate per-participant keys when a member leaves ([#3833](https://github.com/matrix-org/matrix-js-sdk/pull/3833)). Contributed by @dbkr.
* Add E2EE for embedded mode of Element Call ([#3667](https://github.com/matrix-org/matrix-js-sdk/pull/3667)). Contributed by @SimonBrandner.
## 🐛 Bug Fixes
* Shorten TimelineWindow when an event is removed ([#3862](https://github.com/matrix-org/matrix-js-sdk/pull/3862)). Contributed by @andybalaam.
* Ignore receipts pointing at missing or invalid events ([#3817](https://github.com/matrix-org/matrix-js-sdk/pull/3817)). Contributed by @andybalaam.
* Fix members being loaded from server on initial sync (defeating lazy loading) ([#3830](https://github.com/matrix-org/matrix-js-sdk/pull/3830)). Contributed by @BillCarsonFr.
Changes in [30.0.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v30.0.1) (2023-11-13)
==================================================================================================
## 🐛 Bug Fixes
* Ensure `setUserCreator` is called when a store is assigned ([\#3867](https://github.com/matrix-org/matrix-js-sdk/pull/3867)). Fixes vector-im/element-web#26520. Contributed by @MidhunSureshR.
Changes in [30.0.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v30.0.0) (2023-11-07)
==================================================================================================
+1
View File
@@ -4,5 +4,6 @@
# Deep dive
- [Release Process](release.md)
- [Storage notes](storage-notes.md)
- [Unverified devices](warning-on-unverified-devices.md)
+24
View File
@@ -0,0 +1,24 @@
# Release Process
## Hotfix and off-cycle releases
1. Prepare the `staging` branch by using the backport automation and manually merging
2. Go to [Releasing](#Releasing)
## Release candidates
1. Prepare the `staging` branch by running the [branch cut automation](https://github.com/vector-im/element-web/actions/workflows/release_prepare.yml)
2. Go to [Releasing](#Releasing)
## Releasing
1. Open the [Releases page](https://github.com/matrix-org/matrix-js-sdk/releases) and inspect the draft release there
2. Make any modifications to the release notes and tag/version as required
3. Run [workflow](https://github.com/matrix-org/matrix-js-sdk/actions/workflows/release.yml) with the type set appropriately
## Artifacts
Releasing the Matrix JS SDK has just two artifacts:
- Package published to [npm](https://github.com/matrix-org/matrix-js-sdk)
- Docs published to [Github Pages](https://matrix-org.github.io/matrix-js-sdk/)
+9 -9
View File
@@ -1,6 +1,6 @@
{
"name": "matrix-js-sdk",
"version": "30.0.0",
"version": "30.3.0",
"description": "Matrix Client-Server SDK for Javascript",
"engines": {
"node": ">=18.0.0"
@@ -14,10 +14,11 @@
"build:types": "tsc -p tsconfig-build.json --emitDeclarationOnly",
"build:compile": "babel -d lib --verbose --extensions \".ts,.js\" src",
"gendoc": "typedoc",
"lint": "yarn lint:types && yarn lint:js",
"lint": "yarn lint:types && yarn lint:js && yarn lint:workflows",
"lint:js": "eslint --max-warnings 0 src spec && prettier --check .",
"lint:js-fix": "prettier --loglevel=warn --write . && eslint --fix src spec",
"lint:types": "tsc --noEmit",
"lint:workflows": "find .github/workflows -type f \\( -iname '*.yaml' -o -iname '*.yml' \\) | xargs -I {} sh -c 'echo \"Linting {}\"; action-validator \"{}\"'",
"test": "jest",
"test:watch": "jest --watch",
"coverage": "yarn test --coverage"
@@ -51,7 +52,7 @@
],
"dependencies": {
"@babel/runtime": "^7.12.5",
"@matrix-org/matrix-sdk-crypto-wasm": "^2.2.0",
"@matrix-org/matrix-sdk-crypto-wasm": "^3.4.0",
"another-json": "^0.2.0",
"bs58": "^5.0.0",
"content-type": "^1.0.4",
@@ -66,6 +67,8 @@
"uuid": "9"
},
"devDependencies": {
"@action-validator/cli": "^0.5.3",
"@action-validator/core": "^0.5.3",
"@babel/cli": "^7.12.10",
"@babel/core": "^7.12.10",
"@babel/eslint-parser": "^7.12.10",
@@ -92,10 +95,9 @@
"@typescript-eslint/parser": "^5.45.0",
"allchange": "^1.0.6",
"babel-jest": "^29.0.0",
"babelify": "^10.0.0",
"debug": "^4.3.4",
"domexception": "^4.0.0",
"eslint": "8.51.0",
"eslint": "8.54.0",
"eslint-config-google": "^0.14.0",
"eslint-config-prettier": "^9.0.0",
"eslint-import-resolver-typescript": "^3.5.1",
@@ -104,9 +106,9 @@
"eslint-plugin-jsdoc": "^46.0.0",
"eslint-plugin-matrix-org": "^1.0.0",
"eslint-plugin-tsdoc": "^0.2.17",
"eslint-plugin-unicorn": "^48.0.0",
"exorcist": "^2.0.0",
"eslint-plugin-unicorn": "^49.0.0",
"fake-indexeddb": "^5.0.0",
"fetch-mock": "9.11.0",
"fetch-mock-jest": "^1.5.1",
"husky": "^8.0.3",
"jest": "^29.0.0",
@@ -117,9 +119,7 @@
"matrix-mock-request": "^2.5.0",
"prettier": "2.8.8",
"rimraf": "^5.0.0",
"terser": "^5.5.1",
"ts-node": "^10.9.1",
"tsify": "^5.0.2",
"typedoc": "^0.24.0",
"typedoc-plugin-coverage": "^2.1.0",
"typedoc-plugin-mdn-links": "^3.0.3",
+1 -23
View File
@@ -10,28 +10,6 @@ set -e
jq --version > /dev/null || (echo "jq is required: please install it"; kill $$)
if [ "$(git branch -lr | grep origin/develop -c)" -ge 1 ]; then
# When merging to develop, we need revert the `main` and `typings` fields if we adjusted them previously.
for i in main typings browser
do
# If a `lib` prefixed value is present, it means we adjusted the field
# earlier at publish time, so we should revert it now.
if [ "$(jq -r ".matrix_lib_$i" package.json)" != "null" ]; then
# If there's a `src` prefixed value, use that, otherwise delete.
# This is used to delete the `typings` field and reset `main` back
# to the TypeScript source.
src_value=$(jq -r ".matrix_src_$i" package.json)
if [ "$src_value" != "null" ]; then
jq ".$i = .matrix_src_$i" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
else
jq "del(.$i)" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
fi
fi
done
if [ -n "$(git ls-files --modified package.json)" ]; then
echo "Committing develop package.json"
git commit package.json -m "Resetting package fields for development"
fi
"$(dirname "$0")/scripts/release/post-merge-master.sh"
git push origin develop
fi
+1 -12
View File
@@ -175,18 +175,7 @@ echo "yarn version"
# manually commit the result.
yarn version --no-git-tag-version --new-version "$release"
# For the published and dist versions of the package, we copy the
# `matrix_lib_main` and `matrix_lib_typings` fields to `main` and `typings` (if
# they exist). This small bit of gymnastics allows us to use the TypeScript
# source directly for development without needing to build before linting or
# testing.
for i in main typings browser
do
lib_value=$(jq -r ".matrix_lib_$i" package.json)
if [ "$lib_value" != "null" ]; then
jq ".$i = .matrix_lib_$i" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
fi
done
"$(dirname "$0")/scripts/release/pre-release.sh"
# commit yarn.lock if it exists, is versioned, and is modified
if [[ -f yarn.lock && $(git status --porcelain yarn.lock | grep '^ M') ]];
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env node
const fs = require("fs");
async function getRelease(github, dependency) {
let owner;
let repo;
let tag;
if (dependency.includes("/") && dependency.includes("@")) {
owner = dependency.split("/")[0];
repo = dependency.split("/")[1].split("@")[0];
tag = dependency.split("@")[1];
} else {
const upstreamPackageJson = JSON.parse(fs.readFileSync(`./node_modules/${dependency}/package.json`, "utf8"));
[owner, repo] = upstreamPackageJson.repository.url.split("/").slice(-2);
tag = `v${upstreamPackageJson.version}`;
}
const response = await github.rest.repos.getReleaseByTag({
owner,
repo,
tag,
});
return response.data;
}
const HEADING_PREFIX = "## ";
const main = async ({ github, releaseId, dependencies }) => {
const { GITHUB_REPOSITORY } = process.env;
const [owner, repo] = GITHUB_REPOSITORY.split("/");
const sections = new Map();
let heading = null;
for (const dependency of dependencies) {
const release = await getRelease(github, dependency);
for (const line of release.body.split("\n")) {
if (line.startsWith(HEADING_PREFIX)) {
heading = line.trim();
sections.set(heading, []);
continue;
}
if (heading && line) {
sections.get(heading).push(line.trim());
}
}
}
const { data: release } = await github.rest.repos.getRelease({
owner,
repo,
release_id: releaseId,
});
const headings = ["🚨 BREAKING CHANGES", "🦖 Deprecations", "✨ Features", "🐛 Bug Fixes", "🧰 Maintenance"].map(
(h) => HEADING_PREFIX + h,
);
heading = null;
const output = [];
for (const line of [...release.body.split("\n"), null]) {
if (line === null || line.startsWith(HEADING_PREFIX)) {
// If we have a heading, and it's not the first in the list of pending headings, output the section.
// If we're processing the last line (null) then output all remaining sections.
while (headings.length > 0 && (line === null || (heading && headings[0] !== heading))) {
const heading = headings.shift();
if (sections.has(heading)) {
output.push(heading);
output.push(...sections.get(heading));
}
}
if (heading && sections.has(heading)) {
const lastIsBlank = !output.at(-1)?.trim();
if (lastIsBlank) output.pop();
output.push(...sections.get(heading));
if (lastIsBlank) output.push("");
}
heading = line;
}
output.push(line);
}
return output.join("\n");
};
// This is just for testing locally
// Needs environment variables GITHUB_TOKEN & GITHUB_REPOSITORY
if (require.main === module) {
const { Octokit } = require("@octokit/rest");
const github = new Octokit({ auth: process.env.GITHUB_TOKEN });
if (process.argv.length < 4) {
// eslint-disable-next-line no-console
console.error("Usage: node merge-release-notes.js owner/repo:release_id npm-package-name ...");
process.exit(1);
}
const [releaseId, ...dependencies] = process.argv.slice(2);
main({ github, releaseId, dependencies }).then((output) => {
// eslint-disable-next-line no-console
console.log(output);
});
}
module.exports = main;
+22
View File
@@ -0,0 +1,22 @@
#!/bin/bash
# When merging to develop, we need revert the `main` and `typings` fields if we adjusted them previously.
for i in main typings browser
do
# If a `lib` prefixed value is present, it means we adjusted the field earlier at publish time, so we should revert it now.
if [ "$(jq -r ".matrix_lib_$i" package.json)" != "null" ]; then
# If there's a `src` prefixed value, use that, otherwise delete.
# This is used to delete the `typings` field and reset `main` back to the TypeScript source.
src_value=$(jq -r ".matrix_src_$i" package.json)
if [ "$src_value" != "null" ]; then
jq ".$i = .matrix_src_$i" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
else
jq "del(.$i)" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
fi
fi
done
if [ -n "$(git ls-files --modified package.json)" ]; then
echo "Committing develop package.json"
git commit package.json -m "Resetting package fields for development"
fi
+14
View File
@@ -0,0 +1,14 @@
#!/bin/bash
# For the published and dist versions of the package,
# we copy the `matrix_lib_main` and `matrix_lib_typings` fields to `main` and `typings` (if they exist).
# This small bit of gymnastics allows us to use the TypeScript source directly for development without
# needing to build before linting or testing.
for i in main typings browser
do
lib_value=$(jq -r ".matrix_lib_$i" package.json)
if [ "$lib_value" != "null" ]; then
jq ".$i = .matrix_lib_$i" package.json > package.json.new && mv package.json.new package.json && yarn prettier --write package.json
fi
done
+100
View File
@@ -31,9 +31,12 @@ import {
SELF_CROSS_SIGNING_PRIVATE_KEY_BASE64,
SELF_CROSS_SIGNING_PUBLIC_KEY_BASE64,
SIGNED_CROSS_SIGNING_KEYS_DATA,
SIGNED_TEST_DEVICE_DATA,
USER_CROSS_SIGNING_PRIVATE_KEY_BASE64,
} from "../../test-utils/test-data";
import * as testData from "../../test-utils/test-data";
import { E2EKeyResponder } from "../../test-utils/E2EKeyResponder";
import { AccountDataAccumulator } from "../../test-utils/AccountDataAccumulator";
afterEach(() => {
// reset fake-indexeddb after each test, to make sure we don't leak connections
@@ -97,6 +100,12 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("cross-signing (%s)", (backend: s
/** an object which intercepts `/keys/upload` requests on the test homeserver */
new E2EKeyReceiver(homeserverUrl);
// Silence warnings from the backup manager
fetchMock.getOnce(new URL("/_matrix/client/v3/room_keys/version", homeserverUrl).toString(), {
status: 404,
body: { errcode: "M_NOT_FOUND" },
});
await initCrypto(aliceClient);
});
@@ -236,6 +245,53 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("cross-signing (%s)", (backend: s
`[${TEST_USER_ID}].[${TEST_DEVICE_ID}].signatures.[${TEST_USER_ID}].[ed25519:${SELF_CROSS_SIGNING_PUBLIC_KEY_BASE64}]`,
);
});
it("can bootstrapCrossSigning twice", async () => {
mockSetupCrossSigningRequests();
const authDict = { type: "test" };
await bootstrapCrossSigning(authDict);
// a second call should do nothing except GET requests
fetchMock.mockClear();
await bootstrapCrossSigning(authDict);
const calls = fetchMock.calls((url, opts) => opts.method != "GET");
expect(calls.length).toEqual(0);
});
newBackendOnly("will upload existing cross-signing keys to an established secret storage", async () => {
// This rather obscure codepath covers the case that:
// - 4S is set up and working
// - our device has private cross-signing keys, but has not published them to 4S
//
// To arrange that, we call `bootstrapCrossSigning` on our main device, and then (pretend to) set up 4S from
// a *different* device. Then, when we call `bootstrapCrossSigning` again, it should do the honours.
mockSetupCrossSigningRequests();
const accountDataAccumulator = new AccountDataAccumulator();
accountDataAccumulator.interceptGetAccountData();
const authDict = { type: "test" };
await bootstrapCrossSigning(authDict);
// Pretend that another device has uploaded a 4S key
accountDataAccumulator.accountDataEvents.set("m.secret_storage.default_key", { key: "key_id" });
accountDataAccumulator.accountDataEvents.set("m.secret_storage.key.key_id", {
key: "keykeykey",
algorithm: SECRET_STORAGE_ALGORITHM_V1_AES,
});
// Prepare for the cross-signing keys
const p = accountDataAccumulator.interceptSetAccountData(":type(m.cross_signing..*)");
await bootstrapCrossSigning(authDict);
await p;
// The cross-signing keys should have been uploaded
expect(accountDataAccumulator.accountDataEvents.has("m.cross_signing.master")).toBeTruthy();
expect(accountDataAccumulator.accountDataEvents.has("m.cross_signing.self_signing")).toBeTruthy();
expect(accountDataAccumulator.accountDataEvents.has("m.cross_signing.user_signing")).toBeTruthy();
});
});
describe("getCrossSigningStatus()", () => {
@@ -339,4 +395,48 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("cross-signing (%s)", (backend: s
expect(userSigningKeyId).toBe(getPubKey(crossSigningKeys.user_signing_key));
});
});
describe("crossSignDevice", () => {
beforeEach(async () => {
jest.useFakeTimers();
// make sure that there is another device which we can sign
e2eKeyResponder.addDeviceKeys(SIGNED_TEST_DEVICE_DATA);
// Complete initialsync, to get the outgoing requests going
mockInitialApiRequests(aliceClient.getHomeserverUrl());
syncResponder.sendOrQueueSyncResponse({ next_batch: 1 });
await aliceClient.startClient();
await syncPromise(aliceClient);
// Wait for legacy crypto to find the device
await jest.advanceTimersByTimeAsync(10);
const devices = await aliceClient.getCrypto()!.getUserDeviceInfo([aliceClient.getSafeUserId()]);
expect(devices.get(aliceClient.getSafeUserId())!.has(testData.TEST_DEVICE_ID)).toBeTruthy();
});
afterEach(async () => {
jest.useRealTimers();
});
it("fails for an unknown device", async () => {
await expect(aliceClient.getCrypto()!.crossSignDevice("unknown")).rejects.toThrow("Unknown device");
});
it("cross-signs the device", async () => {
mockSetupCrossSigningRequests();
await aliceClient.getCrypto()!.bootstrapCrossSigning({});
fetchMock.mockClear();
await aliceClient.getCrypto()!.crossSignDevice(testData.TEST_DEVICE_ID);
// check that a sig for the device was uploaded
const calls = fetchMock.calls("upload-sigs");
expect(calls.length).toEqual(1);
const body = JSON.parse(calls[0][1]!.body as string);
const deviceSig = body[aliceClient.getSafeUserId()][testData.TEST_DEVICE_ID];
expect(deviceSig).toHaveProperty("signatures");
});
});
});
+93 -122
View File
@@ -96,6 +96,7 @@ import {
getTestOlmAccountKeys,
} from "./olm-utils";
import { ToDevicePayload } from "../../../src/models/ToDeviceMessage";
import { AccountDataAccumulator } from "../../test-utils/AccountDataAccumulator";
afterEach(() => {
// reset fake-indexeddb after each test, to make sure we don't leak connections
@@ -397,6 +398,19 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
expect(aliceClient.getCrypto()).toHaveProperty("globalBlacklistUnverifiedDevices");
});
it("CryptoAPI.getOwnedDeviceKeys returns the correct values", async () => {
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
await startClientAndAwaitFirstSync();
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const deviceKeys = await aliceClient.getCrypto()!.getOwnDeviceKeys();
expect(deviceKeys.curve25519).toEqual(keyReceiver.getDeviceKey());
expect(deviceKeys.ed25519).toEqual(keyReceiver.getSigningKey());
});
it("Alice receives a megolm message", async () => {
expectAliceKeyQuery({ device_keys: { "@alice:localhost": {} }, failures: {} });
await startClientAndAwaitFirstSync();
@@ -692,7 +706,13 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
});
it("prepareToEncrypt", async () => {
expectAliceKeyQuery({ device_keys: { "@alice:localhost": {} }, failures: {} });
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const testDeviceKeys = getTestOlmAccountKeys(testOlmAccount, "@bob:xyz", "DEVICE_ID");
keyResponder.addDeviceKeys(testDeviceKeys);
await startClientAndAwaitFirstSync();
aliceClient.setGlobalErrorOnUnknownDevices(false);
@@ -700,10 +720,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
syncResponder.sendOrQueueSyncResponse(getSyncResponse(["@bob:xyz"]));
await syncPromise(aliceClient);
// we expect alice first to query bob's keys...
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
// ... and then claim one of his OTKs
// Alice should claim one of Bob's OTKs
expectAliceKeyClaim(getTestKeysClaimResponse("@bob:xyz"));
// fire off the prepare request
@@ -720,18 +737,20 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
it("Alice sends a megolm message with GlobalErrorOnUnknownDevices=false", async () => {
aliceClient.setGlobalErrorOnUnknownDevices(false);
expectAliceKeyQuery({ device_keys: { "@alice:localhost": {} }, failures: {} });
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const testDeviceKeys = getTestOlmAccountKeys(testOlmAccount, "@bob:xyz", "DEVICE_ID");
keyResponder.addDeviceKeys(testDeviceKeys);
await startClientAndAwaitFirstSync();
// Alice shares a room with Bob
syncResponder.sendOrQueueSyncResponse(getSyncResponse(["@bob:xyz"]));
await syncPromise(aliceClient);
// Once we send the message, Alice will check Bob's device list (twice, because reasons) ...
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
// ... and claim one of his OTKs ...
// ... and claim one of Bob's OTKs ...
expectAliceKeyClaim(getTestKeysClaimResponse("@bob:xyz"));
// ... and send an m.room_key message
@@ -746,18 +765,20 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
it("We should start a new megolm session after forceDiscardSession", async () => {
aliceClient.setGlobalErrorOnUnknownDevices(false);
expectAliceKeyQuery({ device_keys: { "@alice:localhost": {} }, failures: {} });
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const testDeviceKeys = getTestOlmAccountKeys(testOlmAccount, "@bob:xyz", "DEVICE_ID");
keyResponder.addDeviceKeys(testDeviceKeys);
await startClientAndAwaitFirstSync();
// Alice shares a room with Bob
syncResponder.sendOrQueueSyncResponse(getSyncResponse(["@bob:xyz"]));
await syncPromise(aliceClient);
// Once we send the message, Alice will check Bob's device list (twice, because reasons) ...
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
// ... and claim one of his OTKs ...
// ... and claim one of Bob's OTKs ...
expectAliceKeyClaim(getTestKeysClaimResponse("@bob:xyz"));
// ... and send an m.room_key message
@@ -2052,13 +2073,17 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
});
}
oldBackendOnly("Sending an event initiates a member list sync", async () => {
it("Sending an event initiates a member list sync", async () => {
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const testDeviceKeys = getTestOlmAccountKeys(testOlmAccount, "@bob:xyz", "DEVICE_ID");
keyResponder.addDeviceKeys(testDeviceKeys);
// we expect a call to the /members list...
const memberListPromise = expectMembershipRequest(ROOM_ID, ["@bob:xyz"]);
// then a request for bob's devices...
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
// then a to-device with the room_key
const inboundGroupSessionPromise = expectSendRoomKey("@bob:xyz", testOlmAccount, p2pSession);
@@ -2071,13 +2096,17 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
await Promise.all([sendPromise, megolmMessagePromise, memberListPromise]);
});
oldBackendOnly("loading the membership list inhibits a later load", async () => {
it("loading the membership list inhibits a later load", async () => {
const homeserverUrl = aliceClient.getHomeserverUrl();
keyResponder = new E2EKeyResponder(homeserverUrl);
keyResponder.addKeyReceiver("@alice:localhost", keyReceiver);
const testDeviceKeys = getTestOlmAccountKeys(testOlmAccount, "@bob:xyz", "DEVICE_ID");
keyResponder.addDeviceKeys(testDeviceKeys);
const room = aliceClient.getRoom(ROOM_ID)!;
await Promise.all([room.loadMembersIfNeeded(), expectMembershipRequest(ROOM_ID, ["@bob:xyz"])]);
// expect a request for bob's devices...
expectAliceKeyQuery(getTestKeysQueryResponse("@bob:xyz"));
// then a to-device with the room_key
const inboundGroupSessionPromise = expectSendRoomKey("@bob:xyz", testOlmAccount, p2pSession);
@@ -2410,12 +2439,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
});
describe("Secret Storage and Key Backup", () => {
/**
* The account data events to be returned by the sync.
* Will be updated when fecthMock intercepts calls to PUT `/_matrix/client/v3/user/:userId/account_data/`.
* Will be used by `sendSyncResponseWithUpdatedAccountData`
*/
let accountDataEvents: Map<String, any>;
let accountDataAccumulator: AccountDataAccumulator;
/**
* Create a fake secret storage key
@@ -2428,76 +2452,19 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
beforeEach(async () => {
createSecretStorageKey.mockClear();
accountDataEvents = new Map();
accountDataAccumulator = new AccountDataAccumulator();
expectAliceKeyQuery({ device_keys: { "@alice:localhost": {} }, failures: {} });
await startClientAndAwaitFirstSync();
});
function mockGetAccountData() {
fetchMock.get(
`path:/_matrix/client/v3/user/:userId/account_data/:type`,
(url) => {
const type = url.split("/").pop();
const existing = accountDataEvents.get(type!);
if (existing) {
// return it
return {
status: 200,
body: existing.content,
};
} else {
// 404
return {
status: 404,
body: { errcode: "M_NOT_FOUND", error: "Account data not found." },
};
}
},
{ overwriteRoutes: true },
);
}
/**
* Create a mock to respond to the PUT request `/_matrix/client/v3/user/:userId/account_data/m.cross_signing.${key}`
* Resolved when the cross signing key is uploaded
* https://spec.matrix.org/v1.6/client-server-api/#put_matrixclientv3useruseridaccount_datatype
*/
function awaitCrossSigningKeyUpload(key: string): Promise<Record<string, {}>> {
return new Promise((resolve) => {
// Called when the cross signing key is uploaded
fetchMock.put(
`express:/_matrix/client/v3/user/:userId/account_data/m.cross_signing.${key}`,
(url: string, options: RequestInit) => {
const content = JSON.parse(options.body as string);
const type = url.split("/").pop();
// update account data for sync response
accountDataEvents.set(type!, content);
resolve(content.encrypted);
return {};
},
);
});
}
/**
* Send in the sync response the current account data events, as stored by `accountDataEvents`.
*/
function sendSyncResponseWithUpdatedAccountData() {
try {
syncResponder.sendOrQueueSyncResponse({
next_batch: 1,
account_data: {
events: Array.from(accountDataEvents, ([type, content]) => ({
type: type,
content: content,
})),
},
});
} catch (err) {
// Might fail with "Cannot queue more than one /sync response" if called too often.
// It's ok if it fails here, the sync response is cumulative and will contain
// the latest account data.
}
async function awaitCrossSigningKeyUpload(key: string): Promise<Record<string, {}>> {
const content = await accountDataAccumulator.interceptSetAccountData(`m.cross_signing.${key}`);
return content.encrypted;
}
/**
@@ -2505,28 +2472,18 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
* Resolved when a key is uploaded (ie in `body.content.key`)
* https://spec.matrix.org/v1.6/client-server-api/#put_matrixclientv3useruseridaccount_datatype
*/
function awaitSecretStorageKeyStoredInAccountData(): Promise<string> {
return new Promise((resolve) => {
// This url is called multiple times during the secret storage bootstrap process
// When we received the newly generated key, we return it
fetchMock.put(
"express:/_matrix/client/v3/user/:userId/account_data/:type(m.secret_storage.*)",
(url: string, options: RequestInit) => {
const type = url.split("/").pop();
const content = JSON.parse(options.body as string);
// update account data for sync response
accountDataEvents.set(type!, content);
if (content.key) {
resolve(content.key);
}
sendSyncResponseWithUpdatedAccountData();
return {};
},
{ overwriteRoutes: true },
);
});
async function awaitSecretStorageKeyStoredInAccountData(): Promise<string> {
// eslint-disable-next-line no-constant-condition
while (true) {
const content = await accountDataAccumulator.interceptSetAccountData(":type(m.secret_storage.*)", {
repeat: 1,
overwriteRoutes: true,
});
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
if (content.key) {
return content.key;
}
}
}
function awaitMegolmBackupKeyUpload(): Promise<Record<string, {}>> {
@@ -2537,7 +2494,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
(url: string, options: RequestInit) => {
const content = JSON.parse(options.body as string);
// update account data for sync response
accountDataEvents.set("m.megolm_backup.v1", content);
accountDataAccumulator.accountDataEvents.set("m.megolm_backup.v1", content);
resolve(content.encrypted);
return {};
},
@@ -2602,7 +2559,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
await bootstrapPromise;
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Finally ensure backup is working
await aliceClient.getCrypto()!.checkKeyBackupAndEnable();
@@ -2624,7 +2581,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
);
it("Should create a 4S key", async () => {
mockGetAccountData();
accountDataAccumulator.interceptGetAccountData();
const awaitAccountData = awaitAccountDataUpdate("m.secret_storage.default_key");
@@ -2636,7 +2593,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
const secretStorageKey = await awaitSecretStorageKeyStoredInAccountData();
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Finally, wait for bootstrapSecretStorage to finished
await bootstrapPromise;
@@ -2660,7 +2617,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
await awaitSecretStorageKeyStoredInAccountData();
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Wait for bootstrapSecretStorage to finished
await bootstrapPromise;
@@ -2684,7 +2641,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
await awaitSecretStorageKeyStoredInAccountData();
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Wait for bootstrapSecretStorage to finished
await bootstrapPromise;
@@ -2698,7 +2655,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
await awaitSecretStorageKeyStoredInAccountData();
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Wait for bootstrapSecretStorage to finished
await bootstrapPromise;
@@ -2722,7 +2679,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
const secretStorageKey = await awaitSecretStorageKeyStoredInAccountData();
// Return the newly created key in the sync response
sendSyncResponseWithUpdatedAccountData();
accountDataAccumulator.sendSyncResponseWithUpdatedAccountData(syncResponder);
// Wait for the cross signing keys to be uploaded
const [masterKey, userSigningKey, selfSigningKey] = await Promise.all([
@@ -2875,6 +2832,19 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
const newBackupUploadPromise = awaitMegolmBackupKeyUpload();
// Track calls to scheduleAllGroupSessionsForBackup. This is
// only relevant on legacy encryption.
const scheduleAllGroupSessionsForBackup = jest.fn();
if (backend === "libolm") {
aliceClient.crypto!.backupManager.scheduleAllGroupSessionsForBackup =
scheduleAllGroupSessionsForBackup;
} else {
// With Rust crypto, we don't need to call this function, so
// we call the dummy value here so we pass our later
// expectation.
scheduleAllGroupSessionsForBackup();
}
await aliceClient.getCrypto()!.resetKeyBackup();
await awaitDeleteCalled;
await newBackupStatusUpdate;
@@ -2886,6 +2856,7 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("crypto (%s)", (backend: string,
expect(nextVersion).toBeDefined();
expect(nextVersion).not.toEqual(currentVersion);
expect(nextKey).not.toEqual(currentBackupKey);
expect(scheduleAllGroupSessionsForBackup).toHaveBeenCalled();
// The `deleteKeyBackupVersion` API is deprecated but has been modified to work with both crypto backend
// ensure that it works anyhow
+142 -1
View File
@@ -18,7 +18,7 @@ import fetchMock from "fetch-mock-jest";
import "fake-indexeddb/auto";
import { IDBFactory } from "fake-indexeddb";
import { createClient, CryptoEvent, ICreateClientOpts, MatrixClient, TypedEventEmitter } from "../../../src";
import { createClient, CryptoEvent, ICreateClientOpts, IEvent, MatrixClient, TypedEventEmitter } from "../../../src";
import { SyncResponder } from "../../test-utils/SyncResponder";
import { E2EKeyReceiver } from "../../test-utils/E2EKeyReceiver";
import { E2EKeyResponder } from "../../test-utils/E2EKeyResponder";
@@ -34,6 +34,7 @@ import * as testData from "../../test-utils/test-data";
import { KeyBackupInfo } from "../../../src/crypto-api/keybackup";
import { IKeyBackup } from "../../../src/crypto/backup";
import { flushPromises } from "../../test-utils/flushPromises";
import { defer, IDeferred } from "../../../src/utils";
const ROOM_ID = testData.TEST_ROOM_ID;
@@ -888,6 +889,146 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("megolm-keys backup (%s)", (backe
});
});
describe("Backup Changed from other sessions", () => {
beforeEach(async () => {
fetchMock.get("path:/_matrix/client/v3/room_keys/version", testData.SIGNED_BACKUP_DATA);
// ignore requests to send room key requests
fetchMock.put("express:/_matrix/client/v3/sendToDevice/m.room_key_request/:request_id", {});
aliceClient = await initTestClient();
const aliceCrypto = aliceClient.getCrypto()!;
await aliceCrypto.storeSessionBackupPrivateKey(
Buffer.from(testData.BACKUP_DECRYPTION_KEY_BASE64, "base64"),
testData.SIGNED_BACKUP_DATA.version!,
);
// start after saving the private key
await aliceClient.startClient();
// tell Alice to trust the dummy device that signed the backup, and re-check the backup.
// XXX: should we automatically re-check after a device becomes verified?
await waitForDeviceList();
await aliceClient.getCrypto()!.setDeviceVerified(testData.TEST_USER_ID, testData.TEST_DEVICE_ID);
await aliceClient.getCrypto()!.checkKeyBackupAndEnable();
});
// let aliceClient: MatrixClient;
const SYNC_RESPONSE = {
next_batch: 1,
rooms: { join: { [ROOM_ID]: { timeline: { events: [testData.ENCRYPTED_EVENT] } } } },
};
it("If current backup has changed, the manager should switch to the new one on UTD", async () => {
// =====
// First ensure that the client checks for keys using the backup version 1
/// =====
fetchMock.get(
"express:/_matrix/client/v3/room_keys/keys/:room_id/:session_id",
(url, request) => {
// check that the version is correct
const version = new URLSearchParams(new URL(url).search).get("version");
if (version == "1") {
return testData.CURVE25519_KEY_BACKUP_DATA;
} else {
return {
status: 403,
body: {
current_version: "1",
errcode: "M_WRONG_ROOM_KEYS_VERSION",
error: "Wrong backup version.",
},
};
}
},
{ overwriteRoutes: true },
);
// Send Alice a message that she won't be able to decrypt, and check that she fetches the key from the backup.
syncResponder.sendOrQueueSyncResponse(SYNC_RESPONSE);
await syncPromise(aliceClient);
const room = aliceClient.getRoom(ROOM_ID)!;
const event = room.getLiveTimeline().getEvents()[0];
await advanceTimersUntil(awaitDecryption(event, { waitOnDecryptionFailure: true }));
expect(event.getContent()).toEqual(testData.CLEAR_EVENT.content);
// =====
// Second suppose now that the backup has changed to version 2
/// =====
const newBackup = {
...testData.SIGNED_BACKUP_DATA,
version: "2",
};
fetchMock.get("path:/_matrix/client/v3/room_keys/version", newBackup, { overwriteRoutes: true });
// suppose the new key is now known
const aliceCrypto = aliceClient.getCrypto()!;
await aliceCrypto.storeSessionBackupPrivateKey(
Buffer.from(testData.BACKUP_DECRYPTION_KEY_BASE64, "base64"),
newBackup.version,
);
// A check backup should happen at some point
await aliceCrypto.checkKeyBackupAndEnable();
const awaitHasQueriedNewBackup: IDeferred<void> = defer<void>();
fetchMock.get(
"express:/_matrix/client/v3/room_keys/keys/:room_id/:session_id",
(url, request) => {
// check that the version is correct
const version = new URLSearchParams(new URL(url).search).get("version");
if (version == newBackup.version) {
awaitHasQueriedNewBackup.resolve();
return testData.CURVE25519_KEY_BACKUP_DATA;
} else {
// awaitHasQueriedOldBackup.resolve();
return {
status: 403,
body: {
current_version: "2",
errcode: "M_WRONG_ROOM_KEYS_VERSION",
error: "Wrong backup version.",
},
};
}
},
{ overwriteRoutes: true },
);
// Send Alice a message that she won't be able to decrypt, and check that she fetches the key from the new backup.
const newMessage: Partial<IEvent> = {
type: "m.room.encrypted",
room_id: "!room:id",
sender: "@alice:localhost",
content: {
algorithm: "m.megolm.v1.aes-sha2",
ciphertext:
"AwgAEpABKvf9FqPW52zeHfeVTn90a3jlBLlx7g6VDEkc2089RQUJoWpSJRiK13E83rN41wgGFJccyfoCr7ZDGJeuGYMGETTrgnLQhLs6JmyPf37JYkzxW8uS8rGUKEqTFQriKhibHVLvVacOlSIObUiKU/V3r176XuixqZF/4eyK9A22JNpInbgI10ZUT6LnApH9LR3FpZbE2zImf1uNPuvp7r0xQbW7CcJjqpH+qTPBD5zFdFnMkc2SnbXCsIOaX11Dm0krWfQz7iA26ZnI1nyZnyh7XPrCnJCRsuQH",
device_id: "WVMJGTSSVB",
sender_key: "E5RiY/YCIrHWaF4u416CqvblC6udK2jt9SJ/h1QeLS0",
session_id: "ybnW+LGdUhoS4fHm1DAEphukO3sZ1GCqZD7UQz7L+GA",
},
event_id: "$event2",
origin_server_ts: 1507753887000,
};
const nextSyncResponse = {
next_batch: 2,
rooms: { join: { [ROOM_ID]: { timeline: { events: [newMessage] } } } },
};
syncResponder.sendOrQueueSyncResponse(nextSyncResponse);
await syncPromise(aliceClient);
await awaitHasQueriedNewBackup.promise;
});
});
/** make sure that the client knows about the dummy device */
async function waitForDeviceList(): Promise<void> {
// Completing the initial sync will make the device list download outdated device lists (of which our own
+3 -6
View File
@@ -1259,14 +1259,11 @@ describe.each(Object.entries(CRYPTO_BACKENDS))("verification (%s)", (backend: st
const requestId = await requestPromises.get("m.megolm_backup.v1");
const keyBackupIsCached = emitPromise(aliceClient, CryptoEvent.KeyBackupDecryptionKeyCached);
await sendBackupGossipAndExpectVersion(requestId!, BACKUP_DECRYPTION_KEY_BASE64, matchingBackupInfo);
// We are lacking a way to signal that the secret has been received, so we wait a bit..
jest.useRealTimers();
await new Promise((resolve) => {
setTimeout(resolve, 500);
});
jest.useFakeTimers();
await keyBackupIsCached;
// the backup secret should be cached
const cachedKey = await aliceClient.getCrypto()!.getSessionBackupPrivateKey();
+1 -54
View File
@@ -1204,73 +1204,20 @@ describe("MatrixClient", function () {
describe("requestLoginToken", () => {
it("should hit the expected API endpoint with UIA", async () => {
jest.spyOn(client.http, "getUrl");
httpBackend
.when("GET", "/capabilities")
.respond(200, { capabilities: { "m.get_login_token": { enabled: true } } });
const response = {};
const uiaData = {};
const prom = client.requestLoginToken(uiaData);
httpBackend.when("POST", "/v1/login/get_token", { auth: uiaData }).respond(200, response);
await httpBackend.flush("");
expect(await prom).toStrictEqual(response);
expect(client.http.getUrl).toHaveLastReturnedWith(
expect.objectContaining({
href: "http://alice.localhost.test.server/_matrix/client/v1/login/get_token",
}),
);
});
it("should hit the expected API endpoint without UIA", async () => {
jest.spyOn(client.http, "getUrl");
httpBackend
.when("GET", "/capabilities")
.respond(200, { capabilities: { "m.get_login_token": { enabled: true } } });
const response = { login_token: "xyz", expires_in_ms: 5000 };
const prom = client.requestLoginToken();
httpBackend.when("POST", "/v1/login/get_token", {}).respond(200, response);
await httpBackend.flush("");
// check that expires_in has been populated for compatibility with r0
expect(await prom).toStrictEqual({ ...response, expires_in: 5 });
expect(client.http.getUrl).toHaveLastReturnedWith(
expect.objectContaining({
href: "http://alice.localhost.test.server/_matrix/client/v1/login/get_token",
}),
);
});
it("should still hit the stable endpoint when capability is disabled (but present)", async () => {
jest.spyOn(client.http, "getUrl");
httpBackend
.when("GET", "/capabilities")
.respond(200, { capabilities: { "m.get_login_token": { enabled: false } } });
const response = { login_token: "xyz", expires_in_ms: 5000 };
const prom = client.requestLoginToken();
httpBackend.when("POST", "/v1/login/get_token", {}).respond(200, response);
await httpBackend.flush("");
// check that expires_in has been populated for compatibility with r0
expect(await prom).toStrictEqual({ ...response, expires_in: 5 });
expect(client.http.getUrl).toHaveLastReturnedWith(
expect.objectContaining({
href: "http://alice.localhost.test.server/_matrix/client/v1/login/get_token",
}),
);
});
it("should hit the r0 endpoint for fallback", async () => {
jest.spyOn(client.http, "getUrl");
httpBackend.when("GET", "/capabilities").respond(200, {});
const response = { login_token: "xyz", expires_in: 5 };
const prom = client.requestLoginToken();
httpBackend.when("POST", "/unstable/org.matrix.msc3882/login/token", {}).respond(200, response);
await httpBackend.flush("");
// check that expires_in has been populated for compatibility with r1
expect(await prom).toStrictEqual({ ...response, expires_in_ms: 5000 });
expect(client.http.getUrl).toHaveLastReturnedWith(
expect.objectContaining({
href: "http://alice.localhost.test.server/_matrix/client/unstable/org.matrix.msc3882/login/token",
}),
);
expect(await prom).toStrictEqual(response);
});
});
@@ -28,32 +28,70 @@ import {
NotificationCountType,
RelationType,
Room,
fixNotificationCountOnDecryption,
} from "../../src";
import { TestClient } from "../TestClient";
import { ReceiptType } from "../../src/@types/read_receipts";
import { mkThread } from "../test-utils/thread";
import { SyncState } from "../../src/sync";
const userA = "@alice:localhost";
const userB = "@bob:localhost";
const selfUserId = userA;
const selfAccessToken = "aseukfgwef";
function setupTestClient(): [MatrixClient, HttpBackend] {
const testClient = new TestClient(selfUserId, "DEVICE", selfAccessToken);
const httpBackend = testClient.httpBackend;
const client = testClient.client;
httpBackend!.when("GET", "/versions").respond(200, {});
httpBackend!.when("GET", "/pushrules").respond(200, {});
httpBackend!.when("POST", "/filter").respond(200, { filter_id: "a filter id" });
return [client, httpBackend];
}
describe("Notification count fixing", () => {
let client: MatrixClient | undefined;
beforeEach(() => {
[client] = setupTestClient();
});
it("doesn't increment notification count for events that can't be found in a room", async () => {
const roomId = "!room:localhost";
client!.startClient({ threadSupport: true });
const room = new Room(roomId, client!, selfUserId);
jest.spyOn(client!, "getRoom").mockImplementation((id) => (id === roomId ? room : null));
const event = new MatrixEvent({
room_id: roomId,
type: "m.reaction",
event_id: "$foo",
content: {
"m.relates_to": {
rel_type: RelationType.Annotation,
event_id: "$foo",
key: "x",
},
},
});
jest.spyOn(event, "getPushActions").mockReturnValue({
notify: true,
tweaks: {},
});
fixNotificationCountOnDecryption(client!, event);
expect(room.getUnreadNotificationCount(NotificationCountType.Total)).toBe(0);
});
});
describe("MatrixClient syncing", () => {
const userA = "@alice:localhost";
const userB = "@bob:localhost";
const selfUserId = userA;
const selfAccessToken = "aseukfgwef";
let client: MatrixClient | undefined;
let httpBackend: HttpBackend | undefined;
const setupTestClient = (): [MatrixClient, HttpBackend] => {
const testClient = new TestClient(selfUserId, "DEVICE", selfAccessToken);
const httpBackend = testClient.httpBackend;
const client = testClient.client;
httpBackend!.when("GET", "/versions").respond(200, {});
httpBackend!.when("GET", "/pushrules").respond(200, {});
httpBackend!.when("POST", "/filter").respond(200, { filter_id: "a filter id" });
return [client, httpBackend];
};
beforeEach(() => {
[client, httpBackend] = setupTestClient();
});
+108
View File
@@ -0,0 +1,108 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import fetchMock from "fetch-mock-jest";
import { MockOptionsMethodPut } from "fetch-mock";
import { ISyncResponder } from "./SyncResponder";
/**
* An object which intercepts `account_data` get and set requests via fetch-mock.
*/
export class AccountDataAccumulator {
/**
* The account data events to be returned by the sync.
* Will be updated when fetchMock intercepts calls to PUT `/_matrix/client/v3/user/:userId/account_data/`.
* Will be used by `sendSyncResponseWithUpdatedAccountData`
*/
public accountDataEvents: Map<String, any> = new Map();
/**
* Intercept requests to set a particular type of account data.
*
* Once it is set, its data is stored (for future return by `interceptGetAccountData` etc) and the resolved promise is
* resolved.
*
* @param accountDataType - type of account data to be intercepted
* @param opts - options to pass to fetchMock
* @returns a Promise which will resolve (with the content of the account data) once it is set.
*/
public interceptSetAccountData(accountDataType: string, opts?: MockOptionsMethodPut): Promise<any> {
return new Promise((resolve) => {
// Called when the cross signing key is uploaded
fetchMock.put(
`express:/_matrix/client/v3/user/:userId/account_data/${accountDataType}`,
(url: string, options: RequestInit) => {
const content = JSON.parse(options.body as string);
const type = url.split("/").pop();
// update account data for sync response
this.accountDataEvents.set(type!, content);
resolve(content);
return {};
},
opts,
);
});
}
/**
* Intercept all requests to get account data
*/
public interceptGetAccountData(): void {
fetchMock.get(
`express:/_matrix/client/v3/user/:userId/account_data/:type`,
(url) => {
const type = url.split("/").pop();
const existing = this.accountDataEvents.get(type!);
if (existing) {
// return it
return {
status: 200,
body: existing,
};
} else {
// 404
return {
status: 404,
body: { errcode: "M_NOT_FOUND", error: "Account data not found." },
};
}
},
{ overwriteRoutes: true },
);
}
/**
* Send a sync response the current account data events.
*/
public sendSyncResponseWithUpdatedAccountData(syncResponder: ISyncResponder): void {
try {
syncResponder.sendOrQueueSyncResponse({
next_batch: 1,
account_data: {
events: Array.from(this.accountDataEvents, ([type, content]) => ({
type: type,
content: content,
})),
},
});
} catch (err) {
// Might fail with "Cannot queue more than one /sync response" if called too often.
// It's ok if it fails here, the sync response is cumulative and will contain
// the latest account data.
}
}
}
+1
View File
@@ -315,6 +315,7 @@ export interface IMessageOpts {
event?: boolean;
relatesTo?: IEventRelation;
ts?: number;
unsigned?: IUnsigned;
}
/**
+20
View File
@@ -161,3 +161,23 @@ export const mkThread = ({
return { thread, rootEvent, events };
};
/**
* Create a thread, and make sure the events are added to the thread and the
* room's timeline as if they came in via sync.
*
* Note that mkThread doesn't actually add the events properly to the room.
*/
export const populateThread = ({
room,
client,
authorId,
participantUserIds,
length = 2,
ts = 1,
}: MakeThreadProps): MakeThreadResult => {
const ret = mkThread({ room, client, authorId, participantUserIds, length, ts });
ret.thread.initialEventsFetched = true;
room.addLiveEvents(ret.events);
return ret;
};
+13 -5
View File
@@ -17,7 +17,7 @@ limitations under the License.
import { TextEncoder, TextDecoder } from "util";
import NodeBuffer from "node:buffer";
import { decodeBase64, encodeBase64, encodeUnpaddedBase64 } from "../../src/base64";
import { decodeBase64, encodeBase64, encodeUnpaddedBase64, encodeUnpaddedBase64Url } from "../../src/base64";
describe.each(["browser", "node"])("Base64 encoding (%s)", (env) => {
let origBuffer = Buffer;
@@ -43,19 +43,27 @@ describe.each(["browser", "node"])("Base64 encoding (%s)", (env) => {
global.btoa = undefined;
});
it("Should decode properly encoded data", async () => {
it("Should decode properly encoded data", () => {
const decoded = new TextDecoder().decode(decodeBase64("ZW5jb2RpbmcgaGVsbG8gd29ybGQ="));
expect(decoded).toStrictEqual("encoding hello world");
});
it("Should decode URL-safe base64", async () => {
it("Should encode unpadded URL-safe base64", () => {
const toEncode = "?????";
const data = new TextEncoder().encode(toEncode);
const encoded = encodeUnpaddedBase64Url(data);
expect(encoded).toEqual("Pz8_Pz8");
});
it("Should decode URL-safe base64", () => {
const decoded = new TextDecoder().decode(decodeBase64("Pz8_Pz8="));
expect(decoded).toStrictEqual("?????");
});
it("Encode unpadded should not have padding", async () => {
it("Encode unpadded should not have padding", () => {
const toEncode = "encoding hello world";
const data = new TextEncoder().encode(toEncode);
@@ -68,7 +76,7 @@ describe.each(["browser", "node"])("Base64 encoding (%s)", (env) => {
expect(padding).toStrictEqual("=");
});
it("Decode should be indifferent to padding", async () => {
it("Decode should be indifferent to padding", () => {
const withPadding = "ZW5jb2RpbmcgaGVsbG8gd29ybGQ=";
const withoutPadding = "ZW5jb2RpbmcgaGVsbG8gd29ybGQ";
+4 -20
View File
@@ -356,7 +356,6 @@ describe("Crypto", function () {
let crypto: Crypto;
let mockBaseApis: MatrixClient;
let mockRoomList: RoomList;
let fakeEmitter: EventEmitter;
@@ -390,19 +389,10 @@ describe("Crypto", function () {
isGuest: jest.fn(),
emit: jest.fn(),
} as unknown as MatrixClient;
mockRoomList = {} as unknown as RoomList;
fakeEmitter = new EventEmitter();
crypto = new Crypto(
mockBaseApis,
"@alice:home.server",
"FLIBBLE",
clientStore,
cryptoStore,
mockRoomList,
[],
);
crypto = new Crypto(mockBaseApis, "@alice:home.server", "FLIBBLE", clientStore, cryptoStore, []);
crypto.registerEventHandlers(fakeEmitter as any);
await crypto.init();
});
@@ -1341,15 +1331,9 @@ describe("Crypto", function () {
setRoomEncryption: jest.fn().mockResolvedValue(undefined),
} as unknown as RoomList;
crypto = new Crypto(
mockClient,
"@alice:home.server",
"FLIBBLE",
clientStore,
cryptoStore,
mockRoomList,
[],
);
crypto = new Crypto(mockClient, "@alice:home.server", "FLIBBLE", clientStore, cryptoStore, []);
// @ts-ignore we are injecting a mock into a private property
crypto.roomList = mockRoomList;
});
it("should set the algorithm if called for a known room", async () => {
+1 -1
View File
@@ -85,7 +85,7 @@ describe("CallMembership", () => {
it("considers memberships expired when local age large", () => {
const fakeEvent = makeMockEvent(1000);
fakeEvent.getLocalAge = jest.fn().mockReturnValue(6000);
fakeEvent.localTimestamp = Date.now() - 6000;
const membership = new CallMembership(fakeEvent, membershipTemplate);
expect(membership.isExpired()).toEqual(true);
});
+351 -45
View File
@@ -14,11 +14,12 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import { EventTimeline, EventType, MatrixClient, Room } from "../../../src";
import { EventTimeline, EventType, MatrixClient, MatrixError, MatrixEvent, Room } from "../../../src";
import { CallMembershipData } from "../../../src/matrixrtc/CallMembership";
import { MatrixRTCSession, MatrixRTCSessionEvent } from "../../../src/matrixrtc/MatrixRTCSession";
import { EncryptionKeysEventContent } from "../../../src/matrixrtc/types";
import { randomString } from "../../../src/randomstring";
import { makeMockRoom, mockRTCEvent } from "./mocks";
import { makeMockRoom, makeMockRoomState, mockRTCEvent } from "./mocks";
const membershipTemplate: CallMembershipData = {
call_id: "",
@@ -59,13 +60,14 @@ describe("MatrixRTCSession", () => {
expect(sess?.memberships[0].deviceId).toEqual("AAAAAAA");
expect(sess?.memberships[0].membershipID).toEqual("bloop");
expect(sess?.memberships[0].isExpired()).toEqual(false);
expect(sess?.callId).toEqual("");
});
it("ignores expired memberships events", () => {
const expiredMembership = Object.assign({}, membershipTemplate);
expiredMembership.expires = 1000;
expiredMembership.device_id = "EXPIRED";
const mockRoom = makeMockRoom([membershipTemplate, expiredMembership], () => 10000);
const mockRoom = makeMockRoom([membershipTemplate, expiredMembership], 10000);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
expect(sess?.memberships.length).toEqual(1);
@@ -184,8 +186,15 @@ describe("MatrixRTCSession", () => {
describe("joining", () => {
let mockRoom: Room;
let sendStateEventMock: jest.Mock;
let sendEventMock: jest.Mock;
beforeEach(() => {
sendStateEventMock = jest.fn();
sendEventMock = jest.fn();
client.sendStateEvent = sendStateEventMock;
client.sendEvent = sendEventMock;
mockRoom = makeMockRoom([]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
});
@@ -205,8 +214,6 @@ describe("MatrixRTCSession", () => {
});
it("sends a membership event when joining a call", () => {
client.sendStateEvent = jest.fn();
sess!.joinRoomSession([mockFocus]);
expect(client.sendStateEvent).toHaveBeenCalledWith(
@@ -230,9 +237,6 @@ describe("MatrixRTCSession", () => {
});
it("does nothing if join called when already joined", () => {
const sendStateEventMock = jest.fn();
client.sendStateEvent = sendStateEventMock;
sess!.joinRoomSession([mockFocus]);
expect(client.sendStateEvent).toHaveBeenCalledTimes(1);
@@ -262,7 +266,7 @@ describe("MatrixRTCSession", () => {
const timeElapsed = 60 * 60 * 1000 - 1000;
mockRoom.getLiveTimeline().getState(EventTimeline.FORWARDS)!.getStateEvents = jest
.fn()
.mockReturnValue(mockRTCEvent(eventContent.memberships, mockRoom.roomId, () => timeElapsed));
.mockReturnValue(mockRTCEvent(eventContent.memberships, mockRoom.roomId, timeElapsed));
const eventReSentPromise = new Promise<Record<string, any>>((r) => {
resolveFn = (_roomId: string, _type: string, val: Record<string, any>) => {
@@ -299,15 +303,244 @@ describe("MatrixRTCSession", () => {
jest.useRealTimers();
}
});
it("creates a key when joining", () => {
sess!.joinRoomSession([mockFocus], true);
const keys = sess?.getKeysForParticipant("@alice:example.org", "AAAAAAA");
expect(keys).toHaveLength(1);
const allKeys = sess!.getEncryptionKeys();
expect(allKeys).toBeTruthy();
expect(Array.from(allKeys)).toHaveLength(1);
});
it("sends keys when joining", async () => {
const eventSentPromise = new Promise((resolve) => {
sendEventMock.mockImplementation(resolve);
});
sess!.joinRoomSession([mockFocus], true);
await eventSentPromise;
expect(sendEventMock).toHaveBeenCalledWith(expect.stringMatching(".*"), "io.element.call.encryption_keys", {
call_id: "",
device_id: "AAAAAAA",
keys: [
{
index: 0,
key: expect.stringMatching(".*"),
},
],
});
});
it("retries key sends", async () => {
jest.useFakeTimers();
let firstEventSent = false;
try {
const eventSentPromise = new Promise<void>((resolve) => {
sendEventMock.mockImplementation(() => {
if (!firstEventSent) {
jest.advanceTimersByTime(10000);
firstEventSent = true;
const e = new Error() as MatrixError;
e.data = {};
throw e;
} else {
resolve();
}
});
});
sess!.joinRoomSession([mockFocus], true);
jest.advanceTimersByTime(10000);
await eventSentPromise;
expect(sendEventMock).toHaveBeenCalledTimes(2);
} finally {
jest.useRealTimers();
}
});
it("cancels key send event that fail", async () => {
const eventSentinel = {} as unknown as MatrixEvent;
client.cancelPendingEvent = jest.fn();
sendEventMock.mockImplementation(() => {
const e = new Error() as MatrixError;
e.data = {};
e.event = eventSentinel;
throw e;
});
sess!.joinRoomSession([mockFocus], true);
expect(client.cancelPendingEvent).toHaveBeenCalledWith(eventSentinel);
});
it("Re-sends key if a new member joins", async () => {
jest.useFakeTimers();
try {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const keysSentPromise1 = new Promise((resolve) => {
sendEventMock.mockImplementation(resolve);
});
sess.joinRoomSession([mockFocus], true);
await keysSentPromise1;
sendEventMock.mockClear();
jest.advanceTimersByTime(10000);
const keysSentPromise2 = new Promise((resolve) => {
sendEventMock.mockImplementation(resolve);
});
const onMembershipsChanged = jest.fn();
sess.on(MatrixRTCSessionEvent.MembershipsChanged, onMembershipsChanged);
const member2 = Object.assign({}, membershipTemplate, {
device_id: "BBBBBBB",
});
mockRoom.getLiveTimeline().getState = jest
.fn()
.mockReturnValue(makeMockRoomState([membershipTemplate, member2], mockRoom.roomId, undefined));
sess.onMembershipUpdate();
await keysSentPromise2;
expect(sendEventMock).toHaveBeenCalled();
} finally {
jest.useRealTimers();
}
});
it("Rotates key if a member leaves", async () => {
jest.useFakeTimers();
try {
const member2 = Object.assign({}, membershipTemplate, {
device_id: "BBBBBBB",
});
const mockRoom = makeMockRoom([membershipTemplate, member2]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const onMyEncryptionKeyChanged = jest.fn();
sess.on(
MatrixRTCSessionEvent.EncryptionKeyChanged,
(_key: Uint8Array, _idx: number, participantId: string) => {
if (participantId === `${client.getUserId()}:${client.getDeviceId()}`) {
onMyEncryptionKeyChanged();
}
},
);
const keysSentPromise1 = new Promise<EncryptionKeysEventContent>((resolve) => {
sendEventMock.mockImplementation((_roomId, _evType, payload) => resolve(payload));
});
sess.joinRoomSession([mockFocus], true);
const firstKeysPayload = await keysSentPromise1;
expect(firstKeysPayload.keys).toHaveLength(1);
sendEventMock.mockClear();
const keysSentPromise2 = new Promise<EncryptionKeysEventContent>((resolve) => {
sendEventMock.mockImplementation((_roomId, _evType, payload) => resolve(payload));
});
mockRoom.getLiveTimeline().getState = jest
.fn()
.mockReturnValue(makeMockRoomState([membershipTemplate], mockRoom.roomId, undefined));
sess.onMembershipUpdate();
jest.advanceTimersByTime(10000);
const secondKeysPayload = await keysSentPromise2;
expect(secondKeysPayload.keys).toHaveLength(2);
expect(onMyEncryptionKeyChanged).toHaveBeenCalledTimes(2);
} finally {
jest.useRealTimers();
}
});
it("Doesn't re-send key immediately", async () => {
const realSetImmediate = setImmediate;
jest.useFakeTimers();
try {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const keysSentPromise1 = new Promise((resolve) => {
sendEventMock.mockImplementation(resolve);
});
sess.joinRoomSession([mockFocus], true);
await keysSentPromise1;
sendEventMock.mockClear();
const onMembershipsChanged = jest.fn();
sess.on(MatrixRTCSessionEvent.MembershipsChanged, onMembershipsChanged);
const member2 = Object.assign({}, membershipTemplate, {
device_id: "BBBBBBB",
});
mockRoom.getLiveTimeline().getState = jest
.fn()
.mockReturnValue(makeMockRoomState([membershipTemplate, member2], mockRoom.roomId, undefined));
sess.onMembershipUpdate();
await new Promise((resolve) => {
realSetImmediate(resolve);
});
expect(sendEventMock).not.toHaveBeenCalled();
} finally {
jest.useRealTimers();
}
});
});
it("Does not emits if no membership changes", () => {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const onMembershipsChanged = jest.fn();
sess.on(MatrixRTCSessionEvent.MembershipsChanged, onMembershipsChanged);
sess.onMembershipUpdate();
expect(onMembershipsChanged).not.toHaveBeenCalled();
});
it("Emits on membership changes", () => {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const onMembershipsChanged = jest.fn();
sess.on(MatrixRTCSessionEvent.MembershipsChanged, onMembershipsChanged);
mockRoom.getLiveTimeline().getState = jest
.fn()
.mockReturnValue(makeMockRoomState([], mockRoom.roomId, undefined));
sess.onMembershipUpdate();
expect(onMembershipsChanged).toHaveBeenCalled();
});
it("emits an event at the time a membership event expires", () => {
jest.useFakeTimers();
try {
let eventAge = 0;
const membership = Object.assign({}, membershipTemplate);
const mockRoom = makeMockRoom([membership], () => eventAge);
const mockRoom = makeMockRoom([membership], 0);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
const membershipObject = sess.memberships[0];
@@ -315,7 +548,6 @@ describe("MatrixRTCSession", () => {
const onMembershipsChanged = jest.fn();
sess.on(MatrixRTCSessionEvent.MembershipsChanged, onMembershipsChanged);
eventAge = 61 * 1000 * 1000;
jest.advanceTimersByTime(61 * 1000 * 1000);
expect(onMembershipsChanged).toHaveBeenCalledWith([membershipObject], []);
@@ -326,47 +558,49 @@ describe("MatrixRTCSession", () => {
});
it("prunes expired memberships on update", () => {
client.sendStateEvent = jest.fn();
jest.useFakeTimers();
try {
client.sendStateEvent = jest.fn();
let eventAge = 0;
const mockRoom = makeMockRoom(
[
const mockMemberships = [
Object.assign({}, membershipTemplate, {
device_id: "OTHERDEVICE",
expires: 1000,
}),
],
() => eventAge,
);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
];
const mockRoomNoExpired = makeMockRoom(mockMemberships, 0);
// sanity check
expect(sess.memberships).toHaveLength(1);
expect(sess.memberships[0].deviceId).toEqual("OTHERDEVICE");
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoomNoExpired);
eventAge = 10000;
// sanity check
expect(sess.memberships).toHaveLength(1);
expect(sess.memberships[0].deviceId).toEqual("OTHERDEVICE");
sess.joinRoomSession([mockFocus]);
jest.advanceTimersByTime(10000);
expect(client.sendStateEvent).toHaveBeenCalledWith(
mockRoom!.roomId,
EventType.GroupCallMemberPrefix,
{
memberships: [
{
application: "m.call",
scope: "m.room",
call_id: "",
device_id: "AAAAAAA",
expires: 3600000,
foci_active: [mockFocus],
membershipID: expect.stringMatching(".*"),
},
],
},
"@alice:example.org",
);
sess.joinRoomSession([mockFocus]);
expect(client.sendStateEvent).toHaveBeenCalledWith(
mockRoomNoExpired!.roomId,
EventType.GroupCallMemberPrefix,
{
memberships: [
{
application: "m.call",
scope: "m.room",
call_id: "",
device_id: "AAAAAAA",
expires: 3600000,
foci_active: [mockFocus],
membershipID: expect.stringMatching(".*"),
},
],
},
"@alice:example.org",
);
} finally {
jest.useRealTimers();
}
});
it("fills in created_ts for other memberships on update", () => {
@@ -409,4 +643,76 @@ describe("MatrixRTCSession", () => {
"@alice:example.org",
);
});
it("collects keys from encryption events", () => {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
sess.onCallEncryption({
getType: jest.fn().mockReturnValue("io.element.call.encryption_keys"),
getContent: jest.fn().mockReturnValue({
device_id: "bobsphone",
call_id: "",
keys: [
{
index: 0,
key: "dGhpcyBpcyB0aGUga2V5",
},
],
}),
getSender: jest.fn().mockReturnValue("@bob:example.org"),
} as unknown as MatrixEvent);
const bobKeys = sess.getKeysForParticipant("@bob:example.org", "bobsphone")!;
expect(bobKeys).toHaveLength(1);
expect(bobKeys[0]).toEqual(Buffer.from("this is the key", "utf-8"));
});
it("collects keys at non-zero indices", () => {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
sess.onCallEncryption({
getType: jest.fn().mockReturnValue("io.element.call.encryption_keys"),
getContent: jest.fn().mockReturnValue({
device_id: "bobsphone",
call_id: "",
keys: [
{
index: 4,
key: "dGhpcyBpcyB0aGUga2V5",
},
],
}),
getSender: jest.fn().mockReturnValue("@bob:example.org"),
} as unknown as MatrixEvent);
const bobKeys = sess.getKeysForParticipant("@bob:example.org", "bobsphone")!;
expect(bobKeys).toHaveLength(5);
expect(bobKeys[0]).toBeFalsy();
expect(bobKeys[1]).toBeFalsy();
expect(bobKeys[2]).toBeFalsy();
expect(bobKeys[3]).toBeFalsy();
expect(bobKeys[4]).toEqual(Buffer.from("this is the key", "utf-8"));
});
it("ignores keys event for the local participant", () => {
const mockRoom = makeMockRoom([membershipTemplate]);
sess = MatrixRTCSession.roomSessionForRoom(client, mockRoom);
sess.onCallEncryption({
getType: jest.fn().mockReturnValue("io.element.call.encryption_keys"),
getContent: jest.fn().mockReturnValue({
device_id: client.getDeviceId(),
call_id: "",
keys: [
{
index: 4,
key: "dGhpcyBpcyB0aGUga2V5",
},
],
}),
getSender: jest.fn().mockReturnValue(client.getUserId()),
} as unknown as MatrixEvent);
const myKeys = sess.getKeysForParticipant(client.getUserId()!, client.getDeviceId()!)!;
expect(myKeys).toBeFalsy();
});
});
@@ -14,7 +14,15 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import { ClientEvent, EventTimeline, MatrixClient } from "../../../src";
import {
ClientEvent,
EventTimeline,
EventType,
IRoomTimelineData,
MatrixClient,
MatrixEvent,
RoomEvent,
} from "../../../src";
import { RoomStateEvent } from "../../../src/models/room-state";
import { CallMembershipData } from "../../../src/matrixrtc/CallMembership";
import { MatrixRTCSessionManagerEvents } from "../../../src/matrixrtc/MatrixRTCSessionManager";
@@ -78,4 +86,26 @@ describe("MatrixRTCSessionManager", () => {
expect(onEnded).toHaveBeenCalledWith(room1.roomId, client.matrixRTC.getActiveRoomSession(room1));
});
it("Calls onCallEncryption on encryption keys event", () => {
const room1 = makeMockRoom([membershipTemplate]);
jest.spyOn(client, "getRooms").mockReturnValue([room1]);
jest.spyOn(client, "getRoom").mockReturnValue(room1);
client.emit(ClientEvent.Room, room1);
const onCallEncryptionMock = jest.fn();
client.matrixRTC.getRoomSession(room1).onCallEncryption = onCallEncryptionMock;
const timelineEvent = {
getType: jest.fn().mockReturnValue(EventType.CallEncryptionKeysPrefix),
getContent: jest.fn().mockReturnValue({}),
getSender: jest.fn().mockReturnValue("@mock:user.example"),
getRoomId: jest.fn().mockReturnValue("!room:id"),
sender: {
userId: "@mock:user.example",
},
} as unknown as MatrixEvent;
client.emit(RoomEvent.Timeline, timelineEvent, undefined, undefined, false, {} as IRoomTimelineData);
expect(onCallEncryptionMock).toHaveBeenCalled();
});
});
+8 -17
View File
@@ -18,37 +18,29 @@ import { EventType, MatrixEvent, Room } from "../../../src";
import { CallMembershipData } from "../../../src/matrixrtc/CallMembership";
import { randomString } from "../../../src/randomstring";
export function makeMockRoom(
memberships: CallMembershipData[],
getLocalAge: (() => number) | undefined = undefined,
): Room {
export function makeMockRoom(memberships: CallMembershipData[], localAge: number | null = null): Room {
const roomId = randomString(8);
// Caching roomState here so it does not get recreated when calling `getLiveTimeline.getState()`
const roomState = makeMockRoomState(memberships, roomId, localAge);
return {
roomId: roomId,
getLiveTimeline: jest.fn().mockReturnValue({
getState: jest.fn().mockReturnValue(makeMockRoomState(memberships, roomId, getLocalAge)),
getState: jest.fn().mockReturnValue(roomState),
}),
} as unknown as Room;
}
function makeMockRoomState(memberships: CallMembershipData[], roomId: string, getLocalAge: (() => number) | undefined) {
export function makeMockRoomState(memberships: CallMembershipData[], roomId: string, localAge: number | null = null) {
const event = mockRTCEvent(memberships, roomId, localAge);
return {
getStateEvents: (_: string, stateKey: string) => {
const event = mockRTCEvent(memberships, roomId, getLocalAge);
if (stateKey !== undefined) return event;
return [event];
},
};
}
export function mockRTCEvent(
memberships: CallMembershipData[],
roomId: string,
getLocalAge: (() => number) | undefined,
): MatrixEvent {
const getLocalAgeFn = getLocalAge ?? (() => 10);
export function mockRTCEvent(memberships: CallMembershipData[], roomId: string, localAge: number | null): MatrixEvent {
return {
getType: jest.fn().mockReturnValue(EventType.GroupCallMemberPrefix),
getContent: jest.fn().mockReturnValue({
@@ -56,8 +48,7 @@ export function mockRTCEvent(
}),
getSender: jest.fn().mockReturnValue("@mock:user.example"),
getTs: jest.fn().mockReturnValue(1000),
getLocalAge: getLocalAgeFn,
localTimestamp: Date.now(),
localTimestamp: Date.now() - (localAge ?? 10),
getRoomId: jest.fn().mockReturnValue(roomId),
sender: {
userId: "@mock:user.example",
+279 -21
View File
@@ -14,10 +14,19 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import { MockedObject } from "jest-mock";
import { MatrixEvent, MatrixEventEvent } from "../../../src/models/event";
import { emitPromise } from "../../test-utils/test-utils";
import { Crypto, IEventDecryptionResult } from "../../../src/crypto";
import { IAnnotatedPushRule, PushRuleActionName, TweakName } from "../../../src";
import {
IAnnotatedPushRule,
MatrixClient,
PushRuleActionName,
Room,
THREAD_RELATION_TYPE,
TweakName,
} from "../../../src";
describe("MatrixEvent", () => {
it("should create copies of itself", () => {
@@ -61,31 +70,264 @@ describe("MatrixEvent", () => {
expect(a.toSnapshot().isEquivalentTo(b)).toBe(false);
});
it("should prune clearEvent when being redacted", () => {
const ev = new MatrixEvent({
type: "m.room.message",
content: {
body: "Test",
},
event_id: "$event1:server",
describe("redaction", () => {
it("should prune clearEvent when being redacted", () => {
const ev = createEvent("$event1:server", "Test");
expect(ev.getContent().body).toBe("Test");
expect(ev.getWireContent().body).toBe("Test");
ev.makeEncrypted("m.room.encrypted", { ciphertext: "xyz" }, "", "");
expect(ev.getContent().body).toBe("Test");
expect(ev.getWireContent().body).toBeUndefined();
expect(ev.getWireContent().ciphertext).toBe("xyz");
const mockClient = {} as unknown as MockedObject<MatrixClient>;
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
expect(ev.getContent().body).toBeUndefined();
expect(ev.getWireContent().body).toBeUndefined();
expect(ev.getWireContent().ciphertext).toBeUndefined();
});
expect(ev.getContent().body).toBe("Test");
expect(ev.getWireContent().body).toBe("Test");
ev.makeEncrypted("m.room.encrypted", { ciphertext: "xyz" }, "", "");
expect(ev.getContent().body).toBe("Test");
expect(ev.getWireContent().body).toBeUndefined();
expect(ev.getWireContent().ciphertext).toBe("xyz");
it("should remain in the main timeline when redacted", async () => {
// Given an event in the main timeline
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const ev = createEvent("$event1:server");
const redaction = new MatrixEvent({
type: "m.room.redaction",
redacts: ev.getId(),
await room.addLiveEvents([ev]);
await room.createThreadsTimelineSets();
expect(ev.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([ev.getId()]);
// When I redact it
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
// Then it remains in the main timeline
expect(ev.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([ev.getId()]);
});
ev.makeRedacted(redaction);
expect(ev.getContent().body).toBeUndefined();
expect(ev.getWireContent().body).toBeUndefined();
expect(ev.getWireContent().ciphertext).toBeUndefined();
it("should keep thread roots in both timelines when redacted", async () => {
// Given a thread exists
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const threadRoot = createEvent("$threadroot:server");
const ev = createThreadedEvent("$event1:server", threadRoot.getId()!);
await room.addLiveEvents([threadRoot, ev]);
await room.createThreadsTimelineSets();
expect(threadRoot.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([threadRoot.getId(), ev.getId()]);
// When I redact the thread root
const redaction = createRedaction(ev.getId()!);
threadRoot.makeRedacted(redaction, room);
// Then it remains in the main timeline and the thread
expect(threadRoot.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([threadRoot.getId(), ev.getId()]);
});
it("should move into the main timeline when redacted", async () => {
// Given an event in a thread
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const threadRoot = createEvent("$threadroot:server");
const ev = createThreadedEvent("$event1:server", threadRoot.getId()!);
await room.addLiveEvents([threadRoot, ev]);
await room.createThreadsTimelineSets();
expect(ev.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([threadRoot.getId(), ev.getId()]);
// When I redact it
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
// Then it disappears from the thread and appears in the main timeline
expect(ev.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId(), ev.getId()]);
expect(threadLiveEventIds(room, 0)).not.toContain(ev.getId());
});
it("should move reactions to a redacted event into the main timeline", async () => {
// Given an event in a thread with a reaction
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const threadRoot = createEvent("$threadroot:server");
const ev = createThreadedEvent("$event1:server", threadRoot.getId()!);
const reaction = createReactionEvent("$reaction:server", ev.getId()!);
await room.addLiveEvents([threadRoot, ev, reaction]);
await room.createThreadsTimelineSets();
expect(reaction.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([threadRoot.getId(), ev.getId(), reaction.getId()]);
// When I redact the event
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
// Then the reaction moves into the main timeline
expect(reaction.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId(), ev.getId(), reaction.getId()]);
expect(threadLiveEventIds(room, 0)).not.toContain(reaction.getId());
});
it("should move edits of a redacted event into the main timeline", async () => {
// Given an event in a thread with a reaction
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const threadRoot = createEvent("$threadroot:server");
const ev = createThreadedEvent("$event1:server", threadRoot.getId()!);
const edit = createEditEvent("$edit:server", ev.getId()!);
await room.addLiveEvents([threadRoot, ev, edit]);
await room.createThreadsTimelineSets();
expect(edit.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([threadRoot.getId(), ev.getId(), edit.getId()]);
// When I redact the event
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
// Then the edit moves into the main timeline
expect(edit.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId(), ev.getId(), edit.getId()]);
expect(threadLiveEventIds(room, 0)).not.toContain(edit.getId());
});
it("should move reactions to replies to replies a redacted event into the main timeline", async () => {
// Given an event in a thread with a reaction
const mockClient = createMockClient();
const room = new Room("!roomid:e.xyz", mockClient, "myname");
const threadRoot = createEvent("$threadroot:server");
const ev = createThreadedEvent("$event1:server", threadRoot.getId()!);
const reply1 = createReplyEvent("$reply1:server", ev.getId()!);
const reply2 = createReplyEvent("$reply2:server", reply1.getId()!);
const reaction = createReactionEvent("$reaction:server", reply2.getId()!);
await room.addLiveEvents([threadRoot, ev, reply1, reply2, reaction]);
await room.createThreadsTimelineSets();
expect(reaction.threadRootId).toEqual(threadRoot.getId());
expect(mainTimelineLiveEventIds(room)).toEqual([threadRoot.getId()]);
expect(threadLiveEventIds(room, 0)).toEqual([
threadRoot.getId(),
ev.getId(),
reply1.getId(),
reply2.getId(),
reaction.getId(),
]);
// When I redact the event
const redaction = createRedaction(ev.getId()!);
ev.makeRedacted(redaction, room);
// Then the replies move to the main thread and the reaction disappears
expect(reaction.threadRootId).toBeUndefined();
expect(mainTimelineLiveEventIds(room)).toEqual([
threadRoot.getId(),
ev.getId(),
reply1.getId(),
reply2.getId(),
reaction.getId(),
]);
expect(threadLiveEventIds(room, 0)).not.toContain(reply1.getId());
expect(threadLiveEventIds(room, 0)).not.toContain(reply2.getId());
expect(threadLiveEventIds(room, 0)).not.toContain(reaction.getId());
});
function createMockClient(): MatrixClient {
return {
supportsThreads: jest.fn().mockReturnValue(true),
decryptEventIfNeeded: jest.fn().mockReturnThis(),
getUserId: jest.fn().mockReturnValue("@user:server"),
} as unknown as MockedObject<MatrixClient>;
}
function createEvent(eventId: string, body?: string): MatrixEvent {
return new MatrixEvent({
type: "m.room.message",
content: {
body: body ?? eventId,
},
event_id: eventId,
});
}
function createThreadedEvent(eventId: string, threadRootId: string): MatrixEvent {
return new MatrixEvent({
type: "m.room.message",
content: {
"body": eventId,
"m.relates_to": {
rel_type: THREAD_RELATION_TYPE.name,
event_id: threadRootId,
},
},
event_id: eventId,
});
}
function createEditEvent(eventId: string, repliedToId: string): MatrixEvent {
return new MatrixEvent({
type: "m.room.message",
content: {
"body": "Edited",
"m.new_content": {
body: "Edited",
},
"m.relates_to": {
event_id: repliedToId,
rel_type: "m.replace",
},
},
event_id: eventId,
});
}
function createReplyEvent(eventId: string, repliedToId: string): MatrixEvent {
return new MatrixEvent({
type: "m.room.message",
content: {
"m.relates_to": {
event_id: repliedToId,
key: "x",
rel_type: "m.in_reply_to",
},
},
event_id: eventId,
});
}
function createReactionEvent(eventId: string, reactedToId: string): MatrixEvent {
return new MatrixEvent({
type: "m.reaction",
content: {
"m.relates_to": {
event_id: reactedToId,
key: "x",
rel_type: "m.annotation",
},
},
event_id: eventId,
});
}
function createRedaction(redactedEventid: string): MatrixEvent {
return new MatrixEvent({
type: "m.room.redaction",
redacts: redactedEventid,
});
}
});
describe("applyVisibilityEvent", () => {
@@ -330,3 +572,19 @@ describe("MatrixEvent", () => {
expect(stateEvent.threadRootId).toBeUndefined();
});
});
function mainTimelineLiveEventIds(room: Room): Array<string> {
return room
.getLiveTimeline()
.getEvents()
.map((e) => e.getId()!);
}
function threadLiveEventIds(room: Room, threadIndex: number): Array<string> {
return room
.getThreads()
[threadIndex].getUnfilteredTimelineSet()
.getLiveTimeline()
.getEvents()
.map((e) => e.getId()!);
}
+541
View File
@@ -0,0 +1,541 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { FeatureSupport, MatrixClient, MatrixEvent, ReceiptContent, THREAD_RELATION_TYPE, Thread } from "../../../src";
import { Room } from "../../../src/models/room";
/**
* Note, these tests check the functionality of the RoomReceipts class, but most
* of them access that functionality via the surrounding Room class, because a
* room is required for RoomReceipts to function, and this matches the pattern
* of how this code is used in the wild.
*/
describe("RoomReceipts", () => {
beforeAll(() => {
jest.replaceProperty(Thread, "hasServerSideSupport", FeatureSupport.Stable);
});
afterAll(() => {
jest.restoreAllMocks();
});
it("reports events unread if there are no receipts", () => {
// Given there are no receipts in the room
const room = createRoom();
const [event] = createEvent();
room.addLiveEvents([event]);
// When I ask about any event, then it is unread
expect(room.hasUserReadEvent(readerId, event.getId()!)).toBe(false);
});
it("reports events we sent as read even if there are no (real) receipts", () => {
// Given there are no receipts in the room
const room = createRoom();
const [event] = createEventSentBy(readerId);
room.addLiveEvents([event]);
// When I ask about an event I sent, it is read (because a synthetic
// receipt was created and stored in RoomReceipts)
expect(room.hasUserReadEvent(readerId, event.getId()!)).toBe(true);
});
it("reports read if we receive an unthreaded receipt for this event", () => {
// Given my event exists and is unread
const room = createRoom();
const [event, eventId] = createEvent();
room.addLiveEvents([event]);
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// When we receive a receipt for this event+user
room.addReceipt(createReceipt(readerId, event));
// Then that event is read
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
it("reports read if we receive an unthreaded receipt for a later event", () => {
// Given we have 2 events
const room = createRoom();
const [event1, event1Id] = createEvent();
const [event2] = createEvent();
room.addLiveEvents([event1, event2]);
// When we receive a receipt for the later event
room.addReceipt(createReceipt(readerId, event2));
// Then the earlier one is read
expect(room.hasUserReadEvent(readerId, event1Id)).toBe(true);
});
it("reports read for a non-live event if we receive an unthreaded receipt for a live one", () => {
// Given we have 2 events: one live and one old
const room = createRoom();
const [oldEvent, oldEventId] = createEvent();
const [liveEvent] = createEvent();
room.addLiveEvents([liveEvent]);
createOldTimeline(room, [oldEvent]);
// When we receive a receipt for the live event
room.addReceipt(createReceipt(readerId, liveEvent));
// Then the earlier one is read
expect(room.hasUserReadEvent(readerId, oldEventId)).toBe(true);
});
it("compares by timestamp if two events are in separate old timelines", () => {
// Given we have 2 events, both in old timelines, with event2 after
// event1 in terms of timestamps
const room = createRoom();
const [event1, event1Id] = createEvent();
const [event2, event2Id] = createEvent();
event1.event.origin_server_ts = 1;
event2.event.origin_server_ts = 2;
createOldTimeline(room, [event1]);
createOldTimeline(room, [event2]);
// When we receive a receipt for the older event
room.addReceipt(createReceipt(readerId, event1));
// Then the earlier one is read and the later one is not
expect(room.hasUserReadEvent(readerId, event1Id)).toBe(true);
expect(room.hasUserReadEvent(readerId, event2Id)).toBe(false);
});
it("reports unread if we receive an unthreaded receipt for an earlier event", () => {
// Given we have 2 events
const room = createRoom();
const [event1] = createEvent();
const [event2, event2Id] = createEvent();
room.addLiveEvents([event1, event2]);
// When we receive a receipt for the earlier event
room.addReceipt(createReceipt(readerId, event1));
// Then the later one is unread
expect(room.hasUserReadEvent(readerId, event2Id)).toBe(false);
});
it("reports unread if we receive an unthreaded receipt for a different user", () => {
// Given my event exists and is unread
const room = createRoom();
const [event, eventId] = createEvent();
room.addLiveEvents([event]);
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// When we receive a receipt for another user
room.addReceipt(createReceipt(otherUserId, event));
// Then the event is still unread since the receipt was not for us
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// But it's read for the other person
expect(room.hasUserReadEvent(otherUserId, eventId)).toBe(true);
});
it("reports events we sent as read even if an earlier receipt arrives", () => {
// Given we sent an event after some other event
const room = createRoom();
const [previousEvent] = createEvent();
const [myEvent] = createEventSentBy(readerId);
room.addLiveEvents([previousEvent, myEvent]);
// And I just received a receipt for the previous event
room.addReceipt(createReceipt(readerId, previousEvent));
// When I ask about the event I sent, it is read (because of synthetic receipts)
expect(room.hasUserReadEvent(readerId, myEvent.getId()!)).toBe(true);
});
it("considers events after ones we sent to be unread", () => {
// Given we sent an event, then another event came in
const room = createRoom();
const [myEvent] = createEventSentBy(readerId);
const [laterEvent] = createEvent();
room.addLiveEvents([myEvent, laterEvent]);
// When I ask about the later event, it is unread (because it's after the synthetic receipt)
expect(room.hasUserReadEvent(readerId, laterEvent.getId()!)).toBe(false);
});
it("correctly reports readness even when receipts arrive out of order", () => {
// Given we have 3 events
const room = createRoom();
const [event1] = createEvent();
const [event2, event2Id] = createEvent();
const [event3, event3Id] = createEvent();
room.addLiveEvents([event1, event2, event3]);
// When we receive receipts for the older events out of order
room.addReceipt(createReceipt(readerId, event2));
room.addReceipt(createReceipt(readerId, event1));
// Then we correctly ignore the older receipt
expect(room.hasUserReadEvent(readerId, event2Id)).toBe(true);
expect(room.hasUserReadEvent(readerId, event3Id)).toBe(false);
});
it("reports read if we receive a threaded receipt for this event (main)", () => {
// Given my event exists and is unread
const room = createRoom();
const [event, eventId] = createEvent();
room.addLiveEvents([event]);
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// When we receive a receipt for this event+user
room.addReceipt(createThreadedReceipt(readerId, event, "main"));
// Then that event is read
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
it("reports read if we receive a threaded receipt for this event (non-main)", () => {
// Given my event exists and is unread
const room = createRoom();
const [root, rootId] = createEvent();
const [event, eventId] = createThreadedEvent(root);
setupThread(room, root);
room.addLiveEvents([root, event]);
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// When we receive a receipt for this event on this thread
room.addReceipt(createThreadedReceipt(readerId, event, rootId));
// Then that event is read
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
it("reports read if we receive an threaded receipt for a later event", () => {
// Given we have 2 events in a thread
const room = createRoom();
const [root, rootId] = createEvent();
const [event1, event1Id] = createThreadedEvent(root);
const [event2] = createThreadedEvent(root);
setupThread(room, root);
room.addLiveEvents([root, event1, event2]);
// When we receive a receipt for the later event
room.addReceipt(createThreadedReceipt(readerId, event2, rootId));
// Then the earlier one is read
expect(room.hasUserReadEvent(readerId, event1Id)).toBe(true);
});
it("reports unread if we receive an threaded receipt for an earlier event", () => {
// Given we have 2 events in a thread
const room = createRoom();
const [root, rootId] = createEvent();
const [event1] = createThreadedEvent(root);
const [event2, event2Id] = createThreadedEvent(root);
setupThread(room, root);
room.addLiveEvents([root, event1, event2]);
// When we receive a receipt for the earlier event
room.addReceipt(createThreadedReceipt(readerId, event1, rootId));
// Then the later one is unread
expect(room.hasUserReadEvent(readerId, event2Id)).toBe(false);
});
it("reports unread if we receive an threaded receipt for a different user", () => {
// Given my event exists and is unread
const room = createRoom();
const [root, rootId] = createEvent();
const [event, eventId] = createThreadedEvent(root);
setupThread(room, root);
room.addLiveEvents([root, event]);
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// When we receive a receipt for another user
room.addReceipt(createThreadedReceipt(otherUserId, event, rootId));
// Then the event is still unread since the receipt was not for us
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// But it's read for the other person
expect(room.hasUserReadEvent(otherUserId, eventId)).toBe(true);
});
it("reports unread if we receive a receipt for a later event in a different thread", () => {
// Given 2 events exist in different threads
const room = createRoom();
const [root1] = createEvent();
const [root2] = createEvent();
const [thread1, thread1Id] = createThreadedEvent(root1);
const [thread2] = createThreadedEvent(root2);
setupThread(room, root1);
setupThread(room, root2);
room.addLiveEvents([root1, root2, thread1, thread2]);
// When we receive a receipt for the later event
room.addReceipt(createThreadedReceipt(readerId, thread2, root2.getId()!));
// Then the old one is still unread since the receipt was not for this thread
expect(room.hasUserReadEvent(readerId, thread1Id)).toBe(false);
});
it("correctly reports readness even when threaded receipts arrive out of order", () => {
// Given we have 3 events
const room = createRoom();
const [root, rootId] = createEvent();
const [event1] = createThreadedEvent(root);
const [event2, event2Id] = createThreadedEvent(root);
const [event3, event3Id] = createThreadedEvent(root);
setupThread(room, root);
room.addLiveEvents([root, event1, event2, event3]);
// When we receive receipts for the older events out of order
room.addReceipt(createThreadedReceipt(readerId, event2, rootId));
room.addReceipt(createThreadedReceipt(readerId, event1, rootId));
// Then we correctly ignore the older receipt
expect(room.hasUserReadEvent(readerId, event2Id)).toBe(true);
expect(room.hasUserReadEvent(readerId, event3Id)).toBe(false);
});
it("correctly reports readness when mixing threaded and unthreaded receipts", () => {
// Given we have a setup from this presentation:
// https://docs.google.com/presentation/d/1H1gxRmRFAm8d71hCILWmpOYezsvdlb7cB6ANl-20Gns/edit?usp=sharing
//
// Main1----\
// | ---Thread1a <- threaded receipt
// | |
// | Thread1b
// threaded receipt -> Main2--\
// | ----------------Thread2a <- unthreaded receipt
// Main3 |
// Thread2b <- threaded receipt
//
const room = createRoom();
const [main1, main1Id] = createEvent();
const [main2, main2Id] = createEvent();
const [main3, main3Id] = createEvent();
const [thread1a, thread1aId] = createThreadedEvent(main1);
const [thread1b, thread1bId] = createThreadedEvent(main1);
const [thread2a, thread2aId] = createThreadedEvent(main2);
const [thread2b, thread2bId] = createThreadedEvent(main2);
setupThread(room, main1);
setupThread(room, main2);
room.addLiveEvents([main1, thread1a, thread1b, main2, thread2a, main3, thread2b]);
// And the timestamps on the events are consistent with the order above
main1.event.origin_server_ts = 1;
thread1a.event.origin_server_ts = 2;
thread1b.event.origin_server_ts = 3;
main2.event.origin_server_ts = 4;
thread2a.event.origin_server_ts = 5;
main3.event.origin_server_ts = 6;
thread2b.event.origin_server_ts = 7;
// (Note: in principle, we have the information needed to order these
// events without using their timestamps, since they all came in via
// addLiveEvents. In reality, some of them would have come in via the
// /relations API, making it impossible to get the correct ordering
// without MSC4033, which is why we fall back to timestamps. I.e. we
// definitely could fix the code to make the above
// timestamp-manipulation unnecessary, but it would only make this test
// neater, not actually help in the real world.)
// When the receipts arrive
room.addReceipt(createThreadedReceipt(readerId, main2, "main"));
room.addReceipt(createThreadedReceipt(readerId, thread1a, main1Id));
room.addReceipt(createReceipt(readerId, thread2a));
room.addReceipt(createThreadedReceipt(readerId, thread2b, main2Id));
// Then we correctly identify that only main3 is unread
expect(room.hasUserReadEvent(readerId, main1Id)).toBe(true);
expect(room.hasUserReadEvent(readerId, main2Id)).toBe(true);
expect(room.hasUserReadEvent(readerId, main3Id)).toBe(false);
expect(room.hasUserReadEvent(readerId, thread1aId)).toBe(true);
expect(room.hasUserReadEvent(readerId, thread1bId)).toBe(true);
expect(room.hasUserReadEvent(readerId, thread2aId)).toBe(true);
expect(room.hasUserReadEvent(readerId, thread2bId)).toBe(true);
});
describe("dangling receipts", () => {
it("reports unread if the unthreaded receipt is in a dangling state", () => {
const room = createRoom();
const [event, eventId] = createEvent();
// When we receive a receipt for this event+user
room.addReceipt(createReceipt(readerId, event));
// The event is not added in the room
// So the receipt is in a dangling state
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// Add the event to the room
// The receipt is removed from the dangling state
room.addLiveEvents([event]);
// Then the event is read
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
it("reports unread if the threaded receipt is in a dangling state", () => {
const room = createRoom();
const [root, rootId] = createEvent();
const [event, eventId] = createThreadedEvent(root);
setupThread(room, root);
// When we receive a receipt for this event+user
room.addReceipt(createThreadedReceipt(readerId, event, rootId));
// The event is not added in the room
// So the receipt is in a dangling state
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// Add the events to the room
// The receipt is removed from the dangling state
room.addLiveEvents([root, event]);
// Then the event is read
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
it("should handle multiple dangling receipts for the same event", () => {
const room = createRoom();
const [event, eventId] = createEvent();
// When we receive a receipt for this event+user
room.addReceipt(createReceipt(readerId, event));
// We receive another receipt in the same event for another user
room.addReceipt(createReceipt(otherUserId, event));
// The event is not added in the room
// So the receipt is in a dangling state
expect(room.hasUserReadEvent(readerId, eventId)).toBe(false);
// Add the event to the room
// The two receipts should be processed
room.addLiveEvents([event]);
// Then the event is read
// We expect that the receipt of `otherUserId` didn't replace/erase the receipt of `readerId`
expect(room.hasUserReadEvent(readerId, eventId)).toBe(true);
});
});
});
function createFakeClient(): MatrixClient {
return {
getUserId: jest.fn(),
getEventMapper: jest.fn().mockReturnValue(jest.fn()),
isInitialSyncComplete: jest.fn().mockReturnValue(true),
supportsThreads: jest.fn().mockReturnValue(true),
fetchRoomEvent: jest.fn().mockResolvedValue({}),
paginateEventTimeline: jest.fn(),
canSupport: { get: jest.fn() },
} as unknown as MatrixClient;
}
const senderId = "sender:s.ss";
const readerId = "reader:r.rr";
const otherUserId = "other:o.oo";
function createRoom(): Room {
return new Room("!rid", createFakeClient(), "@u:s.nz", { timelineSupport: true });
}
let idCounter = 0;
function nextId(): string {
return "$" + (idCounter++).toString(10);
}
/**
* Create an event and return it and its ID.
*/
function createEvent(): [MatrixEvent, string] {
return createEventSentBy(senderId);
}
/**
* Create an event with the supplied sender and return it and its ID.
*/
function createEventSentBy(customSenderId: string): [MatrixEvent, string] {
const event = new MatrixEvent({ sender: customSenderId, event_id: nextId() });
return [event, event.getId()!];
}
/**
* Create an event in the thread of the supplied root and return it and its ID.
*/
function createThreadedEvent(root: MatrixEvent): [MatrixEvent, string] {
const rootEventId = root.getId()!;
const event = new MatrixEvent({
sender: senderId,
event_id: nextId(),
content: {
"m.relates_to": {
event_id: rootEventId,
rel_type: THREAD_RELATION_TYPE.name,
["m.in_reply_to"]: {
event_id: rootEventId,
},
},
},
});
return [event, event.getId()!];
}
function createReceipt(userId: string, referencedEvent: MatrixEvent): MatrixEvent {
const content: ReceiptContent = {
[referencedEvent.getId()!]: {
"m.read": {
[userId]: {
ts: 123,
},
},
},
};
return new MatrixEvent({
type: "m.receipt",
content,
});
}
function createThreadedReceipt(userId: string, referencedEvent: MatrixEvent, threadId: string): MatrixEvent {
const content: ReceiptContent = {
[referencedEvent.getId()!]: {
"m.read": {
[userId]: {
ts: 123,
thread_id: threadId,
},
},
},
};
return new MatrixEvent({
type: "m.receipt",
content,
});
}
/**
* Create a timeline in the timeline set that is not the live timeline.
*/
function createOldTimeline(room: Room, events: MatrixEvent[]) {
const oldTimeline = room.getUnfilteredTimelineSet().addTimeline();
room.getUnfilteredTimelineSet().addEventsToTimeline(events, true, oldTimeline);
}
/**
* Perform the hacks required for this room to create a thread based on the root
* event supplied.
*/
function setupThread(room: Room, root: MatrixEvent) {
const thread = room.createThread(root.getId()!, root, [root], false);
thread.initialEventsFetched = true;
}
+69 -32
View File
@@ -19,7 +19,7 @@ import { mocked } from "jest-mock";
import { MatrixClient, PendingEventOrdering } from "../../../src/client";
import { Room, RoomEvent } from "../../../src/models/room";
import { FeatureSupport, Thread, THREAD_RELATION_TYPE, ThreadEvent } from "../../../src/models/thread";
import { makeThreadEvent, mkThread } from "../../test-utils/thread";
import { makeThreadEvent, mkThread, populateThread } from "../../test-utils/thread";
import { TestClient } from "../../TestClient";
import { emitPromise, mkEdit, mkMessage, mkReaction, mock } from "../../test-utils/test-utils";
import { Direction, EventStatus, EventType, MatrixEvent } from "../../../src";
@@ -149,20 +149,38 @@ describe("Thread", () => {
});
it("considers other events with no RR as unread", () => {
const { thread, events } = mkThread({
// Given a long thread exists
const { thread, events } = populateThread({
room,
client,
authorId: myUserId,
participantUserIds: [myUserId],
authorId: "@other:foo.com",
participantUserIds: ["@other:foo.com"],
length: 25,
ts: 190,
});
// Before alice's last unthreaded receipt
expect(thread.hasUserReadEvent("@alice:example.org", events.at(1)!.getId() ?? "")).toBeTruthy();
const event1 = events.at(1)!;
const event2 = events.at(2)!;
const event24 = events.at(24)!;
// After alice's last unthreaded receipt
expect(thread.hasUserReadEvent("@alice:example.org", events.at(-1)!.getId() ?? "")).toBeFalsy();
// And we have read the second message in it with an unthreaded receipt
const receipt = new MatrixEvent({
type: "m.receipt",
room_id: room.roomId,
content: {
// unthreaded receipt for the second message in the thread
[event2.getId()!]: {
[ReceiptType.Read]: {
[myUserId]: { ts: 200 },
},
},
},
});
room.addReceipt(receipt);
// Then we have read the first message in the thread, and not the last
expect(thread.hasUserReadEvent(myUserId, event1.getId()!)).toBe(true);
expect(thread.hasUserReadEvent(myUserId, event24.getId()!)).toBe(false);
});
it("considers event as read if there's a more recent unthreaded receipt", () => {
@@ -481,13 +499,13 @@ describe("Thread", () => {
// And a thread with an added event (with later timestamp)
const userId = "user1";
const { thread, message } = await createThreadAndEvent(client, 1, 100, userId);
const { thread, message2 } = await createThreadAnd2Events(client, 1, 100, 200, userId);
// Then a receipt was added to the thread
const receipt = thread.getReadReceiptForUserId(userId);
expect(receipt).toBeTruthy();
expect(receipt?.eventId).toEqual(message.getId());
expect(receipt?.data.ts).toEqual(100);
expect(receipt?.eventId).toEqual(message2.getId());
expect(receipt?.data.ts).toEqual(200);
expect(receipt?.data.thread_id).toEqual(thread.id);
// (And the receipt was synthetic)
@@ -505,14 +523,14 @@ describe("Thread", () => {
// And a thread with an added event with a lower timestamp than its other events
const userId = "user1";
const { thread } = await createThreadAndEvent(client, 200, 100, userId);
const { thread, message1 } = await createThreadAnd2Events(client, 300, 200, 100, userId);
// Then no receipt was added to the thread (the receipt is still
// for the thread root). This happens because since we have no
// Then the receipt is for the first message, because its
// timestamp is later. This happens because since we have no
// recursive relations support, we know that sometimes events
// appear out of order, so we have to check their timestamps as
// a guess of the correct order.
expect(thread.getReadReceiptForUserId(userId)?.eventId).toEqual(thread.rootEvent?.getId());
expect(thread.getReadReceiptForUserId(userId)?.eventId).toEqual(message1.getId());
});
});
@@ -530,11 +548,11 @@ describe("Thread", () => {
// And a thread with an added event (with later timestamp)
const userId = "user1";
const { thread, message } = await createThreadAndEvent(client, 1, 100, userId);
const { thread, message2 } = await createThreadAnd2Events(client, 1, 100, 200, userId);
// Then a receipt was added to the thread
const receipt = thread.getReadReceiptForUserId(userId);
expect(receipt?.eventId).toEqual(message.getId());
expect(receipt?.eventId).toEqual(message2.getId());
});
it("Creates a local echo receipt even for events BEFORE an existing receipt", async () => {
@@ -550,22 +568,24 @@ describe("Thread", () => {
// And a thread with an added event with a lower timestamp than its other events
const userId = "user1";
const { thread, message } = await createThreadAndEvent(client, 200, 100, userId);
const { thread, message2 } = await createThreadAnd2Events(client, 300, 200, 100, userId);
// Then a receipt was added to the thread, because relations
// recursion is available, so we trust the server to have
// provided us with events in the right order.
// Then a receipt was added for the last message, even though it
// has lower ts, because relations recursion is available, so we
// trust the server to have provided us with events in the right
// order.
const receipt = thread.getReadReceiptForUserId(userId);
expect(receipt?.eventId).toEqual(message.getId());
expect(receipt?.eventId).toEqual(message2.getId());
});
});
async function createThreadAndEvent(
async function createThreadAnd2Events(
client: MatrixClient,
rootTs: number,
eventTs: number,
message1Ts: number,
message2Ts: number,
userId: string,
): Promise<{ thread: Thread; message: MatrixEvent }> {
): Promise<{ thread: Thread; message1: MatrixEvent; message2: MatrixEvent }> {
const room = new Room("room1", client, userId);
// Given a thread
@@ -576,24 +596,41 @@ describe("Thread", () => {
participantUserIds: [],
ts: rootTs,
});
// Sanity: the current receipt is for the thread root
expect(thread.getReadReceiptForUserId(userId)?.eventId).toEqual(thread.rootEvent?.getId());
// Sanity: there is no read receipt on the thread yet because the
// thread events don't get properly added to the room by mkThread.
expect(thread.getReadReceiptForUserId(userId)).toBeNull();
const awaitTimelineEvent = new Promise<void>((res) => thread.on(RoomEvent.Timeline, () => res()));
// When we add a message that is before the latest receipt
const message = makeThreadEvent({
// Add a message with ts message1Ts
const message1 = makeThreadEvent({
event: true,
rootEventId: thread.id,
replyToEventId: thread.id,
user: userId,
room: room.roomId,
ts: eventTs,
ts: message1Ts,
});
await thread.addEvent(message, false, true);
await thread.addEvent(message1, false, true);
await awaitTimelineEvent;
return { thread, message };
// Sanity: the thread now has a properly-added event, so this event
// has a synthetic receipt.
expect(thread.getReadReceiptForUserId(userId)?.eventId).toEqual(message1.getId());
// Add a message with ts message2Ts
const message2 = makeThreadEvent({
event: true,
rootEventId: thread.id,
replyToEventId: thread.id,
user: userId,
room: room.roomId,
ts: message2Ts,
});
await thread.addEvent(message2, false, true);
await awaitTimelineEvent;
return { thread, message1, message2 };
}
function createClientWithEventMapper(canSupport: Map<Feature, ServerSupport> = new Map()): MatrixClient {
+2
View File
@@ -106,6 +106,8 @@ describe("fixNotificationCountOnDecryption", () => {
mockClient,
);
room.addLiveEvents([event]);
THREAD_ID = event.getId()!;
threadEvent = mkEvent({
type: EventType.RoomMessage,
+66
View File
@@ -0,0 +1,66 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { decodeBase64 } from "../../src/base64";
import {
randomLowercaseString,
randomString,
randomUppercaseString,
secureRandomBase64Url,
} from "../../src/randomstring";
describe("Random strings", () => {
it.each([8, 16, 32])("secureRandomBase64 generates %i valid base64 bytes", (n: number) => {
const randb641 = secureRandomBase64Url(n);
const randb642 = secureRandomBase64Url(n);
expect(randb641).not.toEqual(randb642);
const decoded = decodeBase64(randb641);
expect(decoded).toHaveLength(n);
});
it.each([8, 16, 32])("randomString generates string of %i characters", (n: number) => {
const rand1 = randomString(n);
const rand2 = randomString(n);
expect(rand1).not.toEqual(rand2);
expect(rand1).toHaveLength(n);
});
it.each([8, 16, 32])("randomLowercaseString generates lowercase string of %i characters", (n: number) => {
const rand1 = randomLowercaseString(n);
const rand2 = randomLowercaseString(n);
expect(rand1).not.toEqual(rand2);
expect(rand1).toHaveLength(n);
expect(rand1.toLowerCase()).toEqual(rand1);
});
it.each([8, 16, 32])("randomUppercaseString generates lowercase string of %i characters", (n: number) => {
const rand1 = randomUppercaseString(n);
const rand2 = randomUppercaseString(n);
expect(rand1).not.toEqual(rand2);
expect(rand1).toHaveLength(n);
expect(rand1.toUpperCase()).toEqual(rand1);
});
});
+16
View File
@@ -43,8 +43,10 @@ const THREAD_ID = "$thread_event_id";
const ROOM_ID = "!123:matrix.org";
describe("Read receipt", () => {
let threadRoot: MatrixEvent;
let threadEvent: MatrixEvent;
let roomEvent: MatrixEvent;
let editOfThreadRoot: MatrixEvent;
beforeEach(() => {
httpBackend = new MockHttpBackend();
@@ -57,6 +59,15 @@ describe("Read receipt", () => {
client.isGuest = () => false;
client.supportsThreads = () => true;
threadRoot = utils.mkEvent({
event: true,
type: EventType.RoomMessage,
user: "@bob:matrix.org",
room: ROOM_ID,
content: { body: "This is the thread root" },
});
threadRoot.event.event_id = THREAD_ID;
threadEvent = utils.mkEvent({
event: true,
type: EventType.RoomMessage,
@@ -82,6 +93,9 @@ describe("Read receipt", () => {
body: "Hello from a room",
},
});
editOfThreadRoot = utils.mkEdit(threadRoot, client, "@bob:matrix.org", ROOM_ID);
editOfThreadRoot.setThreadId(THREAD_ID);
});
describe("sendReceipt", () => {
@@ -208,6 +222,7 @@ describe("Read receipt", () => {
it.each([
{ getEvent: () => roomEvent, destinationId: MAIN_ROOM_TIMELINE },
{ getEvent: () => threadEvent, destinationId: THREAD_ID },
{ getEvent: () => editOfThreadRoot, destinationId: MAIN_ROOM_TIMELINE },
])("adds the receipt to $destinationId", ({ getEvent, destinationId }) => {
const event = getEvent();
const userId = "@bob:example.org";
@@ -225,6 +240,7 @@ describe("Read receipt", () => {
it("should not allow an older unthreaded receipt to clobber a `main` threaded one", () => {
const userId = client.getSafeUserId();
const room = new Room(ROOM_ID, client, userId);
room.findEventById = jest.fn().mockReturnValue({} as MatrixEvent);
const unthreadedReceipt: WrappedReceipt = {
eventId: "$olderEvent",
+7 -2
View File
@@ -27,6 +27,7 @@ import { M_BEACON } from "../../src/@types/beacon";
import { MatrixClient } from "../../src/client";
import { DecryptionError } from "../../src/crypto/algorithms";
import { defer } from "../../src/utils";
import { Room } from "../../src/models/room";
describe("RoomState", function () {
const roomId = "!foo:bar";
@@ -362,9 +363,11 @@ describe("RoomState", function () {
});
it("does not add redacted beacon info events to state", () => {
const mockClient = {} as unknown as MockedObject<MatrixClient>;
const redactedBeaconEvent = makeBeaconInfoEvent(userA, roomId);
const redactionEvent = new MatrixEvent({ type: "m.room.redaction" });
redactedBeaconEvent.makeRedacted(redactionEvent);
const room = new Room(roomId, mockClient, userA);
redactedBeaconEvent.makeRedacted(redactionEvent, room);
const emitSpy = jest.spyOn(state, "emit");
state.setStateEvents([redactedBeaconEvent]);
@@ -394,11 +397,13 @@ describe("RoomState", function () {
});
it("destroys and removes redacted beacon events", () => {
const mockClient = {} as unknown as MockedObject<MatrixClient>;
const beaconId = "$beacon1";
const beaconEvent = makeBeaconInfoEvent(userA, roomId, { isLive: true }, beaconId);
const redactedBeaconEvent = makeBeaconInfoEvent(userA, roomId, { isLive: true }, beaconId);
const redactionEvent = new MatrixEvent({ type: "m.room.redaction", redacts: beaconEvent.getId() });
redactedBeaconEvent.makeRedacted(redactionEvent);
const room = new Room(roomId, mockClient, userA);
redactedBeaconEvent.makeRedacted(redactionEvent, room);
state.setStateEvents([beaconEvent]);
const beaconInstance = state.beacons.get(getBeaconInfoIdentifier(beaconEvent));
+231 -85
View File
@@ -1743,12 +1743,70 @@ describe("Room", function () {
});
describe("hasUserReadUpTo", function () {
it("should acknowledge if an event has been read", function () {
it("returns true if there is a receipt for this event (main timeline)", function () {
const ts = 13787898424;
room.addLiveEvents([eventToAck]);
room.addReceipt(mkReceipt(roomId, [mkRecord(eventToAck.getId()!, "m.read", userB, ts)]));
room.findEventById = jest.fn().mockReturnValue({ getThread: jest.fn() } as unknown as MatrixEvent);
expect(room.hasUserReadEvent(userB, eventToAck.getId()!)).toEqual(true);
});
it("return false for an unknown event", function () {
it("returns true if there is a receipt for a later event (main timeline)", async function () {
// Given some events exist in the room
const events: MatrixEvent[] = [
utils.mkMessage({
room: roomId,
user: userA,
msg: "1111",
event: true,
}),
utils.mkMessage({
room: roomId,
user: userA,
msg: "2222",
event: true,
}),
utils.mkMessage({
room: roomId,
user: userA,
msg: "3333",
event: true,
}),
];
await room.addLiveEvents(events);
// When I add a receipt for the latest one
room.addReceipt(mkReceipt(roomId, [mkRecord(events[2].getId()!, "m.read", userB, 102)]));
// Then the older ones are read too
expect(room.hasUserReadEvent(userB, events[0].getId()!)).toEqual(true);
expect(room.hasUserReadEvent(userB, events[1].getId()!)).toEqual(true);
});
describe("threads enabled", () => {
beforeEach(() => {
jest.spyOn(room.client, "supportsThreads").mockReturnValue(true);
});
afterEach(() => {
jest.restoreAllMocks();
});
it("returns true if there is an unthreaded receipt for a later event in a thread", async () => {
// Given a thread exists in the room
const { thread, events } = mkThread({ room, length: 3 });
thread.initialEventsFetched = true;
await room.addLiveEvents(events);
// When I add an unthreaded receipt for the latest thread message
room.addReceipt(mkReceipt(roomId, [mkRecord(events[2].getId()!, "m.read", userB, 102)]));
// Then the main timeline message is read
expect(room.hasUserReadEvent(userB, events[0].getId()!)).toEqual(true);
});
});
it("returns false for an unknown event", function () {
expect(room.hasUserReadEvent(userB, "unknown_event")).toEqual(false);
});
});
@@ -3147,106 +3205,194 @@ describe("Room", function () {
const client = new TestClient(userA).client;
const room = new Room(roomId, client, userA);
it("handles missing receipt type", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
return receiptType === ReceiptType.ReadPrivate ? ({ eventId: "eventId" } as WrappedReceipt) : null;
};
expect(room.getEventReadUpTo(userA)).toEqual("eventId");
});
describe("prefers newer receipt", () => {
it("should compare correctly using timelines", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1" } as WrappedReceipt;
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2" } as WrappedReceipt;
}
return null;
};
for (let i = 1; i <= 2; i++) {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: (event1, event2) => {
return event1 === `eventId${i}` ? 1 : -1;
},
} as EventTimelineSet);
expect(room.getEventReadUpTo(userA)).toEqual(`eventId${i}`);
}
describe("invalid receipts", () => {
beforeEach(() => {
// Clear the spies on logger.warn
jest.clearAllMocks();
});
describe("correctly compares by timestamp", () => {
it("should correctly compare, if we have all receipts", () => {
it("ignores receipts pointing at missing events", () => {
// Given a receipt exists
room.getReadReceiptForUserId = (): WrappedReceipt | null => {
return { eventId: "missingEventId" } as WrappedReceipt;
};
// But the event ID it contains does not refer to an event we have
room.findEventById = jest.fn().mockReturnValue(null);
// When we ask what they have read
// Then we say "nothing"
expect(room.getEventReadUpTo(userA)).toBeNull();
});
it("ignores receipts pointing at the wrong thread", () => {
// Given a threaded receipt exists
room.getReadReceiptForUserId = (): WrappedReceipt | null => {
return { eventId: "wrongThreadEventId", data: { ts: 0, thread_id: "thread1" } } as WrappedReceipt;
};
// But the event it refers to is in a thread
room.findEventById = jest.fn().mockReturnValue({ threadRootId: "thread2" } as MatrixEvent);
// When we ask what they have read
// Then we say "nothing"
expect(room.getEventReadUpTo(userA)).toBeNull();
expect(logger.warn).toHaveBeenCalledWith(
"Ignoring receipt because its thread_id (thread1) disagrees with the thread root (thread2) " +
"of the referenced event (event ID = wrongThreadEventId)",
);
});
it("accepts unthreaded receipts pointing at an event in a thread", () => {
// Given an unthreaded receipt exists
room.getReadReceiptForUserId = (): WrappedReceipt | null => {
return { eventId: "inThreadEventId" } as WrappedReceipt;
};
// And the event it refers to is in a thread
room.findEventById = jest.fn().mockReturnValue({ threadRootId: "thread2" } as MatrixEvent);
// When we ask what they have read
// Then we say the event
expect(room.getEventReadUpTo(userA)).toEqual("inThreadEventId");
});
it("accepts main thread receipts pointing at an event in main timeline", () => {
// Given a threaded receipt exists, in main thread
room.getReadReceiptForUserId = (): WrappedReceipt | null => {
return { eventId: "mainThreadEventId", data: { ts: 12, thread_id: "main" } } as WrappedReceipt;
};
// And the event it refers to is in a thread
room.findEventById = jest.fn().mockReturnValue({ threadRootId: undefined } as MatrixEvent);
// When we ask what they have read
// Then we say the event
expect(room.getEventReadUpTo(userA)).toEqual("mainThreadEventId");
});
it("accepts main thread receipts pointing at a thread root", () => {
// Given a threaded receipt exists, in main thread
room.getReadReceiptForUserId = (): WrappedReceipt | null => {
return { eventId: "rootId", data: { ts: 12, thread_id: "main" } } as WrappedReceipt;
};
// And the event it refers to is in a thread, because it is a thread root
room.findEventById = jest
.fn()
.mockReturnValue({ isThreadRoot: true, threadRootId: "thread1" } as MatrixEvent);
// When we ask what they have read
// Then we say the event
expect(room.getEventReadUpTo(userA)).toEqual("rootId");
});
});
describe("valid receipts", () => {
beforeEach(() => {
// When we look up the event referred to by the receipt, it exists
room.findEventById = jest.fn().mockReturnValue({} as MatrixEvent);
});
it("handles missing receipt type", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
return receiptType === ReceiptType.ReadPrivate ? ({ eventId: "eventId" } as WrappedReceipt) : null;
};
expect(room.getEventReadUpTo(userA)).toEqual("eventId");
});
describe("prefers newer receipt", () => {
it("should compare correctly using timelines", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1" } as WrappedReceipt;
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2" } as WrappedReceipt;
}
return null;
};
for (let i = 1; i <= 2; i++) {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: (_1, _2) => null,
} as EventTimelineSet);
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1", data: { ts: i === 1 ? 2 : 1 } } as WrappedReceipt;
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: i === 2 ? 2 : 1 } } as WrappedReceipt;
}
return null;
};
compareEventOrdering: (event1: string, _event2: string) => {
return event1 === `eventId${i}` ? 1 : -1;
},
findEventById: jest.fn().mockReturnValue({} as MatrixEvent),
} as unknown as EventTimelineSet);
expect(room.getEventReadUpTo(userA)).toEqual(`eventId${i}`);
}
});
it("should correctly compare, if private read receipt is missing", () => {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: (_1, _2) => null,
} as EventTimelineSet);
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: 1 } } as WrappedReceipt;
describe("correctly compares by timestamp", () => {
it("should correctly compare, if we have all receipts", () => {
for (let i = 1; i <= 2; i++) {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: () => null,
findEventById: jest.fn().mockReturnValue({} as MatrixEvent),
} as unknown as EventTimelineSet);
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1", data: { ts: i === 1 ? 2 : 1 } } as WrappedReceipt;
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: i === 2 ? 2 : 1 } } as WrappedReceipt;
}
return null;
};
expect(room.getEventReadUpTo(userA)).toEqual(`eventId${i}`);
}
return null;
};
});
expect(room.getEventReadUpTo(userA)).toEqual(`eventId2`);
});
});
it("should correctly compare, if private read receipt is missing", () => {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: () => null,
findEventById: jest.fn().mockReturnValue({} as MatrixEvent),
} as unknown as EventTimelineSet);
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: 1 } } as WrappedReceipt;
}
return null;
};
describe("fallback precedence", () => {
beforeAll(() => {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: (_1, _2) => null,
} as EventTimelineSet);
expect(room.getEventReadUpTo(userA)).toEqual(`eventId2`);
});
});
it("should give precedence to m.read.private", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1", data: { ts: 123 } };
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: 123 } };
}
return null;
};
describe("fallback precedence", () => {
beforeAll(() => {
room.getUnfilteredTimelineSet = () =>
({
compareEventOrdering: () => null,
findEventById: jest.fn().mockReturnValue({} as MatrixEvent),
} as unknown as EventTimelineSet);
});
expect(room.getEventReadUpTo(userA)).toEqual(`eventId1`);
});
it("should give precedence to m.read.private", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.ReadPrivate) {
return { eventId: "eventId1", data: { ts: 123 } };
}
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId2", data: { ts: 123 } };
}
return null;
};
it("should give precedence to m.read", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId3" } as WrappedReceipt;
}
return null;
};
expect(room.getEventReadUpTo(userA)).toEqual(`eventId1`);
});
expect(room.getEventReadUpTo(userA)).toEqual(`eventId3`);
it("should give precedence to m.read", () => {
room.getReadReceiptForUserId = (userId, ignore, receiptType): WrappedReceipt | null => {
if (receiptType === ReceiptType.Read) {
return { eventId: "eventId3" } as WrappedReceipt;
}
return null;
};
expect(room.getEventReadUpTo(userA)).toEqual(`eventId3`);
});
});
});
});
@@ -3564,7 +3710,7 @@ describe("Room", function () {
expect(room.polls.get(pollStartEvent.getId()!)).toBeTruthy();
const redactedEvent = new MatrixEvent({ type: "m.room.redaction" });
pollStartEvent.makeRedacted(redactedEvent);
pollStartEvent.makeRedacted(redactedEvent, room);
await flushPromises();
@@ -23,6 +23,7 @@ import { OutgoingRequestProcessor } from "../../../src/rust-crypto/OutgoingReque
import { KeyClaimManager } from "../../../src/rust-crypto/KeyClaimManager";
import { TypedEventEmitter } from "../../../src/models/typed-event-emitter";
import { HttpApiEvent, HttpApiEventHandlerMap, MatrixHttpApi } from "../../../src";
import { logger, LogSpan } from "../../../src/logger";
afterEach(() => {
fetchMock.mockReset();
@@ -93,7 +94,7 @@ describe("KeyClaimManager", () => {
olmMachine.markRequestAsSent.mockResolvedValueOnce(undefined);
// fire off the request
await keyClaimManager.ensureSessionsForUsers([u1, u2]);
await keyClaimManager.ensureSessionsForUsers(new LogSpan(logger, "test"), [u1, u2]);
// check that all the calls were made
expect(olmMachine.getMissingSessions).toHaveBeenCalledWith([u1, u2]);
@@ -119,12 +120,13 @@ describe("KeyClaimManager", () => {
let markRequestAsSentPromise = awaitCallToMarkRequestAsSent();
// fire off two requests, and keep track of whether their promises resolve
const span = new LogSpan(logger, "test");
let req1Resolved = false;
keyClaimManager.ensureSessionsForUsers([u1]).then(() => {
keyClaimManager.ensureSessionsForUsers(span, [u1]).then(() => {
req1Resolved = true;
});
let req2Resolved = false;
const req2 = keyClaimManager.ensureSessionsForUsers([u2]).then(() => {
const req2 = keyClaimManager.ensureSessionsForUsers(span, [u2]).then(() => {
req2Resolved = true;
});
@@ -24,7 +24,7 @@ import {
KeysUploadRequest,
RoomMessageRequest,
SignatureUploadRequest,
SigningKeysUploadRequest,
UploadSigningKeysRequest,
ToDeviceRequest,
} from "@matrix-org/matrix-sdk-crypto-wasm";
@@ -173,10 +173,10 @@ describe("OutgoingRequestProcessor", () => {
httpBackend.verifyNoOutstandingRequests();
});
it("should handle SigningKeysUploadRequests without UIA", async () => {
it("should handle UploadSigningKeysRequest without UIA", async () => {
// first, mock up a request as we might expect to receive it from the Rust layer ...
const testReq = { foo: "bar" };
const outgoingRequest = new SigningKeysUploadRequest(JSON.stringify(testReq));
const outgoingRequest = new UploadSigningKeysRequest(JSON.stringify(testReq));
// ... then poke the request into the OutgoingRequestProcessor under test
const reqProm = processor.makeOutgoingRequest(outgoingRequest);
@@ -200,10 +200,10 @@ describe("OutgoingRequestProcessor", () => {
httpBackend.verifyNoOutstandingRequests();
});
it("should handle SigningKeysUploadRequests with UIA", async () => {
it("should handle UploadSigningKeysRequest with UIA", async () => {
// first, mock up a request as we might expect to receive it from the Rust layer ...
const testReq = { foo: "bar" };
const outgoingRequest = new SigningKeysUploadRequest(JSON.stringify(testReq));
const outgoingRequest = new UploadSigningKeysRequest(JSON.stringify(testReq));
// also create a UIA callback
const authCallback: UIAuthCallback<Object> = async (makeRequest) => {
@@ -0,0 +1,237 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { Mocked } from "jest-mock";
import * as RustSdkCryptoJs from "@matrix-org/matrix-sdk-crypto-wasm";
import { OutgoingRequest, OutgoingRequestProcessor } from "../../../src/rust-crypto/OutgoingRequestProcessor";
import { OutgoingRequestsManager } from "../../../src/rust-crypto/OutgoingRequestsManager";
import { defer, IDeferred } from "../../../src/utils";
import { logger } from "../../../src/logger";
describe("OutgoingRequestsManager", () => {
/** the OutgoingRequestsManager implementation under test */
let manager: OutgoingRequestsManager;
/** a mock OutgoingRequestProcessor */
let processor: Mocked<OutgoingRequestProcessor>;
/** a mocked-up OlmMachine which manager is connected to */
let olmMachine: Mocked<RustSdkCryptoJs.OlmMachine>;
beforeEach(async () => {
olmMachine = {
outgoingRequests: jest.fn(),
} as unknown as Mocked<RustSdkCryptoJs.OlmMachine>;
processor = {
makeOutgoingRequest: jest.fn(),
} as unknown as Mocked<OutgoingRequestProcessor>;
manager = new OutgoingRequestsManager(logger, olmMachine, processor);
});
describe("Call doProcessOutgoingRequests", () => {
it("The call triggers handling of the machine outgoing requests", async () => {
const request1 = new RustSdkCryptoJs.KeysQueryRequest("foo", "{}");
const request2 = new RustSdkCryptoJs.KeysUploadRequest("foo2", "{}");
olmMachine.outgoingRequests.mockImplementationOnce(async () => {
return [request1, request2];
});
processor.makeOutgoingRequest.mockImplementationOnce(async () => {
return;
});
await manager.doProcessOutgoingRequests();
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(1);
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(2);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request1);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request2);
});
it("Stack and batch calls to doProcessOutgoingRequests while one is already running", async () => {
const request1 = new RustSdkCryptoJs.KeysQueryRequest("foo", "{}");
const request2 = new RustSdkCryptoJs.KeysUploadRequest("foo2", "{}");
const request3 = new RustSdkCryptoJs.KeysBackupRequest("foo3", "{}", "1");
const firstOutgoingRequestDefer = defer<OutgoingRequest[]>();
olmMachine.outgoingRequests
.mockImplementationOnce(async (): Promise<OutgoingRequest[]> => {
return firstOutgoingRequestDefer.promise;
})
.mockImplementationOnce(async () => {
return [request3];
});
const firstRequest = manager.doProcessOutgoingRequests();
// stack 2 additional requests while the first one is still running
const secondRequest = manager.doProcessOutgoingRequests();
const thirdRequest = manager.doProcessOutgoingRequests();
// let the first request complete
firstOutgoingRequestDefer.resolve([request1, request2]);
await firstRequest;
await secondRequest;
await thirdRequest;
// outgoingRequests should be called twice in total, as the second and third requests are
// processed in the same loop.
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(2);
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(3);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request1);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request2);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request3);
});
it("Process 3 consecutive calls to doProcessOutgoingRequests while not blocking previous ones", async () => {
const request1 = new RustSdkCryptoJs.KeysQueryRequest("foo", "{}");
const request2 = new RustSdkCryptoJs.KeysUploadRequest("foo2", "{}");
const request3 = new RustSdkCryptoJs.KeysBackupRequest("foo3", "{}", "1");
// promises which will resolve when OlmMachine.outgoingRequests is called
const outgoingRequestCalledPromises: Promise<void>[] = [];
// deferreds which will provide the results of OlmMachine.outgoingRequests
const outgoingRequestResultDeferreds: IDeferred<OutgoingRequest[]>[] = [];
for (let i = 0; i < 3; i++) {
const resultDeferred = defer<OutgoingRequest[]>();
const calledPromise = new Promise<void>((resolve) => {
olmMachine.outgoingRequests.mockImplementationOnce(() => {
resolve();
return resultDeferred.promise;
});
});
outgoingRequestCalledPromises.push(calledPromise);
outgoingRequestResultDeferreds.push(resultDeferred);
}
const call1 = manager.doProcessOutgoingRequests();
// First call will start an iteration and for now is awaiting on outgoingRequests
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(1);
// Make a new call now: this will request a new iteration
const call2 = manager.doProcessOutgoingRequests();
// let the first iteration complete
outgoingRequestResultDeferreds[0].resolve([request1]);
// The first call should now complete
await call1;
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(1);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request1);
// Wait for the second iteration to fire and be waiting on `outgoingRequests`
await outgoingRequestCalledPromises[1];
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(2);
// Stack a new call that should be processed in an additional iteration.
const call3 = manager.doProcessOutgoingRequests();
outgoingRequestResultDeferreds[1].resolve([request2]);
await call2;
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(2);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request2);
// Wait for the third iteration to fire and be waiting on `outgoingRequests`
await outgoingRequestCalledPromises[2];
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(3);
outgoingRequestResultDeferreds[2].resolve([request3]);
await call3;
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(3);
expect(processor.makeOutgoingRequest).toHaveBeenCalledWith(request3);
// ensure that no other iteration is going on
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(3);
});
it("Should not bubble exceptions if server request is rejected", async () => {
const request = new RustSdkCryptoJs.KeysQueryRequest("foo", "{}");
olmMachine.outgoingRequests.mockImplementationOnce(async () => {
return [request];
});
processor.makeOutgoingRequest.mockImplementationOnce(async () => {
throw new Error("Some network error");
});
await manager.doProcessOutgoingRequests();
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(1);
});
});
describe("Calling stop on the manager should stop ongoing work", () => {
it("When the manager is stopped after outgoingRequests() call, do not make sever requests", async () => {
const request1 = new RustSdkCryptoJs.KeysQueryRequest("foo", "{}");
const firstOutgoingRequestDefer = defer<OutgoingRequest[]>();
olmMachine.outgoingRequests.mockImplementationOnce(async (): Promise<OutgoingRequest[]> => {
return firstOutgoingRequestDefer.promise;
});
const firstRequest = manager.doProcessOutgoingRequests();
// stop
manager.stop();
// let the first request complete
firstOutgoingRequestDefer.resolve([request1]);
await firstRequest;
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(0);
});
it("When the manager is stopped while doing server calls, it should stop before the next sever call", async () => {
const request1 = new RustSdkCryptoJs.KeysQueryRequest("11", "{}");
const request2 = new RustSdkCryptoJs.KeysUploadRequest("12", "{}");
const firstRequestDefer = defer<void>();
olmMachine.outgoingRequests.mockImplementationOnce(async (): Promise<OutgoingRequest[]> => {
return [request1, request2];
});
processor.makeOutgoingRequest
.mockImplementationOnce(async () => {
manager.stop();
return firstRequestDefer.promise;
})
.mockImplementationOnce(async () => {
return;
});
const firstRequest = manager.doProcessOutgoingRequests();
firstRequestDefer.resolve();
await firstRequest;
// should have been called once but not twice
expect(processor.makeOutgoingRequest).toHaveBeenCalledTimes(1);
});
});
});
@@ -0,0 +1,598 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { Mocked, SpyInstance } from "jest-mock";
import * as RustSdkCryptoJs from "@matrix-org/matrix-sdk-crypto-wasm";
import { OlmMachine } from "@matrix-org/matrix-sdk-crypto-wasm";
import fetchMock from "fetch-mock-jest";
import { PerSessionKeyBackupDownloader } from "../../../src/rust-crypto/PerSessionKeyBackupDownloader";
import { logger } from "../../../src/logger";
import { defer, IDeferred } from "../../../src/utils";
import { RustBackupCryptoEventMap, RustBackupCryptoEvents, RustBackupManager } from "../../../src/rust-crypto/backup";
import * as TestData from "../../test-utils/test-data";
import {
ConnectionError,
CryptoEvent,
HttpApiEvent,
HttpApiEventHandlerMap,
IHttpOpts,
IMegolmSessionData,
MatrixHttpApi,
TypedEventEmitter,
} from "../../../src";
import * as testData from "../../test-utils/test-data";
import { BackupDecryptor } from "../../../src/common-crypto/CryptoBackend";
import { KeyBackupSession } from "../../../src/crypto-api/keybackup";
describe("PerSessionKeyBackupDownloader", () => {
/** The downloader under test */
let downloader: PerSessionKeyBackupDownloader;
const mockCipherKey: Mocked<KeyBackupSession> = {} as unknown as Mocked<KeyBackupSession>;
// matches the const in PerSessionKeyBackupDownloader
const BACKOFF_TIME = 5000;
let mockEmitter: TypedEventEmitter<RustBackupCryptoEvents, RustBackupCryptoEventMap>;
let mockHttp: MatrixHttpApi<IHttpOpts & { onlyData: true }>;
let mockRustBackupManager: Mocked<RustBackupManager>;
let mockOlmMachine: Mocked<OlmMachine>;
let mockBackupDecryptor: Mocked<BackupDecryptor>;
let expectedSession: { [roomId: string]: { [sessionId: string]: IDeferred<void> } };
function expectSessionImported(roomId: string, sessionId: string) {
const deferred = defer<void>();
if (!expectedSession[roomId]) {
expectedSession[roomId] = {};
}
expectedSession[roomId][sessionId] = deferred;
return deferred.promise;
}
function mockClearSession(sessionId: string): Mocked<IMegolmSessionData> {
return {
session_id: sessionId,
} as unknown as Mocked<IMegolmSessionData>;
}
beforeEach(async () => {
mockEmitter = new TypedEventEmitter() as TypedEventEmitter<RustBackupCryptoEvents, RustBackupCryptoEventMap>;
mockHttp = new MatrixHttpApi(new TypedEventEmitter<HttpApiEvent, HttpApiEventHandlerMap>(), {
baseUrl: "http://server/",
prefix: "",
onlyData: true,
});
mockBackupDecryptor = {
decryptSessions: jest.fn(),
} as unknown as Mocked<BackupDecryptor>;
mockBackupDecryptor.decryptSessions.mockImplementation(async (ciphertexts) => {
const sessionId = Object.keys(ciphertexts)[0];
return [mockClearSession(sessionId)];
});
mockRustBackupManager = {
getActiveBackupVersion: jest.fn(),
requestKeyBackupVersion: jest.fn(),
importBackedUpRoomKeys: jest.fn(),
createBackupDecryptor: jest.fn().mockReturnValue(mockBackupDecryptor),
on: jest.fn().mockImplementation((event, listener) => {
mockEmitter.on(event, listener);
}),
off: jest.fn().mockImplementation((event, listener) => {
mockEmitter.off(event, listener);
}),
} as unknown as Mocked<RustBackupManager>;
mockOlmMachine = {
getBackupKeys: jest.fn(),
} as unknown as Mocked<OlmMachine>;
downloader = new PerSessionKeyBackupDownloader(logger, mockOlmMachine, mockHttp, mockRustBackupManager);
expectedSession = {};
mockRustBackupManager.importBackedUpRoomKeys.mockImplementation(async (keys) => {
const roomId = keys[0].room_id;
const sessionId = keys[0].session_id;
const deferred = expectedSession[roomId] && expectedSession[roomId][sessionId];
if (deferred) {
deferred.resolve();
}
});
jest.useFakeTimers();
});
afterEach(() => {
expectedSession = {};
downloader.stop();
fetchMock.mockReset();
jest.useRealTimers();
});
describe("Given valid backup available", () => {
beforeEach(async () => {
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA.version!);
mockOlmMachine.getBackupKeys.mockResolvedValue({
backupVersion: TestData.SIGNED_BACKUP_DATA.version!,
decryptionKey: RustSdkCryptoJs.BackupDecryptionKey.fromBase64(TestData.BACKUP_DECRYPTION_KEY_BASE64),
} as unknown as RustSdkCryptoJs.BackupKeys);
mockRustBackupManager.requestKeyBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA);
});
it("Should download and import a missing key from backup", async () => {
const awaitKeyImported = defer<void>();
const roomId = "!roomId";
const sessionId = "sessionId";
const expectAPICall = new Promise<void>((resolve) => {
fetchMock.get(`path:/_matrix/client/v3/room_keys/keys/${roomId}/${sessionId}`, (url, request) => {
resolve();
return TestData.CURVE25519_KEY_BACKUP_DATA;
});
});
mockRustBackupManager.importBackedUpRoomKeys.mockImplementation(async (keys) => {
awaitKeyImported.resolve();
});
mockBackupDecryptor.decryptSessions.mockResolvedValue([TestData.MEGOLM_SESSION_DATA]);
downloader.onDecryptionKeyMissingError(roomId, sessionId);
await expectAPICall;
await awaitKeyImported.promise;
expect(mockRustBackupManager.createBackupDecryptor).toHaveBeenCalledTimes(1);
});
it("Should not hammer the backup if the key is requested repeatedly", async () => {
const blockOnServerRequest = defer<void>();
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/!roomId/:session_id`, async (url, request) => {
await blockOnServerRequest.promise;
return [mockCipherKey];
});
const awaitKey2Imported = defer<void>();
mockRustBackupManager.importBackedUpRoomKeys.mockImplementation(async (keys) => {
if (keys[0].session_id === "sessionId2") {
awaitKey2Imported.resolve();
}
});
// @ts-ignore access to private function
const spy = jest.spyOn(downloader, "queryKeyBackup");
// Call 3 times for same key
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
// Call again for a different key
downloader.onDecryptionKeyMissingError("!roomId", "sessionId2");
// Allow the first server request to complete
blockOnServerRequest.resolve();
await awaitKey2Imported.promise;
expect(spy).toHaveBeenCalledTimes(2);
});
it("should continue to next key if current not in backup", async () => {
fetchMock.get(`path:/_matrix/client/v3/room_keys/keys/!roomA/sessionA0`, {
status: 404,
body: {
errcode: "M_NOT_FOUND",
error: "No backup found",
},
});
fetchMock.get(`path:/_matrix/client/v3/room_keys/keys/!roomA/sessionA1`, mockCipherKey);
// @ts-ignore access to private function
const spy: SpyInstance = jest.spyOn(downloader, "queryKeyBackup");
const expectImported = expectSessionImported("!roomA", "sessionA1");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await jest.runAllTimersAsync();
expect(spy).toHaveBeenCalledTimes(1);
expect(spy).toHaveLastReturnedWith(Promise.resolve({ ok: false, error: "MISSING_DECRYPTION_KEY" }));
downloader.onDecryptionKeyMissingError("!roomA", "sessionA1");
await jest.runAllTimersAsync();
expect(spy).toHaveBeenCalledTimes(2);
await expectImported;
});
it("Should not query repeatedly for a key not in backup", async () => {
fetchMock.get(`path:/_matrix/client/v3/room_keys/keys/!roomA/sessionA0`, {
status: 404,
body: {
errcode: "M_NOT_FOUND",
error: "No backup found",
},
});
// @ts-ignore access to private function
const spy: SpyInstance = jest.spyOn(downloader, "queryKeyBackup");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await jest.runAllTimersAsync();
expect(spy).toHaveBeenCalledTimes(1);
const returnedPromise = spy.mock.results[0].value;
await expect(returnedPromise).rejects.toThrow("Failed to get key from backup: MISSING_DECRYPTION_KEY");
// Should not query again for a key not in backup
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await jest.runAllTimersAsync();
expect(spy).toHaveBeenCalledTimes(1);
// advance time to retry
jest.advanceTimersByTime(BACKOFF_TIME + 10);
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await jest.runAllTimersAsync();
expect(spy).toHaveBeenCalledTimes(2);
await expect(spy.mock.results[1].value).rejects.toThrow(
"Failed to get key from backup: MISSING_DECRYPTION_KEY",
);
});
it("Should stop properly", async () => {
// Simulate a call to stop while request is in flight
const blockOnServerRequest = defer<void>();
const requestRoomKeyCalled = defer<void>();
// Mock the request to block
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, async (url, request) => {
requestRoomKeyCalled.resolve();
await blockOnServerRequest.promise;
return mockCipherKey;
});
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA1");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA2");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA3");
await requestRoomKeyCalled.promise;
downloader.stop();
blockOnServerRequest.resolve();
// let the first request complete
await jest.runAllTimersAsync();
expect(mockRustBackupManager.importBackedUpRoomKeys).not.toHaveBeenCalled();
expect(
fetchMock.calls(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`).length,
).toStrictEqual(1);
});
});
describe("Given no usable backup available", () => {
let getConfigSpy: SpyInstance;
beforeEach(async () => {
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(null);
mockOlmMachine.getBackupKeys.mockResolvedValue(null);
// @ts-ignore access to private function
getConfigSpy = jest.spyOn(downloader, "getOrCreateBackupConfiguration");
});
it("Should not query server if no backup", async () => {
fetchMock.get("path:/_matrix/client/v3/room_keys/version", {
status: 404,
body: { errcode: "M_NOT_FOUND", error: "No current backup version." },
});
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
await jest.runAllTimersAsync();
expect(getConfigSpy).toHaveBeenCalledTimes(1);
expect(getConfigSpy).toHaveReturnedWith(Promise.resolve(null));
});
it("Should not query server if backup not active", async () => {
// there is a backup
fetchMock.get("path:/_matrix/client/v3/room_keys/version", testData.SIGNED_BACKUP_DATA);
// but it's not trusted
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(null);
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
await jest.runAllTimersAsync();
expect(getConfigSpy).toHaveBeenCalledTimes(1);
expect(getConfigSpy).toHaveReturnedWith(Promise.resolve(null));
});
it("Should stop if backup key is not cached", async () => {
// there is a backup
fetchMock.get("path:/_matrix/client/v3/room_keys/version", testData.SIGNED_BACKUP_DATA);
// it is trusted
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA.version!);
// but the key is not cached
mockOlmMachine.getBackupKeys.mockResolvedValue(null);
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
await jest.runAllTimersAsync();
expect(getConfigSpy).toHaveBeenCalledTimes(1);
expect(getConfigSpy).toHaveReturnedWith(Promise.resolve(null));
});
it("Should stop if backup key cached as wrong version", async () => {
// there is a backup
fetchMock.get("path:/_matrix/client/v3/room_keys/version", testData.SIGNED_BACKUP_DATA);
// it is trusted
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA.version!);
// but the cached key has the wrong version
mockOlmMachine.getBackupKeys.mockResolvedValue({
backupVersion: "0",
decryptionKey: RustSdkCryptoJs.BackupDecryptionKey.fromBase64(TestData.BACKUP_DECRYPTION_KEY_BASE64),
} as unknown as RustSdkCryptoJs.BackupKeys);
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
await jest.runAllTimersAsync();
expect(getConfigSpy).toHaveBeenCalledTimes(1);
expect(getConfigSpy).toHaveReturnedWith(Promise.resolve(null));
});
it("Should stop if backup key version does not match the active one", async () => {
// there is a backup
fetchMock.get("path:/_matrix/client/v3/room_keys/version", testData.SIGNED_BACKUP_DATA);
// The sdk is out of sync, the trusted version is the old one
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue("0");
// key for old backup cached
mockOlmMachine.getBackupKeys.mockResolvedValue({
backupVersion: "0",
decryptionKey: RustSdkCryptoJs.BackupDecryptionKey.fromBase64(TestData.BACKUP_DECRYPTION_KEY_BASE64),
} as unknown as RustSdkCryptoJs.BackupKeys);
downloader.onDecryptionKeyMissingError("!roomId", "sessionId");
await jest.runAllTimersAsync();
expect(getConfigSpy).toHaveBeenCalledTimes(1);
expect(getConfigSpy).toHaveReturnedWith(Promise.resolve(null));
});
});
describe("Given Backup state update", () => {
it("After initial sync, when backup becomes trusted it should request keys for past requests", async () => {
// there is a backup
mockRustBackupManager.requestKeyBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA);
// but at this point it's not trusted and we don't have the key
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(null);
mockOlmMachine.getBackupKeys.mockResolvedValue(null);
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, mockCipherKey);
const a0Imported = expectSessionImported("!roomA", "sessionA0");
const a1Imported = expectSessionImported("!roomA", "sessionA1");
const b1Imported = expectSessionImported("!roomB", "sessionB1");
const c1Imported = expectSessionImported("!roomC", "sessionC1");
// During initial sync several keys are requested
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA1");
downloader.onDecryptionKeyMissingError("!roomB", "sessionB1");
downloader.onDecryptionKeyMissingError("!roomC", "sessionC1");
await jest.runAllTimersAsync();
// @ts-ignore access to private property
expect(downloader.hasConfigurationProblem).toEqual(true);
// Now the backup becomes trusted
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA.version!);
// And we have the key in cache
mockOlmMachine.getBackupKeys.mockResolvedValue({
backupVersion: TestData.SIGNED_BACKUP_DATA.version!,
decryptionKey: RustSdkCryptoJs.BackupDecryptionKey.fromBase64(TestData.BACKUP_DECRYPTION_KEY_BASE64),
} as unknown as RustSdkCryptoJs.BackupKeys);
// In that case the sdk would fire a backup status update
mockEmitter.emit(CryptoEvent.KeyBackupStatus, true);
await jest.runAllTimersAsync();
await a0Imported;
await a1Imported;
await b1Imported;
await c1Imported;
});
});
describe("Error cases", () => {
beforeEach(async () => {
// there is a backup
mockRustBackupManager.requestKeyBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA);
// It's trusted
mockRustBackupManager.getActiveBackupVersion.mockResolvedValue(TestData.SIGNED_BACKUP_DATA.version!);
// And we have the key in cache
mockOlmMachine.getBackupKeys.mockResolvedValue({
backupVersion: TestData.SIGNED_BACKUP_DATA.version!,
decryptionKey: RustSdkCryptoJs.BackupDecryptionKey.fromBase64(TestData.BACKUP_DECRYPTION_KEY_BASE64),
} as unknown as RustSdkCryptoJs.BackupKeys);
});
it("Should wait on rate limit error", async () => {
// simulate rate limit error
fetchMock.get(
`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`,
{
status: 429,
body: {
errcode: "M_LIMIT_EXCEEDED",
error: "Too many requests",
retry_after_ms: 5000,
},
},
{ overwriteRoutes: true },
);
const keyImported = expectSessionImported("!roomA", "sessionA0");
// @ts-ignore
const originalImplementation = downloader.queryKeyBackup.bind(downloader);
// @ts-ignore access to private function
const keyQuerySpy: SpyInstance = jest.spyOn(downloader, "queryKeyBackup");
const rateDeferred = defer<void>();
keyQuerySpy.mockImplementation(
// @ts-ignore
async (targetRoomId: string, targetSessionId: string, configuration: any) => {
try {
return await originalImplementation(targetRoomId, targetSessionId, configuration);
} catch (err: any) {
if (err.name === "KeyDownloadRateLimitError") {
rateDeferred.resolve();
}
throw err;
}
},
);
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await rateDeferred.promise;
expect(keyQuerySpy).toHaveBeenCalledTimes(1);
await expect(keyQuerySpy.mock.results[0].value).rejects.toThrow(
"Failed to get key from backup: rate limited",
);
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, mockCipherKey, {
overwriteRoutes: true,
});
// Advance less than the retry_after_ms
jest.advanceTimersByTime(100);
// let any pending callbacks in PromiseJobs run
await Promise.resolve();
// no additional call should have been made
expect(keyQuerySpy).toHaveBeenCalledTimes(1);
// The loop should resume after the retry_after_ms
jest.advanceTimersByTime(5000);
// let any pending callbacks in PromiseJobs run
await Promise.resolve();
await keyImported;
expect(keyQuerySpy).toHaveBeenCalledTimes(2);
});
it("After a network error the same key is retried", async () => {
// simulate connectivity error
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, () => {
throw new ConnectionError("fetch failed", new Error("fetch failed"));
});
// @ts-ignore
const originalImplementation = downloader.queryKeyBackup.bind(downloader);
// @ts-ignore
const keyQuerySpy: SpyInstance = jest.spyOn(downloader, "queryKeyBackup");
const errorDeferred = defer<void>();
keyQuerySpy.mockImplementation(
// @ts-ignore
async (targetRoomId: string, targetSessionId: string, configuration: any) => {
try {
return await originalImplementation(targetRoomId, targetSessionId, configuration);
} catch (err: any) {
if (err.name === "KeyDownloadError") {
errorDeferred.resolve();
}
throw err;
}
},
);
const keyImported = expectSessionImported("!roomA", "sessionA0");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
await errorDeferred.promise;
await Promise.resolve();
await expect(keyQuerySpy.mock.results[0].value).rejects.toThrow(
"Failed to get key from backup: NETWORK_ERROR",
);
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, mockCipherKey, {
overwriteRoutes: true,
});
// Advance less than the retry_after_ms
jest.advanceTimersByTime(100);
// let any pending callbacks in PromiseJobs run
await Promise.resolve();
// no additional call should have been made
expect(keyQuerySpy).toHaveBeenCalledTimes(1);
// The loop should resume after the retry_after_ms
jest.advanceTimersByTime(BACKOFF_TIME + 100);
await Promise.resolve();
await keyImported;
});
it("On Unknown error on import skip the key and continue", async () => {
const keyImported = defer<void>();
mockRustBackupManager.importBackedUpRoomKeys
.mockImplementationOnce(async () => {
throw new Error("Didn't work");
})
.mockImplementationOnce(async (sessions) => {
const roomId = sessions[0].room_id;
const sessionId = sessions[0].session_id;
if (roomId === "!roomA" && sessionId === "sessionA1") {
keyImported.resolve();
}
return;
});
fetchMock.get(`express:/_matrix/client/v3/room_keys/keys/:roomId/:sessionId`, mockCipherKey, {
overwriteRoutes: true,
});
// @ts-ignore access to private function
const keyQuerySpy: SpyInstance = jest.spyOn(downloader, "queryKeyBackup");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA0");
downloader.onDecryptionKeyMissingError("!roomA", "sessionA1");
await jest.runAllTimersAsync();
await keyImported.promise;
expect(keyQuerySpy).toHaveBeenCalledTimes(2);
expect(mockRustBackupManager.importBackedUpRoomKeys).toHaveBeenCalledTimes(2);
});
});
});
+49 -45
View File
@@ -50,6 +50,8 @@ import {
import * as testData from "../../test-utils/test-data";
import { defer } from "../../../src/utils";
import { logger } from "../../../src/logger";
import { OutgoingRequestsManager } from "../../../src/rust-crypto/OutgoingRequestsManager";
import { Curve25519AuthData } from "../../../src/crypto-api/keybackup";
const TEST_USER = "@alice:example.com";
const TEST_DEVICE_ID = "TEST_DEVICE";
@@ -347,6 +349,8 @@ describe("RustCrypto", () => {
makeOutgoingRequest: jest.fn(),
} as unknown as Mocked<OutgoingRequestProcessor>;
const outgoingRequestsManager = new OutgoingRequestsManager(logger, olmMachine, outgoingRequestProcessor);
rustCrypto = new RustCrypto(
logger,
olmMachine,
@@ -357,6 +361,7 @@ describe("RustCrypto", () => {
{} as CryptoCallbacks,
);
rustCrypto["outgoingRequestProcessor"] = outgoingRequestProcessor;
rustCrypto["outgoingRequestsManager"] = outgoingRequestsManager;
});
it("should poll for outgoing messages and send them", async () => {
@@ -395,50 +400,6 @@ describe("RustCrypto", () => {
await awaitCallToMakeOutgoingRequest();
expect(olmMachine.outgoingRequests).toHaveBeenCalledTimes(2);
});
it("stops looping when stop() is called", async () => {
for (let i = 0; i < 5; i++) {
outgoingRequestQueue.push([new KeysQueryRequest("1234", "{}")]);
}
let makeRequestPromise = awaitCallToMakeOutgoingRequest();
rustCrypto.onSyncCompleted({});
expect(rustCrypto["outgoingRequestLoopRunning"]).toBeTruthy();
// go a couple of times round the loop
let resolveMakeRequest = await makeRequestPromise;
makeRequestPromise = awaitCallToMakeOutgoingRequest();
resolveMakeRequest();
resolveMakeRequest = await makeRequestPromise;
makeRequestPromise = awaitCallToMakeOutgoingRequest();
resolveMakeRequest();
// a second sync while this is going on shouldn't make any difference
rustCrypto.onSyncCompleted({});
resolveMakeRequest = await makeRequestPromise;
outgoingRequestProcessor.makeOutgoingRequest.mockReset();
resolveMakeRequest();
// now stop...
rustCrypto.stop();
// which should (eventually) cause the loop to stop with no further calls to outgoingRequests
olmMachine.outgoingRequests.mockReset();
await new Promise((resolve) => {
setTimeout(resolve, 100);
});
expect(rustCrypto["outgoingRequestLoopRunning"]).toBeFalsy();
expect(outgoingRequestProcessor.makeOutgoingRequest).not.toHaveBeenCalled();
expect(olmMachine.outgoingRequests).not.toHaveBeenCalled();
// we sent three, so there should be 2 left
expect(outgoingRequestQueue.length).toEqual(2);
});
});
describe(".getEventEncryptionInfo", () => {
@@ -685,6 +646,7 @@ describe("RustCrypto", () => {
it("should call getDevice", async () => {
olmMachine.getDevice.mockResolvedValue({
free: jest.fn(),
isCrossSigningTrusted: jest.fn().mockReturnValue(false),
isLocallyTrusted: jest.fn().mockReturnValue(false),
isCrossSignedByOwner: jest.fn().mockReturnValue(false),
@@ -911,7 +873,7 @@ describe("RustCrypto", () => {
});
it("returns a verified UserVerificationStatus when the UserIdentity is verified", async () => {
olmMachine.getIdentity.mockResolvedValue({ isVerified: jest.fn().mockReturnValue(true) });
olmMachine.getIdentity.mockResolvedValue({ free: jest.fn(), isVerified: jest.fn().mockReturnValue(true) });
const userVerificationStatus = await rustCrypto.getUserVerificationStatus(testData.TEST_USER_ID);
expect(userVerificationStatus.isVerified()).toBeTruthy();
@@ -970,6 +932,48 @@ describe("RustCrypto", () => {
await rustCrypto.onUserIdentityUpdated(new RustSdkCryptoJs.UserId(testData.TEST_USER_ID));
expect(await keyBackupStatusPromise).toBe(true);
});
it("does not back up keys that came from backup", async () => {
const rustCrypto = await makeTestRustCrypto();
const olmMachine: OlmMachine = rustCrypto["olmMachine"];
await olmMachine.enableBackupV1(
(testData.SIGNED_BACKUP_DATA.auth_data as Curve25519AuthData).public_key,
testData.SIGNED_BACKUP_DATA.version!,
);
// we import two keys: one "from backup", and one "from export"
const [backedUpRoomKey, exportedRoomKey] = testData.MEGOLM_SESSION_DATA_ARRAY;
await rustCrypto.importBackedUpRoomKeys([backedUpRoomKey]);
await rustCrypto.importRoomKeys([exportedRoomKey]);
// we ask for the keys that should be backed up
const roomKeysRequest = await olmMachine.backupRoomKeys();
expect(roomKeysRequest).toBeTruthy();
const roomKeys = JSON.parse(roomKeysRequest!.body);
// we expect that the key "from export" is present
expect(roomKeys).toMatchObject({
rooms: {
[exportedRoomKey.room_id]: {
sessions: {
[exportedRoomKey.session_id]: {},
},
},
},
});
// we expect that the key "from backup" is not present
expect(roomKeys).not.toMatchObject({
rooms: {
[backedUpRoomKey.room_id]: {
sessions: {
[backedUpRoomKey.session_id]: {},
},
},
},
});
});
});
});
+19 -2
View File
@@ -62,9 +62,13 @@ describe("VerificationRequest", () => {
describe("startVerification", () => {
let request: RustVerificationRequest;
let machine: Mocked<RustSdkCryptoJs.OlmMachine>;
let inner: Mocked<RustSdkCryptoJs.VerificationRequest>;
beforeEach(() => {
request = makeTestRequest();
inner = makeMockedInner();
machine = { getDevice: jest.fn() } as unknown as Mocked<RustSdkCryptoJs.OlmMachine>;
request = makeTestRequest(inner, machine);
});
it("does not permit methods other than SAS", async () => {
@@ -73,7 +77,15 @@ describe("VerificationRequest", () => {
);
});
it("raises an error if the other device is unknown", async () => {
await expect(request.startVerification("m.sas.v1")).rejects.toThrow(
"startVerification(): other device is unknown",
);
});
it("raises an error if starting verification does not produce a verifier", async () => {
jest.spyOn(inner, "otherDeviceId", "get").mockReturnValue(new RustSdkCryptoJs.DeviceId("other_device"));
machine.getDevice.mockResolvedValue({} as RustSdkCryptoJs.Device);
await expect(request.startVerification("m.sas.v1")).rejects.toThrow(
"Still no verifier after startSas() call",
);
@@ -118,11 +130,13 @@ describe("isVerificationEvent", () => {
/** build a RustVerificationRequest with default parameters */
function makeTestRequest(
inner?: RustSdkCryptoJs.VerificationRequest,
olmMachine?: RustSdkCryptoJs.OlmMachine,
outgoingRequestProcessor?: OutgoingRequestProcessor,
): RustVerificationRequest {
inner ??= makeMockedInner();
olmMachine ??= {} as RustSdkCryptoJs.OlmMachine;
outgoingRequestProcessor ??= {} as OutgoingRequestProcessor;
return new RustVerificationRequest(inner, outgoingRequestProcessor, []);
return new RustVerificationRequest(olmMachine, inner, outgoingRequestProcessor, []);
}
/** Mock up a rust-side VerificationRequest */
@@ -133,5 +147,8 @@ function makeMockedInner(): Mocked<RustSdkCryptoJs.VerificationRequest> {
phase: jest.fn().mockReturnValue(RustSdkCryptoJs.VerificationRequestPhase.Created),
isPassive: jest.fn().mockReturnValue(false),
timeRemainingMillis: jest.fn(),
get otherDeviceId() {
return undefined;
},
} as unknown as Mocked<RustSdkCryptoJs.VerificationRequest>;
}
+62
View File
@@ -22,12 +22,15 @@ import { Room } from "../../src/models/room";
import { EventTimeline } from "../../src/models/event-timeline";
import { TimelineIndex, TimelineWindow } from "../../src/timeline-window";
import { mkMessage } from "../test-utils/test-utils";
import { MatrixEvent } from "../../src/models/event";
const ROOM_ID = "roomId";
const USER_ID = "userId";
const mockClient = {
getEventTimeline: jest.fn(),
paginateEventTimeline: jest.fn(),
supportsThreads: jest.fn(),
getUserId: jest.fn().mockReturnValue(USER_ID),
} as unknown as MockedObject<MatrixClient>;
/*
@@ -64,6 +67,23 @@ function addEventsToTimeline(timeline: EventTimeline, numEvents: number, toStart
}
}
function createEvents(numEvents: number): Array<MatrixEvent> {
const ret = [];
for (let i = 0; i < numEvents; i++) {
ret.push(
mkMessage({
room: ROOM_ID,
user: USER_ID,
event: true,
unsigned: { age: 1 },
}),
);
}
return ret;
}
/*
* create a pair of linked timelines
*/
@@ -412,4 +432,46 @@ describe("TimelineWindow", function () {
expect(timelineWindow.canPaginate(EventTimeline.FORWARDS)).toBe(true);
});
});
function idsOf(events: Array<MatrixEvent>): Array<string> {
return events.map((e) => (e ? e.getId() ?? "MISSING_ID" : "MISSING_EVENT"));
}
describe("removing events", () => {
it("should shorten if removing an event within the window makes it overflow", function () {
// Given a room with events in two timelines
const room = new Room(ROOM_ID, mockClient, USER_ID, { timelineSupport: true });
const timelineSet = room.getUnfilteredTimelineSet();
const liveTimeline = room.getLiveTimeline();
const oldTimeline = room.addTimeline();
liveTimeline.setNeighbouringTimeline(oldTimeline, EventTimeline.BACKWARDS);
oldTimeline.setNeighbouringTimeline(liveTimeline, EventTimeline.FORWARDS);
const oldEvents = createEvents(5);
const liveEvents = createEvents(5);
const [, , e3, e4, e5] = oldEvents;
const [, e7, e8, e9, e10] = liveEvents;
room.addLiveEvents(liveEvents);
room.addEventsToTimeline(oldEvents, true, oldTimeline);
// And 2 windows over the timelines in this room
const oldWindow = new TimelineWindow(mockClient, timelineSet);
oldWindow.load(e5.getId(), 6);
expect(idsOf(oldWindow.getEvents())).toEqual(idsOf([e5, e4, e3]));
const newWindow = new TimelineWindow(mockClient, timelineSet);
newWindow.load(e9.getId(), 4);
expect(idsOf(newWindow.getEvents())).toEqual(idsOf([e7, e8, e9, e10]));
// When I remove an event
room.removeEvent(e8.getId()!);
// Then the affected timeline is shortened (because it would have
// been too long with the removed event gone)
expect(idsOf(newWindow.getEvents())).toEqual(idsOf([e7, e9, e10]));
// And the unaffected one is not
expect(idsOf(oldWindow.getEvents())).toEqual(idsOf([e5, e4, e3]));
});
});
});
-6
View File
@@ -250,12 +250,6 @@ export interface LoginTokenPostResponse {
* The token to use with `m.login.token` to authenticate.
*/
login_token: string;
/**
* Expiration in seconds.
*
* @deprecated this is only provided for compatibility with original revision of [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882).
*/
expires_in: number;
/**
* Expiration in milliseconds.
*/
+4
View File
@@ -55,6 +55,7 @@ export enum EventType {
CallReplaces = "m.call.replaces",
CallAssertedIdentity = "m.call.asserted_identity",
CallAssertedIdentityPrefix = "org.matrix.call.asserted_identity",
CallEncryptionKeysPrefix = "io.element.call.encryption_keys",
KeyVerificationRequest = "m.key.verification.request",
KeyVerificationStart = "m.key.verification.start",
KeyVerificationCancel = "m.key.verification.cancel",
@@ -93,6 +94,9 @@ export enum EventType {
// Group call events
GroupCallPrefix = "org.matrix.msc3401.call",
GroupCallMemberPrefix = "org.matrix.msc3401.call.member",
// MatrixRTC events
CallNotify = "org.matrix.msc4075.call.notify",
}
export enum RelationType {
+10 -1
View File
@@ -54,7 +54,16 @@ export function encodeUnpaddedBase64(uint8Array: ArrayBuffer | Uint8Array): stri
}
/**
* Decode a base64 string to a typed array of uint8.
* Encode a typed array of uint8 as unpadded base64 using the URL-safe encoding.
* @param uint8Array - The data to encode.
* @returns The unpadded base64.
*/
export function encodeUnpaddedBase64Url(uint8Array: ArrayBuffer | Uint8Array): string {
return encodeUnpaddedBase64(uint8Array).replace("+", "-").replace("/", "_");
}
/**
* Decode a base64 (or base64url) string to a typed array of uint8.
* @param base64 - The base64 to decode.
* @returns The decoded data.
*/
+52 -70
View File
@@ -51,7 +51,7 @@ import { decodeBase64, encodeBase64 } from "./base64";
import { IExportedDevice as IExportedOlmDevice } from "./crypto/OlmDevice";
import { IOlmDevice } from "./crypto/algorithms/megolm";
import { TypedReEmitter } from "./ReEmitter";
import { IRoomEncryption, RoomList } from "./crypto/RoomList";
import { IRoomEncryption } from "./crypto/RoomList";
import { logger, Logger } from "./logger";
import { SERVICE_TYPES } from "./service-types";
import {
@@ -536,21 +536,11 @@ export interface IThreadsCapability extends ICapability {}
export interface IGetLoginTokenCapability extends ICapability {}
/**
* @deprecated use {@link IGetLoginTokenCapability} instead
*/
export type IMSC3882GetLoginTokenCapability = IGetLoginTokenCapability;
export const GET_LOGIN_TOKEN_CAPABILITY = new NamespacedValue(
"m.get_login_token",
"org.matrix.msc3882.get_login_token",
);
/**
* @deprecated use {@link GET_LOGIN_TOKEN_CAPABILITY} instead
*/
export const UNSTABLE_MSC3882_CAPABILITY = GET_LOGIN_TOKEN_CAPABILITY;
export const UNSTABLE_MSC2666_SHARED_ROOMS = "uk.half-shot.msc2666";
export const UNSTABLE_MSC2666_MUTUAL_ROOMS = "uk.half-shot.msc2666.mutual_rooms";
export const UNSTABLE_MSC2666_QUERY_MUTUAL_ROOMS = "uk.half-shot.msc2666.query_mutual_rooms";
@@ -905,7 +895,7 @@ interface IRoomHierarchy {
export interface TimestampToEventResponse {
event_id: string;
origin_server_ts: string;
origin_server_ts: number;
}
interface IWhoamiResponse {
@@ -961,6 +951,7 @@ type CryptoEvents =
| CryptoEvent.KeyBackupStatus
| CryptoEvent.KeyBackupFailed
| CryptoEvent.KeyBackupSessionsRemaining
| CryptoEvent.KeyBackupDecryptionKeyCached
| CryptoEvent.RoomKeyRequest
| CryptoEvent.RoomKeyRequestCancellation
| CryptoEvent.VerificationRequest
@@ -1227,7 +1218,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
public reEmitter = new TypedReEmitter<EmittedEvents, ClientEventHandlerMap>(this);
public olmVersion: [number, number, number] | null = null; // populated after initCrypto
public usingExternalCrypto = false;
public store: Store;
private _store!: Store;
public deviceId: string | null;
public credentials: { userId: string | null };
@@ -1281,7 +1272,6 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
protected cryptoStore?: CryptoStore;
protected verificationMethods?: VerificationMethod[];
protected fallbackICEServerAllowed = false;
protected roomList: RoomList;
protected syncApi?: SlidingSyncSdk | SyncApi;
public roomNameGenerator?: ICreateClientOpts["roomNameGenerator"];
public pushRules?: IPushRules;
@@ -1342,7 +1332,6 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
this.usingExternalCrypto = opts.usingExternalCrypto ?? false;
this.store = opts.store || new StubStore();
this.store.setUserCreator((userId) => User.createUser(userId, this));
this.deviceId = opts.deviceId || null;
this.sessionId = randomString(10);
@@ -1438,10 +1427,6 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
this.livekitServiceURL = opts.livekitServiceURL;
// List of which rooms have encryption enabled: separate from crypto because
// we still want to know which rooms are encrypted even if crypto is disabled:
// we don't want to start sending unencrypted events to them.
this.roomList = new RoomList(this.cryptoStore);
this.roomNameGenerator = opts.roomNameGenerator;
this.toDeviceMessageQueue = new ToDeviceMessageQueue(this);
@@ -1505,6 +1490,18 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
this.ignoredInvites = new IgnoredInvites(this);
this._secretStorage = new ServerSideSecretStorageImpl(this, opts.cryptoCallbacks ?? {});
// having lots of event listeners is not unusual. 0 means "unlimited".
this.setMaxListeners(0);
}
public set store(newStore: Store) {
this._store = newStore;
this._store.setUserCreator((userId) => User.createUser(userId, this));
}
public get store(): Store {
return this._store;
}
/**
@@ -2231,10 +2228,6 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
this.logger.debug("Crypto: Starting up crypto store...");
await this.cryptoStore.startup();
// initialise the list of encrypted rooms (whether or not crypto is enabled)
this.logger.debug("Crypto: initialising roomlist...");
await this.roomList.init();
const userId = this.getUserId();
if (userId === null) {
throw new Error(
@@ -2249,15 +2242,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
);
}
const crypto = new Crypto(
this,
userId,
this.deviceId,
this.store,
this.cryptoStore,
this.roomList,
this.verificationMethods!,
);
const crypto = new Crypto(this, userId, this.deviceId, this.store, this.cryptoStore, this.verificationMethods!);
this.reEmitter.reEmit(crypto, [
CryptoEvent.KeyBackupFailed,
@@ -2358,6 +2343,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
CryptoEvent.KeyBackupStatus,
CryptoEvent.KeyBackupSessionsRemaining,
CryptoEvent.KeyBackupFailed,
CryptoEvent.KeyBackupDecryptionKeyCached,
]);
}
@@ -2392,6 +2378,8 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
*
* @returns base64-encoded ed25519 key. Null if crypto is
* disabled.
*
* @deprecated Prefer {@link CryptoApi.getOwnDeviceKeys}
*/
public getDeviceEd25519Key(): string | null {
return this.crypto?.getDeviceEd25519Key() ?? null;
@@ -2402,6 +2390,8 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
*
* @returns base64-encoded curve25519 key. Null if crypto is
* disabled.
*
* @deprecated Use {@link CryptoApi.getOwnDeviceKeys}
*/
public getDeviceCurve25519Key(): string | null {
return this.crypto?.getDeviceCurve25519Key() ?? null;
@@ -3276,7 +3266,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
// we don't have an m.room.encrypted event, but that might be because
// the server is hiding it from us. Check the store to see if it was
// previously encrypted.
return this.roomList.isRoomEncrypted(roomId);
return this.crypto?.isRoomEncrypted(roomId) ?? false;
}
/**
@@ -3639,6 +3629,9 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
/**
* Marks all group sessions as needing to be backed up and schedules them to
* upload in the background as soon as possible.
*
* (This is done automatically as part of {@link CryptoApi.resetKeyBackup},
* so there is probably no need to call this manually.)
*/
public async scheduleAllGroupSessionsForBackup(): Promise<void> {
if (!this.crypto) {
@@ -3651,6 +3644,10 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
/**
* Marks all group sessions as needing to be backed up without scheduling
* them to upload in the background.
*
* (This is done automatically as part of {@link CryptoApi.resetKeyBackup},
* so there is probably no need to call this manually.)
*
* @returns Promise which resolves to the number of sessions requiring a backup.
*/
public flagAllGroupSessionsForBackup(): Promise<number> {
@@ -3970,10 +3967,9 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
backupDecryptor.free();
}
await this.cryptoBackend.importRoomKeys(keys, {
await this.cryptoBackend.importBackedUpRoomKeys(keys, {
progressCallback,
untrusted,
source: "backup",
});
/// in case entering the passphrase would add a new signature?
@@ -4006,7 +4002,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
throw new Error("End-to-end encryption disabled");
}
const roomEncryption = this.roomList.getRoomEncryption(roomId);
const roomEncryption = this.crypto?.getRoomEncryption(roomId);
if (!roomEncryption) {
// unknown room, or unencrypted room
this.logger.error("Unknown room. Not sharing decryption keys");
@@ -5167,7 +5163,7 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
const room = this.getRoom(event.getRoomId());
if (room && this.credentials.userId) {
room.addLocalEchoReceipt(this.credentials.userId, event, receiptType);
room.addLocalEchoReceipt(this.credentials.userId, event, receiptType, unthreaded);
}
return promise;
}
@@ -8045,50 +8041,23 @@ export class MatrixClient extends TypedEventEmitter<EmittedEvents, ClientEventHa
/**
* Make a request for an `m.login.token` to be issued as per
* [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882).
* The server may require User-Interactive auth.
* https://spec.matrix.org/v1.7/client-server-api/#post_matrixclientv1loginget_token
*
* Compatibility with unstable implementations of MSC3882 is deprecated and will be removed in a future release.
* The server may require User-Interactive auth.
*
* @param auth - Optional. Auth data to supply for User-Interactive auth.
* @returns Promise which resolves: On success, the token response
* or UIA auth data.
*/
public async requestLoginToken(auth?: AuthDict): Promise<UIAResponse<LoginTokenPostResponse>> {
// use capabilities to determine which revision of the MSC is being used
const capabilities = await this.getCapabilities();
let endpoint: string;
if (capabilities[GET_LOGIN_TOKEN_CAPABILITY.name]) {
// use the stable endpoint
endpoint = `${ClientPrefix.V1}/login/get_token`;
} else if (capabilities[GET_LOGIN_TOKEN_CAPABILITY.altName!]) {
// newer unstable r1 endpoint
endpoint = `${ClientPrefix.Unstable}/org.matrix.msc3882/login/get_token`;
} else {
// old unstable r0 endpoint
endpoint = `${ClientPrefix.Unstable}/org.matrix.msc3882/login/token`;
}
const body: UIARequest<{}> = { auth };
const res = await this.http.authedRequest<UIAResponse<LoginTokenPostResponse>>(
return this.http.authedRequest<UIAResponse<LoginTokenPostResponse>>(
Method.Post,
endpoint,
"/login/get_token",
undefined, // no query params
body,
{ prefix: "" },
{ prefix: ClientPrefix.V1 },
);
// the representation of expires_in changed from unstable revision 0 to unstable revision 1 so we cross populate
if ("login_token" in res) {
if (typeof res.expires_in_ms === "number") {
res.expires_in = Math.floor(res.expires_in_ms / 1000);
} else if (typeof res.expires_in === "number") {
res.expires_in_ms = res.expires_in * 1000;
}
}
return res;
}
/**
@@ -9858,6 +9827,19 @@ export function fixNotificationCountOnDecryption(cli: MatrixClient, event: Matri
const room = cli.getRoom(event.getRoomId());
if (!room || !ourUserId || !eventId) return;
// Due to threads, we can get relation events (eg. edits & reactions) that never get
// added to a timeline and so cannot be found in their own room (their edit / reaction
// still applies to the event it needs to, so it doesn't matter too much). However, if
// we try to process notification about this event, we'll get very confused because we
// won't be able to find the event in the room, so will assume it must be unread, even
// if it's actually read. We therefore skip anything that isn't in the room. This isn't
// *great*, so if we can fix the homeless events (eg. with MSC4023) then we should probably
// remove this workaround.
if (!room.findEventById(eventId)) {
logger.info(`Decrypted event ${event.getId()} is not in room ${room.roomId}: ignoring`);
return;
}
const isThreadEvent = !!event.threadRootId && !event.isThreadRoot;
let hasReadEvent;
@@ -9942,7 +9924,7 @@ export function threadIdForReceipt(event: MatrixEvent): string {
* @returns true if this event is considered to be in the main timeline as far
* as receipts are concerned.
*/
function inMainTimelineForReceipt(event: MatrixEvent): boolean {
export function inMainTimelineForReceipt(event: MatrixEvent): boolean {
if (!event.threadRootId) {
// Not in a thread: then it is in the main timeline
return true;
+10 -1
View File
@@ -17,7 +17,7 @@ limitations under the License.
import type { IDeviceLists, IToDeviceEvent } from "../sync-accumulator";
import { IClearEvent, MatrixEvent } from "../models/event";
import { Room } from "../models/room";
import { CryptoApi } from "../crypto-api";
import { CryptoApi, ImportRoomKeysOpts } from "../crypto-api";
import { CrossSigningInfo, UserTrustLevel } from "../crypto/CrossSigning";
import { IEncryptedEventInfo } from "../crypto/api";
import { KeyBackupInfo, KeyBackupSession } from "../crypto-api/keybackup";
@@ -108,6 +108,15 @@ export interface CryptoBackend extends SyncCryptoCallbacks, CryptoApi {
* @param privKey - The private decryption key.
*/
getBackupDecryptor(backupInfo: KeyBackupInfo, privKey: ArrayLike<number>): Promise<BackupDecryptor>;
/**
* Import a list of room keys restored from backup
*
* @param keys - a list of session export objects
* @param opts - options object
* @returns a promise which resolves once the keys have been imported
*/
importBackedUpRoomKeys(keys: IMegolmSessionData[], opts?: ImportRoomKeysOpts): Promise<void>;
}
/** The methods which crypto implementations should expose to the Sync api
+34 -3
View File
@@ -46,6 +46,13 @@ export interface CryptoApi {
*/
getVersion(): string;
/**
* Get the public part of the device keys for the current device.
*
* @returns The public device keys.
*/
getOwnDeviceKeys(): Promise<OwnDeviceKeys>;
/**
* Perform any background tasks that can be done before a message is ready to
* send, in order to speed up sending of the message.
@@ -162,7 +169,7 @@ export interface CryptoApi {
/**
* Mark the given device as locally verified.
*
* Marking a devices as locally verified has much the same effect as completing the verification dance, or receiving
* Marking a device as locally verified has much the same effect as completing the verification dance, or receiving
* a cross-signing signature for it.
*
* @param userId - owner of the device
@@ -175,6 +182,21 @@ export interface CryptoApi {
*/
setDeviceVerified(userId: string, deviceId: string, verified?: boolean): Promise<void>;
/**
* Cross-sign one of our own devices.
*
* This will create a signature for the device using our self-signing key, and publish that signature.
* Cross-signing a device indicates, to our other devices and to other users, that we have verified that it really
* belongs to us.
*
* Requires that cross-signing has been set up on this device (normally by calling {@link bootstrapCrossSigning}.
*
* *Note*: Do not call this unless you have verified, somehow, that the device is genuine!
*
* @param deviceId - ID of the device to be signed.
*/
crossSignDevice(deviceId: string): Promise<void>;
/**
* Checks whether cross signing:
* - is enabled on this account and trusted by this device
@@ -575,9 +597,10 @@ export interface ImportRoomKeyProgressData {
export interface ImportRoomKeysOpts {
/** Reports ongoing progress of the import process. Can be used for feedback. */
progressCallback?: (stage: ImportRoomKeyProgressData) => void;
// TODO, the rust SDK will always such imported keys as untrusted
/** @deprecated the rust SDK will always such imported keys as untrusted */
untrusted?: boolean;
source?: String; // TODO: Enum (backup, file, ??)
/** @deprecated not useful externally */
source?: string;
}
/**
@@ -749,5 +772,13 @@ export enum EventShieldReason {
MISMATCHED_SENDER_KEY,
}
/** The result of a call to {@link CryptoApi.getOwnDeviceKeys} */
export interface OwnDeviceKeys {
/** Public part of the Ed25519 fingerprint key for the current device, base64 encoded. */
ed25519: string;
/** Public part of the Curve25519 identity key for the current device, base64 encoded. */
curve25519: string;
}
export * from "./crypto-api/verification";
export * from "./crypto-api/keybackup";
+8 -1
View File
@@ -29,6 +29,13 @@ export interface IRoomEncryption {
}
/* eslint-enable camelcase */
/**
* Information about the encryption settings of rooms. Loads this information
* from the supplied crypto store when `init()` is called, and saves it to the
* crypto store whenever it is updated via `setRoomEncryption()`. Can supply
* full information about a room's encryption via `getRoomEncryption()`, or just
* answer whether or not a room has encryption via `isRoomEncrypted`.
*/
export class RoomList {
// Object of roomId -> room e2e info object (body of the m.room.encryption event)
private roomEncryption: Record<string, IRoomEncryption> = {};
@@ -43,7 +50,7 @@ export class RoomList {
});
}
public getRoomEncryption(roomId: string): IRoomEncryption {
public getRoomEncryption(roomId: string): IRoomEncryption | null {
return this.roomEncryption[roomId] || null;
}
+83 -2
View File
@@ -64,7 +64,7 @@ import {
IUploadKeySignaturesResponse,
MatrixClient,
} from "../client";
import type { IRoomEncryption, RoomList } from "./RoomList";
import { IRoomEncryption, RoomList } from "./RoomList";
import { IKeyBackupInfo } from "./keybackup";
import { ISyncStateData } from "../sync";
import { CryptoStore } from "./store/base";
@@ -98,6 +98,7 @@ import {
KeyBackupCheck,
KeyBackupInfo,
VerificationRequest as CryptoApiVerificationRequest,
OwnDeviceKeys,
} from "../crypto-api";
import { Device, DeviceMap } from "../models/device";
import { deviceInfoToDevice } from "./device-converter";
@@ -231,6 +232,18 @@ export enum CryptoEvent {
KeyBackupStatus = "crypto.keyBackupStatus",
KeyBackupFailed = "crypto.keyBackupFailed",
KeyBackupSessionsRemaining = "crypto.keyBackupSessionsRemaining",
/**
* Fires when a new valid backup decryption key is in cache.
* This will happen when a secret is received from another session, from secret storage,
* or when a new backup is created from this session.
*
* The payload is the version of the backup for which we have the key for.
*
* This event is only fired by the rust crypto backend.
*/
KeyBackupDecryptionKeyCached = "crypto.keyBackupDecryptionKeyCached",
KeySignatureUploadFailure = "crypto.keySignatureUploadFailure",
/** @deprecated Use `VerificationRequestReceived`. */
VerificationRequest = "crypto.verification.request",
@@ -296,6 +309,13 @@ export type CryptoEventHandlerMap = {
[CryptoEvent.KeyBackupStatus]: (enabled: boolean) => void;
[CryptoEvent.KeyBackupFailed]: (errcode: string) => void;
[CryptoEvent.KeyBackupSessionsRemaining]: (remaining: number) => void;
/**
* Fires when the backup decryption key is received and cached.
*
* @param version - The version of the backup for which we have the key for.
*/
[CryptoEvent.KeyBackupDecryptionKeyCached]: (version: string) => void;
[CryptoEvent.KeySignatureUploadFailure]: (
failures: IUploadKeySignaturesResponse["failures"],
source: "checkOwnCrossSigningTrust" | "afterCrossSigningLocalKeyChange" | "setDeviceVerification",
@@ -365,6 +385,7 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
public readonly dehydrationManager: DehydrationManager;
public readonly secretStorage: LegacySecretStorage;
private readonly roomList: RoomList;
private readonly reEmitter: TypedReEmitter<CryptoEvent, CryptoEventHandlerMap>;
private readonly verificationMethods: Map<VerificationMethod, typeof VerificationBase>;
public readonly supportedAlgorithms: string[];
@@ -453,10 +474,13 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
private readonly deviceId: string,
private readonly clientStore: IStore,
public readonly cryptoStore: CryptoStore,
private readonly roomList: RoomList,
verificationMethods: Array<VerificationMethod | (typeof VerificationBase & { NAME: string })>,
) {
super();
logger.debug("Crypto: initialising roomlist...");
this.roomList = new RoomList(cryptoStore);
this.reEmitter = new TypedReEmitter(this);
if (verificationMethods) {
@@ -606,6 +630,9 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
// (this is important for key backups & things)
this.deviceList.startTrackingDeviceList(this.userId);
logger.debug("Crypto: initialising roomlist...");
await this.roomList.init();
logger.log("Crypto: checking for key backup...");
this.backupManager.checkAndStart();
}
@@ -1191,6 +1218,7 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
await this.storeSessionBackupPrivateKey(privateKey);
await this.backupManager.checkAndStart();
await this.backupManager.scheduleAllGroupSessionsForBackup();
}
/**
@@ -1876,6 +1904,14 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
return new LibOlmBackupDecryptor(algorithm);
}
/**
* Implementation of {@link CryptoBackend#importBackedUpRoomKeys}.
*/
public importBackedUpRoomKeys(keys: IMegolmSessionData[], opts: ImportRoomKeysOpts = {}): Promise<void> {
opts.source = "backup";
return this.importRoomKeys(keys, opts);
}
/**
* Store a set of keys as our own, trusted, cross-signing keys.
*
@@ -1968,6 +2004,8 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
* Get the Ed25519 key for this device
*
* @returns base64-encoded ed25519 key.
*
* @deprecated Use {@link CryptoApi#getOwnDeviceKeys}.
*/
public getDeviceEd25519Key(): string | null {
return this.olmDevice.deviceEd25519Key;
@@ -1977,11 +2015,29 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
* Get the Curve25519 key for this device
*
* @returns base64-encoded curve25519 key.
*
* @deprecated Use {@link CryptoApi#getOwnDeviceKeys}
*/
public getDeviceCurve25519Key(): string | null {
return this.olmDevice.deviceCurve25519Key;
}
/**
* Implementation of {@link CryptoApi#getOwnDeviceKeys}.
*/
public async getOwnDeviceKeys(): Promise<OwnDeviceKeys> {
if (!this.olmDevice.deviceCurve25519Key) {
throw new Error("Curve25519 key not yet created");
}
if (!this.olmDevice.deviceEd25519Key) {
throw new Error("Ed25519 key not yet created");
}
return {
ed25519: this.olmDevice.deviceEd25519Key,
curve25519: this.olmDevice.deviceCurve25519Key,
};
}
/**
* Set the global override for whether the client should ever send encrypted
* messages to unverified devices. This provides the default for rooms which
@@ -2306,6 +2362,15 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
await this.setDeviceVerification(userId, deviceId, verified);
}
/**
* Blindly cross-sign one of our other devices.
*
* Implementation of {@link CryptoApi#crossSignDevice}.
*/
public async crossSignDevice(deviceId: string): Promise<void> {
await this.setDeviceVerified(this.userId, deviceId, true);
}
/**
* Update the blocked/verified state of the given device
*
@@ -4186,6 +4251,22 @@ export class Crypto extends TypedEventEmitter<CryptoEvent, CryptoEventHandlerMap
obj.signatures = recursiveMapToObject(sigs);
if (unsigned !== undefined) obj.unsigned = unsigned;
}
/**
* @returns true if the room with the supplied ID is encrypted. False if the
* room is not encrypted, or is unknown to us.
*/
public isRoomEncrypted(roomId: string): boolean {
return this.roomList.isRoomEncrypted(roomId);
}
/**
* @returns information about the encryption on the room with the supplied
* ID, or null if the room is not encrypted or unknown to us.
*/
public getRoomEncryption(roomId: string): IRoomEncryption | null {
return this.roomList.getRoomEncryption(roomId);
}
}
/**
+1 -1
View File
@@ -50,7 +50,7 @@ export class MatrixHttpApi<O extends IHttpOpts> extends FetchHttpApi<O> {
const abortController = opts.abortController ?? new AbortController();
// If the file doesn't have a mime type, use a default since the HS errors if we don't supply one.
const contentType = opts.type ?? (file as File).type ?? "application/octet-stream";
const contentType = (opts.type ?? (file as File).type) || "application/octet-stream";
const fileName = opts.name ?? (file as File).name;
const upload = {
+50 -9
View File
@@ -18,7 +18,18 @@ limitations under the License.
import loglevel from "loglevel";
/** Logger interface used within the js-sdk codebase */
export interface Logger {
export interface Logger extends BaseLogger {
/**
* Create a child logger.
*
* @param namespace - name to add to the current logger to generate the child. Some implementations of `Logger`
* use this as a prefix; others use a different mechanism.
*/
getChild(namespace: string): Logger;
}
/** The basic interface for a logger which doesn't support children */
interface BaseLogger {
/**
* Output trace message to the logger, with stack trace.
*
@@ -53,14 +64,6 @@ export interface Logger {
* @param msg - Data to log.
*/
error(...msg: any[]): void;
/**
* Create a child logger.
*
* @param namespace - name to add to the current logger to generate the child. Some implementations of `Logger`
* use this as a prefix; others use a different mechanism.
*/
getChild(namespace: string): Logger;
}
// This is to demonstrate, that you can use any namespace you want.
@@ -139,3 +142,41 @@ function getPrefixedLogger(prefix: string): PrefixedLogger {
export const logger = loglevel.getLogger(DEFAULT_NAMESPACE) as PrefixedLogger;
logger.setLevel(loglevel.levels.DEBUG, false);
extendLogger(logger);
/**
* A "span" for grouping related log lines together.
*
* The current implementation just adds the name at the start of each log line.
*
* This offers a lighter-weight alternative to 'child' loggers returned by {@link Logger#getChild}. In particular,
* it's not possible to apply individual filters to the LogSpan such as setting the verbosity level. On the other hand,
* no reference to the LogSpan is retained in the logging framework, so it is safe to make lots of them over the course
* of an application's life and just drop references to them when the job is done.
*/
export class LogSpan implements BaseLogger {
private readonly name;
public constructor(private readonly parent: BaseLogger, name: string) {
this.name = name + ":";
}
public trace(...msg: any[]): void {
this.parent.trace(this.name, ...msg);
}
public debug(...msg: any[]): void {
this.parent.debug(this.name, ...msg);
}
public info(...msg: any[]): void {
this.parent.info(this.name, ...msg);
}
public warn(...msg: any[]): void {
this.parent.warn(this.name, ...msg);
}
public error(...msg: any[]): void {
this.parent.error(this.name, ...msg);
}
}
+1 -1
View File
@@ -91,7 +91,7 @@ export class CallMembership {
}
public isExpired(): boolean {
return this.getAbsoluteExpiry() < this.parentEvent.getTs() + this.parentEvent.getLocalAge();
return this.getMsUntilExpiry() <= 0;
}
public getActiveFoci(): Focus[] {
+346 -4
View File
@@ -22,12 +22,32 @@ import { MatrixClient } from "../client";
import { EventType } from "../@types/event";
import { CallMembership, CallMembershipData } from "./CallMembership";
import { Focus } from "./focus";
import { MatrixEvent } from "../matrix";
import { randomString } from "../randomstring";
import { MatrixError, MatrixEvent } from "../matrix";
import { randomString, secureRandomBase64Url } from "../randomstring";
import { EncryptionKeysEventContent } from "./types";
import { decodeBase64, encodeUnpaddedBase64 } from "../base64";
const MEMBERSHIP_EXPIRY_TIME = 60 * 60 * 1000;
const MEMBER_EVENT_CHECK_PERIOD = 2 * 60 * 1000; // How often we check to see if we need to re-send our member event
const CALL_MEMBER_EVENT_RETRY_DELAY_MIN = 3000;
const UPDATE_ENCRYPTION_KEY_THROTTLE = 3000;
// A delay after a member leaves before we create and publish a new key, because people
// tend to leave calls at the same time
const MAKE_KEY_DELAY = 3000;
// The delay between creating and sending a new key and starting to encrypt with it. This gives others
// a chance to receive the new key to minimise the chance they don't get media they can't decrypt.
// The total time between a member leaving and the call switching to new keys is therefore
// MAKE_KEY_DELAY + SEND_KEY_DELAY
const USE_KEY_DELAY = 5000;
const getParticipantId = (userId: string, deviceId: string): string => `${userId}:${deviceId}`;
const getParticipantIdFromMembership = (m: CallMembership): string => getParticipantId(m.sender!, m.deviceId);
function keysEqual(a: Uint8Array, b: Uint8Array): boolean {
if (a === b) return true;
return a && b && a.length === b.length && a.every((x, i) => x === b[i]);
}
export enum MatrixRTCSessionEvent {
// A member joined, left, or updated a property of their membership.
@@ -36,6 +56,8 @@ export enum MatrixRTCSessionEvent {
// separate from MembershipsChanged, ie. independent of whether our member event
// has succesfully gone through.
JoinStateChanged = "join_state_changed",
// The key used to encrypt media has changed
EncryptionKeyChanged = "encryption_key_changed",
}
export type MatrixRTCSessionEventHandlerMap = {
@@ -44,6 +66,11 @@ export type MatrixRTCSessionEventHandlerMap = {
newMemberships: CallMembership[],
) => void;
[MatrixRTCSessionEvent.JoinStateChanged]: (isJoined: boolean) => void;
[MatrixRTCSessionEvent.EncryptionKeyChanged]: (
key: Uint8Array,
encryptionKeyIndex: number,
participantId: string,
) => void;
};
/**
@@ -51,6 +78,9 @@ export type MatrixRTCSessionEventHandlerMap = {
* This class doesn't deal with media at all, just membership & properties of a session.
*/
export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, MatrixRTCSessionEventHandlerMap> {
// The session Id of the call, this is the call_id of the call Member event.
private _callId: string | undefined;
// How many ms after we joined the call, that our membership should expire, or undefined
// if we're not yet joined
private relativeExpiry: number | undefined;
@@ -65,12 +95,29 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
private memberEventTimeout?: ReturnType<typeof setTimeout>;
private expiryTimeout?: ReturnType<typeof setTimeout>;
private keysEventUpdateTimeout?: ReturnType<typeof setTimeout>;
private makeNewKeyTimeout?: ReturnType<typeof setTimeout>;
private setNewKeyTimeouts = new Set<ReturnType<typeof setTimeout>>();
private activeFoci: Focus[] | undefined;
private updateCallMembershipRunning = false;
private needCallMembershipUpdate = false;
private manageMediaKeys = false;
// userId:deviceId => array of keys
private encryptionKeys = new Map<string, Array<Uint8Array>>();
private lastEncryptionKeyUpdateRequest?: number;
/**
* The callId (sessionId) of the call.
*
* It can be undefined since the callId is only known once the first membership joins.
* The callId is the property that, per definition, groups memberships into one call.
*/
public get callId(): string | undefined {
return this._callId;
}
/**
* Returns all the call memberships for a room, oldest first
*/
@@ -143,6 +190,7 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
public memberships: CallMembership[],
) {
super();
this._callId = memberships[0]?.callId;
this.setExpiryTimer();
}
@@ -175,18 +223,28 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
* This will not subscribe to updates: remember to call subscribe() separately if
* desired.
* This method will return immediately and the session will be joined in the background.
*
* @param activeFoci - The list of foci to set as currently active in the call member event
* @param manageMediaKeys - If true, generate and share a a media key for this participant,
* and emit MatrixRTCSessionEvent.EncryptionKeyChanged when
* media keys for other participants become available.
*/
public joinRoomSession(activeFoci: Focus[]): void {
public joinRoomSession(activeFoci: Focus[], manageMediaKeys?: boolean): void {
if (this.isJoined()) {
logger.info(`Already joined to session in room ${this.room.roomId}: ignoring join call`);
return;
}
logger.info(`Joining call session in room ${this.room.roomId}`);
logger.info(`Joining call session in room ${this.room.roomId} with manageMediaKeys=${manageMediaKeys}`);
this.activeFoci = activeFoci;
this.relativeExpiry = MEMBERSHIP_EXPIRY_TIME;
this.manageMediaKeys = manageMediaKeys ?? false;
this.membershipId = randomString(5);
this.emit(MatrixRTCSessionEvent.JoinStateChanged, true);
if (manageMediaKeys) {
this.makeNewSenderKey();
this.requestKeyEventSend();
}
// We don't wait for this, mostly because it may fail and schedule a retry, so this
// function returning doesn't really mean anything at all.
this.triggerCallMembershipEventUpdate();
@@ -207,9 +265,30 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
return new Promise((resolve) => resolve(false));
}
const userId = this.client.getUserId();
const deviceId = this.client.getDeviceId();
if (!userId) throw new Error("No userId");
if (!deviceId) throw new Error("No deviceId");
// clear our encryption keys as we're done with them now (we'll
// make new keys if we rejoin). We leave keys for other participants
// as they may still be using the same ones.
this.encryptionKeys.set(getParticipantId(userId, deviceId), []);
if (this.makeNewKeyTimeout !== undefined) {
clearTimeout(this.makeNewKeyTimeout);
this.makeNewKeyTimeout = undefined;
}
for (const t of this.setNewKeyTimeouts) {
clearTimeout(t);
}
this.setNewKeyTimeouts.clear();
logger.info(`Leaving call session in room ${this.room.roomId}`);
this.relativeExpiry = undefined;
this.activeFoci = undefined;
this.manageMediaKeys = false;
this.membershipId = undefined;
this.emit(MatrixRTCSessionEvent.JoinStateChanged, false);
@@ -228,6 +307,167 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
});
}
public getKeysForParticipant(userId: string, deviceId: string): Array<Uint8Array> | undefined {
return this.encryptionKeys.get(getParticipantId(userId, deviceId));
}
/**
* A map of keys used to encrypt and decrypt (we are using a symmetric
* cipher) given participant's media. This also includes our own key
*/
public getEncryptionKeys(): IterableIterator<[string, Array<Uint8Array>]> {
return this.encryptionKeys.entries();
}
private getNewEncryptionKeyIndex(): number {
const userId = this.client.getUserId();
const deviceId = this.client.getDeviceId();
if (!userId) throw new Error("No userId!");
if (!deviceId) throw new Error("No deviceId!");
return (this.getKeysForParticipant(userId, deviceId)?.length ?? 0) % 16;
}
/**
* Sets an encryption key at a specified index for a participant.
* The encryption keys for the local participanmt are also stored here under the
* user and device ID of the local participant.
* @param userId - The user ID of the participant
* @param deviceId - Device ID of the participant
* @param encryptionKeyIndex - The index of the key to set
* @param encryptionKeyString - The string represenation of the key to set in base64
* @param delayBeforeuse - If true, delay before emitting a key changed event. Useful when setting
* encryption keys for the local participant to allow time for the key to
* be distributed.
*/
private setEncryptionKey(
userId: string,
deviceId: string,
encryptionKeyIndex: number,
encryptionKeyString: string,
delayBeforeuse = false,
): void {
const keyBin = decodeBase64(encryptionKeyString);
const participantId = getParticipantId(userId, deviceId);
const encryptionKeys = this.encryptionKeys.get(participantId) ?? [];
if (keysEqual(encryptionKeys[encryptionKeyIndex], keyBin)) return;
encryptionKeys[encryptionKeyIndex] = keyBin;
this.encryptionKeys.set(participantId, encryptionKeys);
if (delayBeforeuse) {
const useKeyTimeout = setTimeout(() => {
this.setNewKeyTimeouts.delete(useKeyTimeout);
logger.info(`Delayed-emitting key changed event for ${participantId} idx ${encryptionKeyIndex}`);
this.emit(MatrixRTCSessionEvent.EncryptionKeyChanged, keyBin, encryptionKeyIndex, participantId);
}, USE_KEY_DELAY);
this.setNewKeyTimeouts.add(useKeyTimeout);
} else {
this.emit(MatrixRTCSessionEvent.EncryptionKeyChanged, keyBin, encryptionKeyIndex, participantId);
}
}
/**
* Generate a new sender key and add it at the next available index
* @param delayBeforeUse - If true, wait for a short period before settign the key for the
* media encryptor to use. If false, set the key immediately.
*/
private makeNewSenderKey(delayBeforeUse = false): void {
const userId = this.client.getUserId();
const deviceId = this.client.getDeviceId();
if (!userId) throw new Error("No userId");
if (!deviceId) throw new Error("No deviceId");
const encryptionKey = secureRandomBase64Url(16);
const encryptionKeyIndex = this.getNewEncryptionKeyIndex();
logger.info("Generated new key at index " + encryptionKeyIndex);
this.setEncryptionKey(userId, deviceId, encryptionKeyIndex, encryptionKey, delayBeforeUse);
}
/**
* Requests that we resend our keys to the room. May send a keys event immediately
* or queue for alter if one has already been sent recently.
*/
private requestKeyEventSend(): void {
if (!this.manageMediaKeys) return;
if (
this.lastEncryptionKeyUpdateRequest &&
this.lastEncryptionKeyUpdateRequest + UPDATE_ENCRYPTION_KEY_THROTTLE > Date.now()
) {
logger.info("Last encryption key event sent too recently: postponing");
if (this.keysEventUpdateTimeout === undefined) {
this.keysEventUpdateTimeout = setTimeout(this.sendEncryptionKeysEvent, UPDATE_ENCRYPTION_KEY_THROTTLE);
}
return;
}
this.sendEncryptionKeysEvent();
}
/**
* Re-sends the encryption keys room event
*/
private sendEncryptionKeysEvent = async (): Promise<void> => {
if (this.keysEventUpdateTimeout !== undefined) {
clearTimeout(this.keysEventUpdateTimeout);
this.keysEventUpdateTimeout = undefined;
}
this.lastEncryptionKeyUpdateRequest = Date.now();
logger.info("Sending encryption keys event");
if (!this.isJoined()) return;
const userId = this.client.getUserId();
const deviceId = this.client.getDeviceId();
if (!userId) throw new Error("No userId");
if (!deviceId) throw new Error("No deviceId");
const myKeys = this.getKeysForParticipant(userId, deviceId);
if (!myKeys) {
logger.warn("Tried to send encryption keys event but no keys found!");
return;
}
try {
await this.client.sendEvent(this.room.roomId, EventType.CallEncryptionKeysPrefix, {
keys: myKeys.map((key, index) => {
return {
index,
key: encodeUnpaddedBase64(key),
};
}),
device_id: deviceId,
call_id: "",
} as EncryptionKeysEventContent);
logger.debug(
`Embedded-E2EE-LOG updateEncryptionKeyEvent participantId=${userId}:${deviceId} numSent=${myKeys.length}`,
this.encryptionKeys,
);
} catch (error) {
const matrixError = error as MatrixError;
if (matrixError.event) {
// cancel the pending event: we'll just generate a new one with our latest
// keys when we resend
this.client.cancelPendingEvent(matrixError.event);
}
if (this.keysEventUpdateTimeout === undefined) {
const resendDelay = matrixError.data?.retry_after_ms ?? 5000;
logger.warn(`Failed to send m.call.encryption_key, retrying in ${resendDelay}`, error);
this.keysEventUpdateTimeout = setTimeout(this.sendEncryptionKeysEvent, resendDelay);
} else {
logger.info("Not scheduling key resend as another re-send is already pending");
}
}
};
/**
* Sets a timer for the soonest membership expiry
*/
@@ -254,10 +494,78 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
return this.memberships[0];
}
public onCallEncryption = (event: MatrixEvent): void => {
const userId = event.getSender();
const content = event.getContent<EncryptionKeysEventContent>();
const deviceId = content["device_id"];
const callId = content["call_id"];
if (!userId) {
logger.warn(`Received m.call.encryption_keys with no userId: callId=${callId}`);
return;
}
// We currently only handle callId = ""
if (callId !== "") {
logger.warn(
`Received m.call.encryption_keys with unsupported callId: userId=${userId}, deviceId=${deviceId}, callId=${callId}`,
);
return;
}
if (!Array.isArray(content.keys)) {
logger.warn(`Received m.call.encryption_keys where keys wasn't an array: callId=${callId}`);
return;
}
if (userId === this.client.getUserId() && deviceId === this.client.getDeviceId()) {
// We store our own sender key in the same set along with keys from others, so it's
// important we don't allow our own keys to be set by one of these events (apart from
// the fact that we don't need it anyway because we already know our own keys).
logger.info("Ignoring our own keys event");
return;
}
for (const key of content.keys) {
if (!key) {
logger.info("Ignoring false-y key in keys event");
continue;
}
const encryptionKey = key.key;
const encryptionKeyIndex = key.index;
if (
!encryptionKey ||
encryptionKeyIndex === undefined ||
encryptionKeyIndex === null ||
callId === undefined ||
callId === null ||
typeof deviceId !== "string" ||
typeof callId !== "string" ||
typeof encryptionKey !== "string" ||
typeof encryptionKeyIndex !== "number"
) {
logger.warn(
`Malformed call encryption_key: userId=${userId}, deviceId=${deviceId}, encryptionKeyIndex=${encryptionKeyIndex} callId=${callId}`,
);
} else {
logger.debug(
`Embedded-E2EE-LOG onCallEncryption userId=${userId}:${deviceId} encryptionKeyIndex=${encryptionKeyIndex}`,
this.encryptionKeys,
);
this.setEncryptionKey(userId, deviceId, encryptionKeyIndex, encryptionKey);
}
}
};
public onMembershipUpdate = (): void => {
const oldMemberships = this.memberships;
this.memberships = MatrixRTCSession.callMembershipsForRoom(this.room);
this._callId = this._callId ?? this.memberships[0]?.callId;
const changed =
oldMemberships.length != this.memberships.length ||
oldMemberships.some((m, i) => !CallMembership.equal(m, this.memberships[i]));
@@ -267,6 +575,29 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
this.emit(MatrixRTCSessionEvent.MembershipsChanged, oldMemberships, this.memberships);
}
const isMyMembership = (m: CallMembership): boolean =>
m.sender === this.client.getUserId() && m.deviceId === this.client.getDeviceId();
if (this.manageMediaKeys && this.isJoined() && this.makeNewKeyTimeout === undefined) {
const oldMebershipIds = new Set(
oldMemberships.filter((m) => !isMyMembership(m)).map(getParticipantIdFromMembership),
);
const newMebershipIds = new Set(
this.memberships.filter((m) => !isMyMembership(m)).map(getParticipantIdFromMembership),
);
const anyLeft = Array.from(oldMebershipIds).some((x) => !newMebershipIds.has(x));
const anyJoined = Array.from(newMebershipIds).some((x) => !oldMebershipIds.has(x));
if (anyLeft) {
logger.debug(`Member(s) have left: queueing sender key rotation`);
this.makeNewKeyTimeout = setTimeout(this.onRotateKeyTimeout, MAKE_KEY_DELAY);
} else if (anyJoined) {
logger.debug(`New member(s) have joined: re-sending keys`);
this.requestKeyEventSend();
}
}
this.setExpiryTimer();
};
@@ -449,4 +780,15 @@ export class MatrixRTCSession extends TypedEventEmitter<MatrixRTCSessionEvent, M
await this.triggerCallMembershipEventUpdate();
}
}
private onRotateKeyTimeout = (): void => {
if (!this.manageMediaKeys) return;
this.makeNewKeyTimeout = undefined;
logger.info("Making new sender key for key rotation");
this.makeNewSenderKey(true);
// send immediately: if we're about to start sending with a new key, it's
// important we get it out to others as soon as we can.
this.sendEncryptionKeysEvent();
};
}
+19 -2
View File
@@ -17,10 +17,11 @@ limitations under the License.
import { logger } from "../logger";
import { MatrixClient, ClientEvent } from "../client";
import { TypedEventEmitter } from "../models/typed-event-emitter";
import { Room } from "../models/room";
import { Room, RoomEvent } from "../models/room";
import { RoomState, RoomStateEvent } from "../models/room-state";
import { MatrixEvent } from "../models/event";
import { MatrixRTCSession } from "./MatrixRTCSession";
import { EventType } from "../@types/event";
export enum MatrixRTCSessionManagerEvents {
// A member has joined the MatrixRTC session, creating an active session in a room where there wasn't previously
@@ -62,6 +63,7 @@ export class MatrixRTCSessionManager extends TypedEventEmitter<MatrixRTCSessionM
}
this.client.on(ClientEvent.Room, this.onRoom);
this.client.on(RoomEvent.Timeline, this.onTimeline);
this.client.on(RoomStateEvent.Events, this.onRoomState);
}
@@ -72,6 +74,7 @@ export class MatrixRTCSessionManager extends TypedEventEmitter<MatrixRTCSessionM
this.roomSessions.clear();
this.client.removeListener(ClientEvent.Room, this.onRoom);
this.client.removeListener(RoomEvent.Timeline, this.onTimeline);
this.client.removeListener(RoomStateEvent.Events, this.onRoomState);
}
@@ -95,6 +98,18 @@ export class MatrixRTCSessionManager extends TypedEventEmitter<MatrixRTCSessionM
return this.roomSessions.get(room.roomId)!;
}
private onTimeline = (event: MatrixEvent): void => {
if (event.getType() !== EventType.CallEncryptionKeysPrefix) return;
const room = this.client.getRoom(event.getRoomId());
if (!room) {
logger.error(`Got room state event for unknown room ${event.getRoomId()}!`);
return;
}
this.getRoomSession(room).onCallEncryption(event);
};
private onRoom = (room: Room): void => {
this.refreshRoom(room);
};
@@ -106,7 +121,9 @@ export class MatrixRTCSessionManager extends TypedEventEmitter<MatrixRTCSessionM
return;
}
this.refreshRoom(room);
if (event.getType() == EventType.GroupCallMemberPrefix) {
this.refreshRoom(room);
}
};
private refreshRoom(room: Room): void {
+35
View File
@@ -0,0 +1,35 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { IMentions } from "../matrix";
export interface EncryptionKeyEntry {
index: number;
key: string;
}
export interface EncryptionKeysEventContent {
keys: EncryptionKeyEntry[];
device_id: string;
call_id: string;
}
export type CallNotifyType = "ring" | "notify";
export interface ICallNotifyContent {
"application": string;
"m.mentions": IMentions;
"notify_type": CallNotifyType;
"call_id": string;
}
+2 -3
View File
@@ -162,9 +162,8 @@ export class MSC3089Branch {
if (!event) throw new Error("Failed to find event");
// Sometimes the event isn't decrypted for us, so do that. We specifically set `emit: true`
// to ensure that the relations system in the sdk will function.
await this.client.decryptEventIfNeeded(event, { emit: true, isRetry: true });
// Sometimes the event isn't decrypted for us, so do that.
await this.client.decryptEventIfNeeded(event);
return event;
}
+139
View File
@@ -0,0 +1,139 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { MatrixEvent } from "./event";
import { Room } from "./room";
import { inMainTimelineForReceipt, threadIdForReceipt } from "../client";
/**
* Determine the order of two events in a room.
*
* In principle this should use the same order as the server, but in practice
* this is difficult for events that were not received over the Sync API. See
* MSC4033 for details.
*
* This implementation leans on the order of events within their timelines, and
* falls back to comparing event timestamps when they are in different
* timelines.
*
* See https://github.com/matrix-org/matrix-js-sdk/issues/3325 for where we are
* tracking the work to fix this.
*
* @param room - the room we are looking in
* @param leftEventId - the id of the first event
* @param rightEventId - the id of the second event
* @returns -1 if left \< right, 1 if left \> right, 0 if left == right, null if
* we can't tell (because we can't find the events).
*/
export function compareEventOrdering(room: Room, leftEventId: string, rightEventId: string): number | null {
const leftEvent = room.findEventById(leftEventId);
const rightEvent = room.findEventById(rightEventId);
if (!leftEvent || !rightEvent) {
// Without the events themselves, we can't find their thread or
// timeline, or guess based on timestamp, so we just don't know.
return null;
}
// Check whether the events are in the main timeline
const isLeftEventInMainTimeline = inMainTimelineForReceipt(leftEvent);
const isRightEventInMainTimeline = inMainTimelineForReceipt(rightEvent);
if (isLeftEventInMainTimeline && isRightEventInMainTimeline) {
return compareEventsInMainTimeline(room, leftEventId, rightEventId, leftEvent, rightEvent);
} else {
// At least one event is not in the timeline, so we can't use the room's
// unfiltered timeline set.
return compareEventsInThreads(leftEventId, rightEventId, leftEvent, rightEvent);
}
}
function compareEventsInMainTimeline(
room: Room,
leftEventId: string,
rightEventId: string,
leftEvent: MatrixEvent,
rightEvent: MatrixEvent,
): number | null {
// Get the timeline set that contains all the events.
const timelineSet = room.getUnfilteredTimelineSet();
// If they are in the same timeline, compareEventOrdering does what we need
const compareSameTimeline = timelineSet.compareEventOrdering(leftEventId, rightEventId);
if (compareSameTimeline !== null) {
return compareSameTimeline;
}
// Find which timeline each event is in. Refuse to provide an ordering if we
// can't find either of the events.
const leftTimeline = timelineSet.getTimelineForEvent(leftEventId);
if (leftTimeline === timelineSet.getLiveTimeline()) {
// The left event is part of the live timeline, so it must be after the
// right event (since they are not in the same timeline or we would have
// returned after compareEventOrdering.
return 1;
}
const rightTimeline = timelineSet.getTimelineForEvent(rightEventId);
if (rightTimeline === timelineSet.getLiveTimeline()) {
// The right event is part of the live timeline, so it must be after the
// left event.
return -1;
}
// They are in older timeline sets (because they were fetched by paging up).
return guessOrderBasedOnTimestamp(leftEvent, rightEvent);
}
function compareEventsInThreads(
leftEventId: string,
rightEventId: string,
leftEvent: MatrixEvent,
rightEvent: MatrixEvent,
): number | null {
const leftEventThreadId = threadIdForReceipt(leftEvent);
const rightEventThreadId = threadIdForReceipt(rightEvent);
const leftThread = leftEvent.getThread();
if (leftThread && leftEventThreadId === rightEventThreadId) {
// They are in the same thread, so we can ask the thread's timeline to
// figure it out for us
return leftThread.timelineSet.compareEventOrdering(leftEventId, rightEventId);
} else {
return guessOrderBasedOnTimestamp(leftEvent, rightEvent);
}
}
/**
* Guess the order of events based on server timestamp. This is not good, but
* difficult to avoid without MSC4033.
*
* See https://github.com/matrix-org/matrix-js-sdk/issues/3325
*/
function guessOrderBasedOnTimestamp(leftEvent: MatrixEvent, rightEvent: MatrixEvent): number {
const leftTs = leftEvent.getTs();
const rightTs = rightEvent.getTs();
if (leftTs < rightTs) {
return -1;
} else if (leftTs > rightTs) {
return 1;
} else {
return 0;
}
}
+18 -7
View File
@@ -839,7 +839,10 @@ export class EventTimelineSet extends TypedEventEmitter<EmittedEvents, EventTime
const data: IRoomTimelineData = {
timeline: timeline,
liveEvent: timeline == this.liveTimeline,
// The purpose of this method is inserting events in the middle of the
// timeline, so the events are, by definition, not live (whether or not
// we're adding them to the live timeline).
liveEvent: false,
};
this.emit(RoomEvent.Timeline, event, this.room, false, false, data);
}
@@ -899,11 +902,10 @@ export class EventTimelineSet extends TypedEventEmitter<EmittedEvents, EventTime
* @param eventId1 - The id of the first event
* @param eventId2 - The id of the second event
* @returns a number less than zero if eventId1 precedes eventId2, and
* greater than zero if eventId1 succeeds eventId2. zero if they are the
* same event; null if we can't tell (either because we don't know about one
* of the events, or because they are in separate timelines which don't join
* up).
* @returns -1 if eventId1 precedes eventId2, and +1 eventId1 succeeds
* eventId2. 0 if they are the same event; null if we can't tell (either
* because we don't know about one of the events, or because they are in
* separate timelines which don't join up).
*/
public compareEventOrdering(eventId1: string, eventId2: string): number | null {
if (eventId1 == eventId2) {
@@ -935,7 +937,16 @@ export class EventTimelineSet extends TypedEventEmitter<EmittedEvents, EventTime
idx2 = idx;
}
}
return idx1! - idx2!;
const difference = idx1! - idx2!;
// Return the sign of difference.
if (difference < 0) {
return -1;
} else if (difference > 0) {
return 1;
} else {
return 0;
}
}
// the events are in different timelines. Iterate through the
+65 -5
View File
@@ -45,6 +45,8 @@ import { DecryptionError } from "../crypto/algorithms";
import { CryptoBackend } from "../common-crypto/CryptoBackend";
import { WITHHELD_MESSAGES } from "../crypto/OlmDevice";
import { IAnnotatedPushRule } from "../@types/PushRules";
import { Room } from "./room";
import { EventTimeline } from "./event-timeline";
export { EventStatus } from "./event-status";
@@ -175,11 +177,23 @@ interface IKeyRequestRecipient {
}
export interface IDecryptOptions {
// Emits "event.decrypted" if set to true
/** Whether to emit {@link MatrixEventEvent.Decrypted} events on successful decryption. Defaults to true.
*/
emit?: boolean;
// True if this is a retry (enables more logging)
/**
* True if this is a retry, after receiving an update to the session key. (Enables more logging.)
*
* This is only intended for use within the js-sdk.
*
* @internal
*/
isRetry?: boolean;
// whether the message should be re-decrypted if it was previously successfully decrypted with an untrusted key
/**
* Whether the message should be re-decrypted if it was previously successfully decrypted with an untrusted key.
* Defaults to `false`.
*/
forceRedecryptIfUntrusted?: boolean;
}
@@ -390,7 +404,13 @@ export class MatrixEvent extends TypedEventEmitter<MatrixEventEmittedEvents, Mat
});
this.txnId = event.txn_id;
this.localTimestamp = Date.now() - (this.getAge() ?? 0);
// The localTimestamp is calculated using the age.
// Some events lack an `age` property, either because they are EDUs such as typing events,
// or due to server-side bugs such as https://github.com/matrix-org/synapse/issues/8429.
// The fallback in these cases will be to use the origin_server_ts.
// For EDUs, the origin_server_ts also is not defined so we use Date.now().
const age = this.getAge();
this.localTimestamp = age !== undefined ? Date.now() - age : this.getTs() ?? Date.now();
this.reEmitter = new TypedReEmitter(this);
}
@@ -1135,13 +1155,19 @@ export class MatrixEvent extends TypedEventEmitter<MatrixEventEmittedEvents, Mat
return this.visibility;
}
/**
* @deprecated In favor of the overload that includes a Room argument
*/
public makeRedacted(redactionEvent: MatrixEvent): void;
/**
* Update the content of an event in the same way it would be by the server
* if it were redacted before it was sent to us
*
* @param redactionEvent - event causing the redaction
* @param room - the room in which the event exists
*/
public makeRedacted(redactionEvent: MatrixEvent): void {
public makeRedacted(redactionEvent: MatrixEvent, room: Room): void;
public makeRedacted(redactionEvent: MatrixEvent, room?: Room): void {
// quick sanity-check
if (!redactionEvent.event) {
throw new Error("invalid redactionEvent in makeRedacted");
@@ -1185,9 +1211,43 @@ export class MatrixEvent extends TypedEventEmitter<MatrixEventEmittedEvents, Mat
}
}
// If the redacted event was in a thread (but not thread root), move it
// to the main timeline. This will change if MSC3389 is merged.
if (room && !this.isThreadRoot && this.threadRootId && this.threadRootId !== this.getId()) {
this.moveAllRelatedToMainTimeline(room);
redactionEvent.moveToMainTimeline(room);
}
this.invalidateExtensibleEvent();
}
private moveAllRelatedToMainTimeline(room: Room): void {
const thread = this.thread;
this.moveToMainTimeline(room);
// If we dont have access to the thread, we can only move this
// event, not things related to it.
if (thread) {
for (const event of thread.events) {
if (event.getRelation()?.event_id === this.getId()) {
event.moveAllRelatedToMainTimeline(room);
}
}
}
}
private moveToMainTimeline(room: Room): void {
// Remove it from its thread
this.thread?.timelineSet.removeEvent(this.getId()!);
this.setThread(undefined);
// And insert it into the main timeline
const timeline = room.getLiveTimeline();
// We use insertEventIntoTimeline to insert it in timestamp order,
// because we don't know where it should go (until we have MSC4033).
timeline.getTimelineSet().insertEventIntoTimeline(this, timeline, timeline.getState(EventTimeline.FORWARDS)!);
}
/**
* Check if this event has been redacted
*
+137 -38
View File
@@ -26,15 +26,30 @@ import { EventType } from "../@types/event";
import { EventTimelineSet } from "./event-timeline-set";
import { MapWithDefault } from "../utils";
import { NotificationCountType } from "./room";
import { logger } from "../logger";
import { inMainTimelineForReceipt, threadIdForReceipt } from "../client";
export function synthesizeReceipt(userId: string, event: MatrixEvent, receiptType: ReceiptType): MatrixEvent {
/**
* Create a synthetic receipt for the given event
* @param userId - The user ID if the receipt sender
* @param event - The event that is to be acknowledged
* @param receiptType - The type of receipt
* @param unthreaded - the receipt is unthreaded
* @returns a new event with the synthetic receipt in it
*/
export function synthesizeReceipt(
userId: string,
event: MatrixEvent,
receiptType: ReceiptType,
unthreaded = false,
): MatrixEvent {
return new MatrixEvent({
content: {
[event.getId()!]: {
[receiptType]: {
[userId]: {
ts: event.getTs(),
thread_id: event.threadRootId ?? MAIN_ROOM_TIMELINE,
...(!unthreaded && { thread_id: threadIdForReceipt(event) }),
},
},
},
@@ -94,15 +109,115 @@ export abstract class ReadReceipt<
}
/**
* Get the ID of the event that a given user has read up to, or null if we
* have received no read receipts from them.
* Get the ID of the event that a given user has read up to, or null if:
* - we have received no read receipts for them, or
* - the receipt we have points at an event we don't have, or
* - the thread ID in the receipt does not match the thread root of the
* referenced event.
*
* (The event might not exist if it is not loaded, and the thread ID might
* not match if the event has moved thread because it was redacted.)
*
* @param userId - The user ID to get read receipt event ID for
* @param ignoreSynthesized - If true, return only receipts that have been
* sent by the server, not implicit ones generated
* by the JS SDK.
* @returns ID of the latest event that the given user has read, or null.
* sent by the server, not implicit ones generated
* by the JS SDK.
* @returns ID of the latest existing event that the given user has read, or null.
*/
public getEventReadUpTo(userId: string, ignoreSynthesized = false): string | null {
// Find what the latest receipt says is the latest event we have read
const latestReceipt = this.getLatestReceipt(userId, ignoreSynthesized);
if (!latestReceipt) {
return null;
}
return this.receiptPointsAtConsistentEvent(latestReceipt) ? latestReceipt.eventId : null;
}
/**
* Returns true if the event pointed at by this receipt exists, and its
* threadRootId is consistent with the thread information in the receipt.
*/
private receiptPointsAtConsistentEvent(receipt: WrappedReceipt): boolean {
const event = this.findEventById(receipt.eventId);
if (!event) {
// If the receipt points at a non-existent event, we have multiple
// possibilities:
//
// 1. We don't have the event because it's not loaded yet - probably
// it's old and we're best off ignoring the receipt - we can just
// send a new one when we read a new event.
//
// 2. We have a bug e.g. we misclassified this event into the wrong
// thread.
//
// 3. The referenced event moved out of this thread (e.g. because it
// was deleted.)
//
// 4. The receipt had the incorrect thread ID (due to a bug in a
// client, or malicious behaviour).
// This receipt is not "valid" because it doesn't point at an event
// we have. We want to pretend it doesn't exist.
return false;
}
if (!receipt.data?.thread_id) {
// If this is an unthreaded receipt, it could point at any event, so
// there is no need to validate further - this receipt is valid.
return true;
}
// Otherwise it is a threaded receipt...
if (receipt.data.thread_id === MAIN_ROOM_TIMELINE) {
// The receipt is for the main timeline: we check that the event is
// in the main timeline.
// Check if the event is in the main timeline
const eventIsInMainTimeline = inMainTimelineForReceipt(event);
if (eventIsInMainTimeline) {
// The receipt is for the main timeline, and so is the event, so
// the receipt is valid.
return true;
}
} else {
// The receipt is for a different thread (not the main timeline)
if (event.threadRootId === receipt.data.thread_id) {
// If the receipt and event agree on the thread ID, the receipt
// is valid.
return true;
}
}
// The receipt thread ID disagrees with the event thread ID. There are 2
// possibilities:
//
// 1. The event moved to a different thread after the receipt was
// created. This can happen if the event was redacted because that
// moves it to the main timeline.
//
// 2. There is a bug somewhere - either we put the event into the wrong
// thread, or someone sent an incorrect receipt.
//
// In many cases, we won't get here because the call to findEventById
// would have already returned null. We include this check to cover
// cases when `this` is a room, meaning findEventById will find events
// in any thread, and to be defensive against unforeseen code paths.
logger.warn(
`Ignoring receipt because its thread_id (${receipt.data.thread_id}) disagrees ` +
`with the thread root (${event.threadRootId}) of the referenced event ` +
`(event ID = ${receipt.eventId})`,
);
// This receipt is not "valid" because it disagrees with us about what
// thread the event is in. We want to pretend it doesn't exist.
return false;
}
private getLatestReceipt(userId: string, ignoreSynthesized: boolean): WrappedReceipt | null {
// XXX: This is very very ugly and I hope I won't have to ever add a new
// receipt type here again. IMHO this should be done by the server in
// some more intelligent manner or the client should just use timestamps
@@ -118,10 +233,10 @@ export abstract class ReadReceipt<
// The public receipt is more likely to drift out of date so the private
// one has precedence
if (!comparison) return privateReadReceipt?.eventId ?? publicReadReceipt?.eventId ?? null;
if (!comparison) return privateReadReceipt ?? publicReadReceipt ?? null;
// If public read receipt is older, return the private one
return (comparison < 0 ? privateReadReceipt?.eventId : publicReadReceipt?.eventId) ?? null;
return (comparison < 0 ? privateReadReceipt : publicReadReceipt) ?? null;
}
public addReceiptToStructure(
@@ -229,6 +344,13 @@ export abstract class ReadReceipt<
public abstract setUnread(type: NotificationCountType, count: number): void;
/**
* Look in this room/thread's timeline to find an event. If `this` is a
* room, we look in all threads, but if `this` is a thread, we look only
* inside this thread.
*/
public abstract findEventById(eventId: string): MatrixEvent | undefined;
/**
* This issue should also be addressed on synapse's side and is tracked as part
* of https://github.com/matrix-org/synapse/issues/14837
@@ -256,9 +378,10 @@ export abstract class ReadReceipt<
* @param userId - The user ID if the receipt sender
* @param e - The event that is to be acknowledged
* @param receiptType - The type of receipt
* @param unthreaded - the receipt is unthreaded
*/
public addLocalEchoReceipt(userId: string, e: MatrixEvent, receiptType: ReceiptType): void {
this.addReceipt(synthesizeReceipt(userId, e, receiptType), true);
public addLocalEchoReceipt(userId: string, e: MatrixEvent, receiptType: ReceiptType, unthreaded = false): void {
this.addReceipt(synthesizeReceipt(userId, e, receiptType, unthreaded), true);
}
/**
@@ -284,33 +407,7 @@ export abstract class ReadReceipt<
* @param eventId - The event ID to check if the user read.
* @returns True if the user has read the event, false otherwise.
*/
public hasUserReadEvent(userId: string, eventId: string): boolean {
const readUpToId = this.getEventReadUpTo(userId, false);
if (readUpToId === eventId) return true;
if (
this.timeline?.length &&
this.timeline[this.timeline.length - 1].getSender() &&
this.timeline[this.timeline.length - 1].getSender() === userId
) {
// It doesn't matter where the event is in the timeline, the user has read
// it because they've sent the latest event.
return true;
}
for (let i = this.timeline?.length - 1; i >= 0; --i) {
const ev = this.timeline[i];
// If we encounter the target event first, the user hasn't read it
// however if we encounter the readUpToId first then the user has read
// it. These rules apply because we're iterating bottom-up.
if (ev.getId() === eventId) return false;
if (ev.getId() === readUpToId) return true;
}
// We don't know if the user has read it, so assume not.
return false;
}
public abstract hasUserReadEvent(userId: string, eventId: string): boolean;
/**
* Returns the most recent unthreaded receipt for a given user
@@ -318,6 +415,8 @@ export abstract class ReadReceipt<
* @returns an unthreaded Receipt. Can be undefined if receipts have been disabled
* or a user chooses to use private read receipts (or we have simply not received
* a receipt from this user yet).
*
* @deprecated use `hasUserReadEvent` or `getEventReadUpTo` instead
*/
public abstract getLastUnthreadedReceiptFor(userId: string): Receipt | undefined;
}
+435
View File
@@ -0,0 +1,435 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { MAIN_ROOM_TIMELINE, Receipt, ReceiptContent } from "../@types/read_receipts";
import { threadIdForReceipt } from "../client";
import { Room, RoomEvent } from "./room";
import { MatrixEvent } from "./event";
import { logger } from "../logger";
/**
* The latest receipts we have for a room.
*/
export class RoomReceipts {
private room: Room;
private threadedReceipts: ThreadedReceipts;
private unthreadedReceipts: ReceiptsByUser;
private danglingReceipts: DanglingReceipts;
public constructor(room: Room) {
this.room = room;
this.threadedReceipts = new ThreadedReceipts(room);
this.unthreadedReceipts = new ReceiptsByUser(room);
this.danglingReceipts = new DanglingReceipts();
// We listen for timeline events so we can process dangling receipts
room.on(RoomEvent.Timeline, this.onTimelineEvent);
}
/**
* Remember the receipt information supplied. For each receipt:
*
* If we don't have the event for this receipt, store it as "dangling" so we
* can process it later.
*
* Otherwise store it per-user in either the threaded store for its
* thread_id, or the unthreaded store if there is no thread_id.
*
* Ignores any receipt that is before an existing receipt for the same user
* (in the same thread, if applicable). "Before" is defined by the
* unfilteredTimelineSet of the room.
*/
public add(receiptContent: ReceiptContent, synthetic: boolean): void {
/*
Transform this structure:
{
"$EVENTID": {
"m.read|m.read.private": {
"@user:example.org": {
"ts": 1661,
"thread_id": "main|$THREAD_ROOT_ID" // or missing/undefined for an unthreaded receipt
}
}
},
...
}
into maps of:
threaded :: threadid :: userId :: ReceiptInfo
unthreaded :: userId :: ReceiptInfo
dangling :: eventId :: DanglingReceipt
*/
for (const [eventId, eventReceipt] of Object.entries(receiptContent)) {
for (const [receiptType, receiptsByUser] of Object.entries(eventReceipt)) {
for (const [userId, receipt] of Object.entries(receiptsByUser)) {
const referencedEvent = this.room.findEventById(eventId);
if (!referencedEvent) {
this.danglingReceipts.add(
new DanglingReceipt(eventId, receiptType, userId, receipt, synthetic),
);
} else if (receipt.thread_id) {
this.threadedReceipts.set(
receipt.thread_id,
eventId,
receiptType,
userId,
receipt.ts,
synthetic,
);
} else {
this.unthreadedReceipts.set(eventId, receiptType, userId, receipt.ts, synthetic);
}
}
}
}
}
/**
* Look for dangling receipts for the given event ID,
* and add them to the thread of unthread receipts if found.
* @param eventId - the event ID to look for
*/
private onTimelineEvent = (event: MatrixEvent): void => {
const eventId = event.getId();
if (!eventId) return;
const danglingReceipts = this.danglingReceipts.remove(eventId);
danglingReceipts?.forEach((danglingReceipt) => {
// The receipt is a thread receipt
if (danglingReceipt.receipt.thread_id) {
this.threadedReceipts.set(
danglingReceipt.receipt.thread_id,
danglingReceipt.eventId,
danglingReceipt.receiptType,
danglingReceipt.userId,
danglingReceipt.receipt.ts,
danglingReceipt.synthetic,
);
} else {
this.unthreadedReceipts.set(
eventId,
danglingReceipt.receiptType,
danglingReceipt.userId,
danglingReceipt.receipt.ts,
danglingReceipt.synthetic,
);
}
});
};
public hasUserReadEvent(userId: string, eventId: string): boolean {
const unthreaded = this.unthreadedReceipts.get(userId);
if (unthreaded) {
if (isAfterOrSame(unthreaded.eventId, eventId, this.room)) {
// The unthreaded receipt is after this event, so we have read it.
return true;
}
}
const event = this.room.findEventById(eventId);
if (!event) {
// We don't know whether the user has read it - default to caution and say no.
// This shouldn't really happen and feels like it ought to be an exception: let's
// log a warn for now.
logger.warn(
`hasUserReadEvent event ID ${eventId} not found in room ${this.room.roomId}: this shouldn't happen!`,
);
return false;
}
const threadId = threadIdForReceipt(event);
const threaded = this.threadedReceipts.get(threadId, userId);
if (threaded) {
if (isAfterOrSame(threaded.eventId, eventId, this.room)) {
// The threaded receipt is after this event, so we have read it.
return true;
}
}
// TODO: what if they sent the second-last event in the thread?
if (this.userSentLatestEventInThread(threadId, userId)) {
// The user sent the latest message in this event's thread, so we
// consider everything in the thread to be read.
//
// Note: maybe we don't need this because synthetic receipts should
// do this job for us?
return true;
}
// Neither of the receipts were after the event, so it's unread.
return false;
}
/**
* @returns true if the thread with this ID can be found, and the supplied
* user sent the latest message in it.
*/
private userSentLatestEventInThread(threadId: string, userId: String): boolean {
const timeline =
threadId === MAIN_ROOM_TIMELINE
? this.room.getLiveTimeline().getEvents()
: this.room.getThread(threadId)?.timeline;
return !!(timeline && timeline.length > 0 && timeline[timeline.length - 1].getSender() === userId);
}
}
// --- implementation details ---
/**
* The information "inside" a receipt once it has been stored inside
* RoomReceipts - what eventId it refers to, its type, and its ts.
*
* Does not contain userId or threadId since these are stored as keys of the
* maps in RoomReceipts.
*/
class ReceiptInfo {
public constructor(public eventId: string, public receiptType: string, public ts: number) {}
}
/**
* Everything we know about a receipt that is "dangling" because we can't find
* the event to which it refers.
*/
class DanglingReceipt {
public constructor(
public eventId: string,
public receiptType: string,
public userId: string,
public receipt: Receipt,
public synthetic: boolean,
) {}
}
class UserReceipts {
private room: Room;
/**
* The real receipt for this user.
*/
private real: ReceiptInfo | undefined;
/**
* The synthetic receipt for this user. If this is defined, it is later than real.
*/
private synthetic: ReceiptInfo | undefined;
public constructor(room: Room) {
this.room = room;
this.real = undefined;
this.synthetic = undefined;
}
public set(synthetic: boolean, receiptInfo: ReceiptInfo): void {
if (synthetic) {
this.synthetic = receiptInfo;
} else {
this.real = receiptInfo;
}
// Preserve the invariant: synthetic is only defined if it's later than real
if (this.synthetic && this.real) {
if (isAfterOrSame(this.real.eventId, this.synthetic.eventId, this.room)) {
this.synthetic = undefined;
}
}
}
/**
* Return the latest receipt we have - synthetic if we have one (and it's
* later), otherwise real.
*/
public get(): ReceiptInfo | undefined {
// Relies on the invariant that synthetic is only defined if it's later than real.
return this.synthetic ?? this.real;
}
/**
* Return the latest receipt we have of the specified type (synthetic or not).
*/
public getByType(synthetic: boolean): ReceiptInfo | undefined {
return synthetic ? this.synthetic : this.real;
}
}
/**
* The latest receipt info we have, either for a single thread, or all the
* unthreaded receipts for a room.
*
* userId: ReceiptInfo
*/
class ReceiptsByUser {
private room: Room;
/** map of userId: UserReceipts */
private data: Map<String, UserReceipts>;
public constructor(room: Room) {
this.room = room;
this.data = new Map<string, UserReceipts>();
}
/**
* Add the supplied receipt to our structure, if it is not earlier than the
* one we already hold for this user.
*/
public set(eventId: string, receiptType: string, userId: string, ts: number, synthetic: boolean): void {
const userReceipts = getOrCreate(this.data, userId, () => new UserReceipts(this.room));
const existingReceipt = userReceipts.getByType(synthetic);
if (existingReceipt && isAfter(existingReceipt.eventId, eventId, this.room)) {
// The new receipt is before the existing one - don't store it.
return;
}
// Possibilities:
//
// 1. there was no existing receipt, or
// 2. the existing receipt was before this one, or
// 3. we were unable to compare the receipts.
//
// In the case of 3 it's difficult to decide what to do, so the
// most-recently-received receipt wins.
//
// Case 3 can only happen if the events for these receipts have
// disappeared, which is quite unlikely since the new one has just been
// checked, and the old one was checked before it was inserted here.
//
// We go ahead and store this receipt (replacing the other if it exists)
userReceipts.set(synthetic, new ReceiptInfo(eventId, receiptType, ts));
}
/**
* Find the latest receipt we have for this user. (Note - there is only one
* receipt per user, because we are already inside a specific thread or
* unthreaded list.)
*
* If there is a later synthetic receipt for this user, return that.
* Otherwise, return the real receipt.
*
* @returns the found receipt info, or undefined if we have no receipt for this user.
*/
public get(userId: string): ReceiptInfo | undefined {
return this.data.get(userId)?.get();
}
}
/**
* The latest threaded receipts we have for a room.
*/
class ThreadedReceipts {
private room: Room;
/** map of threadId: ReceiptsByUser */
private data: Map<string, ReceiptsByUser>;
public constructor(room: Room) {
this.room = room;
this.data = new Map<string, ReceiptsByUser>();
}
/**
* Add the supplied receipt to our structure, if it is not earlier than one
* we already hold for this user in this thread.
*/
public set(
threadId: string,
eventId: string,
receiptType: string,
userId: string,
ts: number,
synthetic: boolean,
): void {
const receiptsByUser = getOrCreate(this.data, threadId, () => new ReceiptsByUser(this.room));
receiptsByUser.set(eventId, receiptType, userId, ts, synthetic);
}
/**
* Find the latest threaded receipt for the supplied user in the supplied thread.
*
* @returns the found receipt info or undefined if we don't have one.
*/
public get(threadId: string, userId: string): ReceiptInfo | undefined {
return this.data.get(threadId)?.get(userId);
}
}
/**
* All the receipts that we have received but can't process because we can't
* find the event they refer to.
*
* We hold on to them so we can process them if their event arrives later.
*/
class DanglingReceipts {
/**
* eventId: DanglingReceipt[]
*/
private data = new Map<string, Array<DanglingReceipt>>();
/**
* Remember the supplied dangling receipt.
*/
public add(danglingReceipt: DanglingReceipt): void {
const danglingReceipts = getOrCreate(this.data, danglingReceipt.eventId, () => []);
danglingReceipts.push(danglingReceipt);
}
/**
* Remove and return the dangling receipts for the given event ID.
* @param eventId - the event ID to look for
* @returns the found dangling receipts, or undefined if we don't have one.
*/
public remove(eventId: string): Array<DanglingReceipt> | undefined {
const danglingReceipts = this.data.get(eventId);
this.data.delete(eventId);
return danglingReceipts;
}
}
function getOrCreate<K, V>(m: Map<K, V>, key: K, createFn: () => V): V {
const found = m.get(key);
if (found) {
return found;
} else {
const created = createFn();
m.set(key, created);
return created;
}
}
/**
* Is left after right (or the same)?
*
* Only returns true if both events can be found, and left is after or the same
* as right.
*
* @returns left \>= right
*/
function isAfterOrSame(leftEventId: string, rightEventId: string, room: Room): boolean {
const comparison = room.compareEventOrdering(leftEventId, rightEventId);
return comparison !== null && comparison >= 0;
}
/**
* Is left strictly after right?
*
* Only returns true if both events can be found, and left is strictly after right.
*
* @returns left \> right
*/
function isAfter(leftEventId: string, rightEventId: string, room: Room): boolean {
const comparison = room.compareEventOrdering(leftEventId, rightEventId);
return comparison !== null && comparison > 0;
}
+64 -6
View File
@@ -66,6 +66,8 @@ import { IStateEventWithRoomId } from "../@types/search";
import { RelationsContainer } from "./relations-container";
import { ReadReceipt, synthesizeReceipt } from "./read-receipt";
import { isPollEvent, Poll, PollEvent } from "./poll";
import { RoomReceipts } from "./room-receipts";
import { compareEventOrdering } from "./compare-event-ordering";
// These constants are used as sane defaults when the homeserver doesn't support
// the m.room_versions capability. In practice, KNOWN_SAFE_ROOM_VERSION should be
@@ -236,8 +238,9 @@ export type RoomEventHandlerMap = {
*
* @param event - The matrix redaction event
* @param room - The room containing the redacted event
* @param threadId - The thread containing the redacted event (before it was redacted)
*/
[RoomEvent.Redaction]: (event: MatrixEvent, room: Room) => void;
[RoomEvent.Redaction]: (event: MatrixEvent, room: Room, threadId?: string) => void;
/**
* Fires when an event that was previously redacted isn't anymore.
* This happens when the redaction couldn't be sent and
@@ -431,6 +434,12 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
*/
private visibilityEvents = new Map<string, MatrixEvent[]>();
/**
* The latest receipts (synthetic and real) for each user in each thread
* (and unthreaded).
*/
private roomReceipts = new RoomReceipts(this);
/**
* Construct a new Room.
*
@@ -549,7 +558,7 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
const decryptionPromises = events
.slice(readReceiptTimelineIndex)
.reverse()
.map((event) => this.client.decryptEventIfNeeded(event, { isRetry: true }));
.map((event) => this.client.decryptEventIfNeeded(event));
await Promise.allSettled(decryptionPromises);
}
@@ -567,7 +576,7 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
.getEvents()
.slice(0) // copy before reversing
.reverse()
.map((event) => this.client.decryptEventIfNeeded(event, { isRetry: true }));
.map((event) => this.client.decryptEventIfNeeded(event));
await Promise.allSettled(decryptionPromises);
}
@@ -2113,6 +2122,12 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
* Relations (other than m.thread), redactions, replies to a thread root live only in the main timeline
* Relations, redactions, replies where the parent cannot be found live in no timelines but should be aggregated regardless.
* Otherwise, the event lives in the main timeline only.
*
* Note: when a redaction is applied, the redacted event, events relating
* to it, and the redaction event itself, will all move to the main thread.
* This method classifies them as inside the thread of the redacted event.
* They are moved later as part of makeRedacted.
* This will change if MSC3389 is merged.
*/
public eventShouldLiveIn(
event: MatrixEvent,
@@ -2329,7 +2344,8 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
// if we know about this event, redact its contents now.
const redactedEvent = redactId ? this.findEventById(redactId) : undefined;
if (redactedEvent) {
redactedEvent.makeRedacted(event);
const threadRootId = redactedEvent.threadRootId;
redactedEvent.makeRedacted(event, this);
// If this is in the current state, replace it with the redacted version
if (redactedEvent.isState()) {
@@ -2342,7 +2358,7 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
}
}
this.emit(RoomEvent.Redaction, event, this);
this.emit(RoomEvent.Redaction, event, this, threadRootId);
// TODO: we stash user displaynames (among other things) in
// RoomMember objects which are then attached to other events
@@ -2495,7 +2511,7 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
}
if (redactedEvent) {
redactedEvent.markLocallyRedacted(event);
this.emit(RoomEvent.Redaction, event, this);
this.emit(RoomEvent.Redaction, event, this, redactedEvent.threadRootId);
}
}
} else {
@@ -2927,6 +2943,10 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
*/
public addReceipt(event: MatrixEvent, synthetic = false): void {
const content = event.getContent<ReceiptContent>();
this.roomReceipts.add(content, synthetic);
// TODO: delete the following code when it has been replaced by RoomReceipts
Object.keys(content).forEach((eventId: string) => {
Object.keys(content[eventId]).forEach((receiptType: ReceiptType | string) => {
Object.keys(content[eventId][receiptType]).forEach((userId: string) => {
@@ -2988,6 +3008,7 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
});
});
});
// End of code to delete when replaced by RoomReceipts
// send events after we've regenerated the structure & cache, otherwise things that
// listened for the event would read stale data.
@@ -3574,6 +3595,19 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
return this.oldestThreadedReceiptTs;
}
/**
* Determines if the given user has read a particular event ID with the known
* history of the room. This is not a definitive check as it relies only on
* what is available to the room at the time of execution.
*
* @param userId - The user ID to check the read state of.
* @param eventId - The event ID to check if the user read.
* @returns true if the user has read the event, false otherwise.
*/
public hasUserReadEvent(userId: string, eventId: string): boolean {
return this.roomReceipts.hasUserReadEvent(userId, eventId);
}
/**
* Returns the most recent unthreaded receipt for a given user
* @param userId - the MxID of the User
@@ -3607,6 +3641,30 @@ export class Room extends ReadReceipt<RoomEmittedEvents, RoomEventHandlerMap> {
thread.fixupNotifications(userId);
}
}
/**
* Determine the order of two events in this room.
*
* In principle this should use the same order as the server, but in practice
* this is difficult for events that were not received over the Sync API. See
* MSC4033 for details.
*
* This implementation leans on the order of events within their timelines, and
* falls back to comparing event timestamps when they are in different
* timelines.
*
* See https://github.com/matrix-org/matrix-js-sdk/issues/3325 for where we are
* tracking the work to fix this.
*
* @param leftEventId - the id of the first event
* @param rightEventId - the id of the second event
* @returns -1 if left \< right, 1 if left \> right, 0 if left == right, null if
* we can't tell (because we can't find the events).
*/
public compareEventOrdering(leftEventId: string, rightEventId: string): number | null {
return compareEventOrdering(this, leftEventId, rightEventId);
}
}
// a map from current event status to a list of allowed next statuses
+28 -4
View File
@@ -143,6 +143,9 @@ export class Thread extends ReadReceipt<ThreadEmittedEvents, ThreadEventHandlerM
public constructor(public readonly id: string, public rootEvent: MatrixEvent | undefined, opts: IThreadOpts) {
super();
// each Event in the thread adds a reemitter, so we could hit the listener limit.
this.setMaxListeners(1000);
if (!opts?.room) {
// Logging/debugging for https://github.com/vector-im/element-web/issues/22141
// Hope is that we end up with a more obvious stack trace.
@@ -228,8 +231,8 @@ export class Thread extends ReadReceipt<ThreadEmittedEvents, ThreadEventHandlerM
}
};
private onRedaction = async (event: MatrixEvent): Promise<void> => {
if (event.threadRootId !== this.id) return; // ignore redactions for other timelines
private onRedaction = async (event: MatrixEvent, room: Room, threadRootId?: string): Promise<void> => {
if (threadRootId !== this.id) return; // ignore redactions for other timelines
if (this.replyCount <= 0) {
for (const threadEvent of this.timeline) {
this.clearEventMetadata(threadEvent);
@@ -368,7 +371,7 @@ export class Thread extends ReadReceipt<ThreadEmittedEvents, ThreadEventHandlerM
if (!Thread.hasServerSideSupport) {
// When there's no server-side support, just add it to the end of the timeline.
this.addEventToTimeline(event, toStartOfTimeline);
this.client.decryptEventIfNeeded(event, {});
this.client.decryptEventIfNeeded(event);
} else if (!toStartOfTimeline && this.initialEventsFetched && isNewestReply) {
// When we've asked for the event to be added to the end, and we're
// not in the initial state, and this event belongs at the end, add it.
@@ -745,6 +748,27 @@ export class Thread extends ReadReceipt<ThreadEmittedEvents, ThreadEventHandlerM
* @returns ID of the latest event that the given user has read, or null.
*/
public getEventReadUpTo(userId: string, ignoreSynthesized?: boolean): string | null {
// TODO: we think the implementation here is not right. Here is a sketch
// of the right answer:
//
// for event in timeline.events.reversed():
// if room.hasUserReadEvent(event):
// return event
// return null
//
// If this is too slow, we might be able to improve it by trying walking
// forward from the threaded receipt in this thread. We could alternate
// between backwards-from-front and forwards-from-threaded-receipt to
// improve our chances of hitting the right answer sooner.
//
// Either way, it's still fundamentally slow because we have to walk
// events.
//
// We also might just want to limit the time we spend on this by giving
// up after, say, 100 events.
//
// --- andyb
const isCurrentUser = userId === this.client.getUserId();
const lastReply = this.timeline[this.timeline.length - 1];
if (isCurrentUser && lastReply) {
@@ -813,7 +837,7 @@ export class Thread extends ReadReceipt<ThreadEmittedEvents, ThreadEventHandlerM
}
}
return super.hasUserReadEvent(userId, eventId);
return this.room.hasUserReadEvent(userId, eventId);
}
public setUnread(type: NotificationCountType, count: number): void {
+10
View File
@@ -15,10 +15,20 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import { encodeUnpaddedBase64Url } from "./base64";
import { crypto } from "./crypto/crypto";
const LOWERCASE = "abcdefghijklmnopqrstuvwxyz";
const UPPERCASE = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
const DIGITS = "0123456789";
export function secureRandomBase64Url(len: number): string {
const key = new Uint8Array(len);
crypto.getRandomValues(key);
return encodeUnpaddedBase64Url(key);
}
export function randomString(len: number): string {
return randomStringFrom(len, UPPERCASE + LOWERCASE + DIGITS);
}
+46 -23
View File
@@ -14,12 +14,12 @@ See the License for the specific language governing permissions and
limitations under the License.
*/
import { OlmMachine, CrossSigningStatus } from "@matrix-org/matrix-sdk-crypto-wasm";
import { OlmMachine, CrossSigningStatus, CrossSigningBootstrapRequests } from "@matrix-org/matrix-sdk-crypto-wasm";
import * as RustSdkCryptoJs from "@matrix-org/matrix-sdk-crypto-wasm";
import { BootstrapCrossSigningOpts } from "../crypto-api";
import { logger } from "../logger";
import { OutgoingRequest, OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { UIAuthCallback } from "../interactive-auth";
import { ServerSideSecretStorage } from "../secret-storage";
@@ -57,7 +57,7 @@ export class CrossSigningIdentity {
olmDeviceStatus.hasMaster && olmDeviceStatus.hasUserSigning && olmDeviceStatus.hasSelfSigning;
// Log all relevant state for easier parsing of debug logs.
logger.log("bootStrapCrossSigning: starting", {
logger.log("bootstrapCrossSigning: starting", {
setupNewCrossSigning: opts.setupNewCrossSigning,
olmDeviceHasMaster: olmDeviceStatus.hasMaster,
olmDeviceHasUserSigning: olmDeviceStatus.hasUserSigning,
@@ -66,18 +66,25 @@ export class CrossSigningIdentity {
});
if (olmDeviceHasKeys) {
if (!privateKeysInSecretStorage) {
if (!(await this.secretStorage.hasKey())) {
logger.warn(
"bootstrapCrossSigning: Olm device has private keys, but secret storage is not yet set up; doing nothing for now.",
);
// the keys should get uploaded to 4S once that is set up.
} else if (!privateKeysInSecretStorage) {
// the device has the keys but they are not in 4S, so update it
logger.log("bootStrapCrossSigning: Olm device has private keys: exporting to secret storage");
logger.log("bootstrapCrossSigning: Olm device has private keys: exporting to secret storage");
await this.exportCrossSigningKeysToStorage();
} else {
logger.log("bootStrapCrossSigning: Olm device has private keys and they are saved in 4S, do nothing");
logger.log(
"bootstrapCrossSigning: Olm device has private keys and they are saved in secret storage; doing nothing",
);
}
} /* (!olmDeviceHasKeys) */ else {
if (privateKeysInSecretStorage) {
// they are in 4S, so import from there
logger.log(
"bootStrapCrossSigning: Cross-signing private keys not found locally, but they are available " +
"bootstrapCrossSigning: Cross-signing private keys not found locally, but they are available " +
"in secret storage, reading storage and caching locally",
);
await this.olmMachine.importCrossSigningKeys(
@@ -91,13 +98,16 @@ export class CrossSigningIdentity {
this.olmMachine.userId,
this.olmMachine.deviceId,
);
// Sign the device with our cross-signing key and upload the signature
const request: RustSdkCryptoJs.SignatureUploadRequest = await device.verify();
await this.outgoingRequestProcessor.makeOutgoingRequest(request);
try {
// Sign the device with our cross-signing key and upload the signature
const request: RustSdkCryptoJs.SignatureUploadRequest = await device.verify();
await this.outgoingRequestProcessor.makeOutgoingRequest(request);
} finally {
device.free();
}
} else {
logger.log(
"bootStrapCrossSigning: Cross-signing private keys not found locally or in secret storage, creating new keys",
"bootstrapCrossSigning: Cross-signing private keys not found locally or in secret storage, creating new keys",
);
await this.resetCrossSigning(opts.authUploadDeviceSigningKeys);
}
@@ -105,7 +115,7 @@ export class CrossSigningIdentity {
// TODO: we might previously have bootstrapped cross-signing but not completed uploading the keys to the
// server -- in which case we should call OlmDevice.bootstrap_cross_signing. How do we know?
logger.log("bootStrapCrossSigning: complete");
logger.log("bootstrapCrossSigning: complete");
}
/** Reset our cross-signing keys
@@ -118,18 +128,31 @@ export class CrossSigningIdentity {
private async resetCrossSigning(authUploadDeviceSigningKeys?: UIAuthCallback<void>): Promise<void> {
// XXX: We must find a way to make this atomic, currently if the user does not remember his account password
// or 4S passphrase/key the process will fail in a bad state, with keys rotated but not uploaded or saved in 4S.
const outgoingRequests: Array<OutgoingRequest> = await this.olmMachine.bootstrapCrossSigning(true);
const outgoingRequests: CrossSigningBootstrapRequests = await this.olmMachine.bootstrapCrossSigning(true);
// If 4S is configured we need to udpate it.
if (await this.secretStorage.hasKey()) {
// If 4S is configured we need to update it.
if (!(await this.secretStorage.hasKey())) {
logger.warn(
"resetCrossSigning: Secret storage is not yet set up; not exporting keys to secret storage yet.",
);
// the keys should get uploaded to 4S once that is set up.
} else {
// Update 4S before uploading cross-signing keys, to stay consistent with legacy that asks
// 4S passphrase before asking for account password.
// Ultimately should be made atomic and resistent to forgotten password/passphrase.
// Ultimately should be made atomic and resistant to forgotten password/passphrase.
logger.log("resetCrossSigning: exporting to secret storage");
await this.exportCrossSigningKeysToStorage();
}
logger.log("bootStrapCrossSigning: publishing keys to server");
for (const req of outgoingRequests) {
await this.outgoingRequestProcessor.makeOutgoingRequest(req, authUploadDeviceSigningKeys);
logger.log("resetCrossSigning: publishing keys to server");
for (const req of [
outgoingRequests.uploadKeysRequest,
outgoingRequests.uploadSigningKeysRequest,
outgoingRequests.uploadSignaturesRequest,
]) {
if (req) {
await this.outgoingRequestProcessor.makeOutgoingRequest(req, authUploadDeviceSigningKeys);
}
}
}
@@ -142,17 +165,17 @@ export class CrossSigningIdentity {
const exported: RustSdkCryptoJs.CrossSigningKeyExport | null = await this.olmMachine.exportCrossSigningKeys();
/* istanbul ignore else (this function is only called when we know the olm machine has keys) */
if (exported?.masterKey) {
this.secretStorage.store("m.cross_signing.master", exported.masterKey);
await this.secretStorage.store("m.cross_signing.master", exported.masterKey);
} else {
logger.error(`Cannot export MSK to secret storage, private key unknown`);
}
if (exported?.self_signing_key) {
this.secretStorage.store("m.cross_signing.self_signing", exported.self_signing_key);
await this.secretStorage.store("m.cross_signing.self_signing", exported.self_signing_key);
} else {
logger.error(`Cannot export SSK to secret storage, private key unknown`);
}
if (exported?.userSigningKey) {
this.secretStorage.store("m.cross_signing.user_signing", exported.userSigningKey);
await this.secretStorage.store("m.cross_signing.user_signing", exported.userSigningKey);
} else {
logger.error(`Cannot export USK to secret storage, private key unknown`);
}
+7 -3
View File
@@ -17,6 +17,7 @@ limitations under the License.
import { OlmMachine, UserId } from "@matrix-org/matrix-sdk-crypto-wasm";
import { OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { LogSpan } from "../logger";
/**
* KeyClaimManager: linearises calls to OlmMachine.getMissingSessions to avoid races
@@ -52,7 +53,7 @@ export class KeyClaimManager {
*
* @param userList - list of userIDs to claim
*/
public ensureSessionsForUsers(userList: Array<UserId>): Promise<void> {
public ensureSessionsForUsers(logger: LogSpan, userList: Array<UserId>): Promise<void> {
// The Rust-SDK requires that we only have one getMissingSessions process in flight at once. This little dance
// ensures that, by only having one call to ensureSessionsForUsersInner active at once (and making them
// queue up in order).
@@ -61,19 +62,22 @@ export class KeyClaimManager {
// any errors in the previous claim will have been reported already, so there is nothing to do here.
// we just throw away the error and start anew.
})
.then(() => this.ensureSessionsForUsersInner(userList));
.then(() => this.ensureSessionsForUsersInner(logger, userList));
this.currentClaimPromise = prom;
return prom;
}
private async ensureSessionsForUsersInner(userList: Array<UserId>): Promise<void> {
private async ensureSessionsForUsersInner(logger: LogSpan, userList: Array<UserId>): Promise<void> {
// bail out quickly if we've been stopped.
if (this.stopped) {
throw new Error(`Cannot ensure Olm sessions: shutting down`);
}
logger.info("Checking for missing Olm sessions");
const claimRequest = await this.olmMachine.getMissingSessions(userList);
if (claimRequest) {
logger.info("Making /keys/claim request");
await this.outgoingRequestProcessor.makeOutgoingRequest(claimRequest);
}
logger.info("Olm sessions prepared");
}
}
+3 -3
View File
@@ -23,7 +23,7 @@ import {
RoomMessageRequest,
SignatureUploadRequest,
ToDeviceRequest,
SigningKeysUploadRequest,
UploadSigningKeysRequest,
} from "@matrix-org/matrix-sdk-crypto-wasm";
import { logger } from "../logger";
@@ -62,7 +62,7 @@ export class OutgoingRequestProcessor {
) {}
public async makeOutgoingRequest<T>(
msg: OutgoingRequest | SigningKeysUploadRequest,
msg: OutgoingRequest | UploadSigningKeysRequest,
uiaCallback?: UIAuthCallback<T>,
): Promise<void> {
let resp: string;
@@ -92,7 +92,7 @@ export class OutgoingRequestProcessor {
`/_matrix/client/v3/rooms/${encodeURIComponent(msg.room_id)}/send/` +
`${encodeURIComponent(msg.event_type)}/${encodeURIComponent(msg.txn_id)}`;
resp = await this.rawJsonRequest(Method.Put, path, {}, msg.body);
} else if (msg instanceof SigningKeysUploadRequest) {
} else if (msg instanceof UploadSigningKeysRequest) {
await this.makeRequestWithUIA(
Method.Post,
"/_matrix/client/v3/keys/device_signing/upload",
+141
View File
@@ -0,0 +1,141 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import { OlmMachine } from "@matrix-org/matrix-sdk-crypto-wasm";
import { OutgoingRequest, OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { Logger } from "../logger";
import { defer, IDeferred } from "../utils";
/**
* OutgoingRequestsManager: responsible for processing outgoing requests from the OlmMachine.
* Ensure that only one loop is going on at once, and that the requests are processed in order.
*/
export class OutgoingRequestsManager {
/** whether {@link stop} has been called */
private stopped = false;
/** whether {@link outgoingRequestLoop} is currently running */
private outgoingRequestLoopRunning = false;
/**
* If there are additional calls to doProcessOutgoingRequests() while there is a current call running
* we need to remember in order to call `doProcessOutgoingRequests` again (as there could be new requests).
*
* If this is defined, it is an indication that we need to do another iteration; in this case the deferred
* will resolve once that next iteration completes. If it is undefined, there have been no new calls
* to `doProcessOutgoingRequests` since the current iteration started.
*/
private nextLoopDeferred?: IDeferred<void>;
public constructor(
private readonly logger: Logger,
private readonly olmMachine: OlmMachine,
public readonly outgoingRequestProcessor: OutgoingRequestProcessor,
) {}
/**
* Shut down as soon as possible the current loop of outgoing requests processing.
*/
public stop(): void {
this.stopped = true;
}
/**
* Process the OutgoingRequests from the OlmMachine.
*
* This should be called at the end of each sync, to process any OlmMachine OutgoingRequests created by the rust sdk.
* In some cases if OutgoingRequests need to be sent immediately, this can be called directly.
*
* Calls to doProcessOutgoingRequests() are processed synchronously, one after the other, in order.
* If doProcessOutgoingRequests() is called while another call is still being processed, it will be queued.
* Multiple calls to doProcessOutgoingRequests() when a call is already processing will be batched together.
*/
public doProcessOutgoingRequests(): Promise<void> {
// Flag that we need at least one more iteration of the loop.
//
// It is important that we do this even if the loop is currently running. There is potential for a race whereby
// a request is added to the queue *after* `OlmMachine.outgoingRequests` checks the queue, but *before* it
// returns. In such a case, the item could sit there unnoticed for some time.
//
// In order to circumvent the race, we set a flag which tells the loop to go round once again even if the
// queue appears to be empty.
if (!this.nextLoopDeferred) {
this.nextLoopDeferred = defer();
}
// ... and wait for it to complete.
const result = this.nextLoopDeferred.promise;
// set the loop going if it is not already.
if (!this.outgoingRequestLoopRunning) {
this.outgoingRequestLoop().catch((e) => {
// this should not happen; outgoingRequestLoop should return any errors via `nextLoopDeferred`.
/* istanbul ignore next */
this.logger.error("Uncaught error in outgoing request loop", e);
});
}
return result;
}
private async outgoingRequestLoop(): Promise<void> {
/* istanbul ignore if */
if (this.outgoingRequestLoopRunning) {
throw new Error("Cannot run two outgoing request loops");
}
this.outgoingRequestLoopRunning = true;
try {
while (!this.stopped && this.nextLoopDeferred) {
const deferred = this.nextLoopDeferred;
// reset `nextLoopDeferred` so that any future calls to `doProcessOutgoingRequests` are queued
// for another additional iteration.
this.nextLoopDeferred = undefined;
// make the requests and feed the results back to the `nextLoopDeferred`
await this.processOutgoingRequests().then(deferred.resolve, deferred.reject);
}
} finally {
this.outgoingRequestLoopRunning = false;
}
if (this.nextLoopDeferred) {
// the loop was stopped, but there was a call to `doProcessOutgoingRequests`. Make sure that
// we reject the promise in case anything is waiting for it.
this.nextLoopDeferred.reject(new Error("OutgoingRequestsManager was stopped"));
}
}
/**
* Make a single request to `olmMachine.outgoingRequests` and do the corresponding requests.
*/
private async processOutgoingRequests(): Promise<void> {
if (this.stopped) return;
const outgoingRequests: OutgoingRequest[] = await this.olmMachine.outgoingRequests();
for (const request of outgoingRequests) {
if (this.stopped) return;
try {
await this.outgoingRequestProcessor.makeOutgoingRequest(request);
} catch (e) {
// as part of the loop we silently ignore errors, but log them.
// The rust sdk will retry the request later as it won't have been marked as sent.
this.logger.error(`Failed to process outgoing request ${request.type}: ${e}`);
}
}
}
}
@@ -0,0 +1,474 @@
/*
Copyright 2023 The Matrix.org Foundation C.I.C.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
import * as RustSdkCryptoJs from "@matrix-org/matrix-sdk-crypto-wasm";
import { OlmMachine } from "@matrix-org/matrix-sdk-crypto-wasm";
import { Curve25519AuthData, KeyBackupSession } from "../crypto-api/keybackup";
import { Logger } from "../logger";
import { ClientPrefix, IHttpOpts, MatrixError, MatrixHttpApi, Method } from "../http-api";
import { RustBackupManager } from "./backup";
import { CryptoEvent } from "../matrix";
import { encodeUri, sleep } from "../utils";
import { BackupDecryptor } from "../common-crypto/CryptoBackend";
// The minimum time to wait between two retries in case of errors. To avoid hammering the server.
const KEY_BACKUP_BACKOFF = 5000; // ms
/**
* Enumerates the different kind of errors that can occurs when downloading and importing a key from backup.
*/
enum KeyDownloadErrorCode {
/** The requested key is not in the backup. */
MISSING_DECRYPTION_KEY = "MISSING_DECRYPTION_KEY",
/** A network error occurred while trying to download the key from backup. */
NETWORK_ERROR = "NETWORK_ERROR",
/** The loop has been stopped. */
STOPPED = "STOPPED",
}
class KeyDownloadError extends Error {
public constructor(public readonly code: KeyDownloadErrorCode) {
super(`Failed to get key from backup: ${code}`);
this.name = "KeyDownloadError";
}
}
class KeyDownloadRateLimitError extends Error {
public constructor(public readonly retryMillis: number) {
super(`Failed to get key from backup: rate limited`);
this.name = "KeyDownloadRateLimitError";
}
}
/** Details of a megolm session whose key we are trying to fetch. */
type SessionInfo = { roomId: string; megolmSessionId: string };
/** Holds the current backup decryptor and version that should be used. */
type Configuration = {
backupVersion: string;
decryptor: BackupDecryptor;
};
/**
* Used when an 'unable to decrypt' error occurs. It attempts to download the key from the backup.
*
* The current backup API lacks pagination, which can lead to lengthy key retrieval times for large histories (several 10s of minutes).
* To mitigate this, keys are downloaded on demand as decryption errors occurs.
* While this approach may result in numerous requests, it improves user experience by reducing wait times for message decryption.
*
* The PerSessionKeyBackupDownloader is resistant to backup configuration changes: it will automatically resume querying when
* the backup is configured correctly.
*/
export class PerSessionKeyBackupDownloader {
private stopped = false;
/** The version and decryption key to use with current backup if all set up correctly */
private configuration: Configuration | null = null;
/** We remember when a session was requested and not found in backup to avoid query again too soon.
* Map of session_id to timestamp */
private sessionLastCheckAttemptedTime: Map<string, number> = new Map();
/** The logger to use */
private readonly logger: Logger;
/** Whether the download loop is running. */
private downloadLoopRunning = false;
/** The list of requests that are queued. */
private queuedRequests: SessionInfo[] = [];
/** Remembers if we have a configuration problem. */
private hasConfigurationProblem = false;
/** The current server backup version check promise. To avoid doing a server call if one is in flight. */
private currentBackupVersionCheck: Promise<Configuration | null> | null = null;
/**
* Creates a new instance of PerSessionKeyBackupDownloader.
*
* @param backupManager - The backup manager to use.
* @param olmMachine - The olm machine to use.
* @param http - The http instance to use.
* @param logger - The logger to use.
*/
public constructor(
logger: Logger,
private readonly olmMachine: OlmMachine,
private readonly http: MatrixHttpApi<IHttpOpts & { onlyData: true }>,
private readonly backupManager: RustBackupManager,
) {
this.logger = logger.getChild("[PerSessionKeyBackupDownloader]");
backupManager.on(CryptoEvent.KeyBackupStatus, this.onBackupStatusChanged);
backupManager.on(CryptoEvent.KeyBackupFailed, this.onBackupStatusChanged);
backupManager.on(CryptoEvent.KeyBackupDecryptionKeyCached, this.onBackupStatusChanged);
}
/**
* Called when a MissingRoomKey or UnknownMessageIndex decryption error is encountered.
*
* This will try to download the key from the backup if there is a trusted active backup.
* In case of success the key will be imported and the onRoomKeysUpdated callback will be called
* internally by the rust-sdk and decryption will be retried.
*
* @param roomId - The room ID of the room where the error occurred.
* @param megolmSessionId - The megolm session ID that is missing.
*/
public onDecryptionKeyMissingError(roomId: string, megolmSessionId: string): void {
// Several messages encrypted with the same session may be decrypted at the same time,
// so we need to be resistant and not query several time the same session.
if (this.isAlreadyInQueue(roomId, megolmSessionId)) {
// There is already a request queued for this session, no need to queue another one.
this.logger.trace(`Not checking key backup for session ${megolmSessionId} as it is already queued`);
return;
}
if (this.wasRequestedRecently(megolmSessionId)) {
// We already tried to download this session recently and it was not in backup, no need to try again.
this.logger.trace(
`Not checking key backup for session ${megolmSessionId} as it was already requested recently`,
);
return;
}
// We always add the request to the queue, even if we have a configuration problem (can't access backup).
// This is to make sure that if the configuration problem is resolved, we will try to download the key.
// This will happen after an initial sync, at this point the backup will not yet be trusted and the decryption
// key will not be available, but it will be just after the verification.
// We don't need to persist it because currently on refresh the sdk will retry to decrypt the messages in error.
this.queuedRequests.push({ roomId, megolmSessionId });
// Start the download loop if it's not already running.
this.downloadKeysLoop();
}
public stop(): void {
this.stopped = true;
this.backupManager.off(CryptoEvent.KeyBackupStatus, this.onBackupStatusChanged);
this.backupManager.off(CryptoEvent.KeyBackupFailed, this.onBackupStatusChanged);
this.backupManager.off(CryptoEvent.KeyBackupDecryptionKeyCached, this.onBackupStatusChanged);
}
/**
* Called when the backup status changes (CryptoEvents)
* This will trigger a check of the backup configuration.
*/
private onBackupStatusChanged = (): void => {
// we want to force check configuration, so we clear the current one.
this.hasConfigurationProblem = false;
this.configuration = null;
this.getOrCreateBackupConfiguration().then((configuration) => {
if (configuration) {
// restart the download loop if it was stopped
this.downloadKeysLoop();
}
});
};
/** Returns true if the megolm session is already queued for download. */
private isAlreadyInQueue(roomId: string, megolmSessionId: string): boolean {
return this.queuedRequests.some((info) => {
return info.roomId == roomId && info.megolmSessionId == megolmSessionId;
});
}
/**
* Marks the session as not found in backup, to avoid retrying to soon for a key not in backup
*
* @param megolmSessionId - The megolm session ID that is missing.
*/
private markAsNotFoundInBackup(megolmSessionId: string): void {
const now = Date.now();
this.sessionLastCheckAttemptedTime.set(megolmSessionId, now);
// if too big make some cleaning to keep under control
if (this.sessionLastCheckAttemptedTime.size > 100) {
this.sessionLastCheckAttemptedTime = new Map(
Array.from(this.sessionLastCheckAttemptedTime).filter((sid, ts) => {
return Math.max(now - ts, 0) < KEY_BACKUP_BACKOFF;
}),
);
}
}
/** Returns true if the session was requested recently. */
private wasRequestedRecently(megolmSessionId: string): boolean {
const lastCheck = this.sessionLastCheckAttemptedTime.get(megolmSessionId);
if (!lastCheck) return false;
return Math.max(Date.now() - lastCheck, 0) < KEY_BACKUP_BACKOFF;
}
private async getBackupDecryptionKey(): Promise<RustSdkCryptoJs.BackupKeys | null> {
try {
return await this.olmMachine.getBackupKeys();
} catch (e) {
return null;
}
}
/**
* Requests a key from the server side backup.
*
* @param version - The backup version to use.
* @param roomId - The room ID of the room where the error occurred.
* @param sessionId - The megolm session ID that is missing.
*/
private async requestRoomKeyFromBackup(
version: string,
roomId: string,
sessionId: string,
): Promise<KeyBackupSession> {
const path = encodeUri("/room_keys/keys/$roomId/$sessionId", {
$roomId: roomId,
$sessionId: sessionId,
});
return await this.http.authedRequest<KeyBackupSession>(Method.Get, path, { version }, undefined, {
prefix: ClientPrefix.V3,
});
}
private async downloadKeysLoop(): Promise<void> {
if (this.downloadLoopRunning) return;
// If we have a configuration problem, we don't want to try to download.
// If any configuration change is detected, we will retry and restart the loop.
if (this.hasConfigurationProblem) return;
this.downloadLoopRunning = true;
try {
while (this.queuedRequests.length > 0) {
// we just peek the first one without removing it, so if a new request for same key comes in while we're
// processing this one, it won't queue another request.
const request = this.queuedRequests[0];
try {
// The backup could have changed between the time we queued the request and now, so we need to check
const configuration = await this.getOrCreateBackupConfiguration();
if (!configuration) {
// Backup is not configured correctly, so stop the loop.
this.downloadLoopRunning = false;
return;
}
const result = await this.queryKeyBackup(request.roomId, request.megolmSessionId, configuration);
if (this.stopped) {
return;
}
// We got the encrypted key from backup, let's try to decrypt and import it.
try {
await this.decryptAndImport(request, result, configuration);
} catch (e) {
this.logger.error(
`Error while decrypting and importing key backup for session ${request.megolmSessionId}`,
e,
);
}
// now remove the request from the queue as we've processed it.
this.queuedRequests.shift();
} catch (err) {
if (err instanceof KeyDownloadError) {
switch (err.code) {
case KeyDownloadErrorCode.MISSING_DECRYPTION_KEY:
this.markAsNotFoundInBackup(request.megolmSessionId);
// continue for next one
this.queuedRequests.shift();
break;
case KeyDownloadErrorCode.NETWORK_ERROR:
// We don't want to hammer if there is a problem, so wait a bit.
await sleep(KEY_BACKUP_BACKOFF);
break;
case KeyDownloadErrorCode.STOPPED:
// If the downloader was stopped, we don't want to retry.
this.downloadLoopRunning = false;
return;
}
} else if (err instanceof KeyDownloadRateLimitError) {
// we want to retry after the backoff time
await sleep(err.retryMillis);
}
}
}
} finally {
// all pending request have been processed, we can stop the loop.
this.downloadLoopRunning = false;
}
}
/**
* Query the backup for a key.
*
* @param targetRoomId - ID of the room that the session is used in.
* @param targetSessionId - ID of the session for which to check backup.
* @param configuration - The backup configuration to use.
*/
private async queryKeyBackup(
targetRoomId: string,
targetSessionId: string,
configuration: Configuration,
): Promise<KeyBackupSession> {
this.logger.debug(`Checking key backup for session ${targetSessionId}`);
if (this.stopped) throw new KeyDownloadError(KeyDownloadErrorCode.STOPPED);
try {
const res = await this.requestRoomKeyFromBackup(configuration.backupVersion, targetRoomId, targetSessionId);
this.logger.debug(`Got key from backup for sessionId:${targetSessionId}`);
return res;
} catch (e) {
if (this.stopped) throw new KeyDownloadError(KeyDownloadErrorCode.STOPPED);
this.logger.info(`No luck requesting key backup for session ${targetSessionId}: ${e}`);
if (e instanceof MatrixError) {
const errCode = e.data.errcode;
if (errCode == "M_NOT_FOUND") {
// Unfortunately the spec doesn't give us a way to differentiate between a missing key and a wrong version.
// Synapse will return:
// - "error": "Unknown backup version" if the version is wrong.
// - "error": "No room_keys found" if the key is missing.
// It's useful to know if the key is missing or if the version is wrong.
// As it's not spec'ed, we fall back on considering the key is not in backup.
// Notice that this request will be lost if instead the backup got out of sync (updated from other session).
throw new KeyDownloadError(KeyDownloadErrorCode.MISSING_DECRYPTION_KEY);
}
if (errCode == "M_LIMIT_EXCEEDED") {
const waitTime = e.data.retry_after_ms;
if (waitTime > 0) {
this.logger.info(`Rate limited by server, waiting ${waitTime}ms`);
throw new KeyDownloadRateLimitError(waitTime);
} else {
// apply the default backoff time
throw new KeyDownloadRateLimitError(KEY_BACKUP_BACKOFF);
}
}
}
throw new KeyDownloadError(KeyDownloadErrorCode.NETWORK_ERROR);
}
}
private async decryptAndImport(
sessionInfo: SessionInfo,
data: KeyBackupSession,
configuration: Configuration,
): Promise<void> {
const sessionsToImport: Record<string, KeyBackupSession> = { [sessionInfo.megolmSessionId]: data };
const keys = await configuration!.decryptor.decryptSessions(sessionsToImport);
for (const k of keys) {
k.room_id = sessionInfo.roomId;
}
await this.backupManager.importBackedUpRoomKeys(keys);
}
/**
* Gets the current backup configuration or create one if it doesn't exist.
*
* When a valid configuration is found it is cached and returned for subsequent calls.
* Otherwise, if a check is forced or a check has not yet been done, a new check is done.
*
* @returns The backup configuration to use or null if there is a configuration problem.
*/
private async getOrCreateBackupConfiguration(): Promise<Configuration | null> {
if (this.configuration) {
return this.configuration;
}
// We already tried to check the configuration and it failed.
// We don't want to try again immediately, we will retry if a configuration change is detected.
if (this.hasConfigurationProblem) {
return null;
}
// This method can be called rapidly by several emitted CryptoEvent, so we need to make sure that we don't
// query the server several times.
if (this.currentBackupVersionCheck != null) {
this.logger.debug(`Already checking server version, use current promise`);
return await this.currentBackupVersionCheck;
}
this.currentBackupVersionCheck = this.internalCheckFromServer();
try {
return await this.currentBackupVersionCheck;
} finally {
this.currentBackupVersionCheck = null;
}
}
private async internalCheckFromServer(): Promise<Configuration | null> {
let currentServerVersion = null;
try {
currentServerVersion = await this.backupManager.requestKeyBackupVersion();
} catch (e) {
this.logger.debug(`Backup: error while checking server version: ${e}`);
this.hasConfigurationProblem = true;
return null;
}
this.logger.debug(`Got current backup version from server: ${currentServerVersion?.version}`);
if (currentServerVersion?.algorithm != "m.megolm_backup.v1.curve25519-aes-sha2") {
this.logger.info(`Unsupported algorithm ${currentServerVersion?.algorithm}`);
this.hasConfigurationProblem = true;
return null;
}
if (!currentServerVersion?.version) {
this.logger.info(`No current key backup`);
this.hasConfigurationProblem = true;
return null;
}
const activeVersion = await this.backupManager.getActiveBackupVersion();
if (activeVersion == null || currentServerVersion.version != activeVersion) {
// Either the current backup version on server side is not trusted, or it is out of sync with the active version on the client side.
this.logger.info(
`The current backup version on the server (${currentServerVersion.version}) is not trusted. Version we are currently backing up to: ${activeVersion}`,
);
this.hasConfigurationProblem = true;
return null;
}
const authData = currentServerVersion.auth_data as Curve25519AuthData;
const backupKeys = await this.getBackupDecryptionKey();
if (!backupKeys?.decryptionKey) {
this.logger.debug(`Not checking key backup for session (no decryption key)`);
this.hasConfigurationProblem = true;
return null;
}
if (activeVersion != backupKeys.backupVersion) {
this.logger.debug(
`Version for which we have a decryption key (${backupKeys.backupVersion}) doesn't match the version we are backing up to (${activeVersion})`,
);
this.hasConfigurationProblem = true;
return null;
}
if (authData.public_key != backupKeys.decryptionKey.megolmV1PublicKey.publicKeyBase64) {
this.logger.debug(`getBackupDecryptor key mismatch error`);
this.hasConfigurationProblem = true;
return null;
}
const backupDecryptor = this.backupManager.createBackupDecryptor(backupKeys.decryptionKey);
this.hasConfigurationProblem = false;
this.configuration = {
decryptor: backupDecryptor,
backupVersion: activeVersion,
};
return this.configuration;
}
}
+77 -11
View File
@@ -23,15 +23,16 @@ import {
HistoryVisibility as RustHistoryVisibility,
ToDeviceRequest,
} from "@matrix-org/matrix-sdk-crypto-wasm";
import * as RustSdkCryptoJs from "@matrix-org/matrix-sdk-crypto-wasm";
import { EventType } from "../@types/event";
import { IContent, MatrixEvent } from "../models/event";
import { Room } from "../models/room";
import { Logger, logger } from "../logger";
import { Logger, logger, LogSpan } from "../logger";
import { KeyClaimManager } from "./KeyClaimManager";
import { RoomMember } from "../models/room-member";
import { OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { HistoryVisibility } from "../@types/partials";
import { OutgoingRequestsManager } from "./OutgoingRequestsManager";
/**
* RoomEncryptor: responsible for encrypting messages to a given room
@@ -41,21 +42,35 @@ import { HistoryVisibility } from "../@types/partials";
export class RoomEncryptor {
private readonly prefixedLogger: Logger;
/** whether the room members have been loaded and tracked for the first time */
private lazyLoadedMembersResolved = false;
/**
* @param olmMachine - The rust-sdk's OlmMachine
* @param keyClaimManager - Our KeyClaimManager, which manages the queue of one-time-key claim requests
* @param outgoingRequestProcessor - The OutgoingRequestProcessor, which sends outgoing requests
* @param outgoingRequestManager - The OutgoingRequestManager, which manages the queue of outgoing requests.
* @param room - The room we want to encrypt for
* @param encryptionSettings - body of the m.room.encryption event currently in force in this room
*/
public constructor(
private readonly olmMachine: OlmMachine,
private readonly keyClaimManager: KeyClaimManager,
private readonly outgoingRequestProcessor: OutgoingRequestProcessor,
private readonly outgoingRequestManager: OutgoingRequestsManager,
private readonly room: Room,
private encryptionSettings: IContent,
) {
this.prefixedLogger = logger.getChild(`[${room.roomId} encryption]`);
// start tracking devices for any users already known to be in this room.
// Do not load members here, would defeat lazy loading.
const members = room.getJoinedMembers();
// At this point just mark the known members as tracked, it might not be the full list of members
// because of lazy loading. This is fine, because we will get a member list update when sending a message for
// the first time, see `RoomEncryptor#ensureEncryptionSession`
this.olmMachine
.updateTrackedUsers(members.map((u) => new RustSdkCryptoJs.UserId(u.userId)))
.catch((e) => this.prefixedLogger.error("Error initializing tracked users", e));
}
/**
@@ -96,23 +111,70 @@ export class RoomEncryptor {
*
* @param globalBlacklistUnverifiedDevices - When `true`, it will not send encrypted messages to unverified devices
*/
public async ensureEncryptionSession(globalBlacklistUnverifiedDevices: boolean): Promise<void> {
public async prepareForEncryption(globalBlacklistUnverifiedDevices: boolean): Promise<void> {
const logger = new LogSpan(this.prefixedLogger, "prepareForEncryption");
await this.ensureEncryptionSession(logger, globalBlacklistUnverifiedDevices);
}
/**
* Prepare to encrypt events in this room.
*
* This ensures that we have a megolm session ready to use and that we have shared its key with all the devices
* in the room.
*
* @param logger - a place to write diagnostics to
* @param globalBlacklistUnverifiedDevices - When `true`, it will not send encrypted messages to unverified devices
*/
private async ensureEncryptionSession(logger: LogSpan, globalBlacklistUnverifiedDevices: boolean): Promise<void> {
if (this.encryptionSettings.algorithm !== "m.megolm.v1.aes-sha2") {
throw new Error(
`Cannot encrypt in ${this.room.roomId} for unsupported algorithm '${this.encryptionSettings.algorithm}'`,
);
}
logger.debug("Starting encryption");
const members = await this.room.getEncryptionTargetMembers();
this.prefixedLogger.debug(
// If this is the first time we are sending a message to the room, we may not yet have seen all the members
// (so the Crypto SDK might not have a device list for them). So, if this is the first time we are encrypting
// for this room, give the SDK the full list of members, to be on the safe side.
//
// This could end up being racy (if two calls to ensureEncryptionSession happen at the same time), but that's
// not a particular problem, since `OlmMachine.updateTrackedUsers` just adds any users that weren't already tracked.
if (!this.lazyLoadedMembersResolved) {
await this.olmMachine.updateTrackedUsers(members.map((u) => new RustSdkCryptoJs.UserId(u.userId)));
logger.debug(`Updated tracked users`);
this.lazyLoadedMembersResolved = true;
// Query keys in case we don't have them for newly tracked members.
// It's important after loading members for the first time, as likely most of them won't be
// known yet and will be unable to decrypt messages despite being in the room for long.
// This must be done before ensuring sessions. If not the devices of these users are not
// known yet and will not get the room key.
// We don't have API to only get the keys queries related to this member list, so we just
// process the pending requests from the olmMachine. (usually these are processed
// at the end of the sync, but we can't wait for that).
// XXX future improvement process only KeysQueryRequests for the users that have never been queried.
logger.debug(`Processing outgoing requests`);
await this.outgoingRequestManager.doProcessOutgoingRequests();
} else {
// If members are already loaded it's less critical to await on key queries.
// We might still want to trigger a processOutgoingRequests here.
// The call to `ensureSessionsForUsers` below will wait a bit on in-flight key queries we are
// interested in. If a sync handling happens in the meantime, and some new members are added to the room
// or have new devices it would give us a chance to query them before sending.
// It's less critical due to the racy nature of this process.
logger.debug(`Processing outgoing requests in background`);
this.outgoingRequestManager.doProcessOutgoingRequests();
}
logger.debug(
`Encrypting for users (shouldEncryptForInvitedMembers: ${this.room.shouldEncryptForInvitedMembers()}):`,
members.map((u) => `${u.userId} (${u.membership})`),
);
const userList = members.map((u) => new UserId(u.userId));
await this.keyClaimManager.ensureSessionsForUsers(userList);
this.prefixedLogger.debug("Sessions for users are ready; now sharing room key");
await this.keyClaimManager.ensureSessionsForUsers(logger, userList);
const rustEncryptionSettings = new EncryptionSettings();
rustEncryptionSettings.historyVisibility = toRustHistoryVisibility(this.room.getHistoryVisibility());
@@ -143,7 +205,7 @@ export class RoomEncryptor {
);
if (shareMessages) {
for (const m of shareMessages) {
await this.outgoingRequestProcessor.makeOutgoingRequest(m);
await this.outgoingRequestManager.outgoingRequestProcessor.makeOutgoingRequest(m);
}
}
}
@@ -168,8 +230,10 @@ export class RoomEncryptor {
* @param globalBlacklistUnverifiedDevices - When `true`, it will not send encrypted messages to unverified devices
*/
public async encryptEvent(event: MatrixEvent, globalBlacklistUnverifiedDevices: boolean): Promise<void> {
await this.ensureEncryptionSession(globalBlacklistUnverifiedDevices);
const logger = new LogSpan(this.prefixedLogger, event.getTxnId() ?? "");
await this.ensureEncryptionSession(logger, globalBlacklistUnverifiedDevices);
logger.debug("Encrypting actual message content");
const encryptedContent = await this.olmMachine.encryptRoomEvent(
new RoomId(this.room.roomId),
event.getType(),
@@ -182,6 +246,8 @@ export class RoomEncryptor {
this.olmMachine.identityKeys.curve25519.toBase64(),
this.olmMachine.identityKeys.ed25519.toBase64(),
);
logger.debug("Encrypted event successfully");
}
}
+91 -14
View File
@@ -34,6 +34,7 @@ import { OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { sleep } from "../utils";
import { BackupDecryptor } from "../common-crypto/CryptoBackend";
import { IEncryptedPayload } from "../crypto/aes";
import { ImportRoomKeyProgressData, ImportRoomKeysOpts } from "../crypto-api";
/** Authentification of the backup info, depends on algorithm */
type AuthData = KeyBackupInfo["auth_data"];
@@ -154,8 +155,7 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
logger.info(
`handleBackupSecretReceived: A valid backup decryption key has been received and stored in cache.`,
);
await this.olmMachine.saveBackupDecryptionKey(backupDecryptionKey, backupCheck.backupInfo.version);
await this.saveBackupDecryptionKey(backupDecryptionKey, backupCheck.backupInfo.version);
return true;
} catch (e) {
logger.warn("handleBackupSecretReceived: Invalid backup decryption key", e);
@@ -164,6 +164,59 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
return false;
}
public async saveBackupDecryptionKey(
backupDecryptionKey: RustSdkCryptoJs.BackupDecryptionKey,
version: string,
): Promise<void> {
await this.olmMachine.saveBackupDecryptionKey(backupDecryptionKey, version);
// Emit an event that we have a new backup decryption key, so that the sdk can start
// importing keys from backup if needed.
this.emit(CryptoEvent.KeyBackupDecryptionKeyCached, version);
}
/**
* Import a list of room keys previously exported by exportRoomKeys
*
* @param keys - a list of session export objects
* @param opts - options object
* @returns a promise which resolves once the keys have been imported
*/
public async importRoomKeys(keys: IMegolmSessionData[], opts?: ImportRoomKeysOpts): Promise<void> {
const jsonKeys = JSON.stringify(keys);
await this.olmMachine.importExportedRoomKeys(jsonKeys, (progress: BigInt, total: BigInt): void => {
const importOpt: ImportRoomKeyProgressData = {
total: Number(total),
successes: Number(progress),
stage: "load_keys",
failures: 0,
};
opts?.progressCallback?.(importOpt);
});
}
/**
* Implementation of {@link CryptoBackend#importBackedUpRoomKeys}.
*/
public async importBackedUpRoomKeys(keys: IMegolmSessionData[], opts?: ImportRoomKeysOpts): Promise<void> {
const keysByRoom: Map<RustSdkCryptoJs.RoomId, Map<string, IMegolmSessionData>> = new Map();
for (const key of keys) {
const roomId = new RustSdkCryptoJs.RoomId(key.room_id);
if (!keysByRoom.has(roomId)) {
keysByRoom.set(roomId, new Map());
}
keysByRoom.get(roomId)!.set(key.session_id, key);
}
await this.olmMachine.importBackedUpRoomKeys(keysByRoom, (progress: BigInt, total: BigInt): void => {
const importOpt: ImportRoomKeyProgressData = {
total: Number(total),
successes: Number(progress),
stage: "load_keys",
failures: 0,
};
opts?.progressCallback?.(importOpt);
});
}
private keyBackupCheckInProgress: Promise<KeyBackupCheck | null> | null = null;
/** Helper for `checkKeyBackup` */
@@ -260,7 +313,7 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
}
this.backupKeysLoopRunning = true;
logger.log(`Starting loop for ${this.activeBackupVersion}.`);
logger.log(`Backup: Starting keys upload loop for backup version:${this.activeBackupVersion}.`);
// wait between 0 and `maxDelay` seconds, to avoid backup
// requests from different clients hitting the server all at
@@ -273,27 +326,41 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
while (!this.stopped) {
// Get a batch of room keys to upload
const request: RustSdkCryptoJs.KeysBackupRequest | null = await this.olmMachine.backupRoomKeys();
let request: RustSdkCryptoJs.KeysBackupRequest | null = null;
try {
request = await this.olmMachine.backupRoomKeys();
} catch (err) {
logger.error("Backup: Failed to get keys to backup from rust crypto-sdk", err);
}
if (!request || this.stopped || !this.activeBackupVersion) {
logger.log(`Ending loop for ${this.activeBackupVersion}.`);
logger.log(`Backup: Ending loop for version ${this.activeBackupVersion}.`);
return;
}
try {
await this.outgoingRequestProcessor.makeOutgoingRequest(request);
numFailures = 0;
const keyCount: RustSdkCryptoJs.RoomKeyCounts = await this.olmMachine.roomKeyCounts();
const remaining = keyCount.total - keyCount.backedUp;
this.emit(CryptoEvent.KeyBackupSessionsRemaining, remaining);
if (this.stopped) break;
try {
const keyCount = await this.olmMachine.roomKeyCounts();
const remaining = keyCount.total - keyCount.backedUp;
this.emit(CryptoEvent.KeyBackupSessionsRemaining, remaining);
} catch (err) {
logger.error("Backup: Failed to get key counts from rust crypto-sdk", err);
}
} catch (err) {
numFailures++;
logger.error("Error processing backup request for rust crypto-sdk", err);
logger.error("Backup: Error processing backup request for rust crypto-sdk", err);
if (err instanceof MatrixError) {
const errCode = err.data.errcode;
if (errCode == "M_NOT_FOUND" || errCode == "M_WRONG_ROOM_KEYS_VERSION") {
await this.disableKeyBackup();
logger.log(`Backup: Failed to upload keys to current vesion: ${errCode}.`);
try {
await this.disableKeyBackup();
} catch (error) {
logger.error("Backup: An error occurred while disabling key backup:", error);
}
this.emit(CryptoEvent.KeyBackupFailed, err.data.errcode!);
// There was an active backup and we are out of sync with the server
// force a check server side
@@ -325,7 +392,7 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
*
* @returns Information object from API or null if there is no active backup.
*/
private async requestKeyBackupVersion(): Promise<KeyBackupInfo | null> {
public async requestKeyBackupVersion(): Promise<KeyBackupInfo | null> {
try {
return await this.http.authedRequest<KeyBackupInfo>(
Method.Get,
@@ -379,7 +446,7 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
},
);
this.olmMachine.saveBackupDecryptionKey(randomKey, res.version);
await this.saveBackupDecryptionKey(randomKey, res.version);
return {
version: res.version,
@@ -417,6 +484,14 @@ export class RustBackupManager extends TypedEventEmitter<RustBackupCryptoEvents,
prefix: ClientPrefix.V3,
});
}
/**
* Creates a new backup decryptor for the given private key.
* @param decryptionKey - The private key to use for decryption.
*/
public createBackupDecryptor(decryptionKey: RustSdkCryptoJs.BackupDecryptionKey): BackupDecryptor {
return new RustBackupDecryptor(decryptionKey);
}
}
/**
@@ -489,10 +564,12 @@ export class RustBackupDecryptor implements BackupDecryptor {
export type RustBackupCryptoEvents =
| CryptoEvent.KeyBackupStatus
| CryptoEvent.KeyBackupSessionsRemaining
| CryptoEvent.KeyBackupFailed;
| CryptoEvent.KeyBackupFailed
| CryptoEvent.KeyBackupDecryptionKeyCached;
export type RustBackupCryptoEventMap = {
[CryptoEvent.KeyBackupStatus]: (enabled: boolean) => void;
[CryptoEvent.KeyBackupSessionsRemaining]: (remaining: number) => void;
[CryptoEvent.KeyBackupFailed]: (errCode: string) => void;
[CryptoEvent.KeyBackupDecryptionKeyCached]: (version: string) => void;
};
+4
View File
@@ -67,6 +67,10 @@ export async function initRustCrypto(
storePrefix ?? undefined,
(storePrefix && storePassphrase) ?? undefined,
);
// Disable room key requests, per https://github.com/vector-im/element-web/issues/26524.
olmMachine.roomKeyRequestsEnabled = false;
const rustCrypto = new RustCrypto(logger, olmMachine, http, userId, deviceId, secretStorage, cryptoCallbacks);
await olmMachine.registerRoomKeyUpdatedCallback((sessions: RustSdkCryptoJs.RoomKeyInfo[]) =>
rustCrypto.onRoomKeysUpdated(sessions),
+221 -244
View File
@@ -25,11 +25,11 @@ import { Room } from "../models/room";
import { RoomMember } from "../models/room-member";
import { BackupDecryptor, CryptoBackend, OnSyncCompletedData } from "../common-crypto/CryptoBackend";
import { Logger } from "../logger";
import { ClientPrefix, IHttpOpts, MatrixHttpApi, Method } from "../http-api";
import { IHttpOpts, MatrixHttpApi, Method } from "../http-api";
import { RoomEncryptor } from "./RoomEncryptor";
import { OutgoingRequest, OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { OutgoingRequestProcessor } from "./OutgoingRequestProcessor";
import { KeyClaimManager } from "./KeyClaimManager";
import { encodeUri, MapWithDefault } from "../utils";
import { MapWithDefault } from "../utils";
import {
BackupTrustInfo,
BootstrapCrossSigningOpts,
@@ -44,11 +44,10 @@ import {
EventShieldColour,
EventShieldReason,
GeneratedSecretStorageKey,
ImportRoomKeyProgressData,
ImportRoomKeysOpts,
KeyBackupCheck,
KeyBackupInfo,
KeyBackupSession,
OwnDeviceKeys,
UserVerificationStatus,
VerificationRequest,
} from "../crypto-api";
@@ -65,13 +64,15 @@ import { isVerificationEvent, RustVerificationRequest, verificationMethodIdentif
import { EventType, MsgType } from "../@types/event";
import { CryptoEvent } from "../crypto";
import { TypedEventEmitter } from "../models/typed-event-emitter";
import { RustBackupCryptoEventMap, RustBackupCryptoEvents, RustBackupDecryptor, RustBackupManager } from "./backup";
import { RustBackupCryptoEventMap, RustBackupCryptoEvents, RustBackupManager } from "./backup";
import { TypedReEmitter } from "../ReEmitter";
import { randomString } from "../randomstring";
import { ClientStoppedError } from "../errors";
import { ISignatures } from "../@types/signed";
import { encodeBase64 } from "../base64";
import { DecryptionError } from "../crypto/algorithms";
import { OutgoingRequestsManager } from "./OutgoingRequestsManager";
import { PerSessionKeyBackupDownloader } from "./PerSessionKeyBackupDownloader";
const ALL_VERIFICATION_METHODS = ["m.sas.v1", "m.qr_code.scan.v1", "m.qr_code.show.v1", "m.reciprocate.v1"];
@@ -80,8 +81,6 @@ interface ISignableObject {
unsigned?: object;
}
const KEY_BACKUP_CHECK_RATE_LIMIT = 5000; // ms
/**
* An implementation of {@link CryptoBackend} using the Rust matrix-sdk-crypto.
*
@@ -93,16 +92,6 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
/** whether {@link stop} has been called */
private stopped = false;
/** whether {@link outgoingRequestLoop} is currently running */
private outgoingRequestLoopRunning = false;
/**
* whether we check the outgoing requests queue again after the current check finishes.
*
* This should never be `true` unless `outgoingRequestLoopRunning` is also true.
*/
private outgoingRequestLoopOneMoreLoop = false;
/** mapping of roomId → encryptor class */
private roomEncryptors: Record<string, RoomEncryptor> = {};
@@ -111,8 +100,9 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
private outgoingRequestProcessor: OutgoingRequestProcessor;
private crossSigningIdentity: CrossSigningIdentity;
private readonly backupManager: RustBackupManager;
private outgoingRequestsManager: OutgoingRequestsManager;
private sessionLastCheckAttemptedTime: Record<string, number> = {}; // When did we last try to check the server for a given session id?
private readonly perSessionBackupDownloader: PerSessionKeyBackupDownloader;
private readonly reemitter = new TypedReEmitter<RustCryptoEvents, RustCryptoEventMap>(this);
@@ -143,14 +133,30 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
) {
super();
this.outgoingRequestProcessor = new OutgoingRequestProcessor(olmMachine, http);
this.outgoingRequestsManager = new OutgoingRequestsManager(
this.logger,
olmMachine,
this.outgoingRequestProcessor,
);
this.keyClaimManager = new KeyClaimManager(olmMachine, this.outgoingRequestProcessor);
this.eventDecryptor = new EventDecryptor(this.logger, olmMachine, this);
this.backupManager = new RustBackupManager(olmMachine, http, this.outgoingRequestProcessor);
this.perSessionBackupDownloader = new PerSessionKeyBackupDownloader(
this.logger,
this.olmMachine,
this.http,
this.backupManager,
);
this.eventDecryptor = new EventDecryptor(this.logger, olmMachine, this.perSessionBackupDownloader);
this.reemitter.reEmit(this.backupManager, [
CryptoEvent.KeyBackupStatus,
CryptoEvent.KeyBackupSessionsRemaining,
CryptoEvent.KeyBackupFailed,
CryptoEvent.KeyBackupDecryptionKeyCached,
]);
this.crossSigningIdentity = new CrossSigningIdentity(olmMachine, this.outgoingRequestProcessor, secretStorage);
@@ -159,75 +165,6 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
this.checkKeyBackupAndEnable();
}
/**
* Starts an attempt to retrieve a session from a key backup, if enough time
* has elapsed since the last check for this session id.
*
* If a backup is found, it is decrypted and imported.
*
* @param targetRoomId - ID of the room that the session is used in.
* @param targetSessionId - ID of the session for which to check backup.
*/
public startQueryKeyBackupRateLimited(targetRoomId: string, targetSessionId: string): void {
const now = new Date().getTime();
const lastCheck = this.sessionLastCheckAttemptedTime[targetSessionId];
if (!lastCheck || now - lastCheck > KEY_BACKUP_CHECK_RATE_LIMIT) {
this.sessionLastCheckAttemptedTime[targetSessionId!] = now;
this.queryKeyBackup(targetRoomId, targetSessionId).catch((e) => {
this.logger.error(`Unhandled error while checking key backup for session ${targetSessionId}`, e);
});
} else {
const lastCheckStr = new Date(lastCheck).toISOString();
this.logger.debug(
`Not checking key backup for session ${targetSessionId} (last checked at ${lastCheckStr})`,
);
}
}
/**
* Helper for {@link RustCrypto#startQueryKeyBackupRateLimited}.
*
* Requests the backup and imports it. Doesn't do any rate-limiting.
*
* @param targetRoomId - ID of the room that the session is used in.
* @param targetSessionId - ID of the session for which to check backup.
*/
private async queryKeyBackup(targetRoomId: string, targetSessionId: string): Promise<void> {
const backupKeys: RustSdkCryptoJs.BackupKeys = await this.olmMachine.getBackupKeys();
if (!backupKeys.decryptionKey) {
this.logger.debug(`Not checking key backup for session ${targetSessionId} (no decryption key)`);
return;
}
this.logger.debug(`Checking key backup for session ${targetSessionId}`);
const version = backupKeys.backupVersion;
const path = encodeUri("/room_keys/keys/$roomId/$sessionId", {
$roomId: targetRoomId,
$sessionId: targetSessionId,
});
let res: KeyBackupSession;
try {
res = await this.http.authedRequest<KeyBackupSession>(Method.Get, path, { version }, undefined, {
prefix: ClientPrefix.V3,
});
} catch (e) {
this.logger.info(`No luck requesting key backup for session ${targetSessionId}: ${e}`);
return;
}
if (this.stopped) return;
const backupDecryptor = new RustBackupDecryptor(backupKeys.decryptionKey);
const sessionsToImport: Record<string, KeyBackupSession> = { [targetSessionId]: res };
const keys = await backupDecryptor.decryptSessions(sessionsToImport);
for (const k of keys) {
k.room_id = targetRoomId;
}
await this.importRoomKeys(keys);
}
/**
* Return the OlmMachine only if {@link RustCrypto#stop} has not been called.
*
@@ -267,6 +204,8 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
this.keyClaimManager.stop();
this.backupManager.stop();
this.outgoingRequestsManager.stop();
this.perSessionBackupDownloader.stop();
// make sure we close() the OlmMachine; doing so means that all the Rust objects will be
// cleaned up; in particular, the indexeddb connections will be closed, which means they
@@ -372,11 +311,29 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
return `Rust SDK ${versions.matrix_sdk_crypto} (${versions.git_sha}), Vodozemac ${versions.vodozemac}`;
}
/**
* Implementation of {@link CryptoApi#getOwnDeviceKeys}.
*/
public async getOwnDeviceKeys(): Promise<OwnDeviceKeys> {
const device: RustSdkCryptoJs.Device = await this.olmMachine.getDevice(
this.olmMachine.userId,
this.olmMachine.deviceId,
);
// could be undefined if there is no such algorithm for that device.
if (device.curve25519Key && device.ed25519Key) {
return {
ed25519: device.ed25519Key.toBase64(),
curve25519: device.curve25519Key.toBase64(),
};
}
throw new Error("Device keys not found");
}
public prepareToEncrypt(room: Room): void {
const encryptor = this.roomEncryptors[room.roomId];
if (encryptor) {
encryptor.ensureEncryptionSession(this.globalBlacklistUnverifiedDevices);
encryptor.prepareForEncryption(this.globalBlacklistUnverifiedDevices);
}
}
@@ -390,17 +347,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
}
public async importRoomKeys(keys: IMegolmSessionData[], opts?: ImportRoomKeysOpts): Promise<void> {
// TODO when backup support will be added we would need to expose the `from_backup` flag in the bindings
const jsonKeys = JSON.stringify(keys);
await this.olmMachine.importRoomKeys(jsonKeys, (progress: BigInt, total: BigInt) => {
const importOpt: ImportRoomKeyProgressData = {
total: Number(total),
successes: Number(progress),
stage: "load_keys",
failures: 0,
};
opts?.progressCallback?.(importOpt);
});
return await this.backupManager.importRoomKeys(keys, opts);
}
/**
@@ -426,6 +373,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
await this.outgoingRequestProcessor.makeOutgoingRequest(request);
}
const userIdentity = await this.olmMachine.getIdentity(rustTrackedUser);
userIdentity?.free();
return userIdentity !== undefined;
} else if (downloadUncached) {
// Download the cross signing keys and check if the master key is available
@@ -455,7 +403,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
*/
public async getUserDeviceInfo(userIds: string[], downloadUncached = false): Promise<DeviceMap> {
const deviceMapByUserId = new Map<string, Map<string, Device>>();
const rustTrackedUsers: Set<RustSdkCryptoJs.UserId> = await this.olmMachine.trackedUsers();
const rustTrackedUsers: Set<RustSdkCryptoJs.UserId> = await this.getOlmMachineOrThrow().trackedUsers();
// Convert RustSdkCryptoJs.UserId to a `Set<string>`
const trackedUsers = new Set<string>();
@@ -505,7 +453,10 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
// To fix this, we explicitly call `.free` on each of the objects, which tells the rust code to drop the
// allocated memory and decrement the refcounts for the crypto store.
const userDevices: RustSdkCryptoJs.UserDevices = await this.olmMachine.getUserDevices(rustUserId);
// Wait for up to a second for any in-flight device list requests to complete.
// The reason for this isn't so much to avoid races (some level of raciness is
// inevitable for this method) but to make testing easier.
const userDevices: RustSdkCryptoJs.UserDevices = await this.olmMachine.getUserDevices(rustUserId, 1);
try {
const deviceArray: RustSdkCryptoJs.Device[] = userDevices.devices();
try {
@@ -563,7 +514,34 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
if (!device) {
throw new Error(`Unknown device ${userId}|${deviceId}`);
}
await device.setLocalTrust(verified ? RustSdkCryptoJs.LocalTrust.Verified : RustSdkCryptoJs.LocalTrust.Unset);
try {
await device.setLocalTrust(
verified ? RustSdkCryptoJs.LocalTrust.Verified : RustSdkCryptoJs.LocalTrust.Unset,
);
} finally {
device.free();
}
}
/**
* Blindly cross-sign one of our other devices.
*
* Implementation of {@link CryptoApi#crossSignDevice}.
*/
public async crossSignDevice(deviceId: string): Promise<void> {
const device: RustSdkCryptoJs.Device | undefined = await this.olmMachine.getDevice(
new RustSdkCryptoJs.UserId(this.userId),
new RustSdkCryptoJs.DeviceId(deviceId),
);
if (!device) {
throw new Error(`Unknown device ${deviceId}`);
}
try {
const outgoingRequest: RustSdkCryptoJs.SignatureUploadRequest = await device.verify();
await this.outgoingRequestProcessor.makeOutgoingRequest(outgoingRequest);
} finally {
device.free();
}
}
/**
@@ -579,13 +557,16 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
);
if (!device) return null;
return new DeviceVerificationStatus({
signedByOwner: device.isCrossSignedByOwner(),
crossSigningVerified: device.isCrossSigningTrusted(),
localVerified: device.isLocallyTrusted(),
trustCrossSignedDevices: this._trustCrossSignedDevices,
});
try {
return new DeviceVerificationStatus({
signedByOwner: device.isCrossSignedByOwner(),
crossSigningVerified: device.isCrossSigningTrusted(),
localVerified: device.isLocallyTrusted(),
trustCrossSignedDevices: this._trustCrossSignedDevices,
});
} finally {
device.free();
}
}
/**
@@ -593,11 +574,13 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
*/
public async getUserVerificationStatus(userId: string): Promise<UserVerificationStatus> {
const userIdentity: RustSdkCryptoJs.UserIdentity | RustSdkCryptoJs.OwnUserIdentity | undefined =
await this.olmMachine.getIdentity(new RustSdkCryptoJs.UserId(userId));
await this.getOlmMachineOrThrow().getIdentity(new RustSdkCryptoJs.UserId(userId));
if (userIdentity === undefined) {
return new UserVerificationStatus(false, false, false);
}
return new UserVerificationStatus(userIdentity.isVerified(), false, false);
const verified = userIdentity.isVerified();
userIdentity.free();
return new UserVerificationStatus(verified, false, false);
}
/**
@@ -622,42 +605,51 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
const userIdentity: RustSdkCryptoJs.OwnUserIdentity | undefined = await this.olmMachine.getIdentity(
new RustSdkCryptoJs.UserId(this.userId),
);
const crossSigningStatus: RustSdkCryptoJs.CrossSigningStatus = await this.olmMachine.crossSigningStatus();
const privateKeysOnDevice =
crossSigningStatus.hasMaster && crossSigningStatus.hasUserSigning && crossSigningStatus.hasSelfSigning;
if (!userIdentity || !privateKeysOnDevice) {
// The public or private keys are not available on this device
if (!userIdentity) {
// The public keys are not available on this device
return null;
}
if (!userIdentity.isVerified()) {
// We have both public and private keys, but they don't match!
return null;
}
try {
const crossSigningStatus: RustSdkCryptoJs.CrossSigningStatus = await this.olmMachine.crossSigningStatus();
let key: string;
switch (type) {
case CrossSigningKey.Master:
key = userIdentity.masterKey;
break;
case CrossSigningKey.SelfSigning:
key = userIdentity.selfSigningKey;
break;
case CrossSigningKey.UserSigning:
key = userIdentity.userSigningKey;
break;
default:
// Unknown type
const privateKeysOnDevice =
crossSigningStatus.hasMaster && crossSigningStatus.hasUserSigning && crossSigningStatus.hasSelfSigning;
if (!privateKeysOnDevice) {
// The private keys are not available on this device
return null;
}
}
const parsedKey: CrossSigningKeyInfo = JSON.parse(key);
// `keys` is an object with { [`ed25519:${pubKey}`]: pubKey }
// We assume only a single key, and we want the bare form without type
// prefix, so we select the values.
return Object.values(parsedKey.keys)[0];
if (!userIdentity.isVerified()) {
// We have both public and private keys, but they don't match!
return null;
}
let key: string;
switch (type) {
case CrossSigningKey.Master:
key = userIdentity.masterKey;
break;
case CrossSigningKey.SelfSigning:
key = userIdentity.selfSigningKey;
break;
case CrossSigningKey.UserSigning:
key = userIdentity.userSigningKey;
break;
default:
// Unknown type
return null;
}
const parsedKey: CrossSigningKeyInfo = JSON.parse(key);
// `keys` is an object with { [`ed25519:${pubKey}`]: pubKey }
// We assume only a single key, and we want the bare form without type
// prefix, so we select the values.
return Object.values(parsedKey.keys)[0];
} finally {
userIdentity.free();
}
}
/**
@@ -705,6 +697,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
}
// Create a new storage key and add it to secret storage
this.logger.info("bootstrapSecretStorage: creating new secret storage key");
const recoveryKey = await createSecretStorageKey();
await this.addSecretStorageKeyToSecretStorage(recoveryKey);
}
@@ -719,6 +712,8 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
hasPrivateKeys &&
(isNewSecretStorageKeyNeeded || !(await secretStorageContainsCrossSigningKeys(this.secretStorage)))
) {
this.logger.info("bootstrapSecretStorage: cross-signing keys not yet exported; doing so now.");
const crossSigningPrivateKeys: RustSdkCryptoJs.CrossSigningKeyExport =
await this.olmMachine.exportCrossSigningKeys();
@@ -801,6 +796,8 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
Boolean(userIdentity?.masterKey) &&
Boolean(userIdentity?.selfSigningKey) &&
Boolean(userIdentity?.userSigningKey);
userIdentity?.free();
const privateKeysInSecretStorage = await secretStorageContainsCrossSigningKeys(this.secretStorage);
const crossSigningStatus: RustSdkCryptoJs.CrossSigningStatus | null =
await this.getOlmMachineOrThrow().crossSigningStatus();
@@ -871,6 +868,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
.map(
(request) =>
new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
@@ -901,6 +899,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
if (request) {
return new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
@@ -918,23 +917,32 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
if (!userIdentity) throw new Error(`unknown userId ${userId}`);
// Transform the verification methods into rust objects
const methods = this._supportedVerificationMethods.map((method) =>
verificationMethodIdentifierToMethod(method),
);
// Get the request content to send to the DM room
const verificationEventContent: string = await userIdentity.verificationRequestContent(methods);
try {
// Transform the verification methods into rust objects
const methods = this._supportedVerificationMethods.map((method) =>
verificationMethodIdentifierToMethod(method),
);
// Get the request content to send to the DM room
const verificationEventContent: string = await userIdentity.verificationRequestContent(methods);
// Send the request content to send to the DM room
const eventId = await this.sendVerificationRequestContent(roomId, verificationEventContent);
// Send the request content to send to the DM room
const eventId = await this.sendVerificationRequestContent(roomId, verificationEventContent);
// Get a verification request
const request: RustSdkCryptoJs.VerificationRequest = await userIdentity.requestVerification(
new RustSdkCryptoJs.RoomId(roomId),
new RustSdkCryptoJs.EventId(eventId),
methods,
);
return new RustVerificationRequest(request, this.outgoingRequestProcessor, this._supportedVerificationMethods);
// Get a verification request
const request: RustSdkCryptoJs.VerificationRequest = await userIdentity.requestVerification(
new RustSdkCryptoJs.RoomId(roomId),
new RustSdkCryptoJs.EventId(eventId),
methods,
);
return new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
);
} finally {
userIdentity.free();
}
}
/**
@@ -996,12 +1004,21 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
throw new Error("cannot request verification for this device when there is no existing cross-signing key");
}
const [request, outgoingRequest]: [RustSdkCryptoJs.VerificationRequest, RustSdkCryptoJs.ToDeviceRequest] =
await userIdentity.requestVerification(
this._supportedVerificationMethods.map(verificationMethodIdentifierToMethod),
try {
const [request, outgoingRequest]: [RustSdkCryptoJs.VerificationRequest, RustSdkCryptoJs.ToDeviceRequest] =
await userIdentity.requestVerification(
this._supportedVerificationMethods.map(verificationMethodIdentifierToMethod),
);
await this.outgoingRequestProcessor.makeOutgoingRequest(outgoingRequest);
return new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
);
await this.outgoingRequestProcessor.makeOutgoingRequest(outgoingRequest);
return new RustVerificationRequest(request, this.outgoingRequestProcessor, this._supportedVerificationMethods);
} finally {
userIdentity.free();
}
}
/**
@@ -1026,12 +1043,21 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
throw new Error("Not a known device");
}
const [request, outgoingRequest]: [RustSdkCryptoJs.VerificationRequest, RustSdkCryptoJs.ToDeviceRequest] =
await device.requestVerification(
this._supportedVerificationMethods.map(verificationMethodIdentifierToMethod),
try {
const [request, outgoingRequest]: [RustSdkCryptoJs.VerificationRequest, RustSdkCryptoJs.ToDeviceRequest] =
await device.requestVerification(
this._supportedVerificationMethods.map(verificationMethodIdentifierToMethod),
);
await this.outgoingRequestProcessor.makeOutgoingRequest(outgoingRequest);
return new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
);
await this.outgoingRequestProcessor.makeOutgoingRequest(outgoingRequest);
return new RustVerificationRequest(request, this.outgoingRequestProcessor, this._supportedVerificationMethods);
} finally {
device.free();
}
}
/**
@@ -1062,7 +1088,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
throw new Error("storeSessionBackupPrivateKey: version is required");
}
await this.olmMachine.saveBackupDecryptionKey(
await this.backupManager.saveBackupDecryptionKey(
RustSdkCryptoJs.BackupDecryptionKey.fromBase64(base64Key),
version,
);
@@ -1164,7 +1190,14 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
throw new Error(`getBackupDecryptor key mismatch error`);
}
return new RustBackupDecryptor(backupDecryptionKey);
return this.backupManager.createBackupDecryptor(backupDecryptionKey);
}
/**
* Implementation of {@link CryptoBackend#importBackedUpRoomKeys}.
*/
public async importBackedUpRoomKeys(keys: IMegolmSessionData[], opts?: ImportRoomKeysOpts): Promise<void> {
return await this.backupManager.importBackedUpRoomKeys(keys, opts);
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@@ -1270,15 +1303,11 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
this.roomEncryptors[room.roomId] = new RoomEncryptor(
this.olmMachine,
this.keyClaimManager,
this.outgoingRequestProcessor,
this.outgoingRequestsManager,
room,
config,
);
}
// start tracking devices for any users already known to be in this room.
const members = await room.getEncryptionTargetMembers();
await this.olmMachine.updateTrackedUsers(members.map((u) => new RustSdkCryptoJs.UserId(u.userId)));
}
/** called by the sync loop after processing each sync.
@@ -1290,7 +1319,9 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
public onSyncCompleted(syncState: OnSyncCompletedData): void {
// Processing the /sync may have produced new outgoing requests which need sending, so kick off the outgoing
// request loop, if it's not already running.
this.outgoingRequestLoop();
this.outgoingRequestsManager.doProcessOutgoingRequests().catch((e) => {
this.logger.warn("onSyncCompleted: Error processing outgoing requests", e);
});
}
/**
@@ -1314,7 +1345,12 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
if (request) {
this.emit(
CryptoEvent.VerificationRequestReceived,
new RustVerificationRequest(request, this.outgoingRequestProcessor, this._supportedVerificationMethods),
new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
),
);
}
}
@@ -1531,6 +1567,7 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
this.emit(
CryptoEvent.VerificationRequestReceived,
new RustVerificationRequest(
this.olmMachine,
request,
this.outgoingRequestProcessor,
this._supportedVerificationMethods,
@@ -1540,68 +1577,10 @@ export class RustCrypto extends TypedEventEmitter<RustCryptoEvents, RustCryptoEv
}
// that may have caused us to queue up outgoing requests, so make sure we send them.
this.outgoingRequestLoop();
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Outgoing requests
//
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/** start the outgoing request loop if it is not already running */
private outgoingRequestLoop(): void {
if (this.outgoingRequestLoopRunning) {
// The loop is already running, but we have reason to believe that there may be new items in the queue.
//
// There is potential for a race whereby the item is added *after* `OlmMachine.outgoingRequests` checks
// the queue, but *before* it returns. In such a case, the item could sit there unnoticed for some time.
//
// In order to circumvent the race, we set a flag which tells the loop to go round once again even if the
// queue appears to be empty.
this.outgoingRequestLoopOneMoreLoop = true;
return;
}
// fire off the loop in the background
this.outgoingRequestLoopInner().catch((e) => {
this.logger.error("Error processing outgoing-message requests from rust crypto-sdk", e);
this.outgoingRequestsManager.doProcessOutgoingRequests().catch((e) => {
this.logger.warn("onKeyVerificationRequest: Error processing outgoing requests", e);
});
}
private async outgoingRequestLoopInner(): Promise<void> {
/* istanbul ignore if */
if (this.outgoingRequestLoopRunning) {
throw new Error("Cannot run two outgoing request loops");
}
this.outgoingRequestLoopRunning = true;
try {
while (!this.stopped) {
// we clear the "one more loop" flag just before calling `OlmMachine.outgoingRequests()`, so we can tell
// if `this.outgoingRequestLoop()` was called while `OlmMachine.outgoingRequests()` was running.
this.outgoingRequestLoopOneMoreLoop = false;
const outgoingRequests: Object[] = await this.olmMachine.outgoingRequests();
if (this.stopped) {
// we've been told to stop while `outgoingRequests` was running: exit the loop without processing
// any of the returned requests (anything important will happen next time the client starts.)
return;
}
if (outgoingRequests.length === 0 && !this.outgoingRequestLoopOneMoreLoop) {
// `OlmMachine.outgoingRequests` returned no messages, and there was no call to
// `this.outgoingRequestLoop()` while it was running. We can stop the loop for a while.
return;
}
for (const msg of outgoingRequests) {
await this.outgoingRequestProcessor.makeOutgoingRequest(msg as OutgoingRequest);
}
}
} finally {
this.outgoingRequestLoopRunning = false;
}
}
}
class EventDecryptor {
@@ -1617,14 +1596,10 @@ class EventDecryptor {
public constructor(
private readonly logger: Logger,
private readonly olmMachine: RustSdkCryptoJs.OlmMachine,
private readonly crypto: RustCrypto,
private readonly perSessionBackupDownloader: PerSessionKeyBackupDownloader,
) {}
public async attemptEventDecryption(event: MatrixEvent): Promise<IEventDecryptionResult> {
this.logger.info(
`Attempting decryption of event ${event.getId()} in ${event.getRoomId()} from ${event.getSender()}`,
);
// add the event to the pending list *before* attempting to decrypt.
// then, if the key turns up while decryption is in progress (and
// decryption fails), we will schedule a retry.
@@ -1662,7 +1637,7 @@ class EventDecryptor {
session: content.sender_key + "|" + content.session_id,
},
);
this.crypto.startQueryKeyBackupRateLimited(
this.perSessionBackupDownloader.onDecryptionKeyMissingError(
event.getRoomId()!,
event.getWireContent().session_id!,
);
@@ -1676,7 +1651,7 @@ class EventDecryptor {
session: content.sender_key + "|" + content.session_id,
},
);
this.crypto.startQueryKeyBackupRateLimited(
this.perSessionBackupDownloader.onDecryptionKeyMissingError(
event.getRoomId()!,
event.getWireContent().session_id!,
);
@@ -1845,4 +1820,6 @@ type RustCryptoEventMap = {
* Fires when the trust status of a user changes.
*/
[CryptoEvent.UserTrustStatusChanged]: (userId: string, userTrustLevel: UserVerificationStatus) => void;
[CryptoEvent.KeyBackupDecryptionKeyCached]: (version: string) => void;
} & RustBackupCryptoEventMap;
+24 -3
View File
@@ -57,11 +57,13 @@ export class RustVerificationRequest
/**
* Construct a new RustVerificationRequest to wrap the rust-level `VerificationRequest`.
*
* @param inner - VerificationRequest from the Rust SDK
* @param outgoingRequestProcessor - `OutgoingRequestProcessor` to use for making outgoing HTTP requests
* @param supportedVerificationMethods - Verification methods to use when `accept()` is called
* @param olmMachine - The `OlmMachine` from the underlying rust crypto sdk.
* @param inner - VerificationRequest from the Rust SDK.
* @param outgoingRequestProcessor - `OutgoingRequestProcessor` to use for making outgoing HTTP requests.
* @param supportedVerificationMethods - Verification methods to use when `accept()` is called.
*/
public constructor(
private readonly olmMachine: RustSdkCryptoJs.OlmMachine,
private readonly inner: RustSdkCryptoJs.VerificationRequest,
private readonly outgoingRequestProcessor: OutgoingRequestProcessor,
private readonly supportedVerificationMethods: string[],
@@ -135,6 +137,15 @@ export class RustVerificationRequest
return this.inner.otherDeviceId?.toString();
}
/** Get the other device involved in the verification, if it is known */
private async getOtherDevice(): Promise<undefined | RustSdkCryptoJs.Device> {
const otherDeviceId = this.inner.otherDeviceId;
if (!otherDeviceId) {
return undefined;
}
return await this.olmMachine.getDevice(this.inner.otherUserId, otherDeviceId, 5);
}
/** True if the other party in this request is one of this user's own devices. */
public get isSelfVerification(): boolean {
return this.inner.isSelfVerification();
@@ -322,6 +333,11 @@ export class RustVerificationRequest
throw new Error(`Unsupported verification method ${method}`);
}
// make sure that we have a list of the other user's devices (workaround https://github.com/matrix-org/matrix-rust-sdk/issues/2896)
if (!(await this.getOtherDevice())) {
throw new Error("startVerification(): other device is unknown");
}
const res:
| [RustSdkCryptoJs.Sas, RustSdkCryptoJs.RoomMessageRequest | RustSdkCryptoJs.ToDeviceRequest]
| undefined = await this.inner.startSas();
@@ -392,6 +408,11 @@ export class RustVerificationRequest
* Implementation of {@link Crypto.VerificationRequest#generateQRCode}.
*/
public async generateQRCode(): Promise<Buffer | undefined> {
// make sure that we have a list of the other user's devices (workaround https://github.com/matrix-org/matrix-rust-sdk/issues/2896)
if (!(await this.getOtherDevice())) {
throw new Error("generateQRCode(): other device is unknown");
}
const innerVerifier: RustSdkCryptoJs.Qr | undefined = await this.inner.generateQrCode();
// If we are unable to generate a QRCode, we return undefined
if (!innerVerifier) return;
+23
View File
@@ -21,6 +21,7 @@ import { logger } from "./logger";
import { MatrixClient } from "./client";
import { EventTimelineSet } from "./models/event-timeline-set";
import { MatrixEvent } from "./models/event";
import { Room, RoomEvent } from "./models/room";
/**
* @internal
@@ -74,6 +75,10 @@ export class TimelineWindow {
* are received from /sync; you should arrange to call {@link TimelineWindow#paginate}
* on {@link RoomEvent.Timeline} events.
*
* <p>Note that constructing an instance of this class for a room adds a
* listener for RoomEvent.Timeline events which is never removed. In theory
* this should not cause a leak since the EventEmitter uses weak mappings.
*
* @param client - MatrixClient to be used for context/pagination
* requests.
*
@@ -87,6 +92,7 @@ export class TimelineWindow {
opts: IOpts = {},
) {
this.windowLimit = opts.windowLimit || 1000;
timelineSet.room?.on(RoomEvent.Timeline, this.onTimelineEvent.bind(this));
}
/**
@@ -193,6 +199,23 @@ export class TimelineWindow {
return false;
}
private onTimelineEvent(_event?: MatrixEvent, _room?: Room, _atStart?: boolean, removed?: boolean): void {
if (removed) {
this.onEventRemoved();
}
}
/**
* If an event was removed, meaning this window is longer than the timeline,
* shorten the window.
*/
private onEventRemoved(): void {
const events = this.getEvents();
if (events.length > 0 && events[events.length - 1] === undefined && this.end) {
this.end.index--;
}
}
/**
* Check if this window can be extended
*
-1
View File
@@ -133,7 +133,6 @@ export class GroupCallEventHandler {
break;
}
logger.debug(`GroupCallEventHandler createGroupCallForRoom() processed room (roomId=${room.roomId})`);
this.getRoomDeferred(room.roomId).resolve!();
}
+844 -857
View File
File diff suppressed because it is too large Load Diff