v2.4.3

Prepare changelog for v2.4.3
v2.4.3-rc.1
2019-11-04 13:51:50 +00:00 · 2019-11-04 13:51:49 +00:00 · 2019-10-30 16:40:25 +00:00 · 2019-10-30 16:40:25 +00:00 · 2019-10-26 13:45:24 -04:00 · 2019-10-25 23:24:30 -04:00
70 changed files with 3683 additions and 1183 deletions
@@ -53,6 +53,7 @@ module.exports = {
        // rules we do not want from the google styleguide
        "object-curly-spacing": ["off"],
        "spaced-comment": ["off"],
+        "guard-for-in": ["off"],

        // in principle we prefer single quotes, but life is too short
        quotes: ["off"],
@@ -0,0 +1,2 @@
+patreon: matrixdotorg
+liberapay: matrixdotorg
@@ -1,3 +1,321 @@
+Changes in [2.4.3](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.3) (2019-11-04)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.3-rc.1...v2.4.3)
+
+ * No changes since rc.1
+
+Changes in [2.4.3-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.3-rc.1) (2019-10-30)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.2...v2.4.3-rc.1)
+
+ * fix the path in references to logger.js
+   [\#1056](https://github.com/matrix-org/matrix-js-sdk/pull/1056)
+ * verification in DMs
+   [\#1050](https://github.com/matrix-org/matrix-js-sdk/pull/1050)
+ * Properly documented the function possible returns
+   [\#1054](https://github.com/matrix-org/matrix-js-sdk/pull/1054)
+ * Downgrade to Bluebird 3.5.5 to fix Firefox
+   [\#1055](https://github.com/matrix-org/matrix-js-sdk/pull/1055)
+ * Upgrade safe deps to latest major version
+   [\#1053](https://github.com/matrix-org/matrix-js-sdk/pull/1053)
+ * Don't include .js in the import string.
+   [\#1052](https://github.com/matrix-org/matrix-js-sdk/pull/1052)
+
+Changes in [2.4.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.2) (2019-10-18)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.2-rc.1...v2.4.2)
+
+ * No changes since v2.4.2-rc.1
+
+Changes in [2.4.2-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.2-rc.1) (2019-10-09)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.1...v2.4.2-rc.1)
+
+ * Log state of Olm sessions
+   [\#1047](https://github.com/matrix-org/matrix-js-sdk/pull/1047)
+ * Add method to get access to all timelines
+   [\#1048](https://github.com/matrix-org/matrix-js-sdk/pull/1048)
+
+Changes in [2.4.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.1) (2019-10-01)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.0...v2.4.1)
+
+ * Upgrade deps
+   [\#1046](https://github.com/matrix-org/matrix-js-sdk/pull/1046)
+ * Ignore crypto events with no content
+   [\#1043](https://github.com/matrix-org/matrix-js-sdk/pull/1043)
+
+Changes in [2.4.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.0) (2019-09-27)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.4.0-rc.1...v2.4.0)
+
+ * Clean Yarn cache during release
+   [\#1045](https://github.com/matrix-org/matrix-js-sdk/pull/1045)
+
+Changes in [2.4.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.4.0-rc.1) (2019-09-25)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.2...v2.4.0-rc.1)
+
+ * Remove id_server from creds for interactive auth
+   [\#1044](https://github.com/matrix-org/matrix-js-sdk/pull/1044)
+ * Remove IS details from requestToken to HS
+   [\#1041](https://github.com/matrix-org/matrix-js-sdk/pull/1041)
+ * Add support for sending MSISDN tokens to alternate URLs
+   [\#1040](https://github.com/matrix-org/matrix-js-sdk/pull/1040)
+ * Add separate 3PID add and bind APIs
+   [\#1038](https://github.com/matrix-org/matrix-js-sdk/pull/1038)
+ * Bump eslint-utils from 1.4.0 to 1.4.2
+   [\#1037](https://github.com/matrix-org/matrix-js-sdk/pull/1037)
+ * Handle WebRTC security errors as non-fatal
+   [\#1036](https://github.com/matrix-org/matrix-js-sdk/pull/1036)
+ * Check for r0.6.0 support in addition to unstable feature flags
+   [\#1035](https://github.com/matrix-org/matrix-js-sdk/pull/1035)
+ * Update room members on member event redaction
+   [\#1030](https://github.com/matrix-org/matrix-js-sdk/pull/1030)
+ * Support hidden read receipts
+   [\#1028](https://github.com/matrix-org/matrix-js-sdk/pull/1028)
+ * Do 3pid lookups in lowercase
+   [\#1029](https://github.com/matrix-org/matrix-js-sdk/pull/1029)
+ * Add Synapse admin functions for deactivating a user
+   [\#1027](https://github.com/matrix-org/matrix-js-sdk/pull/1027)
+ * Fix addPendingEvent with pending event order == chronological
+   [\#1026](https://github.com/matrix-org/matrix-js-sdk/pull/1026)
+ * Add AutoDiscovery.getRawClientConfig() for easy .well-known lookups
+   [\#1024](https://github.com/matrix-org/matrix-js-sdk/pull/1024)
+ * Don't convert errors to JSON if they are JSON already
+   [\#1025](https://github.com/matrix-org/matrix-js-sdk/pull/1025)
+ * Send id_access_token to HS for use in proxied IS requests
+   [\#1022](https://github.com/matrix-org/matrix-js-sdk/pull/1022)
+ * Clean up JSON handling in identity server requests
+   [\#1023](https://github.com/matrix-org/matrix-js-sdk/pull/1023)
+ * Use the v2 (hashed) lookup for identity server queries
+   [\#1021](https://github.com/matrix-org/matrix-js-sdk/pull/1021)
+ * Add getIdServer() & doesServerRequireIdServerParam()
+   [\#1018](https://github.com/matrix-org/matrix-js-sdk/pull/1018)
+ * Make requestToken endpoints work without ID Server
+   [\#1019](https://github.com/matrix-org/matrix-js-sdk/pull/1019)
+ * Fix setIdentityServer
+   [\#1016](https://github.com/matrix-org/matrix-js-sdk/pull/1016)
+ * Change ICE fallback server and make fallback opt-in
+   [\#1015](https://github.com/matrix-org/matrix-js-sdk/pull/1015)
+ * Throw an exception if trying to do an ID server request with no ID server
+   [\#1014](https://github.com/matrix-org/matrix-js-sdk/pull/1014)
+ * Add setIdentityServerUrl
+   [\#1013](https://github.com/matrix-org/matrix-js-sdk/pull/1013)
+ * Add matrix base API to report an event
+   [\#1011](https://github.com/matrix-org/matrix-js-sdk/pull/1011)
+ * Fix POST body for v2 IS requests
+   [\#1010](https://github.com/matrix-org/matrix-js-sdk/pull/1010)
+ * Add API for bulk lookup on the Identity Server
+   [\#1009](https://github.com/matrix-org/matrix-js-sdk/pull/1009)
+ * Remove deprecated authedRequestWithPrefix and requestWithPrefix
+   [\#1000](https://github.com/matrix-org/matrix-js-sdk/pull/1000)
+ * Add API for checking IS account info
+   [\#1007](https://github.com/matrix-org/matrix-js-sdk/pull/1007)
+ * Support rewriting push rules when our internal defaults change
+   [\#1006](https://github.com/matrix-org/matrix-js-sdk/pull/1006)
+ * Upgrade dependencies
+   [\#1005](https://github.com/matrix-org/matrix-js-sdk/pull/1005)
+
+Changes in [2.3.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.2) (2019-09-16)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.2-rc.1...v2.3.2)
+
+ * [Release] Fix addPendingEvent with pending event order == chronological
+   [\#1034](https://github.com/matrix-org/matrix-js-sdk/pull/1034)
+
+Changes in [2.3.2-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.2-rc.1) (2019-09-13)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.1...v2.3.2-rc.1)
+
+ * Synapse admin functions to release
+   [\#1033](https://github.com/matrix-org/matrix-js-sdk/pull/1033)
+ * [To Release] Add matrix base API to report an event
+   [\#1032](https://github.com/matrix-org/matrix-js-sdk/pull/1032)
+
+Changes in [2.3.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.1) (2019-09-12)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.1-rc.1...v2.3.1)
+
+ * No changes since rc.1
+
+Changes in [2.3.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.1-rc.1) (2019-09-11)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.0...v2.3.1-rc.1)
+
+ * Update room members on member event redaction
+   [\#1031](https://github.com/matrix-org/matrix-js-sdk/pull/1031)
+
+Changes in [2.3.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.0) (2019-08-05)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.3.0-rc.1...v2.3.0)
+
+ * [release] Support rewriting push rules when our internal defaults change
+   [\#1008](https://github.com/matrix-org/matrix-js-sdk/pull/1008)
+
+Changes in [2.3.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.3.0-rc.1) (2019-07-31)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.2.0...v2.3.0-rc.1)
+
+ * Add support for IS v2 API with authentication
+   [\#1002](https://github.com/matrix-org/matrix-js-sdk/pull/1002)
+ * Tombstone bugfixes
+   [\#1001](https://github.com/matrix-org/matrix-js-sdk/pull/1001)
+ * Support for MSC2140 (terms of service for IS/IM)
+   [\#988](https://github.com/matrix-org/matrix-js-sdk/pull/988)
+ * Add a request method to /devices
+   [\#994](https://github.com/matrix-org/matrix-js-sdk/pull/994)
+
+Changes in [2.2.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.2.0) (2019-07-18)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.2.0-rc.2...v2.2.0)
+
+ * Upgrade lodash dependencies
+
+Changes in [2.2.0-rc.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.2.0-rc.2) (2019-07-12)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.2.0-rc.1...v2.2.0-rc.2)
+
+ * Fix regression from 2.2.0-rc.1 in request to /devices
+   [\#995](https://github.com/matrix-org/matrix-js-sdk/pull/995)
+
+Changes in [2.2.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.2.0-rc.1) (2019-07-12)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.1.1...v2.2.0-rc.1)
+
+ * End the verification timer when verification is done
+   [\#993](https://github.com/matrix-org/matrix-js-sdk/pull/993)
+ * Stabilize usage of stably stable APIs (in a stable way)
+   [\#990](https://github.com/matrix-org/matrix-js-sdk/pull/990)
+ * Expose original_event for /relations
+   [\#987](https://github.com/matrix-org/matrix-js-sdk/pull/987)
+ * Process ephemeral events outside timeline handling
+   [\#989](https://github.com/matrix-org/matrix-js-sdk/pull/989)
+ * Don't accept any locally known edits earlier than the last known server-side
+   aggregated edit
+   [\#986](https://github.com/matrix-org/matrix-js-sdk/pull/986)
+ * Get edit date transparently from server aggregations or local echo
+   [\#984](https://github.com/matrix-org/matrix-js-sdk/pull/984)
+ * Add a function to flag keys for backup without scheduling a backup
+   [\#982](https://github.com/matrix-org/matrix-js-sdk/pull/982)
+ * Block read marker and read receipt from advancing into pending events
+   [\#981](https://github.com/matrix-org/matrix-js-sdk/pull/981)
+ * Upgrade dependencies
+   [\#977](https://github.com/matrix-org/matrix-js-sdk/pull/977)
+ * Add default push rule to ignore reactions
+   [\#976](https://github.com/matrix-org/matrix-js-sdk/pull/976)
+ * Fix exception whilst syncing
+   [\#979](https://github.com/matrix-org/matrix-js-sdk/pull/979)
+ * Include the error object when raising Session.logged_out
+   [\#975](https://github.com/matrix-org/matrix-js-sdk/pull/975)
+
+Changes in [2.1.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.1.1) (2019-07-11)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.1.0...v2.1.1)
+
+ * Process emphemeral events outside timeline handling
+   [\#989](https://github.com/matrix-org/matrix-js-sdk/pull/989)
+
+Changes in [2.1.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.1.0) (2019-07-08)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.1.0-rc.1...v2.1.0)
+
+ * Fix exception whilst syncing
+   [\#979](https://github.com/matrix-org/matrix-js-sdk/pull/979)
+
+Changes in [2.1.0-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.1.0-rc.1) (2019-07-03)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.0.1...v2.1.0-rc.1)
+
+ * Handle self read receipts for fixing e2e notification counts
+   [\#974](https://github.com/matrix-org/matrix-js-sdk/pull/974)
+ * Add redacts field to event.toJSON
+   [\#973](https://github.com/matrix-org/matrix-js-sdk/pull/973)
+ * Handle associated event send failures
+   [\#972](https://github.com/matrix-org/matrix-js-sdk/pull/972)
+ * Remove irrelevant debug line from timeline handling
+   [\#971](https://github.com/matrix-org/matrix-js-sdk/pull/971)
+ * Handle relations in encrypted rooms
+   [\#969](https://github.com/matrix-org/matrix-js-sdk/pull/969)
+ * Relations endpoint support
+   [\#967](https://github.com/matrix-org/matrix-js-sdk/pull/967)
+ * Disable event encryption for reactions
+   [\#968](https://github.com/matrix-org/matrix-js-sdk/pull/968)
+ * Change the known safe room version to version 4
+   [\#966](https://github.com/matrix-org/matrix-js-sdk/pull/966)
+ * Check for lazy-loading support in the spec versions instead
+   [\#965](https://github.com/matrix-org/matrix-js-sdk/pull/965)
+ * Use camelCase instead of underscore
+   [\#963](https://github.com/matrix-org/matrix-js-sdk/pull/963)
+ * Time out verification attempts after 10 minutes of inactivity
+   [\#961](https://github.com/matrix-org/matrix-js-sdk/pull/961)
+ * Don't handle key verification requests which are immediately cancelled
+   [\#962](https://github.com/matrix-org/matrix-js-sdk/pull/962)
+
+Changes in [2.0.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.0.1) (2019-06-19)
+================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.0.1-rc.2...v2.0.1)
+
+ No changes since rc.2
+
+Changes in [2.0.1-rc.2](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.0.1-rc.2) (2019-06-18)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.0.1-rc.1...v2.0.1-rc.2)
+
+ * return 'sending' status for an event that is only locally redacted
+   [\#960](https://github.com/matrix-org/matrix-js-sdk/pull/960)
+ * Key verification request fixes
+   [\#954](https://github.com/matrix-org/matrix-js-sdk/pull/954)
+ * Add flag to force saving sync store
+   [\#956](https://github.com/matrix-org/matrix-js-sdk/pull/956)
+ * Expose the inhibit_login flag to register
+   [\#953](https://github.com/matrix-org/matrix-js-sdk/pull/953)
+ *  Support redactions and relations of/with unsent events.
+   [\#947](https://github.com/matrix-org/matrix-js-sdk/pull/947)
+
+Changes in [2.0.1-rc.1](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.0.1-rc.1) (2019-06-12)
+==========================================================================================================
+[Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v2.0.0...v2.0.1-rc.1)
+
+ * Fix content uploads for modern browsers
+   [\#952](https://github.com/matrix-org/matrix-js-sdk/pull/952)
+ * Don't overlap auth submissions with polls
+   [\#951](https://github.com/matrix-org/matrix-js-sdk/pull/951)
+ * Add funding details for GitHub sponsor button
+   [\#945](https://github.com/matrix-org/matrix-js-sdk/pull/945)
+ * Fix backup sig validation with multiple sigs
+   [\#944](https://github.com/matrix-org/matrix-js-sdk/pull/944)
+ * Don't send another token request while one's in flight
+   [\#943](https://github.com/matrix-org/matrix-js-sdk/pull/943)
+ * Don't poll UI auth again until current poll finishes
+   [\#942](https://github.com/matrix-org/matrix-js-sdk/pull/942)
+ * Provide the discovered URLs when a liveliness error occurs
+   [\#938](https://github.com/matrix-org/matrix-js-sdk/pull/938)
+ * Encode event IDs when redacting events
+   [\#941](https://github.com/matrix-org/matrix-js-sdk/pull/941)
+ * add missing logger
+   [\#940](https://github.com/matrix-org/matrix-js-sdk/pull/940)
+ * verification: don't error if we don't know about some keys
+   [\#939](https://github.com/matrix-org/matrix-js-sdk/pull/939)
+ * Local echo for redactions
+   [\#937](https://github.com/matrix-org/matrix-js-sdk/pull/937)
+ * Refresh safe room versions when the server looks more modern than us
+   [\#934](https://github.com/matrix-org/matrix-js-sdk/pull/934)
+ * Add v4 as a safe room version
+   [\#935](https://github.com/matrix-org/matrix-js-sdk/pull/935)
+ * Disable guard-for-in rule
+   [\#933](https://github.com/matrix-org/matrix-js-sdk/pull/933)
+ * Extend loglevel logging for the whole project
+   [\#924](https://github.com/matrix-org/matrix-js-sdk/pull/924)
+ * fix(login): saves access_token and user_id after login for all login types
+   [\#930](https://github.com/matrix-org/matrix-js-sdk/pull/930)
+ * Do not try to request thumbnails with non-integer sizes
+   [\#929](https://github.com/matrix-org/matrix-js-sdk/pull/929)
+ * Revert "Add a bunch of debugging to .well-known IS validation"
+   [\#928](https://github.com/matrix-org/matrix-js-sdk/pull/928)
+ * Add a bunch of debugging to .well-known IS validation
+   [\#927](https://github.com/matrix-org/matrix-js-sdk/pull/927)
+
 Changes in [2.0.0](https://github.com/matrix-org/matrix-js-sdk/releases/tag/v2.0.0) (2019-05-31)
 ================================================================================================
 [Full Changelog](https://github.com/matrix-org/matrix-js-sdk/compare/v1.2.0...v2.0.0)
@@ -1,8 +1,7 @@
 Matrix Javascript SDK
 =====================
-[![Build Status](http://matrix.org/jenkins/buildStatus/icon?job=JavascriptSDK)](http://matrix.org/jenkins/job/JavascriptSDK/)

-This is the [Matrix](https://matrix.org) Client-Server v1/v2 alpha SDK for
+This is the [Matrix](https://matrix.org) Client-Server r0 SDK for
 JavaScript. This SDK can be run in a browser or in Node.js.

 Quickstart
@@ -297,7 +296,7 @@ Then visit ``http://localhost:8005`` to see the API docs.
 End-to-end encryption support
 =============================

-The SDK supports end-to-end encryption via the and Megolm protocols, using
+The SDK supports end-to-end encryption via the Olm and Megolm protocols, using
 [libolm](https://gitlab.matrix.org/matrix-org/olm). It is left up to the 
 application to make libolm available, via the ``Olm`` global.

@@ -323,13 +322,13 @@ To provide the Olm library in a browser application:
 
 To provide the Olm library in a node.js application:

- * ``yarn add https://packages.matrix.org/npm/olm/olm-3.0.0.tgz``
+ * ``yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz``
   (replace the URL with the latest version you want to use from
    https://packages.matrix.org/npm/olm/)
 * ``global.Olm = require('olm');`` *before* loading ``matrix-js-sdk``.

 If you want to package Olm as dependency for your node.js application, you can
-use ``yarn add https://packages.matrix.org/npm/olm/olm-3.0.0.tgz``. If your
+use ``yarn add https://packages.matrix.org/npm/olm/olm-3.1.4.tgz``. If your
 application also works without e2e crypto enabled, add ``--optional`` to mark it
 as an optional dependency.

@@ -1,6 +1,6 @@
 {
  "name": "matrix-js-sdk",
-  "version": "2.0.0",
+  "version": "2.4.3",
  "description": "Matrix Client-Server SDK for Javascript",
  "main": "index.js",
  "scripts": {
@@ -17,7 +17,7 @@
    "build": "babel -s -d lib src && rimraf dist && mkdir dist && browserify -d browser-index.js | exorcist dist/browser-matrix.js.map > dist/browser-matrix.js && terser -c -m -o dist/browser-matrix.min.js --source-map \"content='dist/browser-matrix.js.map'\" dist/browser-matrix.js",
    "dist": "yarn build",
    "watch": "watchify -d browser-index.js -o 'exorcist dist/browser-matrix.js.map > dist/browser-matrix.js' -v",
-    "lint": "eslint --max-warnings 101 src spec",
+    "lint": "eslint --max-warnings 93 src spec",
    "prepare": "yarn clean && yarn build && git rev-parse HEAD > git-revision.txt"
  },
  "repository": {
@@ -54,11 +54,11 @@
  "dependencies": {
    "another-json": "^0.2.0",
    "babel-runtime": "^6.26.0",
-    "bluebird": "^3.5.0",
+    "bluebird": "3.5.5",
    "browser-request": "^0.3.3",
    "bs58": "^4.0.1",
    "content-type": "^1.0.2",
-    "loglevel": "1.6.1",
+    "loglevel": "^1.6.4",
    "qs": "^6.5.2",
    "request": "^2.88.0",
    "unhomoglyph": "^1.0.2"
@@ -75,19 +75,19 @@
    "eslint": "^5.12.0",
    "eslint-config-google": "^0.7.1",
    "eslint-plugin-babel": "^5.3.0",
-    "exorcist": "^0.4.0",
+    "exorcist": "^1.0.1",
    "expect": "^1.20.2",
    "istanbul": "^0.4.5",
    "jsdoc": "^3.5.5",
    "lolex": "^1.5.2",
    "matrix-mock-request": "^1.2.3",
-    "mocha": "^5.2.0",
+    "mocha": "^6.2.1",
    "mocha-jenkins-reporter": "^0.4.0",
-    "olm": "https://packages.matrix.org/npm/olm/olm-3.1.0.tgz",
-    "rimraf": "^2.5.4",
-    "source-map-support": "^0.4.11",
-    "sourceify": "^0.1.0",
-    "terser": "^4.0.0",
+    "olm": "https://packages.matrix.org/npm/olm/olm-3.1.4.tgz",
+    "rimraf": "^3.0.0",
+    "source-map-support": "^0.5.13",
+    "sourceify": "^1.0.0",
+    "terser": "^4.3.8",
    "watchify": "^3.11.1"
  },
  "browserify": {
@@ -195,6 +195,11 @@ if [ $dodist -eq 0 ]; then
    pushd "$builddir"
    git clone "$projdir" .
    git checkout "$rel_branch"
+    # We use Git branch / commit dependencies for some packages, and Yarn seems
+    # to have a hard time getting that right. See also
+    # https://github.com/yarnpkg/yarn/issues/4734. As a workaround, we clean the
+    # global cache here to ensure we get the right thing.
+    yarn cache clean
    yarn install
    # We haven't tagged yet, so tell the dist script what version
    # it's building
@@ -27,6 +27,7 @@ import MockHttpBackend from 'matrix-mock-request';
 import expect from 'expect';
 import Promise from 'bluebird';
 import LocalStorageCryptoStore from '../lib/crypto/store/localStorage-crypto-store';
+import logger from '../src/logger';

 /**
 * Wrapper for a MockStorageApi, MockHttpBackend and MatrixClient
@@ -82,7 +83,7 @@ TestClient.prototype.toString = function() {
 * @return {Promise}
 */
 TestClient.prototype.start = function() {
-    console.log(this + ': starting');
+    logger.log(this + ': starting');
    this.httpBackend.when("GET", "/pushrules").respond(200, {});
    this.httpBackend.when("POST", "/filter").respond(200, { filter_id: "fid" });
    this.expectDeviceKeyUpload();
@@ -100,7 +101,7 @@ TestClient.prototype.start = function() {
        this.httpBackend.flushAllExpected(),
        testUtils.syncPromise(this.client),
    ]).then(() => {
-        console.log(this + ': started');
+        logger.log(this + ': started');
    });
 };

@@ -122,7 +123,7 @@ TestClient.prototype.expectDeviceKeyUpload = function() {
        expect(content.one_time_keys).toBe(undefined);
        expect(content.device_keys).toBeTruthy();

-        console.log(self + ': received device keys');
+        logger.log(self + ': received device keys');
        // we expect this to happen before any one-time keys are uploaded.
        expect(Object.keys(self.oneTimeKeys).length).toEqual(0);

@@ -159,7 +160,7 @@ TestClient.prototype.awaitOneTimeKeyUpload = function() {
              expect(content.device_keys).toBe(undefined);
              expect(content.one_time_keys).toBeTruthy();
              expect(content.one_time_keys).toNotEqual({});
-              console.log('%s: received %i one-time keys', this,
+              logger.log('%s: received %i one-time keys', this,
                          Object.keys(content.one_time_keys).length);
              this.oneTimeKeys = content.one_time_keys;
              return {one_time_key_counts: {
@@ -223,11 +224,11 @@ TestClient.prototype.getSigningKey = function() {
 * @returns {Promise} promise which completes once the sync has been flushed
 */
 TestClient.prototype.flushSync = function() {
-    console.log(`${this}: flushSync`);
+    logger.log(`${this}: flushSync`);
    return Promise.all([
        this.httpBackend.flush('/sync', 1),
        testUtils.syncPromise(this.client),
    ]).then(() => {
-        console.log(`${this}: flushSync completed`);
+        logger.log(`${this}: flushSync completed`);
    });
 };
@@ -20,6 +20,7 @@ import Promise from 'bluebird';

 import TestClient from '../TestClient';
 import testUtils from '../test-utils';
+import logger from '../../src/logger';

 const ROOM_ID = "!room:id";

@@ -71,7 +72,7 @@ function getSyncResponse(roomMembers) {

 describe("DeviceList management:", function() {
    if (!global.Olm) {
-        console.warn('not running deviceList tests: Olm not present');
+        logger.warn('not running deviceList tests: Olm not present');
        return;
    }

@@ -108,7 +109,7 @@ describe("DeviceList management:", function() {

            return aliceTestClient.flushSync();
        }).then(function() {
-            console.log("Forcing alice to download our device keys");
+            logger.log("Forcing alice to download our device keys");

            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(200, {
                device_keys: {
@@ -121,7 +122,7 @@ describe("DeviceList management:", function() {
                aliceTestClient.httpBackend.flush('/keys/query', 1),
            ]);
        }).then(function() {
-            console.log("Telling alice to send a megolm message");
+            logger.log("Telling alice to send a megolm message");

            aliceTestClient.httpBackend.when(
                'PUT', '/send/',
@@ -36,6 +36,7 @@ import Promise from 'bluebird';
 const utils = require("../../lib/utils");
 const testUtils = require("../test-utils");
 const TestClient = require('../TestClient').default;
+import logger from '../../src/logger';

 let aliTestClient;
 const roomId = "!room:localhost";
@@ -95,7 +96,7 @@ function expectBobQueryKeys() {

    const aliKeys = {};
    aliKeys[aliDeviceId] = aliTestClient.deviceKeys;
-    console.log("query result will be", aliKeys);
+    logger.log("query result will be", aliKeys);

    bobTestClient.httpBackend.when(
        "POST", "/keys/query",
@@ -334,7 +335,7 @@ function recvMessage(httpBackend, client, sender, message) {
            if (event.getType() == "m.room.member") {
                return;
            }
-            console.log(client.credentials.userId + " received event",
+            logger.log(client.credentials.userId + " received event",
                        event);

            client.removeListener("event", onEvent);
@@ -607,7 +608,7 @@ describe("MatrixClient crypto", function() {

                const eventPromise = new Promise((resolve, reject) => {
                    const onEvent = function(event) {
-                        console.log(bobUserId + " received event",
+                        logger.log(bobUserId + " received event",
                                    event);
                        resolve(event);
                    };
@@ -734,7 +735,7 @@ describe("MatrixClient crypto", function() {

        return Promise.resolve()
            .then(() => {
-                console.log(aliTestClient + ': starting');
+                logger.log(aliTestClient + ': starting');
                httpBackend.when("GET", "/pushrules").respond(200, {});
                httpBackend.when("POST", "/filter").respond(200, { filter_id: "fid" });
                aliTestClient.expectDeviceKeyUpload();
@@ -746,7 +747,7 @@ describe("MatrixClient crypto", function() {
                aliTestClient.client.startClient({});

                return httpBackend.flushAllExpected().then(() => {
-                    console.log(aliTestClient + ': started');
+                    logger.log(aliTestClient + ': started');
                });
            })
            .then(() => httpBackend.when("POST", "/keys/upload")
@@ -755,7 +756,7 @@ describe("MatrixClient crypto", function() {
                    expect(content.one_time_keys).toNotEqual({});
                    expect(Object.keys(content.one_time_keys).length)
                        .toBeGreaterThanOrEqualTo(1);
-                    console.log('received %i one-time keys',
+                    logger.log('received %i one-time keys',
                                Object.keys(content.one_time_keys).length);
                    // cancel futher calls by telling the client
                    // we have more than we need
@@ -302,11 +302,32 @@ describe("MatrixClient events", function() {
        });

        it("should emit Session.logged_out on M_UNKNOWN_TOKEN", function() {
-            httpBackend.when("GET", "/sync").respond(401, { errcode: 'M_UNKNOWN_TOKEN' });
+            const error = { errcode: 'M_UNKNOWN_TOKEN' };
+            httpBackend.when("GET", "/sync").respond(401, error);

            let sessionLoggedOutCount = 0;
-            client.on("Session.logged_out", function(event, member) {
+            client.on("Session.logged_out", function(errObj) {
                sessionLoggedOutCount++;
+                expect(errObj.data).toEqual(error);
+            });
+
+            client.startClient();
+
+            return httpBackend.flushAllExpected().then(function() {
+                expect(sessionLoggedOutCount).toEqual(
+                    1, "Session.logged_out fired wrong number of times",
+                );
+            });
+        });
+
+        it("should emit Session.logged_out on M_UNKNOWN_TOKEN (soft logout)", function() {
+            const error = { errcode: 'M_UNKNOWN_TOKEN', soft_logout: true };
+            httpBackend.when("GET", "/sync").respond(401, error);
+
+            let sessionLoggedOutCount = 0;
+            client.on("Session.logged_out", function(errObj) {
+                sessionLoggedOutCount++;
+                expect(errObj.data).toEqual(error);
            });

            client.startClient();
@@ -5,6 +5,7 @@ const sdk = require("../..");
 const HttpBackend = require("matrix-mock-request");
 const utils = require("../test-utils");
 const EventTimeline = sdk.EventTimeline;
+import logger from '../../src/logger';

 const baseUrl = "http://localhost.or.something";
 const userId = "@alice:localhost";
@@ -84,7 +85,7 @@ function startClient(httpBackend, client) {
    // set up a promise which will resolve once the client is initialised
    const deferred = Promise.defer();
    client.on("sync", function(state) {
-        console.log("sync", state);
+        logger.log("sync", state);
        if (state != "SYNCING") {
            return;
        }
@@ -669,11 +670,11 @@ describe("MatrixClient event timelines", function() {
                // initiate the send, and set up checks to be done when it completes
                // - but note that it won't complete until after the /sync does, below.
                client.sendTextMessage(roomId, "a body", TXN_ID).then(function(res) {
-                    console.log("sendTextMessage completed");
+                    logger.log("sendTextMessage completed");
                    expect(res.event_id).toEqual(event.event_id);
                    return client.getEventTimeline(timelineSet, event.event_id);
                }).then(function(tl) {
-                    console.log("getEventTimeline completed (2)");
+                    logger.log("getEventTimeline completed (2)");
                    expect(tl.getEvents().length).toEqual(2);
                    expect(tl.getEvents()[1].getContent().body).toEqual("a body");
                }),
@@ -684,7 +685,7 @@ describe("MatrixClient event timelines", function() {
                ]).then(function() {
                    return client.getEventTimeline(timelineSet, event.event_id);
                }).then(function(tl) {
-                    console.log("getEventTimeline completed (1)");
+                    logger.log("getEventTimeline completed (1)");
                    expect(tl.getEvents().length).toEqual(2);
                    expect(tl.getEvents()[1].event).toEqual(event);

@@ -48,7 +48,7 @@ describe("MatrixClient", function() {
        const buf = new Buffer('hello world');
        it("should upload the file", function(done) {
            httpBackend.when(
-                "POST", "/_matrix/media/v1/upload",
+                "POST", "/_matrix/media/r0/upload",
            ).check(function(req) {
                expect(req.rawData).toEqual(buf);
                expect(req.queryParams.filename).toEqual("hi.txt");
@@ -87,7 +87,7 @@ describe("MatrixClient", function() {

        it("should parse the response if rawResponse=false", function(done) {
            httpBackend.when(
-                "POST", "/_matrix/media/v1/upload",
+                "POST", "/_matrix/media/r0/upload",
            ).check(function(req) {
                expect(req.opts.json).toBeFalsy();
            }).respond(200, { "content_uri": "uri" });
@@ -107,7 +107,7 @@ describe("MatrixClient", function() {

        it("should parse errors into a MatrixError", function(done) {
            httpBackend.when(
-                "POST", "/_matrix/media/v1/upload",
+                "POST", "/_matrix/media/r0/upload",
            ).check(function(req) {
                expect(req.rawData).toEqual(buf);
                expect(req.opts.json).toBeFalsy();
@@ -355,9 +355,9 @@ describe("MatrixClient", function() {
                return client._crypto._olmDevice.sign(anotherjson.stringify(b));
            };

-            console.log("Ed25519: " + ed25519key);
-            console.log("boris:", sign(borisKeys.dev1));
-            console.log("chaz:", sign(chazKeys.dev2));
+            logger.log("Ed25519: " + ed25519key);
+            logger.log("boris:", sign(borisKeys.dev1));
+            logger.log("chaz:", sign(chazKeys.dev2));
            */

            httpBackend.when("POST", "/keys/query").check(function(req) {
@@ -396,7 +396,7 @@ describe("MatrixClient", function() {
        const auth = {a: 1};
        it("should pass through an auth dict", function(done) {
            httpBackend.when(
-                "DELETE", "/_matrix/client/unstable/devices/my_device",
+                "DELETE", "/_matrix/client/r0/devices/my_device",
            ).check(function(req) {
                expect(req.data).toEqual({auth: auth});
            }).respond(200);
@@ -23,6 +23,7 @@ import expect from 'expect';
 const utils = require('../../lib/utils');
 const testUtils = require('../test-utils');
 const TestClient = require('../TestClient').default;
+import logger from '../../src/logger';

 const ROOM_ID = "!room:id";

@@ -203,7 +204,7 @@ function getSyncResponse(roomMembers) {

 describe("megolm", function() {
    if (!global.Olm) {
-        console.warn('not running megolm tests: Olm not present');
+        logger.warn('not running megolm tests: Olm not present');
        return;
    }
    const Olm = global.Olm;
@@ -416,7 +417,7 @@ describe("megolm", function() {

            return new Promise((resolve, reject) => {
                event.once('Event.decrypted', (ev) => {
-                    console.log(`${Date.now()} event ${event.getId()} now decrypted`);
+                    logger.log(`${Date.now()} event ${event.getId()} now decrypted`);
                    resolve(ev);
                });
            });
@@ -555,7 +556,7 @@ describe("megolm", function() {
            ).respond(200, function(path, content) {
                const ct = content.ciphertext;
                const r = inboundGroupSession.decrypt(ct);
-                console.log('Decrypted received megolm message', r);
+                logger.log('Decrypted received megolm message', r);

                expect(r.message_index).toEqual(0);
                const decrypted = JSON.parse(r.plaintext);
@@ -600,7 +601,7 @@ describe("megolm", function() {

            return aliceTestClient.flushSync();
        }).then(function() {
-            console.log('Forcing alice to download our device keys');
+            logger.log('Forcing alice to download our device keys');

            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(
                200, getTestKeysQueryResponse('@bob:xyz'),
@@ -611,10 +612,10 @@ describe("megolm", function() {
                aliceTestClient.httpBackend.flush('/keys/query', 1),
            ]);
        }).then(function() {
-            console.log('Telling alice to block our device');
+            logger.log('Telling alice to block our device');
            aliceTestClient.client.setDeviceBlocked('@bob:xyz', 'DEVICE_ID');

-            console.log('Telling alice to send a megolm message');
+            logger.log('Telling alice to send a megolm message');
            aliceTestClient.httpBackend.when(
                'PUT', '/send/',
            ).respond(200, {
@@ -656,7 +657,7 @@ describe("megolm", function() {

            return aliceTestClient.flushSync();
        }).then(function() {
-            console.log("Fetching bob's devices and marking known");
+            logger.log("Fetching bob's devices and marking known");

            aliceTestClient.httpBackend.when('POST', '/keys/query').respond(
                200, getTestKeysQueryResponse('@bob:xyz'),
@@ -669,17 +670,17 @@ describe("megolm", function() {
                aliceTestClient.client.setDeviceKnown('@bob:xyz', 'DEVICE_ID');
            });
        }).then(function() {
-            console.log('Telling alice to send a megolm message');
+            logger.log('Telling alice to send a megolm message');

            aliceTestClient.httpBackend.when(
                'PUT', '/sendToDevice/m.room.encrypted/',
            ).respond(200, function(path, content) {
-                console.log('sendToDevice: ', content);
+                logger.log('sendToDevice: ', content);
                const m = content.messages['@bob:xyz'].DEVICE_ID;
                const ct = m.ciphertext[testSenderKey];
                expect(ct.type).toEqual(1); // normal message
                const decrypted = JSON.parse(p2pSession.decrypt(ct.type, ct.body));
-                console.log('decrypted sendToDevice:', decrypted);
+                logger.log('decrypted sendToDevice:', decrypted);
                expect(decrypted.type).toEqual('m.room_key');
                megolmSessionId = decrypted.content.session_id;
                return {};
@@ -688,7 +689,7 @@ describe("megolm", function() {
            aliceTestClient.httpBackend.when(
                'PUT', '/send/',
            ).respond(200, function(path, content) {
-                console.log('/send:', content);
+                logger.log('/send:', content);
                expect(content.session_id).toEqual(megolmSessionId);
                return {
                    event_id: '$event_id',
@@ -704,14 +705,14 @@ describe("megolm", function() {
                }),
            ]);
        }).then(function() {
-            console.log('Telling alice to block our device');
+            logger.log('Telling alice to block our device');
            aliceTestClient.client.setDeviceBlocked('@bob:xyz', 'DEVICE_ID');

-            console.log('Telling alice to send another megolm message');
+            logger.log('Telling alice to send another megolm message');
            aliceTestClient.httpBackend.when(
                'PUT', '/send/',
            ).respond(200, function(path, content) {
-                console.log('/send:', content);
+                logger.log('/send:', content);
                expect(content.session_id).toNotEqual(megolmSessionId);
                return {
                    event_id: '$event_id',
@@ -792,7 +793,7 @@ describe("megolm", function() {
            aliceTestClient.httpBackend.when(
                'PUT', '/sendToDevice/m.room.encrypted/',
            ).respond(200, function(path, content) {
-                console.log("sendToDevice: ", content);
+                logger.log("sendToDevice: ", content);
                const m = content.messages[aliceTestClient.userId].DEVICE_ID;
                const ct = m.ciphertext[testSenderKey];
                expect(ct.type).toEqual(0); // pre-key message
@@ -812,7 +813,7 @@ describe("megolm", function() {
            ).respond(200, function(path, content) {
                const ct = content.ciphertext;
                const r = inboundGroupSession.decrypt(ct);
-                console.log('Decrypted received megolm message', r);
+                logger.log('Decrypted received megolm message', r);
                decrypted = JSON.parse(r.plaintext);

                return {
@@ -865,7 +866,7 @@ describe("megolm", function() {
            return aliceTestClient.flushSync();
        }).then(function() {
            // this will block
-            console.log('Forcing alice to download our device keys');
+            logger.log('Forcing alice to download our device keys');
            downloadPromise = aliceTestClient.client.downloadKeys(['@bob:xyz']);

            // so will this.
@@ -14,11 +14,12 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */

-// try to load the olm library.
+import logger from '../src/logger';

+// try to load the olm library.
 try {
    global.Olm = require('olm');
-    console.log('loaded libolm');
+    logger.log('loaded libolm');
 } catch (e) {
-    console.warn("unable to run crypto tests: libolm not available");
+    logger.warn("unable to run crypto tests: libolm not available");
 }
@@ -5,6 +5,7 @@ import Promise from 'bluebird';
 // load olm before the sdk if possible
 import './olm-loader';

+import logger from '../src/logger';
 import sdk from '..';
 const MatrixEvent = sdk.MatrixEvent;

@@ -25,7 +26,7 @@ module.exports.syncPromise = function(client, count) {

    const p = new Promise((resolve, reject) => {
        const cb = (state) => {
-            console.log(`${Date.now()} syncPromise(${count}): ${state}`);
+            logger.log(`${Date.now()} syncPromise(${count}): ${state}`);
            if (state == 'SYNCING') {
                resolve();
            } else {
@@ -48,8 +49,8 @@ module.exports.syncPromise = function(client, count) {
 module.exports.beforeEach = function(context) {
    const desc = context.currentTest.fullTitle();

-    console.log(desc);
-    console.log(new Array(1 + desc.length).join("="));
+    logger.log(desc);
+    logger.log(new Array(1 + desc.length).join("="));
 };

 /**
@@ -232,11 +233,11 @@ module.exports.awaitDecryption = function(event) {
        return Promise.resolve(event);
    }

-    console.log(`${Date.now()} event ${event.getId()} is being decrypted; waiting`);
+    logger.log(`${Date.now()} event ${event.getId()} is being decrypted; waiting`);

    return new Promise((resolve, reject) => {
        event.once('Event.decrypted', (ev) => {
-            console.log(`${Date.now()} event ${event.getId()} now decrypted`);
+            logger.log(`${Date.now()} event ${event.getId()} now decrypted`);
            resolve(ev);
        });
    });
@@ -275,7 +275,7 @@ describe("AutoDiscovery", function() {
                    "m.homeserver": {
                        state: "FAIL_ERROR",
                        error: AutoDiscovery.ERROR_INVALID_HOMESERVER,
-                        base_url: null,
+                        base_url: "https://example.org",
                    },
                    "m.identity_server": {
                        state: "PROMPT",
@@ -304,7 +304,7 @@ describe("AutoDiscovery", function() {
                    "m.homeserver": {
                        state: "FAIL_ERROR",
                        error: AutoDiscovery.ERROR_INVALID_HOMESERVER,
-                        base_url: null,
+                        base_url: "https://example.org",
                    },
                    "m.identity_server": {
                        state: "PROMPT",
@@ -335,7 +335,7 @@ describe("AutoDiscovery", function() {
                    "m.homeserver": {
                        state: "FAIL_ERROR",
                        error: AutoDiscovery.ERROR_INVALID_HOMESERVER,
-                        base_url: null,
+                        base_url: "https://example.org",
                    },
                    "m.identity_server": {
                        state: "PROMPT",
@@ -528,7 +528,7 @@ describe("AutoDiscovery", function() {
                    "m.identity_server": {
                        state: "FAIL_ERROR",
                        error: AutoDiscovery.ERROR_INVALID_IDENTITY_SERVER,
-                        base_url: null,
+                        base_url: "https://identity.example.org",
                    },
                };

@@ -569,7 +569,7 @@ describe("AutoDiscovery", function() {
                    "m.identity_server": {
                        state: "FAIL_ERROR",
                        error: AutoDiscovery.ERROR_INVALID_IDENTITY_SERVER,
-                        base_url: null,
+                        base_url: "https://identity.example.org",
                    },
                };

@@ -31,7 +31,7 @@ describe("ContentRepo", function() {
        function() {
            const mxcUri = "mxc://server.name/resourceid";
            expect(ContentRepo.getHttpUriForMxc(baseUrl, mxcUri)).toEqual(
-                baseUrl + "/_matrix/media/v1/download/server.name/resourceid",
+                baseUrl + "/_matrix/media/r0/download/server.name/resourceid",
            );
        });

@@ -43,7 +43,7 @@ describe("ContentRepo", function() {
        function() {
            const mxcUri = "mxc://server.name/resourceid";
            expect(ContentRepo.getHttpUriForMxc(baseUrl, mxcUri, 32, 64, "crop")).toEqual(
-                baseUrl + "/_matrix/media/v1/thumbnail/server.name/resourceid" +
+                baseUrl + "/_matrix/media/r0/thumbnail/server.name/resourceid" +
                "?width=32&height=64&method=crop",
            );
        });
@@ -52,7 +52,7 @@ describe("ContentRepo", function() {
        function() {
            const mxcUri = "mxc://server.name/resourceid#automade";
            expect(ContentRepo.getHttpUriForMxc(baseUrl, mxcUri, 32)).toEqual(
-                baseUrl + "/_matrix/media/v1/thumbnail/server.name/resourceid" +
+                baseUrl + "/_matrix/media/r0/thumbnail/server.name/resourceid" +
                "?width=32#automade",
            );
        });
@@ -61,7 +61,7 @@ describe("ContentRepo", function() {
        function() {
            const mxcUri = "mxc://server.name/resourceid#automade";
            expect(ContentRepo.getHttpUriForMxc(baseUrl, mxcUri)).toEqual(
-                baseUrl + "/_matrix/media/v1/download/server.name/resourceid#automade",
+                baseUrl + "/_matrix/media/r0/download/server.name/resourceid#automade",
            );
        });
    });
@@ -73,21 +73,21 @@ describe("ContentRepo", function() {

        it("should set w/h by default to 96", function() {
            expect(ContentRepo.getIdenticonUri(baseUrl, "foobar")).toEqual(
-                baseUrl + "/_matrix/media/v1/identicon/foobar" +
+                baseUrl + "/_matrix/media/unstable/identicon/foobar" +
                "?width=96&height=96",
            );
        });

        it("should be able to set custom w/h", function() {
            expect(ContentRepo.getIdenticonUri(baseUrl, "foobar", 32, 64)).toEqual(
-                baseUrl + "/_matrix/media/v1/identicon/foobar" +
+                baseUrl + "/_matrix/media/unstable/identicon/foobar" +
                "?width=32&height=64",
            );
        });

        it("should URL encode the identicon string", function() {
            expect(ContentRepo.getIdenticonUri(baseUrl, "foo#bar", 32, 64)).toEqual(
-                baseUrl + "/_matrix/media/v1/identicon/foo%23bar" +
+                baseUrl + "/_matrix/media/unstable/identicon/foo%23bar" +
                "?width=32&height=64",
            );
        });
@@ -19,6 +19,7 @@ import DeviceList from '../../../lib/crypto/DeviceList';
 import MemoryCryptoStore from '../../../lib/crypto/store/memory-crypto-store.js';
 import testUtils from '../../test-utils';
 import utils from '../../../lib/utils';
+import logger from '../../../src/logger';

 import expect from 'expect';
 import Promise from 'bluebird';
@@ -134,7 +135,7 @@ describe('DeviceList', function() {
        }).then(() => {
            // uh-oh; user restarts before second request completes. The new instance
            // should know we never got a complete device list.
-            console.log("Creating new devicelist to simulate app reload");
+            logger.log("Creating new devicelist to simulate app reload");
            downloadSpy.reset();
            const dl2 = createTestDeviceList();
            const queryDefer3 = Promise.defer();
@@ -10,6 +10,7 @@ import MockStorageApi from '../../../MockStorageApi';
 import testUtils from '../../../test-utils';
 import OlmDevice from '../../../../lib/crypto/OlmDevice';
 import Crypto from '../../../../lib/crypto';
+import logger from '../../../../src/logger';

 const MatrixEvent = sdk.MatrixEvent;
 const MegolmDecryption = algorithms.DECRYPTION_CLASSES['m.megolm.v1.aes-sha2'];
@@ -21,7 +22,7 @@ const Olm = global.Olm;

 describe("MegolmDecryption", function() {
    if (!global.Olm) {
-        console.warn('Not running megolm unit tests: libolm not present');
+        logger.warn('Not running megolm unit tests: libolm not present');
        return;
    }

@@ -20,6 +20,7 @@ import expect from 'expect';
 import MemoryCryptoStore from '../../../../lib/crypto/store/memory-crypto-store.js';
 import MockStorageApi from '../../../MockStorageApi';
 import testUtils from '../../../test-utils';
+import logger from '../../../../src/logger';

 import OlmDevice from '../../../../lib/crypto/OlmDevice';
 import olmlib from '../../../../lib/crypto/olmlib';
@@ -45,7 +46,7 @@ async function setupSession(initiator, opponent) {

 describe("OlmDecryption", function() {
    if (!global.Olm) {
-        console.warn('Not running megolm unit tests: libolm not present');
+        logger.warn('Not running megolm unit tests: libolm not present');
        return;
    }

@@ -28,6 +28,7 @@ import testUtils from '../../test-utils';

 import OlmDevice from '../../../lib/crypto/OlmDevice';
 import Crypto from '../../../lib/crypto';
+import logger from '../../../src/logger';

 const Olm = global.Olm;

@@ -112,7 +113,7 @@ function makeTestClient(sessionStore, cryptoStore) {

 describe("MegolmBackup", function() {
    if (!global.Olm) {
-        console.warn('Not running megolm backup unit tests: libolm not present');
+        logger.warn('Not running megolm backup unit tests: libolm not present');
        return;
    }

@@ -13,15 +13,15 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 */
+import logger from '../../../../src/logger';

 try {
    global.Olm = require('olm');
 } catch (e) {
-    console.warn("unable to run device verification tests: libolm not available");
+    logger.warn("unable to run device verification tests: libolm not available");
 }

 import expect from 'expect';
-
 import DeviceInfo from '../../../../lib/crypto/deviceinfo';

 import {ShowQRCode, ScanQRCode} from '../../../../lib/crypto/verification/QRCode';
@@ -30,7 +30,7 @@ const Olm = global.Olm;

 describe("QR code verification", function() {
    if (!global.Olm) {
-        console.warn('Not running device verification tests: libolm not present');
+        logger.warn('Not running device verification tests: libolm not present');
        return;
    }

@@ -13,11 +13,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 */
+import logger from '../../../../src/logger';

 try {
    global.Olm = require('olm');
 } catch (e) {
-    console.warn("unable to run device verification tests: libolm not available");
+    logger.warn("unable to run device verification tests: libolm not available");
 }

 import expect from 'expect';
@@ -32,7 +33,7 @@ import {makeTestClients} from './util';

 describe("verification request", function() {
    if (!global.Olm) {
-        console.warn('Not running device verification unit tests: libolm not present');
+        logger.warn('Not running device verification unit tests: libolm not present');
        return;
    }

@@ -68,8 +69,14 @@ describe("verification request", function() {
        bob.on("crypto.verification.request", (request) => {
            const bobVerifier = request.beginKeyVerification(verificationMethods.SAS);
            bobVerifier.verify();
+
+            // XXX: Private function access (but it's a test, so we're okay)
+            bobVerifier._endTimer();
        });
        const aliceVerifier = await alice.requestVerification("@bob:example.com");
        expect(aliceVerifier).toBeAn(SAS);
+
+        // XXX: Private function access (but it's a test, so we're okay)
+        aliceVerifier._endTimer();
    });
 });
@@ -13,11 +13,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 */
+import logger from '../../../../src/logger';

 try {
    global.Olm = require('olm');
 } catch (e) {
-    console.warn("unable to run device verification tests: libolm not available");
+    logger.warn("unable to run device verification tests: libolm not available");
 }

 import expect from 'expect';
@@ -37,7 +38,7 @@ import {makeTestClients} from './util';

 describe("SAS verification", function() {
    if (!global.Olm) {
-        console.warn('Not running device verification unit tests: libolm not present');
+        logger.warn('Not running device verification unit tests: libolm not present');
        return;
    }

@@ -56,6 +57,9 @@ describe("SAS verification", function() {
        await sas.verify()
            .catch(spy);
        expect(spy).toHaveBeenCalled();
+
+        // Cancel the SAS for cleanup (we started a verification, so abort)
+        sas.cancel();
    });

    describe("verification", function() {
@@ -269,4 +273,127 @@ describe("SAS verification", function() {
        expect(bob.setDeviceVerified)
            .toNotHaveBeenCalled();
    });
+
+    describe("verification in DM", function() {
+        let alice;
+        let bob;
+        let aliceSasEvent;
+        let bobSasEvent;
+        let aliceVerifier;
+        let bobPromise;
+
+        beforeEach(async function() {
+            [alice, bob] = await makeTestClients(
+                [
+                    {userId: "@alice:example.com", deviceId: "Osborne2"},
+                    {userId: "@bob:example.com", deviceId: "Dynabook"},
+                ],
+                {
+                    verificationMethods: [verificationMethods.SAS],
+                },
+            );
+
+            alice.setDeviceVerified = expect.createSpy();
+            alice.getDeviceEd25519Key = () => {
+                return "alice+base64+ed25519+key";
+            };
+            alice.getStoredDevice = () => {
+                return DeviceInfo.fromStorage(
+                    {
+                        keys: {
+                            "ed25519:Dynabook": "bob+base64+ed25519+key",
+                        },
+                    },
+                    "Dynabook",
+                );
+            };
+            alice.downloadKeys = () => {
+                return Promise.resolve();
+            };
+
+            bob.setDeviceVerified = expect.createSpy();
+            bob.getStoredDevice = () => {
+                return DeviceInfo.fromStorage(
+                    {
+                        keys: {
+                            "ed25519:Osborne2": "alice+base64+ed25519+key",
+                        },
+                    },
+                    "Osborne2",
+                );
+            };
+            bob.getDeviceEd25519Key = () => {
+                return "bob+base64+ed25519+key";
+            };
+            bob.downloadKeys = () => {
+                return Promise.resolve();
+            };
+
+            aliceSasEvent = null;
+            bobSasEvent = null;
+
+            bobPromise = new Promise((resolve, reject) => {
+                bob.on("event", async (event) => {
+                    const content = event.getContent();
+                    if (event.getType() === "m.room.message"
+                        && content.msgtype === "m.key.verification.request") {
+                        expect(content.methods).toInclude(SAS.NAME);
+                        expect(content.to).toBe(bob.getUserId());
+                        const verifier = bob.acceptVerificationDM(event, SAS.NAME);
+                        verifier.on("show_sas", (e) => {
+                            if (!e.sas.emoji || !e.sas.decimal) {
+                                e.cancel();
+                            } else if (!aliceSasEvent) {
+                                bobSasEvent = e;
+                            } else {
+                                try {
+                                    expect(e.sas).toEqual(aliceSasEvent.sas);
+                                    e.confirm();
+                                    aliceSasEvent.confirm();
+                                } catch (error) {
+                                    e.mismatch();
+                                    aliceSasEvent.mismatch();
+                                }
+                            }
+                        });
+                        await verifier.verify();
+                        resolve();
+                    }
+                });
+            });
+
+            aliceVerifier = await alice.requestVerificationDM(
+                bob.getUserId(), "!room_id", [verificationMethods.SAS],
+            );
+            aliceVerifier.on("show_sas", (e) => {
+                if (!e.sas.emoji || !e.sas.decimal) {
+                    e.cancel();
+                } else if (!bobSasEvent) {
+                    aliceSasEvent = e;
+                } else {
+                    try {
+                        expect(e.sas).toEqual(bobSasEvent.sas);
+                        e.confirm();
+                        bobSasEvent.confirm();
+                    } catch (error) {
+                        e.mismatch();
+                        bobSasEvent.mismatch();
+                    }
+                }
+            });
+        });
+
+        it("should verify a key", async function() {
+            await Promise.all([
+                aliceVerifier.verify(),
+                bobPromise,
+            ]);
+
+            // make sure Alice and Bob verified each other
+            expect(alice.setDeviceVerified)
+                .toHaveBeenCalledWith(bob.getUserId(), bob.deviceId);
+            expect(bob.setDeviceVerified)
+                .toHaveBeenCalledWith(alice.getUserId(), alice.deviceId);
+        });
+    });
 });
@@ -43,6 +43,25 @@ export async function makeTestClients(userInfos, options) {
            }
        }
    };
+    const sendEvent = function(room, type, content) {
+        // make up a unique ID as the event ID
+        const eventId = "$" + this.makeTxnId(); // eslint-disable-line babel/no-invalid-this
+        const event = new MatrixEvent({
+            sender: this.getUserId(), // eslint-disable-line babel/no-invalid-this
+            type: type,
+            content: content,
+            room_id: room,
+            event_id: eventId,
+        });
+        for (const client of clients) {
+            setTimeout(
+                () => client.emit("event", event),
+                0,
+            );
+        }
+
+        return {event_id: eventId};
+    };

    for (const userInfo of userInfos) {
        const client = (new TestClient(
@@ -54,6 +73,7 @@ export async function makeTestClients(userInfos, options) {
        }
        clientMap[userInfo.userId][userInfo.deviceId] = client;
        client.sendToDevice = sendToDevice;
+        client.sendEvent = sendEvent;
        clients.push(client);
    }

@@ -21,6 +21,7 @@ import testUtils from '../test-utils';

 import expect from 'expect';
 import Promise from 'bluebird';
+import logger from '../../src/logger';

 describe("MatrixEvent", () => {
    beforeEach(function() {
@@ -48,7 +49,7 @@ describe("MatrixEvent", () => {
            const crypto = {
                decryptEvent: function() {
                    ++callCount;
-                    console.log(`decrypt: ${callCount}`);
+                    logger.log(`decrypt: ${callCount}`);
                    if (callCount == 1) {
                        // schedule a second decryption attempt while
                        // the first one is still running.
@@ -24,6 +24,7 @@ const InteractiveAuth = sdk.InteractiveAuth;
 const MatrixError = sdk.MatrixError;

 import expect from 'expect';
+import logger from '../../src/logger';

 // Trivial client object to test interactive auth
 // (we do not need TestClient here)
@@ -64,7 +65,7 @@ describe("InteractiveAuth", function() {

        // first we expect a call here
        stateUpdated.andCall(function(stage) {
-            console.log('aaaa');
+            logger.log('aaaa');
            expect(stage).toEqual("logintype");
            ia.submitAuthDict({
                type: "logintype",
@@ -75,7 +76,7 @@ describe("InteractiveAuth", function() {
        // .. which should trigger a call here
        const requestRes = {"a": "b"};
        doRequest.andCall(function(authData) {
-            console.log('cccc');
+            logger.log('cccc');
            expect(authData).toEqual({
                session: "sessionId",
                type: "logintype",
@@ -106,7 +107,7 @@ describe("InteractiveAuth", function() {

        // first we expect a call to doRequest
        doRequest.andCall(function(authData) {
-            console.log("request1", authData);
+            logger.log("request1", authData);
            expect(authData).toEqual({});
            const err = new MatrixError({
                session: "sessionId",
@@ -132,7 +133,7 @@ describe("InteractiveAuth", function() {

            // submitAuthDict should trigger another call to doRequest
            doRequest.andCall(function(authData) {
-                console.log("request2", authData);
+                logger.log("request2", authData);
                expect(authData).toEqual({
                    session: "sessionId",
                    type: "logintype",
@@ -7,6 +7,7 @@ const utils = require("../test-utils");

 import expect from 'expect';
 import lolex from 'lolex';
+import logger from '../../src/logger';

 describe("MatrixClient", function() {
    const userId = "@alice:bar";
@@ -69,7 +70,7 @@ describe("MatrixClient", function() {
            "MatrixClient[UT] RECV " + method + " " + path + "  " +
            "EXPECT " + (next ? next.method : next) + " " + (next ? next.path : next)
        );
-        console.log(logLine);
+        logger.log(logLine);

        if (!next) { // no more things to return
            if (pendingLookup) {
@@ -91,7 +92,7 @@ describe("MatrixClient", function() {
            return pendingLookup.promise;
        }
        if (next.path === path && next.method === method) {
-            console.log(
+            logger.log(
                "MatrixClient[UT] Matched. Returning " +
                (next.error ? "BAD" : "GOOD") + " response",
            );
@@ -153,12 +154,9 @@ describe("MatrixClient", function() {
        });
        // FIXME: We shouldn't be yanking _http like this.
        client._http = [
-            "authedRequest", "authedRequestWithPrefix", "getContentUri",
-            "request", "requestWithPrefix", "uploadContent",
+            "authedRequest", "getContentUri", "request", "uploadContent",
        ].reduce((r, k) => { r[k] = expect.createSpy(); return r; }, {});
        client._http.authedRequest.andCall(httpReq);
-        client._http.authedRequestWithPrefix.andCall(httpReq);
-        client._http.requestWithPrefix.andCall(httpReq);
        client._http.request.andCall(httpReq);

        // set reasonable working defaults
@@ -180,9 +178,6 @@ describe("MatrixClient", function() {
        client._http.authedRequest.andCall(function() {
            return Promise.defer().promise;
        });
-        client._http.authedRequestWithPrefix.andCall(function() {
-            return Promise.defer().promise;
-        });
    });

    it("should not POST /filter if a matching filter already exists", async function() {
@@ -353,7 +348,7 @@ describe("MatrixClient", function() {
        function syncChecker(expectedStates, done) {
            return function syncListener(state, old) {
                const expected = expectedStates.shift();
-                console.log(
+                logger.log(
                    "'sync' curr=%s old=%s EXPECT=%s", state, old, expected,
                );
                if (!expected) {
@@ -104,7 +104,7 @@ describe("Room", function() {
                    user_ids: [userA],
                },
            });
-            room.addLiveEvents([typing]);
+            room.addEphemeralEvents([typing]);
            expect(room.currentState.setTypingEvent).toHaveBeenCalledWith(typing);
        });

@@ -48,7 +48,7 @@ describe("MatrixScheduler", function() {
        clock.uninstall();
    });

-    it("should process events in a queue in a FIFO manner", function(done) {
+    it("should process events in a queue in a FIFO manner", async function() {
        retryFn = function() {
            return 0;
        };
@@ -57,28 +57,30 @@ describe("MatrixScheduler", function() {
        };
        const deferA = Promise.defer();
        const deferB = Promise.defer();
-        let resolvedA = false;
+        let yieldedA = false;
        scheduler.setProcessFunction(function(event) {
-            if (resolvedA) {
+            if (yieldedA) {
                expect(event).toEqual(eventB);
                return deferB.promise;
            } else {
+                yieldedA = true;
                expect(event).toEqual(eventA);
                return deferA.promise;
            }
        });
-        scheduler.queueEvent(eventA);
-        scheduler.queueEvent(eventB).done(function() {
-            expect(resolvedA).toBe(true);
-            done();
-        });
-        deferA.resolve({});
-        resolvedA = true;
-        deferB.resolve({});
+        const abPromise = Promise.all([
+            scheduler.queueEvent(eventA),
+            scheduler.queueEvent(eventB),
+        ]);
+        deferB.resolve({b: true});
+        deferA.resolve({a: true});
+        const [a, b] = await abPromise;
+        expect(a.a).toEqual(true);
+        expect(b.b).toEqual(true);
    });

    it("should invoke the retryFn on failure and wait the amount of time specified",
-    function(done) {
+    async function() {
        const waitTimeMs = 1500;
        const retryDefer = Promise.defer();
        retryFn = function() {
@@ -97,24 +99,26 @@ describe("MatrixScheduler", function() {
                return defer.promise;
            } else if (procCount === 2) {
                // don't care about this defer
-                return Promise.defer().promise;
+                return new Promise();
            }
            expect(procCount).toBeLessThan(3);
        });

        scheduler.queueEvent(eventA);
+        // as queueing doesn't start processing synchronously anymore (see commit bbdb5ac)
+        // wait just long enough before it does
+        await Promise.resolve();
        expect(procCount).toEqual(1);
        defer.reject({});
-        retryDefer.promise.done(function() {
-            expect(procCount).toEqual(1);
-            clock.tick(waitTimeMs);
-            expect(procCount).toEqual(2);
-            done();
-        });
+        await retryDefer.promise;
+        expect(procCount).toEqual(1);
+        clock.tick(waitTimeMs);
+        await Promise.resolve();
+        expect(procCount).toEqual(2);
    });

    it("should give up if the retryFn on failure returns -1 and try the next event",
-    function(done) {
+    async function() {
        // Queue A & B.
        // Reject A and return -1 on retry.
        // Expect B to be tried next and the promise for A to be rejected.
@@ -122,8 +126,8 @@ describe("MatrixScheduler", function() {
            return -1;
        };
        queueFn = function() {
- return "yep";
-};
+            return "yep";
+        };

        const deferA = Promise.defer();
        const deferB = Promise.defer();
@@ -142,13 +146,17 @@ describe("MatrixScheduler", function() {

        const globalA = scheduler.queueEvent(eventA);
        scheduler.queueEvent(eventB);
-
+        // as queueing doesn't start processing synchronously anymore (see commit bbdb5ac)
+        // wait just long enough before it does
+        await Promise.resolve();
        expect(procCount).toEqual(1);
        deferA.reject({});
-        globalA.catch(function() {
+        try {
+            await globalA;
+        } catch(err) {
+            await Promise.resolve();
            expect(procCount).toEqual(2);
-            done();
-        });
+        }
    });

    it("should treat each queue separately", function(done) {
@@ -300,7 +308,11 @@ describe("MatrixScheduler", function() {
                expect(ev).toEqual(eventA);
                return defer.promise;
            });
-            expect(procCount).toEqual(1);
+            // as queueing doesn't start processing synchronously anymore (see commit bbdb5ac)
+            // wait just long enough before it does
+            Promise.resolve().then(() => {
+                expect(procCount).toEqual(1);
+            });
        });

        it("should not call the processFn if there are no queued events", function() {
@@ -34,12 +34,18 @@ export default class Reemitter {
    }

    reEmit(source, eventNames) {
+        // We include the source as the last argument for event handlers which may need it,
+        // such as read receipt listeners on the client class which won't have the context
+        // of the room.
+        const forSource = (handler, ...args) => {
+            handler(...args, source);
+        };
        for (const eventName of eventNames) {
            if (this.boundHandlers[eventName] === undefined) {
                this.boundHandlers[eventName] = this._handleEvent.bind(this, eventName);
            }
-            const boundHandler = this.boundHandlers[eventName];

+            const boundHandler = forSource.bind(this, this.boundHandlers[eventName]);
            source.on(eventName, boundHandler);
        }
    }
@@ -17,7 +17,7 @@ limitations under the License.
 /** @module auto-discovery */

 import Promise from 'bluebird';
-const logger = require("./logger");
+import logger from './logger';
 import { URL as NodeURL } from "url";

 // Dev note: Auto discovery is part of the spec.
@@ -256,6 +256,11 @@ export class AutoDiscovery {
        if (!hsVersions || !hsVersions.raw["versions"]) {
            logger.error("Invalid /versions response");
            clientConfig["m.homeserver"].error = AutoDiscovery.ERROR_INVALID_HOMESERVER;
+
+            // Supply the base_url to the caller because they may be ignoring liveliness
+            // errors, like this one.
+            clientConfig["m.homeserver"].base_url = hsUrl;
+
            return Promise.resolve(clientConfig);
        }

@@ -311,6 +316,11 @@ export class AutoDiscovery {
                logger.error("Invalid /api/v1 response");
                failingClientConfig["m.identity_server"].error =
                    AutoDiscovery.ERROR_INVALID_IDENTITY_SERVER;
+
+                // Supply the base_url to the caller because they may be ignoring
+                // liveliness errors, like this one.
+                failingClientConfig["m.identity_server"].base_url = isUrl;
+
                return Promise.resolve(failingClientConfig);
            }
        }
@@ -419,6 +429,26 @@ export class AutoDiscovery {
        return AutoDiscovery.fromDiscoveryConfig(wellknown.raw);
    }

+    /**
+     * Gets the raw discovery client configuration for the given domain name.
+     * Should only be used if there's no validation to be done on the resulting
+     * object, otherwise use findClientConfig().
+     * @param {string} domain The domain to get the client config for.
+     * @returns {Promise<object>} Resolves to the domain's client config. Can
+     * be an empty object.
+     */
+    static async getRawClientConfig(domain) {
+        if (!domain || typeof(domain) !== "string" || domain.length === 0) {
+            throw new Error("'domain' must be a string of non-zero length");
+        }
+
+        const response = await this._fetchWellKnownObject(
+            `https://${domain}/.well-known/matrix/client`,
+        );
+        if (!response) return {};
+        return response.raw || {};
+    }
+
    /**
     * Sanitizes a given URL to ensure it is either an HTTP or HTTP URL and
     * is suitable for the requirements laid out by .well-known auto discovery.
@@ -1,6 +1,8 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
+Copyright 2019 The Matrix.org Foundation C.I.C.
+Copyright 2019 Michael Telatynski <7t3chguy@gmail.com>

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -23,8 +25,23 @@ limitations under the License.
 * @module base-apis
 */

+import { SERVICE_TYPES } from './service-types';
+import logger from './logger';
+
 const httpApi = require("./http-api");
 const utils = require("./utils");
+const PushProcessor = require("./pushprocessor");
+
+function termsUrlForService(serviceType, baseUrl) {
+    switch (serviceType) {
+        case SERVICE_TYPES.IS:
+            return baseUrl + httpApi.PREFIX_IDENTITY_V2 + '/terms';
+        case SERVICE_TYPES.IM:
+            return baseUrl + '/_matrix/integrations/v1/terms';
+        default:
+            throw new Error('Unsupported service type');
+    }
+}

 /**
 * Low-level wrappers for the Matrix APIs
@@ -46,6 +63,15 @@ const utils = require("./utils");
 *
 * @param {string} opts.accessToken The access_token for this user.
 *
+ * @param {IdentityServerProvider} [opts.identityServer]
+ * Optional. A provider object with one function `getAccessToken`, which is a
+ * callback that returns a Promise<String> of an identity access token to supply
+ * with identity requests. If the object is unset, no access token will be
+ * supplied.
+ * See also https://github.com/vector-im/riot-web/issues/10615 which seeks to
+ * replace the previous approach of manual access tokens params with this
+ * callback throughout the SDK.
+ *
 * @param {Number=} opts.localTimeoutMs Optional. The default maximum amount of
 * time to wait before timing out HTTP requests. If not specified, there is no
 * timeout.
@@ -62,6 +88,7 @@ function MatrixBaseApis(opts) {

    this.baseUrl = opts.baseUrl;
    this.idBaseUrl = opts.idBaseUrl;
+    this.identityServer = opts.identityServer;

    const httpOpts = {
        baseUrl: opts.baseUrl,
@@ -100,6 +127,15 @@ MatrixBaseApis.prototype.getIdentityServerUrl = function(stripProto=false) {
    return this.idBaseUrl;
 };

+/**
+ * Set the Identity Server URL of this client
+ * @param {string} url New Identity Server URL
+ */
+MatrixBaseApis.prototype.setIdentityServerUrl = function(url) {
+    this.idBaseUrl = utils.ensureNoTrailingSlash(url);
+    this._http.setIdBaseUrl(this.idBaseUrl);
+};
+
 /**
 * Get the access token associated with this account.
 * @return {?String} The access_token or null
@@ -151,13 +187,14 @@ MatrixBaseApis.prototype.isUsernameAvailable = function(username) {
 *     threepid uses during registration in the ID server. Set 'msisdn' to
 *     true to bind msisdn.
 * @param {string} guestAccessToken
+ * @param {string} inhibitLogin
 * @param {module:client.callback} callback Optional.
 * @return {module:client.Promise} Resolves: TODO
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixBaseApis.prototype.register = function(
    username, password,
-    sessionId, auth, bindThreepids, guestAccessToken,
+    sessionId, auth, bindThreepids, guestAccessToken, inhibitLogin,
    callback,
 ) {
    // backwards compat
@@ -166,6 +203,10 @@ MatrixBaseApis.prototype.register = function(
    } else if (bindThreepids === null || bindThreepids === undefined) {
        bindThreepids = {};
    }
+    if (typeof inhibitLogin === 'function') {
+        callback = inhibitLogin;
+        inhibitLogin = undefined;
+    }

    if (auth === undefined || auth === null) {
        auth = {};
@@ -192,6 +233,9 @@ MatrixBaseApis.prototype.register = function(
    if (guestAccessToken !== undefined && guestAccessToken !== null) {
        params.guest_access_token = guestAccessToken;
    }
+    if (inhibitLogin !== undefined && inhibitLogin !== null) {
+        params.inhibit_login = inhibitLogin;
+    }
    // Temporary parameter added to make the register endpoint advertise
    // msisdn flows. This exists because there are clients that break
    // when given stages they don't recognise. This parameter will cease
@@ -383,9 +427,8 @@ MatrixBaseApis.prototype.deactivateAccount = function(auth, erase) {
        body.erase = erase;
    }

-    return this._http.authedRequestWithPrefix(
+    return this._http.authedRequest(
        undefined, "POST", '/account/deactivate', undefined, body,
-        httpApi.PREFIX_R0,
    );
 };

@@ -430,6 +473,37 @@ MatrixBaseApis.prototype.createRoom = function(options, callback) {
        callback, "POST", "/createRoom", undefined, options,
    );
 };
+/**
+ * Fetches relations for a given event
+ * @param {string} roomId the room of the event
+ * @param {string} eventId the id of the event
+ * @param {string} relationType the rel_type of the relations requested
+ * @param {string} eventType the event type of the relations requested
+ * @param {Object} opts options with optional values for the request.
+ * @param {Object} opts.from the pagination token returned from a previous request as `next_batch` to return following relations.
+ * @return {Object} the response, with chunk and next_batch.
+ */
+MatrixBaseApis.prototype.fetchRelations =
+    async function(roomId, eventId, relationType, eventType, opts) {
+    const queryParams = {};
+    if (opts.from) {
+        queryParams.from = opts.from;
+    }
+    const queryString = utils.encodeParams(queryParams);
+    const path = utils.encodeUri(
+        "/rooms/$roomId/relations/$eventId/$relationType/$eventType?" + queryString, {
+            $roomId: roomId,
+            $eventId: eventId,
+            $relationType: relationType,
+            $eventType: eventType,
+        });
+    const response = await this._http.authedRequest(
+        undefined, "GET", path, null, null, {
+            prefix: httpApi.PREFIX_UNSTABLE,
+        },
+    );
+    return response;
+};

 /**
 * @param {string} roomId
@@ -887,32 +961,6 @@ MatrixBaseApis.prototype.sendStateEvent = function(roomId, eventType, content, s
    );
 };

-/**
- * @param {string} roomId
- * @param {string} eventId
- * @param {string} [txnId]  transaction id. One will be made up if not
- *    supplied.
- * @param {module:client.callback} callback Optional.
- * @return {module:client.Promise} Resolves: TODO
- * @return {module:http-api.MatrixError} Rejects: with an error response.
- */
-MatrixBaseApis.prototype.redactEvent = function(
-    roomId, eventId, txnId, callback,
-) {
-    if (arguments.length === 3) {
-        callback = txnId;
-    }
-
-    const path = utils.encodeUri("/rooms/$roomId/redact/$eventId/$txnId", {
-        $roomId: roomId,
-        $eventId: eventId,
-        $txnId: txnId ? txnId : this.makeTxnId(),
-    });
-
-    return this._http.authedRequest(callback, "PUT", path, undefined, {});
-};
-
-
 /**
 * @param {string} roomId
 * @param {Number} limit
@@ -945,10 +993,13 @@ MatrixBaseApis.prototype.roomInitialSync = function(roomId, limit, callback) {
 * @param {string} rrEventId ID of the event tracked by the read receipt. This is here
 * for convenience because the RR and the RM are commonly updated at the same time as
 * each other. Optional.
+ * @param {object} opts Options for the read markers.
+ * @param {object} opts.hidden True to hide the read receipt from other users. <b>This
+ * property is currently unstable and may change in the future.</b>
 * @return {module:client.Promise} Resolves: the empty object, {}.
 */
 MatrixBaseApis.prototype.setRoomReadMarkersHttpRequest =
-                                function(roomId, rmEventId, rrEventId) {
+                                function(roomId, rmEventId, rrEventId, opts) {
    const path = utils.encodeUri("/rooms/$roomId/read_markers", {
        $roomId: roomId,
    });
@@ -956,6 +1007,7 @@ MatrixBaseApis.prototype.setRoomReadMarkersHttpRequest =
    const content = {
        "m.fully_read": rmEventId,
        "m.read": rrEventId,
+        "m.hidden": Boolean(opts ? opts.hidden : false),
    };

    return this._http.authedRequest(
@@ -1287,10 +1339,16 @@ MatrixBaseApis.prototype.getThreePids = function(callback) {
 };

 /**
+ * Add a 3PID to your homeserver account and optionally bind it to an identity
+ * server as well. An identity server is required as part of the `creds` object.
+ *
+ * This API is deprecated, and you should instead use `addThreePidOnly`
+ * for homeservers that support it.
+ *
 * @param {Object} creds
 * @param {boolean} bind
 * @param {module:client.callback} callback Optional.
- * @return {module:client.Promise} Resolves: TODO
+ * @return {module:client.Promise} Resolves: on success
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixBaseApis.prototype.addThreePid = function(creds, bind, callback) {
@@ -1304,6 +1362,75 @@ MatrixBaseApis.prototype.addThreePid = function(creds, bind, callback) {
    );
 };

+/**
+ * Add a 3PID to your homeserver account. This API does not use an identity
+ * server, as the homeserver is expected to handle 3PID ownership validation.
+ *
+ * You can check whether a homeserver supports this API via
+ * `doesServerSupportSeparateAddAndBind`.
+ *
+ * @param {Object} data A object with 3PID validation data from having called
+ * `account/3pid/<medium>/requestToken` on the homeserver.
+ * @return {module:client.Promise} Resolves: on success
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.addThreePidOnly = function(data) {
+    const path = "/account/3pid/add";
+    return this._http.authedRequest(
+        undefined, "POST", path, null, data, {
+            prefix: httpApi.PREFIX_UNSTABLE,
+        },
+    );
+};
+
+/**
+ * Bind a 3PID for discovery onto an identity server via the homeserver. The
+ * identity server handles 3PID ownership validation and the homeserver records
+ * the new binding to track where all 3PIDs for the account are bound.
+ *
+ * You can check whether a homeserver supports this API via
+ * `doesServerSupportSeparateAddAndBind`.
+ *
+ * @param {Object} data A object with 3PID validation data from having called
+ * `validate/<medium>/requestToken` on the identity server. It should also
+ * contain `id_server` and `id_access_token` fields as well.
+ * @return {module:client.Promise} Resolves: on success
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.bindThreePid = function(data) {
+    const path = "/account/3pid/bind";
+    return this._http.authedRequest(
+        undefined, "POST", path, null, data, {
+            prefix: httpApi.PREFIX_UNSTABLE,
+        },
+    );
+};
+
+/**
+ * Unbind a 3PID for discovery on an identity server via the homeserver. The
+ * homeserver removes its record of the binding to keep an updated record of
+ * where all 3PIDs for the account are bound.
+ *
+ * @param {string} medium The threepid medium (eg. 'email')
+ * @param {string} address The threepid address (eg. 'bob@example.com')
+ *        this must be as returned by getThreePids.
+ * @return {module:client.Promise} Resolves: on success
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.unbindThreePid = function(medium, address) {
+    const path = "/account/3pid/unbind";
+    const data = {
+        medium,
+        address,
+        id_server: this.getIdentityServerUrl(true),
+    };
+    return this._http.authedRequest(
+        undefined, "POST", path, null, data, {
+            prefix: httpApi.PREFIX_UNSTABLE,
+        },
+    );
+};
+
 /**
 * @param {string} medium The threepid medium (eg. 'email')
 * @param {string} address The threepid address (eg. 'bob@example.com')
@@ -1318,9 +1445,7 @@ MatrixBaseApis.prototype.deleteThreePid = function(medium, address) {
        'medium': medium,
        'address': address,
    };
-    return this._http.authedRequestWithPrefix(
-        undefined, "POST", path, null, data, httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "POST", path, null, data);
 };

 /**
@@ -1353,10 +1478,8 @@ MatrixBaseApis.prototype.setPassword = function(authDict, newPassword, callback)
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixBaseApis.prototype.getDevices = function() {
-    const path = "/devices";
-    return this._http.authedRequestWithPrefix(
-        undefined, "GET", path, undefined, undefined,
-        httpApi.PREFIX_UNSTABLE,
+    return this._http.authedRequest(
+        undefined, 'GET', "/devices", undefined, undefined,
    );
 };

@@ -1373,11 +1496,7 @@ MatrixBaseApis.prototype.setDeviceDetails = function(device_id, body) {
        $device_id: device_id,
    });

-
-    return this._http.authedRequestWithPrefix(
-        undefined, "PUT", path, undefined, body,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, body);
 };

 /**
@@ -1399,10 +1518,7 @@ MatrixBaseApis.prototype.deleteDevice = function(device_id, auth) {
        body.auth = auth;
    }

-    return this._http.authedRequestWithPrefix(
-        undefined, "DELETE", path, undefined, body,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "DELETE", path, undefined, body);
 };

 /**
@@ -1420,10 +1536,8 @@ MatrixBaseApis.prototype.deleteMultipleDevices = function(devices, auth) {
        body.auth = auth;
    }

-    return this._http.authedRequestWithPrefix(
-        undefined, "POST", "/delete_devices", undefined, body,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    const path = "/delete_devices";
+    return this._http.authedRequest(undefined, "POST", path, undefined, body);
 };


@@ -1465,7 +1579,9 @@ MatrixBaseApis.prototype.setPusher = function(pusher, callback) {
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
 MatrixBaseApis.prototype.getPushRules = function(callback) {
-    return this._http.authedRequest(callback, "GET", "/pushrules/");
+    return this._http.authedRequest(callback, "GET", "/pushrules/").then(rules => {
+        return PushProcessor.rewriteDefaultRules(rules);
+    });
 };

 /**
@@ -1599,9 +1715,7 @@ MatrixBaseApis.prototype.uploadKeysRequest = function(content, opts, callback) {
    } else {
        path = "/keys/upload";
    }
-    return this._http.authedRequestWithPrefix(
-        callback, "POST", path, undefined, content, httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(callback, "POST", path, undefined, content);
 };

 /**
@@ -1636,10 +1750,7 @@ MatrixBaseApis.prototype.downloadKeysForUsers = function(userIds, opts) {
        content.device_keys[u] = {};
    });

-    return this._http.authedRequestWithPrefix(
-        undefined, "POST", "/keys/query", undefined, content,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "POST", "/keys/query", undefined, content);
 };

 /**
@@ -1667,10 +1778,8 @@ MatrixBaseApis.prototype.claimOneTimeKeys = function(devices, key_algorithm) {
        query[deviceId] = key_algorithm;
    }
    const content = {one_time_keys: queries};
-    return this._http.authedRequestWithPrefix(
-        undefined, "POST", "/keys/claim", undefined, content,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    const path = "/keys/claim";
+    return this._http.authedRequest(undefined, "POST", path, undefined, content);
 };

 /**
@@ -1689,10 +1798,8 @@ MatrixBaseApis.prototype.getKeyChanges = function(oldToken, newToken) {
        to: newToken,
    };

-    return this._http.authedRequestWithPrefix(
-        undefined, "GET", "/keys/changes", qps, undefined,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    const path = "/keys/changes";
+    return this._http.authedRequest(undefined, "GET", path, qps, undefined);
 };


@@ -1700,10 +1807,36 @@ MatrixBaseApis.prototype.getKeyChanges = function(oldToken, newToken) {
 // ==========================

 /**
- * Requests an email verification token directly from an Identity Server.
+ * Register with an Identity Server using the OpenID token from the user's
+ * Homeserver, which can be retrieved via
+ * {@link module:client~MatrixClient#getOpenIdToken}.
 *
- * Note that the Home Server offers APIs to proxy this API for specific
- * situations, allowing for better feedback to the user.
+ * Note that the `/account/register` endpoint (as well as IS authentication in
+ * general) was added as part of the v2 API version.
+ *
+ * @param {object} hsOpenIdToken
+ * @return {module:client.Promise} Resolves: with object containing an Identity
+ * Server access token.
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.registerWithIdentityServer = function(hsOpenIdToken) {
+    if (!this.idBaseUrl) {
+        throw new Error("No Identity Server base URL set");
+    }
+
+    const uri = this.idBaseUrl + httpApi.PREFIX_IDENTITY_V2 + "/account/register";
+    return this._http.requestOtherUrl(
+        undefined, "POST", uri,
+        null, hsOpenIdToken,
+    );
+};
+
+/**
+ * Requests an email verification token directly from an identity server.
+ *
+ * This API is used as part of binding an email for discovery on an identity
+ * server. The validation data that results should be passed to the
+ * `bindThreePid` method to complete the binding process.
 *
 * @param {string} email The email address to request a token for
 * @param {string} clientSecret A secret binary string generated by the client.
@@ -1715,26 +1848,122 @@ MatrixBaseApis.prototype.getKeyChanges = function(oldToken, newToken) {
 * @param {string} nextLink Optional If specified, the client will be redirected
 *                 to this link after validation.
 * @param {module:client.callback} callback Optional.
+ * @param {string} identityAccessToken The `access_token` field of the identity
+ * server `/account/register` response (see {@link registerWithIdentityServer}).
+ *
 * @return {module:client.Promise} Resolves: TODO
 * @return {module:http-api.MatrixError} Rejects: with an error response.
- * @throws Error if No ID server is set
+ * @throws Error if no identity server is set
 */
-MatrixBaseApis.prototype.requestEmailToken = function(email, clientSecret,
-                                                    sendAttempt, nextLink, callback) {
+MatrixBaseApis.prototype.requestEmailToken = async function(
+    email,
+    clientSecret,
+    sendAttempt,
+    nextLink,
+    callback,
+    identityAccessToken,
+) {
    const params = {
        client_secret: clientSecret,
        email: email,
        send_attempt: sendAttempt,
        next_link: nextLink,
    };
-    return this._http.idServerRequest(
-        callback, "POST", "/validate/email/requestToken",
-        params, httpApi.PREFIX_IDENTITY_V1,
-    );
+
+    try {
+        const response = await this._http.idServerRequest(
+            undefined, "POST", "/validate/email/requestToken",
+            params, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+        );
+        // TODO: Fold callback into above call once v1 path below is removed
+        if (callback) callback(null, response);
+        return response;
+    } catch (err) {
+        if (err.cors === "rejected" || err.httpStatus === 404) {
+            // Fall back to deprecated v1 API for now
+            // TODO: Remove this path once v2 is only supported version
+            // See https://github.com/vector-im/riot-web/issues/10443
+            logger.warn("IS doesn't support v2, falling back to deprecated v1");
+            return await this._http.idServerRequest(
+                callback, "POST", "/validate/email/requestToken",
+                params, httpApi.PREFIX_IDENTITY_V1,
+            );
+        }
+        if (callback) callback(err);
+        throw err;
+    }
 };

 /**
- * Submits an MSISDN token to the identity server
+ * Requests a MSISDN verification token directly from an identity server.
+ *
+ * This API is used as part of binding a MSISDN for discovery on an identity
+ * server. The validation data that results should be passed to the
+ * `bindThreePid` method to complete the binding process.
+ *
+ * @param {string} phoneCountry The ISO 3166-1 alpha-2 code for the country in
+ *                 which phoneNumber should be parsed relative to.
+ * @param {string} phoneNumber The phone number, in national or international
+ *                 format
+ * @param {string} clientSecret A secret binary string generated by the client.
+ *                 It is recommended this be around 16 ASCII characters.
+ * @param {number} sendAttempt If an identity server sees a duplicate request
+ *                 with the same sendAttempt, it will not send another SMS.
+ *                 To request another SMS to be sent, use a larger value for
+ *                 the sendAttempt param as was used in the previous request.
+ * @param {string} nextLink Optional If specified, the client will be redirected
+ *                 to this link after validation.
+ * @param {module:client.callback} callback Optional.
+ * @param {string} identityAccessToken The `access_token` field of the Identity
+ * Server `/account/register` response (see {@link registerWithIdentityServer}).
+ *
+ * @return {module:client.Promise} Resolves: TODO
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ * @throws Error if no identity server is set
+ */
+MatrixBaseApis.prototype.requestMsisdnToken = async function(
+    phoneCountry,
+    phoneNumber,
+    clientSecret,
+    sendAttempt,
+    nextLink,
+    callback,
+    identityAccessToken,
+) {
+    const params = {
+        client_secret: clientSecret,
+        country: phoneCountry,
+        phone_number: phoneNumber,
+        send_attempt: sendAttempt,
+        next_link: nextLink,
+    };
+
+    try {
+        const response = await this._http.idServerRequest(
+            undefined, "POST", "/validate/msisdn/requestToken",
+            params, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+        );
+        // TODO: Fold callback into above call once v1 path below is removed
+        if (callback) callback(null, response);
+        return response;
+    } catch (err) {
+        if (err.cors === "rejected" || err.httpStatus === 404) {
+            // Fall back to deprecated v1 API for now
+            // TODO: Remove this path once v2 is only supported version
+            // See https://github.com/vector-im/riot-web/issues/10443
+            logger.warn("IS doesn't support v2, falling back to deprecated v1");
+            return await this._http.idServerRequest(
+                callback, "POST", "/validate/msisdn/requestToken",
+                params, httpApi.PREFIX_IDENTITY_V1,
+            );
+        }
+        if (callback) callback(err);
+        throw err;
+    }
+};
+
+/**
+ * Submits a MSISDN token to the identity server
 *
 * This is used when submitting the code sent by SMS to a phone number.
 * The ID server has an equivalent API for email but the js-sdk does
@@ -1744,46 +1973,323 @@ MatrixBaseApis.prototype.requestEmailToken = function(email, clientSecret,
 * @param {string} sid The sid given in the response to requestToken
 * @param {string} clientSecret A secret binary string generated by the client.
 *                 This must be the same value submitted in the requestToken call.
- * @param {string} token The token, as enetered by the user.
+ * @param {string} msisdnToken The MSISDN token, as enetered by the user.
+ * @param {string} identityAccessToken The `access_token` field of the Identity
+ * Server `/account/register` response (see {@link registerWithIdentityServer}).
 *
 * @return {module:client.Promise} Resolves: Object, currently with no parameters.
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 * @throws Error if No ID server is set
 */
-MatrixBaseApis.prototype.submitMsisdnToken = function(sid, clientSecret, token) {
+MatrixBaseApis.prototype.submitMsisdnToken = async function(
+    sid,
+    clientSecret,
+    msisdnToken,
+    identityAccessToken,
+) {
    const params = {
        sid: sid,
        client_secret: clientSecret,
-        token: token,
+        token: msisdnToken,
    };
-    return this._http.idServerRequest(
-        undefined, "POST", "/validate/msisdn/submitToken",
-        params, httpApi.PREFIX_IDENTITY_V1,
+
+    try {
+        return await this._http.idServerRequest(
+            undefined, "POST", "/validate/msisdn/submitToken",
+            params, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+        );
+    } catch (err) {
+        if (err.cors === "rejected" || err.httpStatus === 404) {
+            // Fall back to deprecated v1 API for now
+            // TODO: Remove this path once v2 is only supported version
+            // See https://github.com/vector-im/riot-web/issues/10443
+            logger.warn("IS doesn't support v2, falling back to deprecated v1");
+            return await this._http.idServerRequest(
+                undefined, "POST", "/validate/msisdn/submitToken",
+                params, httpApi.PREFIX_IDENTITY_V1,
+            );
+        }
+        throw err;
+    }
+};
+
+/**
+ * Submits a MSISDN token to an arbitrary URL.
+ *
+ * This is used when submitting the code sent by SMS to a phone number in the
+ * newer 3PID flow where the homeserver validates 3PID ownership (as part of
+ * `requestAdd3pidMsisdnToken`). The homeserver response may include a
+ * `submit_url` to specify where the token should be sent, and this helper can
+ * be used to pass the token to this URL.
+ *
+ * @param {string} url The URL to submit the token to
+ * @param {string} sid The sid given in the response to requestToken
+ * @param {string} clientSecret A secret binary string generated by the client.
+ *                 This must be the same value submitted in the requestToken call.
+ * @param {string} msisdnToken The MSISDN token, as enetered by the user.
+ *
+ * @return {module:client.Promise} Resolves: Object, currently with no parameters.
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.submitMsisdnTokenOtherUrl = function(
+    url,
+    sid,
+    clientSecret,
+    msisdnToken,
+) {
+    const params = {
+        sid: sid,
+        client_secret: clientSecret,
+        token: msisdnToken,
+    };
+
+    return this._http.requestOtherUrl(
+        undefined, "POST", url, undefined, params,
    );
 };

+/**
+ * Gets the V2 hashing information from the identity server. Primarily useful for
+ * lookups.
+ * @param {string} identityAccessToken The access token for the identity server.
+ * @returns {Promise<object>} The hashing information for the identity server.
+ */
+MatrixBaseApis.prototype.getIdentityHashDetails = function(identityAccessToken) {
+    return this._http.idServerRequest(
+        undefined, "GET", "/hash_details",
+        null, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+    );
+};
+
+/**
+ * Performs a hashed lookup of addresses against the identity server. This is
+ * only supported on identity servers which have at least the version 2 API.
+ * @param {Array<Array<string,string>>} addressPairs An array of 2 element arrays.
+ * The first element of each pair is the address, the second is the 3PID medium.
+ * Eg: ["email@example.org", "email"]
+ * @param {string} identityAccessToken The access token for the identity server.
+ * @returns {Promise<Array<{address, mxid}>>} A collection of address mappings to
+ * found MXIDs. Results where no user could be found will not be listed.
+ */
+MatrixBaseApis.prototype.identityHashedLookup = async function(
+    addressPairs, // [["email@example.org", "email"], ["10005550000", "msisdn"]]
+    identityAccessToken,
+) {
+    const params = {
+        // addresses: ["email@example.org", "10005550000"],
+        // algorithm: "sha256",
+        // pepper: "abc123"
+    };
+
+    // Get hash information first before trying to do a lookup
+    const hashes = await this.getIdentityHashDetails(identityAccessToken);
+    if (!hashes || !hashes['lookup_pepper'] || !hashes['algorithms']) {
+        throw new Error("Unsupported identity server: bad response");
+    }
+
+    params['pepper'] = hashes['lookup_pepper'];
+
+    const localMapping = {
+        // hashed identifier => plain text address
+        // For use in this function's return format
+    };
+
+    // When picking an algorithm, we pick the hashed over no hashes
+    if (hashes['algorithms'].includes('sha256')) {
+        // Abuse the olm hashing
+        const olmutil = new global.Olm.Utility();
+        params["addresses"] = addressPairs.map(p => {
+            const addr = p[0].toLowerCase(); // lowercase to get consistent hashes
+            const med = p[1].toLowerCase();
+            const hashed = olmutil.sha256(`${addr} ${med} ${params['pepper']}`)
+                .replace(/\+/g, '-').replace(/\//g, '_'); // URL-safe base64
+            // Map the hash to a known (case-sensitive) address. We use the case
+            // sensitive version because the caller might be expecting that.
+            localMapping[hashed] = p[0];
+            return hashed;
+        });
+        params["algorithm"] = "sha256";
+    } else if (hashes['algorithms'].includes('none')) {
+        params["addresses"] = addressPairs.map(p => {
+            const addr = p[0].toLowerCase(); // lowercase to get consistent hashes
+            const med = p[1].toLowerCase();
+            const unhashed = `${addr} ${med}`;
+            // Map the unhashed values to a known (case-sensitive) address. We use
+            // the case sensitive version because the caller might be expecting that.
+            localMapping[unhashed] = p[0];
+            return unhashed;
+        });
+        params["algorithm"] = "none";
+    } else {
+        throw new Error("Unsupported identity server: unknown hash algorithm");
+    }
+
+    const response = await this._http.idServerRequest(
+        undefined, "POST", "/lookup",
+        params, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+    );
+
+    if (!response || !response['mappings']) return []; // no results
+
+    const foundAddresses = [/* {address: "plain@example.org", mxid} */];
+    for (const hashed of Object.keys(response['mappings'])) {
+        const mxid = response['mappings'][hashed];
+        const plainAddress = localMapping[hashed];
+        if (!plainAddress) {
+            throw new Error("Identity server returned more results than expected");
+        }
+
+        foundAddresses.push({address: plainAddress, mxid});
+    }
+    return foundAddresses;
+};
+
 /**
 * Looks up the public Matrix ID mapping for a given 3rd party
 * identifier from the Identity Server
+ *
 * @param {string} medium The medium of the threepid, eg. 'email'
 * @param {string} address The textual address of the threepid
 * @param {module:client.callback} callback Optional.
+ * @param {string} identityAccessToken The `access_token` field of the Identity
+ * Server `/account/register` response (see {@link registerWithIdentityServer}).
+ *
 * @return {module:client.Promise} Resolves: A threepid mapping
 *                                 object or the empty object if no mapping
 *                                 exists
 * @return {module:http-api.MatrixError} Rejects: with an error response.
 */
-MatrixBaseApis.prototype.lookupThreePid = function(medium, address, callback) {
-    const params = {
-        medium: medium,
-        address: address,
-    };
-    return this._http.idServerRequest(
-        callback, "GET", "/lookup",
-        params, httpApi.PREFIX_IDENTITY_V1,
-    );
+MatrixBaseApis.prototype.lookupThreePid = async function(
+    medium,
+    address,
+    callback,
+    identityAccessToken,
+) {
+    try {
+        // Note: we're using the V2 API by calling this function, but our
+        // function contract requires a V1 response. We therefore have to
+        // convert it manually.
+        const response = await this.identityHashedLookup(
+            [[address, medium]], identityAccessToken,
+        );
+        const result = response.find(p => p.address === address);
+        if (!result) {
+            // TODO: Fold callback into above call once v1 path below is removed
+            if (callback) callback(null, {});
+            return {};
+        }
+
+        const mapping = {
+            address,
+            medium,
+            mxid: result.mxid,
+
+            // We can't reasonably fill these parameters:
+            // not_before
+            // not_after
+            // ts
+            // signatures
+        };
+
+        // TODO: Fold callback into above call once v1 path below is removed
+        if (callback) callback(null, mapping);
+        return mapping;
+    } catch (err) {
+        if (err.cors === "rejected" || err.httpStatus === 404) {
+            // Fall back to deprecated v1 API for now
+            // TODO: Remove this path once v2 is only supported version
+            // See https://github.com/vector-im/riot-web/issues/10443
+            const params = {
+                medium: medium,
+                address: address,
+            };
+            logger.warn("IS doesn't support v2, falling back to deprecated v1");
+            return await this._http.idServerRequest(
+                callback, "GET", "/lookup",
+                params, httpApi.PREFIX_IDENTITY_V1,
+            );
+        }
+        if (callback) callback(err, undefined);
+        throw err;
+    }
 };

+/**
+ * Looks up the public Matrix ID mappings for multiple 3PIDs.
+ *
+ * @param {Array.<Array.<string>>} query Array of arrays containing
+ * [medium, address]
+ * @param {string} identityAccessToken The `access_token` field of the Identity
+ * Server `/account/register` response (see {@link registerWithIdentityServer}).
+ *
+ * @return {module:client.Promise} Resolves: Lookup results from IS.
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.bulkLookupThreePids = async function(
+    query,
+    identityAccessToken,
+) {
+    try {
+        // Note: we're using the V2 API by calling this function, but our
+        // function contract requires a V1 response. We therefore have to
+        // convert it manually.
+        const response = await this.identityHashedLookup(
+            // We have to reverse the query order to get [address, medium] pairs
+            query.map(p => [p[1], p[0]]), identityAccessToken,
+        );
+
+        const v1results = [];
+        for (const mapping of response) {
+            const originalQuery = query.find(p => p[1] === mapping.address);
+            if (!originalQuery) {
+                throw new Error("Identity sever returned unexpected results");
+            }
+
+            v1results.push([
+                originalQuery[0], // medium
+                mapping.address,
+                mapping.mxid,
+            ]);
+        }
+
+        return {threepids: v1results};
+    } catch (err) {
+        if (err.cors === "rejected" || err.httpStatus === 404) {
+            // Fall back to deprecated v1 API for now
+            // TODO: Remove this path once v2 is only supported version
+            // See https://github.com/vector-im/riot-web/issues/10443
+            const params = {
+                threepids: query,
+            };
+            logger.warn("IS doesn't support v2, falling back to deprecated v1");
+            return await this._http.idServerRequest(
+                undefined, "POST", "/bulk_lookup", params,
+                httpApi.PREFIX_IDENTITY_V1, identityAccessToken,
+            );
+        }
+        throw err;
+    }
+};
+
+/**
+ * Get account info from the Identity Server. This is useful as a neutral check
+ * to verify that other APIs are likely to approve access by testing that the
+ * token is valid, terms have been agreed, etc.
+ *
+ * @param {string} identityAccessToken The `access_token` field of the Identity
+ * Server `/account/register` response (see {@link registerWithIdentityServer}).
+ *
+ * @return {module:client.Promise} Resolves: an object with account info.
+ * @return {module:http-api.MatrixError} Rejects: with an error response.
+ */
+MatrixBaseApis.prototype.getIdentityAccount = function(
+    identityAccessToken,
+) {
+    return this._http.idServerRequest(
+        undefined, "GET", "/account",
+        undefined, httpApi.PREFIX_IDENTITY_V2, identityAccessToken,
+    );
+};

 // Direct-to-device messaging
 // ==========================
@@ -1810,10 +2316,7 @@ MatrixBaseApis.prototype.sendToDevice = function(
        messages: contentMap,
    };

-    return this._http.authedRequestWithPrefix(
-        undefined, "PUT", path, undefined, body,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "PUT", path, undefined, body);
 };

 // Third party Lookup API
@@ -1825,9 +2328,8 @@ MatrixBaseApis.prototype.sendToDevice = function(
 * @return {module:client.Promise} Resolves to the result object
 */
 MatrixBaseApis.prototype.getThirdpartyProtocols = function() {
-    return this._http.authedRequestWithPrefix(
+    return this._http.authedRequest(
        undefined, "GET", "/thirdparty/protocols", undefined, undefined,
-        httpApi.PREFIX_UNSTABLE,
    ).then((response) => {
        // sanity check
        if (!response || typeof(response) !== 'object') {
@@ -1852,10 +2354,7 @@ MatrixBaseApis.prototype.getThirdpartyLocation = function(protocol, params) {
        $protocol: protocol,
    });

-    return this._http.authedRequestWithPrefix(
-        undefined, "GET", path, params, undefined,
-        httpApi.PREFIX_UNSTABLE,
-    );
+    return this._http.authedRequest(undefined, "GET", path, params, undefined);
 };

 /**
@@ -1871,12 +2370,45 @@ MatrixBaseApis.prototype.getThirdpartyUser = function(protocol, params) {
        $protocol: protocol,
    });

-    return this._http.authedRequestWithPrefix(
-        undefined, "GET", path, params, undefined,
-        httpApi.PREFIX_UNSTABLE,
+    return this._http.authedRequest(undefined, "GET", path, params, undefined);
+};
+
+MatrixBaseApis.prototype.getTerms = function(serviceType, baseUrl) {
+    const url = termsUrlForService(serviceType, baseUrl);
+    return this._http.requestOtherUrl(
+        undefined, 'GET', url,
    );
 };

+MatrixBaseApis.prototype.agreeToTerms = function(
+    serviceType, baseUrl, accessToken, termsUrls,
+) {
+    const url = termsUrlForService(serviceType, baseUrl);
+    const headers = {
+        Authorization: "Bearer " + accessToken,
+    };
+    return this._http.requestOtherUrl(
+        undefined, 'POST', url, null, { user_accepts: termsUrls }, { headers },
+    );
+};
+
+/**
+ * Reports an event as inappropriate to the server, which may then notify the appropriate people.
+ * @param {string} roomId The room in which the event being reported is located.
+ * @param {string} eventId The event to report.
+ * @param {number} score The score to rate this content as where -100 is most offensive and 0 is inoffensive.
+ * @param {string} reason The reason the content is being reported. May be blank.
+ * @returns {module:client.Promise} Resolves to an empty object if successful
+ */
+MatrixBaseApis.prototype.reportEvent = function(roomId, eventId, score, reason) {
+    const path = utils.encodeUri("/rooms/$roomId/report/$eventId", {
+        $roomId: roomId,
+        $eventId: eventId,
+    });
+
+    return this._http.authedRequest(undefined, "POST", path, null, {score, reason});
+};
+
 /**
 * MatrixBaseApis object
 */
@@ -47,14 +47,14 @@ module.exports = {
            }
        }
        let serverAndMediaId = mxc.slice(6); // strips mxc://
-        let prefix = "/_matrix/media/v1/download/";
+        let prefix = "/_matrix/media/r0/download/";
        const params = {};

        if (width) {
-            params.width = width;
+            params.width = Math.round(width);
        }
        if (height) {
-            params.height = height;
+            params.height = Math.round(height);
        }
        if (resizeMethod) {
            params.method = resizeMethod;
@@ -62,7 +62,7 @@ module.exports = {
        if (utils.keys(params).length > 0) {
            // these are thumbnailing params so they probably want the
            // thumbnailing API...
-            prefix = "/_matrix/media/v1/thumbnail/";
+            prefix = "/_matrix/media/r0/thumbnail/";
        }

        const fragmentOffset = serverAndMediaId.indexOf("#");
@@ -83,6 +83,7 @@ module.exports = {
     * @param {Number} width The desired width of the image in pixels. Default: 96.
     * @param {Number} height The desired height of the image in pixels. Default: 96.
     * @return {string} The complete URL to the identicon.
+     * @deprecated This is no longer in the specification.
     */
    getIdenticonUri: function(baseUrl, identiconString, width, height) {
        if (!identiconString) {
@@ -99,7 +100,7 @@ module.exports = {
            height: height,
        };

-        const path = utils.encodeUri("/_matrix/media/v1/identicon/$ident", {
+        const path = utils.encodeUri("/_matrix/media/unstable/identicon/$ident", {
            $ident: identiconString,
        });
        return baseUrl + path +
@@ -462,7 +462,7 @@ OlmDevice.prototype.createInboundSession = async function(
 */
 OlmDevice.prototype.getSessionIdsForDevice = async function(theirDeviceIdentityKey) {
    if (this._sessionsInProgress[theirDeviceIdentityKey]) {
-        console.log("waiting for session to be created");
+        logger.log("waiting for session to be created");
        try {
            await this._sessionsInProgress[theirDeviceIdentityKey];
        } catch (e) {
@@ -594,6 +594,11 @@ OlmDevice.prototype.encryptMessage = async function(
        'readwrite', [IndexedDBCryptoStore.STORE_SESSIONS],
        (txn) => {
            this._getSession(theirDeviceIdentityKey, sessionId, txn, (sessionInfo) => {
+                const sessionDesc = sessionInfo.session.describe();
+                console.log(
+                    "Session ID " + sessionId + " to " +
+                    theirDeviceIdentityKey + ": " + sessionDesc,
+                );
                res = sessionInfo.session.encrypt(payloadString);
                this._saveSession(theirDeviceIdentityKey, sessionInfo, txn);
            });
@@ -621,6 +626,11 @@ OlmDevice.prototype.decryptMessage = async function(
        'readwrite', [IndexedDBCryptoStore.STORE_SESSIONS],
        (txn) => {
            this._getSession(theirDeviceIdentityKey, sessionId, txn, (sessionInfo) => {
+                const sessionDesc = sessionInfo.session.describe();
+                console.log(
+                    "Session ID " + sessionId + " to " +
+                    theirDeviceIdentityKey + ": " + sessionDesc,
+                );
                payloadString = sessionInfo.session.decrypt(messageType, ciphertext);
                sessionInfo.lastReceivedMessageTs = Date.now();
                this._saveSession(theirDeviceIdentityKey, sessionInfo, txn);
@@ -23,8 +23,8 @@ limitations under the License.
 */

 import Promise from 'bluebird';
+import logger from '../../logger';

-const logger = require("../../logger");
 const utils = require("../../utils");
 const olmlib = require("../olmlib");
 const base = require("./base");
@@ -278,7 +278,7 @@ MegolmEncryption.prototype._prepareNewSession = async function() {
        ).catch((e) => {
            // This throws if the upload failed, but this is fine
            // since it will have written it to the db and will retry.
-            console.log("Failed to back up group session", e);
+            logger.log("Failed to back up group session", e);
        });
    }

@@ -955,7 +955,7 @@ MegolmDecryption.prototype.onRoomKeyEvent = function(event) {
            ).catch((e) => {
                // This throws if the upload failed, but this is fine
                // since it will have written it to the db and will retry.
-                console.log("Failed to back up group session", e);
+                logger.log("Failed to back up group session", e);
            });
        }
    }).catch((e) => {
@@ -1088,7 +1088,7 @@ MegolmDecryption.prototype.importRoomKey = function(session) {
            ).catch((e) => {
                // This throws if the upload failed, but this is fine
                // since it will have written it to the db and will retry.
-                console.log("Failed to back up group session", e);
+                logger.log("Failed to back up group session", e);
            });
        }
        // have another go at decrypting events sent with this session.
@@ -22,7 +22,7 @@ limitations under the License.
 */
 import Promise from 'bluebird';

-const logger = require("../../logger");
+import logger from '../../logger';
 const utils = require("../../utils");
 const olmlib = require("../olmlib");
 const DeviceInfo = require("../deviceinfo");
@@ -25,7 +25,7 @@ const anotherjson = require('another-json');
 import Promise from 'bluebird';
 import {EventEmitter} from 'events';

-const logger = require("../logger");
+import logger from '../logger';
 const utils = require("../utils");
 const OlmDevice = require("./OlmDevice");
 const olmlib = require("./olmlib");
@@ -197,11 +197,11 @@ utils.inherits(Crypto, EventEmitter);
 * Returns a promise which resolves once the crypto module is ready for use.
 */
 Crypto.prototype.init = async function() {
-    console.log("Crypto: initialising Olm...");
+    logger.log("Crypto: initialising Olm...");
    await global.Olm.init();
-    console.log("Crypto: initialising Olm device...");
+    logger.log("Crypto: initialising Olm device...");
    await this._olmDevice.init();
-    console.log("Crypto: loading device list...");
+    logger.log("Crypto: loading device list...");
    await this._deviceList.load();

    // build our device keys: these will later be uploaded
@@ -210,7 +210,7 @@ Crypto.prototype.init = async function() {
    this._deviceKeys["curve25519:" + this._deviceId] =
        this._olmDevice.deviceCurve25519Key;

-    console.log("Crypto: fetching own devices...");
+    logger.log("Crypto: fetching own devices...");
    let myDevices = this._deviceList.getRawStoredDevicesForUser(
        this._userId,
    );
@@ -221,7 +221,7 @@ Crypto.prototype.init = async function() {

    if (!myDevices[this._deviceId]) {
        // add our own deviceinfo to the cryptoStore
-        console.log("Crypto: adding this device to the store...");
+        logger.log("Crypto: adding this device to the store...");
        const deviceInfo = {
            keys: this._deviceKeys,
            algorithms: this._supportedAlgorithms,
@@ -236,7 +236,7 @@ Crypto.prototype.init = async function() {
        this._deviceList.saveIfDirty();
    }

-    console.log("Crypto: checking for key backup...");
+    logger.log("Crypto: checking for key backup...");
    this._checkAndStartKeyBackup();
 };

@@ -247,9 +247,9 @@ Crypto.prototype.init = async function() {
 * to it.
 */
 Crypto.prototype._checkAndStartKeyBackup = async function() {
-    console.log("Checking key backup status...");
+    logger.log("Checking key backup status...");
    if (this._baseApis.isGuest()) {
-        console.log("Skipping key backup check since user is guest");
+        logger.log("Skipping key backup check since user is guest");
        this._checkedForBackup = true;
        return null;
    }
@@ -257,7 +257,7 @@ Crypto.prototype._checkAndStartKeyBackup = async function() {
    try {
        backupInfo = await this._baseApis.getKeyBackupVersion();
    } catch (e) {
-        console.log("Error checking for active key backup", e);
+        logger.log("Error checking for active key backup", e);
        if (e.httpStatus / 100 === 4) {
            // well that's told us. we won't try again.
            this._checkedForBackup = true;
@@ -269,27 +269,27 @@ Crypto.prototype._checkAndStartKeyBackup = async function() {
    const trustInfo = await this.isKeyBackupTrusted(backupInfo);

    if (trustInfo.usable && !this.backupInfo) {
-        console.log(
+        logger.log(
            "Found usable key backup v" + backupInfo.version +
            ": enabling key backups",
        );
        this._baseApis.enableKeyBackup(backupInfo);
    } else if (!trustInfo.usable && this.backupInfo) {
-        console.log("No usable key backup: disabling key backup");
+        logger.log("No usable key backup: disabling key backup");
        this._baseApis.disableKeyBackup();
    } else if (!trustInfo.usable && !this.backupInfo) {
-        console.log("No usable key backup: not enabling key backup");
+        logger.log("No usable key backup: not enabling key backup");
    } else if (trustInfo.usable && this.backupInfo) {
        // may not be the same version: if not, we should switch
        if (backupInfo.version !== this.backupInfo.version) {
-            console.log(
+            logger.log(
                "On backup version " + this.backupInfo.version + " but found " +
                "version " + backupInfo.version + ": switching.",
            );
            this._baseApis.disableKeyBackup();
            this._baseApis.enableKeyBackup(backupInfo);
        } else {
-            console.log("Backup version " + backupInfo.version + " still current");
+            logger.log("Backup version " + backupInfo.version + " still current");
        }
    }

@@ -359,7 +359,7 @@ Crypto.prototype.isKeyBackupTrusted = async function(backupInfo) {
    for (const keyId of Object.keys(mySigs)) {
        const keyIdParts = keyId.split(':');
        if (keyIdParts[0] !== 'ed25519') {
-            console.log("Ignoring unknown signature type: " + keyIdParts[0]);
+            logger.log("Ignoring unknown signature type: " + keyIdParts[0]);
            continue;
        }
        const sigInfo = { deviceId: keyIdParts[1] }; // XXX: is this how we're supposed to get the device ID?
@@ -371,7 +371,9 @@ Crypto.prototype.isKeyBackupTrusted = async function(backupInfo) {
            try {
                await olmlib.verifySignature(
                    this._olmDevice,
-                    backupInfo.auth_data,
+                    // verifySignature modifies the object so we need to copy
+                    // if we verify more than one sig
+                    Object.assign({}, backupInfo.auth_data),
                    this._userId,
                    device.deviceId,
                    device.getFingerprint(),
@@ -754,6 +756,132 @@ Crypto.prototype.setDeviceVerification = async function(
 };


+function verificationEventHandler(target, userId, roomId, eventId) {
+    return function(event) {
+        // listen for events related to this verification
+        if (event.getRoomId() !== roomId
+            || event.getSender() !== userId) {
+            return;
+        }
+        const content = event.getContent();
+        if (!content["m.relates_to"]) {
+            return;
+        }
+        const relatesTo
+              = content["m.relationship"] || content["m.relates_to"];
+        if (!relatesTo.rel_type
+            || relatesTo.rel_type !== "m.reference"
+            || !relatesTo.event_id
+            || relatesTo.event_id !== eventId) {
+            return;
+        }
+
+        // the event seems to be related to this verification, so pass it on to
+        // the verification handler
+        target.handleEvent(event);
+    };
+}
+
+Crypto.prototype.requestVerificationDM = async function(userId, roomId, methods) {
+    let methodMap;
+    if (methods) {
+        methodMap = new Map();
+        for (const method of methods) {
+            if (typeof method === "string") {
+                methodMap.set(method, defaultVerificationMethods[method]);
+            } else if (method.NAME) {
+                methodMap.set(method.NAME, method);
+            }
+        }
+    } else {
+        methodMap = this._baseApis._crypto._verificationMethods;
+    }
+
+    let eventId = undefined;
+    const listenPromise = new Promise((_resolve, _reject) => {
+        const listener = (event) => {
+            // listen for events related to this verification
+            if (event.getRoomId() !== roomId
+                || event.getSender() !== userId) {
+                return;
+            }
+            const relatesTo = event.getRelation();
+            if (!relatesTo || !relatesTo.rel_type
+                || relatesTo.rel_type !== "m.reference"
+                || !relatesTo.event_id
+                || relatesTo.event_id !== eventId) {
+                return;
+            }
+
+            const content = event.getContent();
+            // the event seems to be related to this verification
+            switch (event.getType()) {
+            case "m.key.verification.start": {
+                const verifier = new (methodMap.get(content.method))(
+                    this._baseApis, userId, content.from_device, eventId,
+                    roomId, event,
+                );
+                verifier.handler = verificationEventHandler(
+                    verifier, userId, roomId, eventId,
+                );
+                // this handler gets removed when the verification finishes
+                // (see the verify method of crypto/verification/Base.js)
+                this._baseApis.on("event", verifier.handler);
+                resolve(verifier);
+                break;
+            }
+            case "m.key.verification.cancel": {
+                reject(event);
+                break;
+            }
+            }
+        };
+        this._baseApis.on("event", listener);
+
+        const resolve = (...args) => {
+            this._baseApis.off("event", listener);
+            _resolve(...args);
+        };
+        const reject = (...args) => {
+            this._baseApis.off("event", listener);
+            _reject(...args);
+        };
+    });
+
+    const res = await this._baseApis.sendEvent(
+        roomId, "m.room.message",
+        {
+            body: this._baseApis.getUserId() + " is requesting to verify " +
+                "your key, but your client does not support in-chat key " +
+                "verification.  You will need to use legacy key " +
+                "verification to verify keys.",
+            msgtype: "m.key.verification.request",
+            to: userId,
+            from_device: this._baseApis.getDeviceId(),
+            methods: [...methodMap.keys()],
+        },
+    );
+    eventId = res.event_id;
+
+    return listenPromise;
+};
+
+Crypto.prototype.acceptVerificationDM = function(event, Method) {
+    if (typeof(Method) === "string") {
+        Method = defaultVerificationMethods[Method];
+    }
+    const content = event.getContent();
+    const verifier = new Method(
+        this._baseApis, event.getSender(), content.from_device, event.getId(),
+        event.getRoomId(),
+    );
+    verifier.handler = verificationEventHandler(
+        verifier, event.getSender(), event.getRoomId(), event.getId(),
+    );
+    this._baseApis.on("event", verifier.handler);
+    return verifier;
+};
+
 Crypto.prototype.requestVerification = function(userId, methods, devices) {
    if (!methods) {
        // .keys() returns an iterator, so we need to explicitly turn it into an array
@@ -801,20 +929,7 @@ Crypto.prototype.beginKeyVerification = function(
        this._verificationTransactions.set(userId, new Map());
    }
    transactionId = transactionId || randomString(32);
-    if (method instanceof Array) {
-        if (method.length !== 2
-            || !this._verificationMethods.has(method[0])
-            || !this._verificationMethods.has(method[1])) {
-            throw newUnknownMethodError();
-        }
-        /*
-        return new TwoPartVerification(
-            this._verificationMethods[method[0]],
-            this._verificationMethods[method[1]],
-            userId, deviceId, transactionId,
-        );
-        */
-    } else if (this._verificationMethods.has(method)) {
+    if (this._verificationMethods.has(method)) {
        const verifier = new (this._verificationMethods.get(method))(
            this._baseApis, userId, deviceId, transactionId,
        );
@@ -949,6 +1064,15 @@ Crypto.prototype.forceDiscardSession = function(roomId) {
 *   the device query is always inhibited as the members are not tracked.
 */
 Crypto.prototype.setRoomEncryption = async function(roomId, config, inhibitDeviceQuery) {
+    // ignore crypto events with no algorithm defined
+    // This will happen if a crypto event is redacted before we fetch the room state
+    // It would otherwise just throw later as an unknown algorithm would, but we may
+    // as well catch this here
+    if (!config.algorithm) {
+        console.log("Ignoring setRoomEncryption with no algorithm");
+        return;
+    }
+
    // if state is being replayed from storage, we might already have a configuration
    // for this room as they are persisted as well.
    // We just need to make sure the algorithm is initialized in this case.
@@ -1176,7 +1300,7 @@ Crypto.prototype.scheduleKeyBackupSend = async function(maxDelay = 10000) {
                numFailures = 0;
            } catch (err) {
                numFailures++;
-                console.log("Key backup request failed", err);
+                logger.log("Key backup request failed", err);
                if (err.data) {
                    if (
                        err.data.errcode == 'M_NOT_FOUND' ||
@@ -1286,6 +1410,18 @@ Crypto.prototype.backupGroupSession = async function(
 * upload in the background as soon as possible.
 */
 Crypto.prototype.scheduleAllGroupSessionsForBackup = async function() {
+    await this.flagAllGroupSessionsForBackup();
+
+    // Schedule keys to upload in the background as soon as possible.
+    this.scheduleKeyBackupSend(0 /* maxDelay */);
+};
+
+/**
+ * Marks all group sessions as needing to be backed up without scheduling
+ * them to upload in the background.
+ * @returns {Promise<int>} Resolves to the number of sessions requiring a backup.
+ */
+Crypto.prototype.flagAllGroupSessionsForBackup = async function() {
    await this._cryptoStore.doTxn(
        'readwrite',
        [
@@ -1303,9 +1439,7 @@ Crypto.prototype.scheduleAllGroupSessionsForBackup = async function() {

    const remaining = await this._cryptoStore.countSessionsNeedingBackup();
    this.emit("crypto.keyBackupSessionsRemaining", remaining);
-
-    // Schedule keys to upload in the background as soon as possible.
-    this.scheduleKeyBackupSend(0 /* maxDelay */);
+    return remaining;
 };

 /* eslint-disable valid-jsdoc */    //https://github.com/eslint/eslint/issues/7307
@@ -1646,6 +1780,11 @@ Crypto.prototype._onRoomKeyEvent = function(event) {
 * @param {module:models/event.MatrixEvent} event verification request event
 */
 Crypto.prototype._onKeyVerificationRequest = function(event) {
+    if (event.isCancelled()) {
+        logger.warn("Ignoring flagged verification request from " + event.getSender());
+        return;
+    }
+
    const content = event.getContent();
    if (!("from_device" in content) || typeof content.from_device !== "string"
        || !("transaction_id" in content) || typeof content.from_device !== "string"
@@ -1665,6 +1804,11 @@ Crypto.prototype._onKeyVerificationRequest = function(event) {
    }

    const sender = event.getSender();
+    if (sender === this._userId && content.from_device === this._deviceId) {
+        // ignore requests from ourselves, because it doesn't make sense for a
+        // device to verify itself
+        return;
+    }
    if (this._verificationTransactions.has(sender)) {
        if (this._verificationTransactions.get(sender).has(content.transaction_id)) {
            // transaction already exists: cancel it and drop the existing
@@ -1717,7 +1861,7 @@ Crypto.prototype._onKeyVerificationRequest = function(event) {
            },
        );
    } else {
-        // notify the application that of the verification request, so it can
+        // notify the application of the verification request, so it can
        // decide what to do with it
        const request = {
            event: event,
@@ -1757,6 +1901,11 @@ Crypto.prototype._onKeyVerificationRequest = function(event) {
 * @param {module:models/event.MatrixEvent} event verification start event
 */
 Crypto.prototype._onKeyVerificationStart = function(event) {
+    if (event.isCancelled()) {
+        logger.warn("Ignoring flagged verification start from " + event.getSender());
+        return;
+    }
+
    const sender = event.getSender();
    const content = event.getContent();
    const transactionId = content.transaction_id;
@@ -1790,22 +1939,6 @@ Crypto.prototype._onKeyVerificationStart = function(event) {
            transaction_id: content.transactionId,
        }));
        return;
-    } else if (content.next_method) {
-        if (!this._verificationMethods.has(content.next_method)) {
-            cancel(newUnknownMethodError({
-                transaction_id: content.transactionId,
-            }));
-            return;
-        } else {
-            /* TODO:
-            const verification = new TwoPartVerification(
-                this._verificationMethods[content.method],
-                this._verificationMethods[content.next_method],
-                userId, deviceId,
-            );
-            this.emit(verification.event_type, verification);
-            this.emit(verification.first.event_type, verification);*/
-        }
    } else {
        const verifier = new (this._verificationMethods.get(content.method))(
            this._baseApis, sender, deviceId, content.transaction_id,
@@ -1860,8 +1993,6 @@ Crypto.prototype._onKeyVerificationStart = function(event) {

                    handler.request.resolve(verifier);
                }
-            } else {
-                // FIXME: make sure we're in a two-part verification, and the start matches the second part
            }
        }
        this._baseApis.emit("crypto.verification.start", verifier);
@@ -1882,7 +2013,7 @@ Crypto.prototype._onKeyVerificationMessage = function(event) {
    if (!handler) {
        return;
    } else if (event.getType() === "m.key.verification.cancel") {
-        console.log(event);
+        logger.log(event);
        if (handler.verifier) {
            handler.verifier.cancel(event);
        } else if (handler.request && handler.request.cancel) {
@@ -24,7 +24,7 @@ limitations under the License.
 import Promise from 'bluebird';
 const anotherjson = require('another-json');

-const logger = require("../logger");
+import logger from '../logger';
 const utils = require("../utils");

 /**
@@ -694,7 +694,7 @@ function promiseifyTxn(txn) {
            if (txn._mx_abortexception !== undefined) {
                reject(txn._mx_abortexception);
            } else {
-                console.log("Error performing indexeddb txn", event);
+                logger.log("Error performing indexeddb txn", event);
                reject(event.target.error);
            }
        };
@@ -702,7 +702,7 @@ function promiseifyTxn(txn) {
            if (txn._mx_abortexception !== undefined) {
                reject(txn._mx_abortexception);
            } else {
-                console.log("Error performing indexeddb txn", event);
+                logger.log("Error performing indexeddb txn", event);
                reject(event.target.error);
            }
        };
@@ -90,7 +90,7 @@ export default class IndexedDBCryptoStore {
            };

            req.onerror = (ev) => {
-                console.log("Error connecting to indexeddb", ev);
+                logger.log("Error connecting to indexeddb", ev);
                reject(ev.target.error);
            };

@@ -160,7 +160,7 @@ export default class IndexedDBCryptoStore {
            };

            req.onerror = (ev) => {
-                console.log("Error deleting data from indexeddb", ev);
+                logger.log("Error deleting data from indexeddb", ev);
                reject(ev.target.error);
            };

@@ -17,7 +17,7 @@ limitations under the License.
 import Promise from 'bluebird';

 import logger from '../../logger';
-import MemoryCryptoStore from './memory-crypto-store.js';
+import MemoryCryptoStore from './memory-crypto-store';

 /**
 * Internal module. Partial localStorage backed storage for e2e.
@@ -21,6 +21,10 @@ limitations under the License.

 import {MatrixEvent} from '../../models/event';
 import {EventEmitter} from 'events';
+import logger from '../../logger';
+import {newTimeoutError} from "./Error";
+
+const timeoutException = new Error("Verification timed out");

 export default class VerificationBase extends EventEmitter {
    /**
@@ -43,27 +47,66 @@ export default class VerificationBase extends EventEmitter {
     *
     * @param {string} transactionId the transaction ID to be used when sending events
     *
-     * @param {object} startEvent the m.key.verification.start event that
+     * @param {string} [roomId] the room to use for verification
+     *
+     * @param {object} [startEvent] the m.key.verification.start event that
     * initiated this verification, if any
     *
-     * @param {object} request the key verification request object related to
+     * @param {object} [request] the key verification request object related to
     * this verification, if any
-     *
-     * @param {object} parent parent verification for this verification, if any
     */
-    constructor(baseApis, userId, deviceId, transactionId, startEvent, request, parent) {
+    constructor(baseApis, userId, deviceId, transactionId, roomId, startEvent, request) {
        super();
        this._baseApis = baseApis;
        this.userId = userId;
        this.deviceId = deviceId;
        this.transactionId = transactionId;
-        this.startEvent = startEvent;
-        this.request = request;
-        this._parent = parent;
+        if (typeof(roomId) === "string" || roomId instanceof String) {
+            this.roomId = roomId;
+            this.startEvent = startEvent;
+            this.request = request;
+        } else {
+            // if room ID was omitted, but start event and request were not
+            this.startEvent= roomId;
+            this.request = startEvent;
+        }
+        this.cancelled = false;
        this._done = false;
        this._promise = null;
+        this._transactionTimeoutTimer = null;
+
+        // At this point, the verification request was received so start the timeout timer.
+        this._resetTimer();
+
+        if (this.roomId) {
+            this._send = this._sendMessage;
+        } else {
+            this._send = this._sendToDevice;
+        }
    }

+    _resetTimer() {
+        logger.info("Refreshing/starting the verification transaction timeout timer");
+        if (this._transactionTimeoutTimer !== null) {
+            clearTimeout(this._transactionTimeoutTimer);
+        }
+        this._transactionTimeoutTimer = setTimeout(() => {
+            if (!this._done && !this.cancelled) {
+                logger.info("Triggering verification timeout");
+                this.cancel(timeoutException);
+            }
+        }, 10 * 60 * 1000); // 10 minutes
+    }
+
+    _endTimer() {
+        if (this._transactionTimeoutTimer !== null) {
+            clearTimeout(this._transactionTimeoutTimer);
+            this._transactionTimeoutTimer = null;
+        }
+    }
+
+    /* send a message to the other participant, using to-device messages
+     */
    _sendToDevice(type, content) {
        if (this._done) {
            return Promise.reject(new Error("Verification is already done"));
@@ -74,6 +117,21 @@ export default class VerificationBase extends EventEmitter {
        });
    }

+    /* send a message to the other participant, using in-roomm messages
+     */
+    _sendMessage(type, content) {
+        if (this._done) {
+            return Promise.reject(new Error("Verification is already done"));
+        }
+        // FIXME: if MSC1849 decides to use m.relationship instead of
+        // m.relates_to, we should follow suit here
+        content["m.relates_to"] = {
+            rel_type: "m.reference",
+            event_id: this.transactionId,
+        };
+        return this._baseApis.sendEvent(this.roomId, type, content);
+    }
+
    _waitForEvent(type) {
        if (this._done) {
            return Promise.reject(new Error("Verification is already done"));
@@ -91,6 +149,7 @@ export default class VerificationBase extends EventEmitter {
        } else if (e.getType() === this._expectedEvent) {
            this._expectedEvent = undefined;
            this._rejectEvent = undefined;
+            this._resetTimer();
            this._resolveEvent(e);
        } else {
            this._expectedEvent = undefined;
@@ -108,17 +167,27 @@ export default class VerificationBase extends EventEmitter {
    }

    done() {
+        this._endTimer(); // always kill the activity timer
        if (!this._done) {
+            if (this.roomId) {
+                // verification in DM requires a done message
+                this._send("m.key.verification.done", {});
+            }
            this._resolve();
        }
    }

    cancel(e) {
+        this._endTimer(); // always kill the activity timer
        if (!this._done) {
+            this.cancelled = true;
            if (this.userId && this.deviceId && this.transactionId) {
                // send a cancellation to the other user (if it wasn't
                // cancelled by the other user)
-                if (e instanceof MatrixEvent) {
+                if (e === timeoutException) {
+                    const timeoutEvent = newTimeoutError();
+                    this._send(timeoutEvent.getType(), timeoutEvent.getContent());
+                } else if (e instanceof MatrixEvent) {
                    const sender = e.getSender();
                    if (sender !== this.userId) {
                        const content = e.getContent();
@@ -127,9 +196,9 @@ export default class VerificationBase extends EventEmitter {
                            content.reason = content.reason || content.body
                                || "Unknown reason";
                            content.transaction_id = this.transactionId;
-                            this._sendToDevice("m.key.verification.cancel", content);
+                            this._send("m.key.verification.cancel", content);
                        } else {
-                            this._sendToDevice("m.key.verification.cancel", {
+                            this._send("m.key.verification.cancel", {
                                code: "m.unknown",
                                reason: content.body || "Unknown reason",
                                transaction_id: this.transactionId,
@@ -137,7 +206,7 @@ export default class VerificationBase extends EventEmitter {
                        }
                    }
                } else {
-                    this._sendToDevice("m.key.verification.cancel", {
+                    this._send("m.key.verification.cancel", {
                        code: "m.unknown",
                        reason: e.toString(),
                        transaction_id: this.transactionId,
@@ -145,7 +214,9 @@ export default class VerificationBase extends EventEmitter {
                }
            }
            if (this._promise !== null) {
-                this._reject(e);
+                // when we cancel without a promise, we end up with a promise
+                // but no reject function. If cancel is called again, we'd error.
+                if (this._reject) this._reject(e);
            } else {
                this._promise = Promise.reject(e);
            }
@@ -167,15 +238,24 @@ export default class VerificationBase extends EventEmitter {
        this._promise = new Promise((resolve, reject) => {
            this._resolve = (...args) => {
                this._done = true;
+                this._endTimer();
+                if (this.handler) {
+                    this._baseApis.off("event", this.handler);
+                }
                resolve(...args);
            };
            this._reject = (...args) => {
                this._done = true;
+                this._endTimer();
+                if (this.handler) {
+                    this._baseApis.off("event", this.handler);
+                }
                reject(...args);
            };
        });
        if (this._doVerification && !this._started) {
            this._started = true;
+            this._resetTimer(); // restart the timeout
            Promise.resolve(this._doVerification())
                .then(this.done.bind(this), this.cancel.bind(this));
        }
@@ -183,17 +263,29 @@ export default class VerificationBase extends EventEmitter {
    }

    async _verifyKeys(userId, keys, verifier) {
+        // we try to verify all the keys that we're told about, but we might
+        // not know about all of them, so keep track of the keys that we know
+        // about, and ignore the rest
+        const verifiedDevices = [];
+
        for (const [keyId, keyInfo] of Object.entries(keys)) {
            const deviceId = keyId.split(':', 2)[1];
            const device = await this._baseApis.getStoredDevice(userId, deviceId);
            if (!device) {
-                throw new Error(`Could not find device ${deviceId}`);
+                logger.warn(`verification: Could not find device ${deviceId} to verify`);
            } else {
                await verifier(keyId, device, keyInfo);
+                verifiedDevices.push(deviceId);
            }
        }
-        for (const keyId of Object.keys(keys)) {
-            const deviceId = keyId.split(':', 2)[1];
+
+        // if none of the keys could be verified, then error because the app
+        // should be informed about that
+        if (!verifiedDevices.length) {
+            throw new Error("No devices could be verified");
+        }
+
+        for (const deviceId of verifiedDevices) {
            await this._baseApis.setDeviceVerified(userId, deviceId);
        }
    }
@@ -213,9 +213,10 @@ export default class SAS extends Base {
            message_authentication_codes: MAC_LIST,
            // FIXME: allow app to specify what SAS methods can be used
            short_authentication_string: SAS_LIST,
-            transaction_id: this.transactionId,
        };
-        this._sendToDevice("m.key.verification.start", initialMessage);
+        // NOTE: this._send will modify initialMessage to include the
+        // transaction_id field, or the m.relationship/m.relates_to field
+        this._send("m.key.verification.start", initialMessage);


        let e = await this._waitForEvent("m.key.verification.accept");
@@ -235,7 +236,7 @@ export default class SAS extends Base {
        const hashCommitment = content.commitment;
        const olmSAS = new global.Olm.SAS();
        try {
-            this._sendToDevice("m.key.verification.key", {
+            this._send("m.key.verification.key", {
                key: olmSAS.get_pubkey(),
            });

@@ -306,7 +307,7 @@ export default class SAS extends Base {
        const olmSAS = new global.Olm.SAS();
        try {
            const commitmentStr = olmSAS.get_pubkey() + anotherjson.stringify(content);
-            this._sendToDevice("m.key.verification.accept", {
+            this._send("m.key.verification.accept", {
                key_agreement_protocol: keyAgreement,
                hash: hashMethod,
                message_authentication_code: macMethod,
@@ -320,7 +321,7 @@ export default class SAS extends Base {
            // FIXME: make sure event is properly formed
            content = e.getContent();
            olmSAS.set_their_key(content.key);
-            this._sendToDevice("m.key.verification.key", {
+            this._send("m.key.verification.key", {
                key: olmSAS.get_pubkey(),
            });

@@ -369,7 +370,7 @@ export default class SAS extends Base {
            keyId,
            baseInfo + "KEY_IDS",
        );
-        this._sendToDevice("m.key.verification.mac", { mac, keys });
+        this._send("m.key.verification.mac", { mac, keys });
    }

    async _checkMAC(olmSAS, content, method) {
@@ -1,5 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2019 The Matrix.org Foundation C.I.C.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -22,6 +23,7 @@ import Promise from 'bluebird';
 const parseContentType = require('content-type').parse;

 const utils = require("./utils");
+import logger from './logger';

 // we use our own implementation of setTimeout, so that if we get suspended in
 // the middle of a /sync, we cancel the sync as soon as we awake, rather than
@@ -45,10 +47,15 @@ module.exports.PREFIX_R0 = "/_matrix/client/r0";
 module.exports.PREFIX_UNSTABLE = "/_matrix/client/unstable";

 /**
- * URI path for the identity API
+ * URI path for v1 of the the identity API
 */
 module.exports.PREFIX_IDENTITY_V1 = "/_matrix/identity/api/v1";

+/**
+ * URI path for the v2 identity API
+ */
+module.exports.PREFIX_IDENTITY_V2 = "/_matrix/identity/v2";
+
 /**
 * URI path for the media repo API
 */
@@ -89,6 +96,13 @@ module.exports.MatrixHttpApi = function MatrixHttpApi(event_emitter, opts) {
 };

 module.exports.MatrixHttpApi.prototype = {
+    /**
+     * Sets the baase URL for the identity server
+     * @param {string} url The new base url
+     */
+    setIdBaseUrl: function(url) {
+        this.opts.idBaseUrl = url;
+    },

    /**
     * Get the content repository url with query parameters.
@@ -101,7 +115,7 @@ module.exports.MatrixHttpApi.prototype = {
        };
        return {
            base: this.opts.baseUrl,
-            path: "/_matrix/media/v1/upload",
+            path: "/_matrix/media/r0/upload",
            params: params,
        };
    },
@@ -164,9 +178,21 @@ module.exports.MatrixHttpApi.prototype = {
        const contentType = opts.type || file.type || 'application/octet-stream';
        const fileName = opts.name || file.name;

-        // we used to recommend setting file.stream to the thing to upload on
-        // nodejs.
-        const body = file.stream ? file.stream : file;
+        // We used to recommend setting file.stream to the thing to upload on
+        // Node.js. As of 2019-06-11, this is still in widespread use in various
+        // clients, so we should preserve this for simple objects used in
+        // Node.js. File API objects (via either the File or Blob interfaces) in
+        // the browser now define a `stream` method, which leads to trouble
+        // here, so we also check the type of `stream`.
+        let body = file;
+        if (body.stream && typeof body.stream !== "function") {
+            logger.warn(
+                "Using `file.stream` as the content to upload. Future " +
+                "versions of the js-sdk will change this to expect `file` to " +
+                "be the content directly.",
+            );
+            body = body.stream;
+        }

        // backwards-compatibility hacks where we used to do different things
        // between browser and node.
@@ -175,7 +201,7 @@ module.exports.MatrixHttpApi.prototype = {
            if (global.XMLHttpRequest) {
                rawResponse = false;
            } else {
-                console.warn(
+                logger.warn(
                    "Returning the raw JSON from uploadContent(). Future " +
                    "versions of the js-sdk will change this default, to " +
                    "return the parsed object. Set opts.rawResponse=false " +
@@ -188,7 +214,7 @@ module.exports.MatrixHttpApi.prototype = {
        let onlyContentUri = opts.onlyContentUri;
        if (!rawResponse && onlyContentUri === undefined) {
            if (global.XMLHttpRequest) {
-                console.warn(
+                logger.warn(
                    "Returning only the content-uri from uploadContent(). " +
                    "Future versions of the js-sdk will change this " +
                    "default, to return the whole response object. Set " +
@@ -278,7 +304,7 @@ module.exports.MatrixHttpApi.prototype = {
                    });
                }
            });
-            let url = this.opts.baseUrl + "/_matrix/media/v1/upload";
+            let url = this.opts.baseUrl + "/_matrix/media/r0/upload";

            const queryArgs = [];

@@ -314,7 +340,7 @@ module.exports.MatrixHttpApi.prototype = {

            promise = this.authedRequest(
                opts.callback, "POST", "/upload", queryParams, body, {
-                    prefix: "/_matrix/media/v1",
+                    prefix: "/_matrix/media/r0",
                    headers: {"Content-Type": contentType},
                    json: false,
                    bodyParser: bodyParser,
@@ -355,7 +381,18 @@ module.exports.MatrixHttpApi.prototype = {
        return this.uploads;
    },

-    idServerRequest: function(callback, method, path, params, prefix) {
+    idServerRequest: function(
+        callback,
+        method,
+        path,
+        params,
+        prefix,
+        accessToken,
+    ) {
+        if (!this.opts.idBaseUrl) {
+            throw new Error("No Identity Server base URL set");
+        }
+
        const fullUri = this.opts.idBaseUrl + prefix + path;

        if (callback !== undefined && !utils.isFunction(callback)) {
@@ -368,13 +405,17 @@ module.exports.MatrixHttpApi.prototype = {
            uri: fullUri,
            method: method,
            withCredentials: false,
-            json: false,
+            json: true, // we want a JSON response if we can
            _matrix_opts: this.opts,
+            headers: {},
        };
-        if (method == 'GET') {
+        if (method === 'GET') {
            opts.qs = params;
-        } else {
-            opts.form = params;
+        } else if (typeof params === "object") {
+            opts.json = params;
+        }
+        if (accessToken) {
+            opts.headers['Authorization'] = `Bearer ${accessToken}`;
        }

        const defer = Promise.defer();
@@ -382,12 +423,7 @@ module.exports.MatrixHttpApi.prototype = {
            opts,
            requestCallback(defer, callback, this.opts.onlyData),
        );
-        // ID server does not always take JSON, so we can't use requests' 'json'
-        // option as we do with the home server, but it does return JSON, so
-        // parse it manually
-        return defer.promise.then(function(response) {
-            return JSON.parse(response);
-        });
+        return defer.promise;
    },

    /**
@@ -457,7 +493,7 @@ module.exports.MatrixHttpApi.prototype = {
        const self = this;
        requestPromise.catch(function(err) {
            if (err.errcode == 'M_UNKNOWN_TOKEN') {
-                self.event_emitter.emit("Session.logged_out");
+                self.event_emitter.emit("Session.logged_out", err);
            } else if (err.errcode == 'M_CONSENT_NOT_GIVEN') {
                self.event_emitter.emit(
                    "no_consent",
@@ -512,76 +548,6 @@ module.exports.MatrixHttpApi.prototype = {
        );
    },

-    /**
-     * Perform an authorised request to the homeserver with a specific path
-     * prefix which overrides the default for this call only. Useful for hitting
-     * different Matrix Client-Server versions.
-     * @param {Function} callback Optional. The callback to invoke on
-     * success/failure. See the promise return values for more information.
-     * @param {string} method The HTTP method e.g. "GET".
-     * @param {string} path The HTTP path <b>after</b> the supplied prefix e.g.
-     * "/createRoom".
-     * @param {Object} queryParams A dict of query params (these will NOT be
-     * urlencoded).
-     * @param {Object} data The HTTP JSON body.
-     * @param {string} prefix The full prefix to use e.g.
-     * "/_matrix/client/v2_alpha".
-     * @param {Number=} localTimeoutMs The maximum amount of time to wait before
-     * timing out the request. If not specified, there is no timeout.
-     * @return {module:client.Promise} Resolves to <code>{data: {Object},
-     * headers: {Object}, code: {Number}}</code>.
-     * If <code>onlyData</code> is set, this will resolve to the <code>data</code>
-     * object only.
-     * @return {module:http-api.MatrixError} Rejects with an error if a problem
-     * occurred. This includes network problems and Matrix-specific error JSON.
-     *
-     * @deprecated prefer authedRequest with opts.prefix
-     */
-    authedRequestWithPrefix: function(callback, method, path, queryParams, data,
-                                      prefix, localTimeoutMs) {
-        return this.authedRequest(
-            callback, method, path, queryParams, data, {
-                localTimeoutMs: localTimeoutMs,
-                prefix: prefix,
-            },
-        );
-    },
-
-    /**
-     * Perform a request to the homeserver without any credentials but with a
-     * specific path prefix which overrides the default for this call only.
-     * Useful for hitting different Matrix Client-Server versions.
-     * @param {Function} callback Optional. The callback to invoke on
-     * success/failure. See the promise return values for more information.
-     * @param {string} method The HTTP method e.g. "GET".
-     * @param {string} path The HTTP path <b>after</b> the supplied prefix e.g.
-     * "/createRoom".
-     * @param {Object} queryParams A dict of query params (these will NOT be
-     * urlencoded).
-     * @param {Object} data The HTTP JSON body.
-     * @param {string} prefix The full prefix to use e.g.
-     * "/_matrix/client/v2_alpha".
-     * @param {Number=} localTimeoutMs The maximum amount of time to wait before
-     * timing out the request. If not specified, there is no timeout.
-     * @return {module:client.Promise} Resolves to <code>{data: {Object},
-     * headers: {Object}, code: {Number}}</code>.
-     * If <code>onlyData</code> is set, this will resolve to the <code>data</code>
-     * object only.
-     * @return {module:http-api.MatrixError} Rejects with an error if a problem
-     * occurred. This includes network problems and Matrix-specific error JSON.
-     *
-     * @deprecated prefer request with opts.prefix
-     */
-    requestWithPrefix: function(callback, method, path, queryParams, data, prefix,
-                                localTimeoutMs) {
-        return this.request(
-            callback, method, path, queryParams, data, {
-                localTimeoutMs: localTimeoutMs,
-                prefix: prefix,
-            },
-        );
-    },
-
    /**
     * Perform a request to an arbitrary URL.
     * @param {Function} callback Optional. The callback to invoke on
@@ -870,7 +836,8 @@ function parseErrorResponse(response, body) {
    let err;
    if (contentType) {
        if (contentType.type === 'application/json') {
-            err = new module.exports.MatrixError(JSON.parse(body));
+            const jsonBody = typeof(body) === 'object' ? body : JSON.parse(body);
+            err = new module.exports.MatrixError(jsonBody);
        } else if (contentType.type === 'text/plain') {
            err = new Error(`Server returned ${httpStatus} error: ${body}`);
        }
@@ -22,6 +22,7 @@ import Promise from 'bluebird';
 const url = require("url");

 const utils = require("./utils");
+import logger from './logger';

 const EMAIL_STAGE_TYPE = "m.login.email.identity";
 const MSISDN_STAGE_TYPE = "m.login.msisdn";
@@ -48,11 +49,18 @@ const MSISDN_STAGE_TYPE = "m.login.msisdn";
 * @param {object?} opts.authData error response from the last request. If
 *    null, a request will be made with no auth before starting.
 *
- * @param {function(object?, bool?): module:client.Promise} opts.doRequest
- *     called with the new auth dict to submit the request and a flag set
- *     to true if this request is a background request. Should return a
- *     promise which resolves to the successful response or rejects with a
- *     MatrixError.
+ * @param {function(object?): module:client.Promise} opts.doRequest
+ *     called with the new auth dict to submit the request. Also passes a
+ *     second deprecated arg which is a flag set to true if this request
+ *     is a background request. The busyChanged callback should be used
+ *     instead of the backfround flag. Should return a promise which resolves
+ *     to the successful response or rejects with a MatrixError.
+ *
+ * @param {function(bool): module:client.Promise} opts.busyChanged
+ *     called whenever the interactive auth logic becomes busy submitting
+ *     information provided by the user or finsihes. After this has been
+ *     called with true the UI should indicate that a request is in progress
+ *     until it is called again with false.
 *
 * @param {function(string, object?)} opts.stateUpdated
 *     called when the status of the UI auth changes, ie. when the state of
@@ -100,6 +108,7 @@ function InteractiveAuth(opts) {
    this._matrixClient = opts.matrixClient;
    this._data = opts.authData || {};
    this._requestCallback = opts.doRequest;
+    this._busyChangedCallback = opts.busyChanged;
    // startAuthStage included for backwards compat
    this._stateUpdatedCallback = opts.stateUpdated || opts.startAuthStage;
    this._resolveFunc = null;
@@ -111,9 +120,14 @@ function InteractiveAuth(opts) {
    this._clientSecret = opts.clientSecret || this._matrixClient.generateClientSecret();
    this._emailSid = opts.emailSid;
    if (this._emailSid === undefined) this._emailSid = null;
+    this._requestingEmailToken = false;

    this._chosenFlow = null;
    this._currentStage = null;
+
+    // if we are currently trying to submit an auth dict (which includes polling)
+    // the promise the will resolve/reject when it completes
+    this._submitPromise = null;
 }

 InteractiveAuth.prototype = {
@@ -134,7 +148,10 @@ InteractiveAuth.prototype = {
            // if we have no flows, try a request (we'll have
            // just a session ID in _data if resuming)
            if (!this._data.flows) {
-                this._doRequest(this._data);
+                if (this._busyChangedCallback) this._busyChangedCallback(true);
+                this._doRequest(this._data).finally(() => {
+                    if (this._busyChangedCallback) this._busyChangedCallback(false);
+                });
            } else {
                this._startNextAuthStage();
            }
@@ -146,24 +163,30 @@ InteractiveAuth.prototype = {
     * completed out-of-band. If so, the attemptAuth promise will
     * be resolved.
     */
-    poll: function() {
+    poll: async function() {
        if (!this._data.session) return;
+        // if we currently have a request in flight, there's no point making
+        // another just to check what the status is
+        if (this._submitPromise) return;

        let authDict = {};
        if (this._currentStage == EMAIL_STAGE_TYPE) {
            // The email can be validated out-of-band, but we need to provide the
            // creds so the HS can go & check it.
            if (this._emailSid) {
-                const idServerParsedUrl = url.parse(
-                    this._matrixClient.getIdentityServerUrl(),
-                );
+                const creds = {
+                    sid: this._emailSid,
+                    client_secret: this._clientSecret,
+                };
+                if (await this._matrixClient.doesServerRequireIdServerParam()) {
+                    const idServerParsedUrl = url.parse(
+                        this._matrixClient.getIdentityServerUrl(),
+                    );
+                    creds.id_server = idServerParsedUrl.host;
+                }
                authDict = {
                    type: EMAIL_STAGE_TYPE,
-                    threepid_creds: {
-                        sid: this._emailSid,
-                        client_secret: this._clientSecret,
-                        id_server: idServerParsedUrl.host,
-                    },
+                    threepid_creds: creds,
                };
            }
        }
@@ -220,18 +243,44 @@ InteractiveAuth.prototype = {
     *    in the attemptAuth promise being rejected. This can be set to true
     *    for requests that just poll to see if auth has been completed elsewhere.
     */
-    submitAuthDict: function(authData, background) {
+    submitAuthDict: async function(authData, background) {
        if (!this._resolveFunc) {
            throw new Error("submitAuthDict() called before attemptAuth()");
        }

+        if (!background && this._busyChangedCallback) {
+            this._busyChangedCallback(true);
+        }
+
+        // if we're currently trying a request, wait for it to finish
+        // as otherwise we can get multiple 200 responses which can mean
+        // things like multiple logins for register requests.
+        // (but discard any expections as we only care when its done,
+        // not whether it worked or not)
+        while (this._submitPromise) {
+            try {
+                await this._submitPromise;
+            } catch (e) {
+            }
+        }
+
        // use the sessionid from the last request.
        const auth = {
            session: this._data.session,
        };
        utils.extend(auth, authData);

-        this._doRequest(auth, background);
+        try {
+            // NB. the 'background' flag is deprecated by the busyChanged
+            // callback and is here for backwards compat
+            this._submitPromise = this._doRequest(auth, background);
+            await this._submitPromise;
+        } finally {
+            this._submitPromise = null;
+            if (!background && this._busyChangedCallback) {
+                this._busyChangedCallback(false);
+            }
+        }
    },

    /**
@@ -283,7 +332,7 @@ InteractiveAuth.prototype = {
                    // We ignore all failures here (even non-UI auth related ones)
                    // since we don't want to suddenly fail if the internet connection
                    // had a blip whilst we were polling
-                    console.log(
+                    logger.log(
                        "Background poll request failed doing UI auth: ignoring",
                        error,
                    );
@@ -304,12 +353,14 @@ InteractiveAuth.prototype = {

            if (
                !this._emailSid &&
+                !this._requestingEmailToken &&
                this._chosenFlow.stages.includes('m.login.email.identity')
            ) {
                // If we've picked a flow with email auth, we send the email
                // now because we want the request to fail as soon as possible
                // if the email address is not valid (ie. already taken or not
                // registered, depending on what the operation is).
+                this._requestingEmailToken = true;
                try {
                    const requestTokenResult = await this._requestEmailTokenCallback(
                        this._inputs.emailAddress,
@@ -332,6 +383,8 @@ InteractiveAuth.prototype = {
                    // the failure up as the user can't complete auth if we can't
                    // send the email, foe whatever reason.
                    this._rejectFunc(e);
+                } finally {
+                    this._requestingEmailToken = false;
                }
            }
        }
@@ -383,9 +436,9 @@ InteractiveAuth.prototype = {
        if (this._chosenFlow === null) {
            this._chosenFlow = this._chooseFlow();
        }
-        console.log("Active flow => %s", JSON.stringify(this._chosenFlow));
+        logger.log("Active flow => %s", JSON.stringify(this._chosenFlow));
        const nextStage = this._firstUncompletedStage(this._chosenFlow);
-        console.log("Next stage: %s", nextStage);
+        logger.log("Next stage: %s", nextStage);
        return nextStage;
    },

@@ -1,6 +1,7 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2017 Vector Creations Ltd
+Copyright 2019 The Matrix.org Foundation C.I.C.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -76,6 +77,7 @@ module.exports.InteractiveAuth = require("./interactive-auth");
 /** The {@link module:auto-discovery|AutoDiscovery} class. */
 module.exports.AutoDiscovery = require("./autodiscovery").AutoDiscovery;

+module.exports.SERVICE_TYPES = require('./service-types').SERVICE_TYPES;

 module.exports.MemoryCryptoStore =
    require("./crypto/store/memory-crypto-store").default;
@@ -20,6 +20,8 @@ limitations under the License.
 const EventEmitter = require("events").EventEmitter;
 const utils = require("../utils");
 const EventTimeline = require("./event-timeline");
+import {EventStatus} from "./event";
+import logger from '../logger';
 import Relations from './relations';

 // var DEBUG = false;
@@ -28,7 +30,7 @@ const DEBUG = true;
 let debuglog;
 if (DEBUG) {
    // using bind means that we get to keep useful line numbers in the console
-    debuglog = console.log.bind(console);
+    debuglog = logger.log.bind(logger);
 } else {
    debuglog = function() {};
 }
@@ -90,6 +92,13 @@ function EventTimelineSet(room, opts) {
 }
 utils.inherits(EventTimelineSet, EventEmitter);

+/**
+ * Get all the timelines in this set
+ * @return {module:models/event-timeline~EventTimeline[]} the timelines in this set
+ */
+EventTimelineSet.prototype.getTimelines = function() {
+    return this._timelines;
+};
 /**
 * Get the filter object this timeline set is filtered on, if any
 * @return {?Filter} the optional filter for this timelineSet
@@ -422,7 +431,7 @@ EventTimelineSet.prototype.addEventsToTimeline = function(events, toStartOfTimel
        }

        // time to join the timelines.
-        console.info("Already have timeline for " + eventId +
+        logger.info("Already have timeline for " + eventId +
                     " - joining timeline " + timeline + " to " +
                     existingTimeline);

@@ -436,15 +445,14 @@ EventTimelineSet.prototype.addEventsToTimeline = function(events, toStartOfTimel
        if (backwardsIsLive || forwardsIsLive) {
            // The live timeline should never be spliced into a non-live position.
            // We use independent logging to better discover the problem at a glance.
-            console.warn({backwardsIsLive, forwardsIsLive}); // debugging
            if (backwardsIsLive) {
-                console.warn(
+                logger.warn(
                    "Refusing to set a preceding existingTimeLine on our " +
                    "timeline as the existingTimeLine is live (" + existingTimeline + ")",
                );
            }
            if (forwardsIsLive) {
-                console.warn(
+                logger.warn(
                    "Refusing to set our preceding timeline on a existingTimeLine " +
                    "as our timeline is live (" + timeline + ")",
                );
@@ -464,8 +472,8 @@ EventTimelineSet.prototype.addEventsToTimeline = function(events, toStartOfTimel
    // timeline we ended up on.
    if (lastEventWasNew || !didUpdate) {
        if (direction === EventTimeline.FORWARDS && timeline === this._liveTimeline) {
-            console.warn({lastEventWasNew, didUpdate}); // for debugging
-            console.warn(
+            logger.warn({lastEventWasNew, didUpdate}); // for debugging
+            logger.warn(
                `Refusing to set forwards pagination token of live timeline ` +
                `${timeline} to ${paginationToken}`,
            );
@@ -687,9 +695,12 @@ EventTimelineSet.prototype.compareEventOrdering = function(eventId1, eventId2) {
 * The type of relation involved, such as "m.annotation", "m.reference", "m.replace", etc.
 * @param {String} eventType
 * The relation event's type, such as "m.reaction", etc.
+ * @throws If <code>eventId</code>, <code>relationType</code> or <code>eventType</code>
+ * are not valid.
 *
- * @returns {Relations}
- * A container for relation events.
+ * @returns {?Relations}
+ * A container for relation events or undefined if there are no relation events for
+ * the relationType.
 */
 EventTimelineSet.prototype.getRelationsForEvent = function(
    eventId, relationType, eventType,
@@ -748,6 +759,10 @@ EventTimelineSet.prototype.aggregateRelations = function(event) {
        return;
    }

+    if (event.isRedacted() || event.status === EventStatus.CANCELLED) {
+        return;
+    }
+
    // If the event is currently encrypted, wait until it has been decrypted.
    if (event.isBeingDecrypted()) {
        event.once("Event.decrypted", () => {
@@ -24,13 +24,14 @@ limitations under the License.
 import Promise from 'bluebird';
 import {EventEmitter} from 'events';
 import utils from '../utils.js';
+import logger from '../logger';

 /**
 * Enum for event statuses.
 * @readonly
 * @enum {string}
 */
-module.exports.EventStatus = {
+const EventStatus = {
    /** The event was not sent and will no longer be retried. */
    NOT_SENT: "not_sent",

@@ -48,6 +49,7 @@ module.exports.EventStatus = {
    /** The event was cancelled before it was successfully sent. */
    CANCELLED: "cancelled",
 };
+module.exports.EventStatus = EventStatus;

 const interns = {};
 function intern(str) {
@@ -123,6 +125,8 @@ module.exports.MatrixEvent = function MatrixEvent(
    this.forwardLooking = true;
    this._pushActions = null;
    this._replacingEvent = null;
+    this._localRedactionEvent = null;
+    this._isCancelled = false;

    this._clearEvent = {};

@@ -227,6 +231,9 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @return {Object} The event content JSON, or an empty object.
     */
    getOriginalContent: function() {
+        if (this._localRedactionEvent) {
+            return {};
+        }
        return this._clearEvent.content || this.event.content || {};
    },

@@ -238,7 +245,9 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @return {Object} The event content JSON, or an empty object.
     */
    getContent: function() {
-        if (this._replacingEvent) {
+        if (this._localRedactionEvent) {
+            return {};
+        } else if (this._replacingEvent) {
            return this._replacingEvent.getContent()["m.new_content"] || {};
        } else {
            return this.getOriginalContent();
@@ -395,7 +404,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        // new info.
        //
        if (this._decryptionPromise) {
-            console.log(
+            logger.log(
                `Event ${this.getId()} already being decrypted; queueing a retry`,
            );
            this._retryDecryption = true;
@@ -469,7 +478,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
                if (e.name !== "DecryptionError") {
                    // not a decryption error: log the whole exception as an error
                    // (and don't bother with a retry)
-                    console.error(
+                    logger.error(
                        `Error decrypting event (id=${this.getId()}): ${e.stack || e}`,
                    );
                    this._decryptionPromise = null;
@@ -495,7 +504,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
                //
                if (this._retryDecryption) {
                    // decryption error, but we have a retry queued.
-                    console.log(
+                    logger.log(
                        `Got error decrypting event (id=${this.getId()}: ` +
                        `${e}), but retrying`,
                    );
@@ -504,7 +513,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {

                // decryption error, no retries queued. Warn about the error and
                // set it to m.bad.encrypted.
-                console.warn(
+                logger.warn(
                    `Error decrypting event (id=${this.getId()}): ${e.detailedString}`,
                );

@@ -665,6 +674,27 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        return this.event.unsigned || {};
    },

+    unmarkLocallyRedacted: function() {
+        const value = this._localRedactionEvent;
+        this._localRedactionEvent = null;
+        if (this.event.unsigned) {
+            this.event.unsigned.redacted_because = null;
+        }
+        return !!value;
+    },
+
+    markLocallyRedacted: function(redactionEvent) {
+        if (this._localRedactionEvent) {
+            return;
+        }
+        this.emit("Event.beforeRedaction", this, redactionEvent);
+        this._localRedactionEvent = redactionEvent;
+        if (!this.event.unsigned) {
+            this.event.unsigned = {};
+        }
+        this.event.unsigned.redacted_because = redactionEvent.event;
+    },
+
    /**
     * Update the content of an event in the same way it would be by the server
     * if it were redacted before it was sent to us
@@ -678,6 +708,8 @@ utils.extend(module.exports.MatrixEvent.prototype, {
            throw new Error("invalid redaction_event in makeRedacted");
        }

+        this._localRedactionEvent = null;
+
        this.emit("Event.beforeRedaction", this, redaction_event);

        this._replacingEvent = null;
@@ -723,6 +755,15 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        return Boolean(this.getUnsigned().redacted_because);
    },

+    /**
+     * Check if this event is a redaction of another event
+     *
+     * @return {boolean} True if this event is a redaction
+     */
+    isRedaction: function() {
+        return this.getType() === "m.room.redaction";
+    },
+
    /**
     * Get the push actions, if known, for this event
     *
@@ -746,9 +787,26 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @param {Object} event the object to assign to the `event` property
     */
    handleRemoteEcho: function(event) {
+        const oldUnsigned = this.getUnsigned();
+        const oldId = this.getId();
        this.event = event;
+        // if this event was redacted before it was sent, it's locally marked as redacted.
+        // At this point, we've received the remote echo for the event, but not yet for
+        // the redaction that we are sending ourselves. Preserve the locally redacted
+        // state by copying over redacted_because so we don't get a flash of
+        // redacted, not-redacted, redacted as remote echos come in
+        if (oldUnsigned.redacted_because) {
+            if (!this.event.unsigned) {
+                this.event.unsigned = {};
+            }
+            this.event.unsigned.redacted_because = oldUnsigned.redacted_because;
+        }
        // successfully sent.
        this.setStatus(null);
+        if (this.getId() !== oldId) {
+            // emit the event if it changed
+            this.emit("Event.localEventIdReplaced", this);
+        }
    },

    /**
@@ -771,6 +829,11 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        this.emit("Event.status", this, status);
    },

+    replaceLocalEventId(eventId) {
+        this.event.event_id = eventId;
+        this.emit("Event.localEventIdReplaced", this);
+    },
+
    /**
     * Get whether the event is a relation event, and of a given type if
     * `relType` is passed in.
@@ -806,7 +869,11 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @param {MatrixEvent?} newEvent the event with the replacing content, if any.
     */
    makeReplaced(newEvent) {
-        if (this.isRedacted()) {
+        // don't allow redacted events to be replaced.
+        // if newEvent is null we allow to go through though,
+        // as with local redaction, the replacing event might get
+        // cancelled, which should be reflected on the target event.
+        if (this.isRedacted() && newEvent) {
            return;
        }
        if (this._replacingEvent !== newEvent) {
@@ -816,15 +883,25 @@ utils.extend(module.exports.MatrixEvent.prototype, {
    },

    /**
-     * Returns the status of the event, or the replacing event in case `makeReplace` has been called.
+     * Returns the status of any associated edit or redaction
+     * (not for reactions/annotations as their local echo doesn't affect the orignal event),
+     * or else the status of the event.
     *
     * @return {EventStatus}
     */
-    replacementOrOwnStatus() {
+    getAssociatedStatus() {
        if (this._replacingEvent) {
            return this._replacingEvent.status;
-        } else {
-            return this.status;
+        } else if (this._localRedactionEvent) {
+            return this._localRedactionEvent.status;
+        }
+        return this.status;
+    },
+
+    getServerAggregatedRelation(relType) {
+        const relations = this.getUnsigned()["m.relations"];
+        if (relations) {
+            return relations[relType];
        }
    },

@@ -834,11 +911,18 @@ utils.extend(module.exports.MatrixEvent.prototype, {
     * @return {string?}
     */
    replacingEventId() {
-        return this._replacingEvent && this._replacingEvent.getId();
+        const replaceRelation = this.getServerAggregatedRelation("m.replace");
+        if (replaceRelation) {
+            return replaceRelation.event_id;
+        } else if (this._replacingEvent) {
+            return this._replacingEvent.getId();
+        }
    },

    /**
     * Returns the event replacing the content of this event, if any.
+     * Replacements are aggregated on the server, so this would only
+     * return an event in case it came down the sync, or for local echo of edits.
     *
     * @return {MatrixEvent?}
     */
@@ -846,6 +930,90 @@ utils.extend(module.exports.MatrixEvent.prototype, {
        return this._replacingEvent;
    },

+    /**
+     * Returns the origin_server_ts of the event replacing the content of this event, if any.
+     *
+     * @return {Date?}
+     */
+    replacingEventDate() {
+        const replaceRelation = this.getServerAggregatedRelation("m.replace");
+        if (replaceRelation) {
+            const ts = replaceRelation.origin_server_ts;
+            if (Number.isFinite(ts)) {
+                return new Date(ts);
+            }
+        } else if (this._replacingEvent) {
+            return this._replacingEvent.getDate();
+        }
+    },
+
+    /**
+     * Returns the event that wants to redact this event, but hasn't been sent yet.
+     * @return {MatrixEvent} the event
+     */
+    localRedactionEvent() {
+        return this._localRedactionEvent;
+    },
+
+    /**
+     * For relations and redactions, returns the event_id this event is referring to.
+     *
+     * @return {string?}
+     */
+    getAssociatedId() {
+        const relation = this.getRelation();
+        if (relation) {
+            return relation.event_id;
+        } else if (this.isRedaction()) {
+            return this.event.redacts;
+        }
+    },
+
+    /**
+     * Checks if this event is associated with another event. See `getAssociatedId`.
+     *
+     * @return {bool}
+     */
+    hasAssocation() {
+        return !!this.getAssociatedId();
+    },
+
+    /**
+     * Update the related id with a new one.
+     *
+     * Used to replace a local id with remote one before sending
+     * an event with a related id.
+     *
+     * @param {string} eventId the new event id
+     */
+    updateAssociatedId(eventId) {
+        const relation = this.getRelation();
+        if (relation) {
+            relation.event_id = eventId;
+        } else if (this.isRedaction()) {
+            this.event.redacts = eventId;
+        }
+    },
+
+    /**
+     * Flags an event as cancelled due to future conditions. For example, a verification
+     * request event in the same sync transaction may be flagged as cancelled to warn
+     * listeners that a cancellation event is coming down the same pipe shortly.
+     * @param {boolean} cancelled Whether the event is to be cancelled or not.
+     */
+    flagCancelled(cancelled = true) {
+        this._isCancelled = cancelled;
+    },
+
+    /**
+     * Gets whether or not the event is flagged as cancelled. See flagCancelled() for
+     * more information.
+     * @returns {boolean} True if the event is cancelled, false otherwise.
+     */
+    isCancelled() {
+        return this._isCancelled;
+    },
+
    /**
     * Summarise the event as JSON for debugging. If encrypted, include both the
     * decrypted and encrypted view of the event. This is named `toJSON` for use
@@ -865,6 +1033,11 @@ utils.extend(module.exports.MatrixEvent.prototype, {
            room_id: this.getRoomId(),
        };

+        // if this is a redaction then attach the redacts key
+        if (this.isRedaction()) {
+            event.redacts = this.event.redacts;
+        }
+
        if (!this.isEncrypted()) {
            return event;
        }
@@ -880,7 +1053,7 @@ utils.extend(module.exports.MatrixEvent.prototype, {
 /* _REDACT_KEEP_KEY_MAP gives the keys we keep when an event is redacted
 *
 * This is specified here:
- *  http://matrix.org/speculator/spec/HEAD/client_server/unstable.html#redactions
+ *  http://matrix.org/speculator/spec/HEAD/client_server/latest.html#redactions
 *
 * Also:
 *  - We keep 'unsigned' since that is created by the local server
@@ -242,12 +242,7 @@ export default class Relations extends EventEmitter {

        redactedEvent.removeListener("Event.beforeRedaction", this._onBeforeRedaction);

-        // Dispatch a redaction event on this collection. `setTimeout` is used
-        // to wait until the next event loop iteration by which time the event
-        // has actually been marked as redacted.
-        setTimeout(() => {
-            this.emit("Relations.redaction");
-        }, 0);
+        this.emit("Relations.redaction");
    }

    /**
@@ -306,10 +301,20 @@ export default class Relations extends EventEmitter {
            // event is known anyway.
            return null;
        }
+
+        // the all-knowning server tells us that the event at some point had
+        // this timestamp for its replacement, so any following replacement should definitely not be less
+        const replaceRelation =
+            this._targetEvent.getServerAggregatedRelation("m.replace");
+        const minTs = replaceRelation && replaceRelation.origin_server_ts;
+
        return this.getRelations().reduce((last, event) => {
            if (event.getSender() !== this._targetEvent.getSender()) {
                return last;
            }
+            if (minTs && minTs > event.getTs()) {
+                return last;
+            }
            if (last && last.getTs() > event.getTs()) {
                return last;
            }
@@ -249,7 +249,7 @@ RoomMember.prototype.getDMInviter = function() {
 * "crop" or "scale".
 * @param {Boolean} allowDefault (optional) Passing false causes this method to
 * return null if the user has no avatar image. Otherwise, a default image URL
- * will be returned. Default: true.
+ * will be returned. Default: true. (Deprecated)
 * @param {Boolean} allowDirectLinks (optional) If true, the avatar URL will be
 * returned even if it is a direct hyperlink rather than a matrix content URL.
 * If false, any non-matrix content URLs will be ignored. Setting this option to
@@ -21,6 +21,7 @@ const EventEmitter = require("events").EventEmitter;

 const utils = require("../utils");
 const RoomMember = require("./room-member");
+import logger from '../logger';

 // possible statuses for out-of-band member loading
 const OOB_STATUS_NOTSTARTED = 1;
@@ -447,7 +448,7 @@ RoomState.prototype.clearOutOfBandMembers = function() {
            delete this.members[userId];
        }
    });
-    console.log(`LL: RoomState removed ${count} members...`);
+    logger.log(`LL: RoomState removed ${count} members...`);
    this._oobMemberFlags.status = OOB_STATUS_NOTSTARTED;
 };

@@ -456,11 +457,11 @@ RoomState.prototype.clearOutOfBandMembers = function() {
 * @param {MatrixEvent[]} stateEvents array of membership state events
 */
 RoomState.prototype.setOutOfBandMembers = function(stateEvents) {
-    console.log(`LL: RoomState about to set ${stateEvents.length} OOB members ...`);
+    logger.log(`LL: RoomState about to set ${stateEvents.length} OOB members ...`);
    if (this._oobMemberFlags.status !== OOB_STATUS_INPROGRESS) {
        return;
    }
-    console.log(`LL: RoomState put in OOB_STATUS_FINISHED state ...`);
+    logger.log(`LL: RoomState put in OOB_STATUS_FINISHED state ...`);
    this._oobMemberFlags.status = OOB_STATUS_FINISHED;
    stateEvents.forEach((e) => this._setOutOfBandMember(e));
 };
@@ -1,6 +1,7 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
 Copyright 2018, 2019 New Vector Ltd
+Copyright 2019 The Matrix.org Foundation C.I.C.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -29,6 +30,7 @@ const ContentRepo = require("../content-repo");
 const EventTimeline = require("./event-timeline");
 const EventTimelineSet = require("./event-timeline-set");

+import logger from '../logger';
 import ReEmitter from '../ReEmitter';

 // These constants are used as sane defaults when the homeserver doesn't support
@@ -37,8 +39,8 @@ import ReEmitter from '../ReEmitter';
 // room versions which are considered okay for people to run without being asked
 // to upgrade (ie: "stable"). Eventually, we should remove these when all homeservers
 // return an m.room_versions capability.
-const KNOWN_SAFE_ROOM_VERSION = '1';
-const SAFE_ROOM_VERSIONS = ['1', '2', '3'];
+const KNOWN_SAFE_ROOM_VERSION = '4';
+const SAFE_ROOM_VERSIONS = ['1', '2', '3', '4'];

 function synthesizeReceipt(userId, event, receiptType) {
    // console.log("synthesizing receipt for "+event.getId());
@@ -209,7 +211,7 @@ utils.inherits(Room, EventEmitter);
 Room.prototype.getVersion = function() {
    const createEvent = this.currentState.getStateEvents("m.room.create", "");
    if (!createEvent) {
-        console.warn("Room " + this.room_id + " does not have an m.room.create event");
+        logger.warn("Room " + this.room_id + " does not have an m.room.create event");
        return '1';
    }
    const ver = createEvent.getContent()['room_version'];
@@ -262,9 +264,36 @@ Room.prototype.getRecommendedVersion = async function() {
        }
    }

+    let result = this._checkVersionAgainstCapability(versionCap);
+    if (result.urgent && result.needsUpgrade) {
+        // Something doesn't feel right: we shouldn't need to update
+        // because the version we're on should be in the protocol's
+        // namespace. This usually means that the server was updated
+        // before the client was, making us think the newest possible
+        // room version is not stable. As a solution, we'll refresh
+        // the capability we're using to determine this.
+        logger.warn(
+            "Refreshing room version capability because the server looks " +
+            "to be supporting a newer room version we don't know about.",
+        );
+
+        const caps = await this._client.getCapabilities(true);
+        versionCap = caps["m.room_versions"];
+        if (!versionCap) {
+            logger.warn("No room version capability - assuming upgrade required.");
+            return result;
+        } else {
+            result = this._checkVersionAgainstCapability(versionCap);
+        }
+    }
+
+    return result;
+};
+
+Room.prototype._checkVersionAgainstCapability = function(versionCap) {
    const currentVersion = this.getVersion();
-    console.log(`[${this.roomId}] Current version: ${currentVersion}`);
-    console.log(`[${this.roomId}] Version capability: `, versionCap);
+    logger.log(`[${this.roomId}] Current version: ${currentVersion}`);
+    logger.log(`[${this.roomId}] Version capability: `, versionCap);

    const result = {
        version: currentVersion,
@@ -273,7 +302,7 @@ Room.prototype.getRecommendedVersion = async function() {
    };

    // If the room is on the default version then nothing needs to change
-    if (currentVersion === versionCap.default) return Promise.resolve(result);
+    if (currentVersion === versionCap.default) return result;

    const stableVersions = Object.keys(versionCap.available)
        .filter((v) => versionCap.available[v] === 'stable');
@@ -286,16 +315,16 @@ Room.prototype.getRecommendedVersion = async function() {
        result.needsUpgrade = true;
        result.urgent = !!this.getVersion().match(/^[0-9]+[0-9.]*$/g);
        if (result.urgent) {
-            console.warn(`URGENT upgrade required on ${this.roomId}`);
+            logger.warn(`URGENT upgrade required on ${this.roomId}`);
        } else {
-            console.warn(`Non-urgent upgrade required on ${this.roomId}`);
+            logger.warn(`Non-urgent upgrade required on ${this.roomId}`);
        }
-        return Promise.resolve(result);
+        return result;
    }

    // The room is on a stable, but non-default, version by this point.
    // No upgrade needed.
-    return Promise.resolve(result);
+    return result;
 };

 /**
@@ -318,13 +347,30 @@ Room.prototype.userMayUpgradeRoom = function(userId) {
 Room.prototype.getPendingEvents = function() {
    if (this._opts.pendingEventOrdering !== "detached") {
        throw new Error(
-            "Cannot call getPendingEventList with pendingEventOrdering == " +
+            "Cannot call getPendingEvents with pendingEventOrdering == " +
                this._opts.pendingEventOrdering);
    }

    return this._pendingEventList;
 };

+/**
+ * Check whether the pending event list contains a given event by ID.
+ *
+ * @param {string} eventId The event ID to check for.
+ * @return {boolean}
+ * @throws If <code>opts.pendingEventOrdering</code> was not 'detached'
+ */
+Room.prototype.hasPendingEvent = function(eventId) {
+    if (this._opts.pendingEventOrdering !== "detached") {
+        throw new Error(
+            "Cannot call hasPendingEvent with pendingEventOrdering == " +
+                this._opts.pendingEventOrdering);
+    }
+
+    return this._pendingEventList.some(event => event.getId() === eventId);
+};
+
 /**
 * Get the live unfiltered timeline for this room.
 *
@@ -471,7 +517,7 @@ Room.prototype._loadMembers = async function() {
    if (rawMembersEvents === null) {
        fromServer = true;
        rawMembersEvents = await this._loadMembersFromServer();
-        console.log(`LL: got ${rawMembersEvents.length} ` +
+        logger.log(`LL: got ${rawMembersEvents.length} ` +
            `members from server for room ${this.roomId}`);
    }
    const memberEvents = rawMembersEvents.map(this._client.getEventMapper());
@@ -515,21 +561,21 @@ Room.prototype.loadMembersIfNeeded = function() {
            const oobMembers = this.currentState.getMembers()
                .filter((m) => m.isOutOfBand())
                .map((m) => m.events.member.event);
-            console.log(`LL: telling store to write ${oobMembers.length}`
+            logger.log(`LL: telling store to write ${oobMembers.length}`
                + ` members for room ${this.roomId}`);
            const store = this._client.store;
            return store.setOutOfBandMembers(this.roomId, oobMembers)
                // swallow any IDB error as we don't want to fail
                // because of this
                .catch((err) => {
-                    console.log("LL: storing OOB room members failed, oh well",
+                    logger.log("LL: storing OOB room members failed, oh well",
                        err);
                });
        }
    }).catch((err) => {
        // as this is not awaited anywhere,
        // at least show the error in the console
-        console.error(err);
+        logger.error(err);
    });

    this._membersPromise = inMemoryUpdate;
@@ -555,9 +601,9 @@ Room.prototype.clearLoadedMembersIfNeeded = async function() {
 */
 Room.prototype._cleanupAfterLeaving = function() {
    this.clearLoadedMembersIfNeeded().catch((err) => {
-        console.error(`error after clearing loaded members from ` +
+        logger.error(`error after clearing loaded members from ` +
            `room ${this.roomId} after leaving`);
-        console.dir(err);
+        logger.log(err);
    });
 };

@@ -735,7 +781,7 @@ Room.prototype.getBlacklistUnverifiedDevices = function() {
 * @param {string} resizeMethod The thumbnail resize method to use, either
 * "crop" or "scale".
 * @param {boolean} allowDefault True to allow an identicon for this room if an
- * avatar URL wasn't explicitly set. Default: true.
+ * avatar URL wasn't explicitly set. Default: true. (Deprecated)
 * @return {?string} the avatar URL or null.
 */
 Room.prototype.getAvatarUrl = function(baseUrl, width, height, resizeMethod,
@@ -769,20 +815,20 @@ Room.prototype.getAvatarUrl = function(baseUrl, width, height, resizeMethod,
 * @return {array} The room's alias as an array of strings
 */
 Room.prototype.getAliases = function() {
-    const alias_strings = [];
+    const aliasStrings = [];

-    const alias_events = this.currentState.getStateEvents("m.room.aliases");
-    if (alias_events) {
-        for (let i = 0; i < alias_events.length; ++i) {
-            const alias_event = alias_events[i];
-            if (utils.isArray(alias_event.getContent().aliases)) {
+    const aliasEvents = this.currentState.getStateEvents("m.room.aliases");
+    if (aliasEvents) {
+        for (let i = 0; i < aliasEvents.length; ++i) {
+            const aliasEvent = aliasEvents[i];
+            if (utils.isArray(aliasEvent.getContent().aliases)) {
                Array.prototype.push.apply(
-                    alias_strings, alias_event.getContent().aliases,
+                    aliasStrings, aliasEvent.getContent().aliases,
                );
            }
        }
    }
-    return alias_strings;
+    return aliasStrings;
 };

 /**
@@ -1004,13 +1050,25 @@ Room.prototype.removeFilteredTimelineSet = function(filter) {
 * @private
 */
 Room.prototype._addLiveEvent = function(event, duplicateStrategy) {
-    if (event.getType() === "m.room.redaction") {
+    if (event.isRedaction()) {
        const redactId = event.event.redacts;

        // if we know about this event, redact its contents now.
        const redactedEvent = this.getUnfilteredTimelineSet().findEventById(redactId);
        if (redactedEvent) {
            redactedEvent.makeRedacted(event);
+
+            // If this is in the current state, replace it with the redacted version
+            if (redactedEvent.getStateKey()) {
+                const currentStateEvent = this.currentState.getStateEvents(
+                    redactedEvent.getType(),
+                    redactedEvent.getStateKey(),
+                );
+                if (currentStateEvent.getId() === redactedEvent.getId()) {
+                    this.currentState.setStateEvents([redactedEvent]);
+                }
+            }
+
            this.emit("Room.redaction", event, this);

            // TODO: we stash user displaynames (among other things) in
@@ -1101,7 +1159,7 @@ Room.prototype.addPendingEvent = function(event, txnId) {

    if (this._opts.pendingEventOrdering == "detached") {
        if (this._pendingEventList.some((e) => e.status === EventStatus.NOT_SENT)) {
-            console.warn("Setting event as NOT_SENT due to messages in the same state");
+            logger.warn("Setting event as NOT_SENT due to messages in the same state");
            event.setStatus(EventStatus.NOT_SENT);
        }
        this._pendingEventList.push(event);
@@ -1110,24 +1168,26 @@ Room.prototype.addPendingEvent = function(event, txnId) {
            // For pending events, add them to the relations collection immediately.
            // (The alternate case below already covers this as part of adding to
            // the timeline set.)
-            // TODO: We should consider whether this means it would be a better
-            // design to lift the relations handling up to the room instead.
-            for (let i = 0; i < this._timelineSets.length; i++) {
-                const timelineSet = this._timelineSets[i];
-                if (timelineSet.getFilter()) {
-                    if (this._filter.filterRoomTimeline([event]).length) {
-                        timelineSet.aggregateRelations(event);
-                    }
-                } else {
-                    timelineSet.aggregateRelations(event);
-                }
+            this._aggregateNonLiveRelation(event);
+        }
+
+        if (event.isRedaction()) {
+            const redactId = event.event.redacts;
+            let redactedEvent = this._pendingEventList &&
+                this._pendingEventList.find(e => e.getId() === redactId);
+            if (!redactedEvent) {
+                redactedEvent = this.getUnfilteredTimelineSet().findEventById(redactId);
+            }
+            if (redactedEvent) {
+                redactedEvent.markLocallyRedacted(event);
+                this.emit("Room.redaction", event, this);
            }
        }
    } else {
        for (let i = 0; i < this._timelineSets.length; i++) {
            const timelineSet = this._timelineSets[i];
            if (timelineSet.getFilter()) {
-                if (this._filter.filterRoomTimeline([event]).length) {
+                if (timelineSet.getFilter().filterRoomTimeline([event]).length) {
                    timelineSet.addEventToTimeline(event,
                        timelineSet.getLiveTimeline(), false);
                }
@@ -1140,6 +1200,30 @@ Room.prototype.addPendingEvent = function(event, txnId) {

    this.emit("Room.localEchoUpdated", event, this, null, null);
 };
+/**
+ * Used to aggregate the local echo for a relation, and also
+ * for re-applying a relation after it's redaction has been cancelled,
+ * as the local echo for the redaction of the relation would have
+ * un-aggregated the relation. Note that this is different from regular messages,
+ * which are just kept detached for their local echo.
+ *
+ * Also note that live events are aggregated in the live EventTimelineSet.
+ * @param {module:models/event.MatrixEvent} event the relation event that needs to be aggregated.
+ */
+Room.prototype._aggregateNonLiveRelation = function(event) {
+    // TODO: We should consider whether this means it would be a better
+    // design to lift the relations handling up to the room instead.
+    for (let i = 0; i < this._timelineSets.length; i++) {
+        const timelineSet = this._timelineSets[i];
+        if (timelineSet.getFilter()) {
+            if (timelineSet.getFilter().filterRoomTimeline([event]).length) {
+                timelineSet.aggregateRelations(event);
+            }
+        } else {
+            timelineSet.aggregateRelations(event);
+        }
+    }
+};

 /**
 * Deal with the echo of a message we sent.
@@ -1161,7 +1245,7 @@ Room.prototype._handleRemoteEcho = function(remoteEvent, localEvent) {
    const oldStatus = localEvent.status;

    // no longer pending
-    delete this._txnToEvent[remoteEvent.transaction_id];
+    delete this._txnToEvent[remoteEvent.getUnsigned().transaction_id];

    // if it's in the pending list, remove it
    if (this._pendingEventList) {
@@ -1229,7 +1313,7 @@ ALLOWED_TRANSITIONS[EventStatus.CANCELLED] =
 * @fires module:client~MatrixClient#event:"Room.localEchoUpdated"
 */
 Room.prototype.updatePendingEvent = function(event, newStatus, newEventId) {
-    console.log(`setting pendingEvent status to ${newStatus} in ${event.getRoomId()}`);
+    logger.log(`setting pendingEvent status to ${newStatus} in ${event.getRoomId()}`);

    // if the message was sent, we expect an event id
    if (newStatus == EventStatus.SENT && !newEventId) {
@@ -1265,7 +1349,7 @@ Room.prototype.updatePendingEvent = function(event, newStatus, newEventId) {

    if (newStatus == EventStatus.SENT) {
        // update the event id
-        event.event.event_id = newEventId;
+        event.replaceLocalEventId(newEventId);

        // if the event was already in the timeline (which will be the case if
        // opts.pendingEventOrdering==chronological), we need to update the
@@ -1276,12 +1360,13 @@ Room.prototype.updatePendingEvent = function(event, newStatus, newEventId) {
    } else if (newStatus == EventStatus.CANCELLED) {
        // remove it from the pending event list, or the timeline.
        if (this._pendingEventList) {
-            utils.removeElement(
-                this._pendingEventList,
-                function(ev) {
-                    return ev.getId() == oldEventId;
-                }, false,
-            );
+            const idx = this._pendingEventList.findIndex(ev => ev.getId() === oldEventId);
+            if (idx !== -1) {
+                const [removedEvent] = this._pendingEventList.splice(idx, 1);
+                if (removedEvent.isRedaction()) {
+                    this._revertRedactionLocalEcho(removedEvent);
+                }
+            }
        }
        this.removeEvent(oldEventId);
    }
@@ -1289,6 +1374,23 @@ Room.prototype.updatePendingEvent = function(event, newStatus, newEventId) {
    this.emit("Room.localEchoUpdated", event, this, oldEventId, oldStatus);
 };

+Room.prototype._revertRedactionLocalEcho = function(redactionEvent) {
+    const redactId = redactionEvent.event.redacts;
+    if (!redactId) {
+        return;
+    }
+    const redactedEvent = this.getUnfilteredTimelineSet()
+        .findEventById(redactId);
+    if (redactedEvent) {
+        redactedEvent.unmarkLocallyRedacted();
+        // re-render after undoing redaction
+        this.emit("Room.redactionCancelled", redactionEvent, this);
+        // reapply relation now redaction failed
+        if (redactedEvent.isRelation()) {
+            this._aggregateNonLiveRelation(redactedEvent);
+        }
+    }
+};

 /**
 * Add some events to this room. This can include state events, message
@@ -1330,28 +1432,33 @@ Room.prototype.addLiveEvents = function(events, duplicateStrategy) {
    }

    for (i = 0; i < events.length; i++) {
-        if (events[i].getType() === "m.typing") {
-            this.currentState.setTypingEvent(events[i]);
-        } else if (events[i].getType() === "m.receipt") {
-            this.addReceipt(events[i]);
-        }
-        // N.B. account_data is added directly by /sync to avoid
-        // having to maintain an event.isAccountData() here
-        else {
-            // TODO: We should have a filter to say "only add state event
-            // types X Y Z to the timeline".
-            this._addLiveEvent(events[i], duplicateStrategy);
-        }
+        // TODO: We should have a filter to say "only add state event
+        // types X Y Z to the timeline".
+        this._addLiveEvent(events[i], duplicateStrategy);
+    }
+};
+
+/**
+ * Adds/handles ephemeral events such as typing notifications and read receipts.
+ * @param {MatrixEvent[]} events A list of events to process
+ */
+Room.prototype.addEphemeralEvents = function(events) {
+    for (const event of events) {
+        if (event.getType() === 'm.typing') {
+            this.currentState.setTypingEvent(event);
+        } else if (event.getType() === 'm.receipt') {
+            this.addReceipt(event);
+        } // else ignore - life is too short for us to care about these events
    }
 };

 /**
 * Removes events from this room.
- * @param {String[]} event_ids A list of event_ids to remove.
+ * @param {String[]} eventIds A list of eventIds to remove.
 */
-Room.prototype.removeEvents = function(event_ids) {
-    for (let i = 0; i < event_ids.length; ++i) {
-        this.removeEvent(event_ids[i]);
+Room.prototype.removeEvents = function(eventIds) {
+    for (let i = 0; i < eventIds.length; ++i) {
+        this.removeEvent(eventIds[i]);
    }
 };

@@ -1367,6 +1474,9 @@ Room.prototype.removeEvent = function(eventId) {
    for (let i = 0; i < this._timelineSets.length; i++) {
        const removed = this._timelineSets[i].removeEvent(eventId);
        if (removed) {
+            if (removed.isRedaction()) {
+                this._revertRedactionLocalEcho(removed);
+            }
            removedAny = true;
        }
    }
@@ -1790,10 +1900,21 @@ module.exports = Room;
 * event).
 *
 * @event module:client~MatrixClient#"Room.redaction"
- * @param {MatrixEvent} event The matrix event which was redacted
+ * @param {MatrixEvent} event The matrix redaction event
 * @param {Room} room The room containing the redacted event
 */

+/**
+ * Fires when an event that was previously redacted isn't anymore.
+ * This happens when the redaction couldn't be sent and
+ * was subsequently cancelled by the user. Redactions have a local echo
+ * which is undone in this scenario.
+ *
+ * @event module:client~MatrixClient#"Room.redactionCancelled"
+ * @param {MatrixEvent} event The matrix redaction event that was cancelled.
+ * @param {Room} room The room containing the unredacted event
+ */
+
 /**
 * Fires whenever the name of a room is updated.
 * @event module:client~MatrixClient#"Room.name"
@@ -43,6 +43,11 @@ const DEFAULT_OVERRIDE_RULES = [
                key: "type",
                pattern: "m.room.tombstone",
            },
+            {
+                kind: "event_match",
+                key: "state_key",
+                pattern: "",
+            },
        ],
        actions: [
            "notify",
@@ -52,6 +57,22 @@ const DEFAULT_OVERRIDE_RULES = [
            },
        ],
    },
+    {
+        // For homeservers which don't support MSC2153 yet
+        rule_id: ".m.rule.reaction",
+        default: true,
+        enabled: true,
+        conditions: [
+            {
+                kind: "event_match",
+                key: "type",
+                pattern: "m.reaction",
+            },
+        ],
+        actions: [
+            "dont_notify",
+        ],
+    },
 ];

 /**
@@ -439,6 +460,38 @@ PushProcessor.actionListToActionsObject = function(actionlist) {
    return actionobj;
 };

+/**
+ * Rewrites conditions on a client's push rules to match the defaults
+ * where applicable. Useful for upgrading push rules to more strict
+ * conditions when the server is falling behind on defaults.
+ * @param {object} incomingRules The client's existing push rules
+ * @returns {object} The rewritten rules
+ */
+PushProcessor.rewriteDefaultRules = function(incomingRules) {
+    let newRules = JSON.parse(JSON.stringify(incomingRules)); // deep clone
+
+    // These lines are mostly to make the tests happy. We shouldn't run into these
+    // properties missing in practice.
+    if (!newRules) newRules = {};
+    if (!newRules.global) newRules.global = {};
+    if (!newRules.global.override) newRules.global.override = [];
+
+    // Fix default override rules
+    newRules.global.override = newRules.global.override.map(r => {
+        const defaultRule = DEFAULT_OVERRIDE_RULES.find(d => d.rule_id === r.rule_id);
+        if (!defaultRule) return r;
+
+        // Copy over the actions, default, and conditions. Don't touch the user's
+        // preference.
+        r.default = defaultRule.default;
+        r.conditions = defaultRule.conditions;
+        r.actions = defaultRule.actions;
+        return r;
+    });
+
+    return newRules;
+};
+
 /**
 * @typedef {Object} PushAction
 * @type {Object}
@@ -24,6 +24,7 @@ limitations under the License.
 */

 "use strict";
+import logger from './logger';

 // we schedule a callback at least this often, to check if we've missed out on
 // some wall-clock time due to being suspended.
@@ -39,7 +40,7 @@ let _realCallbackKey;
 // each is an object with keys [runAt, func, params, key].
 const _callbackList = [];

-// var debuglog = console.log.bind(console);
+// var debuglog = logger.log.bind(logger);
 const debuglog = function() {};

 /**
@@ -170,7 +171,7 @@ function _runCallbacks() {
        try {
            cb.func.apply(global, cb.params);
        } catch (e) {
-            console.error("Uncaught exception in callback function",
+            logger.error("Uncaught exception in callback function",
                          e.stack || e);
        }
    }
@@ -21,6 +21,7 @@ limitations under the License.
 */
 const utils = require("./utils");
 import Promise from 'bluebird';
+import logger from './logger';

 const DEBUG = false;  // set true to enable console logging.

@@ -176,7 +177,8 @@ MatrixScheduler.RETRY_BACKOFF_RATELIMIT = function(event, attempts, err) {
 * @see module:scheduler~queueAlgorithm
 */
 MatrixScheduler.QUEUE_MESSAGES = function(event) {
-    if (event.getType() === "m.room.message") {
+    // enqueue messages or events that associate with another event (redactions and relations)
+    if (event.getType() === "m.room.message" || event.hasAssocation()) {
        // put these events in the 'message' queue.
        return "message";
    }
@@ -219,7 +221,14 @@ function _processQueue(scheduler, queueName) {
    );
    // fire the process function and if it resolves, resolve the deferred. Else
    // invoke the retry algorithm.
-    scheduler._procFn(obj.event).done(function(res) {
+
+    // First wait for a resolved promise, so the resolve handlers for
+    // the deferred of the previously sent event can run.
+    // This way enqueued relations/redactions to enqueued events can receive
+    // the remove id of their target before being sent.
+    Promise.resolve().then(() => {
+        return scheduler._procFn(obj.event);
+    }).then(function(res) {
        // remove this from the queue
        _removeNextEvent(scheduler, queueName);
        debuglog("Queue '%s' sent event %s", queueName, obj.event.getId());
@@ -269,7 +278,7 @@ function _removeNextEvent(scheduler, queueName) {

 function debuglog() {
    if (DEBUG) {
-        console.log(...arguments);
+        logger.log(...arguments);
    }
 }

@@ -0,0 +1,20 @@
+/*
+Copyright 2019 The Matrix.org Foundation C.I.C.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+export const SERVICE_TYPES = Object.freeze({
+    IS: 'SERVICE_TYPE_IS', // An Identity Service
+    IM: 'SERVICE_TYPE_IM', // An Integration Manager
+});
@@ -19,6 +19,7 @@ import Promise from 'bluebird';
 import SyncAccumulator from "../sync-accumulator";
 import utils from "../utils";
 import * as IndexedDBHelpers from "../indexeddb-helpers";
+import logger from '../logger';

 const VERSION = 3;

@@ -146,7 +147,7 @@ LocalIndexedDBStoreBackend.prototype = {
     */
    connect: function() {
        if (!this._disconnected) {
-            console.log(
+            logger.log(
                `LocalIndexedDBStoreBackend.connect: already connected or connecting`,
            );
            return Promise.resolve();
@@ -154,14 +155,14 @@ LocalIndexedDBStoreBackend.prototype = {

        this._disconnected = false;

-        console.log(
+        logger.log(
            `LocalIndexedDBStoreBackend.connect: connecting...`,
        );
        const req = this.indexedDB.open(this._dbName, VERSION);
        req.onupgradeneeded = (ev) => {
            const db = ev.target.result;
            const oldVersion = ev.oldVersion;
-            console.log(
+            logger.log(
                `LocalIndexedDBStoreBackend.connect: upgrading from ${oldVersion}`,
            );
            if (oldVersion < 1) { // The database did not previously exist.
@@ -178,16 +179,16 @@ LocalIndexedDBStoreBackend.prototype = {
        };

        req.onblocked = () => {
-            console.log(
+            logger.log(
                `can't yet open LocalIndexedDBStoreBackend because it is open elsewhere`,
            );
        };

-        console.log(
+        logger.log(
            `LocalIndexedDBStoreBackend.connect: awaiting connection...`,
        );
        return reqAsEventPromise(req).then((ev) => {
-            console.log(
+            logger.log(
                `LocalIndexedDBStoreBackend.connect: connected`,
            );
            this.db = ev.target.result;
@@ -215,7 +216,7 @@ LocalIndexedDBStoreBackend.prototype = {
            this._loadAccountData(),
            this._loadSyncData(),
        ]).then(([accountData, syncData]) => {
-            console.log(
+            logger.log(
                `LocalIndexedDBStoreBackend: loaded initial data`,
            );
            this._syncAccumulator.accumulate({
@@ -273,7 +274,7 @@ LocalIndexedDBStoreBackend.prototype = {
                reject(err);
            };
        }).then((events) => {
-            console.log(`LL: got ${events && events.length}` +
+            logger.log(`LL: got ${events && events.length}` +
                ` membershipEvents from storage for room ${roomId} ...`);
            return events;
        });
@@ -287,7 +288,7 @@ LocalIndexedDBStoreBackend.prototype = {
     * @param {event[]} membershipEvents the membership events to store
     */
    setOutOfBandMembers: async function(roomId, membershipEvents) {
-        console.log(`LL: backend about to store ${membershipEvents.length}` +
+        logger.log(`LL: backend about to store ${membershipEvents.length}` +
            ` members for ${roomId}`);
        const tx = this.db.transaction(["oob_membership_events"], "readwrite");
        const store = tx.objectStore("oob_membership_events");
@@ -306,7 +307,7 @@ LocalIndexedDBStoreBackend.prototype = {
        };
        store.put(markerObject);
        await txnAsPromise(tx);
-        console.log(`LL: backend done storing for ${roomId}!`);
+        logger.log(`LL: backend done storing for ${roomId}!`);
    },

    clearOutOfBandMembers: async function(roomId) {
@@ -341,7 +342,7 @@ LocalIndexedDBStoreBackend.prototype = {
            [roomId, maxStateKey],
        );

-        console.log(`LL: Deleting all users + marker in storage for ` +
+        logger.log(`LL: Deleting all users + marker in storage for ` +
            `room ${roomId}, with key range:`,
            [roomId, minStateKey], [roomId, maxStateKey]);
        await reqAsPromise(writeStore.delete(membersKeyRange));
@@ -354,11 +355,11 @@ LocalIndexedDBStoreBackend.prototype = {
     */
    clearDatabase: function() {
        return new Promise((resolve, reject) => {
-            console.log(`Removing indexeddb instance: ${this._dbName}`);
+            logger.log(`Removing indexeddb instance: ${this._dbName}`);
            const req = this.indexedDB.deleteDatabase(this._dbName);

            req.onblocked = () => {
-                console.log(
+                logger.log(
                    `can't yet delete indexeddb ${this._dbName}` +
                    ` because it is open elsewhere`,
                );
@@ -368,14 +369,14 @@ LocalIndexedDBStoreBackend.prototype = {
                // in firefox, with indexedDB disabled, this fails with a
                // DOMError. We treat this as non-fatal, so that we can still
                // use the app.
-                console.warn(
+                logger.warn(
                    `unable to delete js-sdk store indexeddb: ${ev.target.error}`,
                );
                resolve();
            };

            req.onsuccess = () => {
-                console.log(`Removed indexeddb instance: ${this._dbName}`);
+                logger.log(`Removed indexeddb instance: ${this._dbName}`);
                resolve();
            };
        });
@@ -434,7 +435,7 @@ LocalIndexedDBStoreBackend.prototype = {
     * @return {Promise} Resolves if the data was persisted.
     */
    _persistSyncData: function(nextBatch, roomsData, groupsData) {
-        console.log("Persisting sync data up to ", nextBatch);
+        logger.log("Persisting sync data up to ", nextBatch);
        return Promise.try(() => {
            const txn = this.db.transaction(["sync"], "readwrite");
            const store = txn.objectStore("sync");
@@ -508,7 +509,7 @@ LocalIndexedDBStoreBackend.prototype = {
     * @return {Promise<Object[]>} A list of raw global account events.
     */
    _loadAccountData: function() {
-        console.log(
+        logger.log(
            `LocalIndexedDBStoreBackend: loading account data...`,
        );
        return Promise.try(() => {
@@ -517,7 +518,7 @@ LocalIndexedDBStoreBackend.prototype = {
            return selectQuery(store, undefined, (cursor) => {
                return cursor.value;
            }).then((result) => {
-                console.log(
+                logger.log(
                    `LocalIndexedDBStoreBackend: loaded account data`,
                );
                return result;
@@ -530,7 +531,7 @@ LocalIndexedDBStoreBackend.prototype = {
     * @return {Promise<Object>} An object with "roomsData" and "nextBatch" keys.
     */
    _loadSyncData: function() {
-        console.log(
+        logger.log(
            `LocalIndexedDBStoreBackend: loading sync data...`,
        );
        return Promise.try(() => {
@@ -539,11 +540,11 @@ LocalIndexedDBStoreBackend.prototype = {
            return selectQuery(store, undefined, (cursor) => {
                return cursor.value;
            }).then((results) => {
-                console.log(
+                logger.log(
                    `LocalIndexedDBStoreBackend: loaded sync data`,
                );
                if (results.length > 1) {
-                    console.warn("loadSyncData: More than 1 sync row found.");
+                    logger.warn("loadSyncData: More than 1 sync row found.");
                }
                return (results.length > 0 ? results[0] : {});
            });
@@ -16,6 +16,7 @@ limitations under the License.
 */

 import Promise from 'bluebird';
+import logger from '../logger';

 /**
 * An IndexedDB store backend where the actual backend sits in a web
@@ -140,7 +141,7 @@ RemoteIndexedDBStoreBackend.prototype = {

            // tell the worker the db name.
            this._startPromise = this._doCmd('_setupWorker', [this._dbName]).then(() => {
-                console.log("IndexedDB worker is ready");
+                logger.log("IndexedDB worker is ready");
            });
        }
        return this._startPromise;
@@ -170,13 +171,13 @@ RemoteIndexedDBStoreBackend.prototype = {

        if (msg.command == 'cmd_success' || msg.command == 'cmd_fail') {
            if (msg.seq === undefined) {
-                console.error("Got reply from worker with no seq");
+                logger.error("Got reply from worker with no seq");
                return;
            }

            const def = this._inFlight[msg.seq];
            if (def === undefined) {
-                console.error("Got reply for unknown seq " + msg.seq);
+                logger.error("Got reply for unknown seq " + msg.seq);
                return;
            }
            delete this._inFlight[msg.seq];
@@ -189,7 +190,7 @@ RemoteIndexedDBStoreBackend.prototype = {
                def.reject(error);
            }
        } else {
-            console.warn("Unrecognised message from worker: " + msg);
+            logger.warn("Unrecognised message from worker: " + msg);
        }
    },
 };
@@ -17,6 +17,7 @@ limitations under the License.

 import Promise from 'bluebird';
 import LocalIndexedDBStoreBackend from "./indexeddb-local-backend.js";
+import logger from '../logger';

 /**
 * This class lives in the webworker and drives a LocalIndexedDBStoreBackend
@@ -129,8 +130,8 @@ class IndexedDBStoreWorker {
                result: ret,
            });
        }, (err) => {
-            console.error("Error running command: "+msg.command);
-            console.error(err);
+            logger.error("Error running command: "+msg.command);
+            logger.error(err);
            this.postMessage.call(null, {
                command: 'cmd_fail',
                seq: msg.seq,
@@ -25,6 +25,7 @@ import LocalIndexedDBStoreBackend from "./indexeddb-local-backend.js";
 import RemoteIndexedDBStoreBackend from "./indexeddb-remote-backend.js";
 import User from "../models/user";
 import {MatrixEvent} from "../models/event";
+import logger from '../logger';

 /**
 * This is an internal module. See {@link IndexedDBStore} for the public class.
@@ -124,16 +125,16 @@ IndexedDBStore.exists = function(indexedDB, dbName) {
  */
 IndexedDBStore.prototype.startup = function() {
    if (this.startedUp) {
-        console.log(`IndexedDBStore.startup: already started`);
+        logger.log(`IndexedDBStore.startup: already started`);
        return Promise.resolve();
    }

-    console.log(`IndexedDBStore.startup: connecting to backend`);
+    logger.log(`IndexedDBStore.startup: connecting to backend`);
    return this.backend.connect().then(() => {
-        console.log(`IndexedDBStore.startup: loading presence events`);
+        logger.log(`IndexedDBStore.startup: loading presence events`);
        return this.backend.getUserPresenceEvents();
    }).then((userPresenceEvents) => {
-        console.log(`IndexedDBStore.startup: processing presence events`);
+        logger.log(`IndexedDBStore.startup: processing presence events`);
        userPresenceEvents.forEach(([userId, rawEvent]) => {
            const u = new User(userId);
            if (rawEvent) {
@@ -174,9 +175,9 @@ IndexedDBStore.prototype.getSavedSyncToken = degradable(function() {
 IndexedDBStore.prototype.deleteAllData = degradable(function() {
    MemoryStore.prototype.deleteAllData.call(this);
    return this.backend.clearDatabase().then(() => {
-        console.log("Deleted indexeddb data.");
+        logger.log("Deleted indexeddb data.");
    }, (err) => {
-        console.error(`Failed to delete indexeddb data: ${err}`);
+        logger.error(`Failed to delete indexeddb data: ${err}`);
        throw err;
    });
 });
@@ -197,11 +198,13 @@ IndexedDBStore.prototype.wantsSave = function() {

 /**
 * Possibly write data to the database.
+ *
+ * @param {bool} force True to force a save to happen
 * @return {Promise} Promise resolves after the write completes
 *     (or immediately if no write is performed)
 */
-IndexedDBStore.prototype.save = function() {
-    if (this.wantsSave()) {
+IndexedDBStore.prototype.save = function(force) {
+    if (force || this.wantsSave()) {
        return this._reallySave();
    }
    return Promise.resolve();
@@ -290,18 +293,18 @@ function degradable(func, fallback) {
        try {
            return await func.call(this, ...args);
        } catch (e) {
-            console.error("IndexedDBStore failure, degrading to MemoryStore", e);
+            logger.error("IndexedDBStore failure, degrading to MemoryStore", e);
            this.emit("degraded", e);
            try {
                // We try to delete IndexedDB after degrading since this store is only a
                // cache (the app will still function correctly without the data).
                // It's possible that deleting repair IndexedDB for the next app load,
                // potenially by making a little more space available.
-                console.log("IndexedDBStore trying to delete degraded data");
+                logger.log("IndexedDBStore trying to delete degraded data");
                await this.backend.clearDatabase();
-                console.log("IndexedDBStore delete after degrading succeeeded");
+                logger.log("IndexedDBStore delete after degrading succeeeded");
            } catch (e) {
-                console.warn("IndexedDBStore delete after degrading failed", e);
+                logger.warn("IndexedDBStore delete after degrading failed", e);
            }
            // Degrade the store from being an instance of `IndexedDBStore` to instead be
            // an instance of `MemoryStore` so that future API calls use the memory path
@@ -335,8 +335,10 @@ module.exports.MemoryStore.prototype = {

    /**
     * Save does nothing as there is no backing data store.
+     * @param {bool} force True to force a save (but the memory
+     *     store still can't save anything)
     */
-    save: function() {},
+    save: function(force) {},

    /**
     * Startup does nothing as this store doesn't require starting up.
@@ -22,6 +22,7 @@ limitations under the License.
 */

 const utils = require("../../utils");
+import logger from '../../logger';

 const DEBUG = false;  // set true to enable console logging.
 const E2E_PREFIX = "session.e2e.";
@@ -257,7 +258,7 @@ function removeByPrefix(store, prefix) {

 function debuglog() {
    if (DEBUG) {
-        console.log(...arguments);
+        logger.log(...arguments);
    }
 }

@@ -21,6 +21,7 @@ limitations under the License.
 */

 import utils from "./utils";
+import logger from './logger';


 /**
@@ -168,7 +169,7 @@ class SyncAccumulator {
                }
                break;
            default:
-                console.error("Unknown cateogory: ", category);
+                logger.error("Unknown cateogory: ", category);
        }
    }

@@ -32,6 +32,8 @@ const Group = require('./models/group');
 const utils = require("./utils");
 const Filter = require("./filter");
 const EventTimeline = require("./models/event-timeline");
+const PushProcessor = require("./pushprocessor");
+import logger from './logger';

 import {InvalidStoreError} from './errors';

@@ -58,7 +60,7 @@ function debuglog(...params) {
    if (!DEBUG) {
        return;
    }
-    console.log(...params);
+    logger.log(...params);
 }


@@ -126,7 +128,9 @@ SyncApi.prototype.createRoom = function(roomId) {
        timelineSupport,
        unstableClientRelationAggregation,
    });
-    client.reEmitter.reEmit(room, ["Room.name", "Room.timeline", "Room.redaction",
+    client.reEmitter.reEmit(room, ["Room.name", "Room.timeline",
+                          "Room.redaction",
+                          "Room.redactionCancelled",
                          "Room.receipt", "Room.tags",
                          "Room.timelineReset",
                          "Room.localEchoUpdated",
@@ -392,7 +396,7 @@ SyncApi.prototype._peekPoll = function(peekRoom, token) {
        peekRoom.addLiveEvents(events);
        self._peekPoll(peekRoom, res.end);
    }, function(err) {
-        console.error("[%s] Peek poll failed: %s", peekRoom.roomId, err);
+        logger.error("[%s] Peek poll failed: %s", peekRoom.roomId, err);
        setTimeout(function() {
            self._peekPoll(peekRoom, token);
        }, 30 * 1000);
@@ -454,7 +458,7 @@ SyncApi.prototype._wasLazyLoadingToggled = async function(lazyLoadMembers) {
 SyncApi.prototype._shouldAbortSync = function(error) {
    if (error.errcode === "M_UNKNOWN_TOKEN") {
        // The logout already happened, we just need to stop.
-        console.warn("Token no longer valid - assuming logout");
+        logger.warn("Token no longer valid - assuming logout");
        this.stop();
        return true;
    }
@@ -494,7 +498,7 @@ SyncApi.prototype.sync = function() {

            client.pushRules = result;
        } catch (err) {
-            console.error("Getting push rules failed", err);
+            logger.error("Getting push rules failed", err);
            if (self._shouldAbortSync(err)) return;
            // wait for saved sync to complete before doing anything else,
            // otherwise the sync state will end up being incorrect
@@ -522,7 +526,7 @@ SyncApi.prototype.sync = function() {
                    );
                    debuglog("Created and stored lazy load sync filter");
                } catch (err) {
-                    console.error(
+                    logger.error(
                        "Creating and storing lazy load sync filter failed",
                        err,
                    );
@@ -546,7 +550,7 @@ SyncApi.prototype.sync = function() {
            // we leave the state as 'ERROR' which isn't great since this normally means
            // we're retrying. The client must be stopped before clearing the stores anyway
            // so the app should stop the client, clear the store and start it again.
-            console.warn("InvalidStoreError: store is not usable: stopping sync.");
+            logger.warn("InvalidStoreError: store is not usable: stopping sync.");
            return;
        }
        if (this.opts.lazyLoadMembers && this.opts.crypto) {
@@ -557,7 +561,7 @@ SyncApi.prototype.sync = function() {
            await this.client._storeClientOptions();
            debuglog("Stored client options");
        } catch (err) {
-            console.error("Storing client options failed", err);
+            logger.error("Storing client options failed", err);
            throw err;
        }

@@ -580,7 +584,7 @@ SyncApi.prototype.sync = function() {
                getFilterName(client.credentials.userId), filter,
            );
        } catch (err) {
-            console.error("Getting filter failed", err);
+            logger.error("Getting filter failed", err);
            if (self._shouldAbortSync(err)) return;
            // wait for saved sync to complete before doing anything else,
            // otherwise the sync state will end up being incorrect
@@ -627,7 +631,7 @@ SyncApi.prototype.sync = function() {
                return self._syncFromCache(savedSync);
            }
        }).catch(err => {
-            console.error("Getting saved sync failed", err);
+            logger.error("Getting saved sync failed", err);
        });
        // Now start the first incremental sync request: this can also
        // take a while so if we set it going now, we can wait for it
@@ -699,7 +703,7 @@ SyncApi.prototype._syncFromCache = async function(savedSync) {
    try {
        await this._processSyncResponse(syncEventData, data);
    } catch(e) {
-        console.error("Error processing cached sync", e.stack || e);
+        logger.error("Error processing cached sync", e.stack || e);
    }

    // Don't emit a prepared if we've bailed because the store is invalid:
@@ -774,7 +778,7 @@ SyncApi.prototype._sync = async function(syncOptions) {
    } catch(e) {
        // log the exception with stack if we have it, else fall back
        // to the plain description
-        console.error("Caught /sync error", e.stack || e);
+        logger.error("Caught /sync error", e.stack || e);

        // Emit the exception for client handling
        this.client.emit("sync.unexpectedError", e);
@@ -888,15 +892,15 @@ SyncApi.prototype._onSyncError = function(err, syncOptions) {
        return;
    }

-    console.error("/sync error %s", err);
-    console.error(err);
+    logger.error("/sync error %s", err);
+    logger.error(err);

    if(this._shouldAbortSync(err)) {
        return;
    }

    this._failedSyncCount++;
-    console.log('Number of consecutive failed sync requests:', this._failedSyncCount);
+    logger.log('Number of consecutive failed sync requests:', this._failedSyncCount);

    debuglog("Starting keep-alive");
    // Note that we do *not* mark the sync connection as
@@ -1027,8 +1031,9 @@ SyncApi.prototype._processSyncResponse = async function(
                // honour push rules that were previously cached. Base rules
                // will be updated when we recieve push rules via getPushRules
                // (see SyncApi.prototype.sync) before syncing over the network.
-                if (accountDataEvent.getType() == 'm.push_rules') {
-                    client.pushRules = accountDataEvent.getContent();
+                if (accountDataEvent.getType() === 'm.push_rules') {
+                    const rules = accountDataEvent.getContent();
+                    client.pushRules = PushProcessor.rewriteDefaultRules(rules);
                }
                client.emit("accountData", accountDataEvent);
                return accountDataEvent;
@@ -1040,8 +1045,26 @@ SyncApi.prototype._processSyncResponse = async function(
    if (data.to_device && utils.isArray(data.to_device.events) &&
        data.to_device.events.length > 0
       ) {
+        const cancelledKeyVerificationTxns = [];
        data.to_device.events
            .map(client.getEventMapper())
+            .map((toDeviceEvent) => { // map is a cheap inline forEach
+                // We want to flag m.key.verification.start events as cancelled
+                // if there's an accompanying m.key.verification.cancel event, so
+                // we pull out the transaction IDs from the cancellation events
+                // so we can flag the verification events as cancelled in the loop
+                // below.
+                if (toDeviceEvent.getType() === "m.key.verification.cancel") {
+                    const txnId = toDeviceEvent.getContent()['transaction_id'];
+                    if (txnId) {
+                        cancelledKeyVerificationTxns.push(txnId);
+                    }
+                }
+
+                // as mentioned above, .map is a cheap inline forEach, so return
+                // the unmodified event.
+                return toDeviceEvent;
+            })
            .forEach(
                function(toDeviceEvent) {
                    const content = toDeviceEvent.getContent();
@@ -1050,13 +1073,21 @@ SyncApi.prototype._processSyncResponse = async function(
                            content.msgtype == "m.bad.encrypted"
                    ) {
                        // the mapper already logged a warning.
-                        console.log(
+                        logger.log(
                            'Ignoring undecryptable to-device event from ' +
                                toDeviceEvent.getSender(),
                        );
                        return;
                    }

+                    if (toDeviceEvent.getType() === "m.key.verification.start"
+                        || toDeviceEvent.getType() === "m.key.verification.request") {
+                        const txnId = content['transaction_id'];
+                        if (cancelledKeyVerificationTxns.includes(txnId)) {
+                            toDeviceEvent.flagCancelled();
+                        }
+                    }
+
                    client.emit("toDeviceEvent", toDeviceEvent);
                },
            );
@@ -1216,10 +1247,8 @@ SyncApi.prototype._processSyncResponse = async function(
            room.setSummary(joinObj.summary);
        }

-        // XXX: should we be adding ephemeralEvents to the timeline?
-        // It feels like that for symmetry with room.addAccountData()
-        // there should be a room.addEphemeralEvents() or similar.
-        room.addLiveEvents(ephemeralEvents);
+        // we deliberately don't add ephemeral events to the timeline
+        room.addEphemeralEvents(ephemeralEvents);

        // we deliberately don't add accountData to the timeline
        room.addAccountData(accountDataEvents);
@@ -19,6 +19,7 @@ limitations under the License.

 import Promise from 'bluebird';
 const EventTimeline = require("./models/event-timeline");
+import logger from './logger';

 /**
 * @private
@@ -28,7 +29,7 @@ const DEBUG = false;
 /**
 * @private
 */
-const debuglog = DEBUG ? console.log.bind(console) : function() {};
+const debuglog = DEBUG ? logger.log.bind(logger) : function() {};

 /**
 * the number of times we ask the server for more events before giving up
@@ -1,5 +1,6 @@
 /*
 Copyright 2015, 2016 OpenMarket Ltd
+Copyright 2019 The Matrix.org Foundation C.I.C.

 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -699,3 +700,11 @@ module.exports.globToRegexp = function(glob, extended) {
    }
    return pat;
 };
+
+module.exports.ensureNoTrailingSlash = function(url) {
+    if (url && url.endsWith("/")) {
+        return url.substr(0, url.length - 1);
+    } else {
+        return url;
+    }
+};
@@ -21,6 +21,7 @@ limitations under the License.
 */
 const utils = require("../utils");
 const EventEmitter = require("events").EventEmitter;
+import logger from '../logger';
 const DEBUG = true;  // set true to enable console logging.

 // events: hangup, error(err), replaced(call), state(state, oldState)
@@ -60,9 +61,9 @@ function MatrixCall(opts) {
    this.URL = opts.URL;
    // Array of Objects with urls, username, credential keys
    this.turnServers = opts.turnServers || [];
-    if (this.turnServers.length === 0) {
+    if (this.turnServers.length === 0 && this.client.isFallbackICEServerAllowed()) {
        this.turnServers.push({
-            urls: [MatrixCall.FALLBACK_STUN_SERVER],
+            urls: [MatrixCall.FALLBACK_ICE_SERVER],
        });
    }
    utils.forEach(this.turnServers, function(server) {
@@ -91,8 +92,8 @@ function MatrixCall(opts) {
 }
 /** The length of time a call can be ringing for. */
 MatrixCall.CALL_TIMEOUT_MS = 60000;
-/** The fallback server to use for STUN. */
-MatrixCall.FALLBACK_STUN_SERVER = 'stun:stun.l.google.com:19302';
+/** The fallback ICE server to use for STUN or TURN protocols. */
+MatrixCall.FALLBACK_ICE_SERVER = 'stun:turn.matrix.org';
 /** An error code when the local client failed to create an offer. */
 MatrixCall.ERR_LOCAL_OFFER_FAILED = "local_offer_failed";
 /**
@@ -195,7 +196,7 @@ MatrixCall.prototype.placeScreenSharingCall =
 * @param {string} queueId Arbitrary ID to track the chain of promises to be used
 */
 MatrixCall.prototype.playElement = function(element, queueId) {
-    console.log("queuing play on " + queueId + " and element " + element);
+    logger.log("queuing play on " + queueId + " and element " + element);
    // XXX: FIXME: Does this leak elements, given the old promises
    // may hang around and retain a reference to them?
    if (this.mediaPromises[queueId]) {
@@ -206,10 +207,10 @@ MatrixCall.prototype.playElement = function(element, queueId) {
        // these failures may be non-fatal (as in the case of unmounts)
        this.mediaPromises[queueId] =
            this.mediaPromises[queueId].then(function() {
-                console.log("previous promise completed for " + queueId);
+                logger.log("previous promise completed for " + queueId);
                return element.play();
            }, function() {
-                console.log("previous promise failed for " + queueId);
+                logger.log("previous promise failed for " + queueId);
                return element.play();
            });
    } else {
@@ -224,14 +225,14 @@ MatrixCall.prototype.playElement = function(element, queueId) {
 * @param {string} queueId Arbitrary ID to track the chain of promises to be used
 */
 MatrixCall.prototype.pauseElement = function(element, queueId) {
-    console.log("queuing pause on " + queueId + " and element " + element);
+    logger.log("queuing pause on " + queueId + " and element " + element);
    if (this.mediaPromises[queueId]) {
        this.mediaPromises[queueId] =
            this.mediaPromises[queueId].then(function() {
-                console.log("previous promise completed for " + queueId);
+                logger.log("previous promise completed for " + queueId);
                return element.pause();
            }, function() {
-                console.log("previous promise failed for " + queueId);
+                logger.log("previous promise failed for " + queueId);
                return element.pause();
            });
    } else {
@@ -250,15 +251,15 @@ MatrixCall.prototype.pauseElement = function(element, queueId) {
 * @param {string} queueId Arbitrary ID to track the chain of promises to be used
 */
 MatrixCall.prototype.assignElement = function(element, srcObject, queueId) {
-    console.log("queuing assign on " + queueId + " element " + element + " for " +
+    logger.log("queuing assign on " + queueId + " element " + element + " for " +
        srcObject);
    if (this.mediaPromises[queueId]) {
        this.mediaPromises[queueId] =
            this.mediaPromises[queueId].then(function() {
-                console.log("previous promise completed for " + queueId);
+                logger.log("previous promise completed for " + queueId);
                element.srcObject = srcObject;
            }, function() {
-                console.log("previous promise failed for " + queueId);
+                logger.log("previous promise failed for " + queueId);
                element.srcObject = srcObject;
            });
    } else {
@@ -664,7 +665,7 @@ MatrixCall.prototype._maybeGotUserMediaForAnswer = function(stream) {
        },
    };
    self.peerConn.createAnswer(function(description) {
-        debuglog("Created answer: " + description);
+        debuglog("Created answer: ", description);
        self.peerConn.setLocalDescription(description, function() {
            self._answerContent = {
                version: 0,
@@ -753,7 +754,7 @@ MatrixCall.prototype._receivedAnswer = function(msg) {
 */
 MatrixCall.prototype._gotLocalOffer = function(description) {
    const self = this;
-    debuglog("Created offer: " + description);
+    debuglog("Created offer: ", description);

    if (self.state == 'ended') {
        debuglog("Ignoring newly created offer on call ID " + self.callId +
@@ -1159,7 +1160,7 @@ const callError = function(code, msg) {

 const debuglog = function() {
    if (DEBUG) {
-        console.log(...arguments);
+        logger.log(...arguments);
    }
 };

@@ -1216,24 +1217,9 @@ const _placeCallWithConstraints = function(self, constraints) {
 };

 const _createPeerConnection = function(self) {
-    let servers = self.turnServers;
-    if (self.webRtc.vendor === "mozilla") {
-        // modify turnServers struct to match what mozilla expects.
-        servers = [];
-        for (let i = 0; i < self.turnServers.length; i++) {
-            for (let j = 0; j < self.turnServers[i].urls.length; j++) {
-                servers.push({
-                    url: self.turnServers[i].urls[j],
-                    username: self.turnServers[i].username,
-                    credential: self.turnServers[i].credential,
-                });
-            }
-        }
-    }
-
    const pc = new self.webRtc.RtcPeerConnection({
        iceTransportPolicy: self.forceTURN ? 'relay' : undefined,
-        iceServers: servers,
+        iceServers: self.turnServers,
    });
    pc.oniceconnectionstatechange = hookCallback(self, self._onIceConnectionStateChanged);
    pc.onsignalingstatechange = hookCallback(self, self._onSignallingStateChanged);
@@ -1351,7 +1337,9 @@ module.exports.setVideoInput = function(deviceId) { videoInput = deviceId; };
 * @param {MatrixClient} client The client instance to use.
 * @param {string} roomId The room the call is in.
 * @param {Object?} options DEPRECATED optional options map.
- * @param {boolean} options.forceTURN DEPRECATED whether relay through TURN should be forced. This option is deprecated - use opts.forceTURN when creating the matrix client since it's only possible to set this option on outbound calls.
+ * @param {boolean} options.forceTURN DEPRECATED whether relay through TURN should be
+ * forced. This option is deprecated - use opts.forceTURN when creating the matrix client
+ * since it's only possible to set this option on outbound calls.
 * @return {MatrixCall} the call or null if the browser doesn't support calling.
 */
 module.exports.createNewMatrixCall = function(client, roomId, options) {
@@ -1382,24 +1370,36 @@ module.exports.createNewMatrixCall = function(client, roomId, options) {
            return getUserMedia.apply(w.navigator, arguments);
        };
    }
-    webRtc.RtcPeerConnection = (
-        w.RTCPeerConnection || w.webkitRTCPeerConnection || w.mozRTCPeerConnection
-    );
-    webRtc.RtcSessionDescription = (
-        w.RTCSessionDescription || w.webkitRTCSessionDescription ||
-        w.mozRTCSessionDescription
-    );
-    webRtc.RtcIceCandidate = (
-        w.RTCIceCandidate || w.webkitRTCIceCandidate || w.mozRTCIceCandidate
-    );
-    webRtc.vendor = null;
-    if (w.mozRTCPeerConnection) {
-        webRtc.vendor = "mozilla";
-    } else if (w.webkitRTCPeerConnection) {
-        webRtc.vendor = "webkit";
-    } else if (w.RTCPeerConnection) {
-        webRtc.vendor = "generic";
+
+    // Firefox throws on so little as accessing the RTCPeerConnection when operating in
+    // a secure mode. There's some information at https://bugzilla.mozilla.org/show_bug.cgi?id=1542616
+    // though the concern is that the browser throwing a SecurityError will brick the
+    // client creation process.
+    try {
+        webRtc.RtcPeerConnection = (
+            w.RTCPeerConnection || w.webkitRTCPeerConnection || w.mozRTCPeerConnection
+        );
+        webRtc.RtcSessionDescription = (
+            w.RTCSessionDescription || w.webkitRTCSessionDescription ||
+            w.mozRTCSessionDescription
+        );
+        webRtc.RtcIceCandidate = (
+            w.RTCIceCandidate || w.webkitRTCIceCandidate || w.mozRTCIceCandidate
+        );
+        webRtc.vendor = null;
+        if (w.mozRTCPeerConnection) {
+            webRtc.vendor = "mozilla";
+        } else if (w.webkitRTCPeerConnection) {
+            webRtc.vendor = "webkit";
+        } else if (w.RTCPeerConnection) {
+            webRtc.vendor = "generic";
+        }
+    } catch (e) {
+        logger.error("Failed to set up WebRTC object: possible browser interference?");
+        logger.error(e);
+        return null;
    }
+
    if (!webRtc.RtcIceCandidate || !webRtc.RtcSessionDescription ||
            !webRtc.RtcPeerConnection || !webRtc.getUserMedia) {
        return null; // WebRTC is not supported.