Commit graph

397 commits

Author SHA1 Message Date
Alex Good
3d5590f90a wip 2021-05-13 18:39:05 +01:00
Alex Good
7ccaae92fd Add test for updates inside conflicted list elements 2021-05-12 23:00:03 +01:00
Andrew Jeffery
5713f4263f Update op id 2021-05-12 16:29:24 +01:00
Andrew Jeffery
576a2736f6 Sort diffs by actor id 2021-05-12 15:11:14 +01:00
Andrew Jeffery
31e245e3b7 Handle conflicts with multiple updates 2021-05-12 14:29:14 +01:00
Andrew Jeffery
833149d77f Fix returning an operation for increments 2021-05-12 13:42:06 +01:00
Andrew Jeffery
e5f70ddcb6 Allow del in gen_seq_diff 2021-05-11 22:42:04 +01:00
Andrew Jeffery
e2ec827d66 Ensure we execute diffs for nested objects from lists 2021-05-11 20:45:28 +01:00
Andrew Jeffery
4242415f5b Return empty mapdiff when no root object in pending 2021-05-11 17:01:11 +01:00
Andrew Jeffery
94c02f061c Use actual op.id in update 2021-05-11 16:21:02 +01:00
Andrew Jeffery
cf1ad2c5f8 Use proper key for elem_id 2021-05-11 16:13:37 +01:00
Andrew Jeffery
e9cfd205f6 Remove unused drain 2021-05-11 16:13:22 +01:00
Andrew Jeffery
a5dee2867c Add debug to edits struct 2021-05-11 16:13:08 +01:00
Andrew Jeffery
9ace8892f6 Remove unchanged diff
Uses empty edits or props instead.
2021-05-11 12:48:45 +01:00
alexjg
e92188104f
Ensure that ChangeBytes::compressed contains the original compressed bytes (#112)
* Ensure that ChangeBytes::compressed contains the original compressed bytes, fixes #95

* Fix clippy

* Move bytes into decompress check

Co-authored-by: Andrew Jeffery <dev@jeffas.io>
2021-05-11 10:33:12 +01:00
Andrew Jeffery
765c5287d7
Add event handler logic (#116)
* Add event handlers

Remove partialeq from Backend since I'm not sure of its relevance or
correctness anyway and complicates things with the event handlers.

* Add docs and use id

* Remove partialeq

* Add docs for enum variants

* Check index on remove_handler

* Pass functions through rather than making field pub
2021-05-11 10:15:37 +01:00
Andrew Jeffery
6050f39d46 Change patch diffs to start with mapdiff 2021-05-11 09:26:19 +01:00
Andrew Jeffery
a512728649 Fix expansion of multi delete 2021-05-11 08:38:09 +01:00
Andrew Jeffery
497f554b02 Fix bug in expanded op iterator logic and add tests 2021-05-10 23:25:46 +01:00
Andrew Jeffery
59a24081d2 Switch to expanded_op more 2021-05-10 19:25:43 +01:00
Andrew Jeffery
d794ad4dd5 Update diffedit naming of fields and add comments
This brings the DiffEdit types more in line with the typescript ones and
fixes the names. Also changed to using match guards in the append_edit
function so we can take edit by value.
2021-05-10 13:19:48 +01:00
Andrew Jeffery
9c522cb587 Handle collapsing inserts in edits 2021-05-10 12:27:28 +01:00
Andrew Jeffery
7a08c52dac
Fix clippy (#118) 2021-05-07 17:07:41 +01:00
Andrew Jeffery
9fb8666028
Merge pull request #107 from automerge/encode-ref
SyncState: encode can take a ref
2021-05-07 12:01:51 +01:00
Andrew Jeffery
708db48fbb Fixup after rebase 2021-05-05 21:58:18 +01:00
Alex Good
e86aabb6a3 Implement new sequence patch format 2021-05-05 21:52:57 +01:00
Andrew Jeffery
d2898fd0d7 Add clippy lints 2021-05-04 20:54:50 +01:00
Andrew Jeffery
825cc92b51 Delay cloning until the end 2021-05-04 20:15:25 +01:00
Andrew Jeffery
6da6480c9b Move change into update_history to avoid a clone 2021-05-04 20:04:59 +01:00
Andrew Jeffery
8254ba6231 Change states to use indices into history
This moves to storing the changes only in history and everything else
pointing to that. This reduces duplication and so memory usage,
especially when we may have large changes or documents.
2021-05-04 20:00:00 +01:00
Andrew Jeffery
8f6e663d99 SyncState: encode can take a ref 2021-04-27 15:08:17 +01:00
Andrew Jeffery
3a8447c068
Remove automerge bench harness (#106)
This removes the default wrapper which throws errors with some cli
arguments, e.g. --save-baseline which is nice for being able to compare
versions!
2021-04-27 10:30:07 +01:00
Andrew Jeffery
f18066bb10
Fix formatting (#105) 2021-04-26 21:09:11 +01:00
Orion Henry
7252596691 formatting change 2021-04-26 11:06:41 -07:00
Orion Henry
0e08f00c34 remove direct array access 2021-04-26 11:06:41 -07:00
Orion Henry
98537d9b80 get changes fast 2021-04-26 11:06:41 -07:00
Andrew Jeffery
d1934f2286 Add formatting config 2021-04-26 09:42:36 -07:00
Andrew Jeffery
63d09d5c7e Change count back to isize 2021-04-26 09:40:27 -07:00
Andrew Jeffery
0305acbd49 Change unwrap_or_default to unwrap
Better to not silently ignore the error
2021-04-26 09:40:27 -07:00
Andrew Jeffery
94e0fd7e49 encoding: add documentation and use i64 for rledecoder count 2021-04-26 09:40:27 -07:00
Andrew Jeffery
18ce93603a encodable: Require encode function implementation 2021-04-25 15:19:33 -07:00
Andrew Jeffery
3a8baf7b0a Filter sent hashes down to only what they haven't seen
This means we no longer need to store an unbounded list of hashes while
syncing. This is possible as we know their heads and all changes we send
should be known to us, thus we can look in our history to see whether
they are safely applied at the other end.
2021-04-25 13:40:06 -07:00
Andrew Jeffery
8bb14b73d5 Remove unnecessary conditional
We are deduplicating so it shouldn't matter if either is empty.
2021-04-24 14:25:31 -07:00
Andrew Jeffery
8d3096b823 Make sent_hashes a set instead of a vec
We don't care about the ordering so we can just store them in a hashset
all of the time, avoiding having to rebuild the hashset each time.
2021-04-24 14:25:31 -07:00
Andrew Jeffery
e3de7d970a Only store hashes in sync_state rather than the actual changes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
701f63c624 Separate benchmark into sync and sync with changes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
4c7e0c9167 Mutate the SyncState rather than consuming and returning 2021-04-24 14:25:31 -07:00
Andrew Jeffery
90cbbc7cab Fix ordering of receive_sync_message arguments 2021-04-24 14:25:31 -07:00
Andrew Jeffery
1f338393c3 Move SyncHave to state.rs 2021-04-24 14:25:31 -07:00
Andrew Jeffery
7c2fd94c62 Specify some capacities 2021-04-24 14:25:31 -07:00
Andrew Jeffery
e58ba342f8 Move state to separate module 2021-04-24 14:25:31 -07:00
Andrew Jeffery
21d19cc909 Move bloom filter to separate module 2021-04-24 14:25:31 -07:00
Andrew Jeffery
aebfdc6572 Update patches with pending_changes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
d61f1e1c6e Remove our_need and fix default SyncState 2021-04-24 14:25:31 -07:00
Andrew Jeffery
0a377a858b Catch up with js changes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
6b424d6e97 Take ownership of have 2021-04-24 14:25:31 -07:00
Andrew Jeffery
220580506d Performance updates
Where we do lots of contains checks we should use hashsets rather than
linear scan through a Vec.

Also tried to minimise some copying.
2021-04-24 14:25:31 -07:00
Andrew Jeffery
f8df532d25 Don't have an old_sync_state, just a sync_state 2021-04-24 14:25:31 -07:00
Andrew Jeffery
0655f7ade6 Remove unused log macro 2021-04-24 14:25:31 -07:00
Andrew Jeffery
7c3f52f2e3 Fix need request when head is a false positive 2021-04-24 14:25:31 -07:00
Andrew Jeffery
62f6c6d288 Fix updating sharedHeads when a subset of changes is sent 2021-04-24 14:25:31 -07:00
Andrew Jeffery
6b6135d108 Require vec rather than slice 2021-04-24 14:25:31 -07:00
Andrew Jeffery
85f53685e2 Move to binary representation of changes in api 2021-04-24 14:25:31 -07:00
Andrew Jeffery
7d13091393 Fix bug in get_missing_deps 2021-04-24 14:25:31 -07:00
Andrew Jeffery
68ddb2504e Improve js interop 2021-04-24 14:25:31 -07:00
Andrew Jeffery
e1bdf34471 More borrowing 2021-04-24 14:25:31 -07:00
Andrew Jeffery
00dcd531c1 Use more references 2021-04-24 14:25:31 -07:00
Andrew Jeffery
3fd548cbe1 Use nicer syntax for repeat 2021-04-24 14:25:31 -07:00
Andrew Jeffery
7573c58521 Implement encodable for slice of change hashes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
69bbd8c64d Remove clone 2021-04-24 14:25:31 -07:00
Andrew Jeffery
469b40e371 Remove unnecessary brackets 2021-04-24 14:25:31 -07:00
Andrew Jeffery
34ede85699 Inline probe as usize 2021-04-24 14:25:31 -07:00
Andrew Jeffery
4dd2d42629 Use builtin function to build u32 from bytes 2021-04-24 14:25:31 -07:00
Andrew Jeffery
9cd146a464 Advance heads only needs slice 2021-04-24 14:25:31 -07:00
Andrew Jeffery
8b4632ae07 Use SyncHave::default 2021-04-24 14:25:31 -07:00
Andrew Jeffery
c5d3572670 Remove unwraps in sync 2021-04-24 14:25:31 -07:00
Andrew Jeffery
4b7bbe6710 Rename peer to sync 2021-04-24 14:25:31 -07:00
Andrew Jeffery
cd56799e1c Move all to sync 2021-04-24 14:25:31 -07:00
Andrew Jeffery
766b5621d4 Add more sync work 2021-04-24 14:25:31 -07:00
Andrew Jeffery
3eeb825138 Initial sync work 2021-04-24 14:25:31 -07:00
Andrew Jeffery
137e6d2808 Fix clippy lints 2021-04-07 08:07:53 -07:00
Andrew Jeffery
342504714a Format 2021-04-07 08:07:53 -07:00
Martin Kleppmann
38214ca21e
Merge pull request #65 from automerge/implement-compression
Implement compression
2021-04-02 17:47:39 +01:00
Martin Kleppmann
f63fbe6156 Fix release build 2021-04-02 17:40:52 +01:00
Gregory
f6b3964733 backend: remove unecessary draining of changes to apply. 2021-04-01 09:19:32 -07:00
Andrew Jeffery
d17554dfc4 Remove rc on backend 2021-03-31 08:59:38 -07:00
Andrew Jeffery
02e1a6b6cc Simplify if statement to remove unreachable case
If `op_by_id` contains `succ` then we can get the index straight away,
else we can do the original work. This means we don't need an `else`
case for a third situation as in reality there isn't one.
2021-03-30 09:01:03 -07:00
Andrew Jeffery
3f5a781c71
Simplify while stack not empty (#68) 2021-03-28 21:28:37 +01:00
Alex Good
82cf04ae49 Implement compression 2021-03-27 16:52:43 +00:00
Andrew Jeffery
1e36b62272
Check the checksum in the document on decoding (#62) 2021-03-26 13:59:03 +00:00
Andrew Jeffery
921e9859b2
Fix issue in RleDecoder with null runs (#63)
* Add test to check save and load works on failing input

We loop 100 times due to it being a flaky test, likely due to an
iteration order issue in hashmaps.

* Update printing diff of changes

* Add new case for save then load

* Add another failing test case for save_load

* Update test case

* Update test case to not use empty strings in keys

* Specify to decode into usize in rle decoder

The `decoder.read` call was previously inferring to use isize since the
`self.count` variable is that type. This meant that for a list of null
values >= 64 in length it would read it as negative and so enter an
infinite loop of reading nulls, even if there were other values to read.

This changes the read to use usize but introduces the issue of usize
potentially being bigger than isize. This should be very unlikely for
what it is currently used for though so I'll defer this to later work.

* Add a test case to check the repeated nulls issue in rle

* Update tests

* Specify reading rle len as i64 instead of isize
2021-03-26 13:57:08 +00:00
Alex Good
c103b0638e Ritual abasement at the altar of clippy 2021-03-02 19:35:00 +00:00
Alex Good
4f4674a4a1 Implement cursors in frontend 2021-03-02 16:07:52 +00:00
Alex Good
93d45c381d Run rustfmt 2021-02-15 15:04:05 +00:00
Alex Good
33328796cd Remove mutable reference to actors in OpSet::finalize_diffs 2021-02-12 12:03:10 -05:00
Alex Good
08f324f7e6 Merge ref encoding/decoding and value encoding/decoding 2021-02-12 12:03:10 -05:00
Alex Good
26dfc7ea40 Implement cursor logic in backend 2021-02-12 12:03:10 -05:00
Alex Good
2d31952d77 Implement encoding and decoding of cursor operations 2021-02-12 12:03:10 -05:00
Orion Henry
ef38e6d7da getting some WASM errors trying to access crypto in the browser - switching to SmallRng with a 0 seed for now - deterministic distribution of nodes in the skip list should be fine 2021-02-12 11:49:06 -05:00
Matt Ho
0c44edd919
fixed typos (#43) 2021-02-01 10:57:11 +00:00