Commit graph

975 commits

Author SHA1 Message Date
Alex Good
30b4f1d02e
Fix lifetimes on list_range_at 2023-03-11 00:47:19 +00:00
Orion Henry
c3ceda48e9 first attempt at an api 2023-03-10 18:23:00 -06:00
Orion Henry
4990ca9223 first draft of diff() api 2023-03-10 14:32:21 -06:00
Orion Henry
271d5cbead cleanup based on alex's comments 2023-03-10 12:57:13 -06:00
Orion Henry
aa0fdc7d2d Merge remote-tracking branch 'origin/main' into marks_port 2023-03-10 12:06:58 -06:00
alexjg
cb409b6ffe
docs: timestamp -> time in automerge.change examples (#548) 2023-03-09 18:10:23 +00:00
Conrad Irwin
b34b46fa16
smaller automerge c (#545)
* Fix automerge-c tests on mac

* Generate significantly smaller automerge-c builds

This cuts the size of libautomerge_core.a from 25Mb to 1.6Mb on macOS
and 53Mb to 2.7Mb on Linux.

As a side-effect of setting codegen-units = 1 for all release builds the
optimized wasm files are also 100kb smaller.
2023-03-09 15:09:43 +00:00
Conrad Irwin
7b747b8341
Error instead of corrupt large op counters (#543)
Since b78211ca6, OpIds have been silently truncated to 2**32. This
causes corruption in the case the op id overflows.

This change converts the silent error to a panic, and guards against the
panic on the codepath found by the fuzzer.
2023-03-07 16:49:04 +00:00
Conrad Irwin
2c1970f664
Fix panic on invalid action (#541)
We make the validation on parsing operations in the encoded changes stricter to avoid a possible panic when applying changes.
2023-03-04 12:09:08 +00:00
christine betts
63b761c0d1
Suppress clippy warning in parse.rs + bump toolchain (#542)
* Fix rust error in parse.rs
* Bump toolchain to 1.67.0
2023-03-03 22:42:40 +00:00
Orion Henry
c38d9e883e add unmark() and marks() to unstable api 2023-03-01 10:54:28 -06:00
Orion Henry
2f4cf7b328 enable load callback test 2023-02-28 18:54:03 -06:00
Orion Henry
1c28e9656a add get_marks_at() 2023-02-28 17:17:14 -06:00
Orion Henry
8e818910d1 Merge remote-tracking branch 'origin/main' into marks_port 2023-02-28 14:47:44 -06:00
Orion Henry
ba491c6f72 move wasm observer into automerge 2023-02-28 13:25:07 -06:00
Conrad Irwin
44fa7ac416
Don't panic on missing deps of change chunks (#538)
* Fix doubly-reported ops in load of change chunks

Since c3c04128f5, observers have been
called twice when calling Automerge::load() with change chunks.

* Better handle change chunks with missing deps

Before this change Automerge::load would panic if you passed a change
chunk that was missing a dependency, or multiple change chunks not in
strict dependency order. After this change these cases will error
instead.
2023-02-27 20:12:09 +00:00
Orion Henry
1a539a3d79 add unmark / remove spans/raw_spans/attr 2023-02-27 12:33:55 -06:00
Jason Kankiewicz
8de2fa9bd4
C API 2 (#530)
The AMvalue union, AMlistItem struct, AMmapItem struct, and AMobjItem struct are gone, replaced by the AMitem struct.

The AMchangeHashes, AMchanges, AMlistItems, AMmapItems, AMobjItems, AMstrs, and AMsyncHaves iterators are gone, replaced by the AMitems iterator.

The AMitem struct is opaque, getting and setting values is now achieved exclusively through function calls.

The AMitemsNext(), AMitemsPrev(), and AMresultItem() functions return a pointer to an AMitem struct so you ultimately get the same thing whether you're iterating over a sequence or calling AMmapGet() or AMlistGet().

Calling AMitemResult() on an AMitem struct will produce a new AMresult struct referencing its storage so now the AMresult struct for an iterator can be subsequently freed without affecting the AMitem structs that were filtered out of it.

The storage for a set of AMitem structs can be recombined into a single AMresult struct by passing pointers to their corresponding AMresult structs to AMresultCat().

For C/C++ programmers, I've added AMstrCmp(), AMstrdup(), AM{idxType,objType,status,valType}ToString() and AM{idxType,objType,status,valType}FromString(). It's also now possible to pass arbitrary parameters through AMstack{Item,Items,Result}() to a callback function.
2023-02-25 18:47:00 +00:00
Orion Henry
af9b006bb0 marks name->key, range->start,end, patch callback propogates expceitons and has heads 2023-02-24 17:57:45 -06:00
Orion Henry
01c721e640 remove a fixme 2023-02-24 14:10:29 -06:00
Orion Henry
3beccfb5ee track mark data by reference 2023-02-24 13:38:10 -06:00
Orion Henry
9a6840392e let insert take a conflict flag 2023-02-24 12:26:11 -06:00
Orion Henry
d7d2c29dc7
Merge branch 'main' into marks_port 2023-02-24 09:59:19 -08:00
Orion Henry
2c6e54390b rewrote observe_current_state - added get_marks() api call - cleaned up js interface 2023-02-24 11:36:14 -06:00
Orion Henry
d7f93c5aca get mark patches working on load 2023-02-23 17:51:45 -06:00
Orion Henry
e1d81e01fc mark patch callbacks and js mark api 2023-02-16 19:35:39 -06:00
Philip Schatz
407faefa6e
A few setup fixes (#529)
* include deno in dependencies

* install javascript dependencies

* remove redundant operation
2023-02-15 09:23:02 +00:00
Alex Good
1425af43cd @automerge/automerge@2.0.2 2023-02-15 00:06:23 +00:00
Alex Good
c92d042c87 @automerge/automerge-wasm@0.1.24 and @automerge/automerge@2.0.2-alpha.2 2023-02-14 17:59:23 +00:00
Alex Good
c6a32d8368 Correct logic when skip = B and fix formatting
A few tests were failing which exposed the fact that if skip is `B` (the
out factor of the OpTree) then we set `skip = None` and this causes us
to attempt to return `Skip` in a non root node. I ported the failing
test from JS to Rust and fixed the problem.

I also fixed the formatting issues.
2023-02-14 11:29:55 -06:00
Orion Henry
a9612371e0 rework how skip works to push the logic into node 2023-02-14 11:29:55 -06:00
Conrad Irwin
e2bb0eb6b9 Use our leb128 parser for values
This ensures that values in automerge documents are encoded correctly,
and that no extra data is smuggled in any LEB fields.
2023-02-14 11:29:55 -06:00
Alex Good
02e8ae2c70 Remove nightly from CI 2023-02-14 11:29:55 -06:00
Alex Good
9271b20cf5 Correct logic when skip = B and fix formatting
A few tests were failing which exposed the fact that if skip is `B` (the
out factor of the OpTree) then we set `skip = None` and this causes us
to attempt to return `Skip` in a non root node. I ported the failing
test from JS to Rust and fixed the problem.

I also fixed the formatting issues.
2023-02-14 17:21:59 +00:00
Orion Henry
5e82dbc3c8 rework how skip works to push the logic into node 2023-02-14 17:21:59 +00:00
Orion Henry
a006a32e3f get tests passing + unmark 2023-02-13 10:56:05 -06:00
Conrad Irwin
2cd7427f35 Use our leb128 parser for values
This ensures that values in automerge documents are encoded correctly,
and that no extra data is smuggled in any LEB fields.
2023-02-09 15:46:22 +00:00
Alex Good
a02f70f2b8
Use new columns instead of existing ones
The previous approach of using the key and insert columns of existing
ops was leading to quite confusing code. There's no real cost to
introducing new columns so I've switched the code to do that instead.

Introduce an `expand` and a `mark_name` column. `expand` is a boolean
column and `mark_name` is a RLE encoded string column. Neither of these
columns are encoded if they are empty.

Also move the `MarkData::name` property to use strings interned in
`OpSetMetadata::props` rather than representing the string directly on
the basis that we probably will have a lot of repeated mark names and
we do a bunch of equality checks on them while searching so this will
probably speed things up a bit.

Introduced a new `MaybeBooleanEncoder` (and associated `MaybeBooleanDecoder` and
`MaybeBooleanRange`) types to represent a boolean column which is
entirely skipped if all it contains are `false` values. This allows us
to omit encoding the `expand` column for groups of ops which only ever
set it to `false` which in turn makes us backwards compatible when not
using marks.
2023-02-09 15:09:52 +00:00
Alex Good
11f063cbfe
Remove nightly from CI 2023-02-09 11:06:24 +00:00
Alex Good
f281213a47
tests passing 2023-02-09 10:55:19 +00:00
Alex Good
a44ceacb1c
everything compiles 2023-02-09 10:26:26 +00:00
Orion Henry
290c9e6872 attempt to finish - two issues outstanding 2023-02-08 14:59:41 -06:00
Alex Good
9a7dba09a4
wip 2023-02-08 18:20:38 +00:00
Alex Good
2345176526
DocOp 2023-02-08 17:53:58 +00:00
Orion Henry
9bc424d776 first test passing - needs serialization 2023-02-08 10:43:33 -06:00
Orion Henry
61f9604d0c merge with main 2023-02-06 11:34:13 -06:00
Alex Good
a24d536d16 Move automerge::SequenceTree to automerge_wasm::SequenceTree
The `SequenceTree` is only ever used in `automerge_wasm` so move it
there.
2023-02-05 11:08:33 +00:00
Alex Good
c5fde2802f @automerge/automerge-wasm@0.1.24 and @automerge/automerge@2.0.2-alpha.1 2023-02-03 16:31:46 +00:00
Alex Good
13a775ed9a Speed up loading by generating clocks on demand
Context: currently we store a mapping from ChangeHash -> Clock, where
`Clock` is the set of (ActorId, (Sequence number, max Op)) pairs derived
from the given change and it's dependencies. This clock is used to
determine what operations are visible at a given set of heads.

Problem: populating this mapping for documents with large histories
containing many actors can be very slow as for each change we have to
allocate and merge a bunch of hashmaps.

Solution: instead of creating the clocks on load, create an adjacency
list based representation of the change graph and then derive the clock
from this graph when it is needed. Traversing even large graphs is still
almost as fast as looking up the clock in a hashmap.
2023-02-03 16:15:15 +00:00
Alex Good
1e33c9d9e0 Use Automerge::load instead of load_incremental if empty
Problem: when running the sync protocol for a new document the API
requires that the user create an empty document and then call
`receive_sync_message` on that document. This results in the OpObserver
for the new document being called with every single op in the document
history. For documents with a large history this can be extremely time
consuming, but the OpObserver doesn't need to know about all the hidden
states.

Solution: Modify `Automerge::load_with` and
`Automerge::apply_changes_with` to check if the document is empty before
applying changes. If the document _is_ empty then we don't call the
observer for every change, but instead use
`automerge::observe_current_state` to notify the observer of the new
state once all the changes have been applied.
2023-02-03 10:01:12 +00:00