Commit graph

120 commits

Author SHA1 Message Date
Conrad Irwin
931ee7e77b
Add Fuzz Testing (#498)
* Add fuzz testing for document load

* Fix fuzz crashers and add to test suite
2023-01-25 16:03:05 +00:00
alexjg
819767cc33
fix: use saturating_sub when updating cached text width (#505)
Problem: In `automerge::query::Index::change_vis` we use `-=` to
subtract the width of an operation which is being hidden from the text
widths which we store on the index of each node in the optree. This
index represents the width of all the visible text operations in this
node and below. This was causing an integer underflow error when
encountering some list operations. More specifically, when a
`ScalarValue::Str` in a list was made invisible by a later operation
which contained a _shorter_ string, the width subtracted from the indexed
text widths could be longer than the current index.

Solution: use `saturating_sub` instead. This is technically papering
over the problem because really the width should never go below zero,
but the text widths are only relevant for text objects where the
existing logic works as advertised because we don't have a `set`
operation for text indices. A more robust solution would be to track the
type of the Index (and consequently of the `OpTree`) at the type level,
but time is limited and problems are infinite.

Also, add a lengthy description of the reason we are using
`saturating_sub` so that when I read it in about a month I don't have
to redo the painful debugging process that got me to this commit.
2023-01-23 19:19:55 +00:00
Alex Currie-Clark
78adbc4ff9
Update patch types (#499)
* Update `Patch` types

* Clarify that the splice patch applies to text

* Add Splice patch type to exports

* Add new patches to javascript
2023-01-23 17:02:02 +00:00
Andrew Jeffery
1f7b109dcd
Add From<SmolStr> for ScalarValue::Str (#506) 2023-01-23 17:01:41 +00:00
Conrad Irwin
98e755106f
Fix and simplify lebsize calculations (#503)
Before this change numbits_i64() was incorrect for every value of the
form 0 - 2^x. This only manifested in a visible error if x%7 == 6 (so
for -64, -8192, etc.) at which point `lebsize` would return a value one
too large, causing a panic in commit().
2023-01-23 11:01:05 +00:00
Orion Henry
d745685f5e wip 2023-01-20 13:38:53 -06:00
alexjg
6b0ee6da2e
Bump js to 2.0.1-alpha.5 and automerge-wasm to 0.1.22 (#497) 2023-01-19 22:15:06 +00:00
alexjg
9b44a75f69
fix: don't panic when generating parents for hidden objects (#500)
Problem: the `OpSet::export_key` method uses `query::ElemIdPos` to
determine the index of sequence elements when exporting a key. This
query returned `None` for invisible elements. The `Parents` iterator
which is used to generate paths to objects in patches in
`automerge-wasm` used `export_key`. The end result is that applying a
remote change which deletes an object in a sequence would panic as it
tries to generate a path for an invisible object.

Solution: modify `query::ElemIdPos` to include invisible objects. This
does mean that the path generated will refer to the previous visible
object in the sequence as it's index, but this is probably fine as for
an invisible object the path shouldn't be used anyway.

While we're here also change the return value of `OpSet::export_key` to
an `Option` and make `query::Index::ops` private as obeisance to the
Lady of the Golden Blade.
2023-01-19 21:11:36 +00:00
alexjg
d8baa116e7
automerge-rs: Add ExId::to_bytes (#491)
The `ExId` structure has some internal details which make lookups for
object IDs which were produced by the document doing the looking up
faster. These internal details are quite specific to the implementation
so we don't want to expose them as a public API. On the other hand, we
need to be able to serialize `ExId`s so that FFI clients can hold on to
them without referencing memory which is owned by the document (ahem,
looking at you Java).

Introduce `ExId::to_bytes` and `TryFrom<&[u8]> ExId` implementing a
canonical serialization which includes a version tag, giveing us
compatibility options if we decide to change the implementation.
2023-01-19 17:02:47 +00:00
alexjg
964ae2bd81
Fix SeekOpWithPatch on optrees with only internal optrees (#496)
In #480 we fixed an issue where `SeekOp` calculated an incorrect
insertion index on optrees where the only visible ops were on internal
nodes. We forgot to port this fix to `SeekOpWithPatch`, which has almost
the same logic just with additional work done in order to notify an
`OpObserver` of changes. Add a test and fix to `SeekOpWithPatch`
2023-01-14 11:27:48 +00:00
Alex Good
22e9915fac automerge-wasm: publish release build in Github Action 2023-01-12 12:42:19 +00:00
Alex Good
5c02445bee
Bump automerge-wasm, again
In order to re-trigger the release action we are testing we bump the
version which was de-bumped in the last commit.
2023-01-12 10:39:11 +00:00
Alex Good
3ef60747f4
Roll back automerge-wasm to test release action
The release action we are working conditionally executes based on the
version of `automerge-wasm` in the previous commit. We need to trigger
it even though the version has not changed so we roll back the version
in this commit and the commit immediately following this will bump it
again.
2023-01-12 10:37:11 +00:00
Alex Good
a0d698dc8e
Version bump js and wasm
js: 2.0.1-alpha.3
wasm: 0.1.20
2023-01-12 09:55:12 +00:00
Alex Good
5763210b07
wasm: Allow a choice of text representations
The wasm codebase assumed that clients want to represent text as a
string of characters. This is faster, but in order to enable backwards
compatibility we add a `TextRepresentation` argument to
`automerge_wasm::Automerge::new` to allow clients to choose between a
`string` or `Array<any>` representation. The `automerge_wasm::Observer`
will consult this setting to determine what kind of diffs to generate.
2023-01-10 12:52:19 +00:00
Alex Good
18a3f61704 Update rust toolchain to 1.66 2023-01-10 12:51:56 +00:00
Alex Currie-Clark
0306ade939 Update action name on IncPatch type 2023-01-06 15:23:41 +00:00
Alex Good
8a645bb193 js: Enable typescript for the JS tests
The tsconfig.json was setup to not include the JS tests. Update the
config to include the tests when checking typescript and fix all the
consequent errors. None of this is semantically meaningful _except_ for
a few incorrect usages of the API which were leading to flaky tests.
Hooray for types!
2022-12-22 11:48:06 +00:00
Alex Good
4de0756bb4 Correctly handle ops on optree node boundaries
The `SeekOp` query can produce incorrect results when the optree it is
searching only has visible ops on the internal nodes. Add some tests to
demonstrate the issue as well as a fix.
2022-12-20 20:38:29 +00:00
Alex Good
d678280b57 automerge-cli: Add an examine-sync command
This is useful when receiving sync messages that behave in unexptected
ways
2022-12-19 16:30:14 +00:00
Alex Good
f682db3039 automerge-cli: Add a flag to skip verifiying heads 2022-12-19 16:30:14 +00:00
Alex Good
6da93b6adc Correctly implement colored json
My quickly thrown together implementation had somem mistakes in it which
meant that the JSON produced was malformed.
2022-12-19 16:30:14 +00:00
Alex Good
0f90fe4d02 Add a method for loading a document without verifying heads
This is primarily useful when debugging documents which have been
corrupted somehow so you would like to see the ops even if you can't
trust them. Note that this is _not_ currently useful for performance
reasons as the hash graph is still constructed, just not verified.
2022-12-19 16:30:14 +00:00
alexjg
8aff1296b9
automerge-cli: remove a bunch of bad dependencies (#478)
Automerge CLI depends transitively (via and old version of `clap` and
via `colored_json` on `atty` and `ansi_term`. These crates are both
marked as unmaintained and this generates irritating `cargo deny`
messages. To avoid this, implement colored JSON ourselves using the
`termcolor` crate - colored JSON is pretty mechanical. Also update
criterion and cbindgen dependencies and ignore the criterion tree in
deny.toml as we only ever use it in benchmarks.

All that's left now is a warning about atty in cbindgen, we'll just have
to wait for cbindgen to fix that, it's a build time dependency anyway so
it's not really an issue.
2022-12-14 18:06:19 +00:00
Conrad Irwin
6dad2b7df1
Don't panic on invalid gzip stream (#477)
* Don't panic on invalid gzip stream

Before this change automerge-rs would panic if the gzip data in
a raw column was invalid; after this change the error is propagated
to the caller correctly.
2022-12-14 17:34:22 +00:00
Orion Henry
3229548fc7
update js dependencies and some lint errors (#474) 2022-12-11 21:26:00 +00:00
Orion Henry
b78211ca65
change opid to (u32,u32) - 10% performance uptick (#473) 2022-12-11 18:56:20 +00:00
Orion Henry
1222fc0df1
rewrite opnode to store usize instead of Op (#471) 2022-12-10 10:36:05 +00:00
Orion Henry
2db9e78f2a
Text v2. JS Api now uses text by default (#462) 2022-12-09 23:48:07 +00:00
Conrad Irwin
b05c9e83a4
Use AMbyteSpan for AM{list,map}PutBytes (#464)
* Use AMbyteSpan for byte values

Before this change there was an inconsistency between AMmapPutString
(which took an AMbyteSpan) and AMmapPutBytes (which took a pointer +
length).

Either is fine, but we should do the same in both places. I chose this
path to make it clear that the value passed in was an automerge value,
and to be symmetric with AMvalue.bytes when you do an AMmapGet().

I did not update other APIs (like load) that take a pointer + length, as
that is idiomatic usage for C, and these functions are not operating on
byte values stored in automerge.
2022-12-09 16:11:23 +00:00
Conrad Irwin
c3932e6267
Improve docs for building automerge-c on a mac (#465)
* More detailed instructions in README

I struggled to get the project to build for a while when first getting
started, so have added some instructions; and also some usage
instructions for automerge-c that show more clearly what is happening
without `AMpush()`
2022-12-09 13:46:23 +00:00
Alex Good
becc301877
automerge-wasm@0.1.19 & automerge-js@2.0.1-alpha.2 2022-12-02 15:10:24 +00:00
Alex Good
0ab6a770d8 wasm: improve error messages
The error messages produced by various conversions in `automerge-wasm`
were quite uninformative - often consisting of just returning the
offending value with no description of the problem. The logic of these
error messages was often hard to trace due to the use of `JsValue` to
represent both error conditions and valid values - evidenced by most of
the public functions of `automerge-wasm` having return types of
`Result<JsValue, JsValue>`. Change these return types to mention
specific errors, thus enlisting the compilers help in ensuring that
specific error messages are emitted.
2022-12-02 14:42:55 +00:00
Alex Currie-Clark
2826f4f08c
automerge-wasm: Add deno as a target 2022-12-02 14:42:13 +00:00
Alex Good
de16adbcc5 Explicity create empty changes
Transactions with no ops in them are generally undesirable. They take up
space in the change log but do nothing else. They are not useless
though, it may occasionally be necessary to create an empty change in
order to list all the current heads of the document as dependents of the
empty change.

The current API makes no distinction between empty changes and non-empty
changes. If the user calls `Transaction::commit` a change is created
regardless of whether there are ops to commit. To provide a more useful
API modify `commit` so that if there is a no-op transaction then no
changes are created, but provide explicit methods to create an empty
change via `Transaction::empty_change`, `Automerge::empty_change` and
`Autocommit::empty_change`. Also make these APIs available in Javascript
and C.
2022-12-02 12:12:54 +00:00
Alex Good
ea5688e418 rust: Make fields of Transaction and TransactionInner private
It's tricky to modify these structs with the fields public as every
change requires scanning the codebase for references to make sure you're
not breaking any invariants. Make the fields private to ease
development.
2022-12-02 12:12:54 +00:00
Alex Good
149f870102 rust: Remove Default constraint from OpObserver 2022-12-02 12:12:54 +00:00
Orion Henry
aaddb3c9ea fix error message 2022-11-28 15:43:27 -06:00
Jason Kankiewicz
d3885a3443 Hard-coded automerge-c's initial independent
version number to "0.0.1" for @alexjg.
2022-11-28 00:08:33 -08:00
Jason Kankiewicz
f8428896bd Added a test case for a map key containing NUL
('\0') based on #455.
2022-11-27 23:52:47 -08:00
Jason Kankiewicz
fb0c69cc52 Updated the quickstart example to work with
`AMbyteSpan` values instead of `*const libc::c_char` values.
2022-11-27 23:52:47 -08:00
Jason Kankiewicz
edbb33522d Replaced the C string (*const libc::c_char)
value of the `AMresult::Error` variant with a UTF-8 string view
(`AMbyteSpan`).
2022-11-27 23:52:47 -08:00
Jason Kankiewicz
625f48f33a Fixed clippy violations. 2022-11-27 23:52:47 -08:00
Jason Kankiewicz
7c9f927136 Fixed code formatting violations. 2022-11-27 23:52:47 -08:00
Jason Kankiewicz
b60c310f5c Changed Default::default() calls to be through
the trait.
2022-11-27 23:52:47 -08:00
Jason Kankiewicz
3dd954d5b7 Moved the to_obj_id macro in with AMobjId. 2022-11-27 23:52:47 -08:00
Jason Kankiewicz
3e2e697504 Replaced C string (*const libc::c_char) values
with UTF-8 string view  (`AMbyteSpan`) values except with the
`AMresult::Error` variant.
Added `AMstr()` for creating an `AMbyteSpan` from a C string.
2022-11-27 23:52:47 -08:00
Jason Kankiewicz
a324b02005 Added automerge::AutomergeError::InvalidActorId.
Added `automerge::AutomergeError::InvalidCharacter`.
Alphabetized the `automerge::AutomergeError` variants.
2022-11-27 23:52:47 -08:00
Alex Good
d26cb0c0cb
rust:automerge-test:0.1.0 2022-11-27 16:54:41 +00:00
Alex Good
ed108ba6fc
rust:automerge:0.2.0 2022-11-27 16:44:26 +00:00
Alex Good
484a5bac4f
rust: Add Transactable::base_heads
Sometimes it is necessary to query the heads of a document at the time a
transaction started without having a mutable reference to the
transactable. Add `Transactable::base_heads` to do this.
2022-11-27 16:39:02 +00:00
Alex Good
01350c2b3f
automerge-wasm@0.1.18 and automerge@2.0.1-alpha.1 2022-11-22 19:37:01 +00:00
alexjg
22d60987f6
Dont send duplicate sync messages (#460)
The API of Automerge::generate_sync_message requires that the user keep
track of in flight messages themselves if they want to avoid sending
duplicate messages. To avoid this add a flag to `automerge::sync::State`
to track if there are any in flight messages and return `None` from
`generate_sync_message` if there are.
2022-11-22 18:29:06 +00:00
Orion Henry
ca25ed0ca0
automerge-wasm: Use a SequenceTree in the OpObserver
Generating patches to text objects (a la the edit-trace benchmark) was
very slow due to appending to the back of a Vec. Use the SequenceTree
(effectively a B-tree) instead so as to speed up sequence patch
generation.
2022-11-22 12:13:42 +00:00
Alex Good
03b3da203d
@automerge/automerge-wasm 0.1.16 2022-11-22 00:02:28 +00:00
Jason Kankiewicz
a7656b999b
Add AMobjObjType() (#454)
automerge-c: Add AmobjObjType()
2022-11-07 23:10:53 +00:00
Alex Good
05093071ce
rust/automerge-test: add From<f64> for RealizedObject 2022-11-07 12:08:12 +00:00
Alex Good
bcab3b6e47 Move automerge/tests::helpers to crate automerge-test
The assert_doc and assert_obj macros in automerge/tests::helpers are
useful for writing tests for any application working with automerge
documents. Typically however, you only want these utilities in tests so
rather than packaging them in the main `automerge` crate move them to a
new crate (in the spirit of `tokio_test`)
2022-11-06 19:52:21 +00:00
Alex Good
b53584bec0
Ritual obeisance before the altar of clippy 2022-11-05 22:48:43 +00:00
Orion Henry
91f313bb83 revert compiler flags to max opt 2022-11-04 18:02:32 +00:00
tosti007
6bbed76f0f Update uuid dependency to v1.2.1 2022-11-01 11:39:24 +00:00
Alex Good
5adb6952e9
@automerge/automerge@2.0.0-beta.2 and @automerge/automerge-wasm@0.1.15 2022-10-26 14:03:12 +01:00
Orion Henry
3705212747
js: Add Automerge.clone(_, heads) and Automerge.view
Sometimes you need a cheap copy of a document at a given set of heads
just so you can see what has changed. Cloning the document to do this is
quite expensive when you don't need a writable copy. Add automerge.view
to allow a cheap read only copy of a document at a given set of heads
and add an additional heads argument to clone for when you do want a
writable copy.
2022-10-26 14:01:11 +01:00
Orion Henry
d7d2916acb tiny change that might remove a bloom filter false positive error 2022-10-21 15:15:30 -05:00
Orion Henry
59289f67b1 consolidate inserts and deletes more aggressivly into a single splice 2022-10-18 13:29:56 +01:00
Orion Henry
38205fbcc2 enableFreeze() instead of implicit freeze 2022-10-17 17:35:34 -05:00
Orion Henry
a2704bac4b Merge remote-tracking branch 'origin/main' into f2 2022-10-17 16:32:23 -05:00
Orion Henry
ac90f8f028 Merge remote-tracking branch 'origin/freeze' into f2 2022-10-17 16:21:35 -05:00
Orion Henry
c602e9e7ed update build to match directory restructuring 2022-10-17 16:20:25 -05:00
Alex Good
dd3c6d1303
Move rust workspace into ./rust
After some discussion with PVH I realise that the repo structure in the
last reorg was very rust-centric. In an attempt to put each language on
a level footing move the rust code and project files into ./rust
2022-10-16 19:55:51 +01:00