For some usecases the overhead of compressed columns in the document
format is not worth it. Add `Automerge::save_nocompress` to save without
compressing columns.
Signed-off-by: Alex Good <alex@memoryandthought.me>
This is achieved by liberal use of feature flags. Main additions are:
* Build the OpSet more efficiently when loading from compressed
document storage using a DocObserver as implemented in
`automerge::op_tree::load`
* Reimplement the parsing login in the various types in
`automerge::sync`
There are numerous other small changes required to get the types to line
up.
Signed-off-by: Alex Good <alex@memoryandthought.me>
It is useful to be able to generate a `serde::Value` representation of
an automerge document. We can do this without an intermediate type by
iterating over the keys of the document recursively. Add
`autoeserde::AutoSerde` to implement this.
Signed-off-by: Alex Good <alex@memoryandthought.me>
Implement parsing the binary format using the new parser library and the
new encoding types. This is superior to the previous parsing
implementation in that invalid data should never cause panics and it
exposes and interface to construct an OpSet from a saved document much
more efficiently.
Signed-off-by: Alex Good <alex@memoryandthought.me>
The representation of changes in storage-v2 is different to the existing
representation so add accessor methods to the fields of `Change` and
make all accesses go through them. This allows the change representation
in storage-v2 to be a drop-in.
Signed-off-by: Alex Good <alex@memoryandthought.me>
The existing implementation of the columnar format elides a lot of error
handling (by converting `Err` to `None`) and doesn't allow writing to a
single chunk of memory when encoding. Implement a new set of encoding and
decoding primitives which handle errors more robustly and allow us to
use a single chunk of memory when reading and writing.
Signed-off-by: Alex Good <alex@memoryandthought.me>
Op IDs in the OpSet are represented using an index into a set of actor
IDs. This is efficient but requires conversion when reading and
writing from storage (where the set of actors might be different from
ths in the OpSet). Add a trait for converting between different
representations of an OpID.
Signed-off-by: Alex Good <alex@memoryandthought.me>
We have parsing needs which are slightly more complex than just reading
stuff from a buffer, but not complex enough to justify a dependency on a
parsing library. Implement a simple parser combinator library for use in
parsing the binary storage format.
Signed-off-by: Alex Good <alex@memoryandthought.me>
The colunar storage format allows for values which we do not know the
type of. In order that we can handle these types in a forward compatible
way we add ScalarValue::Unknown.
Signed-off-by: Alex Good <alex@memoryandthought.me>
The new storage implementation is sufficiently large a change that it
warrants a period of testing. To facilitate testing the new and old
implementations side by side we slightly abuse cargo's feature flags and
add a storage-v2 feature which enables the new storage and disables the
old storage.
Note that this commit doesn't use `--all-features` when building the
workspace in scripts/ci/build-test. This will be rectified in a later
commit once the storage-v2 feature is integrated into the other crates
in the workspace.
Signed-off-by: Alex Good <alex@memoryandthought.me>
Weave the original TypeScript code into the C ports of the WASM API's
sync tests.
Fix misnomers in the WASM API's basic and sync unit tests.
Fix misspellings in the WASM API's basic and sync unit tests.
Add symbolic last index specification to `AMlist{Delete,Get,Increment}()`.
Add symbolic last index specification to `AMlistPut{Bool,Bytes,Counter,
F64,Int,Null,Object,Str,Timestamp,Uint}()`.
Prevent `doc::utils::to_str(NULL)` from segfaulting.
Fix some documentation content bugs.
Fix some documentation formatting bugs.
as `AMgetChangeByHash()`.
Add the `AM_CHANGE_HASH_SIZE` macro define constant for
`AMgetChangeByHash()`.
Replace the literal `32` with the `automerge::types::HASH_SIZE` constant.
Expose `automerge::AutoCommit::splice()` as `AMsplice()`.
Add the `automerge::error::AutomergeError::InvalidValueType` variant for
`AMsplice()`.
Add push functionality to `AMspliceText()`.
Fix some documentation content bugs.
Fix some documentation formatting bugs.
Rename `AMdup()` to `AMclone()` to match the WASM API.
Rename `AMgetActor()` to `AMgetActorId()` to match the WASM API.
Rename `AMsetActor()` to `AMsetActorId()` to match the WASM API.
renamed source files.
Defer `BTreeMap` creation until necessary for `AMresult::Changes`.
Add `AMvalueEqual()` to enable direct comparison of two `AMvalue` structs regardless of their respective variants.
quickstart example up to the library as `AMresultStack` so that it can
appear in the README.md and be used to simplify the unit tests.
Promoted `free_results()` to `AMfreeStack()` and `push()` to `AMpush()`.
Added `AMpop()` because no stack should be without one.
`automerge::AutoCommit::get_at()`.
Add `AMmapRange()` to expose `automerge::AutoCommit::map_range()` and
`automerge::AutoCommit::map_range_at()`.
Add `AMmapItems` for `AMlistRange()`.
Add `AMmapItem` for `AMmapItems`.
`automerge::AutoCommit::get_at()`.
Add `AMlistRange()` to expose `automerge::AutoCommit::list_range()` and
`automerge::AutoCommit::list_range_at()`.
Add `AMlistItems` for `AMlistRange()`.
Add `AMlistItem` for `AMlistItems`.
fork()`.
Add `AMobjValues()` to expose `automerge::AutoCommit::values()` and
`automerge::AutoCommit::values_at()`.
Add `AMobjIdActorId()`, `AMobjIdCounter()`, and `AMobjIdIndex()` to expose `automerge::ObjId::Id` fields.
Change `AMactorId` to reference an `automerge::ActorId` instead of
owning one for `AMobjIdActorId()`.
Add `AMactorIdCmp()` for `AMobjIdActorId()` comparison.
Add `AMobjItems` for `AMobjValues()`.
Add `AMobjItem` for `AMobjItems`.
Add `AMobjIdEqual()` for property comparison.
Rename `to_doc!()` to `to_doc_mut!()` and `to_doc_const!()` to `to_doc!()`
for consistency with the Rust standard library.
`unicode-idents` distributes some data tables from unicode.org which
require an additional license. This doesn't affect our licensing because
we don't distribute the data files - just the generated code. Explicitly
allow the Unicode-DFS-2016 license for unicode-idents.
The ordering of opids in the successor and predecessors of an op is
relevant when encoding because inconsistent ordering changes the
hashgraph. This means we must maintain the invariant that opids are
encoded in ascending lamport order. We have been maintaining this
invariant in the encoding implementation - however, this is not ideal
because it requires allocating for every op in the change when we commit
a transaction.
Add `types::OpIds` and use it in place of `Vec<OpId>` for `Op::succ` and
`Op::pred`. `OpIds` maintains the invariant that the IDs it contains
must be ordered with respect to some comparator function - which is
always `OpSetMetadata::lamport_cmp`. Remove the sorting of opids in
SuccEncoder::append.
