Commit graph

944 commits

Author SHA1 Message Date
Jason Kankiewicz
632da04d60
Add the -DFEATURE_FLAG_STORAGE_V2 CMake option
for toggling the "storage-v2" feature flag in a Cargo invocation.
Correct the `AMunknownValue` struct misnomer.
Ease the rebasing of changes to the `AMvalue` struct declaration with
pending upstream changes to same.
2022-08-22 21:18:07 +01:00
Alex Good
8f2d4a494f
Test entire workspace for storage-v2 in CI
Now that all crates support the storage-v2 feature flag of the automerge
crate we update CI to run tests for '--workspace --all-features'

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:48 +01:00
Alex Good
db4cb52750
Add a storage-v2 feature flag to edit-trace
Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:48 +01:00
Alex Good
fc94d43e53
Expose storage-v2 in automerge-wasm
Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
d53d107076
Expose storage-v2 in automerge-c
Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
63dca26fe2
Additional tests for storage-v2
Various tests were required to cover edge cases in the new storage-v2
implementation.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
252a7eb8a5
Add automerge::Automerge::save_nocompress
For some usecases the overhead of compressed columns in the document
format is not worth it. Add `Automerge::save_nocompress` to save without
compressing columns.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
34e919a4c8
Plumb in storage-v2
This is achieved by liberal use of feature flags. Main additions are:

* Build the OpSet more efficiently when loading from compressed
  document storage using a DocObserver as implemented in
  `automerge::op_tree::load`
* Reimplement the parsing login in the various types in
  `automerge::sync`

There are numerous other small changes required to get the types to line
up.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
fc7657bcc6
Add a wrapper to implement Deserialize for Automerge
It is useful to be able to generate a `serde::Value` representation of
an automerge document. We can do this without an intermediate type by
iterating over the keys of the document recursively. Add
`autoeserde::AutoSerde` to implement this.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
771733deac
Implement storage-v2
Implement parsing the binary format using the new parser library and the
new encoding types. This is superior to the previous parsing
implementation in that invalid data should never cause panics and it
exposes and interface to construct an OpSet from a saved document much
more efficiently.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
3a3df45b85
Access change fields through field accessors
The representation of changes in storage-v2 is different to the existing
representation so add accessor methods to the fields of `Change` and
make all accesses go through them. This allows the change representation
in storage-v2 to be a drop-in.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:42 +01:00
Orion Henry
d28767e689 automerge-js v0.1.10 2022-08-22 15:13:08 -05:00
Alex Good
de997e2c50
Reimplement columnar decoding types
The existing implementation of the columnar format elides a lot of error
handling (by converting `Err` to `None`) and doesn't allow writing to a
single chunk of memory when encoding. Implement a new set of encoding and
decoding primitives which handle errors more robustly and allow us to
use a single chunk of memory when reading and writing.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:06:35 +01:00
Alex Good
782f351322
Add types to convert between different Op types
Op IDs in the OpSet are represented using an index into a set of actor
IDs. This is efficient but requires conversion when reading and
writing from storage (where the set of actors might be different from
ths in the OpSet). Add a trait for converting between different
representations of an OpID.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:06:35 +01:00
Alex Good
e1295b9daa
Add a simple parser combinator library
We have parsing needs which are slightly more complex than just reading
stuff from a buffer, but not complex enough to justify a dependency on a
parsing library. Implement a simple parser combinator library for use in
parsing the binary storage format.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:06:35 +01:00
Alex Good
d785c319b8
Add ScalarValue::Unknown
The colunar storage format allows for values which we do not know the
type of. In order that we can handle these types in a forward compatible
way we add ScalarValue::Unknown.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:04:19 +01:00
Orion Henry
88f8976d0a automerge-js 0.1.9 2022-08-22 14:58:13 -05:00
Alex Good
56563a4a60
Add a storage-v2 feature flag
The new storage implementation is sufficiently large a change that it
warrants a period of testing. To facilitate testing the new and old
implementations side by side we slightly abuse cargo's feature flags and
add a storage-v2 feature which enables the new storage and disables the
old storage.

Note that this commit doesn't use `--all-features` when building the
workspace in scripts/ci/build-test. This will be rectified in a later
commit once the storage-v2 feature is integrated into the other crates
in the workspace.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-20 17:36:26 +01:00
Orion Henry
ff90327b52
Merge pull request #414 from jkankiewicz/port_WASM_basic_tests_to_C
Port the WASM API's basic tests to C
2022-08-11 19:00:11 -05:00
Orion Henry
ece66396e8
Merge pull request #417 from tombh/readme-update
Readme updates
2022-08-11 18:58:44 -05:00
Orion Henry
d1a926bcbe fix ownKeys bug in automerge-js 2022-08-11 18:49:42 -05:00
Orion Henry
1a955e1f0d fix some typescript errors - depricate default export of the wasm package 2022-08-11 18:24:21 -05:00
Thomas Buckley-Houston
f89e9ad9cc
Readme updates 2022-08-10 08:46:05 -04:00
Jason Kankiewicz
bc28faee71 Replace NULL with std::ptr::null() within the
safety notes for @alexjg in #414.
2022-08-07 20:04:49 -07:00
Jason Kankiewicz
50981acc5a Replace to_del!() and to_pos!() with
`to_index!()` for @alexjg in #414.
2022-08-07 19:37:48 -07:00
Jason Kankiewicz
7ec17b26a9 Replace `From<&AMvalue<'_>> for Result<
am::ScalarValue, am::AutomergeError>` with `TryFrom<&AMvalue<'_>> for
am::ScalarValue` for @alexjg in #414.
2022-08-07 19:24:47 -07:00
Jason Kankiewicz
825342cbb1 Remove reflexive struct reference from a Doxygen
variable declaration.
2022-08-07 08:07:00 -07:00
Jason Kankiewicz
04d0175113 Add missing past-the-end checks to the unit tests
for `AMmapRange()`.
2022-08-06 16:20:35 -07:00
Jason Kankiewicz
14bd8fbe97 Port the WASM API's basic unit tests to C.
Weave the original TypeScript code into the C ports of the WASM API's
sync tests.
Fix misnomers in the WASM API's basic and sync unit tests.
Fix misspellings in the WASM API's basic and sync unit tests.
2022-08-06 16:18:59 -07:00
Jason Kankiewicz
d48e366272 Fix some documentation content bugs.
Fix some documentation formatting bugs.
2022-08-06 15:56:21 -07:00
Jason Kankiewicz
4217019cbc Expose automerge::AutoCommit::get_all() as AMlistGetAll() and AMmapGetAll().
Add symbolic last index specification to `AMlist{Delete,Get,Increment}()`.
Add symbolic last index specification to `AMlistPut{Bool,Bytes,Counter,
F64,Int,Null,Object,Str,Timestamp,Uint}()`.
Prevent `doc::utils::to_str(NULL)` from segfaulting.
Fix some documentation content bugs.
Fix some documentation formatting bugs.
2022-08-06 15:47:53 -07:00
Jason Kankiewicz
eeb75f74f4 Fix AMstrsCmp().
Fix some documentation content bugs.
Fix some documentation formatting bugs.
2022-08-06 15:07:48 -07:00
Jason Kankiewicz
a22afdd70d Expose automerge::AutoCommit::get_change_by_hash()
as `AMgetChangeByHash()`.
Add the `AM_CHANGE_HASH_SIZE` macro define constant for
`AMgetChangeByHash()`.
Replace the literal `32` with the `automerge::types::HASH_SIZE` constant.
Expose `automerge::AutoCommit::splice()` as `AMsplice()`.
Add the `automerge::error::AutomergeError::InvalidValueType` variant for
`AMsplice()`.
Add push functionality to `AMspliceText()`.
Fix some documentation content bugs.
Fix some documentation formatting bugs.
2022-08-06 15:04:46 -07:00
Orion Henry
5e8f4caed6
Merge pull request #392 from rf-/rf-fix-export-default-syntax
Fix TypeScript syntax error in `automerge-wasm` definitions
2022-08-03 11:01:11 -05:00
Orion Henry
5c6f375f99
Merge pull request #410 from jkankiewicz/add_range_functions_to_C_API
Add range functions to C API
2022-08-03 10:47:44 -05:00
Jason Kankiewicz
3a556c5991 Expose Autocommit::fork_at().
Rename `AMdup()` to `AMclone()` to match the WASM API.
Rename `AMgetActor()` to `AMgetActorId()` to match the WASM API.
Rename `AMsetActor()` to `AMsetActorId()` to match the WASM API.
2022-08-01 07:02:30 -07:00
Orion Henry
1bc5fbb81e
Merge pull request #413 from jkankiewicz/remove_original_C_API_files
Remove original C API files
2022-07-29 16:06:45 -05:00
Jason Kankiewicz
69de8187a5 Update the build system with the added and
renamed source files.
Defer `BTreeMap` creation until necessary for  `AMresult::Changes`.
Add `AMvalueEqual()` to enable direct comparison of two `AMvalue` structs regardless of their respective variants.
2022-07-25 01:41:52 -07:00
Jason Kankiewicz
877744d40b Add equality comparison to the AM* types from
which it was missing.
Add equality comparison to `automerge::sync::message`.
Defer `std::ffi::CString` creation until necessary.
2022-07-25 01:33:50 -07:00
Jason Kankiewicz
14b55c4a73 Fix a bug with the iterators when they pass their
initial positions in reverse.
Rename `AMstrings` to `AMstrs` for consistency with the `AMvalue.str`
field.
2022-07-25 01:23:26 -07:00
Jason Kankiewicz
23fbb4917a Replace _INCLUDED with _H as the suffix for
include guards in C headers like the one generated by cbindgen.
2022-07-25 01:04:35 -07:00
Jason Kankiewicz
877dbbfce8 Simplify the unit tests with AMresultStack et.
al.
2022-07-25 01:00:50 -07:00
Jason Kankiewicz
a22bcb916b Promoted ResultStack/StackNode from the
quickstart example up to the library as `AMresultStack` so that it can
appear in the README.md and be used to simplify the unit tests.
Promoted `free_results()` to `AMfreeStack()` and `push()` to `AMpush()`.
Added `AMpop()` because no stack should be without one.
2022-07-25 00:50:40 -07:00
Jason Kankiewicz
42ab1639db Add heads argument to AMmapGet() to expose
`automerge::AutoCommit::get_at()`.
Add `AMmapRange()` to expose `automerge::AutoCommit::map_range()` and
`automerge::AutoCommit::map_range_at()`.
Add `AMmapItems` for `AMlistRange()`.
Add `AMmapItem` for `AMmapItems`.
2022-07-25 00:11:00 -07:00
Jason Kankiewicz
eba18d1ad6 Add heads argument to AMlistGet() to expose
`automerge::AutoCommit::get_at()`.
Add `AMlistRange()` to expose `automerge::AutoCommit::list_range()` and
`automerge::AutoCommit::list_range_at()`.
Add `AMlistItems` for `AMlistRange()`.
Add `AMlistItem` for `AMlistItems`.
2022-07-24 22:41:32 -07:00
Jason Kankiewicz
ee68645f31 Add AMfork() to expose `automerge::AutoCommit::
fork()`.
Add `AMobjValues()` to expose `automerge::AutoCommit::values()` and
`automerge::AutoCommit::values_at()`.
Add `AMobjIdActorId()`, `AMobjIdCounter()`, and `AMobjIdIndex()` to expose `automerge::ObjId::Id` fields.
Change `AMactorId` to reference an `automerge::ActorId` instead of
owning one for `AMobjIdActorId()`.
Add `AMactorIdCmp()` for `AMobjIdActorId()` comparison.
Add `AMobjItems` for `AMobjValues()`.
Add `AMobjItem` for `AMobjItems`.
Add `AMobjIdEqual()` for property comparison.
Rename `to_doc!()` to `to_doc_mut!()` and `to_doc_const!()` to `to_doc!()`
for consistency with the Rust standard library.
2022-07-24 22:23:54 -07:00
Jason Kankiewicz
cc19a37f01 Remove the makefile for the original C
API to prevent confusion.
2022-07-23 08:48:19 -07:00
Jason Kankiewicz
15c9adf965 Remove the obsolete test suite for the original
C API to prevent confusion.
2022-07-23 08:47:21 -07:00
Jason Kankiewicz
52a558ee4d Cease writing a pristine copy of the generated
header file into the root of the C API's source directory to prevent
confusion.
2022-07-23 08:44:41 -07:00
Alex Good
668b7b86ca Add license for unicode-idents
`unicode-idents` distributes some data tables from unicode.org which
require an additional license. This doesn't affect our licensing because
we don't distribute the data files - just the generated code. Explicitly
allow the Unicode-DFS-2016 license for unicode-idents.
2022-07-18 10:50:32 +01:00