Commit graph

56 commits

Author SHA1 Message Date
Alex Good
837c07b23a
Correctly encode compressed changes in sync messages
Sync messages encode changes as length prefixed byte arrays. We were
calculating the length using the uncompressed bytes of a change but
encoding the bytes of the change using the (possibly) compressed bytes.
This meant that if a change was large enough to compress then it  would
fail to decode. Switch to using uncompressed bytes in sync messages.
2022-10-02 18:59:41 +01:00
Alex Good
9ac8827219
Remove storage-v2 feature flag
Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:21:21 +01:00
Alex Good
771733deac
Implement storage-v2
Implement parsing the binary format using the new parser library and the
new encoding types. This is superior to the previous parsing
implementation in that invalid data should never cause panics and it
exposes and interface to construct an OpSet from a saved document much
more efficiently.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:16:47 +01:00
Alex Good
782f351322
Add types to convert between different Op types
Op IDs in the OpSet are represented using an index into a set of actor
IDs. This is efficient but requires conversion when reading and
writing from storage (where the set of actors might be different from
ths in the OpSet). Add a trait for converting between different
representations of an OpID.

Signed-off-by: Alex Good <alex@memoryandthought.me>
2022-08-22 21:06:35 +01:00
Jason Kankiewicz
a22afdd70d Expose automerge::AutoCommit::get_change_by_hash()
as `AMgetChangeByHash()`.
Add the `AM_CHANGE_HASH_SIZE` macro define constant for
`AMgetChangeByHash()`.
Replace the literal `32` with the `automerge::types::HASH_SIZE` constant.
Expose `automerge::AutoCommit::splice()` as `AMsplice()`.
Add the `automerge::error::AutomergeError::InvalidValueType` variant for
`AMsplice()`.
Add push functionality to `AMspliceText()`.
Fix some documentation content bugs.
Fix some documentation formatting bugs.
2022-08-06 15:04:46 -07:00
Alex Good
d71a734e49 Add OpIds to enforce ordering of Op::succ and Op::pred
The ordering of opids in the successor and predecessors of an op is
relevant when encoding because inconsistent ordering changes the
hashgraph. This means we must maintain the invariant that opids are
encoded in ascending lamport order. We have been maintaining this
invariant in the encoding implementation - however, this is not ideal
because it requires allocating for every op in the change when we commit
a transaction.

Add `types::OpIds` and use it in place of `Vec<OpId>` for `Op::succ` and
`Op::pred`. `OpIds` maintains the invariant that the IDs it contains
must be ordered with respect to some comparator function - which is
always `OpSetMetadata::lamport_cmp`. Remove the sorting of opids in
SuccEncoder::append.
2022-07-17 20:58:47 +01:00
Andrew Jeffery
587adf7418 Add Eq to ObjType 2022-05-24 09:48:55 +01:00
Andrew Jeffery
11fbde47bb Use HASH_SIZE const in ChangeHash definition 2022-05-20 10:04:32 +01:00
Orion Henry
be33f91346 Merge branch 'experiment' 2022-04-27 11:58:53 -04:00
Andrew Jeffery
48e397e82f Add lints 2022-04-23 11:05:43 +01:00
Andrew Jeffery
d667552a98 Add increment observation for observer 2022-04-20 14:44:04 +01:00
Andrew Jeffery
b6fd7ac26e Add op_observer to documents and transactions
This replaces the built-in patches with a more generic mechanism, and
includes a convenience observer which uses the old patches.
2022-04-19 17:30:05 +01:00
Jason Kankiewicz
d4a904414d Squashed commit of the following:
commit e1f8d769f4
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 10 08:53:07 2022 -0500

    update authors

commit 3e5525f1a6
Merge: f4ba1770 1c21abc5
Author: Orion Henry <orionz@users.noreply.github.com>
Date:   Wed Mar 9 14:36:29 2022 -0500

    Merge pull request #304 from jkankiewicz/c_api_exp

    Fix "fmt" workflow step violations

commit 1c21abc5a3
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Wed Mar 9 11:13:01 2022 -0800

    Fix CMake and Rust code formatting issues.

commit f4ba1770a9
Merge: bf1ae609 f41b30d1
Author: Orion Henry <orionz@users.noreply.github.com>
Date:   Wed Mar 9 12:05:58 2022 -0500

    Merge pull request #300 from jkankiewicz/c_api_exp

    Add unit test suites for the `AMlistSet*` and `AMmapSet*` functions

commit f41b30d118
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 22:08:36 2022 -0800

    Added a brief description of the `AmObjType` enum.
    Added the `AmStatus` enum to the enum docs page.

commit af7386a482
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 21:50:52 2022 -0800

    Added a unit test suite for the  `AMlistSet*`
    functions.

commit 1eb70c6eee
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 21:42:42 2022 -0800

    Added the rest of the `AMlistSet*` functions.
    Started the enum tags at `1` so they won't be
    inherently false.
    Alphabetized enum tags for the docs.
    Improved the docs.

commit 6489cba13b
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 18:01:46 2022 -0800

    Alphabetize functions in the docs.

commit 74c245b82d
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 07:54:25 2022 -0800

    Fix a typo in `AMmapSetObject()`'s documentation.

commit b2a879ba4e
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 06:24:22 2022 -0800

    Append missing EOF linefeed.

commit fbf0f29b66
Merge: c56d54b5 bf1ae609
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 01:08:12 2022 -0800

    Merge branch 'c_api_exp' of https://github.com/automerge/automerge-rs into c_api_exp

commit c56d54b565
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 01:07:11 2022 -0800

    Added unit test cases for the new `AMmapSet*`
    functions by @orionz.
    Moved the unit test cases for the `AMmapSet*` functions into their own
    unit test suite.

commit 7e59b55760
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Tue Mar 8 01:01:47 2022 -0800

    Edited the Doxygen documentation.

commit bf1ae60913
Author: Orion Henry <orion.henry@gmail.com>
Date:   Mon Mar 7 11:59:22 2022 -0500

    fmt

commit e82a7cc78e
Merge: a44e69d2 965c2d56
Author: Orion Henry <orionz@users.noreply.github.com>
Date:   Mon Mar 7 11:55:32 2022 -0500

    Merge pull request #299 from jkankiewicz/c_api_exp

    Enable unit testing of the C API

commit 965c2d56c3
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Mon Mar 7 06:37:36 2022 -0800

    Enable unit testing of the C API.

commit a44e69d2c7
Author: Orion Henry <orion.henry@gmail.com>
Date:   Sun Mar 6 14:00:46 2022 -0500

    remove datatype mapset

commit 88153c44e7
Merge: 41512e9c c6194e97
Author: Orion Henry <orionz@users.noreply.github.com>
Date:   Sun Mar 6 10:32:39 2022 -0500

    Merge pull request #298 from jkankiewicz/rebase_c_api_exp

    Rebase the "c_api_exp" branch on the "experiment" branch

commit c6194e9732
Merge: a2d745c8 41512e9c
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Sun Mar 6 01:09:56 2022 -0800

    Merge branch 'c_api_exp' into rebase_c_api_exp

commit a2d745c8d9
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Sun Mar 6 00:44:37 2022 -0800

    Replace the `utils::import_value` function with
    the `utils::import_scalar` function.
    Exclude `# Safety` comments from the documentation.

commit 0681e28b40
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 16:04:17 2022 -0500

    support new as_ref api

commit 916e23fcc2
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:56:27 2022 -0500

    fmt

commit 71cd6a1f18
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:54:38 2022 -0500

    lock data at 64 bit - no c_long

commit e00bd4c201
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:27:55 2022 -0500

    verbose

commit 39d157c554
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 14:56:23 2022 -0500

    clippy cleanup

commit 7f650fb8e0
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Wed Feb 23 02:14:06 2022 -0800

    Added Doxygen documentation generation.
    Renamed `AMDatatype` to `AmDataType`.
    Reorganized the `AmDataType` tags.
    Renamed `AMfree()` to `AMdestroy()`.
    Renamed `AMclone()` to `AMdup()`.

commit b0b803eef8
Author: Orion Henry <orion.henry@gmail.com>
Date:   Tue Feb 22 11:30:42 2022 -0500

    get simple test passing

commit cab9017ffa
Author: Orion Henry <orion.henry@gmail.com>
Date:   Wed Feb 9 15:50:44 2022 -0500

    rework to return a queriable result

commit a557e848f3
Author: Jason Kankiewicz <you@example.com>
Date:   Mon Feb 14 14:38:00 2022 -0800

    Add a CI step to run the CMake build of the C bindings for @alexjg.

commit c8c0c72f3b
Author: Jason Kankiewicz <you@example.com>
Date:   Mon Feb 14 14:09:58 2022 -0800

    Add CMake instructions for @orionz.

commit fb62c4b02a
Author: Jason Kankiewicz <you@example.com>
Date:   Thu Feb 10 23:28:54 2022 -0800

    Add CMake support.

commit 7bc3bb6850
Author: Jason Kankiewicz <you@example.com>
Date:   Thu Feb 10 22:49:53 2022 -0800

    Replace *intptr_t in C function signatures.

commit 60395a2db0
Author: Orion Henry <orion.henry@gmail.com>
Date:   Sun Feb 6 18:59:19 2022 -0500

    am_pop and am_pop_value

commit b1e88047d2
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Feb 3 19:43:36 2022 -0500

    break the ground

commit 41512e9c78
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 16:04:17 2022 -0500

    support new as_ref api

commit bcee6a9623
Merge: cf98f78d 9a89db3f
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:58:19 2022 -0500

    Merge remote-tracking branch 'origin/experiment' into c_api_exp

commit cf98f78dd1
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:56:27 2022 -0500

    fmt

commit 3c1f449c5c
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:54:38 2022 -0500

    lock data at 64 bit - no c_long

commit 2c2ec0b0c5
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 15:27:55 2022 -0500

    verbose

commit b72b9c989a
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Mar 3 14:56:23 2022 -0500

    clippy cleanup

commit 3ba28f91cc
Author: Jason Kankiewicz <jason.kankiewicz@gmail.com>
Date:   Wed Feb 23 02:14:06 2022 -0800

    Added Doxygen documentation generation.
    Renamed `AMDatatype` to `AmDataType`.
    Reorganized the `AmDataType` tags.
    Renamed `AMfree()` to `AMdestroy()`.
    Renamed `AMclone()` to `AMdup()`.

commit 8564e5b753
Author: Orion Henry <orion.henry@gmail.com>
Date:   Tue Feb 22 11:30:42 2022 -0500

    get simple test passing

commit 60835e6ae7
Author: Orion Henry <orion.henry@gmail.com>
Date:   Wed Feb 9 15:50:44 2022 -0500

    rework to return a queriable result

commit 89466d9e8c
Author: Jason Kankiewicz <you@example.com>
Date:   Mon Feb 14 14:38:00 2022 -0800

    Add a CI step to run the CMake build of the C bindings for @alexjg.

commit e2485bd5fd
Author: Jason Kankiewicz <you@example.com>
Date:   Mon Feb 14 14:09:58 2022 -0800

    Add CMake instructions for @orionz.

commit b5cc7dd63d
Author: Jason Kankiewicz <you@example.com>
Date:   Thu Feb 10 23:28:54 2022 -0800

    Add CMake support.

commit 685536f0cf
Author: Jason Kankiewicz <you@example.com>
Date:   Thu Feb 10 22:49:53 2022 -0800

    Replace *intptr_t in C function signatures.

commit c1c6e7bb66
Author: Orion Henry <orion.henry@gmail.com>
Date:   Sun Feb 6 18:59:19 2022 -0500

    am_pop and am_pop_value

commit e68c8d347e
Author: Orion Henry <orion.henry@gmail.com>
Date:   Thu Feb 3 19:43:36 2022 -0500

    break the ground
2022-04-19 08:35:44 -06:00
Andrew Jeffery
4406a5b208 Add range query
This is a way of efficiently getting just the keys and values in a
range.
2022-04-08 18:17:54 +01:00
Andrew Jeffery
e9adc32486 Fixup OpIdSearch's key extraction 2022-04-07 14:51:31 +01:00
Andrew Jeffery
bd2f252e0b Try and fix parent object query 2022-04-07 14:21:17 +01:00
Andrew Jeffery
fa2971a29a Have value be a reference for scalars 2022-04-04 12:47:08 +01:00
Andrew Jeffery
d331ceb6d4 Rename set to put and set_object to put_object 2022-04-01 13:40:58 +01:00
Andrew Jeffery
5cbc977076 More internal renames of del and inc 2022-04-01 13:36:27 +01:00
Orion Henry
ab580df947 Merge remote-tracking branch 'origin/experiment' into getnerate-patches 2022-03-30 13:04:51 -06:00
Martin Kleppmann
a9eddd88cc Bugfix: resurrection of deleted list elements 2022-03-30 13:12:07 -04:00
Martin Kleppmann
7025bb6541 Tests and fixes for list patches 2022-03-30 13:12:07 -04:00
Martin Kleppmann
94ff10f690 Rename and reformat a bit 2022-03-30 13:12:07 -04:00
Martin Kleppmann
26efee509d First patch implementation from pairing session with Orion 2022-03-30 13:12:01 -04:00
Andrew Jeffery
83d298ce8d Add test for broken last_elem 2022-03-27 15:28:49 +01:00
Andrew Jeffery
488df55385 Remove change field on Op as unused
This field was never read from.
2022-03-11 11:40:42 +00:00
Andrew Jeffery
a2cb15e936 Remove obj from the op as it can be gotten from the optree
This makes the Op struct smaller, helping memory usage and cache
coherence.
2022-03-11 11:40:28 +00:00
Andrew Jeffery
42233414b3 Add some documentation 2022-03-09 16:53:26 +00:00
Andrew Jeffery
d00cee1637 Misc API updates
- Commit now returns just a single hash rather than a vec. Since the
  change we create from committing has all of the heads as deps there
  can only be one hash/head after committing.
- Apply changes now takes a Vec rather than a slice. This avoids having
  to clone them inside.
- transact_with now passes the result of the closure to the commit
  options function
- Remove patch struct
- Change receive_sync_message to return a () instead of the
  `Option<Patch>`
- Change `Transaction*` structs to just `*` and use the transaction
  module
- Make CommitOptions fields public
2022-03-09 12:33:20 +00:00
Orion Henry
95f27f362c
Merge pull request #283 from jeffa5/experiment-make
Separate scalars and objects in transaction API
2022-03-04 16:53:17 -05:00
Andrew Jeffery
a4432bdc3d Nothing really Into's ObjType so just take it directly 2022-03-04 13:03:19 +00:00
Andrew Jeffery
2f3fe0e342 Cleanup actor id api
Default can be a footgun and confuse users, it was used internally but
that now uses the `from` impls. Also, opidat wasn't used and doesn't
seem to need to be public.
2022-03-04 12:06:43 +00:00
Andrew Jeffery
52eb193950 Add custom actor enum to avoid caching an unused one 2022-03-02 17:20:44 +00:00
Orion Henry
c8c695618b remove marks 2022-02-10 11:42:15 -05:00
Orion Henry
b53305cf7f Merge branch 'marks' into tmp 2022-02-10 09:42:38 -05:00
Orion Henry
bf184fe980 remove some un needed imports 2022-02-03 14:43:02 -05:00
Orion Henry
bfc051f4fb cleanup / rename 2022-01-31 14:02:24 -05:00
Orion Henry
a2e433348a mark encode/decode/serde 2022-01-31 14:02:24 -05:00
Orion Henry
b794f4803d rework marks as inserts between values 2022-01-31 14:02:24 -05:00
Orion Henry
e679c4f6a0 v0 wip 2022-01-31 14:02:23 -05:00
Orion Henry
b30a2b9cc1 give Counter its own type 2022-01-14 06:27:42 -08:00
Orion Henry
d50062b769 move values into the counter type - remove need for vis_window 2022-01-14 06:27:42 -08:00
Orion Henry
d2a7cc5f75 get sync tests working 2022-01-09 08:05:00 -08:00
Orion Henry
3046cbab35
Replace the OpID API with an object ID
Rather than returning an OpID for every mutation, we now return an
`Option<ObjId>`. This is `Some` only when a `make*` operation was
applied. This `ObjID` is an opaque type which can be used with any
document.
2022-01-01 20:15:02 +00:00
Orion Henry
e275334e72 remove dead code and pubs we don't need 2021-12-21 12:23:55 -05:00
Orion Henry
265ce823da implement all the query_at(), add tests, was, cut dead code 2021-12-21 09:23:34 -08:00
Orion Henry
9c237c7f00 remove expanded_op and internal_op_type 2021-12-17 12:39:26 -05:00
Orion Henry
702117b996 move change_hash out of legacy, removing diff 2021-12-17 12:13:43 -05:00
Orion Henry
90244236c3 move important items out of legacy 2021-12-17 11:33:05 -05:00
Orion Henry
6b517b9894 move things we need out of legacy 2021-12-17 11:04:35 -05:00