classy_table ¶This module implements a standalone minimalist analogue of mnesia’s local_data table with disc_copies storage.
It is used to store classy’s own persistent data. Other applications can also use it for data that doesn’t require replication and is not written too frequently.
Classy tables consist of a RAM cache backed by ETS and a durable write ahead log. Durable operations (write, delete, atomically) are immediately appended to the WAL, while dirty mutations (dirty_write, dirty_delete) only mark certain keys as dirty. All dirtied keys are flushed to WAL only when table is flushed.
The WAL is periodically compacted when its “badness” exceeds the configured threshold. Badness is calculated as a difference between the number of table elements and length of the log. WAL compaction is a blocking operation: all table mutations get blocked while it goes on. Compaction time is proportional to the number of elements stored in the table.
Because of this design, frequent mutations of the same key, e.g.
classy_table:write(Tab, foo, 1), classy_table:write(Tab, foo, 2), classy_table:write(Tab, foo, 3), ...
are rather inefficient.
flush is called or the table server terminates.
They are meant for the situations where some keys are frequently updated,
but these updates can be lost.
There is no automatic flushing of dirty operations, the business code must call classy_table:flush/1 function explicitly.
If it fails to do so, all work for persisting the data will be done on terminate or after a durable mutation, which may be risky or lead to unexpected results.
So, do not use classy tables as a synchronization mechanism between different processes.
on_update callback is executed before operations are persisted to the WAL.
As such, it is not a reliable way to mirror the state of a classy table to other storage.
on_update operations block the table server.
They must not contain any sort of heavy or long-running tasks.
-type rec() :: #classy_kv{k :: term(), v :: term()}.
All classy tables contain key-value data wrapped in #classy_kv record.
-type options() :: #{ets_options => list(),
badness_threshold => pos_integer(),
on_update => on_update_callback()}.
Table creation options.
-type on_update_callback() :: fun((tab(),
on_update_op()) -> _).
-type on_update_op() :: open |
{w, _Key, _Val} |
{d, _Key} |
close.
-type atomic_op(Effect) :: {w, _, _} |
{d, _} |
{then, Effect}.
A type of operation in a batch accepted by atomically.
{w, Key, Value}
equivalent to classy_table:write(Tab, Key, Value)
{d, Key}
equivalent to classy_table:delete(Tab, Key)
{then, Any}
ignored by classy.
If atomic batch was successfully committed,
then batch elements like this are returned as is.
-type tab() :: atom().
Table name that is used as a table identifier for both classy and ETS. All classy tables are named ETS tables.
-spec dump_wal(file:filename(), tab()) -> {ok, list()} |
{error, _}.
Dump WAL for debugging.
Warning: this function reads the entire WAL into memory.
-spec dump_wal(tab()) -> {ok, list()} | {error, _}.
See classy_table:dump_wal/2, uses the default directory.
-spec clear(tab()) -> ok.
Delete all data in the table. This is a durable operation.
on_update callback sees effects of this operation as series of regular deletes.
-spec lookup(tab(), _Key) -> [_Val].
Lookup a value from the table.
WARNING: this function can block the caller until the table is fully restored.
-spec drop(tab()) -> ok.
Drop the table (it must be open)
-spec force_compaction(tab()) -> ok.
Make a checkpoint and truncate the WAL.
-spec flush(tab()) -> ok.
Persist all records that got dirtied prior to this call to WAL.
Flush is atomic, meaning either all or none dirty operations are restored. However, if multiple processes perform unsynchronized dirty writes and flushes in parallel, data can be restored partially.
-spec atomically(tab(), [atomic_op(Effect)]) -> {ok,
[Effect]} |
{error, _}.
Commit a number of operations into a table atomically: either all or none of operations are durably stored by the time this function returns.
Operations are denoted via t:classy_table:atomic_op/1 type.
-spec delete(tab(), _Key) -> ok.
Delete a record from the table. From durability perspective, it has the same properties as classy_table:write/3.
-spec dirty_delete(tab(), _Key) -> ok.
Dirty version of classy_table:delete/2. From durability perspective, it has the same properties as classy_table:dirty_write/3.
-spec write(tab(), _Key, _Val) -> ok.
Update RAM representation of a record, write operation to WAL, sync WAL and then return.
Warning: this is a heavy operation. While this module batches writes, writes or deletes coming from a single process are always interleaved with a datasync.
If some process needs to reliably update a large number of records at once, it’s better to use classy_table:atomically/2.
-spec dirty_write(tab(), _Key, _Val) -> ok.
Update the RAM representation of the record and mark it as dirty. No writes to disk are made until any of the following calls complete:
-spec stop(tab(), timeout()) -> ok | {error, timeout}.
Close the table.
Warning: any process reading the table will become blocked.
-spec open(tab(), options()) -> ok | {error, _}.
Open a table named Tab and block the caller until all data is fully restored.
Note: this function is idempotent.