codec_interface.h File Reference

Codec interface definition and basic elements common to all codecs. More...

#include "volksdata/graph.h"

Include dependency graph for codec_interface.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	VOLK_Codec
	Codec structure. More...

Macros
#define	CHUNK_SIZE   4096

Typedefs
typedef VOLK_rc(*	term_enc_fn_t) (const VOLK_Term *term, char **rep)
	Term encoder callback type.
typedef void *(*	encode_gr_init_fn_t) (const VOLK_Graph *gr, VOLK_CodecFlags flags)
	Initialize a graph encoding loop.
typedef VOLK_rc(*	encode_gr_iter_fn_t) (void *it, char **res)
	Perform one graph encoding iteration.
typedef void(*	encode_gr_done_fn_t) (void *it)
	Finalize a graph encoding operation.
typedef void *(*	encode_ds_init_fn_t) (void)
	Initialize a dataset encoding loop.
typedef VOLK_rc(*	encode_ds_iter_fn_t) (void *it, const VOLK_Graph *gr, char **res)
	Perform one dataset encoding iteration.
typedef void(*	encode_ds_done_fn_t) (void *it)
	Finalize a dataset encoding operation.
typedef void *(*	encode_store_init_fn_t) (VOLK_Store *store)
	Initialize a store encoding loop.
typedef VOLK_rc(*	encode_store_iter_fn_t) (void *it, char **res)
	Perform one store encoding iteration.
typedef void(*	encode_store_done_fn_t) (void *it)
	Finalize a store encoding operation.
typedef VOLK_rc(*	decode_term_fn_t) (const char *rep, VOLK_Term **term)
	Prototype for decoding a string into a VOLK_Term.
typedef VOLK_rc(*	decode_gr_fn_t) (FILE *fh, const char *sh, VOLK_Graph *gr, size_t *ct, char **err)
	Prototype for decoding a complete RDF document file into a graph.
typedef void *(*	decode_ds_init_fn_t) (FILE *fh, const char *sh)
	Prototype for initializing a dataset decoding loop.
typedef VOLK_rc(*	decode_ds_iter_fn_t) (void *it, VOLK_Graph **gr, size_t *ct, char **err)
	Prototype for decoding one graph in a dataset.
typedef void(*	decode_ds_done_fn_t) (void *it)
	Prototype for finalizing a dataset decoder iterator.

Enumerations
enum	VOLK_CodecFlags { VOLK_CODEC_NO_PROLOG = 1<<0 , VOLK_CODEC_TXN = 1<<1 }
	Parse error information. More...
enum	VOLK_CodecFeatures {   VOLK_CODEC_FEAT_ENCODE_TERM = 1<<0 , VOLK_CODEC_FEAT_DECODE_TERM = 1<<1 , VOLK_CODEC_FEAT_ENCODE_GR = 1<<2 , VOLK_CODEC_FEAT_DECODE_GR = 1<<3 ,   VOLK_CODEC_FEAT_ENCODE_DS = 1<<4 , VOLK_CODEC_FEAT_DECODE_DS = 1<<5 , VOLK_CODEC_FEAT_ENCODE_STORE = 1<<6 , VOLK_CODEC_FEAT_DECODE_STORE = 1<<7 }
	Feature flags applicable to a codec. More...

Functions
uint8_t *	uint8_dup (const uint8_t *str)
	strdup() for unsigned char.
uint8_t *	uint8_ndup (const uint8_t *str, size_t size)
	strndup() for unsigned char.
VOLK_rc	escape_lit (const char *in, char **out)
	Add escape character (backslash) to illegal literal characters.
char	unescape_char (const char c)
	Unescape a single character.
uint8_t *	unescape_unicode (const uint8_t *esc_str, size_t size)
	Replace \uxxxx and \Uxxxxxxxx with Unicode bytes.
char *	fmt_header (char *pfx)
	Format an informational header.

Detailed Description

Codec interface definition and basic elements common to all codecs.

A codec can be composed by one or both of the following modules: a writer, which encodes a Volksdata graph into a serialized representation, and/or a parser, which performs the opposite operation.

A writer module can be implemented in plain C, while the parser, except for a trivial one, requires using a lexer and a parser generator, which are written in different languages and compiled into C code before building the library.

Because of this, a codec is necessarily a bit complex and consists of:

• A codec header, placed in include/volksdata/codec/codec_<name>.h. This header defines a codec singleton implementing the interface documented here. It SHOULD include the headers for the writer and/or parser defined for the module. This file MUST be included in the include/volksdata/codec.h header. See that file documentation for details.
• An optional writer header (include/volksdata/codec/writer_<name>.h), declaring the encoding functions implemented in the codec structure. The functions are implemented in src/codec/writer_<name>.c.
• An optional parser header (include/volksdata/codec/parser_<name>.h), declaring the encoding functions implemented in the codec structure.
• If the codec implements a parser, a lexer file MUST be created as src/codec/lexer_<name>.re. This file is parsed by the make codec makefile target and generates headers (include/volksdata/codec/tokens_<name>.h) and sources (src/codec/parser_<name>.c) containing the tokens and patterns for scanning the text to decode. The file is compiled by re2c, which must be installed in order to compile the lexer.
• Also, if the codec implements a parser, a grammar file MUST be created as src/codec/grammar_<name>.y. This file contains the non-terminal symbol definitions as well as referencing the terminals defined in the lexer, and the production rules for the parser. This file is compiled by Lemon. A copy of the Lemon source code is included in this repository and is compiled as part of the codec make target.

Both the parser and writer headers SHOULD include this header (codec_interface.h). This file sits at the bottom of the codec dependency chain and includes all the lower-level Volksdata function definitions.

Definition in file codec_interface.h.