archeme 1.0.2

Command-line tool to describe and visualize architecture schemes.

Archeme

Archeme is a tool to describe and visualize schemes and diagrams. It’s designed primarily for architectural schemes.

Archeme may be used as a stand-alone command-line tool, or as a Python package.

To describe schemes and diagrams, Archeme provides YAML-based DSL (domain-specific language).

To draw them, Archeme uses Graphviz. The engines dot, neato, and fdp are supported.

In fact, Archeme provides some new abstraction level over Graphviz. Unlike PlantUML, Archeme almost doesn’t limit your ability to use the flexible functionality of Graphviz. However, Archeme provides more convenient means to describe elements of the same types using classes, to specify the relative positions of elements, etc. Using YAML as the base of DSL makes it possible to work with diagram descriptions as with data structures—in Archeme itself and in applications integrated with it. So, Archeme can easily combine several scheme descriptions into one.

By using SVG as an output format, you may assign hyperlinks to scheme elements and thereby organize relationships between schemes—for example, in accordance with the C4 methodology.

Installation

To install Archeme, run the command:

$ pip install archeme

After installation, the command archeme will be available in your system.

CLI Usage

Currently Archeme provides 2 actions (commands):

• generate—to generate graph description that can be drawn with Graphviz from YAML-based DSL source;
• merge—to merge (combine, join) multiple DSL sources into a single DSL source.

The generate Command Syntax

Generalized syntax of the generate command is the following:

$ archeme generate [-c|--config <config_file>] -i|--input <input_file> -o|--output <output_file> [-d|--draw <format>]

The -i/--input and -o/--output arguments are required, other arguments are optional.

The -i/--input parameter is used to specify YAML-based DSL source.

The -o/--output parameter points to the output file—graph description for Graphviz.

You may specify an optional config file by using the -c/--config argument. Both config and scheme description files use the same YAML-based DSL. It’s recommended to use a separate config file to specify some common settings that may be reused in multiple scheme descriptions of the same style. If both config file and scheme description set the parameter of the same name, the value specified in scheme description will override the value specified in config.

If the -d/--draw argument is given, Archeme will try to call Graphviz to draw an image from generated graph description, and the Graphviz calling command will be displayed. By default, Archeme doesn’t call Graphviz to visualize generated graph descriptions. The -d/--draw argument may take any output format value allowed by Graphviz (png, svg, etc.). By default, the dot engine is used. You may specify the necessary engine explicitly in DSL, see below.

The merge Command Syntax

Generalized syntax of the merge command is more simple:

$ archeme merge -i|--input <input_file> -o|--output <output_file>

The command takes 2 mandatory arguments: -i/--input for input YAML-based DSL file, and -o/--output for output YAML-based DSL file. Output files of the merge command should be used as input files for the generate command. YAML-based DSL is described below.

YAML-Based DSL

• See also: Archeme DSL Quick Reference

To describe schemes and diagrams, YAML-based DSL is provided.

DSL, same as Graphviz, operates the graph theory terms: so, graph is the whole drawing; node is a functional component of a scheme or a diagram; cluster is a group of nodes that is usually highlighted visually; edge is a line that represents a relation between two nodes.

DSL may be used to:

• specify common settings of multiple elements;
• define scheme structure and relations between elements;
• describe each separate element—node, cluster, edge;
• set relative positions of elements;
• describe how it’s needed to combine multiple scheme descriptions into a single one.

Common Setting

Common settings for multiple elements usually should be specified in the config file.

Graphviz Engine

The engine parameter sets the certain Graphviz engine to draw images. Archeme supports the dot, neato, and fdp engines. The default engine is dot.

Code example:

engine: neato

You may read about the difference between engines in the Graphviz documentation.

In relation to architectural schemes:

• dot is suitable for models with hierarchical relations; using dot, you can’t strictly control relative positions of elements—the positions depend of ranks of elements; dot supports clusters;
• neato allows to set strict fixed positions to each node; it doesn’t support clusters;
• fdp is similar to neato but it supports clusters; however, if clusters are used, fdp re-calculates the positions of nodes, they can’t be strictly fixed.

Settings Of Elements

In elements section, you may specify common settings of the element of different types—graph, node, cluster, edge. In the simplest case, DSL maps to Graphviz syntax. DSL code example:

elements:
    graph:
        newrank: true
        rankdir: TB
    node:
        shape: box
        fixedsize: true
        width: 4
        height: 1
        style:
            - filled
            - rounded
    edge:
        dir: both
        arrowtail: dot

This example will be converted into the following description for Graphviz:

graph [newrank = "true", rankdir: "TB"]
node [shape = "box", fixedsize = "true", width = "4", height = "1", style = "filled, rounded"]
edge [dir = "both", arrowtail = "dot"]

All available parameters for graphs, nodes, edges are described in the Graphviz documentation.

In addition to graph, node, and edge parameters, Archeme supports the analogous cluster parameter that allows to specify default settings for clusters. Code example:

elements:
    cluster:
        labelloc: b
        labeljust: l
        shape: box

Note that you may reduce the number of lines of YAML-based DSL code. So, node settings from the example above may look like this:

node {shape: box, fixedsize: true, width: 4, height: 1, style: [filled, rounded]}

Archeme allows to specify parameters that take lists of values, in different ways. The constructions style: [filled, rounded], style: filled, rounded, style: [filled], style: filled are valid.

Note that if a node, cluster, or edge has some custom style or another parameter with a list of values, it will fully override the default style or whatever. This is how it works in Graphviz, and Archeme doesn’t change this behavior for compatibility reasons.

For example, if the default style for nodes is [filled, rounded], and the custom style of some node is dashed, only dashed will be applied to this node.

To control the settings of elements more flexible, classes are provided.

Classes Definition

You may define one or more classes for each type of elements (node, cluster, edge), and then assign any combination of defined classes to any separate element. Code example:

elements:
    node:
        fixedsize: true
        penwidth: 3
        classes:
            generic:
                shape: box
                width: 4
                height: 1
                style: rounded
            database:
                shape: cylinder
                width: 3
                height: 3
            external_network:
                shape: circle
                width: 2.5
                height: 2.5
                style: filled
                fillcolor: '#f0f0f0'

In this example, 3 classes of nodes with the names generic, database, and external_network are defined. Note that using of classes doesn’t disallow to use global default settings.

Grid Settings

To control the relative positions of elements, Archeme provides the ability to describe nodes arrangement as a text grid. In the elements section of common settings you may define the horizontal and vertical intervals of the grid step. Code example:

elements:
    grid:
        spacing_x: 400
        spacing_y: 250

The spacing_x parameter sets the horizontal interval and defaults to 500; the spacing_y parameter sets the vertical interval and defaults to 200. Grid spacing units are points; 1 point equals to 1/72 inch.

Note that neato and fdp engines use different units for the pos attribute: points and inches respectively. Archeme takes it into account. If you specify engine: neato, do not try to draw the resulting graph description with fdp, and vice versa. The dot engine doesn’t support positioning at all.

Scheme Description

To describe a scheme or a diagram, it’s needed to:

• specify scheme structure—define nodes and clusters;
• specify relations between nodes—define edges;
• control the relative positions of nodes:
  • define a text grid, if the neato or fdp engine is used;
  • specify which nodes should have the same rank, if the dot engine is used.

Structure

To define nodes and clusters, Archeme DSL provides the structure section. You may assign Graphviz attributes and Archeme classes to any node and any cluster. For nodes, the id parameter is only required. All other parameters are optional. The id parameter is used to assign a text identifier to a certain node.

Simple code example:

structure:
    - node:
        id: first
    - node:
        id: second

In this example, 2 nodes with the identifiers first and second are defined.

Note that the value of the structure parameter must be a list. If the dot engine is used, the order of nodes matters for their positioning. If the neato or fdp engine is used, the order of nodes doesn’t matter.

Clusters should be specified at the same level as nodes. Each cluster has nested structure that may include nodes and nested clusters. Code example:

structure:
    - node:
        id: first
    - node:
        id: second
    - cluster:
        structure:
            - node:
                id: third
            - node:
                id: fourth

The following code example shows how it’s possible to assign Graphviz attributes and Archeme classes to nodes and clusters:

structure:
    - node:
        id: first
        label: The First Node
        class:
            - amazing
            - awesome
    - cluster:
        label: The Wonderful Cluster
        style: filled
        fillcolor: '#cccccc'
        class: wonderful
        structure:
            ...

Single class name may be specified as a string; multiple class names should be specified as a list.

Archeme merges attributes of classes. Suppose 2 classes are defined as the following:

classes:
    amazing:
        shape: box
        width: 4
        height: 1
        style:
            - rounded
            - filled
    awesome:
        height: 2
        style:
            - dashed
        fillcolor: '#99ccff'

If both styles are assigned to some node in the order: [amazing, awesome], the resulting attributes assigned to the node will be:

shape: box
width: 4
height: 2
style:
    - rounded
    - filled
    - dashed
fillcolor: '#99ccff'

So, the style attributes will be combined. Contradicting values of the attributes of the same names will be taken from the last class in the sequence (like height in this example).

Relations

To show relations between nodes, edges are used. To define a list of edges, use the edges section. Simple code example:

edges:
    -   tail: first
        head: second
    -   tail: second
        head: third

In this example, 2 edges are defined. These edges specify relations between 3 nodes with the identifiers first, second, and third.

Archeme uses directed graphs only. So each edge should have a tail (“source,” “beginning”) and a head (“target,” “ending”). For example, if an edge represents interaction between a client and a server, it will be alright if a tail will be assigned to a client, and a head—to a server. However, client requests and server responses may be shown as two different counter-directed edges.

You may assign Graphviz attributes and classes to edges like in case with nodes and clusters. Example:

edges:
    -   tail: first
        head: second
        label: HTTP API
        class: two_arrows

Archeme provides extended syntax for the label and xlabel attributes of edges. You may specify protocol and describe transferred data:

    -   tail: first
        head: second
        label:
            protocol: HTTP
            data: Admin API

This code will be transformed into the following construction in a graph description:

"first" -> "second" [label = <<b>HTTP</b><br />Admin API>];

Positioning

Archeme implements a powerful concept to control relative positions of scheme elements. Positioning may be described as a multiline text grid. Suppose you have 10 nodes with the identifiers from first to tenth. Feel free to set their positions in this visual way:

grid: |
    -        -    second    sixth
    first         third
                  fourth               eighth
                                       ninth
                  fifth     seventh
                                       -
                                       tenth

If the engine neato or fdp is used, the nodes will be strictly positioned with the pos attribute. Positions will be calculated using the values of the elements.grid.spacing_x and elements.grid.spacing_y parameters. A hyphen (-) means skipped step if it’s necessary not to place any nodes to the respective horizontal or vertical coordinate.

Grid steps are defined by start positions of node identifiers. So, the nodes second, third, fourth, and fifth from the example will be placed to the third horizontal step of the grid.

The dot engine ignores the pos attribute, but it allows to place nodes with the same rank to the same hierarchy level of the resulting layout. If the dot engine is used, Archeme analyzes a grid and defines groups (subgraphs) of nodes with the same ranks. For the example above and the elements.graph.rankdir attribute set to LR (left to right), Archeme will generate the following groups:

• second, third, fourth, fifth;
• sixth, seventh;
• eighth, ninth, tenth.

If the elements.graph.rankdir attribute is set to TB (top to bottom, default in Graphviz), for the example above Archeme will make the following groups:

• second, sixth;
• first, third;
• fourth, eighth;
• fifth, seventh.

If you don’t plan to use the engines neato and fdp to draw a certain scheme, and use only dot, you don’t need to describe grids. For dot, instead of using grids, you may define groups of nodes of the same ranks explicitly. Example of code that represents the last described case:

groups:
    -   - second
        - sixth
    -   - first
        - third
    -   - fourth
        - eighth
    -   - fifth
        - seventh

Combining Multiple Schemes

To combine multiple schemes or diagrams into a single one, the merge Archeme command is used. Archeme merges multiple scheme descriptions using the concept of modules.

A module is a scheme description that is used as a part of the structure of the resulting combined scheme description. Optionally some nodes of a certain module may be excluded from the resulting combined scheme description.

Each module should have an identifier. To avoid conflicts when some nodes have the same identifiers in different modules, node identifiers are combined with module identifiers in the resulting scheme description. For example, the node api from the module backend will get the identifier backend.api in the resulting combined scheme description.

Archeme works with modules that are described in separate files. To include some module into the resulting scheme structure, you should specify module identifier and module description file. Code example:

structure:
    - module:
        id: backend
        file: path/to/backend.yml
    - module:
        id: frontend
        file: path/to/frontend.yml

It seems most convenient to describe modules in separate files, if Archeme is used as a stand-alone command-line tool. However, Archeme allows to specify a module description directly as a value of the description parameter that should be used instead of the file parameter. This way may be preferred when Archeme is used as a Python package in a third-party application.

In this example, 2 modules with the identifiers backend and frontend are added to the resulting structure.

If you need not to include some module nodes into the resulting structure, use the optional exclude parameter:

structure:
    - module:
        id: frontend
        file: path/to/frontend.yml
        exclude:
            - client_device
            - cdn

In this example, the nodes with the identifiers client_device and cdn of the module frontend will not be taken into account in the resulting combined scheme description.

Feel free to add nodes, clusters, and edges to the resulting scheme description in the usual way:

structure:
    - module:
        id: backend
        file: path/to/backend.yml
    - module:
        id: frontend
        file: path/to/frontend.yml
    - node:
        id: middleware
edges:
    -   tail: frontend.application
        head: middleware
    -   tail: middleware
        head: backend.api

Archeme provides smart merging of grids. Suppose you need to combine 4 modules with the identifiers one, two, three, and four. You may describe a grid that should be used in the resulting scheme description:

grid: |
    one      two
    three    four

Archeme will generate the resulting grid in which, for example, the leftmost nodes of the module two will be placed at the same horizontal position as the leftmost nodes of the module four.

When merging modules, Archeme deeply processes the structure, edges, grid, and groups parameters of each module. Common parameters for multiple elements such as engine and elements will be merged using the simplest algorithm. It’s best to specify them in a separate config file that will not be involved in the merge process.

Examples

Archeme repository includes some examples of architectural schemes.

Each example is located in its own folder that has the following structure:

• source/*—YAML-based DSL sources;
• target/*—generated graph descriptions for Graphviz, and resulting PNG images;
• draw.sh—shell script that executes the commands which create target files.

PNG images drawn with Graphviz are shown below.

1. Foliant Architecture, Simple, Drawn With neato

• Directory with the example files

2. Foliant Architecture, Pretty, Drawn With neato

• Directory with the example files

3. Foliant Architecture, With A Cluster, Drawn With fdp

• Directory with the example files

4. Foliant Architecture, With More Clusters, Drawn With fdp

• Directory with the example files

5. Foliant Architecture, With Many Nested Clusters, Drawn With dot

• Directory with the example files

6. Digital Television System, Two Subsystems, Two Styles, Drawn With dot

• Directory with the example files

Style 1

Subsystem 1

Subsystem 2

Whole System, Reuses Subsystems Descriptions

Style 2

Subsystem 1

Subsystem 2

Whole System, Reuses Subsystems Descriptions

Clouds and client devices icons are made by iconixar.