Literate Minecraft data packs and resource packs.
Project description
Lectern
Literate Minecraft data packs and resource packs.
@function tutorial:greeting
say Hello, world!
Introduction
This markdown file is interspersed with code fragments describing the content of a Minecraft data pack. Using lectern
, you can turn this single file into an actual data pack that can be loaded into the game.
Features
- Turn markdown files into data packs and resource packs
- Merge resources from several markdown files
- Convert data packs and resource packs into markdown snapshots
- Can be used as a
beet
plugin - Highly extensible with custom directives
- Automatically integrates with
pytest-insta
Hmmkay but why?
- Editing data packs involves a lot of jumping around between files, for simple use-cases a single file is a lot easier to work with
- Minecraft packs aggregate various types of files that can have complex interactions with each other, a literate style allows you to document these interactions fluently
- Human-readable, single-file data pack and resource pack snapshots can be really useful to diff and track regressions in Minecraft-related tooling
Installation
The package can be installed with pip
.
$ pip install lectern
Getting started
This is an example of a markdown file that can be turned into a data pack:
# Beginner tutorial
Let's start by creating a simple function:
`@function tutorial:greeting`
```mcfunction
say Hello, world!
```
And now we can make it run when the data pack is loaded!
`@function_tag minecraft:load`
```json
{
"values": ["tutorial:greeting"]
}
```
You can use the lectern
command-line utility to turn the markdown file into a data pack.
$ lectern tutorial.md --data-pack path/to/tutorial_data_pack
If you're using beet
you can use lectern
as a plugin in your pipeline.
{
"pipeline": ["lectern"],
"meta": {
"lectern": {
"load": ["*.md"]
}
}
}
Document formats
lectern
implements two closely-related document formats: markdown and plain text. The markdown format builds upon the plain text format.
The markdown format lets you present the various elements of your data pack or resource pack and how they fit together. It's a format that's meant to support literate programming. You can use it when your document is meant to be read by other people. It allows you to emphasize the important parts, explain tradeoffs and discuss alternatives, implementation details, etc...
`@function tutorial:greeting`
```mcfunction
say Hello, world!
```
On the other hand if you don't intend to produce literate documents you can use the plain text format to author data packs and resource packs as a single file without having to deal with markdown formatting.
@function tutorial:greeting
say Hello, world!
Directives
Data pack and resource pack fragments are code blocks, links or images annotated with a special lectern
directive. Directives are prefixed with the @
symbol and can be followed by zero or more arguments.
@<directive_name> <arg1> <arg2> <arg3>...
lectern
provides directives for including namespaced resources inside data packs and resource packs. These built-in directives all expect a single argument specifying the fully-qualified resource name.
@function tutorial:greeting
@function_tag minecraft:load
Here is a reference of all the supported resources:
Data pack | Resource pack |
---|---|
@advancement |
@blockstate |
@function |
@model |
@loot_table |
@font |
@predicate |
@glyph_sizes |
@recipe |
@truetype_font |
@structure |
@shader_post |
@block_tag |
@shader_program |
@entity_type_tag |
@fragment_shader |
@fluid_tag |
@vertex_shader |
@function_tag |
@text |
@item_tag |
@texture_mcmeta |
@dimension_type |
@texture |
@dimension |
|
@biome |
|
@configured_carver |
|
@configured_feature |
|
@configured_structure_feature |
|
@configured_surface_builder |
|
@noise_settings |
|
@processor_list |
|
@template_pool |
|
@item_modifier |
There are also two built-in directives that can be used to include files using a path relative to the root of the data pack or the resource pack.
@data_pack pack.mcmeta
@resource_pack pack.png
@resource_pack assets/minecraft/textures/block/kelp_plant.png.mcmeta
This is useful for adding files that aren't part of any particular namespace.
If you're using lectern
as a beet
plugin you will be able to require plugins dynamically wih the @require
directive.
@require my_plugins.hello
Finally, the @skip
directive is simply ignored and allows you to end a previous fragment in the plain text format.
@function tutorial:greeting
say Hello, world!
@skip
This will not be included in the output.
Code block fragments
You can include the content of a code block in a data pack or a resource pack by preceding it with a directive surrounded by backticks.
@function tutorial:greeting
say Hello, world!
You can put the directive in an html comment to make it invisible. Here the code block is annotated with the following comment:
<!-- @function_tag minecraft:load -->
{
"values": ["tutorial:greeting"]
}
When using backticks you can surround the code block in a <details>
element to make the code fragment foldable.
@function tutorial:greeting
say Hello, world!
The directive can also be embedded directly inside the code block. You can insert a directive preceded by either #
or //
and the following lines will be included in the specified file.
# @function tutorial:obtained_dead_bush
say You obtained a dead bush!
Embedded directives are stripped from the output. You can use multiple directives in a single code block.
// @loot_table minecraft:blocks/diamond_ore
{
"pools": [
{
"rolls": 1,
"entries": [
{
"type": "minecraft:item",
"name": "minecraft:dead_bush"
}
]
}
]
}
// @advancement tutorial:obtained_dead_bush
{
"criteria": {
"dead_bush": {
"trigger": "minecraft:inventory_changed",
"conditions": {
"items": [
{
"item": "minecraft:dead_bush"
}
]
}
}
},
"requirements": [
[
"dead_bush"
]
],
"rewards": {
"function": "tutorial:obtained_dead_bush"
}
}
Link fragments
Link fragments make it possible to refer to external files, online assets, and to embed binary files in the markdown as data urls. You can create a link fragment by turning a directive surrounded by backticks into a markdown link.
@loot_table minecraft:blocks/yellow_shulker_box
The link itself can be a path to a local file or any url supported by the built-in urlopen
function.
Image fragments
You can include inline markdown images in the output data pack or resource pack by preceding the image with a directive surrounded by backticks.
@data_pack pack.png
Image fragments support the same variations as code block fragments. You can put the directive in a comment or surround the image with a <details>
element to make it foldable.
Modifiers
The behavior of particular directives can be adjusted with modifiers. A modifier is specified between parentheses right after the name of the directive.
@<directive_name>(<modifier>) <arg1> <arg2> <arg3>...
The append
modifier is implemented by all the text-based built-in namespaced resource directives and makes it possible to concatenate the content of the fragment to the already-existing content.
@function(append) tutorial:greeting
say This is added afterwards.
The merge
modifier is similar but instead of concatenating the contents it uses the beet
merging strategy to combine the fragment with the existing file.
@function_tag(merge) minecraft:load
{
"values": ["#tutorial:something_else"]
}
There are also modifiers that are applied to the content of the fragment directly. The base64
modifier will decode the content of the code fragment as base64.
@function_tag(base64) tutorial:something_else
ewogICJ2YWx1ZXMiOiBbInR1dG9yaWFsOnN0cmlwcGVkIl0KfQ==
Finally, there's a strip_final_newline
modifier that removes the final newline at the end of code block fragments. It's mostly used to make sure that lectern
snapshots can reconstruct the original content byte for byte in case the file wasn't terminated by a newline.
@function(strip_final_newline) tutorial:stripped
say This function doesn't have a final newline.
Command-line utility
$ lectern --help
Usage: lectern [OPTIONS] [PATH]...
Literate Minecraft data packs and resource packs.
Options:
-d, --data-pack <path> Extract data pack.
-r, --resource-pack <path> Extract resource pack.
-e, --external-files <path> Emit external files.
-v, --version Show the version and exit.
-h, --help Show this message and exit.
You can extract data packs from markdown files with the -d/--data-pack
option. If the name ends with .zip
the generated data pack will be zipped. Multiple markdown files can be merged together into a single data pack.
$ lectern demo.md --data-pack demo_data_pack
$ lectern demo.md -d demo_data_pack
$ lectern demo.md -d demo_data_pack.zip
$ lectern foo.md bar.md -d demo_data_pack
The -r/--resource-pack
option lets you do exactly the same thing but with resource packs. The two options can be combined to extract a data packs and a resource pack at the same time.
$ lectern demo.md --resource-pack demo_resource_pack
$ lectern demo.md -r demo_resource_pack
$ lectern demo.md -d demo_data_pack -r demo_resource_pack
You can also convert a combination of data packs and resource packs into a single markdown file.
$ lectern demo_data_pack demo.md
$ lectern demo_data_pack.zip demo.md
$ lectern demo_data_pack demo_resource_pack demo.md
$ lectern foo_data_pack bar_data_pack demo.md
The last argument is the name of the generated markdown file. By default, the lectern
utility will bundle binary files into the markdown file as data urls. You can use the -e/--external-files
option to dump the binary files in a given directory instead.
$ lectern demo_data_pack demo.md --external-files files
$ lectern demo_data_pack demo.md -e files
$ lectern demo_data_pack demo.md -e .
All these commands also work with plain text files. lectern
will only use the markdown document format when the filename ends with .md
.
Python API
The API revolves around Document
objects. A lectern
document holds a DataPack
and a ResourcePack
, as well as a dictionary defining the usable directives. The extractors and serializers are also exposed on the document to make it possible to swap them out with custom ones if needed.
from beet import DataPack, ResourcePack
from lectern import Document
document = Document()
assert document.data == DataPack()
assert document.assets == ResourcePack()
The constructor makes it possible to provide existing DataPack
and ResourcePack
instances, some initial text or markdown content, or a path from which to load an existing lectern
document.
Document(data=DataPack(), assets=ResourcePack())
Document(text=...)
Document(markdown=...)
Document(path="path/to/document.md")
Document
instances will compare equal if the underlying data packs and resource packs also compare equal.
You can use the load
method to read a markdown or a plain text file and update the internal data pack and resource pack with the extracted fragments.
document.load("path/to/document.md")
If you already have some text or markdown ready to go, you can use the add_text
and add_markdown
methods.
document.add_text(...)
document.add_markdown(...)
If the markdown content refers to local files you can specify the directory from which the external files should be loaded from with the external_files
argument.
document.add_markdown(..., external_files="path/to/directory")
You can use the get_text
and get_markdown
methods to serialize the entire content of the internal data pack and resource pack. By default the get_markdown
method will produce markdown that embeds binary files as data urls. You can enable emit_external_files
and optionally provide a path prefix to generate a dictionary of associated files instead.
text = document.get_text()
markdown = document.get_markdown()
markdown, external_files = document.get_markdown(emit_external_files=True)
markdown, external_files = document.get_markdown(emit_external_files=True, prefix="path/to/directory")
Finally, the save
method lets you serialize and write the document to a given path. If the filename ends with .md
the generated markdown will bundle binary files as data urls by default. You can use the external_files
argument to emit the binary files in the given directory instead.
document.save("path/to/document.txt")
document.save("path/to/document.md")
document.save("path/to/document.md", external_files="path/to/files")
Custom directives
Directives are simply callable objects that receive the document fragment, the resource pack, and the data pack as arguments.
from beet import DataPack, ResourcePack, Function
from lectern import Document, Fragment
def my_directive(fragment: Fragment, assets: ResourcePack, data: DataPack):
num1, num2 = fragment.expect("num1", "num2")
result = int(num1) + int(num2)
data["demo:output_result"] = Function([f"say {result}"])
document = Document()
document.directives["my_directive"] = my_directive
document.add_text("@my_directive 32 10")
assert document.data.functions["demo:output_result"] == Function(["say 42"])
The expect
method allows you to unpack the directive arguments and automatically raises an error if the user didn't specify the arguments properly. You can use the as_file
method to get the content of the fragment as a specific type of file.
def repeated_function(fragment: Fragment, assets: ResourcePack, data: DataPack):
full_name, count = fragment.expect("full_name", "count")
function = fragment.as_file(Function)
function.lines *= int(count)
data[full_name] = function
The as_file
method will take care of reading the file or downloading it if the directive is used with a link fragment. It will also handle the base64
and strip_final_newline
modifiers.
You can handle custom modifiers by checking the content of the modifier
attribute.
Beet plugin
Using lectern
as a beet
plugin makes it possible to combine your markdown files with arbitrary beet
plugins for further processing. The plugin can load files using the plain text and markdown document formats and emit a snapshot of the beet
context at the end of the build.
{
"pipeline": ["lectern"],
"meta": {
"lectern": {
"load": ["*.md"],
"snapshot": "out/snapshot.md",
"external_files": "out"
}
}
}
You can require the plugin programmatically by using the lectern
plugin factory.
from beet import Context
from lectern import lectern
def my_plugin(ctx: Context):
ctx.require(
lectern(
load=["*.md"],
snapshot="out/snapshot.md",
external_files="out",
)
)
All the configuration is optional. The plugin is a no-op if the load
or snapshot
options are not specified.
You can retrieve the Document
instance with the inject
method. This is useful for adding custom directives.
from beet import Context, DataPack, ResourcePack, Function
from lectern import Document, Fragment
def hello_directive(ctx: Context):
""""Plugin that defines the `@hello <name>` directive."""
document = ctx.inject(Document)
document.directives["hello"] = hello
def hello(fragment: Fragment, assets: ResourcePack, data: DataPack):
name = fragment.expect("name")
function = data.functions.setdefault("hello:greetings", Function([]))
function.lines.append(f"say Hello, {name}!")
It's worth mentioning that lectern
uses the beet
cache to avoid downloading link fragments repeatedly and keeping your build snappy, especially in watch mode. If you need to re-download link fragments you can clear the lectern
cache.
$ beet cache --clear lectern
You can also use a plugin to configure a custom cache timeout if you want to make sure that your assets are re-downloaded periodically.
from beet import Context
def download_every_day(ctx: Context):
ctx.cache["lectern"].timeout(hours=24)
Snapshot testing
A lot of Minecraft tooling involves generating data packs and resource packs. Writing tests for this kind of tooling takes time because you need to painstakingly compare everything that you care about with a reference value. This makes it hard to get good coverage, and then even harder to keep making changes to the code being tested afterwards. You're trading robustness and stability for a shackle that massively slows down development.
That's where snapshot testing comes into play. Snapshot testing allows you to record a reference value and then make sure that your code keeps producing the same results. It provides the necessary tools for reviewing snapshots and updating them as your project evolves.
lectern
documents are useful as snapshot formats because they represent entire data packs and resource packs in a single file that's human-readable and diff-friendly.
pytest-insta
is an extensible snapshot testing plugin for pytest
. When it's installed, lectern
defines three additional snapshot formats.
Extension | Format description |
---|---|
.pack.txt |
Plain text snapshot. |
.pack.md |
Markdown snapshot with bundled binary files. |
.pack.md_external_files |
Directory with a README.md that refers to external binary files. |
You can use these snapshot formats when comparing Document
objects with the snapshot
fixture.
def test_generate(snapshot):
data = generate_some_data_pack()
assert snapshot("pack.txt") == Document(data=data)
If you're using the beet
toolchain, keep in mind that you can get a Document
instance bound to the context object by using the inject
method.
def test_generate_with_beet(snapshot):
ctx = run_beet(...)
assert snapshot("pack.txt") == ctx.inject(Document)
This will save the entire data pack and resource pack in the snapshot. For more details about working with the generated snapshots check out the pytest-insta
documentation.
Contributing
Contributions are welcome. Make sure to first open an issue discussing the problem or the new feature before creating a pull request. The project uses poetry
.
$ poetry install
You can run the tests with poetry run pytest
.
$ poetry run pytest
The project must type-check with pyright
. If you're using VSCode the pylance
extension should report diagnostics automatically. You can also install the type-checker locally with npm install
and run it from the command-line.
$ npm run watch
$ npm run check
The code follows the black
code style. Import statements are sorted with isort
.
$ poetry run isort lectern tests
$ poetry run black lectern tests
$ poetry run black --check lectern tests
License - MIT
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.