LEP-003: Code Generation


Created:Tuesday, August 23, 2022
Author:David Delassus
Category:Compiler Architecture
Status:FINAL

Abstract

This LEP specifies the techniques used by the compiler to generate code in the target language.

Rationale

After parsing the source code into an Abstract Syntax Tree, the compiler needs to generate valid Rust code that can be compiled.

Multiple solutions can achieve this goal:

  • generating Rust AST
  • generating Rust source code, compiling it with rustc
  • generating a Cargo[1] project

Specification

Target Directory

Generated code is located in a folder named .llbuild with the following structure:

flowchart TB .llbuild/ --> Cargo.toml .llbuild/ --> modules/ .llbuild/ --> executable/

The Cargo.toml file in this folder describe a Cargo workspace, whose members are the crates located in the modules/ and executable/ folders.

Module generation

Each Letlang module will be translated to a library crate.

For example, the following module:

module example::main;

# module code

Will produce a crate with the following structure:

flowchart LR .llbuild/ --> modules/ modules/ --> lldep_example_main/ lldep_example_main/ --> Cargo.toml lldep_example_main/ --> src/ src/ --> lib.rs

Executable generation

One of the modules is defined as a main module. This module must provide a public main function.

The compiler will generate an executable crate which depends on that module. Its role is to start the node and spawn the main process:

flowchart LR .llbuild/ --> executable/ executable/ --> Cargo.toml executable/ --> src/ src/ --> main.rs

Dependency resolution

Every time a module use the import statement, the imported module is added as a dependency to the library crate (in the Cargo.toml file).

For external dependencies (standard library, external projects, …), the toolchain will fetch the dependency and compile the library crates in the .llbuild/modules/ folder accordingly.

Generating Rust code

Using the Rust crate askama[2], the compiler uses templates to transform the Letlang AST into source-code.

Finally, the rustfmt binary should be executed on the output, to facilitate debugging.

Rejected Ideas

Generating Rust AST

The Rust crate rustc_ast[3] is unstable and subject to change.

There are too few ressources at the moment to target the AST directly.

Calling rustc directly

Cargo simplifies the crate dependency management, avoiding re-inventing the wheel.

References

ReferenceTitleLink
1Cargohttps://doc.rust-lang.org/cargo/
2askama Rust cratehttps://docs.rs/askama/latest/askama/
3rustc_ast Rust cratehttps://doc.rust-lang.org/beta/nightly-rustc/rustc_ast/index.html