This chapter is an introduction to the more advanced topic of writing an extension for Goal in Go.
Goal is designed to be easily embedded and extended in Go, using a relatively high-level API. The Go API for Goal is provided by a set of packages belonging to a single module, with no dependencies outside Go’s standard library.
As an example, we’ll implement an extension for reading zip files as Goal file system values, based on Go’s standard archive/zip package. Once done, we’ll be able to write the following code in Goal:
import"fs" / assuming GOALLIB points to goal's lib/ dir
/ open the epub file made from Goal docs (epub is zip)
'epubfs:zip.open"goal-docs-gen.epub"
/ read and display the "mimetype" file
say"mimetype: "+ 'epubfs read"mimetype"
/ use fs.ls to get all non-dir file paths in the file system
fpaths:(fs.ls epubfs)..path@&~dir
/ display number of files and list of paths
say"total %d files:\n%s"$(#fpaths),"\n"/fpaths
close epubfs
Executing the script with the extended interpreter will produce:
mimetype: application/epub+zip
total 14 files:
mimetype
EPUB/chap-FAQ.xhtml
EPUB/chap-from-k.xhtml
EPUB/chap-help.xhtml
EPUB/chap-intro.xhtml
EPUB/chap-tutorial.xhtml
EPUB/chap-working-with-tables.xhtml
EPUB/chap-writing-an-extension.xhtml
EPUB/content.opf
EPUB/index.xhtml
EPUB/nav.xhtml
EPUB/stylesheet.css
EPUB/toc.ncx
META-INF/container.xml
We
first create a
goalzipfs
directory somewhere. There, in a
main.go
file, we write:
package main
import (
"os"
"codeberg.org/anaseto/goal"
"codeberg.org/anaseto/goal/cmd"
"codeberg.org/anaseto/goal/help"
gos "codeberg.org/anaseto/goal/os"
)
func main() {
ctx := goal.NewContext() // new evaluation context for Goal code
ctx.Log = os.Stderr // configure logging with \x to stderr
gos.Import(ctx, "") // register all IO/OS primitives with prefix ""
cmd.Exit(cmd.Run(ctx, cmd.Config{Help: help.HelpFunc(), Man: "goal"}))
}
The code above is a commented copy of
cmd/goal/main.go
as found in Goal’s repository, and produces a goal
interpreter equivalent to the default one in just four
short lines of code. We’ll use this as a basis for our
extension.
As you can see, other than importing the relevant packages,
the code creates a new evaluation context with
goal.NewContext,
then configures and registers any extensions (like the
IO/OS primitives), and finally the
cmd.Run
function runs an interpreter using the created
context, handling command-line arguments too, with some
optional extra configuration for REPL help (using Goal’s
default help here). The
cmd.Exit
function handles the return value of
cmd.Run
to format any errors on stderr, and exits the program with
the appropriate status code.
Before actually compiling and running our interpreter, we
need to provide a
go.mod
file for it by running:
$ go mod init goalzipfs
The latest released version of Goal will automatically be
downloaded and added as a dependency in
go.mod
with the following command:
$ go get codeberg.org/anaseto/goal@latest
You can then build a
goalzipfs
executable in the current directory with:
$ go build
Because we haven’t implemented any extensions yet, that
goalzipfs
executable is still the same as the upstream
goal.
Goal values are represented by the opaque struct type
goal.V.
That type can represent both unboxed and boxed values.
Examples of unboxed values are integers and floats: they fit
into a struct’s field, without extra indirection. Boxed
values represent most other values, including strings,
arrays, projections, and so on. They don’t fit into the
struct: they are stored in a field as a Go interface value
type, which points to heap-allocated memory.
Goal allows extensions to define new kinds of boxed value types. In order to do so, one has to define a Go type satisfying the goal.BV interface, which is the interface implemented by all boxed Goal values. That interface is described by a set of three methods.
Append
is used for implementing
$x.
Matches
implements
x~y
for the new kind of value.
Type
implements
@x.
Boxed value types can implement extra methods when more
functionality is desired: in our case, we’ll inherit methods
from the
*zip.ReadCloser
type from Go’s standard
archive/zip
package, which satisfies the
fs.FS
file system interface, as well as the
io.Closer
interface.
// zipFS is a wrapper around zip.ReadCloser that implements the goal.BV, fs.FS
// and io.Closer interfaces.
type zipFS struct {
*zip.ReadCloser
s string // program string representation
}
// Append appends a program representation of the value to dst.
func (fsys *zipFS) Append(ctx *goal.Context, dst []byte, compact bool) []byte {
return append(dst, fsys.s...)
}
// Matches reports whether fsys~y.
func (fsys *zipFS) Matches(y goal.BV) bool {
yv, ok := y.(*zipFS)
return ok && fsys == yv
}
// Type returns "/" for file system types.
func (fsys *zipFS) Type() string {
return "/"
}
As you can see, we defined a struct type with two fields:
one of them simply embeds the
*zip.ReadCloser
type, while the extra
s
field is there simply to provide a program string
representation. Without any extra work, the
*zipFS
type represents a Goal generic value that is supported by
many primitives, with specific behavior for
$x,
@x,
and matching. It is also automatically supported by
close
and all the IO primitives working on file system values,
just because it inherits
Open
and
Close
methods from the
*zip.ReadCloser
type. To make the code compile we need to add
"archive/zip"
to the list of imports, along
"os"
and the others.
Unlike user-defined lambdas, new Goal functions implemented
in Go are variadic, like all the built-in verbs. We’ll
hence define the function that opens a zip file and produces
a
*zipFS
value as a variadic function. We’ll also need to add the
"fmt"
package to the list of imports.
Variadic functions take a
*goal.Context
argument, and a list of Goal values in stack order (last
argument is the first one).
// VFZipOpen implements the zip.open variadic function.
func VFZipOpen(ctx *goal.Context, args []goal.V) goal.V {
if len(args) > 1 {
return goal.Panicf("zip.open : too many arguments (%d)", len(args))
}
s, ok := args[0].BV().(goal.S)
if !ok {
return goal.Panicf("zip.open s : bad type %q in s", args[0].Type())
}
r, err := zip.OpenReader(string(s))
if err != nil {
return gos.NewOSError(err)
}
return goal.NewV(&zipFS{r, fmt.Sprintf("zip.open[%q]", s)})
}
The function is a straightforward wrapper around
zip.OpenReader
that does some additional argument type checking and error
processing, and provides a concrete program string
representation too (mainly useful in REPL for a file system
value). The
goal.NewV
function is used to produce a generic
goal.V
value from a Go type satisfying the
goal.BV
interface of boxed values.
All that’s left is registering the
VFZipOpen
variadic function into our main context
ctx
in the
main
function, before
cmd.Run.
ctx.RegisterMonad("zip.open", VFZipOpen)
This registers a new
zip.open
monadic verb. Often, introducing new syntax is not
desirable, so storing the function into a global instead is
preferable:
ctx.AssignGlobal("zip.open", ctx.RegisterMonad(".zip.open", VFZipOpen))
In that case, introducing new syntax with
RegisterMonad
is avoided by using an invalid identifier
".zip.open",
which is only used for identifying and display purposes.
Running
go build
again will produce an extended interpreter that can run the
Goal code shown in introduction, so you can open an EPUB
file and process it as any other kind of file system value!
As a summary, we reproduce the whole code below:
package main
import (
"archive/zip"
"fmt"
"os"
"codeberg.org/anaseto/goal"
"codeberg.org/anaseto/goal/cmd"
"codeberg.org/anaseto/goal/help"
gos "codeberg.org/anaseto/goal/os"
)
func main() {
ctx := goal.NewContext() // new evaluation context for Goal code
ctx.Log = os.Stderr // configure logging with \x to stderr
gos.Import(ctx, "") // register all IO/OS primitives with prefix ""
ctx.AssignGlobal("zip.open", ctx.RegisterMonad(".zip.open", VFZipOpen))
cmd.Exit(cmd.Run(ctx, cmd.Config{Help: help.HelpFunc(), Man: "goal"}))
}
// zipFS is a wrapper around zip.ReadCloser that implements the goal.BV, fs.FS
// and io.Closer interfaces.
type zipFS struct {
*zip.ReadCloser
s string // program string representation
}
// Append appends a program representation of the value to dst.
func (fsys *zipFS) Append(ctx *goal.Context, dst []byte, compact bool) []byte {
return append(dst, fsys.s...)
}
// Matches reports whether fsys~y.
func (fsys *zipFS) Matches(y goal.BV) bool {
yv, ok := y.(*zipFS)
return ok && fsys == yv
}
// Type returns "/" for file system types.
func (fsys *zipFS) Type() string {
return "/"
}
// VFZipOpen implements the zip.open variadic function.
func VFZipOpen(ctx *goal.Context, args []goal.V) goal.V {
if len(args) > 1 {
return goal.Panicf("zip.open : too many arguments (%d)", len(args))
}
s, ok := args[0].BV().(goal.S)
if !ok {
return goal.Panicf("zip.open s : bad type %q in s", args[0].Type())
}
r, err := zip.OpenReader(string(s))
if err != nil {
return gos.NewOSError(err)
}
return goal.NewV(&zipFS{r, fmt.Sprintf("zip.open[%q]", s)})
}
At this point, you might want to look at the
API docs
of the module’s various packages. As for examples of
extensions, the
os
built-in package is actually written as an extension: it can
be a good first place to look at. Moreover, there are
currently
math
and
io/fs
packages in the repos,
as well as an
archive/zip
package that provides the functionality we implemented in
this tutorial.
We could just have imported it in a similar way as the
os
one.
If you’re also interested in the internals, the
docs/implementation.md
file in goal’s repository gives a short introduction.