TCL scripting in userspace

Backstory

This is the lore of this side-quest.

Basically, I needed a basic scripting language to automate stuff in userspace, for eg. starting USB pollers, listing out devices / processes after the system boots up and so on. ce (the shell) already supports that via it’s mediocre "scripting capabilities", but that wasn’t enough. I needed a language that implements logic and flow control like if, while, break, continue, for and so on. Also variables are pretty useful.

The final goal, which is not yet achieved, is to throw out the /sys/init.cmd ce script entirely and replace it with the new more capable language. Possibly even replace ce itself and just move the edit editor into a separate application, but those are plans for the far future.

Marking out the requirements

Firstly, I w wanted to make a list of interpreters to consider when building my scripting environment. I wanted to rely on existing interpreters, because writing one would take too much time and attention from the project and I would just get lost entirely on this side-quest.

After some research and planning, I’ve managed to mark out these requirements:

No custom / sophisticated build system allowed. You see, MOP3 uses it’s own way of building user applications via a build system made on top of sets of makefiles. This makes it super easy to write my own applications, but makes it super-duper difficult and annoying to port apps with their own special way of being built. Ideally I can just throw the interpreter’s source files into a directory, put Makfile, src.mk and app.mk in there and it will automagically integrate itself into the build process.
Minimal / basic libc support requirement. MOP3’s libu is a very small C library and is does not comply with the standard in a lot of ways. Some functions are standard though, which makes porting a little easier, but we must not assume full Linux / POSIX-compliant environment.
Simple source code to be understood by one person within a short amount of time. I want something that I can quickly get a grasp of and extend / modify to my needs.
Generally okay performance, which also ties into 3.

Potential candidates

This is the list of projects that caught my eye during research:

MicroPython
- Pros:
  - Is Python, so syntax will be familiar and the language will be easy to use.
  - Assumes bare-metal environment, so it would work with our minimal C library support.
- Cons:
  - Has it’s own build system and way of porting to other platforms (ports/), so it doesn’t integrate well into the proejct.
Berry
- Pros:
  - I already have experience with the code base and have ported it before.
  - Simple python-like language.
  - Doesn’t require much in terms of C library support.
- Cons:
  - Uses external tools to generate a parser, which makes the build process not straightforward.
MuJS
- Pros:
  - Is JavaScript, so using it will be super easy.
  - Nice stack-based API for creating native C functions.
  - No build system. Just throw the .c files into the project and go!
- Cons:
  - Requires a bit better C library support, because it assumes a hosted environment.
  - Suuuuper slow boot up time, due to object prototypes needing to be created.
PICOL (TCL)
- Pros:
  - While I don’t know (at the time) TCL, it’s a simple language and I can learn it quickly.
  - Super simple codebase of ~500 LOCs.
  - Minimal C library support required.
  - Everything in one file and no sophisticated build system.
- Cons:
  - A bit bare and will require a lot more work on my part to add useful features.

The winner

The winner is… MuJS? What? I thought we were doing TCL in this article?

MuJS - first try

MuJS was very easy to set up and get going. You just download the zipped release sources from their website and throw the files into your project. No build system, no generated configs, no nothing - just a bunch of .c files. Also in terms of required C library support, I only had to implement like 2-3 new string functions and that’s it.

Unfortunately it’s not so beautiful. As it turns out, MuJS is extremely slow to boot up. Once it’s running it’s quite fast, but the startup is very cold, which makes it not ideal for scripts. Why is this? MuJS at start needs to create all needed object prototypes and built-in classes, which does lots of micro allocations. I assume this is a design choice, because Artifex (company behind MuJS) wants the runtime to work under MuPDF, where the start up is already cold, so we can wait a second for the interpreter to start - and what really matters is interpreter speed, while it’s running. Makes sense, but I had to ditch MuJS unfortunately.

PICOL (TCL)

I’ve never had any experience with TCL programming haha. I randomly found PICOL on GitHub and saw that the implementation is fairly simple and can be easily incorportated into the project (literally one file). It’s also super easy to add custom commands using picol_register_command(). The boot up time of PICOL is actually non-existant and the whole interpreter runs very smoothly.

This is my playground script located in etc/hello.tcl:

#!/sys/mopscript -f

puts "Hello from TCL"

proc fib {n} {
    if {$n <= 1} {
        return $n
    }
    expr [fib [expr $n-1]] + [fib [expr $n-2]]
}

puts [fib 20]

puts [exec /sys/sdutil -lpd -d /devices/ide0]

puts [exec_bg /sys/spin]

As you can see, exec and exec_bg are custom added commands that do not come with PICOL by default.

Overall rating of PICOL

I really like this project and if you need an embedded scripting language, this is a nice option, mostly because of how easily it integrates into existing code.

RATING: ★★★★☆

SOURCE: https://github.com/antirez/picol

Closing thoughts

What a lot of embeddable languages miss is the ability to integrate seemlessly into an existing codebase. What I look for in tools like this is the ability to simply throw the source files into my project and build them alongside my main code. This is what initially drew me to MuJS and made me stick with PICOL. I think more of interpreters like this should take that into consideration.