Scheme-JS: A Scheme Interpreter with Transparent JavaScript Interoperability

An implementation of the Scheme R7RS-Small standard in JavaScript, designed for deep JavaScript interoperability.

Implementation Highlights

• R7RS-Small: Fully conforms to the R7RS-small standard.
• Tail Call Optimization (TCO): Proper tail recursion by the interpreter (though Interleaved JS and Scheme code may still cause stack overflow.)
• First-Class Continuations: Full support for call/cc.
• JavaScript Interop: Seamless calling between Scheme and JavaScript, including shared data representation and transparent boundary crossing. Scheme closures and continuations are first-class JavaScript functions. JavaScript global definitions are automatically visible in the Scheme global environment.
• Browser Scripting: Replace JavaScript in web apps with <script type="text/scheme"> tags. Direct evaluation of Scheme by JavaScript is also supported.
• Node.js REPL: Full-featured interactive REPL with history, multiline support, and parenthesis matching.
• Browser REPL: Interactive browser-based REPL via a custom <scheme-repl> web component.

---

⚠️Trigger Alert: The Scheme implementation described here was vibe coded, so if the very idea of vibe coding disturbs you, you might want to move on to another article or post. This article, however, was written by a human … well, except for parts of the Appendix.

---

Background and History

My background is in programming language design and implementation, and I was deeply immersed in the Scheme world back in the 1990's, especially in my time as a Research Scientist at the MIT A.I. lab working on the MIT Scheme system. Fast forward about 20 years, when I cofounded what is now MIT App Inventor[1] and used Scheme to represent its core intermediate language and implement its runtime environment.

For a while now I've been working, on and off, on a visual programming system that would allow people to build web-based apps. It's mostly meant as a tool for teaching kids (of all ages) programming. It uses a "blocks-based" UI, similar in spirit to App Inventor, Scratch and Snap!. Like App Inventor, the users should be able to build "real", standalone apps. Like Snap!, I want it to be built on a solid semantic and pedagogical base. The Scheme programming language has a long history as such a base[2], so I want the building blocks of my visual language to be based on Scheme.

Consequently, I've been looking for an implementation of the Scheme programming language that had the following:

• A full r7rs-small Scheme, including a numeric tower, proper tail recursion and call/cc.
• Maximal interoperability with JavaScript (in the browser and in node.js).

Unfortunately none of the existing Scheme interpreters or compilers that I could find had quite everything I wanted. The closest were:

• LIPS
• BiwaScheme
• Gambit
• Hoot (Guile compiler to WASM)

At some point I realized that I probably was going to have to create my own implementation of Scheme to meet my needs, but I knew that it would be a difficult and slow endeavor, so I kept putting it off. However, a few things happened relatively recently that gave me some hope. One was that I discovered some research into implementing call/cc in ways that could be implemented by or interoperate with other high-level programming languages. These papers were:

• Continuations from Generalized Stack Inspection[3]
• An Unexceptional Implementation of First-Class Continuations[4]
• A Method for Implementing First-Class Continuations on the JVM and CLR (AI assisted)[5]

I also had some personal communication with Joe Marshall[6] (the author or co-author of the above papers), which gave me some more confidence.

Around the same time I was aware of the increasing capabilities of A.I. assisted programming. I'm a good programmer, I think, but I'm a slow programmer, and the hope that A.I. could accelerate the process energized me. I was also hoping that A.I. could help in understanding the research mentioned above and applying it to JavaScript.

The above confluence of events led me to try and vibe code[7] my way to a new Scheme implementation. It's now complete (of course it may have bugs) with respect to the features mentioned at the top of this article. I should mention, though, that performance was a non-goal for this implementation. In my intended use cases I am hoping that performance won't be an issue.

You can try out the browser-based REPL here.  The GitHub repo is here.

Feature Details

Let's go into a little more detail on the features that I mentioned in the highlights at the top of this article.

R7RS-Small

Scheme-JS is (modulo any bugs) a conformant implementation of Scheme as described in the Scheme R7RS-small standard. It passes what I call the "Chibi compliance tests", which is a fairly comprehensive test suite originally developed for Chibi Scheme. Additionally, we created a test suite which consists of all the examples in the R7RS-standard document.

As a standard-conforming Scheme, Scheme-JS is properly tail recursive (though Interleaved JS and Scheme code may still cause stack overflow.) It also has full support for first-class continuations (i.e. call/cc), in a language which itself supports neither. That was where the research mentioned above came in.

JavaScript Interoperability

There are three elements of JavaScript interoperability in this implementation of Scheme: JS transparency, JS-like syntax and exposing JS operations.

The first element, JS transparency, has to do with being able to call into JavaScript from Scheme, and vice versa, with the least effort on the part of the user who is coding with Scheme-JS. It also means that Scheme data types are "transparently" converted to (or represented as) JavaScript data types, and vice versa, as we cross the JS<->Scheme boundary. Consequently, Scheme vectors are JavaScript arrays, Scheme records are JavaScript objects, Scheme procedures and continuations are directly callable by JavaScript functions.

In order to support a complete Scheme numeric tower and proper exactness, Scheme-JS implements exact numbers using JavaScript BigInts and provides, by default, automatic deep conversion when passed to native JavaScript functions. This obviously has some performance implications, but benchmarking seems to show that the slowdown (over representing all numbers as JS Numbers and having no exact numbers in the numeric tower) is on the order of 10%

The second element is about providing convenient JS-like syntax in Scheme. In Scheme-JS this involved adding a dot notion syntax for accessing and setting JS object properties and adding a syntax for creating JS object literals.

The third element has to do with giving explicit Scheme access to primitive JavaScript operations. Scheme-JS supports that with a set of Scheme procedures. Related to this, JavaScript global definitions (on window or globalThis) are automatically visible in the Scheme global environment.

Some more details about the JavaScript interoperability can be seen in the Appendix.

Node.js REPL

Scheme-JS has a node-based interactive REPL with history, multiline support, and parenthesis matching.

Browser REPL

Scheme-JS has a browser-based REPL which can be embedded in an HTML page via a custom <scheme-repl> web component.

Browser Scripting

You can use Scheme-JS to replace JavaScript in web apps by using <script type="text/scheme"> tags.  You can also directly access the Scheme-JS evaluator functions from JavaScript.

Future Directions

Debugger

One major area of future work will be in debugging support in the REPLs and for the web app scripting use case. Experiments in using Scheme-JS to replace JavaScript in a real web app have shown how important such support is. I am in the process of building Scheme-JS debugging capabilities into the REPLs and integrating debugging capabilities into Chrome dev tools (for the scripting use case).

Performance

There also might be some low hanging fruit that could improve performance.  Longer term, greater performance improvements might come from more fundamental changes.

Vibe Coding

It's worthwhile to note that Scheme-JS has been completely vibe coded, with almost no traditional coding.  That said, I am a very experienced programmer, with considerable knowledge of Scheme and some of its implementation strategies, and I gave the AI models and agents considerable guidance in many situations. For those who are interested, I plan to have a followup article talking in detail about the vibe coding aspects of this project.

I understand that some people use the term "vibe coding" to apply to a one-shot (or few-shot) prompt that describes the app a person wants to build and then the AI goes and generates the code it with little or no further interaction with the person. That is not how I am using the term. The AI and I had a very long series of interactions while building this implementation. 

For those who are interested, I plan to have a followup article talking in detail about the vibe coding aspects of this project.

The creation of live programming in App Inventor.

In the beginning there was a REPL ...

One of the major features of App Inventor^[1] in making it an easy to use programming environment for beginners is live programming.   Users interact directly with the state of the running program and changes take effect instantaneously.  Rather than the edit-compile-load-run cycle typical of compiler-based languages, App Inventor’s live programming presents a phone-oriented analog of the read-eval-print loop (REPL)^[2] found in common implementations of Lisp, Python, and other interpreter-based languages.^[3]

The history of live programming in App Inventor is instructive  not only for the insight it might offer into the feature itself but also as a story of serendipitous engineering, i.e. how unexpected discoveries lead to significant invention - if you’re prepared.

The story starts in 2008 with the choice of Scheme^[4] as the textual programming language that the App Inventor (then known as “Young Android”) blocks language would generate.  We considered a few possibilities, including Python, Java and Scheme.  We discussed various implementations of the various languages considering criteria like ease of implementation, closeness to Android, perception as a "standard" language, availability of libraries, ease of integration with a visual UI, ease of debugging and ease of incremental development.  We also considered ease of use for our target users and availability of documentation because at that point we were still considering allowing users to develop in the target textual language in addition to a visual language.

After considerable debate we decided on Scheme.  Each of the languages had pros and cons and to some extent the decision was largely based on our belief in Scheme as a general base on which to implement pretty much any programming language (at the semantic level). At this point we didn’t quite know what the semantics of our desired language would be but based on experience we were confident of our ability to implement whatever language we wanted in Scheme.

Note that at this point we had no thought of anything like live programming.  Our plan was to have a more or less conventional edit-build-install-test development cycle for App Inventor users.  In fact, one of the reasons that we pick Kawa^[5] as out Scheme implementation was because it had a compiler (into Java class files) and it had just recently demonstrated building an Android app^[6].  We did consider the existence of a read-eval-print loop (REPL) to be valuable, but we considered that to be an element of our own ‘ease of incremental development’ and ‘ease of debugging’, i.e. as part of our development process for testing our code generator and its related Scheme-written runtime.

We were implementing something that we called YAIL (Young Android Intermediate Language) , as a the textual code that was generated from our visual blocks language.  YAIL consisted of S-expressions^[7] (i.e. lots of parentheses) representing Scheme expressions which would denote the resulting App Inventor program.  It were these Scheme expressions that we would test using Kawa’s REPL on our development computers.

The YAIL expressions were a combination of Scheme macros and procedures.  Some of these macros were pretty hairy and it was very useful to be able to test them without going through the entire App Inventor edit-build-install-test process.  We could also test the code generated for many of the basic App Inventor blocks for the number, text and list types.  Eventually, though, we started to build App Inventor components which required functionality that existed on the phone.  For that, the Kawa REPL on our development machines was not very useful.

Around this time, we were looking at the command line for the Kawa REPL^[8] (i.e. ‘ java kawa.repl’) and realized that the REPL was just a Java class implemented by Kawa and contained in its runtime jar file^[9].  It then occurred to us that maybe there was a way to access the REPL in running Android app that was built using Kawa.  A little browsing around the Kawa class documentation led to kawa.TelnetRepl - a class that implemented a REPL that could be accessed using a Telnet^[10] client.   Sure enough, after making sure that App Inventor started up a TelnetRepl in the Android Activity that it created for its generated app, we could use the Android adb^[11] command’s port forwarding capability to enable us to open a Telnet client on our development machines which could talk to a TelnetRepl running in our Android app!  Now we could create App Inventor apps containing components and test out those components interactively by typing into our Telnet client connected to a REPL running within our app^[12].

It wasn’t long before we realized that in addition to testing out components that we built into apps using App Inventor we could also add components and procedures to apps via the REPL by typing the same code that App Inventor would generate to add them.  At this point, we had our ‘aha’ moment.  We realized that if we started out with an empty app^[13] we could potentially add everything incrementally and if we somehow connected App Inventor itself to the REPL^[14] within the app then we could incrementally build the app as the user added (and modified) components, properties, procedures, variables and events!^[15]

It turned out, of course, that although conceptually simple, the implementation wasn’t trivial.  We had to make considerable changes to the Scheme-coded runtime system, mainly to deal with some assumptions that we had initially made about how some App Inventor concepts (particularly procedures, variables and events) were represented in Kawa and in the Java class that Kawa generated for an App Inventor screen.  For example, we were generating code that would represent procedures and variables as methods and fields in the generated Java class.  However, doing so meant that we couldn’t redefine them (as Java isn’t dynamic in that way).  For the liveprogramming mode we needed to loosen the association with Java and rely more on Scheme, which was more dynamic and required an explicit notion of an environment^[16] (for tracking variable bindings).  Implementing live event definition (and redefinition) was also challenging and required an overhaul of the event handler creation and lookup code, which originally was all in Java but got refactored and split between Java and Scheme.  We also had to deal with copying media assets from App Inventor to the Android device.^[17]

To sum up, we had a series of small steps that serendipitously led to a major App Inventor feature, i.e. live programming.  But note that while each step seemed to lead naturally to the next, it wasn't preordained that we would see or take those next steps.  That’s what I meant in the opening paragraph by the necessity of being prepared.  We were prepared because we had experience with environments like Lisp (and especially Lisp machine), Smalltalk^[18] systems and Emacs editors which provided REPLs.   And most notably relevant for live programming, they provided REPLs which ran in and provided access to the context of a running program.

Finally, it’s worth mentioning that major parts of the implementation of live programming have changed over time, (e.g., TelnetRepl and adb) especially with the release of App Inventor 2^[19].  

---

[1] appinventor.mit.edu

[2] http://en.wikipedia.org/wiki/Read-eval-print_loop

[3] App Inventor also provides a compilation capability, which is typically used to package a project to produce an app once development has been completed.

[4] http://en.wikipedia.org/wiki/Scheme_(programming_language)

[5] http://www.gnu.org/software/kawa/

[6] http://per.bothner.com/blog/2008/AndroidHelloScheme/

[7] http://en.wikipedia.org/wiki/S-expression

[8] http://www.gnu.org/software/kawa/Running.html

[9] You may argue that this should have been obvious from the start (and you’d be right) but it just wasn't something that we were noticing until this point.

[10] http://en.wikipedia.org/wiki/Telnet

[11] http://developer.android.com/tools/help/adb.html

[12] There was some fiddling to figure out how to get the REPL into the same namespace as the app.

[13] Originally called the ‘REPL App’ but eventually called the ‘AI Companion’

[14] This wasn’t all that hard, since we already had incorporated Android’s adb library into App Inventor for other reasons.

[15] Eureka.

[16] http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-21.html#%_sec_3.2

[17] Again, adb.

[18] Which is where App Inventor’s ‘Do it’ comes from.

[19] appinventor.mit.edu