In the beginning there was a REPL ...
One of the major features of App Inventor[1] in making it an easy to use programming environment for beginners is live programming. Users interact directly with the state of the running program and changes take effect instantaneously. Rather than the edit-compile-load-run cycle typical of compiler-based languages, App Inventor’s live programming presents a phone-oriented analog of the read-eval-print loop (REPL)[2] found in common implementations of Lisp, Python, and other interpreter-based languages.[3]
The history of live programming in App Inventor is instructive not only for the insight it might offer into the feature itself but also as a story of serendipitous engineering, i.e. how unexpected discoveries lead to significant invention - if you’re prepared.
The story starts in 2008 with the choice of Scheme[4] as the textual programming language that the App Inventor (then known as “Young Android”) blocks language would generate. We considered a few possibilities, including Python, Java and Scheme. We discussed various implementations of the various languages considering criteria like ease of implementation, closeness to Android, perception as a "standard" language, availability of libraries, ease of integration with a visual UI, ease of debugging and ease of incremental development. We also considered ease of use for our target users and availability of documentation because at that point we were still considering allowing users to develop in the target textual language in addition to a visual language.
After considerable debate we decided on Scheme. Each of the languages had pros and cons and to some extent the decision was largely based on our belief in Scheme as a general base on which to implement pretty much any programming language (at the semantic level). At this point we didn’t quite know what the semantics of our desired language would be but based on experience we were confident of our ability to implement whatever language we wanted in Scheme.
Note that at this point we had no thought of anything like live programming. Our plan was to have a more or less conventional edit-build-install-test development cycle for App Inventor users. In fact, one of the reasons that we pick Kawa[5] as out Scheme implementation was because it had a compiler (into Java class files) and it had just recently demonstrated building an Android app[6]. We did consider the existence of a read-eval-print loop (REPL) to be valuable, but we considered that to be an element of our own ‘ease of incremental development’ and ‘ease of debugging’, i.e. as part of our development process for testing our code generator and its related Scheme-written runtime.
We were implementing something that we called YAIL (Young Android Intermediate Language) , as a the textual code that was generated from our visual blocks language. YAIL consisted of S-expressions[7] (i.e. lots of parentheses) representing Scheme expressions which would denote the resulting App Inventor program. It were these Scheme expressions that we would test using Kawa’s REPL on our development computers.
The YAIL expressions were a combination of Scheme macros and procedures. Some of these macros were pretty hairy and it was very useful to be able to test them without going through the entire App Inventor edit-build-install-test process. We could also test the code generated for many of the basic App Inventor blocks for the number, text and list types. Eventually, though, we started to build App Inventor components which required functionality that existed on the phone. For that, the Kawa REPL on our development machines was not very useful.
Around this time, we were looking at the command line for the Kawa REPL[8] (i.e. ‘ java kawa.repl’) and realized that the REPL was just a Java class implemented by Kawa and contained in its runtime jar file[9]. It then occurred to us that maybe there was a way to access the REPL in running Android app that was built using Kawa. A little browsing around the Kawa class documentation led to kawa.TelnetRepl - a class that implemented a REPL that could be accessed using a Telnet[10] client. Sure enough, after making sure that App Inventor started up a TelnetRepl in the Android Activity that it created for its generated app, we could use the Android adb[11] command’s port forwarding capability to enable us to open a Telnet client on our development machines which could talk to a TelnetRepl running in our Android app! Now we could create App Inventor apps containing components and test out those components interactively by typing into our Telnet client connected to a REPL running within our app[12].
It wasn’t long before we realized that in addition to testing out components that we built into apps using App Inventor we could also add components and procedures to apps via the REPL by typing the same code that App Inventor would generate to add them. At this point, we had our ‘aha’ moment. We realized that if we started out with an empty app[13] we could potentially add everything incrementally and if we somehow connected App Inventor itself to the REPL[14] within the app then we could incrementally build the app as the user added (and modified) components, properties, procedures, variables and events![15]
It turned out, of course, that although conceptually simple, the implementation wasn’t trivial. We had to make considerable changes to the Scheme-coded runtime system, mainly to deal with some assumptions that we had initially made about how some App Inventor concepts (particularly procedures, variables and events) were represented in Kawa and in the Java class that Kawa generated for an App Inventor screen. For example, we were generating code that would represent procedures and variables as methods and fields in the generated Java class. However, doing so meant that we couldn’t redefine them (as Java isn’t dynamic in that way). For the liveprogramming mode we needed to loosen the association with Java and rely more on Scheme, which was more dynamic and required an explicit notion of an environment[16] (for tracking variable bindings). Implementing live event definition (and redefinition) was also challenging and required an overhaul of the event handler creation and lookup code, which originally was all in Java but got refactored and split between Java and Scheme. We also had to deal with copying media assets from App Inventor to the Android device.[17]
To sum up, we had a series of small steps that serendipitously led to a major App Inventor feature, i.e. live programming. But note that while each step seemed to lead naturally to the next, it wasn't preordained that we would see or take those next steps. That’s what I meant in the opening paragraph by the necessity of being prepared. We were prepared because we had experience with environments like Lisp (and especially Lisp machine), Smalltalk[18] systems and Emacs editors which provided REPLs. And most notably relevant for live programming, they provided REPLs which ran in and provided access to the context of a running program.
Finally, it’s worth mentioning that major parts of the implementation of live programming have changed over time, (e.g., TelnetRepl and adb) especially with the release of App Inventor 2[19].
[3] App Inventor also provides a compilation capability, which is typically used to package a project to produce an app once development has been completed.
[9] You may argue that this should have been obvious from the start (and you’d be right) but it just wasn't something that we were noticing until this point.
[12] There was some fiddling to figure out how to get the REPL into the same namespace as the app.
[13] Originally called the ‘REPL App’ but eventually called the ‘AI Companion’
[14] This wasn’t all that hard, since we already had incorporated Android’s adb library into App Inventor for other reasons.
[15] Eureka.
[17] Again, adb.
[18] Which is where App Inventor’s ‘Do it’ comes from.
1 comment:
I had a similar experience with Kawa on Android.
The app that I am developing started out as various algorithms developed in chicken scheme and then ported to Android with the NDK.
The wrapper code in Java and JNI was getting ugly, so I tried Kawa instead. I wrote an on-the-fly scheme-to-java compiler and class loader, and then implemented everything else in the scheme REPL (layout, GPS, network, threading) over a Kawa telnet connection.
I posted the compiler/dexer/loader here:
https://sourceware.org/ml/kawa/2014-q4/msg00122.html
I intend to publish the rest of my Kawa Android components to github, but I am trying to figure out how to get paid for this work, since I am currently unemployed.
Post a Comment