A better IDE for uLisp?

[johnsondavies]

uLisp would benefit from a better way of writing and uploading programs. This post discusses some of the possible alternatives, to get feedback and other suggestions.

Currently I think most people use uLisp in one of the following ways:

Write programs directly from a terminal

Pros: No extra software needed.
Cons: No line editing; if you make a mistake you have to enter the function again.

Write programs using the Arduino IDE’s Serial Monitor

The Arduino Serial Monitor is one step up from a terminal. You can type into the input field, and delete to correct mistakes before you click Send. You can also copy and paste functions or even complete uLisp sources into the input field.

Pros: Built into the Arduino IDE.
Cons: No way to store programs, or help with formatting programs.

uLisp Tree Editor

Not sure if anyone uses this. Allows you to edit in-memory functions, but difficult to use; you have to have a mental concept of the function as a tree. See: Using the program editor

SD Card

You can write programs in a text editor and put them on an SD Card, and then read them in. See: Loading uLisp programs from an SD card

Suggested future developments

Line editor built into uLisp

uLisp could be extended with a buffer and line editor to improve its usability from a terminal.

The Lisp Badge has a similar solution. It has a buffer large enough to hold a screenful of text, allowing delete to correct mistakes, and it provides a bracket-highlighting parenthesis matcher to make entering Lisp programs easier.

Pros: Makes using uLisp from a terminal less frustrating.
Cons: Needs RAM for the buffer; a problem on low-memory systems such as the ATmega328P.

uLisp Arduino Core

It might be possible to write a Core (ie plugin) for the Arduino IDE that allows you to:

• Use the Arduino IDE editor to write and save/load uLisp programs.

• When you click Upload the entire source would be sent to the attached microcontroller board via the USB port.

• You could then open the Serial Monitor window and use the uLisp REPL.

Other possible features would be: automatic formatting of Lisp programs.

Does anyone have enough understanding of the Arduino IDE to say whether this would be feasible?

uLisp version of mu editor

The Mu editor is a multi-platform editor designed to act as a simple interface between a computer and Python running on a microcontroller connected via the USB port. It provides an editor pane and a REPL pane. See: About Mu.

It’s open source and configurable via plug-ins; see Mu on GitHub. It might be possible to make a plugin for Mu to enable it to be used as an editor/REPL for uLisp.

Slime

Slime/Swank is an editing environment that that lets you use Emacs with a remote copy of Lisp.

Dave Astels has made a Swank server that lets you use slime-mode in GNU Emacs to interact with uLisp running on a board: see Slime support for uLisp.

There’s also Atom-Slime that would let you interface to uLisp via the Atom editor: Atom-Slime

Pros: An editing environment that understands Lisp and Lisp formatting.
Cons: Requires a copy of Common Lisp running on the host computer. Needs quite a bit of configuration.

I welcome feedback from anyone who has set this up.

[odin]

I think minimal line editing capabilities are a must, at least on the platforms that have any ability to do it. There is precedent for memory-intensive capabilities (arbitrary symbols!) being disabled on systems that can’t handle it.

As for SLIME, I’m not entirely convinced it makes sense to do that. Swank is really not intended as a general protocol, and it’s enough of a moving target that the Clojure folks gave up on trying to follow it, instead building their own protocol.

[Aeneas]

I, too, vote for “some form of line editor”. uLisp should be, INHO, operable from platforms which habe no screen and no input buffer by themselves, i.e. it should be, at least in my eyes, “standalone” operable.

An IBM 7090, on which many famous early LISP programs were written, has had LESS cells and SLOWER speed than some of the microcontrollers running uLisp. uLisp has, IMHO, NO need to “play modest”! 😉

[johnsondavies]

I’ll prototype a version of uLisp with a line editor and make it available for feedback. Which platform would be most useful for the prototype?

[Aeneas]

I vote ARM (particularly runnable on Arduino Due): sort of EVERYBODY should be acquainted with it by now… ☺️ And it is neither too powerful, nor too weak.

[odin]

Which platform would be most useful for the prototype?

I suspect ARM is more broadly available than AVRs that have the memory capacity for it.

[johnsondavies]

It would be nice to include parenthesis matching, at least as an option. Do terminals provide codes for cursor left, cursor right, and highlight character?

[odin]

After a fashion. There is a standard for terminal control codes, Ecma-48, which are commonly known as ANSI codes. Those include cursor position reporting, cursor movement commands (in 2D) and various appearance changes. They’re supported by just about every Unix-like terminal, but can get really hairy if you go beyond the very basics.

[johnsondavies]

Thanks! I’ll try it.

[Aeneas]

Ad parenthesis matching: I learned recently a trick from a C64-Lisp:

A [ can stand in the place of any (, but a ] closes ALL parens up to the previous preceding ]. This is particularly useful for “larger subclauses” in cond-statements or thelike:

(defun f (x) (cond [(>= x 0) (cond ((> x 0) 1) (t 0] (t -1)))

would be a (somewhat ridiculous) example.

While it is not exactly “paren matching”, it is a form of “checkpoint matching”, and in practice feels rather nice. I read, too, that some other Lisps in the 70s or 80s provided similar “mass closing” facilities.

Another way would be to provide a “counter” as soon as the user types “(” or “)”, i.e. printing ({1}defun g ({2}x){1} ({2}+ 1 x){1}){0} when the user types (defun g (x) (+ 1 x)), but I simply fear this will be too visually cluttered.

[Aeneas]

As to backspace: I recently tried something, and it works WELL: many 70s OSes typed the last erased letter. Like if you write “lips” instead of “lisp” and correct the last two letters using two backspaces, it would look like “lipsspsp”. This looks stupid when you “read” it, but while you actually USE it, it feels just OK. Early Unix did that, too.

[odin]

This is where you can start running into weirdness with terminals. Backspace doesn’t always mean the same thing. Sometimes it sends, well, backspace (BS, #x08) and sometimes it sends delete (DEL, #x7F). There is also cancel character (CCH, #x94 or the two-byte sequence #x1B #x54 - ESC T).

Incidentally, the convention of using C’s '\n' character to represent a newline doesn’t work when you’re writing to a terminal. That’s just the line feed (LF, #x0A). You also need to send the carriage return (CR, #x0D) to move the cursor to the start of the line. Pressing return, incidentally, (the big key now often marked ‘enter’) inputs a carriage return. Which you can also input to the terminal using Ctrl+m.

Oh, and you can send an ASCII NUL down the terminal using Ctrl+Space. I haven’t actively tried to mess up uLisp that way, but C programs that don’t expect to see NULs have a tendency to freak out when they do. :)

[johnsondavies]

The Lisp Badge highlights matching parentheses as you type, which is nice:

[ahmetus]

Emacs with inferior-lisp capable enough. Just add this code to your init file, edit port and board info to suit your needs, then you are good to go. When you connect to board, call elisp function (sb-open) from scratch buffer. That’s it.

(defvar port "/dev/ttyUSB0" "esp32doit-devkit-v1")

(defvar bauds 9600 "Bps")

(defun sb-open()
  (let ((serial-buffer (serial-term port bauds)))
  (with-current-buffer
      (rename-buffer "*inferior-lisp*")
    (term-line-mode)
    (setq inferior-lisp-buffer serial-buffer))))

[johnsondavies]

Emacs with inferior-lisp capable enough. Just add this code to your init file, edit port and board info to suit your needs, then you are good to go.

I’d like to set this up to try it out, but I’m a bit confused about where PlatformIO fits in, and where Emacs fits in.

What’s the application running in your video? What’s emacs@prolog-desktop?

Would it be possible to give a simple step-by-step getting started sequence?

Thanks! David

[odin]

PlatformIO is orthogonal to the emacs setup. You can use one without the other. The program seen on the left of the demo is emacs; the window title includes the hostname.

The main question for a step-by-step guide is going to be how familiar people are with emacs.

[ahmetus]

Before I did install ulisp to my Mac with Arduino, and I did test k210 support, it worked like a charm. All graphic examples were great.

Then I installed PlatformIO because Arduino esp32 and maixduino libraries do not support arm64, and I’m using Jetson Nano Dev Kit as my daily Linux machine - which is an Arm machine like Rapsberry Pi, but 64 bit one. Strangely I like this machine more than my Mac Mini, and I usually first try to work with it. There for I needed to install. As odin said; it’s independent from Emacs. It has cli interface. Also PlatformIO has a standart file structure that I followed. I used cli to install esp32 project, then converted ulisp esp32 ino file to two independent files, one header(ulisp.h - in include directory), one cpp - main.cc - in src directory. I used cproto (apt install or brew install) for retrieving function prototypes from ulisp.ino file, then followed compiler error messages, and fixed one by one. Like I moved some functions and definitions to the top of the file.

It works mostly better than Arduino, uploads more quick, and project becomes C++ project. You can use any editor - ide. Official IDE VSCode that I hate. It also has fantastic documentation,

Installation instructions is here : https://docs.platformio.org/en/latest/core/installation.html

and Quick Start Guide : https://docs.platformio.org/en/latest/core/quickstart.html

I explained above and in my videos Emacs part, I think…

I still did not be able to install k210 Arduino libraries or platformio project to my Jetson Nano. I’ll look closely that at weekend. This is the plan.

Sorry for my English, never been good, hope I can be able to explain a bit.

[odin]

Having uLisp as a simple inferior-lisp is a decided improvement on the previous experience, but I wonder if it’s possible to go further by making changes on the uLisp side and a bit more code in emacs. You see, ASCII contains control characters that are no longer really used, but which can be extremely useful if correctly used.

What if we use them to define a protocol that can send clearly delimited code for execution, possibly including some header information? emacs can munge the line before it’s sent, adding the header and encapsulation. It can also do something similar in the reverse direction.

The disadvantage is that this requires more emacs lisp code, but the advantages are that it would be relatively easy to extend to the point where we have completion of symbols from the running image and other such niceties. I’d at least be interested in exploring the possibilities.

[ahmetus]

Good ideas, but there is a swank port for that in here which I did not be able to setup and use, so, basically what you are asking is slime for ulisp. As I said my previous reply, I’m not a Common-Lisp Library writer, I never needed Slime. Emacs and simple inferior-lisp mode, yasnippets which is rich for elisp and common lisp and company-mode usually enough to me. I also use Dash for documentation on Mac - it has Common Lisp Docs, and Zeal for Linux - same. Last additions are Lisp extra font-lock - which its job is coloring base Common-Lisp - Elisp macros, and Rainbow Delimiters modes which can be found on Melpa. Dash and Zeal modes also on Melpa too.

With these modes and necessary configuration of init file, Emacs becomes very good Lisp IDE. If you use Semantic, you can navigate to local and other reference symbols - functions.

C and C++ on Emacs is completely different story, but it is also doable and once you accomplish setup these things, joyful.

[ahmetus]

What can be easily done is symbol - function reference display with eldoc. If it is in source code, I can parse with elisp. Also we can write a parser for flycheck - flymake. Or dedicated ulisp mode. It can be done. But differences will worth the trouble, I’m not sure.