For this module, I really wanted to explore text generation: I thought it would be cool to create a stream-of-consciousness output for my ESP32 based on a corpus of my own writing. I knew from the get-go that I wasn’t going to be able to implement any sort of actual machine learning; I’m totally clueless when it comes to that kind of thing, and the ESP32 wouldn’t be able to handle the size of the corpus required to produce anything discernible.

With that in mind, I decided to piece together fragments of language—parts of speech, phrases, idioms—in predetermined patterns. The goal wasn’t necessarily to produce “believable” (human) output. In fact, I think it’s really interesting to lean into those weird idiosyncracies of generated text that don’t quite sit right. Even though my generated sentences are constrained to a certain number of “skeleton” structures, the output is entertaining. It’s silly at times, and poetic at others. I think of it sort of like a mosaic—a Frankenstein-esque amalgamation—where the tesserae are words and the plaster is computation. It’s naturally a little rough around the edges.

Technical Overview

See the code here.

I’ve written out the steps I took to configure the Arduino IDE here. The most painstaking part of this module was definitely wrestling with the IDE, especially the Mac port snafu mentioned in the Arduino post. Once I had this figured out, I tried printing text to the screen. I figured out pretty quickly that iterative development is your friend when working with this setup. I’d make a small change—like decreasing the font size—and the entire screen would go black.

Putting the Pieces Together

After I had a vision of what I wanted to do, I started by creating some small arrays of patterned fragments: some sentence starters like “i don’t mind”, some nouns. I set up my code structure so that it would randomly choose a sentence starter (using the size of the array) and randomly choose a noun, then string them together, stick a period at the end, and print it out. The text was stationary on the screen at this point, so I printed to the serial port in case anything was cut off. I then built up the word arrays and added some other simple sentence structures to diversify the output a bit. This was the basic functionality I wanted, and I was happy enough with the generative part. However, it was a problem that the text wasn’t always fully visible, so the next step was implementing a scrolling mechanism.

First, I tried using text sprites, but this ended up being more trouble than it was worth. I couldn’t get the scrolling right; the timing was always off, and if I increased the font size beyond tiny, the display would stop working. I had read online that sometimes it’s better to implement text scrolling by just changing the letter offset within a certain time interval and re-drawing the string. It was a lot cruder than the sprite method, but it worked (and now I can say it fits in the general thematic aesthetic)!

I had planned to regenerate a sentence with every iteration of the loop, but I liked the idea of a continuous stream more than disjoint sentences. This is weird to implement, because we don’t have infinite storage, so there has to be a point at which you stop to trash your buffer and generate more stuff. After experimenting with a few options, I went with the one that seemed simplest to code: fill a large (4096-byte) buffer with content at each iteration of the loop function, then scroll over the entire thing. At the end of the loop, trash the buffer and start again.

For future iterations of this project, I would refine the behavior at the end of the loop. Right now, to make sure that the program doesn’t access any memory it shouldn’t, I stop the text generation whenever I detect that the next fragment would be too long to append to the buffer. Because of this, the sentences don’t often end cleanly at the breaks between iterations of the loop. Luckily, the buffer is long enough that this really doesn’t happen often.

Aesthetics

I kept this project simple, visually. I wanted the focus of this project to be the words, which speak their own aesthetic. I had originally used a horizontal orientation, and changed to vertical last-minute to suit the installation. If I were to do this installation again, I would experiment with different fonts.

Visual documentation