Category: Programming from A to Z

I’m interested in exploring ways in which we can physically move through text. This is something that is inherently a part of reading a book or newspaper – there is a tactile sense of how much you have consumed and how much you have left to go. But this is something that we get further away from as we do more and more of our reading on the screen. I started thinking about preexisting ways that we move through or break apart text for different purposes, whether it be text scrolling on teleprompters, lines broken apart on cue cards, or words that are physically assembled into magnetic poetry.

I’ve also been thinking about ways to liberate the screen – to make into something we can touch and hold and make it responsive to it’s own physical environment. There are lots of examples of explorations being done in this direction, like the Interactive Digital Wall by Onomy Labs, but I wanted to embark on some explorations of my own.

In considering how to combine these two interests of physically exploding text and liberating the screen, I considered possible actions that could be used to reveal text with a portable screen including scrolling, attaching, throwing, and banging. For this project I settled on attaching and made a prototype of what I call the “Screen Reader”.

The Screen Reader is a setup that allows you to physically move through text in a linear fashion by attaching a small screen to specified locations on a wall. When the screen is attached to the wall, it shows a word. When you move it to a different location, it shoes a different word. When you reach the end of the line and then return to the beginning, you will reveal a new line of text. The idea is that you can read and reveal a text one word at a time.

The back of the screen is covered with the hook side of conductive velcro. On what would be the wall are patches of loop side conductive velcro. When the screen is attached to any of the available locations, the velcro on the screen creates a connection between the two small pieces of velcro on the wall and in effect closes a switch. Each location where the screen can be attached is a switch. All switch information is monitored by an Arduino which then sends information serially to Processing as to which switch is activated. According to which switch is activated and in what order the switches are activated, the Processing code determines which word to display. For this situation I hooked up the LCD to my computer via the s-video out and showed the applet on the secondary display.

A video of what I showed in class can be seen here:

[QUICKTIME http://itp.nyu.edu/~kh928/a2z/screenreader320.mov 320 257]

I was pretty satisfied with the result. It’s a rough prototype, but I enjoyed turning the act of reading into a more physically engaging activity. I started very simple approach but have ideas that I plan to pursue as to how I could expand this into a larger, more dynamic system, both in terms of content and physical interface.

A script for performance is usually a static entity – something that is written, printed, memorized and performed. But what if it were somewhat more dynamic? What if it were pieced together live from disparate sources? For my midterm, I explored the idea of generating a script through the process of crawling the web. In this example, I crawled a movie script site while using different pattern matching techniques to determine what was sort of lines are spit out. You can see in the example below I was looking for lines with more violent words (slam, wham) and lines with pauses – an attempt to create a sort of ebb and flow in the narrative. Though the lines move along too fast to be read in completion, you can get a sort of general sense of a story.

In this instance I’m returning mostly descriptive, stage direction-type stuff, but the same technique could obviously be used for generating dialogue. I’d like to work on developing more specific rules, so that it’s a bit more elegant and also the work on having better control of the pacing. I’m interested in working from unlikely sources (how would a scene from the New York Times be performed?) and in having two actors work off of two drastically different sources and see how they can play off of each other and if any sense could be made. The main thing I enjoy about this setup is the sense of anticipation and unpredictability – you can never know when the next line will come and you never know ahead of time what the scene will be.

I’ve decided to go with my idea for constructing a dialogue for two people out of language or lines from their favorite “texts”. I’d like to eventually look at all sorts of texts including books, television show scripts, etc. But for the sake of ease and clarity, I’ll be starting with movie scripts. This is inspired by the way people repeat lines back to each other from things they have read or seen. I often wonder – what if these were the only words we could use to communicate? What if our vocabulary was populated only by what we consumed? I guess, in a sense, it is, but it would be interesting to look at it through the lens of popular culture.

Here’s how I see it working:
-Get access to my myspace & Friendster accounts
I’m not sure how to do crawling with anything that’s password-protected. In the end, these don’t necessarily have to be my friends, but since I want to get these people to “perform” the scripts after they are created, I’ll stick with who I know for now.
-Scrape profiles for favorite movies and make list for each friend
-Search for movie scripts
-Accumulate source files
-Analyze each script
It seems as though I should be looking for what sort of language is significant or prominent. I need to figure out how to discern what lines are sort of iconic, representative, or memorable from the movie as a whole.
-Select “pairings”
In other words, decide who will be talking to who. I will probably do this manually, but it would be interesting to think of how to automate this. Should it be random? Should there be some sort of logic to why these two people should be talking to each other?
-Pull out a set number of lines for each person
-Arrange lines in dialogue form to form a new script
I need to figure out some sort of algorithm for this – questions followed by answers, etc.
-Send script to actors with request to generate a “performance”
This could be done via email OR using whatever service I scraped for the information. More likely than not, I’ll have to actually arrange for these people to get together and “perform” the script. The result could be a video created in-person or an audio recording made via telephone.

The way I’m looking at it now, each person will be the embodiment of many characters, or channeling the language of many characters from many movies. The thing I wonder is if I should just focus on the language of one character per movie, or whether it should just be a free-for-all. Also, I wonder if I should just start out simple and do one movie per person. I guess I’ll just experiment and see what’s interesting & engaging and also what I can accomplish in a short amount of time.

I imagine that I will only be able to accomplish chunks of this separately and paste it together myself. It won’t be the most elegant thing, but it’s a good way to get results quickly and see if this would be interesting enough to pursue & polish for the future.

Obviously not for the midterm, but if I could get the full system running, I’d love to create an installation-based scenario where this could be used. There would be two stations next to each other in front of a camera. People enter their favorite movies and then hit “analyze”. While the camera was running, they would then be prompted, line-by-line on their individual screens, with their lines that they would say aloud to each other. The fun part would be that the script would unfold as they were “performing” it – they would never know what the next line was or where the conversation was leading. The video would then be automatically uploaded to a website along with the text of the script. I like the idea because it is seemingly innocuous – listing your favorite movies does not seem particularly personal. But the dialogues might lead to some pretty bizarre places and either way, it causes you to have a full conversation out loud that you might not have had otherwise.

For my first adventure into the world of regular expressions, I made a few interpretations of the “Trouble with Tribbles” script from the original Star Trek television show. The source code can be found here. I looked for tribbles, stage directions, questions, and exclamations. This is one of my favorite results:

UHURA: (coldly)

JONES: (entering)

KIRK: (shows Chekov the packet of wheat)

SPOCK: (stroking a tribble)

BARMAN: (starts producing tribbles from all over the place and putting them on the bar)

SCOTT: (big grin)

MCCOY: (scanning him)

KIRK: (shouting)

KIRK: (a whisper)

More results:
–one word stage directions
–all stage directions
–names with stage directions
–tribbles in stage directions
–one word questions
–questions and exclamations