Dasher: information-efficient text entry

link DonHopkins on Nov 24, 2023 | parent | context | favorite | on: AI is currently just glorified compression I love David MacKay's brilliant work on the Dasher text input system, which draws deeply from his work on information theory -- imagine Dasher integrated with an IDE and code search and Copilot and language model! "Writing is navigating in the library of all possible books." -David MacKay We just allocate more shelf space to the more probable letters. Why isn't Dasher built into every operating system and mobile phone? link) link link DonHopkins on May 18, 2018 | parent | context | favorite | on: Pie Menus: A 30-Year Retrospective: Take a Look an... Dasher is fantastic, because it's based on rock solid information theory, designed by the late David MacKay. Here is the seminal Google Tech Talk about it: ▶ this video Here is a demo of using Dasher by an engineer at Google, Ada Majorek, who has ALS and uses Dasher and a Headmouse to program: link Another one of her demonstrating Dasher: Ada Majorek Introduction - CSUN Dasher link Here’s a more recent presentation about it, that tells all about the latest open source release of Dasher 5: Dasher - CSUN 2016 - Ada Majorek and Raquel Romano link Here's the github repo: Dasher Version 4.11 link >Dasher is a zooming predictive text entry system, designed for situations where keyboard input is impractical (for instance, accessibility or PDAs). It is usable with highly limited amounts of physical input while still allowing high rates of text entry. Ada referred me to this mind bending prototype: D@sher Prototype - An adaptive, hierarchical radial menu. link >( link ) - a really neat way to "dive" through a menu hierarchy/, or through recursively nested options (to build words, letter by letter, swiftly). D@sher takes Dasher, and gives it a twist, making slightly better use of screen revenue. >It also "learns" your typical useage, making more frequently selected options larger than sibling options. This makes it faster to use, each time you use it. >More information here: link and here: link Dasher is even a viable way to input text in VR, just by pointing your head, without a special input device! Text Input with Oculus Rift: link >As part of VR development environment I'm currently writing ( link ), I've implemented dasher ( link ) to input text. One important property of Dasher is that you can pre-train it on a corpus of typical text, and dynamically train it while you use it. It learns the patterns of letters and words you use often, and those become bigger and bigger targets that string together so you can select them even more quickly! Ada Majorek has it configured to toggle between English and her native language so she can switch between writing email to her family abroad and co-workers at google. Now think of what you could do with a version of dasher integrated with a programmer's IDE, that knew the syntax of the programming language you're using, as well as the names of all the variables and functions in scope, plus how often they're used! I have a long term pie in the sky “grand plan” about developing a JavaScript based programmable accessibility system I call “aQuery”, like “jQuery” for accessibility. It would be a great way to deeply integrate Dasher with different input devices and applications across platforms, and make them accessible to people with limited motion, as

I love David MacKay's brilliant work on the Dasher text input system, which draws deeply from his work on information theory -- imagine Dasher integrated with an IDE and code search and Copilot and language model! "Writing is navigating in the library of all possible books." -David MacKay We just allocate more shelf space to the more probable letters. Why isn't Dasher built into every operating system and mobile phone? link) link link DonHopkins on May 18, 2018 | parent | context | favorite | on: Pie Menus: A 30-Year Retrospective: Take a Look an... Dasher is fantastic, because it's based on rock solid information theory, designed by the late David MacKay. Here is the seminal Google Tech Talk about it: ▶ this video Here is a demo of using Dasher by an engineer at Google, Ada Majorek, who has ALS and uses Dasher and a Headmouse to program: link Another one of her demonstrating Dasher: Ada Majorek Introduction - CSUN Dasher link Here’s a more recent presentation about it, that tells all about the latest open source release of Dasher 5: Dasher - CSUN 2016 - Ada Majorek and Raquel Romano link Here's the github repo: Dasher Version 4.11 link >Dasher is a zooming predictive text entry system, designed for situations where keyboard input is impractical (for instance, accessibility or PDAs). It is usable with highly limited amounts of physical input while still allowing high rates of text entry. Ada referred me to this mind bending prototype: D@sher Prototype - An adaptive, hierarchical radial menu. link >( link ) - a really neat way to "dive" through a menu hierarchy/, or through recursively nested options (to build words, letter by letter, swiftly). D@sher takes Dasher, and gives it a twist, making slightly better use of screen revenue. >It also "learns" your typical useage, making more frequently selected options larger than sibling options. This makes it faster to use, each time you use it. >More information here: link and here: link Dasher is even a viable way to input text in VR, just by pointing your head, without a special input device! Text Input with Oculus Rift: link >As part of VR development environment I'm currently writing ( link ), I've implemented dasher ( link ) to input text. One important property of Dasher is that you can pre-train it on a corpus of typical text, and dynamically train it while you use it. It learns the patterns of letters and words you use often, and those become bigger and bigger targets that string together so you can select them even more quickly! Ada Majorek has it configured to toggle between English and her native language so she can switch between writing email to her family abroad and co-workers at google. Now think of what you could do with a version of dasher integrated with a programmer's IDE, that knew the syntax of the programming language you're using, as well as the names of all the variables and functions in scope, plus how often they're used! I have a long term pie in the sky “grand plan” about developing a JavaScript based programmable accessibility system I call “aQuery”, like “jQuery” for accessibility. It would be a great way to deeply integrate Dasher with different input devices and applications across platforms, and make them accessible to people with limited motion, as well as users of VR and AR and mobile devices. link Here’s some

Thank you, that's an interesting response to his quotes, too! Like the proverbial plate of shrimp, this stuff has been coming up a lot recently, so I've been researching and recombining ideas from Will's old talks about Spore and with Brian Eno, David MacKay's Dasher text input system, and I've also dug up some even older unpublished thoughts from Ed Fredkin and John Cocke about a Theory of Dreams, and how they relate to LLMs and compression. Being able to fit a program that generates lots of content on a small floppy disk or cdrom is one goal, less important today, but the important part is that thinking of procedural content generation as decompression of noise (random, user generated, contextual, or environmental) is a useful technique even with today's virtually unlimited storage and high speed delivery. Will speaks about compression in information theoretic and arithmetic coding terms, not just referring to standard compression algorithms like "jpeg" or "mp3", but to information encoding and decoding theory, using compression techniques for procedural generation, like LLMs. Here's a great video of Will Wright and Brian Eno discussing generative systems and demonstrating cellular automata with Mirek's Cellebration to Brian Eno's generative music, at a talk at the Long Now Foundation,: Will Wright and Brian Eno - Generative Systems (excepts from talk): link >Game designer Will Wright and musician Brian Eno discuss the generative systems used in their respective creative works. This clip features original music by Brian Eno. Will Wright and Brian Eno on "Playing with Time." In a dazzling duet Will Wright and Brian Eno give an intense clinic on the joys and techniques of "generative" creation. Playing with Time | Brian Eno and Will Wright (entire talk): link >Will Wright, creator of the video games "Sim City," "The Sims," and the forthcoming "Spore," will speak on playing with time. "Playing with Time" was given on June 26, 02006 as part of Long Now's Seminar series. Generative Music – Brian Eno (1996) (inmotionmagazine.com): link link Sandspiel Studio, Brian Eno, Wave Function Collapse, visual programming, cellular automata, etc: link Here's a simple low-tech pre-LLM example that shows the equivalence of compression and procedural content generation: Take a huge text file of HN postings, and compress it with gzip or compress or some other robust compression algorithm. The better the algorithm, the more the output will look like random noise. Then slice the compressed file in half, and replace the second half with random numbers. Then uncompress it. You'll find that at the point you sliced it, it keeps on writing out almost plausible text for a while, consisting of highly probably snippets of commonly encountered words and phrases, then goes downhill towards incoherence. It's not as coherent or confident as an LLM, but the point is to show how low the bar is for using compression for procedural content generation. LLMs are essentially a form of compression of the world's knowledge or whatever they're trained on, not just word frequencies or pixel patterns, but also concepts and ideas. Ed Fredkin described John Cocke's Theory of Dreams, which describes dreams as a kind of procedural content generation based on running your brain's decoder over random noise inputs: link DonHopkins 1 day ago | parent | context | favorite | on: No one cares about your dreams unless you’re a fam... ON THE SOUL: Ed Fredkin, Unpublished Manuscript link >The John Cocke Theory of Dreams was told to me, on the phone, late one night back in the early 1960’s. John’s complete description was contained in a very short conversation approximately as

I'd love a neural interface to Dasher, which adaptively learns as you use it which letter combinations are the most popular, and adjusts over time to make it easier and faster to input common text. It would be wonderful integrated with a context and language sensitive IDE. link) link link >To make the interface efficient, we use the predictions of a language model to determine how much of the world is devoted to each piece of text. Probable pieces of text are given more space, so they are quick and easy to select. Improbable pieces of text (for example, text with spelling mistakes) are given less space, so they are harder to write. The language model learns all the time: if you use a novel word once, it is easier to write next time. [...] >Imagine a library containing all possible books, ordered alphabetically on a single shelf. Books in which the first letter is "a" are at the left hand side. Books in which the first letter is "z" are at the right. In picture (i) below, the shelf is shown vertically with "left" (a) at the top and "right" (z) at the bottom. The first book in the "a" section reads "aaaaaaaaaaaa..."; somewhere to its right are books that start "all good things must come to an end..."; a tiny bit further to the right are books that start "all good things must come to an enema...". [...] >.... This is exactly how Dasher works, except for one crucial point: we alter the SIZE of the shelf space devoted to each book in proportion to the probability of the corresponding text. For example, not very many books start with an "x", so we devote less space to "x..." books, and more to the more plausible books, thus making it easier to find books that contain probable text. The classic Google Tech Talk by the late David MacKay, the inventor of Dasher: ▶ this video It's based on the concept of concept of arithmetic coding from information theory. link Ada Majorek, who has ALS, uses it with a Headmouse to program (and worked on developing a new open source version of Dasher) and communicate in multiple languages: link

Dasher is fantastic, because it's based on rock solid information theory, designed by the late David MacKay. Here is the seminal Google Tech Talk about it: ▶ this video Here is a demo of using Dasher by an engineer at Google, Ada Majorek, who has ALS and uses Dasher and a Headmouse to program: link Another one of her demonstrating Dasher: Ada Majorek Introduction - CSUN Dasher link Here’s a more recent presentation about it, that tells all about the latest open source release of Dasher 5: Dasher - CSUN 2016 - Ada Majorek and Raquel Romano link Here's the github repo: Dasher Version 4.11 link >Dasher is a zooming predictive text entry system, designed for situations where keyboard input is impractical (for instance, accessibility or PDAs). It is usable with highly limited amounts of physical input while still allowing high rates of text entry. Ada referred me to this mind bending prototype: D@sher Prototype - An adaptive, hierarchical radial menu. link >( link ) - a really neat way to "dive" through a menu hierarchy/, or through recursively nested options (to build words, letter by letter, swiftly). D@sher takes Dasher, and gives it a twist, making slightly better use of screen revenue. >It also "learns" your typical useage, making more frequently selected options larger than sibling options. This makes it faster to use, each time you use it. >More information here: link and here: link Dasher is even a viable way to input text in VR, just by pointing your head, without a special input device! Text Input with Oculus Rift: link >As part of VR development environment I'm currently writing ( link ), I've implemented dasher ( link ) to input text. One important property of Dasher is that you can pre-train it on a corpus of typical text, and dynamically train it while you use it. It learns the patterns of letters and words you use often, and those become bigger and bigger targets that string together so you can select them even more quickly! Ada Majorek has it configured to toggle between English and her native language so she can switch between writing email to her family abroad and co-workers at google. Now think of what you could do with a version of dasher integrated with a programmer's IDE, that knew the syntax of the programming language you're using, as well as the names of all the variables and functions in scope, plus how often they're used! I have a long term pie in the sky “grand plan” about developing a JavaScript based programmable accessibility system I call “aQuery”, like “jQuery” for accessibility. It would be a great way to deeply integrate Dasher with different input devices and applications across platforms, and make them accessible to people with limited motion, as well as users of VR and AR and mobile devices. link Here’s some discussion on hacker news, to which I contributed some comments about Dasher: A History of Palm, Part 1: Before the PalmPilot (lowendmac.com) link

Speaking of IRCAM, does anybody know Joseph "jojo" Francis -- francis@ircam.fr? Scenes don't sprout like blueberry bushes -- or baobab trees with their roots in the sky. Someone does the legwork -- or the scene quietly dies while everyone assumes it was a natural feature of the landscape. Maintenance is fragile and underappreciated until it stops; you do it for the love of the game even when reciprocity is uneven; the process of putting the thing on is often where the real community forms, not the event flyer. Someone spent fifteen years building a local scene and still says it was worth it after Facebook ate the users. Someone else noted that recreational typing on HN rarely becomes meeting people. Sometimes it does, if you treat the thread as a place to organize a hunt instead of only consume takes. Which brings me to a digression that already started elsewhere in this thread (IRCAM, Apple IIes, DX7s) and to the kind of legwork I actually enjoy: preserving a weird primary source, then trying to find the human who wrote it. Does anyone know how to contact the legendary Jojo -- Joseph Francis, once francis@ircam.fr? I asked FJ!!!; he thought Jojo was last seen in LA. Brewster -- if you're reading this -- any breadcrumbs? I'm wandering the stacks the way the Archive wanders the Web: save the strange pages, then go looking for the people. That is producing social fabric, just with a longer half-life. If you knew him from Paris / Usenet / soc.motss, please point him here or forward this. Jojo, if you see this: please email don@donhopkins.com. I'd love to catch up, reminisce, and talk about applying those brilliant (and ridiculous) UI ideas now. I'd rather you tell your own story than have me invent one. His classic Usenet riff, glorious typos intact: link Jojo built it with words decades ago, and boy did he make me come! He invented a whole product line of design-free GUI kits (POSEUR, XYmorph, Quagmire, boraX -- all open, scalable, and extensible, of course), then drops the mask for one paragraph he titles XBorges: >One longs for the day when the responsibility of programming computers falls squarely on the shoulders of the users, where it belongs; they are provided with a set of infinitely configurable instruction codes, on an open, extensible, and scalable n-bit bus, and their task before setting upon work, is the naming of all the operations they want, and encoding them into words, sentences, phrases and storing them for instant retrieval while they use ideas communicated to them from all the users before to choose most wisely within the infinities of possibility. They have at their hands all books written, all interfaces, they merely traverse endless treelike chains of possibility, of choice, of alternate (open, scalable, and extensible) universes; baobob-like roots in the sky and leaves delving gnomishly in circular connections leading to closed-form solutions. >Visual design: patterns on the screen, snow in hell. Thirty years ago that read like absurdist satire about Motif-era configurability. Today it reads like someone accidentally describing prompting, tool use, and semantic navigation -- naming the operations in words, then retrieving them -- while standing in Borges' Library of Babel. David MacKay said writing is navigating in the library of all possible books. Dasher made that literal: you don't type, you steer through probability space toward the text that already exists as possibility. The Internet Archive is busy saving the books that were written. LLMs collapsed a fuzzy corner of the ones that could be. The hard problem Jojo named is still the hard problem: not generating the infinity, but traversing it wisely -- which is also, annoyingly, the hard problem of community. Infinite possible hangouts. Someone still has to pick a time and open the door. Dasher: information-efficient text entry - David MacKay, April 19 2007, Google Tech Talk

I suspect a major reason it's ridiculous something like Graffiti isn't widely available in 2016 is because of patents, FUD and NIH. A text entry system that requires visual attention which you can't use while driving a car might be considered a safety feature. ;) Then again, it might just kill people faster who insist on texting while they're driving without looking at the road. ;( YOLO! BOOM Dasher has different goals and trade-offs than Graffiti, so its useful for different kinds of applications, but it supports some very important features (like alternative alphabets, and language modeling) that are impossible for Graffiti to provide, which many applications require (especially Hiragana, constrained grammars, text messaging and chat with Unicode and emoji symbols, easy training, and accessibility for people with limited motion or alternative input methods). The paper comparing Graffiti and Unistroke [1] measured Graffiti at up to 11.4 words per minute with a consistent 26% correction rate, and Unistroke at up to 15.8 words per minute with decreasing correction rate from 43% to 16%. Dasher is faster and has a steep but easy learning curve (it's self revealing and doesn't require a reference card). David MacKay described an experiment in his Google Tech Talk [2] that measured novice users performance over time. Bottom of the class started at 5 words per minute, and improved to 10 words per minute after an hour of practice; top of the class started at 12 words per minute, and got to 25 words per minute after one hour of practice. Expert dasher users using a hands-free eye tracking interface have been measured at up to 25 words per minute with an error rate of essentially zero. I also think Dasher would be quite useful and natural for head mounted displays, and that is a fruitful avenue for further research. [3] [4] [1] link [2] ▶ this video [3] link [4] link