Applied Sciences Group: High Performance Touch

>I just want to say I am glad pro gaming took over. Yes! I have been crying about latency for nearly 10 years [1]. Computing has always been optimising for throughput. And before Pro Gaming, there just hasn't been a marketable incentive for companies to work on / minimise latency. Now we finally do! Even in the best case scenario, the lowest latency is still 25ms, and in most cases we are still above 50ms. I think it is worth posting [2] Microsoft Research on Input latency. It would be nice if we could get average system end to end latency down to sub 10ms level. Which is what I hope work on VR will bring us. [1] link [2] ▶ this video

End-to-end latency is definitely something that deserves attention. It'd be interesting to modify mouse&kb by adding LEDs in series with the switches. That way you'd get good visual indicator when the event happened without any added latency. Combine that with 144hz monitor and actual high-speed camera (I hear good things about Chronos camera), and you could do very accurate software latency measurements This MSR video from few years back is also pretty cool wrt touch interface latency: ▶ this video

That’s the prevailing statistic, but your original number isn’t that wrong either: Humans can perceive much smaller latencies. If you look at the Card & Miller reference, at least some humans can perceive differences in ~50ms vs 100ms latencies when typing (in my limited testing, it’s likely you can!). There’s some newer research I don’t have handy that I believe found error rates decreased and NSAT improved until around at least 30ms (if not 20ms). On that note, humans can definitely distinguish 60hz vs 120hz reliably (about 8ms faster per frame). Even faster: with a reference (eg when dragging on a touchscreen), humans can distinguish down to at least 1ms vs 10ms of latency: ▶ this video And you can probably distinguish metronomes that are off by about 1-2ms. Much smaller for other things (like metronomes that slightly slower or faster than one another). This is a special interest of mine XD

The F1166 standard permits up to 250ms latency for touch screen response. That is ridiculously high. It practically guarantees people will think it's not working and tap again, undoing any toggle function they might have activated. They won't necessarily notice their mistake. I once accidentally ran a washing machine at 90C temperature because of high-latency touch controls. I instinctively double-pressed the touch surface to reduce temperature because I thought the first touch hadn't registered, causing it to wrap around back to maximum. It did some minor damage to the clothes before I noticed. And that was with more like 200ms latency, not 250ms. Latency on touchscreens is more important than with physical controls because there's no tactile feedback. Even 10ms is obviously imperfect for drag controls. Microsoft Research published a demonstration video: ▶ this video

I would like to try a 1 kHz display. I can definitely tell the difference between 120 and 240 Hz for mouse pointer motion. The motion blur tests mentioned in the article make it obvious as well. And Microsoft research has shown that when using direct touch interaction people can perceive latency all the way down to 1 ms. ▶ this video Also an underrated feature of high frame rate is that it's a universal fix for poorly written software with multiple frames of latency. An application with 10 frames of latency will be faster on a 1 kHz display than a perfectly coded application on a 60 Hz display.

> I seem to remember that a good pencil has lag on the order of 1ms (how long it takes the tip to settle out after high speed motion), but I don't remember where this number came from, so it's untrustworthy. Not sure what lag there is on actual pencil, but there's this somewhat famous video[0] from Microsoft Research, where they test various values of input lag on a touch screen, and demonstrate that even 10ms is noticeable - but 1ms is about enough for it to feel instant. With 1ms lag, drawing on a tablet seems to feel just like drawing on paper (I personally haven't had an opportunity to test a tablet with 1ms lag, so I can't tell). -- [0] - ▶ this video

When I saw this page a few years back I had an idea for a project. I want to create the lowest-latency typing terminal I possibly can, using an FPGA and an LED array. My initial results suggest that I can drive a 64x32 pixel LED array at 4.88kHz, for a roughly 0.2ms latency. For the next step I want to make it capable of injecting artificial latency, and then do A/B testing to determine (1) the smallest amount of latency I can reliably perceive, and (2) the smallest amount of latency that actually bothers me. This idea was also inspired by this work from Microsoft Research, where they do a similar experiment with touch screens: ▶ this video

You can easily observe how significant latency is in videos like this: ▶ this video (Microsoft Research presenting its ultra low latency displays for touch interactions). Many mobile games have you drag and drop things, so it's not like it's just first person shooters that suffer from latency. You're a lay person, you couldn't have known this, you're using words with very specific meaning to streaming (like latency) and you're comparing it to human reaction times, which are measuring something else entirely. You kind of reasoned about from a first principle in a very Paulgrahamarian way, and it led you deeply astray. That happens. And you're not the only person doing this, this is a comment section full of people who play games and parrot stuff they seen in YouTube, and don't have a concrete grasp of what it is they're even talking about, so it's understandable when it's laypeople shouting at laypeople that it's just a bunch of blah. One of the reasons I hate HN and write in throwaways nowadays is that the comments section is a better example of Knoll's law than actual journalism.

Oh thank goodness, so it's not just me who is driven insane by the mouse lag in browser input. And while I understand your cynicism, I don't think it's deliberate as much as it's complacency, given that it's latency issues all the way down when it comes to modern user interfaces. A literal decade ago Microsoft Research uploaded a video highlighting how much of an effect minimizing latency for touchscreens had on the user experience[0]. The vast majority of tablets still utterly suck in this regard a decade later, and I wonder if any manage to get below the 10ms barrier. And of course there is Dan Luu's famous 2017 article on computer latency in general[1]. [0] ▶ this video [1] link