Of interfaces, and Days Made of Glass

[Spenser TR]

When I'm not occupying my mind with Traveller, every now and then I manage to spend time visiting the topics of interface design and user experience. Just like Traveller is way more than "just talking about spaceships flying around," these topics are about much more than just how something looks on a screen.

I could make your eyes glaze over, with just an overview.

Anyway, tonight I stumbled across a couple vids I hadn't seen in a while. Put out by Corning about 6 years ago, a lot of people I know referred to them as "investor ⌧," that is, something meant from the Bar Napkin Stage on to elicit pleasurable responses from venture capitalists interested in developing tech. Such is the nature of my chosen profession.

Among other things, Corning is involved in research into a zillion different applications for glass and glass-like materials. As the "internet of things" comes more and more into our typical days, it's a given that people are thinking about taking the control surfaces we see on tablets and putting them... everywhere. Well, lots of places, anyway.

Think of how we interface with the digital world: through keyboards, mice, touchpads, styluses, but now also fingers, gesture, overlay ( AR ), immersion ( VR ), and aural/voice, just to name a few. More are coming.

Now think of how we might do these things in the far future. Interesting stuff.

The Corning videos are called "A Day Made of Glass" and ( very originally ) "A Day Made of Glass 2." They're a little dated now, but from a human-computer interaction point of view there's still some wonderful visuals and thought processes in them. Some silly stuff too, but that's how it goes, with investor ⌧.

 

[flykiller]

Think of how we interface with the digital world: through keyboards, mice, touchpads, styluses, but now also fingers, gesture, overlay ( AR ), immersion ( VR ), and aural/voice, just to name a few. More are coming.

Now think of how we might do these things in the far future.

could start off with a discussion of the five input senses and the possible discriminatory motion output. for example the army devised a system to aim a gun according to where a soldier turns his helmet, and then where he turns his eye pupils.

and there's the ever-popular "mind scanning".

a second consideration is how those devices might interact with us.

http://www.travellerrpg.com/CotI/Discuss/showpost.php?p=485976&postcount=1

 

[nobby-w]

[ . . .]
Think of how we interface with the digital world: through keyboards, mice, touchpads, styluses, but now also fingers, gesture, overlay ( AR ), immersion ( VR ), and aural/voice, just to name a few. More are coming.
[ . . .]

In the days of light pens there used to be a term called Gorilla Arm Syndrome, and I remember seeing a cartoon about the holographic UIs in Minority report. This showed an office full of very buff office workers using the holographic displays and a job applicant getting rejected with a reply on the lines of 'Your qualifications look great but you just don't have the upper body strength for this job.'

One of the reasons that keyboards and mice stood the test of time (and interactive graphics systems have been a thing for nearly 50 years) is that they have a significant ergonomic property. You can support your arms on the table you are using them on.

Gesture based interfaces work on mobile devices because they save space through not needing a keyboard or external pointing device. Gorilla arm syndrome would be a significant problem in using this type of interface on a workstation screen. Trying to embed the UI into a horizontal desk surface would cause the sort of posture issues in its users that wind up in product liability suits.

For better or worse I think that utopian visions of the internet of things are exactly that. And that's even before we start on the security issues or potential as a mass surveillance platform.

As for speech interfaces, try listening to someone using Dragon Dictate and marvel at just what a Kn*b they sound. Now imagine an entire office doing that, or better yet trying to use it on a noisy factory floor or battlefield.

 

[Spenser TR]

Gorilla arm! Ha! I remember hearing about that before I got into the field. Way before, actually, reading an article about screens at Epcot down in Disneyworld. They spent a ton of cash putting them up because they thought it was cool, and then just as much taking them down shortly thereafter. And yea, there's no shortage of commentary and a little research to suggest that non-horizontal touch interfaces area a challenge. Steve Jobs was pretty adamant that Apple laptops would never have screens that were touch-reactive, because of this.

A very interesting counterpoint to all that is AR, where nerds all over are currently holding their phones out in front of them for shocking lengths of time catching Pokemon.

And that scene from Minority Report - single most popular reason cited in foundation classes I had way back in the day, when we all discussed why we'd gotten into the HCI MS program in Chicago in the first place. Also the most popular ( in my own experience ) example of gesture-based experience models in movies, with the Wii ( of course ) holding that honor in real life.

Mice are brilliant IAs - intelligence amplifiers. They let a person use eyesight and spatial relations for situations where before you used text ( the command line ) and imagination. Or not. The touchpad is just an evolution of this. And in the comfortable position that does not promote gorilla arm. But does let's you get goop on your computer.

And speech... ha. I'm with you there, too. But it's getting better all the time. A long, long time coming. Like VR. But it will definitely happen. It's happening as we speak ( ha ) as far as passive gathering of information ( Siri, Alexa, Fire Stick, etc. ) and spitting that back out at us as advertising, and mostly we don't even know it.

I'm agree about utopian visions, Nobby-W. As a UX professional I could write a pretty long report about the issues, challenges, and reasons -not- to do a lot of what the videos show - but they're not a road map. They're a light house, meant to draw the eye and inspire. Especially to inspire investors to open their checkbooks for Corning, back in 2010, which they most certainly did.

For me, they're also pretty sweet to help me imagine some ways that multi-platform wearables might saturate daily life. The ways information interfaces could respond to your actions, and when. Walking through the arcology, boarding the ship, whenever.

and Flykiller - a lot of the roots of HCI and into mission-critical apps such as cockpits, weapons platforms like HUDs, air traffic control interfaces, and surgical implement design. When computers and then the web et al came along, just a natural fit for the field.

And a very interesting short - dystopian for sure. At least from my point of view. But an interesting thing: when that type of tech evolves, the countermeausres evolve with it. Guy Fawkes masks and glasses that shine UV light to foil facial recognition cameras, and so on. When the tech is just sort of dropped in somewhere instead of evolved, things might go differently in that society. Very interesting stuff.

 

[aramis]

Among other things, Corning is involved in research into a zillion different applications for glass and glass-like materials. As the "internet of things" comes more and more into our typical days, it's a given that people are thinking about taking the control surfaces we see on tablets and putting them... everywhere. Well, lots of places, anyway.

Flat glass touchpanels are already on lots of higher end appliances... especially...
Refrigerators
Induction Stoves
Ovens
Microwaves.
Televisions

Membrane panels in glossy black have be de rigeur for same for 2 decades, now. The nifty shiny black face is just replacing membranes with capacitive touch...

 

[Spenser TR]

Indeed. Designing for shifting contexts ( standing up? Just for a moment? At work? Writing, or just looking, or adjusting? And for households or businesses in Europe? Asia? ) As well as tandem environments where one screen affects another, or a few others, and oh yea maybe it's not a screen, but a little thingy that can only give haptic feedback... all good times for the designer. And the stakeholders who are looking to be made happy, make money, and look smart.

But I digress.

A few ways interfaces might work their way into a game of Traveller:

- the rifle that works only when it recognizes it's you. But not today, because you have dirty gloves on. Or bad breath. Or whatever.

- the cohort that communicates with you by painting on the lens of your sunglasses, because the GD commdot in your ear is not working. Again.

- the ship asking you for the password, and chastising you in front of the other Travellers when you get it wrong. As a way of warning you an enemy is lurking aboard. How many vowels in 'Iishiraanaraagiir'? A lot. Sh*t...

- some ( un )lucky sophont in the group sees into a spectrum humans don't, so they can't see all the freaky subliminals on the touchpads in this startown. Weird.

- with the flick of a wrist, your nav data is now on the big screen for everyone to review. Unfortunately, this is also an obscene gesture in the local vernacular, and here come two big hulks to discuss the matter with you.

- By accident you grabbed Foley's profile when you sat in his pilot chair, and saw how his AR mods were all in what looks like Zho-gibberish. Hmmmmm.

..

Or not. This stuff all happens in my games at the speed of plot. In your games, maybe not at all. But interesting to think about.

 

[Condottiere]

It pricks your trigger finger, and makes a direct comparison of the DNA.

 

[Spenser TR]

...or talks to the dongle in your pocket, or tags around your neck, or the nuances of the trace perspiration on your palm. Or it hears your particular war cry. All things that can vouch for who you are so your rifle works the moment you grab it. Lots of possibilities. But I like the bit of blood... it works artistically, for a weapon.

(^_^

 

[Sstefan]

It pricks your trigger finger, and makes a direct comparison of the DNA.

...

... and then confines you to the sick bay because of "last night's 'adventure' in star town"...


:rofl:

 

[nobby-w]

It pricks your trigger finger, and makes a direct comparison of the DNA.

And generates a huge aftermarket in mods to disable the user identity feature. Authorised user identification systems for guns seem to be popular with pretty much everybody but the folks actually buying guns for self-defence.

I would be willing to bet quite a few credits that we will either never see an actual armed force buy small arms with this type of feature, or that they will have the feature removed after a few high profile 'failure to operate' incidents.

 

[whartung]

Well, I'd paid Real Money™ for practical windows that can absolutely darken electronically. Curtains are simply a pain.

I would like to see more "awareness" in the devices. Especially geo-spatial awareness. Have 3 devices, the one in the middle "knows" which ones are to the left and right of it. There's already some locality that can be expressed, using Bluetooth. So we can know when things are "near", but we don't seem to know yet where they are spatially.

I agree with the issue of "gorilla arm". We did a touch interface years and years ago. It was a simply categorizing system, using basically smart terminals, big boxy buttons.

I don't know how the users reacted to it, frankly. They were sorting and categorizing clothes. So, they were moving about with their arms already, and the touch may have been ancillary to the motions they were already doing.

But it's certainly not efficient, swinging a several pound arm around vs using dextrous fingers. It works for one off kiosks, but it gets old after about 3 steps IMHO.

It's not unusable in my car, the touch screen interface, but it's awkward. I end up anchoring my hand on the console at the corner of the display and then using my fingers to touch the controls.

A multitouch desk would be interesting. Dunno if I'd want it getting rid of a mouse (end up with a callous on my finger). But it could be interesting.

 

[flykiller]

Authorised user identification systems for guns seem to be popular with pretty much everybody but the folks actually buying guns for self-defence.

actually the biggest issue with these is how they fail. the manufacturers want them to fail into "no-shoot" so they don't get sued. the users want them to fail into "shoot" so they don't die. irreconcilable conflict.

heh. and then there's the issue of hacking and remote fail into "no-shoot". it's not a bug, it's a feature ....

 

[flykiller]

- the cohort that communicates with you by painting on the lens of your sunglasses, because the GD commdot in your ear is not working. Again.

with a world-wide internet-of-things there's no particular limit to the number of paths of communications. and with translators there's no particular limit to the number of communication modes.

a traffic light starts blinking morse code at you. in vargr.

And a very interesting short ... when that type of tech evolves, the countermeausres evolve with it.

thanks. if opponents have the resources to do so, sure. in my version of louzy the general citizenry doesn't.

 

[nobby-w]

It's not unusable in my car, the touch screen interface, but it's awkward. I end up anchoring my hand on the console at the corner of the display and then using my fingers to touch the controls.

I think iDrive is just about the worst thing I've ever seen in a car console. It's not a solution to the problem of 'How do we control a the features in a car better?' Rather, it's a solution to the problem of We've made the M5 too fecking complicated.

It's an attractive nuisance that takes your attention off the road while you search through menus. I wouldn't be surprised to see a class action suit started by folks who have been run over by BMW drivers distracted by trying to navigate through the menus on their iDrive system.

I would buy a car without such a system as a matter of preference - push buttons have much better affordance, and they're much safer as you don't have to divert attention to hunting through the menu system.

A multitouch desk would be interesting. Dunno if I'd want it getting rid of a mouse (end up with a callous on my finger). But it could be interesting.

The problem with integrating the display into the desktop is that the ergonomics are really bad. You would get serious neck problems trying to use a system like that for any length of time.

 

[nobby-w]

[ . . . ]
a traffic light starts blinking morse code at you. in vargr.

Network security and the internet of things ...

 

[robject]

I think iDrive is just about the worst thing I've ever seen in a car console. [...] it's a solution to the problem of We've made the M5 too fecking complicated.

It's an attractive nuisance that takes your attention off the road while you search through menus. [...]

I would buy a car without such a system as a matter of preference - push buttons have much better affordance, and they're much safer as you don't have to divert attention to hunting through the menu system.

SYNC. Ford. Microsoft. Driving. Me. Insane.

 

[whartung]

My wifes GM system in her car is nice. Of all things, there's haptic feedback when you touch things -- that alone is really helpful. I have the Chrysler system in my Jeep.

But all we really use it for is the radio or connection to the bluetooth radio, and talking to the phone. For that it all works fine. The A/C is buttons, seat warmers are on the display. I don't have a nab system, I just use my phone for that. Chrysler published a recent patch (I have yet to install) which enables "Hey Siri" on my phone, which will be nice if I ever get around to installing it.

Buttons would be better, but with all the features, you soon get overwhelmed by buttons.

 

[nobby-w]

[ . . . ]
Buttons would be better, but with all the features, you soon get overwhelmed by buttons.

If you have to take your eyes off the road to navigate through the menu system then that has implications for safety. In that case, it could be argued that the car essentially has too many features to be operated safely while driving.

Unnecessary features in safety-critical systems are a no-no. I've come to the view that automotive systems that require a driver to divert attention to a menu in order to operate the features are essentially a design flaw. Menus are avoided (or at least minimised) in glass cockpit systems for much the same reason.

 

[Enoki]

It pricks your trigger finger, and makes a direct comparison of the DNA.

Does it shoot the holder if the DNA doesn't match...? :devil:

 

[Condottiere]

Depends on the embedded personality profile: