Favorite quotation: Before you criticize someone, you should walk a mile in their shoes, that way when you criticize them, you're a mile away and you have their shoes. - Jack Handey

Sunday, July 14, 2013

Text Abstraction Layer

There really needs to be a text abstraction layer in the OS for editing and selecting text.

Ever since I tried Microsoft Word 1.0, I have been frustrated by several things about text editors. One problem that I have is that there is some invisible markup for text formatting that has an associated Unicode character, but there's some that does not, yet the method for selecting text, copying it, and pasting it seems oblivious to this distinction, creating inconsistencies in user experience.

On the one hand, you have invisibles characters that still have standard Unicode values. For example, a new line is actually an invisible "new line character" that causes a new line to be rendered by the software. Tabs are actually a "tab character" in the text itself that is rendered by the software as a tab. Therefore if you select an area containing a tab or a new line, that copies with the text. If you turn on "show invisibles" in many editors, you can see these characters. Maybe there's a better way to approach this, but I can live with this situation.

However on the other hand, each software program that can perform any type of text editing has its own specific markup strings that are concealed from the user. For example in my current text editor, MacJournal 6.0.6, if I'm typing and I press command-B for bold, now the next characters will be bold. If I then type command-B again, and then backspace (I'm going to do it right now), then the next characters are still bold!

This is inconsistent behavior. If I press command-B twice, that should toggle bold text entry on and then off again. Why should it matter if I pressed backspace? Maybe I wanted to change back to normal text entry, then backspace over the last bold character I typed, and re-type it in non-bold text. Why should hitting backspace undo my decision to toggle back to non-bold text?

The reason is because when I press command-B, the text editor inserts an invisible markup string (that only it can see) into the text file. For example in Apple TextEdit for Mountain Lion, lets say I create and save a simple test text file with one line of text inside:

Test test test.

When I save this file, TextEdit will save it to disk as a file called "test.rtf". The file extension, .rtf, is hidden from the user by default, but it's visible if you go into Terminal and navigate to the file. And then if you inspect this file with a good file editor (like HexFiend or vi), you'll discover that it contains much more than meets the eyes. Its actual contents are:


{\fonttbl\f0\fswiss\fcharset0 Helvetica;}




\f0\fs24 \cf0 Test

\b test

\b0 test.}

As you can see there are quite a few hidden characters. If you look past the gobbledegook at the top, you'll see the last three lines are what we actually typed. You'll notice that when we pressed command-B, then typed something, then pressed command be again, then typed some more, TextEdit was creating an RTF file as we went along, inserting invisible markup like "\b " (hex value: 0x5C6220) and "\b0 " (hex value: 0x5C623020) as breadcrumbs so it would know where the bold text begins and ends.

The problem is, this invisible markup cannot be seen by the user, yet it gets affected by editing commands issued by the user! So for example, if we do the following steps:

  • type the middle word "test"
  • press spacebar
  • press command-B
  • press backspace
  • type the last word "test."

then the last word will still be bold. The reason is because even though we couldn't see the string, "\b0 ", when we pressed backspace and erased the last character in the file (which was a space), this caused TextEdit to erase the "\b0 " markup code as well.

Why would TextEdit think we wanted to go back to typing bold because we typed backspace? Shouldn't it only do that if we pressed command-B again? Why did Apple make it that way?

I hypothesize the answer to "why" is because the text editing that the user is doing is not completely abstracted from the text editing that the software is doing, and text in computers fundamentally does not have formatting attributes at the character level. Each character is just a couple of bytes fundamentally. Any formatting like bold must be added to that character via surrounding markup in the file. The software edits the file adding this markup according to what the user is doing. But rather than letting the user's editing occur in a sandboxed environment completely abstracted from the file formatting conventions of the file on the disk, the text editor merely creates the illusion of doing so, while in reality the user is actually directly affecting this disk-level file's invisible markup via editing commands like backspace and return.

Text editors have worked this way since before the Mac even existed. So why are you only now filing a complaint, after 29 years? And how, Mr. Genius, would you propose we solve this "problem" that only you are whining about?

Well, good question. First let me enter an analogy.

Photoshop does not have this problem. If I create a 1x5 pixel bitmap at a bit depth of 8 bits, I will have a 5 byte file:


Now if I change the first, third, and fifth pixels to 50% gray, which represents normal text, I will have:

6C FF 6C FF 6C

Then if I want to change the middle pixel to black, which represents bold text, I will have:

6C FF 00 FF 6C

If I then change my pencil color to gray again, and reduce my canvas size to 1x3 pixels, then back to 1x5 pixels, my pencil color won't change. If Photoshop was TextEdit however, my pencil would change back to black, because my decision to change my insertion preference to bold was encoded into the file itself. Whereas in Photoshop, setting my insertion preference to gray or black doesn't alter the file in any way. This is how it should be.

What the text editor SHOULD do is represent each character like Photoshop represents pixels. Each character should be a series of bits where some of the bits represent the Unicode character it is, and the rest of the bits represent the formatting information about the how the character ought to be rendered, such as its font, style, and everything else. When the user sets their preferences in the text editor, they would not be inserting invisible markup data in between the characters they are typing; rather, they'd just be setting the "color" of their "paintbrush" such that whenever they type, they are generating characters each have metadata telling the computer exactly how to render them on screen. If a user changes their formatting preferences, then hitting the backspace key would not affect it. Of course my scheme would seem to require much more memory. I can't deny that. However, just like images can be compressed using the JPEG scheme, text encoded in my format would be able to be compressed in lossless or even lossy compression formats. No problem.

I'm going to name my concept "textels" so people don't confuse the members of our textel bitmaps confused with plain old characters.

Even more than its usefulness for editing, however, this concept would be even more awesome for text selection.

Have you ever run into a web page where you tried to select text on your mobile device, and it wants to select the whole web page frame? Or it won't let you select individual words in a block of text? Or it will let you select one word, but when you try to select the entire paragraph, it freaks out and selects a bunch of weird non-contiguous blocks of text in seemingly random fashion?

These bad behaviors are all symptoms of the same problem I describe above, except the invisible markup is HTML, XHTML, and CSS, instead of RTF file formatting. Same difference really. The end result is that as you are trying to move the selection point around text on your screen, the browser is interpolating your actions as instructions on how to move an invisible insertion point around inside the invisible code of the webpage, which may or may not directly correspond with what you are seeing or trying to do. This causes all manner of weirdness, because the browser doesn't even really know what you are actually seeing and interacting with; it only ever deals directly with the HTML code itself. You trying to select a block of text inside a selection area might be perfectly rational to you, but to the browser, it's a big no-no because of an invisible HTML tag that makes this text non-selectable in some browsers. Even worse, if you're typing into a text field that has some active JavaScritps controlling it, your text entry can become unbelievably laggy and annoying, because now, the web browser is competing with the JavaScript web app to render the text properly and handle your selections. Argh!

Alright, Mr. Smarty Pants, how do you propose we fix this so-called "issue" of yours, eh?

Well, what the browser SHOULD do is, whenever you try to select text, let you interact with an abstracted textel bitmap layer, so that the browser is seeing the same thing you are. If you select three words that appear in a row, left-to-right, on the screen, then it would actually just select only those three words (not the nine paragraphs below and three paragraphs above that!!). Wouldn't that be nice? When you start editing text in a field on a webpage, it would open a separate panel that would let you compose all your text in an auto-saved environment in case the browser should crash or your session should expire while you're typing (no more lost forum posts... imagine that!!). If there are JavaScript controls on the page, they wouldn't get to touch the text until you entered it from the editing panel into the webpage itself. And if we were really enterprising, our browser's editing panel could smartly find any formatting buttons in the text field, and generate equivalent textel map formatting buttons for the purposes of editing, then, when the panel is dismissed, issue emulated UI interaction to the JavaScript to accomplish the formatting that the user did while in the editing panel (hell yeah!!).

One can always ream.

Saturday, July 06, 2013

Night Mode Suggestion

Sent to Apple:

Summary of the proposed feature:

I suggest that you add a system Setting called "Night Mode." Night Mode would set the UI background from white to black or dark grey, and would set Dynamic Text to be white or a user-definable, easy-on-the-eyes color (perhaps "terminal green" or a pleasing off-white tone). Unlike "Invert Colors," Night Mode would not affect the color of tinted UIButtons, photos, webpages, app icons, etc., improving consistency across the interface.

There is already something like "Night Mode" in Maps app. Maps automatically sets to a dark color scheme based on the time of day. This obviously improves clarity at night, in deference to drivers who are depending on their night vision for safety reasons.

My suggestion is that we, the users and developers, should be able to have a night color scheme in all apps, not just Maps. Night Mode would be able to be set to automatically trigger based on ambient lighting, display brightness level, time of day, or to manually trigger based on a toggle. Developers should be able to make a "night" version of their app's UI that would automatically activate when system is set to night mode if their app is made to support that.

I would even go as far as to say that you should call this "Dynamic UI," to go along with Dynamic Text. Developers would be able to implement Dynamic UI elements into their apps so that when the user changes to Night Mode then all apps that support Dynamic UI would adapt automatically to this.

Summary of the current problem:

iOS 7 contains lots of white space. This is great during daylight hours, especially in direct sunlight, where it improves readability. However, lots of white space is NOT great at night or in a dark room, like a movie theater, concert, a car at night, bedroom, studio, etc. In these dark settings, lots of white pixels induce the opposite effect from what you intend. White will not lend the "calming" effect your HI guidelines suggest it ought to have.

Rather, a white screen disrupts night-vision, interferes with melatonin production in the brain, and increases eye strain. Numerous medical studies (I can provide links if you want) have shown that white screens can disrupt sleep cycles by stimulating the same optic cells that get stimulated by the blue wavelengths from the sun. One study even showed this had a negative impact on cognitive function. I find that even on minimum brightness, the white is still very annoying at night.

Inadequacy of Current Solutions:

The existing solutions to this are not adequate. Below I explain why reduction of display brightness, and invert colors, are both not adequate to resolve this problem, due to how they violate the iOS 7 HI guidelines.

Why Reducing Display Brightness is Inadequate:

If the user sets their device to the minimum brightness, in my experience even then, it's too bright if you're in a dark room. I often find that by having to lower my brightness to reduce eye strain from white areas, now my photos and other app content all appears too dark. This means that all the white in the interface actually competes with user content in dark settings, violating the HI guidelines of "clarity" and "deference."

Why Invert Colors is Inadequate:

Currently the only option to avoid the white background of iOS 7 is the "invert colors" control, but that also breaks iOS 7's "deference" HI guideline by affecting all app content, such as photos and web pages, not just the white background. It also affects the background image of the home screen and all the app icons, making the system feel unfamiliar. I would say that "invert colors" also makes the system less clear by changing how icons appear and making all the system colors backwards, breaking the "clarity" HI guideline also.


Therefore I suggest that you add the "Night Mode" feature, as described above. I think this would go a long way to quieting any criticism people may have of iOS. If it's too bright and white for them, they can go with the dark look. I think even those people who criticize the whiteness of it will find that in bright conditions, it's better that way, so having the option to set it to light or dark based on the environment and user comfort leve would be ideal. With such an option I think iOS 7 will be much better received by the public. Half the time or more, we're in the dark. Please don't blind us.