Interpreters

Over a year ago I posted about integrated development environments (IDE's), and mentioned in passing that interpreters were

An "interpreter" is a program that executes another program line by line, rather than translating it into assembly or machine code. When an interpreter runs a program, it responds to errors differently than would a computer running the compiled version of the same program. Since the compiled program is merely a translation, an error would potentially cause the computer running the program to crash. An interpreter can stop and send an error message.

That was, of course, profoundly inadequate. I'd like to post just a little bit more about what interpreters do.

Strictly speaking, any programming language may be implemented with an interpreter or a compiler; although some exceptions may apply. Disagreement exists over whether Java is an interpreted language or a compiled language, with some (Lowe, p.11) saying it is compiled, and others (Cohn, et al.) saying it is interpreted. Perl may be implemented with a compiler; so may PHP; Lisp; and Python. I don't pretend to be an authority on the subject, but there's a basic concept in logic that the statement "All x is y" is false if even a single example of x ≠ y can be found. I am vulnerable here on the grounds that disagreement may exist over whether a thing is a compiler or an interpreter.

There are several reasons why a language may be implemented with an interpreter rather than a compiler. First, the obvious reason is that you may want a developer environment that allows you to debug the program. With a compiler, you simply get a message, "Runtime error." It might tell you more, but a really sophisticated interpreter can help you find the actual spot where the error occurred, and even help correct your spelling. Since the compiler's function is to translate the entire program into byte code that the machine can read "in one shot" as it were, debugging with a true compiler is a little like finding a needle in a haystack.

Another reason is that an interpreter may be easier to update and be creative with. Ruby was developed with certain syntax innovations ("principle of least surprise"—POLS), and of course it was a lot easier to create an interpreter that could run an increasing number of commands, than a compiler with a fully-revised complement of libraries, ported to that specific model of microprocessor. Also, a compiler generates machine code, or data in the ones and zeros that the microprocessor actually understands. In contrast, an interpreter can be written entirely in a high-level programming language like C, without any knowledge of machine code.
________________________________________________
How do Interpreters/Compilers Work?

There are several similarities between compilers and interpreters at the operational level. The code that is sent to the compiler/interpreter for execution is called the source file; sometimes, programs written explicitly for use with an interpreter are called scripts. Both interpreters and compilers include a scanner and lexer. The scanner module reads the source file one character at a time. The lexer module divides the source file into tiny chunks of one or more characters, called tokens, and specifies the token type to which they belong; the effect is rather like diagramming a sentence. Suppose the source file is as follows

cx = cy + 324;
print "value of cx is ", cx; 

The lexer would produce this:

cx  --> Identifier (variable)
=  --> Symbol (assignment operator)
cy  --> Identifier (variable)
+ --> Symbol (addition operator)
324 --> Numeric constant (integer)
; --> Symbol (end of statement)
print --> Identifier (keyword)
"value of cx is " --> String constant
, --> Symbol (string concatenation operator)
cx --> Identifier (variable)
; --> Symbol (end of statement) 

The ability of the lexer to do this depends on the ability of the scanner to document exactly where each token occurs in the source filer, and its ability to scan backwards and forwards. Sometimes the precise meaning of the file depends on its position with respect to other token types. For example, operators may contain more than a single character (e.g., < as opposed to <=). The lexer may have to pass a message to the scanner to back up and check to see the identity of neighboring characters.

The parser receives the tokens + token types from the lexer and applies the syntax of the language. The parser actually requests the tokens and assesses their appropriateness with respect to the syntax of the language, and sometimes demands additional information from the lexer module.

Parser: Give me the next token
Lexer: Next token is "cx" which is a variable.
Parser: Ok, I have "cx" as a declared integer variable. Give me next token
Lexer: Next token is "=", the assignment operator.
Parser: Ok, the program wants me to assign something to "cx". Next token
  Lexer: The next token is "cy" which is a variable.
  Parser: Ok, I know "cy" is an integer variable. Next token please
  Lexer: The next token is '+', which is an addition operator.
  Parser: Ok, so I need to add something to the value in "cy". Next token please.
      Lexer: The next token is "324", which is an integer.
      Parser: Ok, both "cy" and "324" are integers, so I can add them. Next token please:
      Lexer: The next token is ";" which is end of statement.
  Parser: Ok, I will evaluate "cy + 324" and get the answer
Parser: I'll take the answer from "cy + 324" and assign it to "cx"

Indents are used here to indicate a subroutine. This illustrates what the interpreter/compiler must do in order to add cy and 324. If the parser gets a token that violates the syntax, it will stop processing and send an error message.

The next module is the Interpreter or, with compilers, the Code Generator, which actually executes the code. With interpreters (as opposed to compilers), this is sometimes part of the parser; the parser interprets and converts the statements into bytecode (i.e., intermediate language, passed off to a compiler). In the case of the compiler itself, the code generate produces machine code that can be executed by the microprocessor.

(Special thanks to Scorpions4ever)

---

SOURCES & ADDITIONAL READING: Wikipedia, Interpreter (computing), Interpreted Language;

BOOKS: Doug Lowe, Java for Dummies, Wiley Publishing (2005); Cohn, et al., Java Developer's Reference (1996)

TRON

TRON was designed by Ken Sakamura of the University of Tokyo as a universal operating system standard that would run in real time, like the in-flight systems used on an airline. TRON was never an OS; rather, it was a set of standards, like POSIX, with a strict hierarchy. At the top was the macro-TRON (MTRON), that would act as the set of standards for universal interface across computer systems. Then came CTRON, used in the largest computers, such as supercomputers, mainframes, and telephone switching systems. Then came BTRON, a standard business system architecture, and ITRON, the industrial microcontroller OS. MTRON was evidently never more than an ideal, and no implementations exist. CTRON acquired NTT as a backer, but the architecture was confined to a line of telephone switches.¹ A variant of ITRON, micro-ITRON, has become a fairly common standard for niche applications, such as navigation systems.

Ken Sakamura, who by all accounts is a lovable and politically astute man, contrived to start the T-Engine Forum. I was struck by the way the literature appeared to have been written by Borat:

In an emergency, Toyota’s Prius hybrid automobile can supply the Toyota home with electricity. The house gives new meaning to Le Corbusier’s famous exclamation that a dwelling is "a machine for living."

("T-Engine [...] Ready for Prime Time" [PDF] 2005)

This is Sakamura's second (?) career as a tech guru. The article I linked to strikes me as something of a joke: identify a large number of objects that have some abstract feature in common, such as the potential to store data; then create a "standard" that consists of terminology so abstract it can embrace them; and announce your technology is "ubiquitous." Fortify the article with references to vapid platitudes and toss in a nightmarish, but influential, guru from the past—like Le Corbusier or Buckminster Fuller. Capitalize on the lesser-known tendency of Japanese sarariman to join professional associations just because it's possible, and one can drop any number of impressive names.

I was immensely surprised to learn that TRON, a project to create a single comprehensive OS for the whole of the incipient Japanese IT industry in the '80's, which was launched from the nation's most prestigious university, and briefly embraced by NTT, never went anywhere. It was never much more than a Zen joke (like this T-Engine hardware spec; see figure below).

Exactly why did the TRON project go nowhere? Part of it has to do with the basic difference between the concept of "winning market share," which became an obsession with Japanese industry, and "conquering an industry." In the 1980's, when the Japanese industrial model had reached its ultimate development, there were typically 3-6 Japanese firms in each industry. In the great majority of cases, each firm was a member of a large business group known as a keiretsu. For example, Toyota was part of Mitsui, while Nissan was part of Fuyo. In a few cases, such as Honda, a firm might not be affiliated, but this was the exception. Winning market share and ranking was a matter of immense pride among the affiliated firms, although actual extinction was unlikely.

By the 1980's, the standard business model in Japan included assimilation of existing technical standards, like DOS/V, and excelling within that field. The market was sufficiently segmented that, unless unavoidable, Japanese firms avoiding taking a stand of their own on an industrial standard. Rather like the brilliant politician, whose career consists of avoiding difficult positions and anticipating popular ones, the Japanese technology firms seemed to use standards like decals. Executives were alarmed when they discovered embracing a new technology meant slamming the door on another. Because of this, TRON withered on the vine. It was distinguished by the amazing abilities of its godfather, Prof. Sakamura, to look like a decisive guru.

---

1 My link to the CTRON page is basically Ken Sakamura's plug for the concept. The page is now almost 10 years old, which is about the point when TRON largely vanished as a developmental objective. He mentions several firms, such as Tandem, developing a fault-tolerant devices and switching systems based on this standard, then sums up that it "is a standard" in Japan. Those firms were contractors for NTT. A few enthusiasts, unfamiliar with IT terminology and claims, or prone to hyperbole, have gone so far as to insist it's "the most popular operating system in the world"(ITRON is not an OS). Almost none of the information in the linked article is true or, for that matter, logically coherent enough to be true OR false.

Keiretsu: The keiretsu (business group) model is distinct from the business group found everywhere else in the world in that it has the external linkage (crossholding of shares) and standard convoy structure, with a trading house, commercial bank, and investment bank at its core. At its culmination of development, the Japanese economy was dominated by six keiretsu, each with its own commercial bank. A seventh bank, the Industrial Bank of Japan, was the battle cruiser in the financial convoy system. The entire system was restructured out of existence in the late '90's, as the seven banks were merged into three and the non-financial firms developed a somewhat more conventional relationship.

Mac & Windows

I'm going to have to admit that I do everything in MS Windows. For years I dreamed of a Macintosh, but put aside those dreams when Steve Jobs ended licensing of the Mac OS, and insisted on a monopoly of the proprietary operating system. Since then, the user base of Mac OS has dwindled as a share of the total, and Job's focus has been on Mac as an expression of profound individuality. However, I have observed repeatedly that there is an awful lot of really good creative blogging out there being done by Mac users. I've noticed that the Mac user base, far from being the group of technical novices I would have expected, is actually more technically competent and shrewd than the Windows user base is.

I still have this big ugly pool of frustration with Steve, because I think he spent so much energy marketing Apple products as fashion accessories, but I've gotten older and increasingly I recognize that Apple products have retained a crucial technical edge over Intel ones in many respects. The anarchistic piece of patches and bugs that is Windows essentially uses up all of the skill and talents of coders. The hideously dysfunctional business climate for Windows software developers consigns entire cohorts of coders to arbitrarily-inflicted obsolescence. There's a great Japanese word for this, muda, which IMO describes about 75% of the effort in Windows development. Here we are, twenty years after the development of the GUI, and we're taking up Linux as the alternative to Windows. The fact is that Windows is not a programming environment, it's a cross between the Cold War and the Bosnian Civil War. The Mac OS community is, by this analogy, a stable productive society. So Steve handled the technical issues successfully.

As to the marketing: I have a really tin ear. My ability to anticipate the effect on the public of a certain thing, like the presidential debates, is so poor I've had to give up on the exercise entirely. The sort of reasoning that win arguments nowadays is not my reasoning.

That's where we're going with this: the market of ideas and products as a programming environment—a civil OS, as it were. I'm used to thinking of operating systems as a bunch of specialized code that you install on a computer so the damn thing can open Word files. But it turns out operating systems are a metaphor for society. The components of the program (institutions) work by collecting what we know about pieces of data (individuals) and associating that data in ways that serve the survival of the institution. The system files mostly evolved in different places, and they don't necessarily work together very well together, but it's much to late and too difficult and too controversial to revise them so they make any sort of sense. There's also the core of the OS, which gradually absorbs more functions of the system software (software drivers, social welfare systems and child protective services) but there's always going to be things that the OS cannot do, that have to be done (things that the state cannnot do, that require spontaneous or traditional associations).

And I notice that the programming API (the standards that applications have to fit into in order to run in a programming environment) for our society is set not by political philosophers like John Locke or Thomas Aquinas, but by fashion. This is not a terribly original idea, but it's something I keep overlooking. Karl Marx, John Stuart Mill, W.E.B. DuBois, and Joan Robinson came up with really compelling analytical models to explain how societies interact with their institutions—yes, they did. But these models are terrible at predicting the future, and people who understand them become worse, not better, at reading the responses of their neighbors.

The Mac OS deserves to beat out the Windows OS because it works like a properly engineered design. It's a more effective, waste-avoiding strategy of doing what programmers and computers are supposed to do, than Windows ever will be. It spawns creativity; the anarchy of Windows, as we've seen (and seen, and seen, and seen again) spawns drudgery and frustration. This is not because Windows programmers are stupid; nor do programmers code for Windows because they're masochists. They do it to earn incomes and get their ideas on people's desktops. Windows is the world in which we live; Mac OS is an enclave of sanity. It's like a tiny little corner of Bosnia where Serbs, Croats, and Bosniaks live in harmony. The rest of the country is chaos, and that's the world of computing: massive waste of potential because of chaos. People never actually chose chaos, but that's what we get. So we plan for it, and we assume it.

And that's why fashion sense—a system of hierarchy that explicitly repudiates any logic—is the law of the jungle. Jobs understands that. Why did I not see this before?