Alan Turing’s Linear Algebra Legacies: MATLAB, APL and Q

I’ve always been a history buff, yet having failed to make a career out of it, I sold my soul to what some call scientific computing, and in particular computational finance. Yet only recently have I discovered the very own amazing histories of my accidental career!

A Simple Scientific Computing Backgrounder

In a single sentence, scientific computing entails mathematical or scientific principles in software and computing to drive engineering and science use cases and applications. Computational finance specifically applies scientific computing to financial applications, including but not limited to high frequency trading and market modelling, instrument pricing including options and derivatives, portfolio construction, econometrics, and risk management. Scientific computing matters because:

It is built on matrix/linear algebra
Matrix/linear algebra underpins key discriminative, predictive and generative AI technologies like machine and deep learning, transfer learning, tensor modeling and good old fashioned honest statistics.
It is the basis of quantum computing
Early computing was scientific computing-centric, as this article will show.
Popular scientific computing software options include MATLAB (per this blog), R, Python (with NumPy and SciPy), and Wolfram Mathematica, also the language Q.

Inspired by the Origins of Q and APL

My interest in scientific computing history was fully triggered after joining KX in 2022. KX develops the uniquely efficient, terse, vector processing, no loops, functional programming language Q on which its kdb+ database is centered. Q is a pain in the ass to learn, but for those who love a learning challenge, it is a wonderful experience. For “mortals,” the ultra-efficiency of Q, and the kdb+ database, is readily accessible from common languages like Python and SQL.

I note the coupling of the kdb+ database and Q programming language. Databases command a higher price tag than a language so marketing databases tends to be more profitable, but ultimately databases are *just* managed data structures and transformations built on — and queried by — programming languages. That’s true of all databases, but in kdb+’s case, the database is built in a version of, and queried by, the language Q and the coupling is possibly the tightest in the industry, one reason they’re so efficient. Q gives kdb+ its wings. Q also happens to be a descendent of the equally terse language APL, “A Programming Language,” and the J Programming Language.

Following a somewhat unorthodox internal sales enablement I ran at KX (public version here), I met KX “librarian” Stephen Taylor. He quite liked my “why kdb” angle which strove to highlight efficiency and kdb’s postage stamp footprint rather than kdb’s traditional speeds and feeds. Stephen and I met at the beautiful The Wren coffee house, in an old church on Queen Victoria Street one cold, grey autumn afternoon. We bonded over tales of mathematical programming languages — I regaled him about MATLAB, Java, Python — and our shared love of travel, writing, books, and society. We put the world to rights.

Stephen was then kind enough to introduce me to the Canadian architect of Q/kdb+, Arthur Whitney. Arthur’s father, Eoin Whitney, had worked closely with fellow Canadian and APL and J language inventor Kenneth J Iverson, one of the stars of this story, and introduced his son Arthur Whitney into the APL family, who would take its principles and architect it for modern computing as Q and kdb+.

Both Stephen and Arthur were more than eager to point me to their fantastic online historical archives, succinctly summarized elsewhere by my friends at The Thalesians, itself a scientific computing and computational finance community powerhouse. A cursory read of any of these sources clearly shows APL’s origins care of Iverson in the fifties and sixties, and its focus on linear algebra using vector processing as its beating heart.

In Contrast, MATLAB, Matrices and Expressiveness

That triggered me, purely because I had given the greatest part of my working life to MathWorks, developers of the MATLAB programming language, though I had never fully engaged with its fascinating history.

MATLAB stands for “Matrix Laboratory” (matrices being n x n rather than the vector’s 1 x n) and it matters to my story because it too does linear algebra, but expressively unlike the terse APL/J/Q languages. It uses powerful matrix programming routines under the hood rather than vectors. Both sets of languages apply singular bulk vector/matrix transformations rather than the sequential loops (“for i = 1 to x”) common to computer science operations in C++, Visual Basic, Java, etc.

The upshot is faster, more efficient numerical computation, extremely fast in Q, and necessarily fast in MATLAB and similar, e.g. NumPy in the Python family. MATLAB draws its inspiration from how engineers and mathematicians think, while APL, J and Q strive for absolute efficiency, as Q devotee, Jeff Borror puts it “serious computation in a single line of code.”

Cleve Moler: The MATLAB Visionary

MATLAB’s founding guru and the second hero of this story, Cleve Moler, is an active, jovial and utterly amiable character. I’ve only met him once. Truth be told, it was a bit of a damp squib because I was accompanied by a loud overbearing stetson-wearing Applications Engineer-come-blogger who commanded the conversation while I stood meekly to one side. Fair play, however, that domineering stetson-wearing lad then became a gambling guru in Las Vegas.

Cleve loved his MATLAB history, and rightly so. Many reminiscences are recorded for posterity as easy-reading blogs and videos on the MathWorks web-site. I do recall I think in 2015, after the film The Imitation Game starring Benedict Cumberbatch as Bletchley Park’s Alan Turing, Moler presented for the very first time at a MathWorks “Winter Celebration,” how MATLAB could trace its origins right back to Alan Turing.

Until then, Cleve’s histories had tended to stop with his graduate supervising professor at Stanford, George Forsythe, undergraduate lecturer, CalTech’s John Todd, and influential visiting British computer scientist Jim Wilkinson.

Wilkinson, it turns out, was actually the link to Turing, as he had helped try to implement Turing’s ACE while at the UK’s National Physical Laboratory and they then worked together at the University of Manchester.

Moler tells the Turing connection well in his blog “Alan Turing and His Connections to MATLAB.” In it he notes how in 1936 Turing wrote “On Computable Numbers, with an Application to the Entscheidungsproblem” in response to the German mathematician David Hilbert’s challenge on whether a problem can neither be proved or disproved. Turing’s solution was a theoretical machine, essentially one that computed maths based on binary states. Turing would then take this theoretical concept, blend it with his experiences at Bletchley Park with the Enigma machine and the cryptoanalysis Colossus machine, and in 1946 present his ambitious design for a stored program machine that he called the ACE, or Automatic Computing Engine. It would be powered by sets of subroutines for different tasks.

Soon after, in 1948, Turing published a paper entitled “Rounding-off Errors in Matrix Processing” where he discussed, among other topics, Gaussian elimination and introduced the term “condition number.” Moler’s CalTech undergraduate Professor John Todd, would expand on condition number theory. while in England Jim Wilkinson would lead the charge to develop the ACE at the National Physical Laboratory and then Manchester University alongside Turing, but whose reputation was being unfairly tarnished due to his homosexuality. Wilkinson implemented arithmetic subroutines, including floating-point arithmetic. Indeed Wilkinson would in later years rework ALGOL processes as subroutines in FORTRAN, which became the heartbeat of early MATLAB.

In short, Turing was well aware of linear algebra, and early computers like the ACE would seek to incorporate it to do useful sums. Using computers to playing games and do word processing could wait until later.

The Mystery: Why No APL in MATLAB Histories?

This was my puzzle. Cleve was active in the 1960s and 1970s, but I had never once heard Cleve reference either APL or Iverson. Meanwhile Arthur Whitney told me that he had never knowingly met Cleve, while neither Cleve nor MATLAB seemingly appear in the APL/J archives. Yet APL was all about linear algebra, built decades earlier than MATLAB. Given this, I wanted to know:

Had MATLAB taken inspiration from APL in any way?
Had Cleve Moler met Ken Iverson?
If Cleve could trace MATLAB back to Turing, could Iverson’s APL be traced to Turing too?

The answer to (1) and (2) is actually very simple, needing just a cursory web search. However, I failed in that endeavour for some months, yet took some fascinating detours and re-discovered some fantastic documents from the dawn of (scientific) computing time. Please don’t consider my amateur work proper historical analysis, but the resources are just dreamy, particularly Moler’s blogs and the JSoftware notes. And you should look at them, because the linear algebra literature is where AI and computational econometrics began, and like me, Moler and Iverson had some interests in both disciplines.

Also, my imagination kicked in. I imagined opinionated men in bow ties, suits and slacks, smoking copious quantities of cigarettes, writing equations on blackboards in beautiful old campuses that look like Barton School in The Holdovers, and arguing opinionatedly with other men in bow ties, suits and slacks. I certainly sensed, and will in this article hypothesize, some possible tensions and cliques, but I will note that it wasn’t exclusively men in the circle. I shall reference two female leaders later, Grace Hopper and Olga Taussky-Todd.

Communities, Culture and Geography: West Coast versus East Coast

My first forays indicated distinct trajectories for both Moler and Iverson, geographical and cultural.

Moler was clearly West Coast (New Mexico, Stamford, CalTech, IBM) via his graduate supervising Professor at Stanford, George Forsythe and undergraduate inspiration, CalTech’s John Todd. Forsythe and Todd were in turn connected with numerous British and other international connections including NPL’s, and then Manchester University’s Jim Wilkinson who had followed Turing and inherited his mantle (Turing had died under inauspicious circumstances in 1954 after his societal troubles)

Moler talks fondly of the Gatlinburg Conferences, annual conferences held in Tennessee in the 1960’s, organized by Forsythe and Wilkinson. Moler was invited to attend the third conference as its first graduate student, which he claims raised eyebrows.

Forsythe and Moler would then co-author the influential “Computer Solution of Linear Algebraic Systems” and a decade later “Computer Methods for Mathematical Computations” based on their lectures with code in FORTRAN, rolling in some of Wilkinson’s work, and ultimately introduce early MATLAB.

Meanwhile, I found Iverson firmly on the East Coast at Harvard and then with IBM working with preeminent computer scientist Howard Aiken on the Harvard 1/ASCC (Automatic Sequence Controlled Calculator) computer, also with the young Grace Hopper, inventor of the legacy language Cobol which still makes the world go round.

I find my initial readings of Aiken, perhaps wrongly, as quite gnarly, and (self-)obsessed. Aiken had three wives, I suspect I know why, and he upset IBM by taking more credit than they thought he was due for the Harvard I/ASCC machine. His community looks to have been tightly focussed around Harvard and on the East Coast. I found no immediate evidence of Aiken nor his student Iverson interacting in any significant way with the likes of Forsythe, Todd and Wilkinson, at least at first.

Iverson’s circle meanwhile, post-Aiken, centered on Canada and the Eastern Seaboard. I have already noted Harvard-educated Canadian Eoin Whitney, then his son Arthur who I corresponded with. Another key APL contributor was fellow Canadian Roger Hui, massive APL acolyte who became Iverson’s collaborator on the array language J. Then I note the corporate impact of I P Sharp Associates. My friend KX Librarian Stephen Taylor had regaled me with tales of Sharp Associates at the Wren coffee house where he had hooked up with Whitney, Hui and possibly Iverson, I don’t recall. I read the company as being possibly the most important digital (though their tech might not have been digital as we know it today!) transformation company you’ve never heard of from a time long before digital transformation was a thing. They seem massively influential behind the scenes putting in computing, networking, messaging, and analytics pipes particularly into electrifying capital markets.

In sum, I saw two distinct computational branches. Mentally I saw the academic Cleve Moler and MATLAB descending from the west coast and British folks via collaborative, global facing academic circles, with Kenneth Iverson was in the East, evangelizing APL to friendlies, and quietly technologically upgrading Wall Street.

But It Turns Out Iverson Did Go West, Briefly

Then, Nirvana! I came across my first gobsmacking source of a connection, a paper from 1952 c/o the National Bureau of Standards tucked away on an academic web-site which directly linked APL’s Iverson to George Forsythe, the same George Forsythe who a decade later would supervise and partner with Moler.

George Forsythe I would find also referenced Iverson in his brilliantly titled 1952 monograph “Solving Linear Algebraic Equations Can Be Interesting.” Indeed, they can.

Back in 1952, then, the two pioneers, George Forsythe who would become Moler’s supervisor a decade later, and Kenneth Iverson were somehow in touch. The paper linked above authored by Iverson was not a co-authored paper, but it was a commentary on Forsythe’s lectures. At least I had found a connection, linear algebra-related too, and a decade before Cleve Moler became Forsythe’s student. Bingo!

Then I found the detail behind the supposition, in some remarkable papers collated at a later date discussing the so-called Institute for Numerical Analysis (INA). This was a fascinating Los Angeles-based offshoot of Washington’s National Bureau of Standards. This archive shows that in 1951 Iverson was a research scholar at the INA, which was led by George Forsythe and John Todd, involving British luminaries like Maurice Wilkes (Cambridge University) and elevating Hal Huskey, an American who had worked with Turing’s ACE community.

The INA was certainly international-facing, with tenured (if that’s the right word!) experts from all over the world, for example:

No Jim Wilkinson, but Wilkinson is held in great respect within the doc, in particular for his work on the ACE. As Hal Huskey notes, this [Western Automatic Computer] “machine was to be built under the direction of the author who had joined Curtiss’s group in January 1948. He had spent the previous year at the National Physical Laboratory in Teddington, England, working under Alan Turing with James Wilkinson and others on the Automatic Computing Engine (ACE) project.”

Some of my highlights and observations from the INA retrospective:

The retrospective phrase “Iverson is now known as the originator of the programming language APL” so his legacy was known and assured when they were compiled.
INA is west coast-centric, albeit international-facing. While an offshoot of the Washington-based National Bureau for Standards, it was concerned with the development of the Western Automatic Computer, with meetings to set up the various protocols and standards including the numerical computation standards.
INA feels academic, yet the Harvard team including Aiken are mostly in absentia (I note the name Joseph Walsh in the table above with a Harvard affiliation).
Indeed, Aiken, Iverson’s mentor, gets one single mention in the document, in conjunction with John Von Neumann, where both form part of an advisory committee suggesting that “the Bureau shouldn’t really work on computers.” I really wonder about their motives with this comment. Is there some professional jealousy at play, or might they have simply seen the INA as a bureaucratic academic talking shop?
The role of the British is credited, since the Western Automatic Computer would follow the stored procedure methodology which came from the Eastern Automatic Computer which in turn took its inspiration from “The Manchester Baby.” Manchester is key. This is where Alan Turing’s career ended, under sad circumstances, and where Wilkinson flourished.

So on one hand, then, we have the SWAC (the Standards Western Automatic Computer), closely affiliated to Turing’s ACE with George Forsythe and John Todd at its heart. On the other hand, we have the seemingly independent Aiken working with a young Iverson at Harvard and IBM on the successors to the ASCC (Automatic Sequence Controlled Calculator) aka Harvard Mark I. Yet this retrospective shows us that Iverson overwintered, or rather oversummered, at the INA in 1951, and would author notes on Forsythe’s lectures. He was there, right at the heart of the West Coast community, and by extension, with followers and associates of Turing.

Interestingly, Cleve Moler in his histories of Forsythe would reference the INA, long defunct by the time he had arrived at Stanford in 1961. “The INA was dissolved in 1953 as collateral damage from political scandals at the Bureau of Standards involving McCarthyism and congressional hearings about battery additives.”

That is just fascinating. My reading of the INA is that it was progressive, international, and, just perhaps, a little stigmatized by Turing’s connections and troubles. My suspicion is that McCarthyism might take a dim view of open international, chaotic, academic, collaboration, but frown less on what I see as the corporate managed discipline of Aiken, IBM, Harvard et al, and therefore Iverson. Von Neuman, mentioned with Aiken in their assessment of the Standards entity, would usurp Turing in the computing literature given the social trajectories of the 1950s. But I speculate, and I will gladly be corrected by those who know more than I.

Another speculation. Moler in his blog teasingly includes a picture (link to the blog again here) showing Forsythe at UCLA in the 1950s, some years before he came on the scene. In it, he names those in the front row of the picture, including George Forsythe, John Todd and Olga Taussky-Todd. I believe this picture was taken at an INA meeting. Might Kenneth Iverson perhaps be in the second row? Answers on a postcard, please!

Answers to My 3 Questions

Questions 1 and 2 — Did Cleve Moler meet Ken Iverson, and did MATLAB take inspiration from APL?

My diatribe, fascinating though it may or may not be, was completely pointless, it turned out, to answering my Questions 1 and 2. A simple web search applied long after discovering the docs above would reveal the supremely obvious “MATLAB and APL: Meeting Cleve Moler,” a blog that followed a hearty 2012 GeorgiaTech Dinner at which Moler was present.

I’ll quote the author verbatim keeping all his links. His story is marvellous.

I have often described MATLAB as “APL with a normal character set,” but I didn’t actually know anything about how MATLAB came to be or if there was any relationship. Last night, I got to ask the man who invented MATLAB, Cleve Moler, at the IEEE Computer Society Awards Dinner, where Cleve was named a “Computer Society Pioneer.” When I introduced myself as coming from Georgia Tech, he took notice. “Georgia Tech is a big MATLAB user!” We teach 1200 Engineering students a semester in MATLAB.

Cleve developed MATLAB (in Fortran) as a Matrix Calculator (explicitly, a “MATrix LABoratory”) for his students. There was no explicit tie to APL, but he saw the connections. He said that he’s always seen MATLAB as “portable APL” because he used a traditional character set.

It’s not just the character set though. “Iverson showed me J. I wanted MATLAB to be understandable by normal people.” He said that someone once converted a program he’d written in MATLAB into APL. “I asked what that was. They told me, ‘That’s your program!’ I couldn’t recognize it.” APL is about being uniform about everything, but MATLAB “is a mishmosh of all kinds of things.”

Others joined in the conversation. “What do you think about Mathematica?” Cleve responded, “Mathematica is APL for the 21st century. Mathematica has a uniformity about it.”

Cleve’s ideas about what make a language usable “by normal people” are interesting. The success of MATLAB in terms of its use by so many people in so many different contexts, domains, and application areas give him real authority for making such claims. He sees a “mishmosh” as being easier for people to understand than uniformity. Marvin Minsky famously said that the brain is likely a “kluge.” Do we actually prefer messy languages, with less uniformity, perhaps as a reflection of our “kluge” nature?

So there we have it! Cleve Moler and Kenneth Iverson did meet. I hope they had smiles on their faces as they exchanged their experiences, though it’s possible the meeting was slightly fractious.

However, my Question 3 is less obvious. Can we really trace Iverson, and thus Q through J and APL, back to Turing and his early work? I believe we can, with Iverson engaged with Forsythe and INA, albeit briefly. I’ve found some possible evidence too of a (later) collaboration of Iverson with Wilkinson, but I need to investigate that further. Aiken certainly influenced Iverson, and through Aiken we see a clear trajectory to Von Neuman who spent time with Turing in the 1930s on both sides of the Atlantic. However, it is the shared love of linear algebra that truly binds Moler, Iverson, Forsythe, Wilkinson, Todd, Huskey and Turing.

Final Comments

As such, I see APL/J/Q as uncles and cousins of MATLAB, but expressive MATLAB is built differently to these terse languages. All serve computational linear algebra but from different standpoints, APL/J/Q was developed for absolute scientific computing efficiency, taken to a corporate world via I P Sharp Associates, while MATLAB served clear understandable code, commensurate with the open collaborations of Forsythe, INA and academics across the world. But Iverson did, briefly, straddle both worlds.

I have not discussed other related languages in any depth from the earlier years, for example ALGOL, Pascal, or FORTRAN. I have elsewhere discussed Niklaus Wirth who did impact and engage with APL, and then influenced Moler and MATLAB. Wirth’s PhD supervisor was INA’s Hal Huskey. There’s much more to scientific computing than just MATLAB and APL, and with the rise of open source huge innovation by new heroes.

On the early histories, much credit goes on the MATLAB side to Cleve Moler and the team at MathWorks, and on the APL, J and Q side to the most magnificent Librarian I ever met (apart from my wife, that is!), Stephen Taylor, also to his friend and the genius who is Arthur Whitney. I am eternally grateful, also, to MathWorks and KX for having employed me!