What Can the NSA Teach Us About Debugging?

William F. Friedman wrote training materials teaching would-be cryptographers how to break encrypted messages, revealing a powerful cross-domain synthesis opportunity for modern software developers. This chapter explores how approaches developed for one highly specialized domain (cryptanalysis) offer profound insights when applied to an apparently unrelated field (software debugging).

By recognizing the shared patterns between code-breaking and bug-hunting, both requiring persistence, careful analysis, and intuitive leaps, we gain problem-solving frameworks unavailable to those who remain within traditional software engineering boundaries. This deliberate application of patterns across domains exemplifies cross-domain synthesis at its most practical.

Both cryptanalysts and debuggers face problems with incomplete information, patterns hidden in complexity, and solutions that require systematic yet creative approaches. Friedman's insights from the early days of the NSA provide not just historical interest but actionable approaches for modern technical challenges, demonstrating how cross-domain thinking transforms seemingly disparate fields into sources of mutual illumination.

Digital Computers

Digital computers were created to help with decrypting enemy communications. Once I realized this, it made sense to me that cryptanalysts might have useful insight into how computers work, specifically, insight into problem analysis and debugging. Indeed, they do!

William F. Friedman (1891-1969) was the founding cryptanalyst of the National Security Agency (NSA). A cryptanalyst specializes in making and breaking codes, ciphers, and other methods of secret communication. We will focus on what Friedman had to say, even though the NSA was not founded until 1952.

Context is important. That means we must, of necessity, take a quick dive into Friedman's world of codes and ciphers. We will be stepping back to the 1930s. Friedman wrote a series of books titled *Advanced Military Cryptography*. The series was so significant that it was rewritten several times and, in the 1950s, gained a second author and a new title.

Figure 1, "Military Cryptanalytics Encoded Message," shows the new edition, now titled *Military Cryptanalytics*, by Friedman and Lambros D. Callimahos. The handwritten dedication from Callimahos to Friedman includes the encoded message "Bacon did not write this work."¹

Bacon? What did this book have to do with Bacon?

Friedman married Elizebeth Smith (1892-1980) in 1917. The Friedmans jointly taught the U.S. Army's cryptographic school from 1917-1918. That is an interesting accomplishment for the first year of marriage! That school's graduation photo (Figure 2, "Knowledge is Power") carries an encrypted message. Note that the caption "KNOWLEDGE IS POWE" is cryptographically correct. The graduation class did not have enough students to encode the "R" of "POWER".²

This photo is encoded using the Bacon cipher, with each student either facing the camera or facing away from the camera. Elizebeth is seated at the center of the photo; William is sitting at the far right, looking left.

The encoded message reads, "Knowledge is Power". Cabinet Magazine explains:³

By facing either forward or sideways, the soldiers formed a coded phrase utilizing Francis Bacon's motto, "Knowledge is power," but there were insufficient people to complete the "r", and the [first] "w" was compromised by one soldier looking the wrong way...

By the time he retired from the National Security Agency in 1955, Friedman... had arguably become the most important code-breaker in modern history.

Colonel William F. Friedman kept a very special photograph under the glass plate that covered his desk... Friedman found it so significant that he had a second, larger copy framed for the wall of his study. When he looked at the oblong image, taken in Aurora, Illinois, on a winter's day in 1918, what did Friedman see? He saw seventy-one officers, soon to be sent to the war in France, for whom he had designed a crash course on the theory and practice of cryptology.

This brings out a key skill with debugging: when you see something odd, follow through to the answer if you can. You will become better and better at spotting the answers when debugging a situation.

By the 1930s, with Friedman established as the U.S. Government's chief cryptographer, he continued teaching would-be military cryptanalysts. Let us take a look at what he had to say. Those lessons apply to us today.

With Advanced Military Cryptography, 1935 Edition Friedman began by comparing ciphers (i.e., the encryption of text, including very-long messages) to codebooks.⁴ A codebook is a dictionary or a lookup table that allows you to look up concepts or phrases, and substitute the codebook's code word for the phrase. The codebook was literally a book. Figure 3, "1899 Codebook," shows a page from the 1899 U.S. State Department code book.⁵

By this point, you will be wondering what 19th Century codebooks have to do with debugging modern software. That is an insightful question. The answer is that, with this type of analysis, it is crucial to understand the problem and its context in-depth. We are exploring. Do bear with me; there is a reason.

The codebook idea is similar to how 18th and 19th Century navies used signal flags (Figure 4, "Navy Signal Flags"). Signal midshipmen used a codebook to translate phrases into the appropriate flag hoist.⁶

The key feature here is the codebook's limited vocabulary. If your word or phrase is not in the codebook, you cannot use the word. (There were workarounds, but the workarounds are not germane here.)

Friedman, in 1935, compared the current (1935) situation to the situation as it had been during the Great War (World War I). Armies during the Great War did not use codebooks. They were too limited. They used military ciphers.

It is necessary to add that viewpoints are always undergoing change; what is regarded as wholly impracticable today may, through some unforeseen improvement in technique, become feasible tomorrow, and it is unwise to condemn a system too hastily. For example, before the World War, and indeed for the first two years of that conflict, the use of codebooks in the theater of operations was regarded as wholly impracticable. Colonel Hitt, in his Manual for the Solution of Military Ciphers, published in 1916, says:

"The necessity for exact expression of ideas practically excludes the use of codes for military work, although it is possible that a special tactical code might be useful for preparation of tactical orders."

Also, in the official British Army Manual of Cryptography prepared in 1914 is found:

"Codes will first be considered, but as they do not fulfill the conditions required of a means of secret communication in the field, they need not be dealt with here at length."

As of 1935, however, the pendulum had swung in the other direction. Most major nations' armies now used codebooks for protecting communication.

Friedman then proposed that, with the invention of digital computers (but he did not call them that), the prohibitively expensive calculations would likely become practical:

It need only be pointed out in this connection that today code methods predominate in the secret communication systems of the military, naval, and diplomatic services of practically all the large nations of the world. Nevertheless, it is likely that within the next decade or two the pendulum may once more swing over to the other position and cipher methods may again come to the fore, especially if mechanical and electrical cipher machines are perfected so that their operation becomes practicable for general use.

It is for this reason, if for no other, that the cryptographer who desires to keep abreast of progress must devote considerable attention to the more complicated cipher methods of the past and present time, for with the introduction of mechanical and electrical devices the complexities and difficulties of these hand-operated methods may be eliminated.

Friedman, therefore, was setting out to teach forms of military cryptography that might not appear of immediate use. He was, rather, teaching sound principles without regard to whether they immediately applied. He warned:

Consequently, if among the methods to be set forth herein certain ones appear to the student to fall outside the realm of what is today considered practicable, it should be remembered that the purpose in describing them is to present for his consideration various basic cryptographic principles, and not to set forth methods that may with a high degree of probability be encountered in military cryptography in the immediate future.

Friedman, as we all now know, was correct. The pendulum, so far as typical web software is concerned, has swung the other way. We don't use codebooks. Public-key cryptography, password hashing, certificates, SSL, etc., all use ciphers rather than codes and codebooks.

Intuition

For me, intuition comes to the forefront when it comes to debugging difficult problems. Friedman agrees. Not only does he agree, but he wrote down some useful insight. But to understand his insights, we needed the context. That is why we took this tour.

Knowledge is power.

We now know that when Friedman wrote Military Cryptanalysis, Part I—Monoalphabetic Substitution Systems in 1936, he had something similar to say.⁷

Military Cryptanalysis exposed a concept that I was handling implicitly. Friedman declaimed (page 4):

The fact that the scientific investigator works 50 per cent of his time by non-rational means is, it seems, quite insufficiently recognized. There is without the least doubt an instinct for research, and often the most successful investigators of nature are quite unable to give an account of their reasons for doing such and such an experiment, or for placing side by side two apparently unrelated facts.

When I am trying to solve a difficult software failure, I rely on intuition far more than I rely on specific debugging tools. The more I know about the problem context, the stresses on the system in question, and the surrounding events, the more likely I am to figure out what happened. All of these things feed my intuition.

Friedman also explained (page 3):

An active imagination, or perhaps what Hitt and other writers call intuition, is essential, but mere imagination uncontrolled by a judicious spirit will more often be a hindrance than a help. In practical cryptanalysis the imaginative or intuitive faculties must, in other words, be guided by good judgment, by practical experience, and by as thorough a knowledge of the general situation or extraneous circumstances that led to the sending of the cryptogram as is possible to obtain.

Friedman's advice, in my view, applies equally well to debugging and problem analysis. That's because it is crucial to understand the overall context and to consider the system as a whole. Friedman takes us a step further:

In this respect the many cryptograms exchanged between correspondents whose identities and general affairs, commercial, social, or political, are known are far more readily solved than are isolated cryptograms exchanged between unknown correspondents, dealing with unknown subjects. It is obvious that in the former case there are good data upon which the intuitive powers of the cryptanalyst can be brought to bear, whereas in the latter case no such data are available. Consequently, in the absence of such data, no matter how good the imagination and intuition of the cryptanalyst, these powers are of no particular service to him.

Eric Evans, in Domain-Driven Design: Tackling Complexity in the Heart of Software calls that "knowledge crunching". In Chapter 13, "Refactoring Toward Deeper Insight", Evans explains:

There are three things you have to focus on.

1. Live in the domain.
2. Keep looking at things in a different way.
3. Maintain an unbroken dialog with domain experts. Seeking insight into the domain creates a broader context for the process of refactoring.

Evans explains that we need deep insight into the business itself before we are able to undertake transformational refactoring. Superficial knowledge of the business brings superficial results from modeling and refactoring the software.

The same, I find, is true with issue analysis and resolution. When the problems are difficult, it is the deeper understanding, empathy, if you will, that enables you to find the right answer.

When the breakthrough finally happens:

Intuition, like a flash of lightning, lasts only for a second. It generally comes when one is tormented by a difficult decipherment and when one reviews in his mind the fruitless experiments already tried. Suddenly the light breaks and one finds after a few minutes what previous days of labor were unable to reveal.

However,

Unfortunately, there is no way in which the intuition may be summoned at will, when it is most needed.

Again, in my view, Friedman's insight directly applies to computer problem analysis and debugging. Perhaps this is because computers were designed and invented for cryptanalysis. Thus, it should not surprise us that problem-solving in cryptology sheds light on problem-solving in computer science!

Even if intuition cannot be summoned at will or when most needed, can intuition be taught as a skill? I am not sure, but we can certainly *demonstrate* those flashes of insight when they happen. Friedman's advice applies to developing our own skills in problem analysis and debugging.

How do you find and fix a problem? Certainly many known issues, challenges, or difficulties have known solutions. The more solutions you already know, the quicker you can pass from the known to the unknown.

At this point, Friedman explains, the work begins (page 2):

Success in dealing with unknown ciphers is measured by these four things in the order named.

1. Perseverence
2. Careful methods of analysis
3. Intuition
4. Luck

Cipher work will have little permanent attraction for one who expects results at once, without labor, for there is a vast amount of purely routine labor... before the message begins to appear.

Finally, Friedman warns his students:

Often, indeed, the student will not even know whether he is on the right track until he has performed a large amount of preliminary "spade work" involving many hours of labor. Thus, without at least a willingness to pursue a fair amount of theoretical study, and a more than average amount of patience and perseverance, little skill and experience can be gained in the rather difficult art of cryptanalysis.

Superpowers

The online First Round Review has a useful article, "How to Spot and Magnify the Powers of Your Engineering Superheroes." The article comes from the perspective of tech interviews, but the interview objective is to draw out and identify the candidate's "superpower".⁸

Friedman has been describing the person we might now call "Aquaman."

Aquaman's superpower is diving deeply. This engineer is driven by solving big problems. He may not write any code for weeks on end, but will continue to dive through layers of the API to find and tackle a challenge. This superhero is nimble and acrobatic enough to penetrate operating system, database and controller layers to dig for a bug. His rare ability is being able to understand the code of each of these levels well enough to know what's happening.

Wait! No code for weeks on end? Is that allowed? What happened to the Agile project sprint? And the last sprint, and the one before that? Aquaman must be deployed with care!

"Just so," the article notes:

Early on, we had a bug in the SSL layer because we're built on top of JRuby (on the Java VM). Our Aquaman spent a month hunting the bug through JRuby, and then throughout the Java VM and down into the SSL networking layer. If I didn't have that guy who could hold his breath and had the stamina to dive to the bottom and get that SSL bug, we'd be in trouble. [Our product] would freeze occasionally because of it, and we wouldn't stay in business long with that happening. He persisted, found the bug and fixed it.

For an "Aquaman"," it's the tough ones that keep the boredom away.

However, Friedman has one more lesson for us regarding keeping that boredom away. The lesson is not a good one.

By the end of 1940, Friedman's team had created an exact analog of the Japanese PURPLE cipher machine without ever having seen one. In 1941, Friedman was hospitalized with a "nervous breakdown," widely attributed to the mental strain of his work on PURPLE.

Hang on to your work/life balance. That is important.

Summary

William F. Friedman's cryptanalytic principles demonstrate how powerful cross-domain synthesis can transform technical problem-solving. By applying frameworks from cryptanalysis to software debugging, we gain a structured approach that transcends conventional debugging techniques. This transfer between domains is effective precisely because both fields share fundamental patterns despite their surface differences.

Friedman's success factors (perseverance, careful methods, intuition, and luck) create a mental model applicable across technical domains. This cross-domain application works because the underlying pattern recognition challenges in breaking codes and fixing bugs share deep structural similarities. Both require systematic exploration guided by informed intuition, both involve searching for anomalies in complex patterns, and both demand persistence through tedious investigation.

Particularly significant is Friedman's insight that intuition must be guided by judgment, experience, and thorough knowledge of context. This recognition, that technical problem-solving requires both analytical rigor and creative leaps, applies equally in cryptanalysis, debugging, and other complex technical domains.

---

1. Friedman, William F., and Lambros D. Callimahos. "Military Cryptanalysis Part I," 1956. Image of public domain document. ↩
2. Pre-1925 Public Domain photo from William F. Friedman Collection at the George C. Marshall Foundation in Lexington, VA. ↩
3. Sherman, William H. "How to Make Anything Signify Anything | William H. Sherman." Accessed December 27, 2024. ↩
4. "Advanced Military Cryptography," 1935. ↩
5. Licensed creative commons. 1899 Code Book ↩
6. Public domain image. First recognition of the American Flag by a foreign government ↩
7. Friedman, William F. "Military Cryptanalysis: Part I -- Monoalphabetic Substitution Systems," 1936. ↩
8. First Round Review. "How to Spot and Magnify the Powers of Your Engineering Superheroes," December 14, 2015. ↩