A lot of people have asked me the question “how to read existing code”, and I hope I have a good way. Some people hope to improve their code quality by reading “good projects” (such as the Linux kernel) to improve.
I generally have a hard time answering this question because I rarely get a boost from reading other people’s codes. I have high standards for the code I read because there is so much badly styled code in the world that reading hurts my mind. By the same token, I don’t read poor-quality styled articles.
But that doesn’t mean I can’t communicate and work with other programmers, I have other options. Rather than reading code, I prefer others to explain their code to me, using simple language or graphics to explain their thoughts. With a thought, I naturally know how to turn it into code, and elegant code. I don’t read a lot of people’s code, but if they tell me, I can accept it.
If a colleague asks me to help him improve the code, I won’t pick up the code and dig into it, because I know that looking at the code is often half the effort, or even completely useless. I’ll have them explain to me on the whiteboard what those codes mean.
My Understanding of How to read someone else’s programming code
My colleagues have found that it takes a lot of work to explain to me clearly. Because my requirements are very high, as long as there is something they don’t understand, I will ask them to explain it again.
Still have to draw, and I will let them iterate on the drawing until I can understand it at a glance. If the graphics are 3D, I’ll ask them to compress them to 2D for me, and then generalize them to 3D after I understand them. I can’t understand complex, high-dimensional concepts, they have to make it easy for me.
So it might take a lot of time to talk to me about the code and tech-related queries, but it’s worth it. Once I understand it, I can often dig out points that others can’t see clearly. Explaining things to me can also improve their own thinking and language skills and help them simplify their thinking.
Many times I didn’t read the code at all, and after explaining it to me, they simplified the code themselves. Saved my brainpower and eyesight and they improved too.
Learn PyTorch’s code
The last time I looked at someone else’s code was at Intel, and we changed PyTorch’s code. It was not a pleasant experience, because while many people find PyTorch easy to use, the code inside it is obscure and difficult to understand. PyTorch is not Intel’s own thing, so no one can tell me.
When modifying PyTorch code to add new features, I find it difficult to see where to change from the code itself. Later I found out that the reason is that a lot of code is automatically generated in the compilation framework of PyTorch, which makes you unable to understand where some codes come from.
For example, they have several file formats of their own design, which contain some special text that determines how to generate code at compile time. You have to understand what the files are saying, and that’s not any known language. These files are read in by some Python script, which spits out some weird C++, CUDA, or Python code.
This is actually a DSL, and I’ve explained the problems with DSLs in previous articles. To add functionality to PyTorch, you need to understand how these scripts handle these DSLs and generate code. And these scripts are also messy and sloppy, so it’s a headache and a headache.
In the end, I found that there is no way to fully rely on the code itself to understand it. So how to solve this problem? Fortunately, there are PyTorch in-house engineers on the web who have written a blog explaining how PyTorch organizes code.
Blog author EZ Yang I met once at a PL academic conference. He was studying at MIT at the time, a very smart guy. However, after reading this blog, I can only initially know what it does and which files should be touched, and these may change every day.
Some Points to Lookup
This blog also mentions that there are historical legacy codes in certain directories. If you don’t know what that is, please don’t touch it! Take a look at those directories, there are some templates that use C language macros to process the generated code, and the way it uses C language macros is not the same as ordinary usage. In my opinion, the so-called “macro” (macro) and “metaprogramming” (metaprogramming) themselves are huge misunderstandings, and PyTorch’s use of macros is so strange and clever.
Do you think you can understand PyTorch code after reading this blog? No, there are still all kinds of bumps in the wall every day. A lot of experience comes from tossing and hitting a wall. Multiple people do these things at the same time and then share their experiences.
Discussions are often: “I found out that to do this, you have to add this to this file, then add that to that file… and then it seems to work.” At the next meeting, someone said, “I found out that it’s not like you said. , you have to change here and here, and there is no the key…
” Many people who know it and don’t know why are blind, because “why” has been covered up by PyTorch authors under a pile of confusing DSLs.
My Experience In PyTorch code
So what have I learned from PyTorch code? Nothing at all. I only see all kinds of software development misunderstandings repeated over and over again. Had they gotten my advice early on, it would have been impossible to organize the code like this, with so much macro processing, code generation, and DSL. Deep learning frameworks such as PyTorch are essentially interpreters for some simple programming languages, except that functions written in these languages can be derived.
Many people don’t know that one day I wrote a “deep learning framework” with less than a hundred lines of Scheme code, which is actually a small programming language. While there is no performance to speak of, no GPU acceleration, and imperfect functionality, it captures the essence of larger frameworks like PyTorch—functions written in this language can be automatically differentiated.
This kind of insight is the most critical thing. As long as the key is grasped, the details can be figured out when needed. Dozens of lines of code pondering over and over can often help you see through the secrets hidden in projects with millions of lines.
What’s People’s Misconception
Many people think that looking at large projects can improve themselves, but they do not see that large projects are just extensions of dozens of lines of core code, and many parts are low-level repetitions. Dozens of lines of mediocre or even obscure code, repeated 10,000 times, become hundreds of thousands of lines. Looking at the low-level repetitive parts, you won’t get much improvement.
It is not the large-scale projects with millions of lines that make my programming ability and insight today, but the exercises as small as a few lines and dozens of lines. Don’t underestimate these shortcodes, they are the essence of programming.
Repeatedly ponder these shortcodes, constantly improve and refine the structure inside, and sharpen your own thinking. Gradually, your level of awareness goes beyond these million-line, headache-inducing projects.
So how do I read someone else’s code? Don’t. If I can, I let the author of the code tell me, instead of reading it. If the author isn’t cooperating and I really want to use the code from that project, I’ll mess with it. So how to toss other people’s code? I have another way.