Lately I write demos under thorn with a todo list using C++ under a BriarPig mu-babel license. The run and hex demos were done on 13apr2008; crc on 14apr2008; buf on 20apr2008; in on 27apr2008; ctype on 04may2008; out on 18may2008; slice on 25may2008; quote on 31may2008; escape on 31may2008; mutex on 14jun2008; rand on 16jun2008; stat on 17jun2008; primes on 19jun2008; list on 23jun2008; heap on 29jun2008; iter on 02jul2008 and 04jul2008; atomic on 06jul2008; node on 13jul2008; page on 23jul2008; hash on 27jul2008; book on 27jul2008; pile on 03aug2008; stack on 07aug2008; mu on 12aug2008; toy on 16aug2008; weight on 23aug2008; this menu links all demos:
mu: toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight, jar, card, harp, debug, profile
thorn: todo, names, iovec, assert, log, run, hex, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, list, iter, ctype
symbol boxes ¶
On box I finished material on hash boxes and symbol boxes. This might be the most interesting part of that page, unless I add high level generalities later.
The following excerpts dialog from the symbol section.
"It's actually kinda cool symbols are also C strings," Stu said.
"I guess so," Wil agreed. "It should improve interoperation with C as long as bytes are kept immutable."
"What's the maximum length of a symbol?" Zé asked. "Is it smaller than any other string?"
"Darn it, I avoided discussing size limits so far," Wil mused. "Um, I think the parser imposes a limit of about 1024 bytes in one symbol name before it's split by a read buffer."
"But outside the reader, there's no limit except what will fit inside one book instance?" Zé guessed.
"Basically yes, and the tag's length field is only 16 bits," Wil noted. "If a book is bigger than 64K, then a tag's length field is the limit. You'll ask about string limits now, right?"
"Hey, yeah," Stu chimed in. "How big can a string get? Is it limited to one book in size? 64K currently?"
"As a single contiguous vector of octets, yes," Wil said. "But strings will also use discontiguous iovec formats."
"So strings as discontiguous arrays composed of fragments can be as big as I want?" Zé asked.
"Yes, but that's a slightly involved feature," Wil warned. "It won't be ready until I code it. Not complex though."
lacuna » (stories)
box pairs ¶
Now box describes current basic pair format and code. I added a new length method coping with cycle detection, because it seemed lame to just leave the first simple draft there without further comment about doing a better job.
There's not much point in folding a lot of code into the struct for pairs. So what's there is light, with just enough detail to show intended usage and semantics.
I had a vision of something Zé would do to Dex by page's end — something odd just to be amusing. I'll add it last.
tiny update ¶
Slightly more dialog appears on gc box formats. I spent more time in the gym tonight than writing. I usually workout on nights I don't have my sons.
slower exercise » (stories)
social gyms » (stories)
trash bin ¶
I cut a couple long pieces here after I wrote them. I enjoyed writing them, but they were too personal. (It doesn't take much at all to be too personal.) Too bad I didn't spend that time writing more language docs.
design box ¶
Use of character Zé appears on design and box formats with start of descriptive dialogs appear on box. I would put off posting either so soon, since there's so much left to write about boxes. But so far it's a bit interesting. I'm always surprised when dialog engages more than I expected.
It's not too hard to see I write dialogs linearly, all at one time, mostly in one draft, without much editing. Basically the idea is: characters don't have time to be perfect, they just say whatever comes next. Whatever undone parts remain are characteristic of their personalities. I pick up where I left off when I break overnight or several days.
box editing ¶
I'm editing api for box formats, but it's still too raw to post as-is. Pending material includes a lot of box formats not yet used because I haven't yet reached parts of the toy language that would use them.
As I write pages about a toy language, I'll eventually reach the spot where I stopped working on it last fall; then writing web pages will occur concurrently with writing code. (This is likely to be slightly messy, but who cares?)
I stopped working on it last fall because my day job was getting so involved and absorbing that I couldn't keep context for night time hobby coding in mind. The demand on memory was too great to keep everything "in cache" and I didn't have time to fault in hobby code memory fast enough.
That's one of the reasons I started writing a lot about past code infrastructure, even though it basically took the place of coding any fun new stuff for a long time. Now that I've written a lot down, I can read material here to quickly remind myself of hundreds — actually thousands — of little details that normally stay in mind continuously if not for the force of day job mental traffic flushing my cache too vigorously.
not your tool ¶
The Wil character I use in dialogs expresses himself far more dramatically than I would, but we do have attitudes in common. Just the same, don't confuse my actual beliefs with whatever words I put in Wil's mouth. He's easy to write because he doesn't consider very many complex tradeoffs — he just says: keep your distance. That's a simple perspective.
His use of profane language might make you fear he'll do so again in response to things you say. He's just uncouth enough to make you afraid an ugly, distasteful rant might erupt when you try to apply pressure. That's intentional: I don't want to participate in online conversations much. Having you worry I might express myself the same way suits me fine.
What Wil said so rudely in a recent dialog conveys an important idea I want you to keep in mind: I'm not your tool. Specifically, I'm not a resource you can use. Any and all attempts at manipulation are extremely rude, even if you think you're being subtle. Let's face it: you're not subtle any time what you'd like to happen is even slightly apparent. Invoking "social good" is one of the most blatant, sophomoric forms of persuasion ever — what makes you the arbiter of social good? Nothing! You only represent yourself. Even hinting you have some direct line on social values — with associated group morality — is one of the ugliest forms of intellectual dishonesty. It puts you in company with all sorts of unsavory folk. I'd invite you to feel ashamed by it, but I suspect you already have a handy defense against seeing yourself critically.
Coercion is disgusting, no matter what your motive, if it doesn't involve protecting others from abuse of force. Bullying others in pursuit of your interests and your cronies is not service to society, even if you dress up your personal preference as universal neighborhood improvement.
small update ¶
Just a small update on weight today, addressing an assortment of topics in the left column.
thread popularity ¶
Over the last several months, one the most popular demos under thorn is just a stub page with nothing on it yet: the thread demo, as yet unwritten. By a small margin, it typically has hit counts exceeding every page I finish, and then link with an intro. So I gather threads are a topic everyone wants to know more about. Or perhaps if I have anything novel to say, that's the topic where folks want to see it.
Maybe one thread fan just keeps pinging it like mad.
Anyway, I thought I should say slightly more about the content to appear there later: it's just a pthread wrapper api in C++. It's trivial. It won't tell you anything you don't already know from the Posix pthread api. It's just a way to turn a C function interface into an object-based interface.
The thread demo won't be interesting when it appears. I'd have done it already if there was anything intrinsically fun, but there isn't. Mainly it defines a particular method you override which runs whatever you want done by a thread, typically in the form of a loop — you keep looping until you see an indication you should stop. A thread driven by a request queue, like one in a thread pool, typically blocks reading a thread-safe queue each time through the loop.
I suspect folks want to find out something else. Perhaps other ways to do threads? For example, maybe folks want to know how to do green threads with cooperative scheduling. That's not going to appear in that demo; it's just the pthread api for native threads. Let me think. Why does it draw attention?
At my current day job, the last rev of the system was threaded and had somewhat interesting dataflow between worker threads and the event thread. New folks would read the C++ code involved and be puzzled by absence of clear meaning: how did that code have the effect it did in the system? And what was that effect anyway? So I explained the sequence of events in control flow involved, and which changes in state told other threads what, and in which order. At least one very wonky thing happened along the way.
Nothing about the thread class explained this. And nothing about the thread class I publish will explain anything like that either. The weird stuff is in applications of message passing that interacts with threads, scheduling, and blocking semantics.
The thread class won't clarify any of that. Oh, I guess when I write the demo, I should really add a sample subclass and a unit test that launches a couple dozen threads to perform some monte carlo simulation test. I could show blocking queue use with thread pools, and the odd things you do for joining at the end after waiting on pthread barriers so a test stops cleanly. (But in normal use, other folks never bother to make threads stop running — they just go forever.)
But that won't tell you about other different weird thread usage situations. What is it exactly you want to find out, anyway? All the interesting parts are in a mental model you have to hold, which lets you anticipate what can happen concurrently. The api won't help you there.
traffic spike ¶
I had a large spike of "Code 302 Redirected Request" traffic, I presume caused by the zombie story — which I felt slightly edgy about today since screwing around with the gorilla doesn't seem very smart even if it was just a joke.
profane dialog ¶
A new dialog appears on weight, containing more profanity than I've used on this site before. But I think it works in context; it carefully reads more like fiction, in which folks are supposed to get upset once in a while.
peg api ¶
I added just the class api to peg without any plain english commentary. But it sorely needs explanation, so don't expect to enjoy it much before docs.
Much of the yp peg api actually belongs on imm, because most of peg semantics address encoding of immediate values. I still need to decide how to divide parts in two, and which parts to duplicate on both pages. If I keep much immediate api on peg, most of the explanation will appear on imm, with peg focusing on the view outward toward uses of pegs.
microsoft zombies ¶
This is more a mean-spirited joke than Microsoft bashing; they were just easy to hand when I had the idea.
At lunch we were telling stories about system installations and Raj (a pseudonym, not his real name) mentioned some Vista experiences he seemed to feel ambiguous about. When we left cafeteria, I happened to be walking next to him, so I tried out a little joke I'd been cooking.
I guess the idea popped into my head shortly after someone suggested I install Remote Desktop on my Mac at home, so I could log into my Windows desktop from home, which I use mainly to VNC into Linux desktops. Anyway, the idea of installing Microsoft network software on my Mac gave me a bad feeling, like it would be very trusting of me to do that.
So as we left the building, I cleared my throat and got Raj's attention. "You know," I began casually, "I think they should come out with a new product called Microsoft Zombies™."
Raj tossed back his head and started laughing. Now wait a minute, I thought, I only just started: wait for a punchline. My god he was a good audience. I think he was already confabulating my joke, like he was way ahead of me.
"After you install it," I continued, "you hear some moaning, shuffling, and plaintive demands for ... Braaiins!"
Raj nodded enthusiastically, like he could finish the joke for me. He gave a little zombie shuffle and deep throated "Uhnnn" like a Frankenstein monster.
"Then of course the problem is trying to kill it," I followed through, sending Raj on an extemporaneous riff on killing undead and having them keep coming.
I almost told this story in fictional form, with Zé calling Wil on the phone late at night, begging for help after he explains he installed Microsoft Zombies™ to see what would happen. Wil would hear what sounds like the sound track of a zombie movie in the background, while trying to decide whether Zé is pulling his leg, despite the panicky sound in his voice. I think that would have been much funnier, and oddly effective at making both Zé and Wil slightly more real.
tag api ¶
I added just the class api to tag without any plain english commentary, which will follow soon but not necessarily next since I'd like to push peg and box further too.
You can amuse yourself by guessing what I'll say in code description, and you'll come rather close if you're used to deciphering code without comments. What you see there now is what you'd have to work with if I didn't write any docs.
That's about an hour's worth of editing because I do markup by hand. Recent demos made better progress when I just cranked through code as quickly as possible before writing explanations, because switching back and forth took longer. It's assembly line writing for one person.
Many lines had to be broken into two or more lines to get code under 64 columns. So it now uses more vertical space than the original did. I shoot for 80 columns normally.
path finding ¶
I'm editing code for pegs, boxes, and tags while trying out ideas to describe them. While reducing code lines to 64 columns, I think about how hard it will be to elucidate what I see.
It's a mixed bag. Some parts will be unusually clear (compared to what it might be) and other parts will be almost line noise quality nonsense — the code that is. When I wrote the last draft of pegs and tags, I didn't care even slightly what it looked like, and merely choose short clear names.
Anyway, I suspect how I encode tags and immediate values might disappoint on first sight, because it'll just be bit values with names that go into certain locations. What bits mean depends on how they get interpreted by code using them. And context for code using these bits might not gel quickly.
I know I said parts resembled New Jersey ML runtime code from the early 90's, but it won't be apparent beyond slight similarity in location of tags and some bitfields.
When I get to boxes, I'll include a lot of box formats that aren't yet used, because having them around helps me think about connecting them later. When I say a box is this sort, don't get hung up on the members in such structs because they're subject to change. Early drafts suggest ideas.
property ¶
I plan to add a new section to weight on the role of property as pro and con in motivation. Here's a preview in blog mode, differing slightly from how I write other pages, where I assume attention is very divided. Here I assume more alertness. (Insert "hey you in the back, wakeup" joke here.)
I work for a company in California, and writing a language in my spare time as a hobby is in no way related to my job — it doesn't intersect with business interests of my company, and therefore no claim can be made against what I do on my time, using only my own resources, in this situation.
In other states, they can get away with owning everything you think, even on your own time. But not in California, if it's not related to an employer's business interests. This is one reason I'm working on a programming language: they're a dime a dozen and no one wants a new one. So I'll be left alone.
What everyone really wants is a database, always. Which is why I'm not working on one. I learned my lesson there. Making something seen as valuable in your spare time is a recipe for slowly brewing trouble. Every time I describe a database I once wrote, folks at each job say "that's perfect" and suggest maybe I should work on that more. But my response is always "no freaking way" phrased in the most polite terms possible. No, really, I just couldn't. Shucks.
If I ever added a database to what I'm doing, I wouldn't call it a database. I'd say something else. You know, um, here's a page cache and here's a btree library and here's, blah, blah, blah, but I have no idea why anyone would ever put these together. Yeah, there's some i/o and caching and serialization and stuff, but that's not a database, is it? Of course not. And that transaction stuff? I have no idea how that got in there.
Okay, I'm kidding a little. I'm not that disingenuous. I won't say anything at all. If btrees show up some day, I won't have anything to say about it besides demos. But I'll tie it to language stuff, rather than making it standalone. If it was standalone, someone might mistake it for a database and bother me. Might not happen for a couple years, though, because I'm busy writing a programming language and not a database.
This means I don't want to work for any company with a business in programming languages, because then it would be hard to defend my hobby work at home as unrelated to business interests. I don't even want to work for a company that has a strong opinion about languages. Yes, that tends to rule out a lot of fun things.
I very nearly got burned at Netscape, after AOL bought the place and handed everyone new IP agreements to sign, which included standard "all your base are belong to us" clauses from out of state. (It went to so far as actually stating anything you thought was the property of AOL. It rankled.)
I went to management and said it wouldn't do. They laughed and said, ha ha, that was no problem because it was unenforceable in California. I said, um, it's enforceable when what you do at home in hobby work is related to business interests. They said, no problem, because that won't happen, will it? And I replied, well it already happened because you just assigned me the task of writing a new database, and making a database is exactly what I do at home as hobby work. You assigned me that task at work because it's my hobby and I know all about it — it's not an accident. And they said, Oh.
Then they asked, do you mind? You didn't want to keep that as yours, did you? What's a little IP among friends?
I replied, yes I freaking mind. Make it right or I'm gone.
So they said, okay, and set me up with the lawyers to get official, public, publishable permission to keep putting my stuff in the public domain, both database and language work, since I was public domain oriented at the time. And just getting that done was a royal pain in the ass because drafts of documents kept getting small comically inappropriate phrases added to them, such as requiring that I pass everything I wanted to post publically through a permission process that would allow me to be stonewalled by ignoring me. I made them remove them all, but you know how hard it is to get a lawyer to remove a gotcha they put in there on purpose? I said never again after that.
So I don't have conversations with employers about IP — I just stay clear. I only work on things no one wants like, ugh, Lisp. So sorry, yes it does have to be a syntax you hate. It's what I like. Yes, I know some languages are now all the rage and you like those. No, I can't do one of those just now.
So I have a motive to anti-hype things. Any time what I'm doing sounds interesting to someone who smells money, I have a problem. I almost never mention my home hobby code at work. They only know I have some bee in my bonnet about development environments, scripting, and whatnot — the usual useless drivel so many geeks like. That's how I want it to stay: I'm doing something unpopular no one will ever want to use — as far as I know of course.
cinder blocks ¶
I wrote very little today: toy links short intro content on peg and weight. I suppose I'll document more code first before writing generalities. While interesting, high level descriptions trip my bullshit detector when it's not mixed with code.
fitness ¶
Some weeks I spend almost as much time in the gym as I do writing for this site. This was almost a week like that. (Of course, I don't have time for a social life — but then I don't have one. What? You can afford friends? Show-off.)
During my week off, a trainer talked me into an hour's complimentary personal training since I hadn't sampled it in years. He asked me what I wanted to focus on. I said, "I want to learn back exercises; back's hard to excercise."
"What are your goals?" he asked with warm sympathy. I guess this is part of the selling regime. What are your goals? Lose a zillion pounds? We can do that.
"I just want a stronger back," I replied, trying to kill an urge to laugh. I'm about where I want to be — I look like a gym rat already — I just don't have any back tone. You know there's something wrong when you bench press so many pounds, but vacuuming a floor makes your lower back sore.
The trainer made an interesting facial expression. (It's one I've seen a lot actually — it's says "you're eccentric.") But he was game if that's what I wanted. Maybe most guys say, "I want to look like Superman — how much does that cost?" Or something like that. Hey, you know, I'm too old to be into body building vanity. I just want a vacuum proof back.
So he ran me through a series of useful exercises all targeting my back, some using weights and some on the floor using a big rubber ball a yard in diameter, upon which you place your elbows and try to keep your balance while kicking the ball with alternate knees — which is hard, and effective.
One of the exercises was perfect — my entire lower back achieved that same ache as cleaning the entire house. Super. All I have to do is repeat this a zillion times.
Most of the exercises with weights required I keep increasing the load because I'm, uh, really strong. One with free weights I can do with eighty pounds no problem — about like if you picked a box up off the floor, but using a barbell and safely staying away from bending to the floor. Basically pulling it from your knees to your belly button and holding it there while half bent over. Repeat until exhausted.
Anyway, I added all that to my routine, and I've gone two out of three days since then. The back exercise has somehow increased my metabolism a lot — very noticeably. My workout is a lot longer when I have time: about two hours today. Not enough aerobics, though. Need to work on that.
toy index ¶
I wrote an index page for the toy language with intros for topics on each page. The updates section will list changes to toy language pages. Note today they're all empty except for mu and toy. It's probably going to take weeks before it even starts to look a little fleshed out. It could take until next summer before enough is done for you to get excited about, unless you like ideas without executable code for every bit mentioned.
The toy page also has a few random, interesting, high level, conceptual sections about generalities that won't be covered on other pages. The part on the value of toy languages is good; the section on maps and morphisms might give you ideas; and a piece on viewpoint context might help reduce how tedious the Wil character seems, when you can imagine your options.
javascript ¶
I have not been commenting on JavaScript when I use phrases like "design-by-committee languages" — that's merely true of most languages. It was just a coincidence JavaScript folks had something to announce the same time I posted new stuff on a toy language. They're not related.
I'm only vaguely interested in JavaScript, because I assume I'll later be translating code in other languages into JavaScript when I map a framework so the client side might work in a browser using JavaScript. However, that idea won't in any way affect my plans, thinking, or strategy in doing any part of Lathe. I'll just assume JavaScript specific work won't be hard, and won't be materially different from similar effort targeting other languages.
Several months ago a recruiter called me up and asked how much I knew about JavaScript, after identifying herself as staffing for a well-known Silicon Valley consumer electronics company. (Gee, I wonder how many of those there are?) I told her I didn't know much really, and that was that. I haven't been looking for new opportunities and it's easy to hear in my voice — recruiters like folks who sound hungry.
(My current day job isn't very interesting, and since they know this, they've been taking care to provide incentive to stay. Walking away from above market incentive is tough. At least I think it's above market, but how would I know?)
data structures ¶
I'm basically a data structures guy. I do things to code and data graphs to make them more optimal, or predictable, or thread-safe, or shared, or faster, or lower latency, or incremental, or loss resistant, or correct, or cached, or scaleable, or flexible, or less confusing, or whatever else it is you wish would happen to data in your app or system. You know you want something.
Everybody always has a core representation of something that determines how the whole system behaves, or some particular part of a system upon which your data depends, or is necessary to enable a core feature. You might have several.
The way your code currently does this makes you nervous, because you either know it's already wrong, or it's a problem waiting to happen, or because you need to change it again and you already used all degrees of freedom and it's frozen because you can't imagine how you'll squeeze in another factor without the house of cards collapsing. In fact, the house teeters most of the time now already, making you sweat sometimes.
You're afraid to let most people touch this thing. Your senior people only do so with trepidation, swearing they won't again, making no promises a new change or fix will work.
Usually you'll ask me to work on this thing when I start working for you. And I'm good at it. I make some of the chaos go away, and it does several things better, until you suppose you can demand the impossible since I've been accomodating so far. Sometimes the impossibility is you want it too fast, partly because I've been making it look easy, even when it's not.
How do you describe that on a resume? It doesn't have much to do with the domain each time ... just the infrastructure.
I'm not looking for a job. I just wanted to tell you what I do. I'm the kind of plumber who works on your bottlenecks.
I'm not describing my current day job specifically, because all of them go something like that. It's most fun doing this in code written from scratch. But startups are a pain in the ass, and I'm only into midsize companies these days.
format tinkering ¶
Yes, I see the toy language stuff looks interesting. Please be patient while I play with a little new format detail. I'd like updates to use subdued markup while being clear.
By this weekend bits should start to trickle into place, but it won't increase in volume until I find my pace. Still, updates should occur more often since I won't need to finish a monolithic demo (except when I do another under thorn).
I wrote half a page of history on my past language work as part introduction, and was dismayed to find it didn't work: it would just distract folks from thinking about matters at hand. So that was a waste of time despite being fun. After months of having everything work when writing demos, I thought I was over penning occasional false starts. Maybe structural change brings them back.
lunch chatter ¶
Coming back from lunch today with Gav and Ned (two pseudonyms used before) Wil was amused by Gav's sudden veering observation. Gav noted, "Funny how people come up with odd sayings like 'happy as a clam' — what do you guess it means?"
Gav and Ned are both older engineers, full of funny stories about computing systems way back when. Ned's an Aussie and Gav's a Vietnam veteran. A spectacularly wry sense of humor often peppers Gav's conversation, presumably driven by much past experience with fubared systems.
"What on earth does that mean?" Gav pondered. "What does it mean to be happy as a clam? Is that nonsense or what?"
Wil tried a shot. "I think it means a person who's happy because they're oblivious to the world like a clam," he suggested.
"Yes, that's it," Ned agreed. "Connotations imply happy but out of touch." Ned's articulate and has a fine sense of old school nuance in traditional English, perhaps from an upbringing in Australia. Whereas Wil's command of slightly outdated colloquial English comes from reading vast amounts of science fiction written in the 40's, 50's, and 60's.
"You see," Wil expanded, "sometimes folks are happy only because they don't pay attention to problems."
"How does that apply to another phrase?" Gav vollied. "There's a lot of them like, 'Happy as a pig in shit.' Now there's an interesting expression."
Ned started laughing now that Gav's line of thinking seems to have structure and plan to it. Gav sometimes sets up his own jokes by asking leading serious questions a little earlier.
"Uh," Wil responded gamely, "Maybe that's someone who doesn't mind a little mess — they even revel in it."
Ned and Gav shot pig in shit quips back in forth in sotto voice riffs, not listening to Wil's serious answer since Wil didn't seem to get the punchline had been reached.
"Oh, I have one more," Gav realized. "How about, 'Happy as a <product name> customer'?" Setting Ned off into near paroxysms of laughter. Wil didn't have any answer, but Gav was beyond needing a straight man any more.
toy language ¶
I started fleshing out a section of this site describing the lathe toy langauge. Most new pages are skeletal stubs so far, but it's a good outline of where I'm going over coming months.
|
The new page that actually has something to read is mu, which was previously a stub, but now introduces the new section devoted to programming language material. The intro will continue next on toy, which will index changes more persistently than this log page does.
The image at right is one of many I collected on Sunday while creating several dozen new iconic images for page decoration. It was mainly something fun to do at the end of my vacation week. This one's part of a larger work by Arthur Rackham: one of my favorite illustrators. The shrunken version I embedded in the new gc page is almost too small to read, so I thought it would be nice to show the larger version here because it's so awesome.
Of course this character in Rackham's image isn't collecting garbage, but he is doing a bit of house-keeping that involves moving stuff around from here to there, and that's basically what garbage collection is: it has less to do with garbage than it has to do with preserving valuable content still in use.
Future monolithic demos under thorn will slow down in pace of appearance as I ramp up incremental additions to the toy languge pages under mu.
stack demo ¶
A new stack demo is done, describing technique to create stacks of garbage collection roots efficiently using refcounted pages allocated from a pile of pages and books.
I'm on vacation this week, which is why I had time to squeeze in an extra demo despite spending most of my time hanging out with my teenage sons. For example, tomorrow we're going to the beach in Santa Cruz.
The stack demo I just finished has slightly odd style compared to others because I used a lot more dialog format than usual, in a way resembling the recent hash demo, but with less antagonism between characters since only Stu was involved.
Some of the oddness seems due to my having more time on my hands, letting me get slightly sloppy, probably using a few too many words. However, in this context it was likely a good idea because the material's inherently fuzzy. Let me explain.
The last pile demo was about specific heap mechanism for allocating pages and books, whereas the stack demo is about policy for using pages to effect a gc root strategy with fuzzier constraints. Policy is fuzzy. So the demo was going to feel a little raw no matter how I wrote it. Using more dialog can obscure loose edges because resolution is coarser in the first place.
flight deck ¶
I took my sons to the Great America amusement park a couple days ago, and they're still laughing at my reaction to riding a modern steel roller coaster for the first time in their lives. My younger son does an impression for his friends on his cell phone, which basically goes, "Oh dear god."
It's a little odd I sprinkle my emphatic utterances with words like god since I'm not even remotely theistic, except when I let mythic creation mode loose. Neal Stephenson might say I addressed Athena about design of the coaster I rode. I prefer to call on Wyv, a forgotten Norse dragon god of technology.
pile demo ¶
I finally finished the pile demo showing a heap subclass allocating pages and books. I hate to think how long I spent working up to that page. Writing about memory management takes even longer than coding memory management.
That's the demo I wanted to reach for several weeks now. Now I plan to segue into some gc-box related format demos. I've started the next stack demo showing a class using pages from the pile demo to build scatter-gather stacks of gc-roots to refer to objects in collected memory that move.
code comments ¶
A short section in the pile demo mentions code comments in passing. I've been thinking about a longer treatment since folks are saying lame things about code comments again.
I've explained many times before that comments should focus on why instead of what since code itself shows what it's doing — at least on the lowest level.
But why isn't what I miss most in code with no comments. What I really miss is redundancy letting me find variance between what an code author intends and what code actually does. As near as I can tell, everyone makes mistakes, and it's harder to pinpoint them without some semantic duplication.
Over many years I've learned many different systems in day jobs by reading code with little or no docs with design descriptions. So I find myself making tentative models about what's happening on several levels at once, slowly correcting the models as I gather more evidence, often imitating scientific method.
The most irritating code to read is a system that just barely works, with no slack or repetition whatsoever, without any comments confirming your tentative evolving theories. At any given time you can't quite understand what you've seen earlier because it still depends on parts you haven't yet seen to make sense.
Those crazy bits you saw earlier that look like bugs? They still might work if code you haven't seen yet is odd enough. And until you've seen and understood all the code, you don't quite grasp any one part 100%. It's quite frustrating.
What I really want to see is code redundantly described in terms different from what code says directly, so I have fewer unknowns left than facts and data points I've gathered so far. When I'm half way through I'd like to see everything overdescribed enough that I can predict what's coming and check it for consistency.
I get asked, "What do you think so far?" a lot when I'm learning systems, especially when I've been assigned a problem. An accurate description of what I think as I model a system is hard to describe because we don't have good vocabulary for it.
Mental models I build always have far more parts to them than anyone wants to hear, because they preserve many shades of gray. For example, every piece of code has at least three different versions I'm tracking, which we might call should, intended, and actual, and all three can have bugs. Yes, I'm sorry, this is going to get complex, but that's what I was telling you.
The first is what code should do if a problem is being handled appropriately by the code. This often has an ideal flavor as a general plan which a coder approximates by a specific incarnation which is what he intends to do, which is only similar to what the code should do. Typically neither of these is written down. Instead, you only see what the code actually does, which often matches neither the should nor intended plans.
Okay, now that I've said it once in plain English words, let me say it again using pseudo math notation to make this easier to track, because it's going to get perverse as I expand.
Let's use letters to denote aspects of coding, working backwards from a general plan at Z, which is approximated by a specific coding strategy at Y, which is supported by some code folderol at X. So Z might be "I'm going to persist data this way," and Y might be "I'll use this library and encoding scheme" and X might be "here's my twisty classes supporting Y."
Now X, Y, and Z all have their should, intended, and actual variations — let's represent these with integers 0, 1, and 2, with zero used as the most ideal. Z0 is what the plan should be, Z1 is what the author intended the plan to be, and Z2 is the plan actually embodied. None of these usually match exactly.
Note plans Z0, Z1, and Z2 don't actually appear anywhere because they were ideas, unless someone is saying them to you ("our original idea was to do ABC...") or unless you infer the ideas from the things you see embodied in code.
What you actually see in code is a mixure of Y1, Y2, X1, and X2. Because you're reading the code to solve a problem, the code's not doing currently ideal things Y0 and X0, but you have to infer how those are different from what's actually there.
As code is versioned over time, all three — 0, 1, and 2 — experience drift so the codebase is a mix of historical values for those in a cross product with X, Y, and Z.
When I have conversations with developers, they almost always think I'm discussing subscript 0: what code should be doing. They find it distressing when I discuss the model of 2: what the code is actually doing right now. And it's worse when I reveal 2 is not the same thing as 1: what the code author thought he was doing, but didn't manage to achieve.
Is this creeping you out yet? Well I've got news: having to construct this sort of model to track what's happening isn't a lot of fun either. But it's the only way to sort some things out.
Code comments are instrumental in clarifying 1: what the author thought he was doing — the model an author thought was being pursued but doesn't quite appear in code which is always 2: what code actually does. Without code comments, it's very hard to locate, for example, where Z1 is situated between Z0 and Z2. Why code aims for Z2 — instead of Z0 where it ought to aim — would be a lot easier to work out if you could know how much of either resembled Z1, the plan the author chose as an approximation of ideal Z0. Sometimes all of them have bugs: code not only doesn't work now, it also wouldn't work if it did what the author intended, and wouldn't have worked if the ideal was achieved either.
You see now why I have to be paid to do this. It involves far too much psychology and code archeology when what you really enjoy is making up elegant systems to solve problems. It helps to know what would be elegant if it ever occurred, because you can see how things now differ from better versions. And you can see how much it would cost to change which things at what rate, to pursue goals other folks think are elegant. But you rarely get to pursue what you personally think is good.
