Lately I write demos under thorn with a todo list using C++ under a BriarPig mu-babel license. The run and hex demos were done on 13apr2008; crc on 14apr2008; buf on 20apr2008; in on 27apr2008; ctype on 04may2008; out on 18may2008; slice on 25may2008; quote on 31may2008; escape on 31may2008; mutex on 14jun2008; rand on 16jun2008; stat on 17jun2008; primes on 19jun2008; list on 23jun2008; heap on 29jun2008; iter on 02jul2008 and 04jul2008; atomic on 06jul2008; node on 13jul2008; page on 23jul2008; hash on 27jul2008; book on 27jul2008; pile on 03aug2008; stack on 07aug2008; mu on 12aug2008; toy on 16aug2008; weight on 23aug2008; symbol on 02sep2008; imm on 04sep2008; gc on 06sep2008; map on 14sep2008; meter on 16sep2008; thread on 22oct2008; arc on 05nov2008; this menu links all demos:
mu: toy, peg, imm, tag, box, symbol, token, number, bigint, class, method, reader, writer, eval, env, vm, gc, world, pcode, compiler, asm, lathe, lisp, smalltalk, design, weight, jar, card, harp, debug, profile
thorn: todo, names, iovec, assert, log, run, hex, crc, buf, in, out, quote, escape, compare, file, deck, cow, arc, blob, tree, slice, rand, time, stat, heap, node, primes, page, book, pile, stack, atomic, lock, mutex, thread, map, list, iter, ctype
optimism ¶
You might not guess from my fiction lately, but I've been feeling optimistic. It was around a week ago I noticed I was looking forward to writing more. It's been a long time since I looked forward to anything. So that's a really good change. To encourage that feeling, I've been writing more, that's all.
The next long sequence of scenes is one I've been trying to reach for a while now. It has mythologically interesting themes I get a kick out of. Recent material had bits specifically designed to support what comes next, in the sense of reducing total number of words needed to convey good parts, without smothering it in explanation. (This is typically an interesting challenge in fiction: laying groundwork, fun in itself, that later makes you see a new thing quickly because it follows.)
My normal mental makeup is a kind high in dopamine but low in serotonin—meaning I can get dopamine just by thinking, but serotonin comes only from regular gym visits. That's really the only reason I workout: I go every two or three days because of how it makes me feel the next two days, and that's all. There's no goal except high spirits. Besides caffeine and vitamins, I medicate in no other way. I never drink anymore—the last time I tried was socializing a few years ago. Doesn't do anything for me. Also, I try to eat very well.
I usually have salad for lunch. For the last few years, 90 percent of my lunches have been salads: high in raw vegetables and moderate in protein. The place I frequent every day lately has very fresh, nearly raw vegetables. (In fact, they might be totally raw for all I can tell, but finely shredded beets and carrots are good.) I never use salad dressing any more because it correlates too highly with intestinal trouble. A single bowl of chili is enough to supply flavor to otherwise totally bland victuals.
Do you get the idea I'm offering advice? Exercise several times a week. Avoid sugar and eat lots of vegetables and lean protein. Sleep as much as you can every night. And do something you like all the time. Also, stay away from people like Finch, because it's not worth it. I'm kidding: if you thought all that fiction was parable or allegory, the message isn't about any specific thing. (I suspect it's unavoidable messages are buried in there, but it's not my goal to get anything across.)
simulation ¶
Well, the other shoe fell at work. I won't burden you with analysis of overly subtle office politics. But at the moment it would suit my interests to spend a lot of time—probably evenings at home when I can—writing a simulation answering big questions, as quickly as possible, while timing is right.
context ¶
(See the fiction page first for context.)
armchair criticism ¶
Two hours before Red gave him the ring, Zé sat fidgeting outside Flywheel's office, waiting for his time slot. He spent his short wait supposing the crouching, two-legged science fiction sculpture in the foyer was really a heavily armed futuristic dreadnought biding its time until awakened by masters to defend the perimeter. The shape roughly seemed heavy ostrich legs and body, without any neck or head, all composed of presumably telescoping machine rods like a battle bot vision out of Stephenson's The Diamond Age. Two miniguns depended from the body, along with a pair of skeletal arms, no doubt needed to guide mass in fast heading changes, say when flying through doorways at speed.
It was extremely well done. Zé intended to ask Koi the artist's name. But it looked like several thousand dollars worth of painstaking labor to make an intricate piece like this, and Zé could not afford that. Maybe a miniature copy could be arranged.
(Continued at fiction.)
paintball ¶
The last time Wil visited this place, he was one of many engineers from BigCo playing paintball in a team-building exercise, sneaking through a dark maze of partitions in two opposing teams bent on shooting each other for high scores. In the last fifteen years, Wil attended three such events, all very similar to each other in most ways: interesting, chaotic, fun, unsatisfying, exotic, pointless, and curiously lacking in strategy for a game involving folks hunting one another in semi-darkness with submachine gun-like toys.
(Continued at fiction.)
recovery ¶
Wil came around in a sitting up position answering Falk's questions—he'd been half awake for a time he no longer recalled. Falk demoed features on absurdly lightweight, soft body armor, showing Wil how to adjust it for comfort and fit. It almost seemed a costly, very strong, high-end sports jersey with moderate padding and many points of smooth articulation. But Falk said it was nano fab dragon skin.
(Continued at recovery.)
work: async caching ¶
I finally finished my new spec at work this week, which roughs out a design for what we're doing in coming months. I can't tell you the schedule of course, but there is one. I'm halfway through integrating a big upgrade into the last version. (For the next design, I found an ideal way to do something driving one of the costs to zero, and this also works in the last design.)
Very oddly, I had remarkably little feedback about the new design, either positive or negative. It's like after one long meeting everyone said, "Well I guess that's the way it should go." But that never happens, so I'm waiting for the other shoe to drop.
Much of the code api uses dialogs between Wil and Stu characters to do fast conversational Q&A. It made me nervous when I wrote it that way, so I asked folks to request a stop to dialogs if it seemed awkward. But no one had any comment, except an architect who noticed only Wil says things you need to track.
The very last web page I wrote was an intro to dedup for raw beginners, so new folks can write clients for the new design's api. I managed to write a nice crisp explanation unifying descriptions of 1) traditional compression, 2) our last system, and 3) the new design. I think several folks had an "aha" experience of sorts when new parts seemed minimal in context.
I'm looking forward to coding all the time for a while instead of writing prose about code.
Summarizing my design in high level terms, the phrase "async asymmetric peer-to-peer caching" covers territory involved, but without saying how I do it. The hard part is asymmetry: it's really interesting.
hobby coding: quiet ¶
I currently work very little on Lathe and async pipes beyond some visualization and problem statement refactoring. I wouldn't do even that much, except it's what I'm thinking about when I wake at times.
Fiction is basically usurping all the time I would put into Lathe runtime work. When I get bored with that, I'll go back to Lathe programming language runtime work. The problem with coding for fun should be obvious: after twenty-five years of aggressive hobby pursuit, I don't learn much any more when I write code. It turns out about like I expect. No surprises, ho hum.
In contrast, writing fiction is causing weird insights. And they're hard to describe. It's like I'm cooking up a new batch of axioms about how systems of information affect human minds—enough to get a little parallax so I can see how it affects my own, in a way I didn't see before. It's in the same family as an idea you might call "buying into the program," but weirdly colored. You know a bumper sticker reading, "If you can read this, you're too close"? This is related. As a bumper sticker it might read, "If you got the hook, you already bought the scheme." Once you accept carrier waves embedding a message, it's a little late to treat the carrier's premise with suspicion after decoding.
It's like a one person version of social proof, involving only you. Decoding a message tends to have an unwarranted bias toward telling you the carrier itself is not a subversive message.
fiction: exploration ¶
I wrote a couple more fiction installments, but they need editing. Today's piece takes place in Flywheel's office before Zé gets the ring. It's good enough to post: it satisfies a low quality bar. But it could be better. About five percent impedes another ten or twenty percent. So I'll cut it down first.
Fixing mistakes often involves removing bits saying too much. Oversold parts ruin the tone.
But I worry my standards might creep up faster than I write and post. I'd hate to get stuck always realizing each segment could still be better after each edit.
Last night I drafted a direct continuation of the fragment below, but it didn't work. This is more interesting than it sounds—on one hand, it's disappointing to write something you don't like. But that's how you learn: by figuring out how you did something wrong. So in this case, the way I failed gave me a plan to do better on future first drafts. It was broken because the story was told from the wrong perspective.
So I plan to fix it after the Flywheel office scene by jumping ahead and resuming from Wil's view, which at that point only implies the part cut. Believe me, this is better. I'll probably have Zé lecture someone on the power of context. I like the story better when all the interesting ideas are implied, giving you something to think about. And I guess I just like the sensation of having missed something; it increases a sense of time pressure when you want tension.
There are funny bits in the part I cut. But those funny bits oversold something, which limited the scope too much. More degrees of freedom is better.
context ¶
As always, the following story won't make any sense unless you read the fiction page first.
implants ¶
"When I buy a new phone, it comes with a little user manual telling me how to use it," Wil noted.
"Hey Falk, did you ever get around to writing a new manual?" Finch joked. "I lost my old one."
(Continued at recovery.)
context ¶
As always, the following story won't make any sense unless you read the fiction page first.
upgrades ¶
Despite Falk's feverish activity, Wil finally realized there was nothing he could see in Falk's process. As a programmer, Wil should have expected it: laymen seldom see Wil do anything but type. As he watched Falk, he realized all he could see was nanite coding, or maybe indirect support two or three levels removed. Wil wasn't equipped to grasp anything about his tuneup in any detail. It was all black magic, but it involved Falk thinking a lot, then chasing down details.
(Continued at upgrades.)
context ¶
As always, the following story won't make any sense unless you read the fiction page first.
preview ¶
That night Ulf dreamed again: Wil played Cole in Twelve Monkeys (with his good memory) sent to find Jeffrey before the army of the twelve monkeys could do something inscrutable (but funny). Only this time the homeless guy haunting Cole was Dr. Laughs, the nutjob both Zé and Wil condescended to like. In the dream, Ulf worried Laughs was a vampire and checked his reflection in a mirror. It was visible but distorted.
memento ¶
Wil saw much of Zé and Wil overnight, and a few other segments with Ivy. Finch warned his card only played for him, but he meant to test it. By morning Wil's heart ache was worse, not better. Paranoid ideas thrived: can Finch generate images? Was Ivy real?
(Continued at fiction.)
eye contact ¶
Finch returned a couple hours before noon without Falk—apparently to get Wil slotted and ready.
(Continued at fiction.)
tuneup ¶
Falk did indeed carry a little black bag, as if seeking instant identification as medical doctor. Signalling non-combatant status? Catching sight of Finch while entering Wil's apartment, he put up a hand to hide her eyes.
(Continued at fiction.)
immediate gratification ¶
Fiction has an advantage of immediate gratification at the moment. In contrast, it doesn't seem like it matters whether I pursue programming language work, since by the time I finish anything, the time is long past when feedback would have been interesting (for the amount of effort involved).
Stories rarely have technical problems to solve. But I'm roughly guessing how big a tool of Finch's might be when intended to be worn. Hmm, I guess size might be tuned depending on circumstances—whether more mass was help or hindrance in one's near future.
async pipes ¶
The next code to describe is async pipe api. If you consider Lathe's requirement to avoid all blocking calls, it's not hard to deduce all stream i/o must be based solely on async pipes. Noticing this fact is a useful global generalization: statements that all X's are Y's help eliminate blind alleys in otherwise reasonable sounding assumptions.
But other than actual code, there's not a lot interesting to say about an async pipe api, except that it ought to have parts resembling other async pipes.
narcissitic youth ¶
I haven't screened folks recently, but the last time I did (last year) a few recent college grads didn't make the cut for an in-person interview because they sounded like clock-punchers: after having done their time, it was their due to move up in the world, whether or not interest, aptitude, or passion was present. Note: If I poke around looking for passion and interest, while saying this is what I want to see—and you still manage to hide it completely—then you fail.
A typical central question I ask sounds like this: What do you find interesting? Don't worry, you can't fail just by not grasping the question, because I'll ask it several ways until you manage to prove you find nothing interesting, except maybe status. I'm persistent in my search for a chat about your thought process. At the end of my rope, I'll press to find any problem you ever worked on that seemed mildly interesting to you—I just want to talk about it.
Surprisingly, I have had kids manage to hide experience in solving any kind of problem, high or low, in school or out, small scale or large—even when I dig for twenty minutes straight along many avenues. You have to be pretty dim to not recognize what I want to hear and just make something up. (I don't care whether you make something up, I just want to talk about the thought process; why should I care whether you simulate it on the spot? It's all the same.)
As near as I can tell, sometimes an applicant's attitude resembles this: I went to a nice school and survived without being shot down in flames, or drummed out in ridicule, so I'm ready for the gravy train. After satisfying minimum requirements for success, folks expect success bestowed upon them.
Huh? I'm sorry to say I'm not even slightly interested in your desire for success, unless you expect it as a side effect of working—preferrably at something in which you think you have skill or interest, or something.
Of course, once you own up to ever having worked on a problem, or thought about one, I'll ask a lot of questions. And no matter what you say, I learn a lot about how you think. Among other things, I quickly get a sense of how bright you are, and what sorts of things you're better at doing—which I'll try to verify with more questions. If you picked the wrong problem to discuss, I'm happy to switch to another. In fact, I'll suggest we follow up on several related problems directly adjacent to anything you mention. You can only get a crummy score in choice of problem to discuss by conveying an impression you can't relate to thinking as a tool for solving problems.
Sometimes folks say, "I'm interested in a job, doing something that sounds cool."
(Incidentally, when I'm interviewed and I'm asked what I like to do, I say I do anything that needs doing, but that I very often get assigned problems of great concern and priority to teams I join; so coincidentally I often get what I like: the hard stuff. But shoveling crap can be part of a job, and I don't complain. I'm as far as folks get from being particular and picky. So I get what makes people wince: gnarly stuff, cool.)
To that I might reply, "I want to talk about any problem you ever worked on: I don't care what it is. I want to ask a series of questions about aspects of problems. I just want to get a feel for how you think. I don't even care if you solved a problem."
The only way you can fail is by refusing to mention any problem, of any sort, during your whole life. And yet some people do this: the moral equivalent of a shrug over the phone. Then I persist in the hope it's clear that nothing means: fail.
Even so, I still sometimes get, "I went to school XYZ and I should be set for life."
Good luck with that.
Of course, if you participate in my desire to discuss problems, you're going to get scored. But it's a really complex score: I describe you to a hiring manager in a lot of detail, in terms of how you think, and this rarely involves something as linear as how smart you are. Except, of course, if you obviously seem a genius, I'll say you're very bright, so you do get credit.
(I don't get that many geniuses, and when I do they often say during in-person interviews that they want to be the lead architect. Say what? Did you notice the rest of us have fifteen more years of experience, and some of us are just as smart as you? We explain the invitation is just to be one of the gang.)
Sometimes I get an idea young folks are poor at gauging elders—often making an assumption everyone of a certain age is the same, guessing older engineers must be like parents or teachers, or something, and not peers with many years more experience.
copyright ¶
I will not be happy with anyone re-distributing my stories. Everything here is obviously under copyright: everything is under copyright unless explicitly licensed otherwise. Only code on this site is covered by my code license. If I catch you reprinting my fiction, I won't let you off easily ... unless it's clearly attributed to BriarPig. Then it might be okay.
Alright, if you always note BriarPig's copyright, I won't bust your chops. But I'll be seriously annoyed if I'm not the sole person attributed as the author.
If you want to write stories similar to mine, that's totally cool. If you can manage to nod in my direction, that would be even better—even respectable. But don't use my characters, okay?
background ¶
The following story won't make any sense unless you read the fiction page first.
enigma ¶
Zé boogied for his car, keeping half an eye out for Red and Alley Oop to reappear.
Something about the gold ring bothered him. Why not cast it aside? What if—somehow—Red and Oop can track its location?
Zé guessed it might be precious. But why? It was physically odd—it shouldn't shimmer like that. Red's expression as she threw it said: let's see what you do with this. Was it curse or boon?
He delayed trying it on until crossing the street to his car. Zé smiled at his fear: what if drivers stopped seeing him once he wore the ring?
At his car door he tried his right ring finger: no, it was too small... at first. After hitting his knuckle the ring seemed to stretch a bit. Fine lines flowed faster, with extra gleam. Then it fit perfectly.
What would happen if he tried other fingers? Ignoring this question, Zé started his car quickly, thinking of things to ask Finch.
rewind ¶
Unknown to Zé, Finch already gave Wil clues the night previous after Wil's injury in an accident, during a talk with Finch while walking near sundown 24 hours before Zé received the ring. It went like this.
Ulf stopped Wil as he and Finch were headed over to Vintage Season for coffee before a chat. Ulf blinked and looked uncomfortable before he spoke.
(Moved to fiction.)
smart paper ¶
"You feel a little traumatized," Finch guessed after bandaging a raw spot on Wil's scalp back at his place. She gave Wil pain killers and water, then settled into a chair at the computer they use to control various systems installed in Wil's home. Finch brought up UI for Kip's cameras and checked status.
(Moved to fiction.)
pipes ¶
I've been thinking about pipes, since about ten days ago. Over the last weekend I spent most of my "hobby coding time" reading a fair number of web pages about Unix pipes and related models.
At first I did that just to pick a first style guiding async stream api for Lathe bootstrap context. But then I started having a trickle of brainstorm traffic I didn't want to shut down. So I've been giving it a little rope.
Then I had the most interesting half-dreaming, half-waking brainstorm about pipes while trying to nap two mornings ago. Plumbers began demolishing a facade of wall just on the other side of my bedroom wall at 8AM Tuesday morning after a late night and I woke up totally groggy. So I tried to nap on my couch half out of range of the racket. The result was similar to half sleep when sick with the flu, except it works even better when I'm not sick. The noise kept me just awake enough to keep a lucid thread in many free form associations.
When I finally got up to go to work, I was thankful the plumbers had ruined my sleep, since my thought process napping had seemed incredibly fertile. (In case you wonder about the pun with pipes and plumbers, tempting you to suppose I'm pulling your leg—that's just a coincidence; all this story is quite literal.)
But by today I could no longer quite remember the really good part. What had I found so interesting, exactly? Then I remembered. Turns out the most stimulating insight wasn't technical. I had a quirky view of social interactions among groups of people as a kind of asynchronous calculation using people as processes. This is even funny when you think about how original process theory actually evolved in the opposite direction, reasoning from informal descriptions of calculation process done by a person to something done automatically by machine.
This rumination overlaid visualizations of memory layout designs for pipes, and I had cross over energy bouncing around in the mix.
Anyway, compared to my normal waking experience of concentrating on tech ideas, I vaguely remember having about three times as many complex ideas going at the same time—a bit like being in hypnotic trace to ignore everything not related to what I was studying. Something about the density of it must be what struck me so hard. I harbor an illusion it will start going again when I have time to think about it fully once again.
emotional impact ¶
I wrote a long new chapter of fiction. It was very fun to write but it's not good enough to post. It's a little under my threshold for rejection based on writing quality.
Compared to recent pieces I've written, it has much more plot—which I prefer to lyrical fluff. I was always a big fan of old fashioned science fiction moving so fast one strains to follow narrative. So I let the pace move too fast; and yet, at the same time I wrote too many words. The treatment must be more spare to work.
The end of it includes a story fragment I mentioned early last January. It has a big emotional impact on Wil, which seems to be why I wrote it poorly: I suck at rendering emotional context ... especially when characters can't leverage normal convention to show motive due to personality quirks.
Anyway, it was exciting to write. But when I proofread later, it lacked detail I had in mind but didn't explicitly show. I filled in blanks using my mind instead of words, which doesn't fly. A process of refinement to sharpen end product was obvious, but I knew that would be work. I don't write fiction for a living, so why do I want to turn fun into work? I'm still stewing over that one.
I guess I need to treat it as a first draft. I'm inclined to copy collect into a crisp rendering when I'm in the mood again. But I only want to make it good enough to pass my rejection threshold. Setting too high a bar in quality makes it taste too much like a job.
Agent Finch causes a lot of trouble in the "show, don't tell" department because she knows far more than other characters can verify empirically, leaving them somewhat at the mercy of her pleasure to disclose facts they can't test. I'd have to write a bloody novel to squeeze it out of her by force, and I'm not gonna do that.
My compromise lately is negotiation in presence of need. Finch needs something and has to give in order to get. But she holds too many cards and the result is slanted too far in her favor, unless she's a saint, which doesn't make good fiction—it's too stilted.
archipelago ¶
Since the redhead never introduced herself, Zé tagged her as Red in his mind; as she and Oop receded from view, Zé made sure a path to his car was clear, then finally glanced at his catch to see what Red tossed him.
To his surprise, it looked like a gold ring — quite simple in form at first look until Zé noticed intricate, vanishingly tiny detail dominated by threads — or striations — of lines seeming to flow slowly. Maybe it was optical illusion. Along myriad lines, irridescent gleams traced faint rainbow color like a fresnel grating's nacreous sheen.
Zé slowly raised it closer to his eye, gazing through it's center while rays of dying sunlight cast running glints of color round its inner rim. A flash of déjà vu took hold, recalling a talk he and Ulf had one week ago, to the hour. Zé searched this memory for signs in Ulf's question about Wil.
"What's wrong with Wil?" Ulf asked Zé with a grumpy sigh, ignoring a garish sunset out the window at Wil's place.
Zé slowly brought his sight back home from an inner vision of alternate reality where kids in home-made parachutes climbed rope ladders, single file, up to an acre of drifting property in the sunny winter sky supporting a small, crumbling stone tower where the gardner lived.
"Earth to Zé," Ulf intoned without emotion.
"Ground control to Major Tom," Zé echoed, blinking. "Nothing's wrong with Wil — same as always. Except for a slight spring in his step. Notice that today?"
(Continued at fiction.)
breadth first ¶
Just a bit more content on the new qi page today, sampled lightly below. You'll notice I haven't yet started covering an actual code api. But I think once I do, anyone reading that page will wish I described things with a broader perspective. Well, that's what I'm writing now. My approach will get steadily narrower as I go. Enjoy the overly high level view while it lasts, because soon it will dry up.
Also, you might notice I indulge in quite a bit of comic tone unnecessary for technical presentation. At work I can't write like that — I'm awfully careful to stay rather dry, which is kinda boring. So this is a chance to act out a little, and an opportunity to continue characterizing stock voices used in fiction elsewhere. Also, I hope fun parts help ideas stick better in your mind than they would otherwise.
When I start drafting code api, I expect to swing entirely in the other direction: I won't explain enough. (At work I'm careful to explain everything, which eventually becomes an incredible grind — but it's work, so maximizing value for others is a goal.) I won't need much explanation myself, so I'm going to short-shrift you a little — at least compared to my standard of diligence in day job contexts.
prison model ¶
"Something doesn't fit," Eli puzzled. "If a client process is single threaded and can't block, and Lathe defers blocking operations to a client env, who can't do them either, then who makes blocking calls?"
"Nice analysis," Wil granted. "I couldn't put it better myself. But I think the answer is implied by your process of elimination. The client env is out. Who's left?"
(Continued at qi#prison-model.)
command lines ¶
"We want to invoke Lathe from command lines," Wil said. "Which would run it as a process, of course, unless a command line is routed as a client request to Lathe running as a server somewhere else."
Eli nodded tentatively. "How does that relate to running in a prison process without amenities?"
(Continued at qi#command-lines.)
processes ¶
"Can't put my finger on it," Eli concentrated. "But somehow you're violating the Unix way, which involves doing small things in each process, so you can combine multiple processes to get larger tasks done."
"I had in mind adopting the Unix way in Lathe," Wil countered. "Except by extending the idea of processes into Lathe. I think it'd be cool to connect lightweight processes in Lathe, piping output from one to another."
(Continued at qi#processes.)
