Þ   briarpig  » log  » jul07

doughnuts ¶

     Last night we saw The Simpsons the movie and by the end I was channeling Homer's brand of misplaced practical advice. My oldest son tricked the younger in a race, so I offered him this in Homer's voice: "Remember son, never fight fair!" which seemed funny at the time. I should teach sportsmanship, right?

stack roots ¶

     I was going to discuss the gc root code I was using in my lisp reader and writer today, but it's too late now to go into detail. I have a data structure in þ specifically representing stack frames of reachable gc boxed objects. It's a scatter gather stack composed of pages, with frames mimicing 68K style link and unlink operations, and an RAII C++ class declaring frame size.

     I decided I was likely to build gc based Lisp lists so often I ought to have a convenience class specifically for that in C++. So today I wrote a list class taking box heap and gc root stack instances, one to alloc pairs (cons cells) and the other to make reachable the lists being built by putting box references in frames. And now I can just push_back() the next list element to append.

     Tomorrow I'll probably throw in an iterator class so when I (eg) convert lists into vectors I can write code looking like an STL iteration loop, which actually does the most direct and efficient thing to the Lisp list pairs. I often make code look just like STL even when it's not; might as well use a standard interface.

zooko ¶

     I answered a new email from Zooko on the letters page, where we mention free software licenses and detecting integer overflow. Zooko's a nice and smart guy who last saw me during the worst of my post divorce meltdown several years ago (never subject to discussion here).

personality type ¶

     In private email from an old coworker (a fine human being) I received a casual query about a VP of engineering position to be filled, in case I was interested. (Or in case I might pass word along to others. Maybe I can post info here; I can ask.) The idea is flattering and the work sounds interesting.

     So why wouldn't I normally pursue that kind of thing? I doubt I have the right personality type to manage large groups. I might enjoy it, but I don't have a business mien, and I'm far too casual and egalitarian to front authority and discipline. I don't have the bearing of a VP and don't think I can fake one.

     If you know anything about Myers Briggs and MBTI personality indicators, you might agree an ENTJ or ENFJ type personality fits a VP style of performance better: an extrovert who enjoys setting and imposing a particular viewpoint, and guiding others to follow that viewpoint.

     In contrast, I'm basically an INTP, which means introverted, intuitive, thinker, perceiver. Introversion's a close thing — I don't seem introverted, but I care slightly more about my internal mental experience than what happens in the world. The thinker (over feeler) aspect is close too: my focus is logic but I won't sacrifice your feelings when I think I'm right. In fact, my idea of "right" often has much to do with less strife in human affairs. You might not see I weight thinking over feeling.

     So iNtP might be a better way write to it, with strong intuitive and perceiver aspects. I'm as intuitive as folks get. (And I evaluate by perceiving, not judging, so I never force things into slots because I want them there; I still make decisions easily enough, which can be hard for many perceivers.)

     My personality type is often called architect, which fits well enough — but I'm more a builder/architect since I like hands-on creation more than abstract design. I'm a sort of person good at executing and designing both; I often get this kind of work. I come out visual mathematician on tests grading ability, which means (duh) I'm visual and strongest in math. (Did you think because I can write, I'm better at words?)

     Anyway, I'm no good at playing a heavy. I'm just one of the guys in my current team at work and our team dynamic is well suited toward casual intelligent cooperation. It'd be exciting to be a VP, but I wouldn't have as much fun.

poor choices ¶

     Apple's stock price kills me, because in Nov2002 I said no to a fine Apple job offer — I took an open source position with less salary instead. Stock options in Apple's offer were hard to resist: six thousand priced under $10 each. (That'd now be twelve thousand options after the split.) I fantasize what it'd be like to have them all vested. Woulda coulda shoulda.

heartbeat ¶

     I plan to cut posts here to minimum size for a while, except when I get a kick out of longer pieces. So treat updates like a heartbeat showing the site is active. You can elicit more material by email if you don't mind the cost: publication. (You don't have a free rider problem, do you?) I'm starting to have slightly more fun coding at night, and it takes priority.

lisp repl ¶

     I'm writing my Lisp reader and writer in parallel, so I can write whatever's just been read once I finish reading code, and finally I get more feedback and can enjoy interactivity. I hate writing code for weeks and months without ability to run most of it. That's what it's like when you bootstrap a system that's not already working, but it was worse than I expected.

     Right now I'm just echoing expressions, so it's only a read write loop; the real fun starts later when eval is added. A lot of my work up until now focused on physical representation, and I'm only just now getting to using the heap allocation I spent so long coding. Strings, symbols, and floating point work like a charm of course. I have array types of all the physical number sizes, but I won't code the i/o for a while yet.

floating point ¶

     I was disappointed by how many digits of precision appeared in the long double floating point boxes, so I started reading about 96 and 128 bit floating point formats, and was instantly sorry I did. Gah, turning into a numerics enthusiast is the last thing I need. All the math stuff gets token support only for now. I just don't care about Scheme's numeric tower, so its priority might come last. Maybe I'll farm it out.

demon calls ¶

     Jens Alfke posted an idea provoking a really funny comment by Jon Deal about about collect calls from supernatural entities. When Jens mentioned Nyarlothotep it overflowed my gotta-link meter. Now I need to stop thinking about it, or I might have a fiction attack. (Lovecraft is cool.)

bad timing ¶

     I'd aim to avoid posting twice in a work week henceforth, but that'd only give me a twice weekly schedule, meh. Work was hectic this week, but interesting. Two nights ago I played with social networking for hours, and last night I read Rudy Rucker's Mathematician's in Love halfway through. So I coded little at home this week.

content indexing ¶

     Some of what I do at work is content indexing; occasionally I've a need to make up something, like this week. When the result is fun (like it was this time) it's exactly the sort of thing I can't describe for you: simple but wicked hacks too small be called algorithms, but big enough to get names.

selection filter ¶

     I wrote a piece of code selecting the best way to do something with a probabilistic effect, and stumbled across something good I mustn't disclose. But I've been thinking about what the selection code means; I call it a unit test, but that's not really what it is. (It's an experiment.) It proves a fuzzy, probablistic property of a function whose code doesn't advertise it's effect.

     It's hard to describe to folks at work what this "test" code is comparing and evaluating, but once they understand what it demonstrates, and how it does that, they totally get and appreciate the experiment's effectiveness. I've been wondering what category of code to put it in; it's some kind of isolated metric for distribution and probability.

     The reaction of one of my junior colleagues was strong: he seemed as interested in how do you think like that (to come up with the evaluation scheme) as he was in the final result's meaning. I tried to explain how sometimes when you want to know the answer to something complex, you can code to measure exact effects for target populations.

     The explanation for why it works is vaguely geometric, since I favor that sort of math. I'm toying with an idea for a wee essay on viewing code geometrically, and how reasoning can be more simulation than logic. (That sounds highfalutin. Maybe I'll save it for a day when I'm trying to imply, alternately, I'm both smart and stupid, hoping contradiction teaches you to stop being swayed so easily by minor effects in rhetoric.)

ordinaries ¶

     The headline is a reference to Dash's whining in The Incredibles, which I watched again recently. Dash noted when everyone is special, no one is. At my home we better fit a film named The Ordinaries. I've little in common with Mr. Incredible except age and a minor paunch. (A belly is the one thing that won't behave as your metabolism falls steadily).

     I loved that movie by Brad Bird; I watched it several times when it debuted. It's tempting to write spoofs of Incredibles, but you can't do so without giving the impression you imagine yourself a superhero, when you're more of an antihero. (I wrote a perfectly respectable antihero character a few years ago, but he's retired beyond future cameos.)

anyone can cook ¶

     My boys and I saw Bird's Ratatooie recently, which was excellent and an obvious source of animal metaphors useful to sites named for anthropomorphic pigs. But I'm still unready to be funny; I'd waste the material — maybe later. I relate better to slighted rodents with cooking skill than superheroes (fitting Bird's working the obverse of Incredibles).

map trees ¶

     Normally I'd avoid repeating a topic so soon, but std::map came up again at work today. A few months ago I used an STL map in a unit test timing a new algorithm I used to replace an old one in the engine. I assumed performance of std::map was immaterial, but it turned out slow enough to skew test results when someone profiled the unit test now.

     Maps were used only to hold resulting sets of data, but my timings left elapsed time for map operations in the mix with my target data. The extra time was more than enough to change benchmarked relative speeds significantly for slow and fast algorithms in question. So my experimental technique was shoddy: I assumed map cost was trivial. But a profiler showed large costs in comparison to those I measured. Oops.

     Actually it was a minor mistake — more interesting than alarming. I started dreaming up a new alternative algorithm while we discussed the old one. Apparently my fecundity of inspiration is enough to entertain. To me the description of a problem suggests a way to work around it. If not, I try new descriptions until one does.

app memory ¶

     Bill Seitz has been citing some of my old stuff on storage, and I'm wondering if I should say more about it despite the fact I wrote off most of my past endeavors when I was divorced. (It cost more than everything I had, and more than I'll have for years, but I'm over bitterness and into thorns.) However, in some ways what I pursue now is tangentially related.

     My interest in storage was always secondary to programming languages. However, I didn't think apps were interesting that didn't have memory. And at the time I started focusing on storage, systems were so limited in available resources it wasn't possible to write something I thought interesting without first writing language persistence.

     So this caused an odd situation where I got entangled in my second choice interest — the provision of suitable storage for dynamic systems — while other folks got the fun and easy parts of cool languages. (A thorn acts as a snag that entangles, and entangled is the idea I saw in common to word roots inspiring my name. In knowing, one becomes the enemy.)

     I was always designing my idea of the perfect storage backplane to support a dynamic language, because I wanted what I'd be able to do with the dynamic language given this, which was create a little virtual operating system not terribly different from what's sketched on the gui page. But I was tricked into marrying the sister of the one I wanted.

     From this perspective it's easy to see why total flexibility was my goal, and why what I designed tended to fit plans other folks had, especially when their goals were similar. It's just we couldn't agree on who was going to play horse and who the rider. I wasn't dumb enough to fall for that again after I let my ex get away with it, so I declined to be the tool.

     I stopped working on my storage system when folks I knew spelled out their intention to take the first public domain version and slap a new copyright on their fork, and leave me behind without thanks. So I started working on languages again at that time, since I could always resume storage when I need it, having already possessed myself of expertise.

     This is why my license won't be public domain though I prefer that, and why terms will aim to stop forks without payment for the privilege. I'd be further along now, but I spent a long time trying to keep my skin intact. The good part about where I am these days is that I'm enjoying myself, and I'm free from any silly urge to trust others. :-)

     I don't have the problem I once had of promoting something I dreamed would help others, because I wanted to do good. I only need to explain enough to help my sons use what I have at any given time. In fact, I've incentive to say no more than necessary on how my code works after it's available. Most people can't figure out what code does without an explanation. I'm less generous these days.

site imagery ¶

     I added a paragraph to the license page stub to explain the images used on this site, since copyright on them is less clear; I overstated inclusiveness of copyright in the first draft of that page, and I'm trying to be precise. (I doubt I'll make more hand images: they take too much energy. When they run out, they run out. I can switch to something else.) All the images currently derive from content belonging to others. Maybe time I invested in them was waste.

needing sleep ¶

     I don't sleep as much as many folks. Bill Seitz says (in Sleeping and Snoring) he prefers at least nine hours of sleep. For most of my life I've needed only seven hours a night for perfect rest. On days with an hour gym workout and ten or twelve hours of coding, I might need eight hours for the same effect as seven. Otherwise with no alarm I wake rested after seven hours — weekends too. But getting only six is not enough, and makes me need five minute cat naps to sharpen my mind later in the day.

     For a couple years (we broke up a few months ago) I had a girlfriend who had trouble believing how little sleep I needed, since she needed about nine hours. At first she wondered if I was manic. (I told her Andy Hertzfeld asked me the same thing once, because he wondered if it might explain my output.) She's a therapist who worked for the city many years treating folks with a wide range of disorders, so she was intimately familiar with manic depressives. So at first she wondered if I was in a manic phase when she met me. But over time my sleep schedule didn't change: six 1/2 to seven 1/2 hours. In contrast, manic depressives have wild seesaws in sleep schedule in a regular cyclic period, with manic phases getting far, far less sleep. But that wasn't me.

     All her earlier boyfriends had complained about her having too much energy. She was always the person with the most energy in her experience until she met me. To her I was a near freak in energy level, only explained in her psychological experience in association with manic depressives. So I was on parole for a long time, until she saw I didn't come down. I'm just your basic steady high energy geek who gets melancholic without a drop in energy. Pissy yes, low energy no.

     On rare occasions when I do absolutely nothing for a day but relax and goof off (which is very hard — I just can't do this usually), I find I don't need as much sleep. Then I tend to wake after five to six hours feeling like I had seven. But that much down time bores me to tears. I keep busy because I get bored; doing nothing is painful. Watching television is sometimes like watching paint dry. Unless it's poker, or science. I find reading fiction more stimulating. (The reminds me: I need to add Rudy Rucker to the blog roll soon.)

hash tables ¶

     Yesterday a young coworker asked me why the hash tables I use aren't based on STL. He was surprised to learn std::map was a tree instead of a hashmap, and that actual hashmaps weren't yet standard. (The SGI version of STL has a non-standard hashmap.) Once we got past that, he asked who wrote the hash tables I use. I said I did. I told him even ten years ago I'd written dozens and dozens of hashmap implementations, and I tend to just toss one off whenever I want one. I make them fit exactly what I want done.

     Then he wanted to know what was the advantage to writing your own hashmap instead using one that was debugged. First we made a short token gesture at bashing STL, just to touch that base in a half hearted fashion. (Actually, I told him go ahead and use std::map unless you have a reason not to, like you're going to make a huge multithread safe hashmap whose every byte is used as efficiently as possible.)

     I told him when I fill a gigabyte of address space with many millions of hashmap entries in our product, my goal is to use every byte as efficiently as possible, because I'd rather fit more entries in the same space if I could, because hit rates affect our performance. Then I explained how using STL based data structures tend to use slightly more space per entry than something hand tuned to specifics.

     Then I explained how I need to control the size of critical sections when the hashmap is locked, for every operation we perform, including those visiting every hashmap member, without making any thread wait any longer than I could manage. I need to know exactly what happens in a critical section. And typically I want all space pre-allocated, because messing with the heap is the last thing I want to do.

     Hashmap lore just gets like this when the performance of your product is directly related to how well specific hashmaps work in the central parts of engines.

pretty code ¶

     Today Slashdot featured an article asking Any "Pretty" Code Out There?, to request examples of good C++ code style in applications found in the wild. The first sentence of the summary included this quoted sentence: "Y is unmanageable and really should be rewritten." (Which strikes me as funny since y is the modern equivalent of þ.) Many of the posts talk about tendency of code to become messy and unmanageable over time, even if it started out nice at first. There's something to this.

     I write prettier code at work. In my own personal code I write in a style ugly to many folks: short names, two space indent, and columns under 80. Worst of all, I stack multiple statements on one line. I often comment in such high density it impedes readability without syntax coloring. Here's an example from tokenizing code I wrote last week (parsing a symbol delimited by vertical bars):

     Don't expect code any prettier than this in þ, unless you find economy of effort a beautiful thing. There are places where complex things happen with few lines of code, in an elegant manner. But this isn't one of them.

     Each comment typically has a goal to explain "why" the code is present. I think presence of explanations makes pretty code. Complex methods can have comments on virtually every line. (At work this style helps anyone review my code: they can see what I was thinking.) When I edit code by other folks — usually without comments — I often add comments when I grasp everything.

     This is an example of very low level code that looks almost like C in its cheerful twiddling of member state. Sometimes that's pretty, but sometimes abstraction is better — it depends entirely on whether low or high level code is good in a situation. I use abstraction sparingly, in places I expect substitution.

operator overloading ¶

     I overload operator<<(yo&, T const&) for many types T in my personal code, and sometimes more than one time per type so a wrapper type can indicate whether I mean to show a mere one line cite, or a full debug print, etc. When the token returned by the code above is written with o << r_ltoken.quote(), the following result appears in the unit test output:

     One reason to print tokens this way is to see whether the token correctly knows range of input bytes specifying a symbol, as a separate matter from knowing just the symbol characters to be interned. Here you can see a debugger might know to select vertical bars as well as the content.

     While this looks nice, it's just a kind of pseudo XML that makes no attempt to be readable by an XML parser. (Yes I know some attribute values aren't quoted; do you think everyone else is stupid?) It's a habit I started a few years ago and haven't shaken. It can make you think I care about using XML tools when I don't; it's for my eyes only. Since I'm debug printing internal C++ representations of a Lisp runtime, it'd make sense to use s-expressions instead; I might later.

     In my own code, I try to make everything in C++ printable, because this a primary way I get evidence about what's happening: by printing values to inform myself about state. (There's usually not enough time for this in professional work, even though it's sorely needed.) I've often felt one of the main advantages of standard Lisp practice over C++ was the convention everything can be printed, since it affords easy means to see intermediary states when you wish.

     My out streams track current indent level, which enables one of the nicest good format effects in my printed output: indentation looks appropriate. (Yes, in multithreaded systems, interpretation of interleaved printing and indentation is interesting. But I'm not going to talk about that here. I might not even worry about in early versions of þ until I must. I don't enjoy explaining that sort of thing to others when I get too much argument. However, it's nice for call tree logging.)

templates etc ¶

     I don't actually like C++, but that's what I've been coding in professionally since 1989, basically full time continuously with a couple short interruptions. I try to use few C++ features, but still possibly too many for folks who want only C. Templates and exceptions are things I use as little as I can. By choice I'd only ever use templates for smartrefs in refcounting (which does involve operator overloading).

     The þ library uses templates for refcounting smartrefs, and otherwise has templates only in imitations of STL style types, for use only a rewrite C++ to þ is simpler using work-alike APIs. For the most part, I don't use them now though, so I don't debug or extend template code much. (Which means when you see some, it will look half done.) Later when I port more C++ code to þ, I'll use the þ templates to replace some STL classes.

reluctance ¶

     Today I'm in no mood to write, so I'm only writing to keep form with an arbitrary three day schedule. This doesn't interfere with being able to write. I discovered long ago writing is just as easy when you don't feel like it. If anything, lower standards ease the burden.

false consensus ¶

     I've a big list of things to write about, most related to each other, and most driven by some sociological perspective. I just tackle whatever floats to the top on any given day. So today we're doing false consensus, which concerns your tendency to assume other folks agree with you or — more accurately — to assume other folks are even remotely close to the same page as you.

     This is a large area to tackle at one time, so I'm really just broaching the topic. You might get sick of how much I return to this. I'll stop when you learn the lesson which is this: other people don't think like you, let alone plan similar targets, or pursue similar goals.

     In other words, it's completely your imagination that other folks are going to greet the future you want with open arms. Actually, they often have no ideas at all that resemble your own. But since they don't say anything, you interpret absence of evidence as support for your implicit mental models. Why is that?

     That's a rather optimistic way to interpret lack of evidence, don't you think? When other folks don't demontrate how much their world view differs from yours, you assume everything matches up pretty closely. Hmm, if you're lucky, it only leads to eventual surprise. But if another person has a sense of your assumptions, and wants to lead you around by the nose, it's going to be easy when you don't resist. Ouch. And we haven't even gotten to active disinformation. (It's less necessary when your passive errors are sufficient.)

     There I warned you of person to person pitfalls, but there are even more risks in assuming each person fits a stereotype, that each person you meet does what everyone does, or wants what everyone wants. (Does that sound bizarre already? Why would everyone be alike, anyway?) If you assume every collective social generalization applies to each specific person you see, you deserve what you get if it bites you.

     That's a pet peeve of mine: when folks believe social convention is enforceable prescription, where one size fits all. You meet many folks every day who believe, for example, the only motivation to do anything is to get more money, more status, or more whatever is best for you personally. And under an assumption everyone is like this — grubbing for themselves — it's perfectly okay to screw the other guy proactively, because they'd do it first if they could. Such a charming world to live in (not).

     It's strange to see folks playing by such different rules, yet everyone assuming the same game and same rules are agreed upon by all. By default, every day each thing you do is measured against preconceptions about what 'everyone' wants, and plans, and believes. (Some part of this might sound jejune; if so, you're missing the point. It's directly related to false consensus; I'm just belaboring it.)

nice people ¶

     I like my current coworkers. But then, I accepted the position originally for exactly this reason. My main selection criterion when job hunting is to find nice people I want to be around. Specifically, this rules out the opposite, which is assholes. (I know, I'm avoiding profanity on this site, but my kids have heard me say this a lot. I just never use this word at work, so it's kind of odd using it here.) I have quite a bit of experience with the latter at past jobs, so I'm good at recognizing them.

     (Instead of that a-word, let's use another closely related a-word as a code: arrogance. So when I talk about giving someone an arrogance test to determine AQ — arrogance quotient — you'll know I really mean the other word. In general, you want to work with someone whose IQ is bigger than their AQ, or else they're more trouble than they're worth. Sometime soon I'll write a page purporting to test your AQ, just for laughs.)

     I'll take nice people over more money any day of the week. You can easily predict what sorts of company might have more folks who self-selected themselves for higher AQs. They tend to be "in" places with lots of prestige; while nice folks work in those places too, they can attract a disproportiate number of ambitious folks with high AQs. Unfortunately, those places also attract a lot of really smart people. Just not all of them. You can find places with folks both smart and nice, like where I work now.

     To kill a company as a prospect, I only need to interview one person with a high AQ. Just one. It's a litmus test. During interviews you expect folks to be on better behavior than usual; if they can't avoid being a jerk then, when would they ever? Often folks who can't control it seldom bother to tone it down even slightly. They just go for it and hope the alpha dog thing will work for them. It makes "no" that much more obvious. (Let this be a warning to some of you high AQ folks: folks like me won't tell you rejection is because you're an ass.)

     The guy running the project I joined last summer was extremely collegial, as were all the other folks on the team. Things have been great. When we hire new engineers, we look for someone who's able to be just another one of the guys. A good attitude is one of the competencies we expect of qualifying candidates.

day job ¶

     I like my current day job. It's been very nice for many months now. You'll notice I don't say anything about them, and I'm not about to do so now or in the future. Who I work for is privileged information. But I do publically tell folks in passing what sort of thing I do: it's server side networking optimization in C++ on Linux, where I do almost all threads and locking, among other things.

     I spend a certain amount of time rewriting stuff, because that's what they need done. (If you create a thing now too complex for you to debug or factor any more, then I fix it so afterward it's stable, clear, and debuggable by everyone again, with enough new degrees of freedom to extend easily; then I extend it.) Sometimes I do algorithmic stuff, which means a better way to get it done pops into my mind; then I implement it and prove performance improvements.

     I also diagnose problems from evidence like stack traces, and think of ways to get new evidence to verify or falsify theories folks have about system behavior. Sometimes I write a comically large amounts of code, and occasionally I have architectural talks with the idea guys who don't get their hands dirty often in the code. I'm the guy who actually codes things after any discussion about what's a good idea. I like this role.

     A few months ago I took over the job of making final passes on resumes and making initial phone screens for candidates. I'm not a manager, but I spend a certain amount of time doing this because I seem to be good at it. I find it easy to get a detailed sense of someone's style of thinking after just a half hour talk. I anticipate what other team members will think of candidates, and I'm very good at seeing prime prospects we'd like to sell. I have good taste in engineers; my phone screens strongly predict a desire to make an offer.

     I like coding more though. And lately I've started getting to the interesting stuff. Looks like it's just going to get more fun over the next several months. I enjoy optimizing i/o and storage quite a bit, and now we're going to ramp up the scope one more notch.

moving lips ¶

     This section exists only as an introductory segue for the next. Here I'll describe part of my "interesting" life prior to full time computing. Before I returned to college in the late 80's, I was a data entry clerk in a medium sized law office where, among other things, I entered every bit of time billed to clients. Yes, I did pretend not to see dirt.

     (For example, one litigator would go to a deposition for ten hours, and bill those hours to one client. At the same time he'd take phone calls on behalf of other clients. By virtue of exaggerating length of phone calls, he could bill another eight hours to other clients. Fifteen to eighteen hours in such a day was not uncommon.)

     My youngest brother (I have three brothers and a sister) used to tell hilarious lawyer jokes. One of my favorites began by asking: "How can you tell when a lawyer is lying?" When his lips are moving, of course. Now, before anyone out there gets in a huff over this, let me tell you lawyers are off the hook in my book, since this behavior is in no way exceptional.

     It's true of most folks, not just lawyers. A person whose lips are moving is lying. Usually. There are exceptions, but it's uncommon. Very often the person doesn't know they're lying because they simply have a surreal world view that is false to fact. When they never make an effort to test anything, it's easy to coast along in a state of delusion. Ignorance is bliss.

     Perhaps lawyers do it more on purpose. I'd prefer not to test that myself, though. I became allergic to lawyers during my divorce. (I went through a phase where I watched every popular movie I could about lawyers.) My honesty made me too blind to crafty wolves.

     Please pardon my exaggeration as poetic license. There are honest folks, but they're a distinct minority. Many are geeks, who live steeped in a culture revolving around preservation and accuracy. These honest folks are often easy prey to normal folks who lie almost all the time, including your spouse and the typical power geek who moves and shakes (who calls it dealing).

compliers vs compilers ¶

     In this law office, I often manned the phones during lunch hours when the receptionist was away. I had fun learning to take overlapping phone calls at once, with interleaved interruptions putting folks on hold to take the next call, before slowly popping the call stack.

     During pauses between calls I sat reading the Dragon Book on compilers. It was one of my favorite things to read and re-read during the period of time between Berkeley and my later return to college. (This was around 1986/1987). Okay, so picture this: a book with a large title of compilers is sitting closed in front of me. What do you think the lawyers saw when they read the cover?

     At different times, three different partners stopped in fascination to discuss the book, saying the name with obvious relish and pleasure, intensely curious to know why someone would write such a thing. But each saw the same wrong word instead of "compilers"— to a man they read compliers: a book about people who comply.

     It was obviously a delicious idea to the lawyers. A book instructing you on how to make folks comply. They asked about it with big smiles. But they instantly lost interest when I corrected the mistake; then it became as boring as a book on gardening: something peasants do. Smiles gone, the sharks returned to cruise control.

scripted testing ¶

     During the 90's I tested virtually I every piece of software I wrote in C++ using the Scheme/Smalltalk hybrid interpreter I'd written in the early 90's. I generated glue code using a framework I'd written for foreign object interfaces — because it was object-oriented — along with a sed based scanner written more by Alan Liu than myself. (I worked with Alan at Taligent; he was a wizard at tweaking sed to target more C++ API edge cases.)

     So the glue code generation step was similar in purpose and nature to that in Swig, which came along a few years later in 1996. (I was doing this in 1992. Years later I had folks lecture me on how Swig works; do you know how irritating it is to have someone lecture you on a tool like one you invented? You keep saying, "I know...")

     I tried to share with other C++ coders, but they were consistently horrified by Lisp, so I stopped trying. All the psychic retching kept me from releasing what I'd done, because there seemed little interest. In retrospect, I still think I was right. The syntax was a problem.

     At some point over the years, it became clear many folks were using Python as this sort of scripting of C++ code, in order to get the same kind of interactive testing environment I had in Scheme/Smalltalk. But it was hard for me to stop using a tool I controlled. Why would I?

     Today there's even more languages playing in tech spaces where I work. So I feel devotion to just one scripting language makes even less sense now than ever. Since I know more about Scheme and Smalltalk, I'm most comfortable starting from there. But in principle it's no harder to transform Lisp into another syntax than it is to transform Lisp in one form to Lisp in another.

code vs data ¶

     I came across quasiquote code during an ongoing rewrite of my Scheme reader. I was reminded where I got the spec, when Mark Richer introduced me to Lisp around 1989 or so; he showed me the snippet of code below.

     I worked for Richer during my senior year, and I'd recently become excited by data-driven coding. Richer asked whether I knew Lisp, and I said no. He said I should look into Lisp because both code and data had the same format, and this was cool. I was intrigued.

     In essence, my interest in data-driven code was manipulating representations, explaining processes in terms of data flow, and accounting for all whys and wherefores in terms of inputs and outputs in a kind of demand economy, without resort to mysticism.

     Richer convinced me it was possible Lisp's code aided visibility into such transformations because code itself was data, in the same form as all Lisp data. The syntax of code was just more data, and was just as subject to analysis as any other data. It was a great hook for me.

     But first I tried rolling my own tool based on a vague idea that code and data were the same. I had almost finished writing Richer a btree storage system, and I was looking for a good way to test my system using some kind of scripting language. So I made up one like Lisp.

     My testing experience was fantastic and very effective. I was able to build up code coverage incrementally and interactively, and try out every edge case with little effort and immediate feedback. It worked so well I knew I wanted a real Lisp and not my toy; I was sold.

     I dug up the source code to an early public domain version of XLisp by David Betz and started tinkering, and basically never stopped hacking this and later versions I created along an arc passing through Scheme and into Smalltalk. I had a tendency to rewrite everything as I came to it, since I didn't care for Betz's original style, and I was a bit of a nut when it came to micro-optimization. Every few generations I started from scratch again, until by mid-90's I was trying to implement Dylan on a code base that had never shared anything with the XLisp tree, because it was based more on an imitation of the New Jersey SML runtime.

     Anyway, let's get back to quasiquote. Because I didn't want Richer to have a claim on my Lisp hacks, I explained I'd only be doing that on my own time. But he was still very interested and wanted to help out. Richer was enthusiastic about macros, so he explained this funky idea about partially quoting data using quasiquote. Then he fished out code to implement it from a book; it's listed in the next section. I folded this into the reader by transliterating it into C++. (The first thing I did with Betz's code was rewrite in C++.) The macros I eventually added worked just like MacGambit's macros†, pretty much through pure coincidence.

quasiquote ¶

     Below is the spec I use to expand quasiquoted expressions in my Lisp reader(s). But the actual code uses same language the reader (C++ just now). So this Lisp version is used to decorate the actual code with comments explaining the intended result.

     I wish I could cite the source of this code, but I have no idea who wrote it. (If you find out, let me know so I can cite it properly.) Amusingly enough, when I google for "quasiquote combine-skels", the top hit (of only two hits) is something I wrote myself on Wes Felter's website in 2001.

     In the absence of any other documentation at this time, below I've quoted the long comment I posted on Wes' site to explain the quasiquote code. Sometime later I'll write a shorter, more lucid version, probably when I write this part of the reader again, since at that point I'll fully bring it in mind again. It might enliven an otherwise dull reader description.

     As you can see, I wrote all this in the typical prose style I used in those days, which emphasized breaking up complex ideas into small bites of single-line, short sentences. That's not quite as clear as the style I'm using here. But my new mostly conversational style is partly more clear because I imagine readers a little differently now. Note what's best for you is usually also what's best for me; less taxing is just better.

     (Dear reader, do you wonder how your model changed? It's many things, but most striking is an assumption of shorter attention span, but one degrading less severely given long chains of related ideas, so continuity bolsters attention. I think I also assume you care less, ultimately.)

short term memory ¶

     My short term memory seems to work unlike yours, as near as I can tell. Comparison's not easy.

     Most people act like they're being slapped every ten or twenty seconds or so, with time immediately prior slightly harder to recall, as if looking through a haze, with further steep decay over hours and days. I had my first experience with this three years ago at a party in the city.

     They served brownies containing an ingredient I had never tried before, and my girlfriend and I both helped ourselves. Apparently it was a strong batch, with more effect than amateurs manage. After an hour I reached a state where my short term memory reset about every ten or fifteen seconds, and I had an epiphany. This would explain why so many people seem so stupid: they're forgetting things constantly, even as it's happening.

     Though I wouldn't care to ever try it again (forgetting is just too strange), I found that one experience very enlightening. It was one more clue to better grasp how the world looks to folks closer to the norm. In practical terms, it usually affects how much I expect people to remember at work, which is not much beyond sketches. As a result, I don't waste as much time, which is constructive.

     Incidentally, I can tell who remembers: it roughly corresponds to a group of people I had previously thought of as more awake and more present than others, which is substantially correct, since they aren't taking mini-sleeps minute to minute. What were we talking about?

     Then there's folks who remember everything in three weeks if they still recall it after two minutes. Of course, as you get older it gets steadily harder to wade through all the crap you remember. And it doesn't take quite as well either, like silly putty that's been used a few too many times. Where'd I put my false teeth again?

diamond age ¶

     I just noticed Bill Seitz has a page about Neal Stephenson's Diamond Age, which affected me so strongly ten years ago, and to which I've been making references now and again. (Bill and I conversed recently, and I'm happy to see something at his place I enjoy discussing.)

     Coincidentally, Stephenson and I were born in the same state in the same year, not terribly far apart, and we both lived in the same city for a while, named in Diamond Age as the birth place of Lord Finkle McGraw (cited as M by Seitz). McGraw also grew up there during the same time period as Stephenson and myself. You have to wonder if we share some of his sensibilities. I certainly do, anyway.

     Seitz quotes a fun part of the book (leading up to the word subversive), where Finkle McGraw and Hackworth attempt to address the way in which leading a life at variance with the straight and narrow (the easy and comfortable way, perhaps) might make you into a sort of person better suited — perhaps temperamentally — for doing interesting work.

     I was quite taken with that particular conversation beween Finkle McGraw and Hackworth, because I knew Stephenson was on to something important, since I'd experienced myself something very like the process discussed. The better bespoke engineers had unusual life histories, with sometimes eccentric training paths.

meat packing ¶

     I resemble my maternal grandfather in many ways. He was an uneducated jack-of-all-trades shop mechanic who worked many years in a midwestern meat packing plant — now defunct — that specialized in slaughtering pigs. (So it goes.) He made things with his hands, like a trailer camper he designed from scratch, and he gave me the first set of magnets I loved as a boy. His oldest son, my uncle, was a first rate electrical specialist. I look like both of them.

     This is actually a story about the pigs. I never saw them, but I smelled them every time I crossed the river in my parents' car to visit my grandparents. The bridge passed right by the plant, and the smell of cooked pork was pervasive. I was never able to pass without pondering the quaint traditional manner we used to turn living creatures into packaged food. I never wanted to tour that plant.

     I'm just giving you something to think about, so you can wonder whether the name of my site and recurrent themes on predator, prey, and food, have anything to do with my not-even-slightly traumatic near-pig experiences. (Did you think I was going to go all silence of the pigs on you?) Actually, I'm just messing with you a little. And I'm baiting you to try on your hick bias for size, to see if it still fits. I've noticed many west coast raised geeks have serious social darwinism issues, and assume spawn of midwest working class means genetic lowlife; am I slightly warm? You see any problem with house painters and mechanics siring bright coders? No? Why not?

     To be truthful, I'm a little different from all my siblings: more mentally than physically by a couple standard deviations. Enough so you'd think my father was the milkman, or something. But I sound just like my father, and I looked like a twin of one brother, until we later diverged a lot. Genetics is pretty random, and nurture has a lot to do with how we end up too. As Hackworth might say, leading an interesting life can sometimes change your basic nature.

     Both my sons have ability at math, and the younger also seems to have gotten the near photographic memory I enjoyed through college. The older son seems to have more of the imagination. I've been trying to inculcate in them a temperament to use their own judgment when they suspect someone else is wrong. At an early age I began stretching their minds by inserting random nonsense into stories I read them, which they rapidly saw did not belong. Repetition made them faster. They've since grown up to be quite expert at instantly recognizing any plausible invented nonsense I try on them (but they think it's feeble jokes rather than mental calisthenics).

     I'm hoping when grown, they're less inclined to do what they're told, even though I sometimes find this quality something of a trial now. Their friends often seem so slow and credulous in comparison, so maybe it's working.

language dialects ¶

     A while ago on the λσπ page, I said I wasn't making a new programming language. Since then I wondered whether I was accurate; technically, it changes a language when you add something surprising. So folks expecting Scheme in Lathe mode might consider this dialect significantly different, based on surprises.

     Differences might be enough some folks insist it's not Scheme. Okay. I'll try to avoid saying it's plain vanilla Scheme. But I hope to have a plain vanilla Scheme embedded inside, with Lathe (λþ) as a superset.

smalltalk objects ¶

     The object system in Lathe will be a Smalltalk runtime, so for example, every object will be an instance of some class, just like in Smalltalk. This will be orthogonal to — and in addition to — Scheme types. Eg, each cons cell will be an instance of a pair class. You can extend system classes with new methods (unless they're frozen). It'll be turtles all the way down.

     I did exactly this same thing in an earlier system I developed around 1993 or so, and I liked it; this is why I'm doing it again: to suit myself. I didn't find it confusing or contradictory for values to have simultaneous Scheme and Smalltalk semantics. However, dispatch might look odd to folks with deeply ingrained Scheme habits.

scheme evaluation ¶

     When evaluation order of λþ differs significantly from Scheme, it might make λþ seem a different language. Before Lathe evaluation, first let's consider Scheme's evaluation. (Just pretend all code is interpreted.)

     When Scheme evaluates a list, first it looks at the first element — then evaluation goes one of two directions, depending on whether the first list member is a symbol denoting a special form, which means not all subsequent arguments are necessarily evaluated. For example, an (or …) special form does short circuit evaluation, and stops after the first true value. (Note I'm ignoring macros — pretend they're special forms for the purposes of this discussion.)

     Otherwise, when a first list member is not a special form symbol, the first expression must evaluate to a procedure instance called with all the following evaluated arguments as parameters. In this case, the value of a list expression is whatever value the call returns.

     So Scheme understands special forms (or macros) and procedure calls. Anything else in the first position of an evaluated list is an error. The λþ dialect extends Scheme by giving meaning to some of the error cases: when they look like a message send — receiver followed by symbol & args — message dispatch occurs instead of error.

lathe evaluation ¶

     Scheme uses strictly prefix notation, but not λþ. When the first member of an evaluated list is a special form or evaluates to a procedure, λþ will do the same thing as Scheme. But any other object type in the first position is expected to be a Smalltalk message send — in this case, the second list member must be a symbol, or else an error occurs. Obviously message dispatch is not prefix notation, since a message selector is second.

     Consider the following example.

     This expression might evaluate to true: both subexpressions may call exactly the same code. Sending at: to a vector instance might look up and invoke the procedure also bound to vector-ref. In other words, the following might be true:

     This might bother Schemers a lot, because it creates an entirely new possible meaning for a list expression. Diversity in code expression has good and bad points: the downside is less clarity given more options.

     (Clearly it would be simple to disable Smalltalk style message dispatch when folks are trying to write portable Scheme. So let's not get excited.)

explicit sends ¶

     So how would you send a message to a procedure? Since they're objects too, you can send them messages. But if you can't put a procedure in first position without calling, how do you send a message instead?

     All you need is a new special form that means send. Then you use this special form to send to procedures. All message sends would look like this, but with the send symbol missing and understood.

     I'm inclined to use a very short special form name for send. Suppose you wanted to ask a lambda procedure for its environment. You might do it like this:

     This assumes period-colon (.:) is the special form symbol for send. I've been planning to use period (.) as a prefix on various new non-evaluating special forms, as a convention to help emphasize no evaluation. And the colon is reminiscent of Smalltalk keyword methods.

     As a reader exercise, what do you think I might name the super-send special form? Is it obvious? Add either another period or another colon.

new language ¶

     Is λþ a new language, or is it just a dialect of Scheme? I lean toward the latter. You might beg to differ. It's all the same to me. I just want to get the labels right. Or I want to get the fuzzy ambiguity clearly enunciated.