Category Archives: School

Warewulf, and the Kernel Documentation

Posted on 2012-01-05 by pappp

Earlier, while trying to instrument a failing boot from some peculiar nodes we were trying to provision, I came across the following gem in the Linux kernel documentation, from Documentation/filesystems/ramfs-rootfs-initramfs.txt:

Note: The cpio man page contains some bad advice that will break your initramfs archive if you follow it. It says “A typical way to generate the list of filenames is with the find command; you should give find the -depth option to minimize problems with permissions on directories that are unwritable or not searchable.” Don’t do this when creating initramfs.cpio.gz images, it won’t work. The Linux kernel cpio extractor won’t create files in a directory that doesn’t exist, so the directory entries must go before the files that go in those directories. The above script gets them in the right order.

Yup. If you follow the documentation for the tool, it renders your system unbootable. The linked documentation is actually pretty cool – it explains the rationale for the current state of the boot process, including that charming behavior, and links to the original discussions. But the particular behavior is still kind of psychotic.

Upside: After today’s digging I know all kinds of neat things about the current Linux boot process, which I hadn’t relearned after it changed at the the 2.4/2.6 transition. Similarly, the last couple times we had problems with Warewulf 3 (or, actually, Redhat-isims interfering with Warewulf) brought me back up to speed on interpreting raw packet logs from Wireshark, so this has all been thoroughly educational.
Downside: I have even less idea why the nodes won’t finish booting – the check I was adding was to test our theory that they were running out of memory, and they don’t seem to be.

Posted in Computers, DIY, General, School | Tagged , | 1 Comment

Teaching Embedded Systems (with Arduino)

Posted on 2011-12-14 by pappp

Now that the classes are winding down, I want to write up some internet-accessible notes about the embedded systems unit I designed and taught for EGR199 this semester. The unit went well, and I can see basically the same materials being reused, so having a nice content dump for me or any other instructor to use is worth the effort. Long winded version after the fold.
Continue reading →

Posted in DIY, Electronics, General, Objects, School | Leave a comment

Bookish Dreaming

Posted on 2011-11-27 by pappp

I only remember a dream every year or so, but I realized on the way back from getting breakfast with friends this morning that a book I thought I’d been reading was entirely in a dream. It was a long dream with various dream-space fucked-up-ness to the setting building (No University space is that large, that nice… or has three large highly styled cafe/lounge spaces in the same complex) and interaction with various old acquaintances, but there was one section I didn’t realize was a dream because it was so normal:
< dream content >
I picked up a book (roughly A4 sized, and inch or so thick, nicely black cloth-bound with gold embossing) about code generation for a particular class of exotic hybrid-SIMD machines (I remember details, which are realistic, but not specific enough to pick out which machine) by David Padua (respected figure in parallel computing, who I’ve met at conferences) and a coauthor I couldn’t remember when I woke up. I got the book from a well stocked engineering library, and discussed it with various engineering types I know, including my current adviser.
< /dream content >
Until we were headed back from breakfast and I realized the setting was “improbable,” I was sure it had happened. When I got home I had to see if it was something I may have seen referenced – the content and authors were probably based on “Optimizing data permutations for SIMD devices.” which I read a year or two ago, but it isn’t an exact match. The description I remember also matches a section in Encyclopedia of Parallel Computing (four volume, $1500) book that I’ve never seen before (and now want access to). I also want the dream book, because it would be all kinds of useful for my MS project.
Aren’t brains interesting…

Posted in Computers, General, Navel Gazing, Objects, School | Tagged | Leave a comment

Headed out to SC11 in Seattle, WA. for the week. Technical interest, travel complaints, booth hacks, advertising mockery, and schwag to follow.

Posted on 2011-11-12 by pappp | Leave a comment

Source Scroller

Posted on 2011-11-10 by pappp

I’ve been working on a side project to make a source display widget for the aggregate.org SC’11 exhibit, and it is surprisingly problematic. The goals for the program are as follows:

  • Take a directory of source files as input
  • Automatically perform appropriate syntax highlighting on the source files.
  • Display (On a dark background – intended use is a floating projection display) all the source files consecutively.
  • Scroll automatically through the output in a pleasing, readable way.
  • Be capable of repeating this action unattended indefinitely.

Continue reading →

Posted in Computers, DIY, General, School | Leave a comment

A Day in the Life of the KAOS Lab Thought Process

Posted on 2011-10-28 by pappp

In which a simple “Do you know an easy way to convert an integer into a string of (character) digits?” turned into an expedition into ancient UNIX codebases. The process went something like this:

Labmate: Do you know an easy way to convert an integer into a string of digits?
< both of us type it in to google>
Looks like sprintf() is best, there must be a simple efficent algorithm.
How does printf() do it?
Let’s look in libc!
< grep through the various toolchain sources on my system >
Well, here’s the uClibc implementation… which is a terrifying mess of ambigously named function calls and preprocessor directives.
I wish I had some old UNIX sources to look at, that would be simple.
< a bit of googling later>
Holy crap, this is amazing! Full root images for a bunch of early UNIXes, many of them from dmr himself!
< download v5, grep /usr/source judiciously to find /usr/source/s4/printf.s>
Well crap, it’s in PDP-11 assembly, maybe a later version.
< download v7, grep /usr/src judiciously to find /usr/src/libc/stdio/doprnt.s>
Damn, still in PDP-11 assembly, but this is a fancier algorithm.
Hmm… the most understandable UNIX I’ve ever looked at was old MINIX…
< spin up BasiliskII, in ‘030 mode, use this workaround to make MacMinix run>
< more grepping for justice>
Eventually, we found the printk() from MacMinix 1.5, in all its awful K&R/ANSI transitional C glory

...
#define NO_FLOAT

#ifdef NO_FLOAT
#define MAXDIG 11 /*32 bits in radix 8*/
#else
#define MAXDIG 128 /* this must be enough */
#endif

_PROTOTYPE(void putc, (int ch)); /*user-supplied, should be putk */

PRIVATE _PROTOTYPE( char *_itoa, (char *p, unsigned num, int radix));
#ifndef NO_LONGD
PRIVATE _PROTOTYPE( char *_itoa, (char *p, unsigned long num, int radix));
#endif

PRIVATE char *_iota(p, num, radix)
register char *p;
register unsigned num;
register radix;
{
register i;
register char *q;
q = p + MAXDIG;
do {
i = (int) (num % radix);
i += '0';
if (i > '9') i += 'A' - '0' - 10;
*--q = i;
} while (num = num / radix);
i = p + MAXDIG - q;
do
*p++ = *q++;
while(--i);
return(p);
}
...

Which is, of course, a digit at a time in one of the most straightforward ways imaginable. Minix’s kernel is designed for systems so resource constrained it has a separate prints() that can only handle char and char* to save overhead, so I can’t imagine it uses a sub-optimal technique.
This kind of thing really makes me wish I had learned OSes in the old death-march through the UNIX sources (or at least the old Tenenbaum book with MINIX) way; things are too complicated and opaque now. There always seems to me to be a golden age from around 1970 into the early 1990s where the modern computing abstractions were in place, but the complexity of production hardware and software hadn’t yet grown out of control.
In a related note, as a tool writer, looking at the earliest versions of cc is AMAZING. Most decent programmers should be able to work through the cc from v5 UNIX (574 lines of C for cc proper, 6454 more lines of C and 1606 of PDP-11 assembly in called parts) in a couple hours, and fairly fully understand how it all works. Sadly, (pre-3) MINIX came with a (binary only) CC made with ACK, which is fancy and portable and way, way, way harder to understand. dmr’s simple genius was just that.

Posted in Computers, Entertainment, General, School | Tagged , , | Leave a comment

Power Factor Corrector

Posted on 2011-10-04 by pappp

We’ve been tracking down the failure mode of power supplies in the clusters on campus, and picked up a plug-in “Power Saver” power factor corrector box for around $5 to look at in our experiments. Prices on these things range from about $5 (less than the cost of the components in small quantities… and in a nice wallwart case – this is what we paid) to over $70 (fleecing the morons).
This particular device is a “PowerSaver PowerStar CHT” (or some similar random string, the model is “CHT-001A”), about which a variety of bemusingly improbable claims are made.


Upon opening it up, it contains a large (5uF,450V) capacitor, two LEDs, three quarter-watt resistors, and a single-sided PCB with “Comment” all over the silkscreen where fields were not filled in. As best I can make out, the resistors are a very high impedance voltage divider to step down the 120V/60Hz from the wall to a level the LEDs can handle, and the LEDs are acting as their own rectifier. The capacitor is at least connected across the outlet. Curiously, only about half the board is populated, and I can’t figure out what some of the missing components would do – they appear to be a second independent indicator LED circuit rotated 90 degrees from the populated one.
Under a very limited set of circumstances (linear inductive load, like an AC motor) these things could actually help the power factor, but since most residential power billing is net, with no power factor penalty, it wouldn’t actually reduce bills. In the case of non-linear loads, like say, computer power supplies, it will do nothing useful. Experiment complete, although it may still be plugged in while instrumented for funsises.

Posted in Computers, DIY, Electronics, Entertainment, General, Objects, School | Tagged , , | Leave a comment

Principles of Programming Languages

Posted on 2011-09-17 by pappp

I referred to Bruce J. MacLennan’s Principles of Programming Languages in my previous post, then discovered I can’t find the list anywhere on the ‘net, at least not as I was taught them. They are (AFIK) derived from his book, also named Principles of Programming Languages, so it may not be precisely kosher to put them online, but I’m going to post a copy anyway, because they are terribly useful to refer to.

  1. Abstraction: Avoid requiring something to be stated more than once; factor out the recurring pattern.
  2. Automation: Automate mechanical, tedious, or error-prone activities.
  3. Defense in Depth: Have a series of defenses so that if an error is not caught by one, it will probably be caught by another.
  4. Elegance: Confine your attention ot designs that look good because they are good.
  5. Impossible Error: Making errors impossible to commit is preferable to detecting them after their commission.
  6. Information Hiding: The languages should permit modules designed so that (1) the user has all of the information needed ot use the module correctly, and nothing more, and (2) the implementer has all the information needed to implement the module correctly, and nothing more.
  7. Labeling: Avoid arbitrary sequences more than a few items long. Do not require the user to know the absolute position of an item in a list. Instead, associate a meaningful label with each item and allow the items to occur in any order.
  8. Localized Cost: users should pay only for what they use; avoid distributed costs.
  9. Manifest Interface: All interfaces should be apparent (manifest) in the syntax.
  10. Orthogonality: independent functions should be controlled by independent mechanisms.
  11. Portability: Avoid features or facilities that are dependent on a particular computer or a small class of computers.
  12. Preservation of Information: The languages should allow the representation of information that the user might know and that the compiler might need.
  13. Regularity: Regular rules, without exceptions, are easier to learn, use, describe, and implement.
  14. Responsible Design: Do not ask users what they want, find out what they need.
  15. Security: No program that violates the definition of the language, or its own intended structure, should escape detection.
  16. Simplicity: A language should be as simple as possible. There should be a minimum number of concepts, with simple rules for their combination.
  17. Structure: the static structure of the program should correspond in a simple way to the dynamic structure of the corresponding computations.
  18. Syntactic Consistency: Similar things should look similar, different things different.
  19. Zero-One-Infinity: The only reasonable numbers are zero, one, and infinity.
  20. Avoid Creeping Featurism: Do not add features which are expensive to implement and rarely used. [I think Rafi added this one himself, but I learned them from him, so here it stays. Also, I feel like there was a more elegant phrasing presented at some point.]

I’m not entirely fond of all the rules, particularly 6 and 7, which encourage trusting other people’s code and space inefficiency respectively, but it is generally a wonderful compact way of evaluating languages, that should resonate with anyone who programs.

Posted in Computers, General, School | Tagged | Leave a comment

Too Much Toolchain Time

Posted on 2011-09-04 by pappp

Having my “work project” be finishing the code generator and simulator fronted for LARK, and my “play project” be coaxing uClibc and TCC to build for Android (demonstrably possible, it’s just a question of if I’m clever and/or tenacious enough…) is giving me a little too much time working with compiler toolchains to be healthy. It does make me understand how compilers people get their — characteristic — personality traits, though.

Posted in Computers, DIY, General, Navel Gazing, School | Leave a comment

JSTOR copying

Posted on 2011-07-20 by pappp

Open-Access Advocate Is Arrested for Huge Download
I’m pretty sure I don’t have a big problem with this kind of radical openness, especially with regard to journal publications… although that was a disgustingly unsubtle technique. I’ve always pictured a distributed project of the “Share your documents (and possibly your subscriptions)” darknet forming style… all the contacts are already in place, since that is how academics have been sharing harder to get publications for as long as those terms have had meaning, we just need a protocol to streamline finding and swapping over the ‘net. There is no excuse for journal access still being so closed and expensive.

Posted in Computers, General, School | Leave a comment