Monthly Archives: June 2024

Article note: Apple rejects anything that contains the string "itms-services" so including handler libraries that have it in the list of possible services trigger rejection. They don't document this fact so someone had to reverse engineer it. The corporate controlled coercive content consumption jails that computers have turned into are so fucking disappointing, especially from the company that used to talk about "Wings for the Mind."

Article note: Maximum greasy.

Article note: I've seen recent evidence that public chatterbots can beat the _median_ student at "pick all that apply" questions in computer engineering, in the sense that bots can swing a C and the students did their best to absorb nothing from the class. The real deal is you _can not_ treat summative assessments that aren't regularly revised and taken in a controlled environment as reliable; the stakes and culture are such that a large fraction of students will "optimize" (cheat) to the best of their ability. Which sucks for instructors who have to deal with editing and scheduling and grading old-school exams, and students with test anxiety and such, but less than getting out-competed by a swarm of know-nothing bros whose test taking strategy only requires "Ctrl" "c" and "v."

Article note: Heh, it's a cute little form-factor, and the actual computer is a neat exercise in minimalism. Its "ALU" is a 2 input NOR gate that it bit-serial recirculates. About 20 ICs none of which are designs from after the 80s.

Article note: It isn't really matrix-math-free, it's just matrices of ternaries. That said, I'm a fan of small-range systems for sloppy approximators, (-1,0,1) ternaries map well to LLMs, and not using huge, expensive, power-hungry monstrosities for dumb bullshit is a win for everyone.

Enlarge (credit: Getty Images)

Researchers claim to have developed a new way to run AI language models more efficiently by eliminating matrix multiplication from the process. This fundamentally redesigns neural network operations that are currently accelerated by GPU chips. The findings, detailed in a recent preprint paper from researchers at the University of California Santa Cruz, UC Davis, LuxiTech, and Soochow University, could have deep implications for the environmental impact and operational costs of AI systems.

Matrix multiplication (often abbreviated to "MatMul") is at the center of most neural network computational tasks today, and GPUs are particularly good at executing the math quickly because they can perform large numbers of multiplication operations in parallel. That ability momentarily made Nvidia the most valuable company in the world last week; the company currently holds an estimated 98 percent market share for data center GPUs, which are commonly used to power AI systems like ChatGPT and Google Gemini.

In the new paper, titled "Scalable MatMul-free Language Modeling," the researchers describe creating a custom 2.7 billion parameter model without using MatMul that features similar performance to conventional large language models (LLMs). They also demonstrate running a 1.3 billion parameter model at 23.8 tokens per second on a GPU that was accelerated by a custom-programmed FPGA chip that uses about 13 watts of power (not counting the GPU's power draw). The implication is that a more efficient FPGA "paves the way for the development of more efficient and hardware-friendly architectures," they write.

Article note: It's an absurd $15,000 OAM module with 192GB of RAM on it... but it's also benching out around twice as fast as an Nvidia H100 (and drastically better in some circumstances), which is a $25,000 OAM module. ...if your software will run on AMD's toolchain(s), it's clearly a wildly preferable part, but the question is how many workloads is that reliably true of after all the pooch screwings they've made on the software front.

Article note: Yesssss. The AI Hype is dangerously out of hand, and this is signs of correction.

Article note: Oh that's really sad that the LCM is being dismantled. Nice that SDF is adopting their emulated remote experiential things. Will be interesting to see who can provide a good home for the white elephants like the CDC6500 and KI10.

Article note: Dang, that has been a long (and frequently absurd) saga, and it just lurched to a halt.