Author Archives: pappp

Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Source: Ars Technica

Article note: It isn't really matrix-math-free, it's just matrices of ternaries. That said, I'm a fan of small-range systems for sloppy approximators, (-1,0,1) ternaries map well to LLMs, and not using huge, expensive, power-hungry monstrosities for dumb bullshit is a win for everyone.
Illustration of a brain inside of a light bulb.

Enlarge (credit: Getty Images)

Researchers claim to have developed a new way to run AI language models more efficiently by eliminating matrix multiplication from the process. This fundamentally redesigns neural network operations that are currently accelerated by GPU chips. The findings, detailed in a recent preprint paper from researchers at the University of California Santa Cruz, UC Davis, LuxiTech, and Soochow University, could have deep implications for the environmental impact and operational costs of AI systems.

Matrix multiplication (often abbreviated to "MatMul") is at the center of most neural network computational tasks today, and GPUs are particularly good at executing the math quickly because they can perform large numbers of multiplication operations in parallel. That ability momentarily made Nvidia the most valuable company in the world last week; the company currently holds an estimated 98 percent market share for data center GPUs, which are commonly used to power AI systems like ChatGPT and Google Gemini.

In the new paper, titled "Scalable MatMul-free Language Modeling," the researchers describe creating a custom 2.7 billion parameter model without using MatMul that features similar performance to conventional large language models (LLMs). They also demonstrate running a 1.3 billion parameter model at 23.8 tokens per second on a GPU that was accelerated by a custom-programmed FPGA chip that uses about 13 watts of power (not counting the GPU's power draw). The implication is that a more efficient FPGA "paves the way for the development of more efficient and hardware-friendly architectures," they write.

Read 13 remaining paragraphs | Comments

Posted in News | Leave a comment

A buddy of mine dropped me a link to TI’s new brushed DC motor driver with sensorless speed control, the DRV8214. I spent a few minutes trying to hunt down the mechanism they use to derive speed from back EMF … Continue reading

Posted on by pappp | Leave a comment

Testing AMD’s Giant MI300X

Source: Hacker News

Article note: It's an absurd $15,000 OAM module with 192GB of RAM on it... but it's also benching out around twice as fast as an Nvidia H100 (and drastically better in some circumstances), which is a $25,000 OAM module. ...if your software will run on AMD's toolchain(s), it's clearly a wildly preferable part, but the question is how many workloads is that reliably true of after all the pooch screwings they've made on the software front.
Comments
Posted in News | Leave a comment

Nvidia loses a cool $500B as market questions AI boom

Source: The Register

Article note: Yesssss. The AI Hype is dangerously out of hand, and this is signs of correction.

Cisco was briefly the world's most valuable company too, you know, just before the dot com bust

Nvidia has rapidly lost about $500 billion off its market capitalization amid concerns that the GPU maker may have become overvalued or that the AI market powered by its chips is a bubble set to burst.…

Posted in News | Leave a comment

LHM to permanently close and sell DEC-10 at auction

Source: Hacker News

Article note: Oh that's really sad that the LCM is being dismantled. Nice that SDF is adopting their emulated remote experiential things. Will be interesting to see who can provide a good home for the white elephants like the CDC6500 and KI10.
Comments
Posted in News | Leave a comment

Julian Assange has reached a plea deal with the U.S., allowing him to go free

Source: Hacker News

Article note: Dang, that has been a long (and frequently absurd) saga, and it just lurched to a halt.
Comments
Posted in News | Leave a comment

Apple first company to be found violating DMA

Source: OSNews

Article note: NicCageYouDontSay.jpg Apple's behavior has been actively and intentionally baiting the EU over this, doing the most malicious semi-compliant thing they can come up with at each step.

Today, the European Commission has informed Apple of its preliminary view that its App Store rules are in breach of the Digital Markets Act (DMA), as they prevent app developers from freely steering consumers to alternative channels for offers and content.

In addition, the Commission opened a new non-compliance procedure against Apple over concerns that its new contractual requirements for third-party app developers and app stores, including Apple’s new “Core Technology Fee”, fall short of ensuring effective compliance with Apple’s obligations under the DMA.

↫ European Commission press release

File this in the category for entirely expected news that is the opposite of surprising. Apple has barely even been maliciously compliant with the DMA, and the European Commission is entirely right in pursuing the company for its continued violation of the law. The DMA really isn’t a very complicated law, and the fact the world’s most powerful and wealthiest corporation in the world can’t seem to adapt its products to the privacy and competition laws here in the EU is clearly just a bunch of grandstanding and whining.

In fact, I find that the European Commission is remarkably lenient and cooperative in its dealings with the major technology giants in general, and Apple in particular. They’ve been in talks with Apple for a long time now in preparation for the DMA, the highest-ranking EU officials regularly talked with Apple and Tim Cook, they’ve been given ample warnings, instructions, and additional time to make sure their products do not violate the law – as a European Union citizen, I can tell you no small to medium business or individual EU citizen gets this kind of leniency and silk gloves treatment. Everything Apple is reaping, it sowed all by itself.

As I posted on Mastodon a few days ago:

The EU enacted a new law a while ago that all bottle caps should remain attached to the bottle, to combat plastic trash.

All the bottle and packaging makers, from massive multinationals like Coca Cola and fucking Nestlé to small local producers invested in the development of new caps, changing their production lines, and shipping the new caps. Today, a month before the law goes into effect, it’s basically impossible to find a bottle without an attached cap.

I don’t know, I thought this story was weirdly relevant right now with Apple being a whiny bitch. Imagine being worse than Coca Cola and motherfucking Nestlé.

↫ Thom Holwerda

Apple is in this mess and facing insane fines as high as 10% of their worldwide turnover because spoiled, rich, privileged brats like Tim Cook are not used to anyone ever saying “no”. Silicon Valley has shown, time and time again, from massive data collection for advertising purposes to scraping the entire web for machine learning, that they simply do not understand consent. Now that there’s finally someone big, strong, and powerful enough to not take Silicon Valley’s bullshit, they start throwing tamper tantrums like toddlers.

Apple’s public attacks on the European Union – and their instructions to their PR attack dogs to step it up a notch – are not doing them any favours, either. The EU is, contrary to just about any other government body in the Western world, ridiculously popular among its citizens, and laws that curb the power of megacorps are even more popular. I honestly have no idea who’s running their PR department, because they’re doing a terrible job, at least here in the EU.

Posted in News | Leave a comment

Fedora has been shipping with a broken screen reader for nine years

Source: Hacker News

Article note: It's such a perfect microcosm of accessibility (and platform in general) discussions. _Everyone involved_ is using intentionally charged language and being an asshole. Fedora is - alternately - entirely controlled by IBM or just some poor volunteers doing a community project, depending on which is narratively useful. "Patches welcome" is pretty reasonable for hobby projects... but less so after they tossed a scheme that the worked for affected users, and even less so for commercially supported projects right under a banner declaring how they're making accessibility a priority. The old "bad" X11 system is janky but works, but the new "better" Wayland system is so poorly thought out and inflexibility "securitized" it's not clear if it will ever be able to reach feature parity without an extensive redesign that will undercut other promises. And so on.
Comments
Posted in News | Leave a comment

Some fundraisers pay >90% of the funds to themselves

Source: Hacker News

Article note: Our culture is _packed_ with fake charitable efforts. Every "Round up your purchase for our corporate tax deduction," a tide of purpose made fraud-adjacent spend-millions-to-make-thousands charities, a large proportion of churches (especially if measured by dollars), commercial middlemen who do fundraising-as-a-service and pocket most of it, "medical service foundation" structures, ... Late stage capitalism is a hell of a drug, everything is an extractive hustle. It's hard to envision how we're going to rein this shit in.
Comments
Posted in News | Leave a comment

Review of Linux on Minisforum V3 AMD Ryzen Tablet

Source: Hacker News

Article note: I've been playing with some Minisforum Venus 690 variants (SFF Ryzen boxes) for a side project on campus lately, they are really nice in terms of design and function (but the plastics feel just a hair cheap). Way better cooling/acoustics than I expected. I didn't know they were doing laptops and such, and I'm also interested in a small carryin' around laptop that offers a full Linux environment and also has decent stylus support, so this thing is intriguing.
Comments
Posted in News | Leave a comment