artagnon.com

2043

2020-09-01T16:54:09+02:00

"I'm sorry, but we have to let you go", she says. "Right. What do I need to ink?", I respond, prepared. I don't need this job to pay the bills, and I don't enjoy it. I'd been contemplating resignation for a while now, but a fat severance cheque can't hurt. The HR woman hands me a crisp document on her tablet. "Do you know what you'll do next?", she asks robotically. My mind wanders, and I flippantly say "Maybe I'll move to Antarctica and take up flame-throwing". "Oh, good luck with that", she says, with a meaningless smile. Another robotic response. The lord wonders why she's still employed.

I wander around the sterile campus filled with man-child millionaires typing away furiously on their notebooks, one last time. The memory of the insipid lunch I had at the canteen lingers. Looking forward to a good celebratory dinner though. I have a foggy recollection of the templated interviews I had to sit through eight months ago — it's dizzying to think that I survived here for the better part of a year.

As I head back home on the loop, I carelessly browse some blog entries by people who quit the company. It's been done to death. Everyone thinks that their reasons are somehow more unique or insightful than those of the others, but they all boil down to one reason: stagnation. They all had dreams of becoming billionaires, but got stuck in middle-management. Of course, the posts carefully leave this detail out — instead they talk about "growth". They sound like vegetables in a garden competing for sunlight and soil.

A little electric transports me from the loop station to my suburban home. These internet companies had their glory days in the early 2000s — today, they're just addiction algorithms that keep users hooked onto their platforms, and maximize eyeballs on ads. All of them have sizable neuro departments, doing research on addiction patterns in children and young adults. Work is both unethical and mind-numbing, but it pays really well.

As my mind clears, my link brings up ticket prices to various destinations. Impulsively, I book two one-way tickets to Paris departing the next morning. For the highly educated and cultured, moving countries is never a problem — international travel has really eased up in the last decade. Back in the dark ages, the immigrants would visa-hop until they "settled down". Today, everyone is an immigrant of some kind — who wants to live in the country they grew up in? It's half-past four, and I recognize my neighborhood from the mini.

"We're moving to Paris tomorrow!", I announce, as I get in. Madeline makes a weird anime expression as she looks up from her paperback. I try to identify the soft indie music from the LP, but no avail. "It's about time", she says, after a pause, with a broad unquestioning smile. I'd never consulted her on the destination, but she's not one to take issue with such things. We love surprising each other.

I'm trying to remember how I ever convinced her to move to this suburb in the first place — ah yes, a long vacation in a house facing the sea, with abundant money to buy books, LPs, and scotch didn't sound like a bad prospect. The last month or so has been a bit hard on us though — the cultural poverty of the surroundings was getting a bit much. There isn't much to do when friends from other parts of the world visit us. Much of the culture in the nearby city is centered around tech, a topic that we have little interest in. Our friends are initially fascinated by the high-tech convenience systems that come with wealth, but its novelty starts to wear off pretty quickly.

"Mauzac or Négrette?", she offers, as she gently gets up from the couch. I pick Mauzac, switch the music to Tame Impala, and turn the volume up a notch. Normal people wouldn't bother with analog music, and just shuffle about within their recommendation systems. "Ancient music!", she exclaims, and we kiss tenderly and hug as two young lesbians. She never asks about work, and I have no reason to talk about it either. I pull up some AirBnBs on my link, narrow it down to a few near the center of the city, and let Madeline pick one. I've already made the big decision of the evening.

"You will get that link removed, yes?", Madeline asks with raised eyebrows. "I don't know; it's so convenient", I respond sheepishly. It won't be half as useful in Paris, but I hadn't given the issue much thought. She shakes her head with a grin. "You resisted getting it for the longest time, but this tech job finally broke you". We were among the minority in London for resisting links. Madeline thinks it's an outrageous breach of privacy. I agreed with her for the longest time, but fell into the convenience-trap in the end.

The doorbell chimes, indicating that our bot has arrived with packing boxes. Moving is hardly an ordeal these days — we move every three or four years. Unlike most people, we don't need a reason to move, but rather a reason not to move. Becoming fluent in the local language usually doesn't take us more than a couple of months; Mandarin was the exception — took us a little over six. Prior to our move to this suburb, we had four good years in London. She was working at the local museum, and I was doing my PhD in higher categories.

"Will you miss the high-tech conveniences?", she asks naughtily, as the sophomore girlfriend. She loves doing this to the rigid old man: poking him about what he'll miss everytime something changes. When we moved from London, she asked me whether I'd miss the tube. "I certainly won't miss the 8-hour weekday grind". I was exaggerating — having expended the minimum work in order to keep my dayjob afloat, and used a lot of my time to read and practice oil-on-canvas. These goliath companies are complacent about keeping tabs on their employees, as most of them are addicted to developing the addiction algorithms. The irony. Nevertheless, that should change the line of conversation.

We have a few baseline characteristics, and keep transforming ourselves with every move. "In Madrid, we were true Spanish, in London, we were true Londonders, but I don't know what we became in this suburb", she continues after my thought. The suburbians are poorly-traveled boring people with straight career-lines whose dominant interest is children. Designing children. Even the most conservative ones genetically engineer atleast the hair color and skin tone. "Culturally poor, incredibly wealthy burbians, but without kids?", I offer, in summary. "You remember when our lovely neighbors tried to come over during the first week and offer us their wonderful food?", she asks sarcastically. I raise my eyebrows and shut my eyes in feigned embarrassment. We had to throw away most of the food because it was so insipid.

"I'll let you pick dinner", I state with authority, playing the dominant role. "Bœf bourguignon with cream of artichoke?", she offers coyly. As we move into the kitchen, the bot skirts past us — it's finished with the study, and is heading for the bookshelf in the living room now. We're always a little disappointed when we open the refrigerator. There is literally one weekend market within commuting distance of where we live, where we have to get our ingredients for the week. It's overpriced, and the stuff isn't great, and there's very little diversity. We can't stand the frozen stuff everyone else gets at these gigamarkets. London had a terrible food culture as well: everyone was getting takeout boxes from the local Indian restaurant. "Don't fret. We'll soon be heading to the local butcherie, poissonnerie, and fruit/légume marché every other day, just like in Hong Kong", I say with a broad smile.

As I cleave the beef, she expertly chops the carrots and peels the potatoes while sipping her wine. Many of these burbians get their ingredients shipped home, and have bots cook for them, an idea we find repulsive. "Will you have something to read on the flight tomorrow?", I ask with the concern of a close girlfriend. We both read rapidly, but she's got to a different level during the course of this vacation; burns through two or three paperbacks a day. "You're nearly done with Daisy Johnson's latest, yes?", I ask referring to the book she was reading when I entered. She nods, and we both rush to the living room, to pick a couple of novels set in Paris from the shelf before the bot gets to them. I love my paper books as well; vastly prefer them to reading off the screen or the link.

As the prep phase of the cooking draws to an end, the induction top switches on. In Hong Kong, we used to have a traditional gas top, and often cooked in woks. "We don't have to worry about earning for a while now", she remarks. It's true; with our savings and the generous UBI pay in the EU, why would anyone work for a living? "The last time we had that luxury was in Madrid", I respond, with a tinge of bitterness. "I'll probably write a novel", she says. She's only written short stories so far, and this stint has more than prepared her to write her first novel. "And where is it going to be set?", I ask flatly, as the professor. "Paris, of course. Modern era", she says with a beaming smile. "themed in European film history", she continues. She'd have to learn a few major European languages and cultures. "You're going to be nosed up studying for quite a while then", I say, downplaying the excitement. She knows that I know that we're going to be traveling a lot, with purpose, and merely smiles at the unvoiced thought.

The soup is ready, and we start sipping it in the dining room, while the beef simmers. "I'll work on a project to formalize neuroscience", I say finally. The question has been lingering for some time now. "Quite the challenge", she remarks, adjusting her glasses, playing the nerd. The bedrock of most software construction starts with an understanding of the brain. Most software construction and most neuroscience is stuck in the dark ages though. For the most part, Madeline only uses software that is correct by construction. She's disgusted by recommendation systems, and terrified that a software bug will breach her privacy or hurt her. Practically speaking, this restricts her choice to a subset of pre-neuro software. "I should get my link removed", I say in sudden alarm. "Yes, you should; the Parisians won't take well to it", she responds soberly.

The bourguignon is delicious. Perfect dinner to end this stint. We retire to bed with our books, smoking a couple of cubans.

Aging

2022-04-16T12:43:00+02:00

I'm eighty today, but the sequence of events I'm going to narrate took place when I was in my early twenties. It was a pleasant Friday in San Francisco, and as many people will tell you, SanFran has the ideal weather. The mood at my workplace was jubilant. Added to the fact that the weekend was coming up, our CTO had announced that we would be going public the following Monday. Everyone was dressed in colorful tees and shorts, and we were all trying to estimate what our net worth would be after the IPO, based on the amount of vested shares each of us had. Needless to say, we were doing very well, and were expecting the IPO to make a big splash. Many of us were dating each other in the office; even our CTO was under thirty. As the afternoon wound down, all of us left early, and in high spirits.

I mindlessly took a stroll along the pier with a co-worker, and we got ice-cream to celebrate. She left shortly after, citing that she wanted to spend time with her boyfriend. People had suspected that there was something off with me, but wrote it off as plain weirdness that I'd eventually grow our of. I was careful not to get too close to anyone, for I knew, deep within myself, that I was pretending. I didn't belong here, among these people. If you ask a lay person what this meant, they'd ask if the problem was racism, sexism, or some such social issue, but the problem with me was much deeper than that; I constantly felt like a 200 year-old person.

As I continued my stroll back home alone, I entered a deep state of melancholy. Normal people could live their lives in pursuit of wealth, good-looking partners, and social status. I just didn't get the point. I might be in my early twenties today, but time will fold onto itself until I'm forty, sixty, and eventually eighty. One way to put it, perhaps, would be to say that I couldn't live in the drama of the searing present. There'd be an IPO, which might bump our net worth and social status. We might be able to afford a 3000 sq-ft home in SanFran as a result. Or our company might sink, and we might have to move to a less fashionable city, in order to buy a house. It was inconsequential to me either way: I just didn't see the point of thinking about it; what will happen will happen, whether or not we bite our nails over it.

We're born one day, and we die approximately eighty years later. Society is not structured to accommodate people well beyond their age. That was the truth of what I was doing at a young startup in SanFran: the first-hand life experience, not because I expected to enjoy it, but because it would be an interesting observation to fill the 23rd year of my life. 23, because it was a socially-acceptable age for this.

I felt every groove on my key as I inserted it into my door. I'd already planned my departure, in complete secret. I took off my shoes, and started up my surround-sound system to play soft Icelandic music. Having curled up on the couch comfortably with some good whiskey, I lit up a Cuban cigar. All the creature comforts were attended to; we were all well-paid, and nobody had suspected who I really was. My bookshelf was overflowing with Pulitzer/Booker-prize finalists, and many other rare gems. My e-ink tablet had serious works on mathematics, philosophy, neuroscience, and physics. Normal people usually have some self-help books disguised as psychology, and other baloney disguised as non-fiction. Any serious text would only be read in the context of their jobs.

I started thinking about Reykjavik. My flight was in three hours, and the packing was done; I was departing permanently. I'd spend a good six months relaxing in Iceland, and plan my next steps. I'd sell my stock after the IPO remotely, and give in my resignation (again, via phone) shortly after. The end of my one-year lease on the house was approaching shortly — I'd already asked my landlord to cancel the renewal of my lease, and had arranged for my stuff to be stored in a warehouse via a moving company. I'd packed ten novels to read over the course of six months. I was leaving most unceremoniously.

Was I feeling happy or relieved? Not really. I'd still have to pack the years of my life with stuff to do, until I felt like I'd found home and wanted to put down roots. I wouldn't want to be a pretending-nomad forever, and perhaps I'll always be a misfit, but I was sure there'd be a place I'd be able to call home. Time was abundant.

The re-telling of something that happened over fifty years ago brings me great joy. I'm near the end of my life today, and I can whole-heartedly say, that I have lived a full life. The deep sense of melancholy that I had at the age of 23 only deepened with age, but I learnt to live with it. I live in a place where people understand and love me, and for that, I feel very fortunate.

AI and the war-time economy

2025-12-20T13:18:05+00:00

There are innumerable reasons to hate AI, starting with its environmental impact: the AI boom has been estimated to consume as much power as New York City, and as much water as the global consumption of bottled water in 2025. The reasons could also be personal, from leading to a loss of livelihood, to threatening the way we pursue our passions. The distaste is fueled by visible misuse, including the flood of low-quality AI-generated bug reports and PRs on open-source projects, music and books uploaded to Spotify and Amazon, papers at academic venues, and the recent horrifying McDonalds ad. To make things worse, there is plenty of anecdotal evidence to suggest that AI makes an individual more productive, while there are plenty of studies that suggest the opposite. At the same time, there is no denying that the technology is very widespread across the industry, well beyond just the software industry. For many of us, personal distaste and ethical issues has lead to a AI-denialism stance, where we declare that AI doesn't work for anyone, and espouse borderline conspiracy theories.

The primary feature of AI to pay attention to is that it's a fuzzy non-deterministic system, and measuring productivity gains from AI at an individual level is a bit like studying witchcraft. Today, AI adoption across the industry — be it software, finance, digital humanities, law, journalism, or otherwise — is so widespread, that dismissing it as just "hype" or "herd mentality" is not a cogent position any longer. To a first approximation, corporations are rational capitalist actors looking to maximize their profits, and they are paying good money for AI products for a reason. I only see one conclusion by Occam's razor: that, for many orgs, implementing AI org-wide is a net-positive, although the results vary at an individual-level. Many orgs have run internal productivity metrics and found a 20-30% increase — and the proof is that they continue paying for AI products, even in the light of studies that suggest that AI makes individuals less productive. This would lead us to look for publicly observable metrics to correlate the fact that there's an increase in productivity across the board: for example, we might expect to see quicker software release cycles, an uptick in feature rollout or bug-fixes in products, or in open-source code. Perhaps not the not the most satisfying answer, but orgs are not increasing their production volume with AI adoption, and are instead raking in profits by lowering their costs — and the publicly observable metrics are the layoffs, hiring slump, unemployment, and under-employment in advanced economies. The other problem is that the technology is a poor fit for open-source projects, and I will expand on this next. As an economist put quite poignantly, the current situation is akin to a war-time economy, where the defense companies are raking in profits with huge environmental costs (or the AI adopters today), while everyone else is left to survive on rations.

Open source communities work because the community members are deeply passionate about the project, and treat their work like a hobby — they take pride in their work, and there is nothing to sell. All open source projects must attract and retain new contributors every year to survive. Maintainers often do a lot of work helping new contributors, and the payoff is only seen when a new contributor continues gaining experience, and eventually grows into a maintainer themselves. This key social aspect of an open source project is the reason why AI adoption will be minimal at best: the cost of maintaining LLM-generated code outweighs the marginal benefit of adding a new feature — and since it was low-effort for the new contributor, they are less likely to stay on and learn. The cost-benefit computation is different for a corporation selling a product — revenue is directly correlated to feature rollout. The idea of corporations profiting off open source is nothing new — and in this age, all they have to do is train their LLM on our hand-crafted "artisanal" code.

The economics of AI doesn't add up at all, and the technology cannot even be sold at cost — there is no denying that we live in a bubble similar to the sub-prime mortgage bubble. After the crash, a lot of everyday people will lose a significant portion of their savings for no fault of theirs, and what will probably be left are cheap compressed versions of what we have today, running on commodity hardware.

We have been very irresponsible in going down this AI path — and we're doing irreversible damage to the planet while at it. I hate AI for the reasons outlined, and it is useless to me, as a member of the LLVM community. However, I cannot deny that the technology is widely useful and has increased the productivity of many orgs.

That said, there are also many corporations that have adopted AI with no material benefit. Perhaps the long-term effect on corporations that have adopted AI will be the accumulation of technical debt, leading to shorter-lived corporations.

Banter

2021-09-05T14:16:39+02:00

My heart started racing the moment I saw her. She was standing right there by the coffee machine. The first thing I noticed about her was her hair. Curly hair, that was neatly combed back, with a couple of ornamental clips, pinned haphazardly. Neatly trimmed in line at the neck. She was short, had soft glowing skin, and was speaking very softly, in Russian, with another Russian girl who worked there, that I was acquainted with. She had beautiful legs, dressed in tight jeans, and she was wearing a beautiful red top that day. Exceptionally well-groomed and fit. This was a woman who took great care of herself. In short, stunningly beautiful. I walked rapidly up to the coffee machine, not because I was in a hurry to meet her, but because I always walked rapidly.

"Is the coffee machine broken again?", I asked. "Oh, it just needs some love and taking care of, like a cat." "Purr, Purr", she whispered, as she stroked the coffee machine. I started laughing immediately, and without inhibition. I'm a tall well-built French man, and I was dressed in a plain black North Face tee and cargos. Since my friend had introduced me to it, I always wore jet black Adidas boots, and that always helped making a big entrance.

She was smiling gently at me. Terribly shy, I could tell. "See, it's fixed now?", she said, pointing to her full cup of black coffee. "Do you work here?", I asked gently. "Yes, I'm Irena", she responded, gently turning away in shyness. I recalled that there was an office on our floor with the name Irena that was unoccupied. People didn't talk about her much at work, but I knew that she'd been on vacation. "I'm Etienne. It's nice to finally meet you." "So, you're back from vacation?", I said trying to hide the eagerness in my voice. "Yeah, you know vacations are like that", she said, feigning boredom. She then collected her coffee, and returned to her office with Katina, the other Russian girl she was with. It was post-lunch, and we, as the French, have long lunch breaks. Irena was in the same team as me, and we'd be meeting each other multiple times over the coming months, over multiple lunches and coffee breaks. I was mostly just looking forward to the next day.

There was a talk at IRIF that day, that one of my team members was delivering. I spotted Irena sitting in the front row, feeling uncomfortable because Katina wasn't by her side. Then, as Katina entered the room, Irena waved and tried to get her attention. It was then that she noticed me sitting in the rear, and staring at this amusing ordeal. I merely smiled, but she immediately turned away, embarrassed, and stop waving.

The very next day, I noticed thumping footsteps outside my office, and realized that both Irena and Katina had taken to wearing boots.

Over the next couple of months, our relationship into light casual banter over coffee and lunch. Then, one day, Irena decided to sit next to me at lunch, with Katina by her side. We had a joyful conversation on Master and Margarita. At this point, she decided to sit next to me everyday over all the lunch sessions over the coming months, but for the occasions on which she was cross with me. We were definitely more fond of each other than with the others on the team.

It was my morning routine. Waking up at 5a, and arriving in the office before everyone else, occupying the lounge, and studying mathematics on my tablet. I would wait for Irena to arrive at around 8a, and get my fourth or fifth morning coffee with her, with nobody else watching, and not even Katina by her side. After that, I'd walk to the nearby bakery and get a croissant aux amande, my usual breakfast. On that particular day, I was feeling particularly hungry, and Irena was probably running a little late. I decided to get the croissant myself, and hoped to catch Irena afterward. As I took the stairs to the ground floor, we noticed each other. She had just entered the elevator, and I was walking past it. At this point, she lowered her glasses, and looked at me, definitely feeling a little cross for not having waited for her.

I missed work on three consecutive days, because I had a bout of food poisoning. The next day, when I turned up at work, Irena turned up to get her her coffee, with ruffled hair. She looked distressed, and it seemed to be because she had been missing me. "Where have you been?", she opened with. "Oh, you know. Food poisoning." "Sorry I missed Katina's birthday." "Oh, it's alright, I'm glad you're fine.", she responded. After the usual morning coffee, and somewhat subdued banter, she returned to her office. Moments later, it occurred to me that I could have brought up her hair, so I quietly tiptoed to her office, and peered in. She seemed to have combed her hair at this point, and was no longer looking distressed.

I grew really fond of Irena over the next few months. After carefully evaluating I ought to proceed, I decided to gift her a book. Since I knew her taste in reading at this point, I inked a freshly purchased copy of Vulcan's Hammer with my beautiful cursive handwriting in blue fountain-pen ink. A slightly personal "To Irena, the Fencing-Salsa-Japanese girl, from Etienne" is what I inked, recalling her three primary hobbies.

I wore my favorite shirt to work that day, with excitement. When she arrived for her morning coffee, I opened with "I have a gift for you", and presented her the book. She panicked, and in an attempt to downplay it, she asked me to wait for just a second, rushed to her office, found a copy of an old Systems Engineering textbook, presented it to me and said "So, it's a book exchange", with a serious expression. I smiled politely, disappointed.

Bhutan

2022-01-06T17:02:32+01:00

I woke up feeling like I had a brand new body and mind. We were asleep in a little cabin in the middle of nowhere, in a little village in Bhutan. We'd arrived the previous day, by air, and ridden on top of a bus while jet-lagged to get to find this cabin. We'd all arrived from different countries: Sven from Svalbard, Aamod from Brooklyn, and I'd made it from Montpellier. To add to the chaos of our flights arriving at different times, we had no mobile connectivity in the Thimphu airport, and it took the better part of the previous day to find each other roaming around randomly in various tea shops. There was no plan, we hadn't booked anything in advance, and none of us knew Dzongkha. Yet, somehow, with Aamod's broken Hindi-Urdu, we decided to take a random bus going to an unknown destination, only to find that the bus was full. That's the story of how we ended up on top of the bus. Alright, there was one crucial preparatory step I'm skipping here: we'd gone to our respective banks and exchanged our currencies for Indian rupees, since no bank had heard of Ngultrum; we'd read online that Indian rupees would work in Bhutan. With a limited supply of these Indian rupees in hand, we would manage our expenses over our week-long vacation. When we got off the bus, we found ourselves literally in the middle of nowhere, and trekked a bit until we found a house with a family staying in it. They were very welcoming to us, served us tea, and put us up in their cabin for a small sum.

Aamod was already awake, and gently strumming his ukulele, while Sven was slowly waking up. "Bonjour, Hugo", he sang with a broad smile. "Bonjour!", I responded, cheerfully. Sven had started stirring now, and was nearly awake. We were three very unlikely friends, with very different personalities and vocations. Sven was a jazz musician, Aamod was a student in linguistics and sociology, and I was a mathematician. As an upcoming musician in Svalbard, Sven was the person who was most in touch with nature, and the least in touch with people. Aamod, on the other hand, was a very sociable person, and we were relying on him to handle most of the logistical challenges of this trip. I was a soft-spoken mathematician who had an adventurous streak, but had no creative talent to speak of. Aamod and I had met in San Fransisco, quite randomly at a pizzeria at 4am. We were immediately drawn toward each other, and spent the following day roaming the streets and smoking weed. We'd decided to go to Rejavik the following summer, without a plan, and found Sven playing jazz at a bar there. We loved Sven's music, and spent the better part of that week with him, skinny-dipping in the ice-cold waters, and enjoying the sounds of nature. To be honest, we didn't know each other very well, but both Sven and I were immediately enthusiastic when Aamod proposed that we go backpacking in Bhutan. Many years down the line, we would become close friends, and we'd realize that this week spent together in Bhutan would establish a very strong bond between us.

"Hej, Sven!", I chimed, as Sven sat up in his bed, with a start. "Morning, both. I need some tea", he responded. Aamod stopped strumming, and started scratching his unruly beard. Sven was a heavy coffee drinker, and he was slowly getting accustomed to the fact that only tea would be available in Bhutan. "Get dressed, then. We'll head over to the house — they'll be happy to see us up so early", I said. Indeed, my watch read 6:30a, and we hadn't gone to sleep until 10 the previous day; yet, we all felt like we'd been asleep for a week. We all suited up for the cold, and went knocking on our host's door. The kind old woman who greeted us the previous day appeared at the door and was noticeably surprised. We all smiled and bowed — no words were exchanged. She led us into the large dining area and seated us on the floor. She then automatically proceeded to serve us a milky tea mixed with butter and salt. Sven preferred the milky tea with sugar, and Aamod and I smiled in amusement as he sipped his tea with some discomfort. "You'll get used to it — it's a good drink for the cold weather", said Aamod, gently patting him on his shoulder. When we'd all finished the tea, Sven managed to utter something that sounded like "Dhanyavaad" to the woman, emulating the sound Aamod was constantly making the previous day, to convey his gratitude; this was a warm surprise. The woman smiled generously, and disappeared into the kitchen again to fetch breakfast. When the plates were placed before us, we recognized the food as some sort of barbecued meat along with some boiled and salted peas, with the skin still on. After taking a bite, Aamod and I couldn't recognize the meat; Sven noticed our confusion, and said "Oh, it's really tough sheep meat".

We were eager to get to a nearby village, where we could meet some people, so we caught another random bus from the same spot as which we'd gotten off the previous day. Thankfully, we were riding inside the bus this time, and we kept a lookout for small towns. Once we reached a small town, we got off, and roamed around a bit observing the people and the the unpaved streets. People seemed to be observing us with curiosity, and some of them walked up to us, and asked us where we were from. Aamod is mixed-race Indian-American, Sven is Nordic, and I'm German-French. Sven looks especially curious with his prominent tattoos, facial piercings, long hair, and long well-maintained beard. I have a typical mathematician's beard, and Aamod has an especially unruly one. Neither of us have any visible tattoos or piercings. We were all wearing some kind of jacket, which in itself was enough to make us stand out: everyone there was dressed in some kind of indigenous woolen top garment. There were very few automobiles, and even those were motorcycles. The town wasn't electrified of course, although we did spot some diesel generators at the shops.

I'd brought along a dSLR camera, with which I was awkwardly taking pictures. We decided to make camp in a tea shop, whence Aamod and Sven decided to take turns playing the ukulele. Little by little, people from the town started gathering around us, out of curiosity. At one point, Aamod and Sven began singing, in what would be a long performance for the town. Soon, most of the town was at the tea shop, and everyone was brimming with joy.

On the next day, we headed to a garment shop, and spent the entire morning curiously looking at the various kinds of garments. I picked a shawl with bright colors on it, while Sven picked something tasteful and expensive, and Aamod settled on a long gray flowing top garment. The reason Sven and I didn't pick anything other than shawls is because the garments were somewhat ill-fitting: as Europeans, we were tall and well-built with broad shoulders; Aamod didn't seem to mind the ill-fitting purchase as much though. We did away with our jackets, and got dressed up, excited to see how we'd be perceived now.

As a light late-afternoon snack, we tried some curious yellow rice-based dish, and all three of us were feeling unreasonably sleepy after that. We'd later discover that it was a poppy-based dish, and is meant to be had in the afternoon before a siesta. At this point, we'd found a lake and taken a boat, but none of us was in any condition to swim the cold waters, so we decided to relax for some time in the middle of the lake, and go for a swim another day.

Initially, I was slightly uncomfortable about the fact that we hadn't talked much — I'd come to realize that we never would say more than a few words to each other on this trip; there was an unspoken agreement between us that we'd leave our lives behind on this vacation.

On the fourth day, we discovered that people here smoked opium for leisure, and both Aamod and I were shocked at the prospect of smoking opium. Sven laughed generously, and convinced us to try it. The host had brought us something called a "chillum", a sort of long pipe made of stone, with a piece of cork holding the substance the person wants to smoke in place. Then, he took out his handkerchief, and used it as a filter. A very crude setup overall, but it suffices to say that we had a very relaxing evening on the porch of his house. I enjoyed myself so much, in fact, that I'd buy a chillum and take it back with me, as a reminder of that evening.

We decided to spend the last couple of days of our vacation in Thimphu. We'd made a friend, who spoke English. He told us that he was an ex-con, who'd escaped from prison, under the false pretext of insanity. We all laughed together over his wild stories.

When it was time to take our flights back home, we promised each other that we'd plan a longer vacation the next time around. The airport security personnel freaked out a bit when she saw my chillum, mistaking it for some kind of pipe bomb. I calmed her down, and explained to her what the thing she was holding in her hand was.

The Insomniac

2021-09-05T09:31:15+02:00

I couldn't sleep that night. Tired and frustrated, I got out of bed, took a shower, and dressed for the snowy streets of Boston. Thermals, t-shirt, jeans, and a quirky jacket I'd picked up at the thrift store, that I was quite fond of. I started making plans in my head. I'd take the 5:30am train to Natick after finishing breakfast with a Bloody Mary. There was a diner close to my place that was open through the night. The feeling of frustration was fading, and was quickly being overtaken by a feeling of excitement. I was used to this: I often slept poorly, and had the habit of turning up at work at ungodly hours.

As I opened the front door of my building, a gush of cold air blew in. It was 2a, and the streets were empty, covered in a few inches of snow. Perfect timing. I'd probably run into the most random and interesting people. Picking out my first smoke from the pack, I noticed the first human being ambling about. There was no rush to get to the 24-hour diner.

He looked a bit shabby, but was dressed comfortably for the winter. In his mid-40s, I'd say. Definitely not homeless. A little tipsy, he was drinking a Lagunitas, and was carrying a few more with him. "Hey there, you wanna trade some smokes for a beer, mate?", I said as I approached. "Sure, man. What're you doing up so late?", he said, handing me a beer. "I don't get much sleep anyway. Might as well roam about, am I right?", I said with a smile, as I handed over three Lucky Strikes. "Bloody hell man. My wife kicked me out, and I'm taking the 7a train to Washington." I suspected as much. "Tough luck, mate. No luggage, eh?", I said, with a little sympathy. "No, man. Didn't have enough time to pack. Gonna visit my sis." "I'm heading to South Street to get breakfast. Cherios.", I said, ending the conversation with a wave.

I was already feeling the rush of being out in the cold on no sleep. I'd make the most of my walk to the diner, I thought to myself, as I plugged into my earphones. Bowie, I figured, as I shuffled through my Spotify. It would be a long and satisfying walk, I thought to myself, as I took another swing of the Lagunitas. I'd have some time to think about breakfast, and the new compiler transform I was developing at work. I spotted few other people, who didn't seem to be interested in picking up conversation. There were few homeless people in this part of the world, as the winters were so inhospitable.

As I approached the diner, I unplugged my earphones, and noticed a bunch of people outside smoking and chatting. I loved South Street, especially at odd hours, because it had such weird mix of people: people working night-shifts, strippers, the unemployed, and people who had trouble sleeping. I took a seat at the bar next to a goth girl, and started scanning the menu for items I'd not tried in the past. The diner was cheap, and the food was decent. And, as was the case with all diners, bottomless coffee. It was always lively, but never noisy. This was probably the only 24-hour diner in Boston, and I lived close to it. I felt so lucky.

Within thirty seconds of taking a seat at the bar, the waiter had filled my mug with fresh coffee. "What'll ya have?", she said, in a completely no-nonsense manner. "I'll have the scrambled eggs and bacon, with a side of sausage. And a Bloody Mary to finish off." I'd ordered the usual, realizing that I'd tried everything on the menu already. She made a quick note, and concluded with a "Comin' right up." Although I visited this place a lot, I wasn't acquainted with the waiters. They had a lot of customers, and were constantly working. There was nothing special about me. "Come here often?", I quipped, turning to the goth girl. She had tattoos all over, and was dressed in black overalls. "Yeah, it's my jam.", she said, turning away. She didn't seem to be interested in any further conversation, so I dropped it.

The bacon was tasty, as were the sausages. The servings were all generous, and I was content with my heavy breakfast. The Bloody Mary had just the right amount of salt and celery, and I took my time to finish. I paid by card, and left without saying another word. I was probably the most square person at South Street, if you were to judge by appearances: I didn't have a single tattoo or piercing, and was employed in a well-paid software company. Every now and then, I'd pick up a short conversation with someone there, but in general, I had little in common with the people there. Yet, I enjoyed their presence.

I was really enjoying the mix of the Mary and nicotine, as I noticed the first rays of light from dawn streaming into the streets. As I approached the train station, I noticed an aging woman from the distance. My watch read 5a, and I'd packed a book to read on the station and train, but she seemed to be a little distressed. As she approached me, I realized that the winter air must be a little harsh on her. "Excuse me, but when is the next train?", she asked, politely. I told her that the first Purple line train wouldn't be for another thirty minutes. "Aren't you feeling cold?", I asked her, with concern. "A little", she admitted. She must have been downplaying it to be polite. At this point, I showed her how to enter the station and take a seat in a sheltered area. I then proceeded to offer her my coat, to which she shook her head and said, "It's very kind of you, young man, but I don't want you freezing." "I thought the first train was at 5, but why are you here so early?", she asked, after a brief pause. I explained that I often didn't get sleep, and this was routine for me. We then proceeded to make pleasant conversation about family, friends, and life in Massachusetts.

At forty past five, we started seeing other passengers. Nearly all of them were going to work. Time passed quickly, and I was soon on the train. After a quick visit to the washroom, I took a seat beside the woman, and started reading Infinite Jest. It was superbly written, and would take me weeks to finish.

I got off at Natick at forty past six, and bid the woman goodbye. I smiled to myself for another satisfying morning, as I walked towards the office.

What machines can and can't do

2020-12-01T07:41:57+01:00

We investigate the limitations of statistical methods, a subclass of which is called "machine learning", taking the opportunity to touch on several aspects of the industry, including the future of employment, various industry practices, and privacy. It's written as a wide-audience article, and only assumes a passing familiarity with software and statistics. We start from a seemingly hardline stance, and follow a line of flawed reasoning leading up to the point where humans become near-obsolete, an event commonly known as the "singularity". Finally, we step back to see where we stumbled.

We use the following starting point: using any combination of statistical methods, of any complexity, it is impossible to develop any sort of "understanding" that resembles human intelligence. Instead of trying to tackle questions about the nature of intelligence, or entering a deep philosophical rabbit-hole about whether "intelligence is computable", we start with the modest goal of understanding what statistical methods are.

As a starting point, one might argue that the world is made of up of a collection of "facts" that need to be "memorized" after having seen them stated multiple times correctly. A variation of this is the "monkey brain", in which you train the monkey by asking it to perform a certain task, rewarding it every time it gets it right, and penalizing it every time it gets it wrong. This is the basic idea behind "reinforcement learning", the kind used to get machines to play "games". Speed up the task-to-[reward or penalty] cycle exponentially, and you have a "smart monkey". While the "smart monkey" might make silly mistakes on a bad day for irrational reasons, a "bad day" for a statistical model is the event when it is presented with data that it doesn't know how to fit. The monkey's emotional engine can arguably be emulated by weighting the penalty that the machine receives, based on how fundamental the mistake was, and how many hours of training it had received. The procedure we've described of "training" a machine like this falls under the class of "supervised learning", which, as one might argue, is the same way a human learns from an assignment or exam.

Let us assume that a machine can successfully memorize trillions of "facts", and look into how these might be represented. In most commonly-used applications today, they are encoded in the form of matrices of floating-point numbers, with some weighted arithmetic operations connecting them, as chosen by the person building the model. The model is fed a matrix of floating-point numbers, and outputs a matrix of floating-point numbers. We might argue that the inability to interpret these individual numbers is akin to not being able to interpret the voltages of the individual neurons in the brain, and this doesn't seem to be a problem in practice. The said "facts", now lost in a soup of numbers, can be generalized into a broader class of "things that can be inferred from data". With every attempt at being able to handle a new situation, the model simply does arithmetic on the soup (a large class of which are just matrix multiplications), and the numbers in the soup adjust themselves with every "feedback" we give it. This is the basic idea behind "neural networks". While it's easy to get lost in the details, note that we're still doing statistical analysis. The additional "frills" haven't changed the essential nature of the method: the machine is "learning by example", and attempting to "generalize" it.

Among human beings, learning by example is a very effective way to learn, when starting out. Most programmers would attest to the fact that they started out by copying examples, tweaking them, and checking them with a program (these programs are known as "compilers"). Some would argue that this approach doesn't scale in human beings, because they don't have the computational power to continue learning by example, and "cheat" by reading some kind of specification as they become more experienced. Does it mean that classical programmers (aka. those not engaged in building statistical models) could be replaced by machines in the future?

The key to seeing the viability of this argument is to consider two things: (1) the structural complexity of the task, and (2) the number of examples available to concretely reinforce the patterns, and nullify the anti-patterns. It's no secret that there are several good tools for aiding with "simple" programming languages (otherwise called IDEs), which a reductionist would phrase as "machines are good at dumb tasks". Using simple frequency analysis, the IDE can order its auto-complete suggestions and corrections intelligently. To date, there is no IDE that can assist with the help of the vast repositories of open-source code, but it doesn't require a big leap in imagination to assume that this will be possible in the future. In fact, we should be open to the possibility that compilers start building repositories of error-messages, and run in the background, suggesting even more intelligent fixes, based on how others who encountered a similar error fixed it. A philosophically-minded reader would be interested in whether the "intent" of the programmer can be dissected, and if, by labelling various segments of publicly available code with some kind of abstract "function", the machine can write entire programs with a few prompts. Assuming for a minute that this might be possible, can classical programmers stay relevant by simply moving to newer languages or technologies, on which there are few examples?

This poses a quandary: are we playing some kind of primal game, where the prey constantly has to come up with new strategies to outrun the predator? Some would argue that by refining our statistical methods, one can emulate the process of generalization effectively enough to beat the prey's creativity on the "search space" completely. What if this is the nature of every human specialization? An event, commonly referred to as the "singularity", whereby humans become near-obsolete, is predicted in 2040. Why is this claim so ludicrous?

To better understand where statistical models are successful, let us enumerate some recent accomplishments in the field.

Beating the best human player at Go, a program referred to as AlphaGo. The problem is well-defined, and there are very few starting rules. How does a human become good at Go? Just like in Chess, the rules play an insignificant role in the player's learning, and most of the training is about analyzing previous games. There are huge databases of expertly-played games available, and human players usually try to follow a semi-systematic approach. It could be argued that the machine is inefficient, because it doesn't follow a systematic approach, but the raw computational power makes this handicap look irrelevant. It's a little like arguing that a machine doesn't know the decimal system, or the tables of multiplication, and simply flips 0s and 1s to perform arithmetic. Humans here are the disadvantaged class in both cases, and it is completely unsurprising that, despite certain "inefficiencies", AlphaGo essentially does what a human player does to beat them at it: analyzing lots of games.

A language engine, referred to as GPT-3. How do humans master a natural language? They do it semi-systematically, starting from the association of simple words to real-world objects, then moving on to phrases describing something, learning some construction rules and absorbing culture along the way; and finally by reading a lot of good literature. Of course, the grammar rules play a very small part of the overall learning, so it is unsurprising that GPT-3 is able to produce grammatically correct responses to human-supplied prompts. However, literature talks about things in the real world, and for it to make sense, sufficient interaction with the real world is a prerequisite. It's not a bounded game, where all the necessary information is contained on a map or a board. It is, therefore, entirely unsurprising that GPT-3 can't "know" whether dropping a metallic plate would cause it to melt. It cannot infer the relationship between real-world objects, nor can it differentiate between factual and fictional human experiences. Producing good literature requires a whole lot more than that. This landmark achievement, despite having access to terabytes of text, and an unimaginable amounts of compute power, has failed to infer much more than grammar, sentence construction, and context.

The translation engine behind Google Translate. Someone who has just started using the product with no knowledge of the underlying technology would be puzzled about why this is chalked up as an achievement. Indeed, it's unlikely that we'll even have a half-decent translation engine using purely statistical methods. Translation engines have been around for a long time, and their latest iteration might be a significant improvement over the previous ones, but on an absolute scale, something seems to be off about relying on a pure statistical method to learn natural language.

Medical diagnostics. In this field, it is absolutely imperative to organize every little piece of data about every patient in a highly systematic manner: a missing entry is a life-or-death situation. On millions of these remarkably well-organized records, it should come as no surprise that a machine can serve as a valuable aid to a doctor. Statistical models have been a resounding success in this area. The achievement here, perhaps, should be attributed to the subfield of "computer vision", which studies the best way to encode medical images and videos in the aforementioned number soup.

Proof-search on a well-defined problem phrased in a computer language, using well-defined operations to go from one step of the mathematical proof to the next (in other words, aiding proof-search in programs known as "proof assistants"). These are classic combinatorial problems, with huge search spaces, lots of rules at each node, and clear end goals. Sort of like a video game on steroids. Some progress has been made, and the IDEs are expected to get better. At first glance, this might seem a bit alarming to those who don't understand how little mathematics can be written down in this form, or how painful it is to do so.

Finding protein-folding structures, referred to as AlphaFold. Another classic combinatorial problem, with well-defined rules. The landmark achievement here is that this problem has an enormous search space, and a solution to an important problem in computational biology has been found, which is sure to accelerate research in the area. Statistical models are an unabashed success here, and it has done what humans could only dream of doing using other methods.

Facial recognition. A classic pattern-recognition problem, which can only be learnt by example. It suffices to say that this technology is a resounding success, with the unfortunate side-effect of opening the door to various abusive ways in which it can be used.

Automated support over voice and text. The little "domain knowledge" needed here is all encapsulated within a very narrow context. There are few simple rules of the type "if the human says this, then say that" (the model used to make such decisions are called "decision trees"), and humans learn to identify voice by example. The landmark achievement here is in "speech synthesis", or making the machine do text-to-human-like-speech. Despite that, it should be noted that speech-to-text and speech synthesis is still an unsolved problem for non-standard accents. We could argue that there isn't enough data on the non-standard accents, and yes, that's what this particular problem boils down to.

More classical problems. Statistical methods have been used to improve upon classical data structures and algorithms in computer science (there was one beating quicksort recently). Analyzing the numbers for patterns, then building a data structure that's best suited for those patterns, before deciding how to handle them, sounds pretty sensible. The landmark achievement here is that this model can be trained and executed faster than the corresponding classical algorithm.

Now, we're ready to see machines fall flat on their faces when presented with problems that are effortless for humans.

You're given a sequence of stones, and each stone is represented by a unique symbol. What can you say about the following sequence?

[.*, %\!, &^#?]

A kindergartner would simply say that there are more stones on the right. Open-ended problems like this present a formidable challenge to machines. How is it supposed to begin approaching the problem? It is already stuck with the concrete, which it is looking at much too closely.

Two identical twins, jump off from a height of 10ft simultaneously. The first time, they jump without touching each other, and the second time, they hold hands. What is the difference between the two flight timings?

This thought experiment, if not already familiar, should be a simple task for any human to simulate in their heads. It is often used to illustrate the elegance of a simple physical law. Machines have no idea what to do other than to try some kind of correlation on existing data, or if height of 10ft is relevant.

I've drawn a certain kind of matrix here. Do you see what I'm trying to show you?

*
**
***
****
*****

Anyone with minimal exposure to linear algebra would immediately say that it's a lower-triangular matrix, even if they've never seen it presented that way. Yes, it's vague, but this is a classic abstraction problem, working across domains. The machine, in this case, might identify some kind of right-triangle in the textual picture, but what's the next step? How does this connect to a rectangle filled with numbers? Where are the numbers?

Notice that very little knowledge is needed to solve the above problems, and collecting more data isn't going to change a thing.

Solving these seemingly childish problems might not be of much consequence, but they are prerequisites for tackling more complicated problems:

Given that a hedgehog is a rodent that some people keep as pets, which of these six pictures is the picture of a hedgehog? Derive the equation of motion for a simple pendulum from the axioms of classical mechanics. Watch a film, and critically comment on the acting, screenplay, direction, and storyline. Deliver a talk on black holes, and gauge the level of understanding of the audience from the questions you receive. Check the correctness of the proof of the abc conjecture, by directly reading the document uploaded on arXiv.

To claim that there even is such as thing as a "singularity", requires, at the very least, some kind of strategy to tackle the last problem.

At this point, the data-zealots would interject. If we collected all the data of every second of every human's life for a year, given enough storage space and compute power, we'd be able to tackle at least some of these problems using current methods, and our methods are only going to get better over time, they'd argue. Even if it were true, it is a ridiculous claim, akin to something along the lines of "given enough time, I can write a graphical web browser that runs on bare-metal, from scratch, by writing a sequence of 0s and 1s on a piece of paper". Actually, it's even more ridiculous than that, because you'd need entire galaxies of low-wage unskilled workers to "clean" and "annotate" the collected data to spoon-feed the machine. Is this the "singularity" you talk about, where humans are doing the equivalent of cleaning ditches to keep the omniscient being happy? If you can't do kindergartner-level problems today without human assistance, with so much compute power at your disposal, what hope is there of claiming any "intelligence"? But, they'd argue, what if that machine has an intelligence that's "different", but somehow "subsumes" human intelligence? This is the kind of pseudoscientific rubbish that's spewed by nutty cult leaders. Statistical methods have a firm place in modern society, but as it currently stands, machines are the ones with the dunce hats in play-school.

We're currently in a data-warzone era, where all the big players are racing to collect more data on their users. What kind of data? Boring, inconsequential, everyday lives of human beings. To put it uncharitably, go through everyone's garbage with computing clusters, and you're bound to find some half-eaten chocolates. The models behind AlphaGo, GPT-3, and AlphaFold have the same essential nature as the ones used to keep us endlessly hooked onto social platforms, and to sell us unwanted products via sweet-talking voice-assistants. A lot of industrial AI research is enormously expensive, but the dirty secret is that it is paid for with the privacy of decent unassuming folk.

Let strangers sift through your garbage if you please, but don't do it while being drunk on the "singularity" kool-aid. $\Box$

The pedestrian technology underneath the hype

2025-07-02T14:18:51+01:00

We argue that, underneath all the large-language model hype, there is a pedestrian technology that has the potential to automate a class of problems that classical software can never do.

To study the capabilities of large-language models in the context of writing code, we study two extreme cases. First, the application to a critical project with high cognitive complexity, consisting of 40 million lines of code, and second, the application to a simple from-scratch toy project.

For the first study, I tried incorporating the technology into everyday-contributions to LLVM. The suggestions have over a 95% reject-rate, and the good suggestions appear when you're editing one instance of a mechanical pattern, and want to change all instances.

For example, consider the following transformation:

    if (match(I, m_c_Xor(
          m_CombineOr(m_ZextOrTrunc(m_Specific(P1)), m_Specific(P1)),
          m_CombineOr(m_ZextOrTrunc(m_Specific(P2)), m_Specific(P2)))))

to:

    if (match(I, m_c_Xor(m_CastOrSelf(m_Specific(P1)),
                         m_CastOrSelf(m_Specific(P2)))))

This can, in theory, be done by a find-and-replace with a regex, but nobody ever does that for a couple instances.

As another example, consider updating argument names and corresponding comments like:

/// Checks that \p L and \p R are used together in an XOR in the use-def chain
/// of \p SI's condition, ignoring any casts. The purpose of this function is to
/// ensure that LHSAux from the SimpleRecurrence is used correctly in the CRC
/// computation. We cannot check the correctness of casts at this point, and
/// rely on the KnownBits propagation to check correctness of the CRC
/// computation.
static bool isConditionalOnXorOfPHIs(const SelectInst *SI, const PHINode *L,
                                     const PHINode *R, const Loop &Lp);

to:

/// Checks that \p P1 and \p P2 are used together in an XOR in the use-def chain
/// of \p SI's condition, ignoring any casts. The purpose of this function is to
/// ensure that LHSAux from the SimpleRecurrence is used correctly in the CRC
/// computation. We cannot check the correctness of casts at this point, and
/// rely on the KnownBits propagation to check correctness of the CRC
/// computation.
static bool isConditionalOnXorOfPHIs(const SelectInst *SI, const PHINode *P1,
                                     const PHINode *P2, const Loop &L);

A find-and-replace would match the L in the Loop argument, erroneously replacing it with P1.

You could say that all the automation boils down to a smart and automatic find-and-replace in this study, but it is important to note this class of problems can never be automated by classical software.

For the second study, I tried applying the technology to writing a tree-sitter based LLVM IR parser. The entire task is a mechanical chore of reading docs and ample examples in the test suite, and encoding the knowledge in the parser. To study the limitations, I tried generating the entire parser using a prompt, but the result was entirely unusable. When writing the parser from a clean-slate, the suggestions were actually quite good. Consider the auto-complete with the following snippet:

    binary_instruction: ($) =>

to:

    binary_instruction: ($) =>
      seq($._binary_op, $._type, $.ssa_value, $.ssa_value),
    _binary_op: () =>
      token(
        choice(
          "add",
          "sub",
          "mul",
          "div",
          "mod",
          "shl",
          "shr",
          "and",
          "or",
          "xor",
        ),
      ),
    call_instruction: ($) =>
      seq(
        optional("tail"),
        "call",
        $._type,
        $.atname,
        "(",
        optional($.fn_args),
        ")",
      ),
    compare_instruction: ($) =>
      seq($._compare_op, $._type, $.ssa_value, $.ssa_value),
    _compare_op: () =>
      token(
        choice(
          "eq",
          "ne",
          "slt",
          "sgt",
          "sle",
          "sge",
          "ult",
          "ugt",
          "ule",
          "uge",
        ),
      ),
    cast_instruction: ($) =>
      seq($._cast_op, $._type, $.ssa_value, optional(seq("to", $._type))),
    _cast_op: () =>
      token(
        choice(
          "trunc",
          "zext",
          "sext",
          "fptrunc",
          "fpext",
          "fp_to_ui",
          "fp_to_si",
          "uitofp",
          "sitofp",
          "ptrtoint",
          "inttoptr",
          "bitcast",
        ),
      ),
    extract_instruction: ($) =>
      seq(token("extractvalue"), $._type, $.ssa_value, "[", $.ssa_value, "]"),
    insert_instruction: ($) =>
      seq(
        token("insertvalue"),
        $._type,
        $.ssa_value,
        $.ssa_value,
        "[",
        $.ssa_value,
        "]",
      ),
    load_instruction: ($) => seq(token("load"), $._type, $.ssa_value),
    phi_instruction: ($) =>
      seq(token("phi"), $._type, $.ssa_value, $.ssa_value),
    store_instruction: ($) =>
      seq(token("store"), $._type, $.ssa_value, $.ssa_value),
    switch_instruction: ($) =>
      seq(token("switch"), $._type, $.ssa_value, $.ssa_value),
    branch_instruction: ($) => seq(token("br"), $._type, $.ssa_value),
    return_instruction: ($) => seq(token("ret"), $._type, $.ssa_value),
    unreachable_instruction: ($) => token("unreachable"),

which is actually a smarter version of a copy-paste with some knowledge from the internet. Of course the code needs editing, but this is the kind of copy-pasting you'd normally do before making edits.

The technology was also able to generate 300 tests to exercise the parser, and the tests needed little tweaking to pass: this task can be boiled down to copying reduced-case LLVM IR from the test-suite, and matching them with parser nodes. Of course the generated tests aren't high-quality, but about half the tests are usable as-is, and a lot of tedium has been eliminated.

To conclude, the technology is at alpha-stage, as the automation comes at the cost of putting up with bad visual feedback and fighting the auto-complete nearly all the time, but promises a new class of mechanical automation, whose utility is higher in small codebases following mechanical patterns.

An opinionated history of programming languages

2020-09-30T12:43:32+02:00

We illustrate how different programming languages influnced each other, and include a quick FAQ write up.

An FAQ follows.

Why would someone pick C++ over Rust today?

C++ templates are incredibly powerful, and with the introduction of compile-time expressions, vast portions of modern C++ programs are just compile-time tables. Examples like this make Rust look very tiny:

template <size_t i, typename... Ts, typename CurTy>
void recurseFillChildren(CurTy &E)
{
  using PackTy = std::variant<Ts...>;
  using TyL = std::variant_alternative_t<i - 1, PackTy>;
  static_assert(std::is_same_v<CurTy, TyL>);
  using TyR = std::variant_alternative_t<i, PackTy>;

  for (i32 j = 0; j < E.NChildren; ++j)
  {
    E.Children.push_back(miniParser<TyR>());
    if constexpr (i + 1 < sizeof...(Ts))
      recurseFillChildren<i + 1, Ts...>(E.Children.back());
  }
};

Is there anything that Rust can do that C++ can't?

Rust didn't restrict what macros can do as much as C++ did. As a result, it's possible to do more with them. The particular example of the Pest parser is especially enlightening:

#[derive(Parser)]
#[grammar = "grammar.pest"]

Essentially, "grammar.pest" is processed at compile-time, and you can use definitions parsed from it in your code. However, there are trade-offs; Rust's macros are very slow, and your compile-times blow up if you have recursive macros. std::embed in C++23 will probably do it right.

Why is Haskell classified as a dying language?

Haskell is pleasant to get started with, and write relatively simple programs in. In the 2010-2015 period, there was a lot of intellectual discourse and PL research around it. The high-brow crowd was obsessed with transactional memory, parser combinators, and lenses. Online resources were exploding: LYAH and CatProg enjoyed their bouts of popularity. Several people and companies invested in Haskell heavily in that period. The language is easy to get started with, and has a pleasant development experience for relatively simple programs. The problems started when codebases started growing in size and complexity.

You either need to be able to interactively debug your program or prove that it is correct: in Haskell, you can't do either; the best you can do is to write some QuickCheck tests. Then there's Liquid Haskell that allows you to pepper your Haskell code with invariants that it will check using Z3. Unfortunately, it is very limited in what it can do: good luck checking your monadic combinator library with LH. Moreover, there are no tools to help you debug the most notorious kind of bug seen in a complicated codebase: memory blowups caused by laziness. It suffices to say that tooling is weak. In Atom, the Haskell add-on was terrible, and even today, in VSCode, the Haskell extension is among the most buggy language plugins.

There are over 120 language extensions, which can be turned on/off in each .hs file. The issue is that different extensions interact in subtle ways to produce bugs, and it's very difficult to tell if a new language extension will play well with the others (it often doesn't, until all the bugs are squashed, which can take a few years). The best case scenario plays out like this: GHC rejects your program, and suggests that you turn on some other language extensions; you turn them on, and you're left with a cryptic error message coming from a language extension you're not as familiar with; you spend the next N hours reading whatever little information is available about the more recent language extension, and decide to throw in the towel. The worst case plays out as follows: the type-checker hangs or crashes, and you're on the issue tracker searching for the issue; if you're lucky, you'll find a bug filed using 50~60% of the language extensions you used in your program, and you're not sure if it's the same issue; you file a new issue. In either case, your work has been halted.

There is almost zero documentation on language extensions. Hell, you can't even find the list of available language extensions with some description on any wiki. Looking at the big picture: first, this is a poor way to do software development; as the number of language extensions increase, your testing burden increases exponentially. Second, the problem of having a good type system is already solved by a simple dependent type theory; you study the core, and every new feature is just a small delta that fits in nicely with the overall model. As opposed to having to read detailed papers on each new language extension. And yes, there's a good chance that very few people will be able to understand your code if you're using some esoteric extensions. In summary, language extensions are complicated hacks to compensate for the poverty of Haskell's type system.

In practice, you'll be familiar with ~20 language extensions, and use various combinations of them over and over again, so the problem might not seem as acute as a regular Haskell programmer. However, PL research has shifted away from Haskell for the most part, and the little that happens tends to be unnecessarily complex nonsense that never sees the light of day.

Here's a sample of some simple Haskell code pieced together. I've intentionally left out examples using LinearTypes, because it's unfair to find faults with such a recent language feature.

What does the code mean? What is the intent? Can you specifically tell how each language extension was useful in the snippet? Guess what the code will do on GHC 8.10.1. What will an older version of GHC do? How many more language features are missing?

{-# LANGUAGE MagicHash #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE TypeInType #-}

import GHC.Exts

type family MatchInt (f :: Int) :: () where
  MatchInt ('I# _) = '()

{-# LANGUAGE PolyKinds #-}
{-# LANGUAGE RankNTypes #-}

f = undefined :: (forall (a :: k) m. m a -> Int) -> Int

{-# LANGUAGE PolyKinds #-}

data Proxy a = Proxy

class Foo (t :: k) where foo :: Proxy (a :: t)

{-# LANGUAGE DataKinds              #-}
{-# LANGUAGE FlexibleContexts       #-}
{-# LANGUAGE PolyKinds              #-}
{-# LANGUAGE TypeFamilyDependencies #-}
{-# LANGUAGE TypeOperators, AllowAmbiguousTypes          #-}

type family Dim v

type family v `OfDim` (n :: Dim v) = r | r -> n

(!*^) :: Dim m `OfDim` j -> Dim m `OfDim` i
(!*^) = undefined

Doesn't Go feature a solid compiler/runtime?

It does, but the author didn't feel that it was worthy of an asterisk, because the compiler is working with a really dumb language. Nevertheless, the GC and scheduler are praise-worthy, as is the overall experience with ultra-low compile-times.

Ra: Egyptian ink on parchment

2020-12-27T05:50:10+01:00

A picture of artwork.

The capitalist wet dream

2025-01-04T19:18:03+00:00

Capitalism works by getting a small number of skilled humans to produce something or offer some service that can then be marketed and sold to a vast consumer class. We live in very exciting times, for corporations have stumbled on a piece of technology that will take the capitalist wet dream to its absolute end: eliminating a vast number of skilled humans from the process, generating vast amounts of low-quality content instantaneously. Yes, generative AI will take jobs away from artists, writers, journalists, doctors, therapists, programmers, musicians, actors, lawyers, and several other professions. Anyone who tells you otherwise is lying. Capitalism has always been a race to the bottom: if your product or service offers something valuable at a competitive price, consumers will give you their money. Amazon and Uber are prime examples of this, and both create a sharp divide between the workers they exploit and the customers who're paying them. Whatever happens next with generative AI should therefore come as no surprise to anyone familiar with the basics of capitalism.

In this race to the bottom, there will be a vast number of people who can't afford human labor and are stuck in the free tier of ad-supported AI-generated content. The amount of such content available will explode by five or ten orders of magnitude, leading to greater consumption, and hence higher ad revenues. Most ads in this tier will be similarly AI-generated, as advertisers would also want to cut costs. Of course, corporations will say that the vast majority's quality of life has improved, as these poor people would now have access to a free AI doctor, therapist, and companion, AI summaries of content they can't afford, in addition to the vast amounts of low-quality misinformation and entertainment. The number of skilled humans required will shrink, as will the output: for the average person, buying such content will become a luxury. Wealthy people today already live in separate worlds, and they're already a different class of consumer, so it's not very hard to imagine what the world will be like once this technology becomes widespread.

Lengthy articles and debates on the internet seriously evaluating generative AI technology amuse me. It's supposed to be cheap unglamorous rubbish that you peddle to the masses, and I'm sure the people making the calls at corporations are chuckling too.

AI is something you pay to avoid, just like ads. $\Box$

My 2025 in LLVM: samesign, HashRecognize, and VPlan

2025-12-08T18:13:01+00:00

As the year draws to a close, it helps to look back and reflect on some key changes in LLVM. The article is written from my perspective, summarizing work I landed, along with adjacent work, in chronological order.

In September 2024, Nikita Popov proposed signedness-independent integer-compares, and the necessary work for enabling optimizations with it occupied the first two months of my 2025. In January, Florian Hahn introduced the initial version of a pattern-matcher for ScalarEvolution, which I would flesh out and use to improve code over the course of the year.

March and April were not very productive, as I was on vacation and attending EuroLLVM. In the little time I had, I fixed a vectorization bug and improved code by introducing a new iterator in a fundamental data structure.

In May, I introduced a constant folder in VPlan, guided by Florian, and fixed another vectorization bug. In collaboration with Mel Chen, I made a small improvement to the routine that checks patterns for vectorization.

In June, an analysis to recognize cyclic-redundancy-check loops landed, but required several months of improvements, before finally enabling optimization of CRC loops in September: the work was done in close collaboration with Piotr Fusik. While reviewing one of Florian's patches, I also extended a pattern for vectorization, which Florian built upon.

In July, I was occupied with instruction selection and cost-modeling work for the intrinsics lrint, lround, and ldexp on RISC-V, guided by Craig Topper.

In August, Luke Lau used parameter packs to greatly improve the pattern-matcher in VPlan. Inspired by another of his patches, I would introduce a integer/floating-point compare matcher, using it for optimization benefit. I would also fix some issued related to wrap-flags in VPlan.

In September, I introduced a common-subexpression-elimination transform in VPlan, guided by Florian. I also picked up and completed one of Luke's abandoned patches, a match functor. Nikita would introduce a new transform to strip unnecessary llvm.assume instructions.

In October, I would obsess over the loop-invariant code motion in VPlan, and factor out the legality check for hoisting/sinking, fixing a miscompile. My routine would bail out on memory operations due to aliasing issues, and I was thinking about how to get aliasing information in VPlan.

In November, I would fix long-standing issues related to wrap-flags in VPlan, and make an improvement to a routine to undo sub-optimal widening decisions in VPlan. Florian would build upon my improvement to yield the final version. He would also create specialized hoisting/sinking routines for memory operations, based on the noalias metadata: I plan to re-use the legality checks to improve the general case.

Finally, we're in December. The main outstanding patches are directly unrolling a recipe in VPlan, and introducing a carry-less multiply intrinsic, whose design was guided by Craig and Piotr. Both will hopefully land soon.

Although I wasn't either directly or indirectly involved, llvm-lit --update-tests introduced in August, has completely eliminated the tedium associated with mass test-updates!

An ABI-mismatch bug

2015-08-23T16:25:56-05:00

Our custom max function returns the maximum of two positive integers, and returns the negative integer, given a positive and negative integer. Classic signed wrapping, you'd think. It's not so simple, as the problem reproduces only under the following circumstances:

LLVM code is calling the max function. The library containing the max function is compiled without debugging information. The entire world has been built with XCode 6+.

More information: The LLVM IR is exactly the same between an XCode 5 sandbox, an XCode 6 sandbox, and a GNU/Linux sandbox. The corresponding assembly diff is also clean.

The corresponding C++ code (is actually specialized with short) is:

template <typename T>
FORCEINLINE T operator()(T a, T b) const
{
   return (a >= b ? a : b);
}

While in lldb, the assembly instructions look like:

--------------- [XCode 5]
-> 0x108af1930:  pushq  %rbp
   0x108af1931:  movq   %rsp, %rbp
   0x108af1934:  cmpw   %si, %di
   0x108af1937:  cmovgew %di, %si
   0x108af193b:  movswl %si, %eax
   0x108af193e:  popq   %rbp
   0x108af193f:  retq

--------------- [XCode 6]
-> 0x108d1ac10:  pushq  %rbp
   0x108d1ac11:  movq   %rsp, %rbp
   0x108d1ac14:  cmpl   %esi, %edi
   0x108d1ac16:  cmovgew %di, %si
   0x108d1ac1a:  movswl %si, %eax
   0x108d1ac1d:  popq   %rbp
   0x108d1ac1e:  retq

So, it's comparing the extended registers, but cmpl/cmpw difference is messing up somehow.

After execution of the cmpl/cmpw:

--------------- [XCode 5]
(lldb) register read --format int16_t[] di
      di = {-48}
(lldb) register read --format int16_t[] si
      si = {34}
(lldb) register read --format int16_t[] edi
     edi = {-48 7103}
(lldb) register read --format int16_t[] esi
     esi = {34 7103}
(lldb) register read --format b rflags
  rflags = 0b0000000000000000000000000000000000000000000000000000001010010010

--------------- [XCode 6]
(lldb) register read --format int16_t[] di
      di = {-48}
(lldb) register read --format int16_t[] si
      si = {34}
(lldb) register read --format int16_t[] edi
     edi = {-48 0}
(lldb) register read --format int16_t[] esi
     esi = {34 0}
(lldb) register read --format b rflags
  rflags = 0b0000000000000000000000000000000000000000000000000000001000010010
                                                                     ^
                                                             Sign flag different

--------------- [XCode 6, with C code calling max]
(lldb) register read --format int16_t[] di
      di = {-48}
(lldb) register read --format int16_t[] si
      si = {34}
(lldb) register read --format int16_t[] edi
     edi = {-48 -1}
                 ^
         In the LLVM call, this is zero
(lldb) register read --format int16_t[] esi
     esi = {34 0}

In conclusion, this is a Clang/LLVM version mismatch. It's compiling muIntScalarMax_sint16 to emit a compl instead of a compw. The fact that the bug only reproduces with LLVM is because: LLVM doesn't set up the extended form of the register correctly for negative numbers, when calling a function with a word-sized register.

The SysV ABI does not guarantee that the i16 will be sext'ed to i32. Clang ABI is probably an enhancement over the SysV ABI, and LLVM 3.5 doesn't know about this.

A tour of the LLVM backend

2024-08-04T21:35:05+01:00

Any modern compiler has three main components: a front-end that parses the source language, and translates it to a middle-end intermediate representation, a middle-end that operates on this IR, performing target-independent optimizations with a bit of target-specific information, and a backend that takes the final optimized middle-end IR and emits target-specific assembly. In this article, we illustrate the operation of a the main components of a compiler's backend, namely instruction selection, instruction scheduling, and register allocation, using LLVM as the reference compiler. LLVM is especially good for the purposes of illustration, as it has, in addition to a clean human-readable middle-end IR, a relatively clean human-readable backend IR. We use the RISC-V target for the purposes of concrete illustration, chosen once again for its clean design.

Let us start with a simple C program as our guiding example, and have a look at each step of the backend lowering after running the entire middle-end optimization pipeline, all the way up to the final assembly.

#define N 64

void matmul(const float *restrict x, const float *restrict y,
            float *restrict res) {
  for (int i = 0; i < N; ++i)
    for (int j = 0; j < N; ++j)
      for (int k = 0; k < N; ++k)
        res[N * i + j] += x[N * i + k] * y[N * k + j];
}

First, we run the front-end, and the full middle-end optimization pipeline to obtain the optimized LLVM IR that's handed off to the backend. Running clang --target=riscv64 -march=rv64gc -O3 -emit-llvm -S emits:

define void @matmul(ptr noalias %x, ptr noalias %y, ptr noalias %res) {
entry:
  br label %ph

ph:
  %iv.outer = phi i64 [ 0, %entry ], [ %iv.outer.next, %common.exit ]
  %shl = shl nuw nsw i64 %iv.outer, 6
  br label %common.ph

exit:
  ret void

common.ph:
  %iv.middle = phi i64 [ 0, %ph ], [ %iv.middle.next, %inner.loop.exit ]
  %or = or disjoint i64 %iv.middle, %shl
  %gep.res = getelementptr inbounds float, ptr %res, i64 %or
  %load.res = load float, ptr %gep.res
  %gep.y = getelementptr float, ptr %y, i64 %iv.middle
  br label %inner.loop

common.exit:
  %iv.outer.next = add nuw nsw i64 %iv.outer, 1
  %exit.cond.outer = icmp eq i64 %iv.outer.next, 64
  br i1 %exit.cond.outer, label %exit, label %ph

inner.loop.exit:
  store float %conv.next, ptr %gep.res
  iv.middle.next = add nuw nsw i64 %iv.middle, 1
  %exit.cond.middle = icmp eq i64 %iv.middle.next, 64
  br i1 %exit.cond.middle, label %common.exit, label %common.ph

inner.loop:
  %iv = phi i64 [ 0, %common.ph ], [ %iv.next, %inner.loop ]
  %conv = phi float [ %load.res, %common.ph ], [ %conv.next, %inner.loop ]
  %or = or disjoint i64 %iv, %shl
  %gep.x = getelementptr inbounds float, ptr %x, i64 %or
  %load.x = load float, ptr %gep.x
  %shl.iv = shl nuw nsw i64 %iv, 6
  %gep.y.2 = getelementptr float, ptr %gep.y, i64 %shl.iv
  %load.y = load float, ptr %gep.y.2
  %conv.next = tail call float @llvm.fmuladd.f32(
               float %load.x, float %load.y, float %conv)
  %iv.next = add nuw nsw i64 %iv, 1
  %exit.cond = icmp eq i64 %iv.next, 64
  br i1 %exit.cond, label %inner.loop.exit, label %inner.loop
}

The astute reader will note that we have strategically omitted the vector extension of RISC-V (we use -march=rv64gc instead of -march=rv64gcv) to avoid unnecessarily complicating the exposition due to the vectorization passes in LLVM. Auto-vectorization has already been discussed previously.

The first thing to notice is that the body of the function is cleanly separated into basic blocks: entry, ph, common.ph, inner.loop, inner.loop.exit, common.exit, and exit. Next, notice that the IR is in single-static-assignment (SSA) form, which means is that variables like %iv are only ever defined once. Since this is a loop that requires incrementing the induction variable, a fresh %iv.next variable is created, and %iv is a phi node that selects between the values 0 when jumping into the loop block from entry, and %iv.next when jumping to the inner.loop block from the back-edge of the inner loop. All the instructions (load, getelementptr, store, or, shl etc.) are target-independent instructions that are used throughout the LLVM middle-end. The target-specific instructions will only be seen after instruction selection.

The first step in the lowering process is instruction selection. We can look at the output of this step by running llc --print-after-isel:

Function Live Ins: $x10 in %18, $x11 in %19, $x12 in %20

bb.0 (%ir-block.3):
  successors: %bb.1(0x80000000); %bb.1(100.00%)
  liveins: $x10, $x11, $x12
  %20:gpr = COPY $x12
  %19:gpr = COPY $x11
  %18:gpr = COPY $x10
  %22:gpr = COPY $x0
  %21:gpr = COPY %22:gpr

bb.1 (%ir-block.4):
  successors: %bb.3(0x80000000); %bb.3(100.00%)
  %0:gpr = PHI %18:gpr, %bb.0, %9:gpr, %bb.4
  %1:gpr = PHI %21:gpr, %bb.0, %8:gpr, %bb.4
  %2:gpr = nuw nsw SLLI %1:gpr, 6
  %24:gpr = COPY $x0
  %23:gpr = COPY %24:gpr
  PseudoBR %bb.3

bb.2 (%ir-block.7):
  PseudoRET

bb.3 (%ir-block.8):
  successors: %bb.6(0x80000000); %bb.6(100.00%)
  %3:gpr = PHI %19:gpr, %bb.1, %11:gpr, %bb.5
  %4:gpr = PHI %23:gpr, %bb.1, %10:gpr, %bb.5
  %25:gpr = nuw nsw SLLI %4:gpr, 2
  %26:gpr = ADD killed %25:gpr, %19:gpr
  %27:gpr = LUI 4
  %5:gpr = ADD killed %26:gpr, killed %27:gpr
  %28:gpr = OR %4:gpr, %2:gpr
  %29:gpr = SLLI killed %28:gpr, 2
  %6:gpr = ADD %20:gpr, killed %29:gpr
  %7:fpr32 = FLW %6:gpr, 0
  PseudoBR %bb.6

bb.4 (%ir-block.15):
  successors: %bb.2(0x04000000), %bb.1(0x7c000000);
              %bb.2(3.12%), %bb.1(96.88%)
  %8:gpr = nuw nsw ADDI %1:gpr, 1
  %9:gpr = ADDI %0:gpr, 256
  %33:gpr = ADDI $x0, 64
  BEQ %8:gpr, killed %33:gpr, %bb.2
  PseudoBR %bb.1

bb.5 (%ir-block.18):
  successors: %bb.4(0x04000000), %bb.3(0x7c000000);
              %bb.4(3.12%), %bb.3(96.88%)
  FSW %15:fpr32, %6:gpr, 0
  %10:gpr = nuw nsw ADDI %4:gpr, 1
  %11:gpr = ADDI %3:gpr, 4
  %32:gpr = ADDI $x0, 64
  BEQ %10:gpr, killed %32:gpr, %bb.4
  PseudoBR %bb.3

bb.6 (%ir-block.21):
  successors: %bb.5(0x04000000), %bb.6(0x7c000000);
              %bb.5(3.12%), %bb.6(96.88%)
  %12:gpr = PHI %0:gpr, %bb.3, %17:gpr, %bb.6
  %13:gpr = PHI %3:gpr, %bb.3, %16:gpr, %bb.6
  %14:fpr32 = PHI %7:fpr32, %bb.3, %15:fpr32, %bb.6
  %30:fpr32 = FLW %12:gpr, 0
  %31:fpr32 = FLW %13:gpr, 0
  %15:fpr32 = nofpexcept FMADD_S killed %30:fpr32,
              killed %31:fpr32, %14:fpr32, 7, implicit $frm
  %16:gpr = ADDI %13:gpr, 256
  %17:gpr = ADDI %12:gpr, 4
  BEQ %16:gpr, %5:gpr, %bb.5
  PseudoBR %bb.6

What we see is a relatively straight-forward translation of the middle-end LLVM IR to machine IR (MIR), preserving the basic block structure and SSA-form. The purpose of instruction selection is to select a combination of instructions available on the target that optimally represents the middle-end semantics. Its two primary functions are to replace middle-end SSA variables by virtual registers, and to replace middle-end instructions by target-specific instructions.

Each virtual register has a suffix of either gpr (general-purpose register) or fpr (floating-point register). There are also physical registers in this MIR: $x0, $x10, $x11, $x12, $frm. Per the RISC-V calling convention, the three arguments to the function are in general-purpose registers $x10, $x11, and $x12 (pretty-printed as a0, a1, and a2). $x0 is a special register on RISC-V that always holds the value 0. $frm is the floating-point rounding-mode register. None of these can be virtual registers. A minor point to notice is the annotations on the virtual registers, killed and implicit. The operands of the FMADD_S, %30:fpr32 and %31:fpr32, are marked as killed, meaning that this is the final definition/use of those registers (these annotations are preliminary: the final annotations will be determined at a later stage). The register $frm is marked as implicit, meaning that it implicitly uses this register, and that this argument will be absent in the final assembly. The virtual registers must be turned into physical registers by the register allocator before emitting the final assembly.

Middle-end instructions like @llvm.fmuladd have been replaced by target-specific instructions like FMADD_S (pretty-printed as fmadd.s), and several pseudo-instructions like PseudoBR, PseudoRET, and COPY have been inserted in the process. The pseudo-instructions must be lowered to real instructions before emitting the final assembly.

There is some minor additional information in this MIR: the live-ins, or registers that are live at the point of entry, and the branch frequency information: bb.6 is going to be the successor basic-block of bb.6 97% of the time.

Although the instruction selection infrastructure, SelectionDAG †, is target-independent, we can see that our MIR is clearly specialized for RISC-V. All the information about the registers available on RISC-V are defined in RISCVRegisterInfo.td. The RISC-V instructions themselves come out of the specification RISCVInstrInfo.td, and the target-specific code that specifies how middle-end instructions should be lowered to instructions in this specification via SelectionDAG is RISCVISelLowering.cpp.

In order to understand the various steps after instruction selection, let us inspect the diffs produced at each step by running llc -print-changed=diff.

The first significant pass to run after the instruction selection is early-machinelicm, which is just a backend version of the middle-end loop invariant code motion transform. Next, we have livevars, which analyzes live variables, and marks registers as killed aggressively.

phi-node-elimination runs next, which removes PHI nodes, leaving the MIR in SSA form by inserting pseudo COPY instructions:

  Function Live Ins: $x10 in %18, $x11 in %19, $x12 in %20

  bb.0 (%ir-block.3):
   successors: %bb.1(0x80000000); %bb.1(100.00%)
   liveins: $x10, $x11, $x12
   %20:gpr = COPY killed $x12
   %19:gpr = COPY killed $x11
   %18:gpr = COPY killed $x10
   %22:gpr = COPY $x0
   %21:gpr = COPY killed %22:gpr
   %27:gpr = LUI 4
   %32:gpr = ADDI $x0, 64
+  %36:gpr = COPY killed %18:gpr
+  %37:gpr = COPY killed %21:gpr

 bb.1 (%ir-block.4):
   successors: %bb.3(0x80000000); %bb.3(100.00%)
-  %0:gpr = PHI %18:gpr, %bb.0, %9:gpr, %bb.4
-  %1:gpr = PHI %21:gpr, %bb.0, %8:gpr, %bb.4
+  %1:gpr = COPY killed %37:gpr
+  %0:gpr = COPY killed %36:gpr
   %2:gpr = nuw nsw SLLI %1:gpr, 6
   %24:gpr = COPY $x0
   %23:gpr = COPY killed %24:gpr
+  %38:gpr = COPY %19:gpr
+  %39:gpr = COPY killed %23:gpr
   PseudoBR %bb.3

 bb.2 (%ir-block.7):
   PseudoRET

 bb.3 (%ir-block.8):
   successors: %bb.6(0x80000000); %bb.6(100.00%)
-  %3:gpr = PHI %19:gpr, %bb.1, %11:gpr, %bb.5
-  %4:gpr = PHI %23:gpr, %bb.1, %10:gpr, %bb.5
+  %4:gpr = COPY killed %39:gpr
+  %3:gpr = COPY killed %38:gpr
   %25:gpr = nuw nsw SLLI %4:gpr, 2
   %26:gpr = ADD killed %25:gpr, %19:gpr
   %5:gpr = ADD killed %26:gpr, %27:gpr
   %28:gpr = OR %4:gpr, %2:gpr
   %29:gpr = SLLI killed %28:gpr, 2
   %6:gpr = ADD %20:gpr, killed %29:gpr
   %7:fpr32 = FLW %6:gpr, 0
+  %40:gpr = COPY %0:gpr
+  %41:gpr = COPY %3:gpr
+  %42:fpr32 = COPY killed %7:fpr32
   PseudoBR %bb.6

 bb.4 (%ir-block.15):
   successors: %bb.2(0x04000000), %bb.1(0x7c000000);
               %bb.2(3.12%), %bb.1(96.88%)
   %8:gpr = nuw nsw ADDI killed %1:gpr, 1
   %9:gpr = ADDI killed %0:gpr, 256
-  BEQ %8:gpr, %32:gpr, %bb.2
+  %36:gpr = COPY killed %9:gpr
+  %37:gpr = COPY %8:gpr
+  BEQ killed %8:gpr, %32:gpr, %bb.2
   PseudoBR %bb.1

 bb.5 (%ir-block.18):
   successors: %bb.4(0x04000000), %bb.3(0x7c000000);
               %bb.4(3.12%), %bb.3(96.88%)
   FSW killed %15:fpr32, killed %6:gpr, 0
   %10:gpr = nuw nsw ADDI killed %4:gpr, 1
   %11:gpr = ADDI killed %3:gpr, 4
-  BEQ %10:gpr, %32:gpr, %bb.4
+  %38:gpr = COPY killed %11:gpr
+  %39:gpr = COPY %10:gpr
+  BEQ killed %10:gpr, %32:gpr, %bb.4
   PseudoBR %bb.3

 bb.6 (%ir-block.21):
   successors: %bb.5(0x04000000), %bb.6(0x7c000000);
               %bb.5(3.12%), %bb.6(96.88%)
-  %12:gpr = PHI %0:gpr, %bb.3, %17:gpr, %bb.6
-  %13:gpr = PHI %3:gpr, %bb.3, %16:gpr, %bb.6
-  %14:fpr32 = PHI %7:fpr32, %bb.3, %15:fpr32, %bb.6
+  %14:fpr32 = COPY killed %42:fpr32
+  %13:gpr = COPY killed %41:gpr
+  %12:gpr = COPY killed %40:gpr
   %30:fpr32 = FLW %12:gpr, 0
   %31:fpr32 = FLW %13:gpr, 0
   %15:fpr32 = nofpexcept FMADD_S killed %30:fpr32,
               killed %31:fpr32, killed %14:fpr32,
               7, implicit $frm
   %16:gpr = ADDI killed %13:gpr, 256
   %17:gpr = ADDI killed %12:gpr, 4
-  BEQ %16:gpr, %5:gpr, %bb.5
+  %40:gpr = COPY killed %17:gpr
+  %41:gpr = COPY %16:gpr
+  %42:fpr32 = COPY %15:fpr32
+  BEQ killed %16:gpr, %5:gpr, %bb.5
   PseudoBR %bb.6

The next pass liveintervals undoes many of the killed annotations inserted by livevars, by performing a more sophisticated analysis to determine the actual virtual registers killed in the loop-nest.

The final preparatory step before instruction scheduling and register allocation is register-coalescer, which eliminates many of the COPY instructions, taking the MIR out of SSA form:

 Function Live Ins: $x10 in %18, $x11 in %19, $x12 in %20

 bb.0 (%ir-block.3):
   successors: %bb.1(0x80000000); %bb.1(100.00%)
   liveins: $x10, $x11, $x12
   %20:gpr = COPY $x12
   %19:gpr = COPY $x11
-  %18:gpr = COPY $x10
-  %22:gpr = COPY $x0
-  %21:gpr = COPY %22:gpr
+  %0:gpr = COPY $x10
   %27:gpr = LUI 4
   %32:gpr = ADDI $x0, 64
-  %36:gpr = COPY %18:gpr
-  %37:gpr = COPY %21:gpr
+  %1:gpr = COPY $x0

 bb.1 (%ir-block.4):
   successors: %bb.3(0x80000000); %bb.3(100.00%)
-  %1:gpr = COPY %37:gpr
-  %0:gpr = COPY %36:gpr
   %2:gpr = nuw nsw SLLI %1:gpr, 6
-  %24:gpr = COPY $x0
-  %23:gpr = COPY %24:gpr
-  %38:gpr = COPY %19:gpr
-  %39:gpr = COPY %23:gpr
+  %3:gpr = COPY %19:gpr
+  %4:gpr = COPY $x0
   PseudoBR %bb.3

 bb.2 (%ir-block.7):
   PseudoRET

 bb.3 (%ir-block.8):
   successors: %bb.6(0x80000000); %bb.6(100.00%)
-  %4:gpr = COPY %39:gpr
-  %3:gpr = COPY %38:gpr
   %25:gpr = nuw nsw SLLI %4:gpr, 2
   %26:gpr = ADD %25:gpr, %19:gpr
   %5:gpr = ADD %26:gpr, %27:gpr
   %28:gpr = OR %4:gpr, %2:gpr
   %29:gpr = SLLI %28:gpr, 2
   %6:gpr = ADD %20:gpr, %29:gpr
-  %7:fpr32 = FLW %6:gpr, 0
+  %42:fpr32 = FLW %6:gpr, 0
   %40:gpr = COPY %0:gpr
   %41:gpr = COPY %3:gpr
-  %42:fpr32 = COPY %7:fpr32
   PseudoBR %bb.6

 bb.4 (%ir-block.15):
   successors: %bb.2(0x04000000), %bb.1(0x7c000000);
               %bb.2(3.12%), %bb.1(96.88%)
-  %8:gpr = nuw nsw ADDI %1:gpr, 1
-  %9:gpr = ADDI %0:gpr, 256
-  %36:gpr = COPY %9:gpr
-  %37:gpr = COPY %8:gpr
-  BEQ %8:gpr, %32:gpr, %bb.2
+  %1:gpr = nuw nsw ADDI %1:gpr, 1
+  %0:gpr = ADDI %0:gpr, 256
+  BEQ %1:gpr, %32:gpr, %bb.2
   PseudoBR %bb.1

 bb.5 (%ir-block.18):
   successors: %bb.4(0x04000000), %bb.3(0x7c000000);
               %bb.4(3.12%), %bb.3(96.88%)
-  FSW %15:fpr32, %6:gpr, 0
-  %10:gpr = nuw nsw ADDI %4:gpr, 1
-  %11:gpr = ADDI %3:gpr, 4
-  %38:gpr = COPY %11:gpr
-  %39:gpr = COPY %10:gpr
-  BEQ %10:gpr, %32:gpr, %bb.4
+  FSW %42:fpr32, %6:gpr, 0
+  %4:gpr = nuw nsw ADDI %4:gpr, 1
+  %3:gpr = ADDI %3:gpr, 4
+  BEQ %4:gpr, %32:gpr, %bb.4
   PseudoBR %bb.3

 bb.6 (%ir-block.21):
   successors: %bb.5(0x04000000), %bb.6(0x7c000000);
               %bb.5(3.12%), %bb.6(96.88%)
-  %14:fpr32 = COPY %42:fpr32
-  %13:gpr = COPY %41:gpr
-  %12:gpr = COPY %40:gpr
-  %30:fpr32 = FLW %12:gpr, 0
-  %31:fpr32 = FLW %13:gpr, 0
-  %15:fpr32 = nofpexcept FMADD_S %30:fpr32,
-              %31:fpr32, %14:fpr32, 7, implicit $frm
-  %16:gpr = ADDI %13:gpr, 256
-  %17:gpr = ADDI %12:gpr, 4
-  %40:gpr = COPY %17:gpr
-  %41:gpr = COPY %16:gpr
-  %42:fpr32 = COPY %15:fpr32
-  BEQ %16:gpr, %5:gpr, %bb.5
+  %30:fpr32 = FLW %40:gpr, 0
+  %31:fpr32 = FLW %41:gpr, 0
+  %42:fpr32 = nofpexcept FMADD_S %30:fpr32,
+              %31:fpr32, %42:fpr32, 7, implicit $frm
+  %41:gpr = ADDI %41:gpr, 256
+  %40:gpr = ADDI %40:gpr, 4
+  BEQ %41:gpr, %5:gpr, %bb.5
   PseudoBR %bb.6

The purpose of the instruction scheduler, machine-scheduler, is to re-order instructions to minimize hazards and stalls. The optimal schedule depends on micro-architectural details, and an example of a scheduler descriptor is RISCVSchedSiFive7.td, for the SiFive 7 CPU. In our invocation of Clang, we did not specify a particular CPU, so some generic information for RISC-V is used. The optimal schedule can be seen in the following diff:

 Function Live Ins: $x10 in %18, $x11 in %19, $x12 in %20

 bb.0 (%ir-block.3):
   successors: %bb.1(0x80000000); %bb.1(100.00%)
   liveins: $x10, $x11, $x12
   %20:gpr = COPY $x12
   %19:gpr = COPY $x11
   %0:gpr = COPY $x10
+  %1:gpr = COPY $x0
   %27:gpr = LUI 4
   %32:gpr = ADDI $x0, 64
-  %1:gpr = COPY $x0

 bb.1 (%ir-block.4):
   successors: %bb.3(0x80000000); %bb.3(100.00%)
+  %4:gpr = COPY $x0
   %2:gpr = nuw nsw SLLI %1:gpr, 6
   %3:gpr = COPY %19:gpr
-  %4:gpr = COPY $x0
   PseudoBR %bb.3

 bb.2 (%ir-block.7):
   PseudoRET

 bb.3 (%ir-block.8):
   successors: %bb.6(0x80000000); %bb.6(100.00%)
-  %25:gpr = nuw nsw SLLI %4:gpr, 2
-  %26:gpr = ADD %25:gpr, %19:gpr
-  %5:gpr = ADD %26:gpr, %27:gpr
   %28:gpr = OR %4:gpr, %2:gpr
   %29:gpr = SLLI %28:gpr, 2
   %6:gpr = ADD %20:gpr, %29:gpr
   %42:fpr32 = FLW %6:gpr, 0
+  %25:gpr = nuw nsw SLLI %4:gpr, 2
+  %26:gpr = ADD %25:gpr, %19:gpr
+  %5:gpr = ADD %26:gpr, %27:gpr
   %40:gpr = COPY %0:gpr
   %41:gpr = COPY %3:gpr
   PseudoBR %bb.6

 bb.4 (%ir-block.15):
   successors: %bb.2(0x04000000), %bb.1(0x7c000000);
               %bb.2(3.12%), %bb.1(96.88%)
   %1:gpr = nuw nsw ADDI %1:gpr, 1
   %0:gpr = ADDI %0:gpr, 256
   BEQ %1:gpr, %32:gpr, %bb.2
   PseudoBR %bb.1

 bb.5 (%ir-block.18):
   successors: %bb.4(0x04000000), %bb.3(0x7c000000);
               %bb.4(3.12%), %bb.3(96.88%)
   FSW %42:fpr32, %6:gpr, 0
   %4:gpr = nuw nsw ADDI %4:gpr, 1
   %3:gpr = ADDI %3:gpr, 4
   BEQ %4:gpr, %32:gpr, %bb.4
   PseudoBR %bb.3

 bb.6 (%ir-block.21):
   successors: %bb.5(0x04000000), %bb.6(0x7c000000);
               %bb.5(3.12%), %bb.6(96.88%)
   %30:fpr32 = FLW %40:gpr, 0
   %31:fpr32 = FLW %41:gpr, 0
   %42:fpr32 = nofpexcept FMADD_S %30:fpr32, %31:fpr32,
               %42:fpr32, 7, implicit $frm
   %41:gpr = ADDI %41:gpr, 256
   %40:gpr = ADDI %40:gpr, 4
   BEQ %41:gpr, %5:gpr, %bb.5
   PseudoBR %bb.6

We're now ready to run the greedy register allocator †, whose purpose is to assign every virtual register a physical register, while re-using the limited number of physical registers on the machine, and minimizing register spills. The actual replacement is done by virtregrewriter:

 Function Live Ins: $x10, $x11, $x12

 bb.0 (%ir-block.3):
   successors: %bb.1(0x80000000); %bb.1(100.00%)
   liveins: $x10, $x11, $x12
-  %20:gpr = COPY $x12
-  %19:gpr = COPY $x11
-  %0:gpr = COPY $x10
-  %1:gpr = COPY $x0
-  %27:gpr = LUI 4
-  %32:gpr = ADDI $x0, 64
+  renamable $x16 = COPY $x0
+  renamable $x17 = LUI 4
+  renamable $x5 = ADDI $x0, 64

 bb.1 (%ir-block.4):
   successors: %bb.3(0x80000000); %bb.3(100.00%)
-  %4:gpr = COPY $x0
-  %2:gpr = nuw nsw SLLI %1:gpr, 6
-  %3:gpr = COPY %19:gpr
+  liveins: $x5, $x10, $x11, $x12, $x16, $x17
+  renamable $x29 = COPY $x0
+  renamable $x6 = nuw nsw SLLI renamable $x16, 6
+  renamable $x28 = COPY renamable $x11
   PseudoBR %bb.3

 bb.2 (%ir-block.7):
 ; predecessors: %bb.4

   PseudoRET

 bb.3 (%ir-block.8):
   successors: %bb.6(0x80000000); %bb.6(100.00%)
-  %28:gpr = OR %4:gpr, %2:gpr
-  %29:gpr = SLLI %28:gpr, 2
-  %6:gpr = ADD %20:gpr, %29:gpr
-  %42:fpr32 = FLW %6:gpr, 0
-  %25:gpr = nuw nsw SLLI %4:gpr, 2
-  %26:gpr = ADD %25:gpr, %19:gpr
-  %5:gpr = ADD %26:gpr, %27:gpr
-  %40:gpr = COPY %0:gpr
-  %41:gpr = COPY %3:gpr
+  liveins: $x5, $x6, $x10, $x11, $x12, $x16, $x17, $x28, $x29
+  renamable $x13 = OR renamable $x29, renamable $x6
+  renamable $x13 = SLLI killed renamable $x13, 2
+  renamable $x7 = ADD renamable $x12, killed renamable $x13
+  renamable $f15_f = FLW renamable $x7, 0
+  renamable $x13 = nuw nsw SLLI renamable $x29, 2
+  renamable $x13 = ADD killed renamable $x13, renamable $x11
+  renamable $x14 = ADD killed renamable $x13, renamable $x17
+  renamable $x15 = COPY renamable $x10
+  renamable $x13 = COPY renamable $x28
   PseudoBR %bb.6

 bb.4 (%ir-block.15):
   successors: %bb.2(0x04000000), %bb.1(0x7c000000);
               %bb.2(3.12%), %bb.1(96.88%)
-  %1:gpr = nuw nsw ADDI %1:gpr, 1
-  %0:gpr = ADDI %0:gpr, 256
-  BEQ %1:gpr, %32:gpr, %bb.2
+  liveins: $x5, $x10, $x11, $x12, $x16, $x17
+  renamable $x16 = nuw nsw ADDI killed renamable $x16, 1
+  renamable $x10 = ADDI killed renamable $x10, 256
+  BEQ renamable $x16, renamable $x5, %bb.2
   PseudoBR %bb.1

 bb.5 (%ir-block.18):
   successors: %bb.4(0x04000000), %bb.3(0x7c000000);
               %bb.4(3.12%), %bb.3(96.88%)
-  FSW %42:fpr32, %6:gpr, 0
-  %4:gpr = nuw nsw ADDI %4:gpr, 1
-  %3:gpr = ADDI %3:gpr, 4
-  BEQ %4:gpr, %32:gpr, %bb.4
+  liveins: $x5, $x6, $x7, $x10, $x11, $x12, $x16,
+           $x17, $x28, $x29, $f15_f
+  FSW killed renamable $f15_f, killed renamable $x7, 0
+  renamable $x29 = nuw nsw ADDI killed renamable $x29, 1
+  renamable $x28 = ADDI killed renamable $x28, 4
+  BEQ renamable $x29, renamable $x5, %bb.4
   PseudoBR %bb.3

 bb.6 (%ir-block.21):
   successors: %bb.5(0x04000000), %bb.6(0x7c000000);
               %bb.5(3.12%), %bb.6(96.88%)
-  %30:fpr32 = FLW %40:gpr, 0
-  %31:fpr32 = FLW %41:gpr, 0
-  %42:fpr32 = nofpexcept FMADD_S %30:fpr32, %31:fpr32,
-              %42:fpr32, 7, implicit $frm
-  %41:gpr = ADDI %41:gpr, 256
-  %40:gpr = ADDI %40:gpr, 4
-  BEQ %41:gpr, %5:gpr, %bb.5
+  liveins: $x5, $x6, $x7, $x10, $x11, $x12, $x13, $x14,
+           $x15, $x16, $x17, $x28, $x29, $f15_f
+  renamable $f14_f = FLW renamable $x15, 0
+  renamable $f13_f = FLW renamable $x13, 0
+  renamable $f15_f = nofpexcept FMADD_S killed renamable $f14_f,
+                     killed renamable $f13_f, killed renamable $f15_f,
+                     7, implicit $frm
+  renamable $x13 = ADDI killed renamable $x13, 256
+  renamable $x15 = ADDI killed renamable $x15, 4
+  BEQ renamable $x13, renamable $x14, %bb.5
   PseudoBR %bb.6

Before we emit the assembly, there are two pending steps: first, basic blocks need to be re-ordered, so that the final structure is flat and there are no unnecessary jumps; second, pseudo-instructions like COPY need to be replaced by real instructions. The final MIR after branch-folder and postrapseduos is:

Function Live Ins: $x10, $x11, $x12

bb.0 (%ir-block.3):
  successors: %bb.1(0x80000000); %bb.1(100.00%)
  liveins: $x10, $x11, $x12
  $x16 = ADDI $x0, 0
  renamable $x17 = LUI 4
  renamable $x5 = ADDI $x0, 64

bb.1 (%ir-block.4):
  successors: %bb.2(0x80000000); %bb.2(100.00%)
  liveins: $x5, $x10, $x11, $x12, $x16, $x17
  $x29 = ADDI $x0, 0
  renamable $x6 = nuw nsw SLLI renamable $x16, 6
  $x28 = ADDI $x11, 0

bb.2 (%ir-block.8):
  successors: %bb.3(0x80000000); %bb.3(100.00%)
  liveins: $x5, $x6, $x10, $x11, $x12, $x16, $x17, $x28, $x29
  renamable $x13 = OR renamable $x29, renamable $x6
  renamable $x13 = SLLI killed renamable $x13, 2
  renamable $x7 = ADD renamable $x12, killed renamable $x13
  renamable $f15_f = FLW renamable $x7, 0
  renamable $x13 = nuw nsw SLLI renamable $x29, 2
  renamable $x13 = ADD killed renamable $x13, renamable $x11
  renamable $x14 = ADD killed renamable $x13, renamable $x17
  $x15 = ADDI $x10, 0
  $x13 = ADDI $x28, 0

bb.3 (%ir-block.21):
  successors: %bb.4(0x04000000), %bb.3(0x7c000000);
              %bb.4(3.12%), %bb.3(96.88%)
  liveins: $x5, $x6, $x7, $x10, $x11, $x12, $x13, $x14,
           $x15, $x16, $x17, $x28, $x29, $f15_f
  renamable $f14_f = FLW renamable $x15, 0
  renamable $f13_f = FLW renamable $x13, 0
  renamable $f15_f = nofpexcept FMADD_S killed renamable $f14_f,
                     killed renamable $f13_f, killed renamable $f15_f,
                     7, implicit $frm
  renamable $x13 = ADDI killed renamable $x13, 256
  renamable $x15 = ADDI killed renamable $x15, 4
  BNE renamable $x13, renamable $x14, %bb.3

bb.4 (%ir-block.18):
  successors: %bb.5(0x04000000), %bb.2(0x7c000000);
              %bb.5(3.12%), %bb.2(96.88%)
  liveins: $x5, $x6, $x7, $x10, $x11, $x12, $x16, $x17, $x28, $x29, $f15_f
  FSW killed renamable $f15_f, killed renamable $x7, 0
  renamable $x29 = nuw nsw ADDI killed renamable $x29, 1
  renamable $x28 = ADDI killed renamable $x28, 4
  BNE renamable $x29, renamable $x5, %bb.2

bb.5 (%ir-block.15):
  successors: %bb.6(0x04000000), %bb.1(0x7c000000);
              %bb.6(3.12%), %bb.1(96.88%)
  liveins: $x5, $x10, $x11, $x12, $x16, $x17
  renamable $x16 = nuw nsw ADDI killed renamable $x16, 1
  renamable $x10 = ADDI killed renamable $x10, 256
  BNE renamable $x16, renamable $x5, %bb.1

bb.6 (%ir-block.7):
  PseudoRET

The assembly is produced from the final MIR by AsmPrinter, which doesn't do much more than pretty-printing registers ($x10 is a0, $x11 is a1), and instructions (FMADD_S is fmadd.s), while dropping auxiliary information. This output can be checked against the RISC-V ISA manual.

matmul:                                 # @matmul
# %bb.0:
  li a6, 0
  lui   a7, 4
  li t0, 64
.LBB0_1:                                # =>This Loop Header: Depth=1
                                        #     Child Loop BB0_2 Depth 2
                                        #       Child Loop BB0_3 Depth 3
  li t4, 0
  slli  t1, a6, 6
  mv t3, a1
.LBB0_2:                                #   Parent Loop BB0_1 Depth=1
                                        # =>  This Loop Header: Depth=2
                                        #       Child Loop BB0_3 Depth 3
  or a3, t4, t1
  slli  a3, a3, 2
  add   t2, a2, a3
  flw   fa5, 0(t2)
  slli  a3, t4, 2
  add   a3, a3, a1
  add   a4, a3, a7
  mv a5, a0
  mv a3, t3
.LBB0_3:                                #   Parent Loop BB0_1 Depth=1
                                        #     Parent Loop BB0_2 Depth=2
                                        # =>    This Inner Loop Header: Depth=3
  flw   fa4, 0(a5)
  flw   fa3, 0(a3)
  fmadd.s fa5, fa4, fa3, fa5
  addi  a3, a3, 256
  addi  a5, a5, 4
  bne   a3, a4, .LBB0_3
# %bb.4:                                #   in Loop: Header=BB0_2 Depth=2
  fsw   fa5, 0(t2)
  addi  t4, t4, 1
  addi  t3, t3, 4
  bne   t4, t0, .LBB0_2
# %bb.5:                                #   in Loop: Header=BB0_1 Depth=1
  addi  a6, a6, 1
  addi  a0, a0, 256
  bne   a6, t0, .LBB0_1
# %bb.6:
  ret

There is an alternative instruction selection framework called GlobalISel, and alternative register allocation algorithms in LLVM.

Incrementally updating the Dominator

2016-10-09T16:28:58-05:00

An input program function is represented using a Control Flow Graph (CFG). It's composed of different Statements, which decompose into Expressions, but we will not concern ourselves with Expressions. The main control flow constructs are SplitStatement and MergeStatement. The SplitStatement has outgoing edges to N (N = 2 in the case of IfStatement) other Statements, and the MergeStatement has incoming edges from M other Statements.

The Dominator is a data structure that holds "domination relationship" between any two Statements in the CFG. "A dominates B" means: all paths starting from the start of the CFG to end of the CFG, that flow through A, must necessarily flow through B. It's represented using a tree of things that progressively dominate their children. We want to incrementally update this structure.

Motivation

Any code motion transform will invalidate the Dominator at each step, and running Cooper or Lengauer-Tarjan on each invalidation is expensive even if the algorithm is near-linear. The number of queries is several times the number of reconstructions; we cannot trade off query-efficiency for ease-of-update.

Background

The main supporting data structure we will use for this feat is the Program Structure Tree, which is incrementally updated. The PST is a tree representation of structured control flow: all Control Flow Graph nodes belong to a Region in a nesting of single-entry single-exit Regions. Any control-flow that cannot be classified is boxed in an UnstructuredRegion, which we will not handle.

This is a Region:

for (i : Vec)
  A = i;

This is a ChainRegion comprising of an IfRegion and a BasicBlockRegion:

if (foo)
  A = 15;
else
  A = 21;
quux();

This is an UnstructuredRegion, but it is very much SESE:

jump:
  A = 18;
if (A < 17)
  A = A + 3;
  goto jump;

Updating a vanilla Dominator tree is relatively easy: simply add a child to the correct node. The problem at hand is that we have an O(1)-query Dominator object that we want to keep up to date as we make edits to the IR. How is it O(1)-query? Every node in the tree is assigned DFS start and end times: so if one interval nests within another, the corresponding node is dominated by the other node.

Supporting arbitrary IR edits is both very hard and not a useful goal: we can reduce all our edits to two items, namely inserting a SESE Statement, and inserting a SESE PST Region. Removing a Statement or Region is not interesting, because it just leaves gaps in the numbering, and doesn't affect any existing Dominance relationships if we make no changes after removal. Modifying a Region can be easily expressed as the removal of the old Region/insertion of the new Region.

The PST gives us an excellent mechanism to classify the blob we're injecting into the CFG. The interesting Regions begin with a SplitStatement, and end with a MergeStatement, or begin with a MergeStatement and end with a SpiltStatement: IfRegions, SwitchRegions, and LoopRegions. This is the key property we will use to do the incremental update.

Possible approaches

First, it must be noted that by using tight consecutive DFS start and end timers, we will have no choice but to renumber the descendant tree when something is inserted. So, we leave gaps of 64 between the numbering, to stuff our new SESE Statement or Region into. It may be argued that using fractions is profitable, but we found that space to fit intervals wasn't the bottleneck: the hard problem is determining dominance. When attempting to stuff a new Region into an existing gap, we can imagine centering around the midpoint of the gap, and leaving space for more Regions above and below. Bisecting each gap will exhaust the gaps quickly, and we decided to optimize for insertion only from top to bottom.

One approach we could take is to assume that the Region-to-be-inserted has been removed from a different part of the tree, and has some numbers already that we simply need to adjust them (those numbers will be spaced with gaps of 64, and we need to compress them). This approach wouldn't be able to handle newly created Regions.

Our algorithm

The key insight in our algorithm is that we need to match a Split Statement with its Merge Statement(s), and need to partition our DFS start/end time intervals into two categories: intervals to trust (ie. ones that exist, and should not be modified), and intervals to assign. The DFS start time corresponds to DFS first-visit sequence, and end time corresponds to DFS final-visit sequence. interval1.start_time > interval2.start_time and interval1.end_time < interval2.end_time means that interval1 is nested within interval2, or that the node corresponding to interval2 dominates the node corresponding to interval1.

void DFS(Node *v) {
  assign_start_time(v); // increment a clock as you assign
  for (auto u : v->outEdges()) {
    if (start_time_not_assigned(v)) DFS(v);
  }
  assign_end_time(v); // increment a clock as you assign
}

The reason we need to match a Split to a Merge is that there are two cases:

The Merge dominates the Split, in which case, the Merge dominates everything lying on the outEdges of the Split leading to the Merge. The Split dominates the Merge, in which case, the Split dominates everything on its outEdges leading to the Merge.

The intervals of the each of the outEdges ("branches") must stagger with each other so there's no false dominance relationship between them. Two intervals "staggering" means one not being nested within the other. In the case of Split-dominates-Merge, they must additionally stagger with the Merge.

Our API looks like:

void assignHeads(std::vector<Statement*> branchHeads,
                 std::vector<Statement*> headsToStaggerWith,
                 Statement* nestWithin, Statement* toEnclose);

First, we assign intervals to the split and merge Statements in the Region, by looking at their predecessor and successor. The predecessor is something we need to nest within. Since we decided that we'll only optimize for the case of top-to-bottom insertion, we can simply add one to the start time, and subtract one from the end time to get the necessary start and end times for the Region entry. For the Region exit, it's a little more tricky: we need to leave gaps on either side, because there may be nodes along the inEdge(s) as well as nodes along the outEdge(s). For simplicity, let's simply assign an interval halfway between the Region entry and successor intervals.

Then, we assign intervals to be branchHeads (ie. successors of the SplitStatement whose intervals we need to assign), by staggering with the headsToStaggerWith (ie. successors of the SplitStmt whose intervals we must trust + the MergeStatement if Split-dominates-Merge). nestWithin is the SplitStatement, and toEnclose is nullptr.

Finally, we trivially assign intervals to the rest of the Stmts in the Region, following the Split, by nesting them within the branch heads.

Now, this API generalizes to SESE Statements as well. branchHeads is the singular SESE Statement, headsToStaggerWith may or may not include the containing Region's MergeStatement. headsToStaggerWith will contain all the other successors of the SplitStatement, if the SESE Statement is one of the successors, and may contain the MergeStatement. If the successor has one inEdge, it's used as toEnclose.

The actual interval calculation is quite simple. Subtract the staggerUnion interval (union of all the stagger intervals) from the nestWithin interval: this produces two "gaps" which we need to "dice up" into N intervals (where N is the number of branchHeads). The number of intervals in the left gap versus the right gap is dependent on the ratio of the sizes of the gaps. toEnclose simply tells us whether to pick the left gap or right gap in the case of a single branchHead. It's also useful to check interval exhaustion cases.

There is an elegant way to catch all the cases when the interval has been exhausted. Just throw an exception from the interval-construction class. Catch this exception and abort the incremental update: recompute the Dominator.

Closing notes

Conceptually, the algorithm is simple. The implementation, as leveraged by one transform, passes our entire testsuite (~6k tests).

Our implementation of the Dominator incremental update relies on a correct implementation of the PST incremental update: incorrect information from one incremental update will propagate to the other incremental update and wreak havoc.

The best way to test the implementation is to have a recomputed Dominator as reference, and checking that the dominates-relationship between every two nodes is the same in the incremental update and re-computation case.

Thanks to Sanjoy Das for reviewing the draft.

An introduction to auto-vectorization with LLVM

2024-06-27T20:08:57+01:00

Most modern general purpose CPUs have a vector processing unit (VPU). This unit contains vector registers that can hold multiple integers or floats, and instructions that operate on vector registers. Given two vector registers containing N floats each †, the VPU can perform a single instruction to operate on these two vector registers and store the result in another vector register. Without a VPU, this operation would require N instructions operating on two floating-point registers each relying on the floating-point unit (FPU), or arithmetic logic unit (ALU) in the case of integers.

Major instruction set architectures have vector extensions, and corresponding instructions for the VPU. Examples include x86's SSE/AVX, AArch64's SVE/Neon, and the latest entrant is RISC-V's RVV. In order to generate vector instructions, the programmer could adapt their code to use standardized vector intrinsics. To illustrate, suppose the programmer had the following source code:

void saxpy_golden(size_t n, const float a, const float *x, float *y) {
  for (size_t iv = 0; iv < n; ++iv)
    y[iv] = a * x[iv] + y[iv];
}

If they wanted to use RISC-V vector intrinsics to vectorize it, this is how they would adapt the code:

void saxpy_vec(size_t n, const float a, const float *x, float *y) {
  for (size_t vl; n > 0; n -= vl, x += vl, y += vl) {
    vl = __riscv_vsetvl_e32m8(n);
    vfloat32m8_t vx = __riscv_vle32_v_f32m8(x, vl);
    vfloat32m8_t vy = __riscv_vle32_v_f32m8(y, vl);
    __riscv_vse32_v_f32m8(y, __riscv_vfmacc_vf_f32m8(vy, a, vx, vl), vl);
  }
}

As is evident, this process of hand-vectorizing code is very difficult and error-prone for non-trivial programs. The process is analogous to writing assembly instead of writing the source language and letting the compiler generate assembly. Indeed, compilers today auto-vectorize code, and for the purposes of illustration, let us use Clang/LLVM. First, the source language is lowered to target-independent LLVM IR by Clang, which is the frontend. Next, the vectorizers in the middle-end of LLVM operate on this LLVM IR without vectors in them, and produce LLVM IR with vectors in them. The vectorized LLVM IR is finally lowered to target-specific assembly by the backend of LLVM.

Let us take the same example of saxpy_golden, and see how the loop looks in LLVM IR without vectorization:

loop:
  %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
  %gep.x = getelementptr inbounds float, ptr %x, i64 %iv
  %load.x = load float, ptr %gep.x
  %gep.y = getelementptr inbounds float, ptr %y, i64 %iv
  %load.y = load float, ptr %gep.y
  %fmuladd = tail call float @llvm.fmuladd.f32(
    float %a, float %load.x, float %load.y)
  store float %fmuladd, ptr %gep.y
  %iv.next = add nuw i64 %iv, 1
  %exitcond = icmp eq i64 %iv.next, %n
  br i1 %exitcond, label %exit, label %loop

The first line, loop:, is like a label in C, to which branch instructions (br) can jump to. The next line is creating a new variable %iv using a phi to select either 0 when jumping from the entry block (not shown), or %iv.next when jumping from the back-edge of the loop: this is the induction variable of the loop. Since x and y are pointers (marked by the type ptr), we see getelementptr instructions that act as the address computations: so, %gep.x is %x + %iv, and %gep.y is %y + %iv. The load instructions derefence the pointer: so, %load.x is x[iv] and load.y is y[iv]; both are of type f32, or float. @llvm.fmuladd.f32 is the computation, which does a a * x[iv] + y[iv], and the store writes the result %fmuladd back into %gep.y. The last three instructions make the loop block a loop: %iv.next = %iv + 1 is the "next value" of the loop induction variable, and %exitcond is a boolean (marked by the type i1, integer with bitwidth 1) that acts as the exit-condition of the loop. %exitcond is computed as an integer-equal-comparison between %iv.next and %n, where n is the total number of iterations of the loop. Finally, there is a branch instruction that jumps to either the exit block (not shown), or back to the loop block (this is the back-edge of the loop), depending on the exit condtion.

This is how it changes after vectorization:

loop.preaheder:
  %n.and.minus4 = and i64 %n, -4
  %a.vec.tmp = insertelement <4 x float> poison, float %a, i64 0
  %a.vec = shufflevector <4 x float> %a.vec.tmp,
    <4 x float> poison, <4 x i32> zeroinitializer
  br label %loop

loop:
  %iv = phi i64 [ 0, %loop.preaheder ], [ %iv.next, %loop ]
  %gep.x = getelementptr inbounds float, ptr %x, i64 %iv
  %load.x = load <4 x float>, ptr %gep.x
  %gep.y = getelementptr inbounds float, ptr %y, i64 %iv
  %load.y = load <4 x float>, ptr %gep.y
  %fmuladd = tail call <4 x float> @llvm.fmuladd.v4f32(
    <4 x float> %a.vec, <4 x float> %load.x, <4 x float> %load.y)
  store <4 x float> %fmuladd, ptr %gep.y
  %iv.next = add nuw i64 %iv, 4
  %exitcond = icmp eq i64 %iv.next, %n.and.minus4
  br i1 %exitcond, label %exit, label %loop

The loop now has a loop.preheader block containing an %n.and.minus4 to handle the case when n is not a multiple of four. It also contains an insertelement and shufflevector, which essentially fill the vector %a.vec (that has type <4 x float>) with four copies of %a. The difference in the loop is that %iv.next is now %iv + 4, since it operates on four floats at a time, cutting the number of iterations by a factor of four. The load instructions are now loading four floats at a time, and the vector variant of @llvm.fmuladd.f32, which is @llvm.fmuladd.v4f32, does a %a.vec * %load.x + %load.y, operating on a vector of four floats at a time.

The vectorizer in LLVM is quite sophisticated, and it is fascinating how it manages to vectorize non-trivial examples. To walk through a simple but non-trivial example, consider:

for (size_t i = 0; i < n; ++i)
  y[i] += a * x[i] + i % 2 ? 0 : y[i];

Inspecting the output, we can see what the vectorizer has done. First, we see a new phi:

%vec.ind = phi <4 x i64> [ <i64 0, i64 1, i64 2, i64 3>, %vector.ph ],
  [ %vec.ind.next, %vector.body ]

It has created a new vector induction variable vector, corresponding to i, with four elements, since it has decided to vectorize with a factor of 4. Since %vec.ind is an integer, and the vectors are float, it has done a integer-to-floating-point conversion:

%10 = uitofp <4 x i64> %9 to <4 x float>

For the conditional itself, it has used a select to choose either 0 or the loaded value:

%15 = select <4 x i1> %12, <4 x float> zeroinitializer,
  <4 x float> %wide.load1

For the rest of this article, let us inspect some factors that could get in the way when the loop is expected to be vectorized, omitting cases where the loop is either obviously impossible or unprofitable to vectorize.

The iteration count. In most real-world code, the exact iteration count is unknown and the auto-vectorizers can deal with this. Auto-vectorizers can also deal with non-power-of-2 iteration counts by inserting a scalar epilogue with the remaining iterations. However, if the iteration count of the loop is computable and too small, the loop might get unrolled instead of vectorized. Aliasing information. If there isn't sufficient information to determine whether memory accesses in the loop are aliased to each other, runtime checks may be generated, or auto-vectorization might fail completely because insertion of runtime checks make it unprofitable to vectorize.

To illustrate, let's modify saxpy_golden as follows:

for (size_t i = 0; i < 3; ++i)
  y[i] += a * x[i];

Then, running clang -O3 -fno-unroll-loops -mllvm -vectorize-loops=false -emit-llvm -S test.c -o test.ll followed by opt -passes=loop-vectorize -debug-only=loop-vectorize test.ll outputs:

LV: Found a loop with a very small trip count.
  This loop is worth vectorizing only if no
  scalar iteration overheads are incurred.
LV: Found trip count: 3
LV: Not vectorizing

The overhead it is talking about is runtime checks it needs to add. Adding restrict to the pointers would remove the overhead, and allow it to be vectorized:

void saxpy_golden(const float a, const float *restrict x, float *restrict y)

Even if we were to ramp up the iteration count to 17, unrolling the loop would be preferred over vectorization. Forcing it not to unroll loops using -fno-unroll-loops as before results in successful vectorization. The final output contains a scalar version of the loop as well, marked by scalar.ph, which is required as 17 isn't divisible by 4.

The memory access pattern. A good rule of thumb write loops with uniform row-major access, and avoid complex indexing arithmetic.

The vectorizer has no issues with the following case:

void test(size_t n, size_t m, const float a,
          const float **restrict x, float **restrict y) {
  for (size_t i = 0; i < n; ++i)
    for (size_t j = i; j < m; ++j)
      y[i][j] += a * x[i][j];
}

Now, let's change the assignment statement in the loop to:

y[i][j] += a * x[j][i];

This is unfortunately not vectorized by LLVM, but there is no reason this is not theoretically vectorizable. The reason for this is that an analysis that is used to prove legality of vectorization in LLVM, called LoopAccessAnalysis, is unable to find the bounds of the memory access. Running opt -passes=loop-vectorize test.ll -disable-output -debug-only=loop-accesses outputs:

LAA: We can't vectorize because we can't find the array bounds.

Conditionals within the loop. Certain conditionals are possible to vectorize by either using select, or masking values in the vector using the condition. There is no way to tell if a certain conditional will be vectorized in advance, and depends on whether the pattern is implemented in LLVM's vectorizer.

LLVM is able to vectorize the example discussed previously:

for (size_t i = 0; i < n; ++i)
  y[i] += a * x[i] + i % 2 ? 0 : y[i];

Unfortunately, it is unable to vectorize this:

size_t rdx = 3;
for (size_t i = 0; i < n; ++i)
  rdx = (y[i] > x[i]) ? i : rdx;
return rdx;

The reason is as good as "it is unimplemented":

LV: Not vectorizing: Found an unidentified PHI
  %10 = phi i64 [ %16, %8 ], [ 3, %.preheader ]

Whether the loop is canonical. If, for instance, the loop has multiple exits, LLVM cannot auto-vectorize it.

LLVM struggles with early-exit loops, although there is no reason this is not possible to vectorize:

size_t i = 0;
for (; i < n; ++i)
  if (y[i] > x[i]) {
    y[i] = x[i];
    break;
  }
return i;

The reason is as good as "it is unimplemented":

LV: Found an outside user for :   %.lcssa = phi float [ %11, %6 ]
LV: Not vectorizing: could not determine number of loop iterations.

Function calls within the loop. Except for calls to LLVM intrinsics (like the @llvm.fmuladd in the example discussed previously), the auto-vectorizer will fail if there are function calls within the loop that it didn't inline.

LLVM cannot vectorize:

for (size_t i = 0; i < n; ++i)
  printf("%f\n", a * x[i]);

AArch64 and RISC-V don't include a vector instruction for printf in their instruction set, and this is a fundamental limitation of the instruction set.

LV: Not vectorizing: Found a non-intrinsic callsite
  %13 = tail call i32 (ptr, ...) @printf(
    ptr noundef nonnull dereferenceable(1) @.str,
    double noundef %12)

An example of a vector instruction that's included in both AArch64 and RISC-V is lrint, and vectorizing a program with it will use the vector variant of the @llvm.lrint intrinsic in the target-independent LLVM IR. The output can be inspected on Godbolt.

void test(size_t n, const float a, const float *x, long *y) {
  for (size_t i = 0; i < n; ++i)
    y[i] += lrint(a * x[i]);
}

N is a power of two, bounded by vector register width.

Detecting loops

2016-07-16T16:26:50-05:00

Detecting loops in a directed graph can be tricky, depending on how you define your loop. If you only want to admit "natural loops", where the header of the loop dominates every node in the body, as well as the footer, we have a simple algorithm. If you want to go to the other extreme, and define the most general "strongly connected components", without regard for loops, you have Tarjan's SCC algorithm. However, I'm going to discuss a definition of loop that admits strange loops, but only things we'd intuitively call "loops".

There are five [†] reduced cases we must consider, where backedges are marked by dotted arrows:

Now, let's employ a simple DFS:

void DFS(Node *v) {
  assign_start_time(v); // increment a clock as you assign
  for (auto u : v->outEdges()) {
    if (start_time_not_assigned(v)) DFS(v);
  }
  assign_end_time(v); // increment a clock as you assign
}

We get the following start times and end times:

Let's call the first number "DFS number", and the second number "topo number", to indicate that this is the order that topological sort would have produced, in the absence of backedges (topological sort is meaningless when there are loops).

First pass: do the dfs, and number everything.

Second pass: find the backedge. When start_time[destination] < start_time[source] and end_time[destination] > end_time[source], we have a backedge from source to destination.

Third pass: find all the nodes in the loop. Here, we walk backwards from the source of the backedge until the loop header, and take everything that's "nested within" the loop header start and end times to be within the loop.

In the no loop case, there's nothing to do since no backedges were detected. Note that in the 6/7-2/5 and 6/7-3/4 combinations, both start times and end times are greater in one pair; this is how we identify crossedges.

In the early exit case, 2/7 and 4/5 are nested within 1/8, and we never reach exit target by walking backward from the source of the backedge.

In the multi-entry case, we correctly detect that 1/8 is not nested within 2/7, while 3/6 and 4/5 are: when a node reached via a backward walk doesn't nest within the header, the loop is classified as irreducible.

In the cuddled loops case, everything nests within 1/8, and the inner loop consists of 4/5, 3/6, and 2/7, all of which nest within 2/7. Finally, the shared header case is detected as two nested loops as well: a loop is identified by a unique backedge, not a unique header [‡]. The analysis is weak in that these cases are indistinguishable from normal nested loops.

Hat tip to Sanjoy Das for pointing out the fifth case. You might want to merge loops that share a header in a post-pass.

Inside a register allocator

2016-03-24T17:36:35-05:00

We are going to be discussing LLVM's Fast Register Allocator: you might like to open RegAllocFast.cpp and refer to it as we go through the article.

FastRegAlloc allocates linearly, going through instructions and their operands in order. It uses PhysRegState to keep track of the state of various physical registers: they can be 0 (regDisabled), 1 (regFree), 2 (regReserved), or a virtual register number (a large number). At the time of allocation, the full UseDef information is available (so you have information like LR.LastUse); a register can either be a <def> or a use. If IsImplicit is flipped, it could also be a implicit-def (<imp-def>) or implicit-use (<imp-use>).

%vreg349 = MOVZX32rm8 %vreg351, 1, %noreg, 0, %noreg

turns into

%ESI = MOVZX32rm8 %RDX, 1, %noreg, 0, %noreg

after allocation. The <kill> indicates that the instruction is the last use of %RDX. In another example,

CQO %RAX, %RDX, %RAX

turns into

CQO %RAX, %RDX, %RAX

Notice that the instruction references physical registers even before the allocation. That's because this instruction specifically wants to sign extend %RAX into %RDX (not any other register) for use in a later IDIV. Remember that IDIV operates on %RDX:%RAX as the numerator, and writes the quotient in %RAX, reminder in %RDX.

The instruction is modeled as MachineInstr, and the operands as MachineOperand. Note that a MO doesn't have to be a register: it could also be an immediate value; we use MO.isReg() to find out if it's a register, physical or virtual. In addition to the states shown in the pretty-print, MO can also be EarlyClobber, Dead, or Undef. A Dead MO implies Def, and indicates that the value defined is used no longer.

AllocateBasicBlock, which operates on MachineBasicBlock, can be separated into three different scans, that operate on register MOs. Before the first scan, we set up the LiveIns (registers coming in live from the previous MBB) as regReserved so they aren't clobbered. The first scan operates on physical registers that are allocatable Uses; it calls usePhysReg on them. At this stage, the physical register must be either regDisabled, regReserved, or regFree. It cannot be allocated to a virtual register. Why? Imagine you see:

CQO %RAX, %RDX, %RAX

Now, if %RAX were already allocated to a virtual register, we're basically screwed in this linear walk. Otherwise, kill it, mark it as regFree, and move on: we have done our part by completing the use of the register that was reserved earlier.

The second scan operates only on virtual register Uses. We add the register to LiveVirtRegs, via reloadVirtReg. If it didn't already exist in the map, we allocVirtReg it, which essentially gets the first regFree register not used in an instruction, and calls assignVirtToPhysReg on it. assignVirtToPhysReg is very simple: it just updates the PhysRegState mapping. Finally, reloadVirtReg updates the UsedInInstr map.

The third scan operates on physical and virtual registers that are Defs. If the register is a physical register, it does definePhysReg with regReserved unless MO.isDead(), in which case it's regFree. definePhysReg is very simple: it just puts the register in the state that was requested (regReserved or regFree, in this case).

To think about the problem, if we have,

%RAX = COPY %vreg342

we should always regReserve %RAX, right? Except if %RAX is dead. What about if it's an <imp-def>?

CQO %RAX, %RDX, %RAX

We didn't pass CQO %RAX explicitly, but that doesn't mean that a later instruction (for instance, IDIV) is not relying on this register's value. If we have an instruction between CQO and IDIV, that can potentially clobber %RAX, leading to a regalloc crash. Hence, we regReserve even if MO.isImplicit().

The third scan does defineVirtReg on virtual registers, to grab the first free physical register for the virtual register.

Note: the "pretty-prints" are generated by setting a breakpoint in AllocateBasicBlock and executing p MBB->dump().

The Vectorization-Planner (VPlan) in LLVM

2025-09-19T20:59:39+01:00

In a follow-up to the introductory article on auto-vectorization, we discuss how the loop vectorizer works internally. Nearly all transforms in LLVM work by directly manipulating IR, using the results of various analyses on the IR, while the loop vectorizer operates on a layer of abstraction above the IR: it lifts the LLVM IR to an overlay IR, analyzes and transforms the overlay IR, and finally removes the original IR entirely, replacing it with a final LLVM IR from the direct concretization of the final overlay IR. It also plans various vectorization strategies, costing each one until it finds a winning strategy. This infrastructure is called VPlan.

Let us start with a variant of the example program from the previous article:

void saxpy(const float a, const float *x, float *restrict y) {
  for (size_t iv = 0; iv < 1024; ++iv)
    y[iv] = a * x[iv] + y[iv];
}

As the vectorizer runs near the end of the pipeline, LLVM applies various scalar transforms to yield target-independent IR that the vectorizer gets as input:

loop:
  %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
  %gep.x = getelementptr inbounds float, ptr %x, i64 %iv
  %load.x = load float, ptr %gep.x
  %gep.y = getelementptr inbounds float, ptr %y, i64 %iv
  %load.y = load float, ptr %gep.y
  %fmuladd = tail call float @llvm.fmuladd.f32(
    float %a, float %load.x, float %load.y)
  store float %fmuladd, ptr %gep.y
  %iv.next = add nuw i64 %iv, 1
  %exitcond = icmp eq i64 %iv.next, 1024
  br i1 %exitcond, label %exit, label %loop

The planning stage of the vectorizer normally decides two key factors when attempting to find the winning plan: the vectorization factor (VF), which determines the width of the instructions on the VPU, and interleave count (UF, referring to the old name "unrolling factor"), which determines how many copies of each instruction to create. These decisions are predicated on the cost of the final instructions, which only makes sense when a specific target is provided: for the purposes of this article, let us force the vectorizer to run in a target-independent fashion without interleaving using -force-vector-width=4 -force-vector-interleave=1, and inspect the lifting of the LLVM IR to the VPlan:

VPlan 'Initial VPlan for VF={4},UF>=1' {
Live-in vp<%0> = VF
Live-in vp<%1> = VF * UF
Live-in vp<%2> = vector-trip-count
Live-in ir<1024> = original trip-count

vector.ph:
Successor(s): vector.body

vector.body:
  EMIT vp<%3> = CANONICAL-INDUCTION ir<0>, vp<%index.next>
  vp<%4> = SCALAR-STEPS vp<%3>, ir<1>, vp<%0>
  CLONE ir<%gep.x> = getelementptr inbounds ir<%x>, vp<%4>
  vp<%5> = vector-pointer ir<%gep.x>
  WIDEN ir<%load.x> = load vp<%5>
  CLONE ir<%gep.y> = getelementptr inbounds ir<%y>, vp<%4>
  vp<%6> = vector-pointer ir<%gep.y>
  WIDEN ir<%load.y> = load vp<%6>
  WIDEN-INTRINSIC ir<%fmuladd> =
    call llvm.fmuladd(ir<%a>, ir<%load.x>, ir<%load.y>)
  vp<%7> = vector-pointer ir<%gep.y>
  WIDEN store vp<%7>, ir<%fmuladd>
  EMIT vp<%index.next> = add nuw vp<%3>, vp<%1>
  EMIT branch-on-count vp<%index.next>, vp<%2>
}

First, we notice that each variable has an ir<> or vp<> annotation. The ir variables are variables present in the original LLVM IR or some kind of constants, and the vp variables are fresh variables that are created in the VPlan. Next, we notice new annotations like EMIT, CLONE, and WIDEN in each line in the vector loop: these are termed "recipes", from which the final LLVM IR will be emitted, post transformations.

Near the beginning of the vector loop, we can see that the vectorizer has analyzed the trip-count of the original loop (1024), along with a canonical induction-variable (%iv), and emitted a CANONICAL-INDUCTION recipe along with a SCALAR-STEPS that increments it by VF to account for the fact that the vectorized loop has width VF, and executes 1024 / VF times. This means that certain instructions must either be width VF, or duplicated VF number of times with different steps for correctness: in our snippet, we see that loads and stores were widened, along with the intrinsic fmuladd. getelementptr should not be widened: it should add VF times the original offset to the base pointer, and this is indeed what is done. For instructions that should not be widened, we see an EMIT/CLONE, or "replicate" recipes. This is also where the interleaving-factor would have come in, in a different example, when something that needs to be widened couldn't be widened.

Notice that the final line, branch-on-count, is predicated on an abstract vector-trip-count: since the vectorizer could make different decisions about VF and UF as it plans, the vector-trip-count is only materialized after a decision is made.

We see some cryptic vector-pointer recipes, but these will be optimized out by the time we get to the final VPlan. Also notice a minor detail in the operands of fmuladd: the first operand is %a, which is a scalar, while the other two operands, %load.x and %load.y, are widened; if the first operand isn't fixed up, vectorization of the entire loop would be inhibited for correctness reasons.

The final VPlan is produced post optimizations and materializations:

VPlan 'Final VPlan for VF={4},UF={1}' {
vector.ph:
  EMIT vp<%1> = broadcast ir<%a>
Successor(s): vector.body

vector.body:
  EMIT-SCALAR vp<%index> =
    phi [ ir<0>, vector.ph ], [ vp<%index.next>, vector.body ]
  CLONE ir<%gep.x> = getelementptr inbounds ir<%x>, vp<%index>
  WIDEN ir<%load.x> = load ir<%gep.x>
  CLONE ir<%gep.y> = getelementptr inbounds ir<%y>, vp<%index>
  WIDEN ir<%load.y> = load ir<%gep.y>
  WIDEN-INTRINSIC ir<%fmuladd> = call llvm.fmuladd(
    vp<%1>, ir<%load.x>, ir<%load.y>)
  WIDEN store ir<%gep.y>, ir<%fmuladd>
  EMIT vp<%index.next> = add nuw vp<%index>, ir<4>
  EMIT branch-on-count vp<%index.next>, ir<1024>
}

Notice that the first operand of the fmuladd is now a broadcast of %a, which is essentially a way to fill a vector with a scalar. It lowers to the following LLVM IR:

  %broadcast.splatinsert =
    insertelement <4 x float> poison, float %a, i64 0
  %broadcast.splat =
    shufflevector <4 x float> %broadcast.splatinsert,
    <4 x float> poison, <4 x i32> zeroinitializer

At this point, the VPlan is "executed" to produce the final LLVM IR. The full example can be seen on Godbolt.

VPlan allows us to focus on semantic details of the vectorization, and operate at an abstraction level that is easier to inspect and optimize, and less error-prone, than concrete LLVM IR.

Equality in Mechanized Mathematics

2020-02-15T18:12:59+01:00

Here, we talk about equalities, and provide illustrative examples in vanilla Rocq, SProp, and HoTT.

Equality in Mathematics

In zfc-based mathematics, say in abelian groups, $A \oplus B \simeq B \oplus A$, where the equality is a set-based equality. In mathematics based on category theory, equality is too strong a notion on objects, and only speak about objects "upto unique isomorphism"; morphisms in a $\mathrm{1}$-category can be equal though. In higher categories, and in particular, homotopy theory, we talk about "weak homotopy equivalences" almost fully replacing equality. However, it can be tricky to mechanize a theory based on $\infty$-categories; the best dependent-type-theory model we have is homotopy type theory, commonly referred to as HoTT, which we will discuss in the last section.

Universes in Rocq

First, let us briefly talk about the cumulative universe of Rocq. Prop is a Set that can be promoted to Type seamlessly. The reason for the distinction between Prop and Set is an engineering one: Prop is impredicative, while Set is not, and proofs are erased during extraction.

Definition SetTypeIncl (A : Set) : Type := A.
Definition PropTypeIncl (A : Prop) : Type := A.
Definition PropSetIncl (A : Prop) : Set := A.

There is an $\infty$ hierarchy within the Type universe, and types of Types are Types themselves.

Inhabitants of a Set are sets of primitive things like $\mathbb{N}$ and $\mathbb{R}$, while inhabitants of a Prop are propositions, which could be $\top$, $\bot$, or some arbitrary term, the inhabitant of which acts as the proof.

Propositions

There are two kinds of equalities in vanilla Rocq. The difference is as follows: propositional equality roughly translates to "requires proof obligation to be discharged by the user", whereas definitional equality is a simple syntactic rewriting in the metatheory. A propositional equality between propositions can be formalized as:

Axiom PropositionalProofIrrelevance : ∀ (P : Prop) (a b : P), a = b.

We then get a proof obligation which we discharge using the axiom:

Theorem PropEquality (P : Prop) (a b : P) : a = b.
Proof.
  by apply PropositionalProofIrrelevance.
Qed.

In the general case, two different inhabitants of Prop are unequal. There is, however, a propositional equality existing between two equal propositions:

Theorem PropRefl (P : Prop) (x : P): x = x.
Proof.
  exact eq_refl.
Qed.

where eq_refl is simply the sole inhabitant of an Inductive:

Inductive eq (A : Type) (x : A) : A -> Prop :=
  eq_refl : eq A x x.

The proof-irrelevant SProp

There are three inhabitants of SProp: sUnit corresponding to $\top$, sEmpty corresponding to $\bot$, and sProposition corresponding to a definitionally proof-irrelevant term. The way SProp implements definitional proof-irrelevance is a simple engineering detail: there is hard-coding in Rocq to render two inhabitants of sProposition trivially inter-convertible.

Unfortunately, = doesn't work as expected:

Theorem SPropIrr (P : SProp) (x y : P) : x = y.
Proof.
  by reflexivity.
Abort. (* Type-check fails at Qed.
        * (=) : ∀ A, A -> A -> Prop, but we want to return an SProp. *)

This is because the SProp universe is disjoint from the Prop universe:

Theorem SPropToProp : SProp -> Prop.
Proof.
  by intros x; exact x.
Abort. (* Type-check fails at Qed.
        * SProp is not convertible to Prop. *)

Fortunately, we can do better.

Mere propositions in homotopy type theory

In HoTT, there is just one $\infty$-ladder:

$\mathbb{1}$, and anything that's contractible to it, is the ($\mathrm{-2}$)-truncated Type. A "mere proposition", an inhabitant of hProp, is simply a ($\mathrm{-1}$)-truncated Type. hSet is the $\mathrm{0}$-truncated Type.

Notation Contr := (IsTrunc minus_two).
Notation IsHProp := (IsTrunc minus_two.+1).
Notation IsHSet := (IsTrunc minus_two.+2).

The notion of truncation is central to HoTT, where $A : \mathrm{Type}_n$ can be truncated to a $\mathrm{Type}_m$, whence higher-than-$m$ morphisms are rendered uninteresting. Any two hProps are propositionally equal:

Theorem hPropEquality (P : hProp) (a b : P) : a = b.
Proof.
  by apply path_ishprop.
Qed.

Conceptually, this is as simple and elegant as PropEquality.

Big thanks to Kenji Maillard for the scintillating discussion, and for helping with the examples. Kudos to Hugo Moeneclaey for giving a wonderful introduction to truncation in the last HoTT class. Thanks to Pierre-Marie Pédrot for pointing out some very serious errors.

An inquiry into the Foundations of Mathematics

2020-02-02T11:11:41+01:00

Starting from the question of what consequence Gödel's incompleteness theorem has on the foundations of mathematics, we argue for new foundations, and build some intuition for these new foundations in Coq.

Mathematics can be thought of as a game where you start with some objects (say, sets), some axioms on the objects (typically, ZFC), and use a calculus of logical deductions (such as classical logic) to prove things. To mechanize this game in a proof assistant, one would either use a proof-search or tactic system, and check subject reduction. However, mechanized mathematics is done differently from classical mathematics, and we first investigate whether building new foundations is justifiable.

Gödel's second incompleteness theorem states that any formal system, that can express elementary arithmetic, cannot prove its own consistency, assuming à priori that the system is consistent. It doesn't destroy the platonic view of mathematics as such, but does destroy the view that there is only one correct foundation for mathematics. Further, one would argue that it is precisely this result that makes mathematics exciting: we can keep rewriting foundations to better suit our abstract intuitions, and build higher abstractions, without worrying about whether there is "one true foundation".

Departing from ZFC foundations

Some mathematicians would then argue that we've settled on ZFC and classical logic as foundations, but this is only true of $\sim$80% of recent mathematical literature. Category theory, in particular, breaks away from set-theoretic foundations; objects and morphisms aren't sets. Indeed, set-theoretic questions about categories are ill-formed, although one can encode boolean algebras using topoi.

Moreover, one should aim for a foundation that is conducive to mechanization, so proofs can be checked for correctness in an automated fashion. The strategy that Voevodsky et al. used was to extend Martin-Löf type theory, and build a branch of mechanized mathematics, homotopy type theory, which we will refer to henceforth as HoTT. It is formalized in Coq, whose foundations are based in Martin-Löf Type Theory.

Preliminaries

What the following proposition says, is that, for every inhabitant of $\mathbb{N}$ that you supply, $f$ will supply an inhabitant of $\mathbb{1}$:

Theorem f : nat -> unit.
Proof.
  intros x.
  exact tt.
Qed.

The proof is witness that we can always supply an inhabitant of the type $\mathbb{N} \rightarrow \mathbb{1}$. By Curry-Howard isomorphism:

Definition f' : nat -> unit := fun _ => tt.

In type theory, a function is just a non-dependent version of the $\Pi$ type, where the codomain $B$ is fixed:

The logical connective $\neg$, which we will use in the next section, has the following definition in type theory:

Two intuitions in classical mathematics broken

Our first example is the classic proof-by-contradiction. A proof in constructive mathematics cannot proceed in the following logical sequence for a proposition $P$:

In other words,

This is in line with the general philosophy of constructive mathematics, where non-existence of an inhabitant of a type cannot be proved; to supply a witness is to supply a construction of the inhabitant, and indeed, no such construction might exist. Nevertheless, an axiom of excluded middle can be axiomatized and used to construct-via-contradiction, just like any other axiom:

Axiom LEM : forall (P : Prop), P \/ ~P.

To understand this better, let us take the example of the constructive equivalent of $\mathbb{0} \Rightarrow \textrm {anything}$ in classical mathematics:

Theorem FalseImpliesAnything : forall A, False -> A.
Proof.
  intros A.
  exact (fun x : False => match x with end).
Qed.

Since $\mathbb{0}$ has no inhabitants, the construction of any inhabitant of any type can be proved from it.

It is perhaps prudent to supply another example of a classical statement to illustrate Coq's type-checker:

Theorem TrueDoesNotImplyFalse : True -> False.
Proof.
  (* Due to enforced exhaustive pattern-matching on inhabitants of the LHS,
   * we'd need to match up an inhabitant of True (there is only I)
   * to some inhabitant of False, but there are no inhabitants of False.
   *)
Abort.

In category theory, $\mathbb{0}$ would be the initial object, and there is at most one arrow from the initial object to every other object in the category; $\mathbb{1}$ would be the final object, and there is at most one arrow from every other object in the category to it. The above theorem hypothesizes the existence of a morphism from the final object to the initial object, which of course, cannot exist.

For the second example, let us investigate the classical notion of proof irrelevance. In ZFC, sets and propositions are in different universes of discourse, as opposed to type theory, where proofs objects are first-class. Over the course of its development, the question of proof relevance was raised in type theory. In the following example, $p$ and $q$ are inhabitants of the type $x = y$, and the question is whether $p$ is different from $q$:

The answer, as supplied by HoTT, is: yes, proofs are relevant, and $p$ is different from $q$; a proof is viewed as a homotopy path between two homotopy types, and there is a notion of homotopy equivalence between homotopy paths. In classical mathematics, $p$ would always be equal to $q$.

Lean versus Rocq: The cultural chasm

2020-01-04T15:38:56+01:00

Both Lean and Rocq are proof assistants based on pCIC. Here, we argue that what sets them apart are, in essence, cultural differences.

Lean has much lower startup-cost for pure mathematicians, since its built-in features and math library are great for doing undergraduate-level group theory & topology, masters-level commutative algebra & category theory, but it plateaus quickly thereafter.

Lean seems to have taken a top-down approach, by focusing on writing real proofs as quickly as possible, without compromising on soundness. There are three axioms in Lean: propositional extensionality, quotient soundness, and choice; however, these don't block computation, since computation is done in a VM. They do, however, break good type theoretic properties like strong normalization, subject reduction, and canonicity — this was a conscious design choice.

Rocq, on the other hand, has always been very particular about sound type theoretic foundations. The recent "Coq Coq Correct!" formalizes the Rocq engine proving metatheoretic properties about it, in Rocq, and HoTT is being actively developed to fix many of Rocq's issues.

Working mathematicians have only just started using software for their work, and they often rely on unverified (and sometimes proprietary) tools like magma, sage, and mathematica. Lean is a big step-up, and the good type theoretic properties preserved by Rocq don't seem as important to them.

Lean supports both constructive and classical reasoning, but quotienting in Rocq is painful due to "setoid-hell"; something that would be fixed when types are modeled as $\infty$-groupoids instead of setoids, as in the HoTT line of work.

Quotient-reasoning makes formalizing commutative algebra painless, and it's baked into the Lean kernel. However, quotients are tricky to implement without breaking certain metatheoretic properties that Rocq'ers cherish; nevertheless, there is an implementation in Rocq's math-comp, albeit, without a reduction rule.

Lean's inheritance is another good feature; it disallows diamond-inheritance, and seems like a bit of a hack, but Rocq is definitely missing some form of a well-thought-out ad-hoc polymorphism.

Category theory in Lean has not been developed with higher categories in mind; I'm not sure how one would define $\infty$-categories, since universes are explicit, and since there are no coinductive types: Rocq's cumulativity seems to have been a better design choice.

One would think that Lean is an engineering feat, since it's written in C++: Rocq's math-comp (90k loc) compares to Lean's mathlib (150k loc); math-comp builds in under ten minutes, while mathlib takes over thirty minutes to build! Indeed, due to their design decision to use a VM for computations, computation happens at a speed comparable to Python-bytecode evaluation — they seem to be overhauling this in Lean4 though, by compiling Lean code down to C before execution.

Another reason for the relative slowness of Lean is that they do a lot of automation under the hood, figuring out, for instance, that when the user writes $X \times Y$, $X$ and $Y$ are topological spaces, and that $\times$ here means product topology.

Metamath Zero might be of interest to readers of this article. This article has developed over several discussions on coq-club and Lean Zulip.

Predicativity in Rocq

2020-03-08T07:27:41+01:00

Today, we write a quick specialized note on what impredicativity exactly means, for those reasonably familiar with the Rocq syntax.

Historically speaking, Rocq started out with making Set impredicative and they still carry around the flag --set-impredicative to maintain impredicativity in Sets. Let's check it quickly:

(* Set is predicative. *)
Fail Definition SetPred := (forall X : Set, X) : Set.

We use a little macro to check the impredicativity of various types, since an equivalent definition wouldn't type-check:

(* Checks that all inhabitants of U are of type V. *)
Notation "'check' U V" := ((forall X : U, X) : V) (at level 0, U, V at level 0).

All inhabitants of Prop are of type Prop, for good reason.

(* Prop is impredicative. *)
Definition PropImpr := check Prop Prop.

Attempt to check the predicativity of Type naïvely:

(* Is Type impredicative? This is misleading. *)
Definition TypeNaive := (forall X : Type, X) : Type.

The previous example was misleading, and we clarify it in the next example:

(* In the previous definition, Type is really Type@{i},
 * and the following definition fails with: Universe {Predicativity.153} is unbound. *)
Fail Definition TypePred@{i} := check Type@{i} Type@{i}.

The Type universe is cumulative, not impredicative. We show the cumulativity using algebraic universe notation:

(* Fails due to a bug in Rocq: Unable to handle arbitrary u+k <= v constraints? *)
Definition TypePred'@{i} := ((forall X : Type@{i}, X) : Type@{i+1}).

Definition TypePred'@{i} := check Type@{i} Type@{i+1}.

Gaëtan Gilbert: intricacies of universes in Rocq Hugo Herbelin: syntactic mastery over Rocq

ZFC and propositional logic

2020-01-18T08:02:23+01:00

Using a mélange of mathematical syntaxes, we mechanize some exercises from Tao's excellent book

Overture

Let us informally define sets and set membership $\in$. A set is an unordered collection of objects, with the set itself being an object. Set membership checks whether a given object is contained within a set:

The axiom of substitution uses the definition of equality:

The axiom of substitution is preserved for all sets only defined in terms of $\in$ and $=$.

Thanks to Chaitanya Leena Subramanium for suggesting the exercises and pushing for the use of natural deductions in the last exercise. Thanks to Hugo Herbelin for patiently going through many proofs and pointing out the errors.