Links and Notes for August 26th, 2022

Below are links to articles, highlighted passages[1], and occasional annotations[2] for the week ending on the date in the title, enriching the raw data from Instapaper Likes and Twitter. They are intentionally succinct, else they’d be articles and probably end up in the gigantic backlog of unpublished drafts. YMMV.

Table of Contents

• Economy & Finance
• Public Policy & Politics
• Science & Nature
• Art & Literature
• Philosophy & Sociology
• Technology
• Programming
• Video Games

Economy & Finance

The Asset Economy Strikes Again by Martijn Konings (Boston Review)

“[…] introduced a cycle of inflationary pressures: anticipating price increases, unions demanded wage increases to preserve purchasing power, which in turn helped push prices up. The wage-price spiral that developed in the 1970s represented a major headache for anyone invested in the monetary stability of the postwar order—the middle classes very much included.”

“[…] neoliberalism shifted inflationary pressures from wages to assets. Between 1982 and 2022 the total dollar value of U.S. corporate stock ballooned by a factor of 61; it had little more than doubled between 1962 and 1982. The growth of home values may seem less spectacular—an 18-fold increase over the last half century—but when adjusted for inflation and plotted against real wages, the latter is essentially flat.”

“[…] it is essential to recall how U.S. political economy has been remade to serve the interests of asset owners at the expense of wage-earners. Only by reckoning with this asset economy head on can we envision a future that breaks free of it.”

“systematically measuring purchasing power institutionalizes a strange feedback loop. Since wages are one of the key factors shaping prices, there is always the possibility that a wage increase will partly undo itself.”

“Traditionally, regulators and politicians had considered bailouts a major source of moral hazard, a way of rewarding irresponsible behavior that was impossible to explain to the tax-paying public. Had U.S. authorities continued in this spirit and decided to let failing firms fail, we would now be living in a very different world.”

“Much of what we think of as the sophisticated, fast-paced world of financial innovation is underpinned by the willingness of the U.S. state to put a floor under the value of asset classes.”

Got it in one.

“It was the era of the knowledge economy, and the power of training and education featured prominently in the progressive-neoliberal fantasy of the transubstantiation of labor into capital. As Bill Clinton and Al Gore put it, “what you earn depends on what you learn.””

“The Clinton administration’s proactive embrace of fiscal discipline and balanced budgets, not least through major cuts to welfare spending, largely relieved the Fed of having to police the government on that score. In combination with the steady weakening of organized labor, this arrangement meant that Alan Greenspan could focus on backstopping financial markets and promoting asset inflation without fear of price inflation”

“The experience of the roaring ’90s had reconciled them to the end of wage growth, and they now emerged as the all-too-familiar public figures who think of themselves as politically progressive but are constitutionally incapable of identifying with those who depend on a monthly paycheck.”

“This amounted to a full normalization of the bailout system—indeed, its proactive implementation. As Gerald Epstein and Robert Pollin have demonstrated in these pages, bailouts are not exceptions to the core logic of neoliberalism; they are its modus operandi.”

“Since the mid-2010s, as each round of asset purchases did less and less to help ordinary people and sired more and more new millionaires, the Fed became more aware of the contradictions of its own position. Cautiously, it sought to highlight the inherent difficulty of managing an economic system that had effectively ruled out social spending and wage increases.”

“The ability and willingness of corporations to increase their mark-ups and profits have been far more significant in turning the transitory inflation associated with pandemic supply chain disruptions into sustained upward pressure on consumer prices.”

“This logic requires not simply that workers bear the brunt of the problems that other people create but also that they acknowledge that doing so is in their own interest. Jane Elliott has argued that neoliberalism is not primarily interested in denying people agency, but in ensuring that they actively use their freedom to make their own situation worse. The strange bind that inflation discourse has imposed on workers—wage cuts will hurt you and your family, but all the other options are even worse—is emblematic of that state of affairs.”

“The real issue is not that they are wrong so much as beside the point, given that wages are not responsible for inflation in the first place. Should corporations see additional opportunities for raising prices to boost profits, Summers will just revise his numbers upward, demanding yet more sacrifice. The desire to drive down wages has become untethered from any actual reason for doing so or any plausible justification.”

“The real threat is not so much that the asset economy has definitively run out of steam; it’s that it will figure out new ways to keep going.”

“A new phase of asset-driven growth is, in any case, what the wealthy are counting on: they are currently busy buying up assets because the market lull is a good moment to add to one’s portfolio before things take off again. If the Federal Reserve is successful in what it euphemistically calls “stabilizing inflation,” it will be facing a more extreme version of the problem it navigated in past years, having to push even more liquidity into the system through asset purchases simply to keep the system going.”

Public Policy & Politics

Trump Dinner For One Hundred Thousand Bucks

They Hate US ‘Cause They Ain’t US! by Mark Ashwill (CounterPunch)

“While naturally condemning the terrorists’ heinous acts of mass murder, other more thoughtful voices of reason in that period of hysteria and bloodlust reflected on why young men would sacrifice their lives while slinging a few stones at the empire, taking thousands of innocents with them. It certainly wasn’t the aforementioned “freedoms” that motivated them to commit such acts of terror, nor was it the 72 virgins waiting for them in paradise, or the fact their countries aren’t US.”

“This fantasy enables those who believe in the political Santa Claus of US nationalism to live happily in a state of denial about the problems facing their country and the reasons for a higher quality of life in other countries that could serve as positive role models for the US if only there were eyes to see and ears to hear.”

“This nationwide scourge explains a lot about the current state of disunion. 54% of US adults between the ages of 16 and 74 lack proficiency in literacy. This shocking reality check is based on a report Assessing the Economic Gains of Eradicating Illiteracy Nationally and Regionally in the United States (PDF download) by Jonathan Rothwell, Principal Economist, Gallup, and Nonresident Senior Fellow, Brookings Institution, Metropolitan Policy Program published in 2020 by the Barbara Bush Foundation for Family Literacy. In a nutshell, people who are functionally illiterate cannot use reading, writing, and calculation skills for their own and the community’s development.”

“Under the current system both California, with a population of 40 million, and Wyoming, with 579,000 residents, get two senators. It is estimated that 40% of all US Americans will live in five states by 2040, which means half of the US states will be represented by 18 senators and the other half by 82.”

Senators were intended as a counterweight to mob rule (i.e. If the majority wants to legalize lynching foreigners, that would be democratic, but most people would probably agree that we need some sort of counterweight to help avoid such outcomes. With the increase in population shift, though, it’s not so clear that having the Senate weigh equally to the House of Representatives is the right fit. Eliminating the Senate isn’t either, though.

“In case you’re counting, so far in 2022, some 12,272 people nationwide have died due to firearms—including intentional and accidental killings but not suicides. At the current rate, this year’s total could approach the 2021 number of 20,944, a seven-year high, and exceed 2020’s 19,518 deaths.”

“Blacks are imprisoned for drug offenses at a rate 10 times greater than that of whites even though they both use drugs at about the same rates.”

“[…] the maximum wage offered by Micky Dees is still exploitation, unless you’re a young person living with Mom and Dad with few expenses. Working 40 hours a week at $15 an hour amounts to $31,200 a year – before taxes – flipping burgers, frying fries, and pouring liquid sugar without a vacation. It doesn’t buy you much in most places in the US of 2022.”

“The official US poverty rate was 11.4% in 2020, up 1% from 2019. Federal guidelines consider citizens to be “poor” if they are earning the following amounts per year: $12,880 (1 person), $17,240 (2), $21,960 (3), and $26,500 (4). The US has one of the highest child poverty rates among OECD countries. About one in six children are classified as poor using the conservative income threshold of $26,500 a year for a family of four.”

“Knowing the cost of living in much of the country, it’s clear that millions who earn more than a poverty wage are among the working poor living paycheck to paycheck with a dismally low savings rate. As of July 2022, 203 million Americans (61%) were living paycheck to paycheck,”

“Carlin was spot-on when he said that US Americans “are efficient, professional, compulsive consumers. Shopping – it’s their civic duty. Consumption – it’s the new national pasttime. Fuck baseball. The only true lasting American value that’s left is buyin’ things. People spending money they don’t have on things that they don’t need,” which also applies to their government.”

“In a country in which 10% of citizens owned 89% of stocks and mutual funds in the first quarter of 2021 while the bottom 50% of US households own around 0.5%, and where three men, Elon Musk, Jeff Bezos, and Bill Gates, are worth more than the bottom half of all US Americans, the American Dream is moribund.”

“George Carlin: “Bullshit is the glue that binds us as a nation. Where would we be without our safe, familiar, American bullshit? Land of the free. Home of the brave. The American dream. All men are equal. Justice is blind. The press is free. Your vote counts. Business is honest. The good guys win. The police are on your side. God is watching you. Your standard of living will never decline. Everything is going to be just fine.””

“Instead of trying to call the shots in all four corners of the earth, the US would be well-advised to get its own house in order and worry about what’s left of its fragile representative democracy in a system that is essentially an oligarchy hurtling towards authoritarianism and Christofascism.”

“Having lived in Vietnam for 17 years, I always return to the country of my birth and coming of age as an interested ethnographic researcher, mental notepad always at the ready.”

I felt the same, although I called myself an anthropologist.

“[…] the unalienable rights of life, liberty, and the pursuit of happiness are only available to those who can afford them.”

“The US government and a majority of its people have at least one thing in common: their ongoing obsession with external enemies, most imagined. Their most formidable foes are at home, if not looking back at them in the mirror.”

My First Seventy Years as an Ex-Pat by Victor Grossman (CounterPunch)

“Impressing me most as an American: no layoffs, no unemployment; there were jobs for everyone. Rents averaged less than 10% of most incomes; evictions were forbidden by law. In the early years large apartments were divided up when needed; no-one slept in the streets or went begging. Food pantries were unneeded, even the word was unknown. So was student debt. All education was free and monthly stipends covered basic costs, making all jobbing while at college unnecessary.”

“A monthly medical tax on wages or fees (max. 10%) covered everything: in my case, nine (free) hospital weeks with hepatitis plus four weeks at a health spa to recuperate and four more a year later in Karlsbad. My wife had three rheumatism cures, four weeks each, in the Polish and Harz mountains. All costs were covered and we also got 90% of our salaries.”

“East Germany was occupied by a country it had been taught to hate, whose soldiers had fought it hardest, were often violent in the first weeks, and were poorer and more difficult to love than prosperous, hence generous, gum-chewing GI’s, who came from a wealthy, undamaged homeland.”

“[…] the GDR had probably come closer than any country in the world to achieving that legendary goal of abolishing poverty, while sharply decreasing the frightful, growing rich-poor gap based on an obscene profit system.”

“The GDR citizenry took all its amazing social advantages for granted and dreamt of scarce bananas and unavailable VWs, of Golden Arch and Golden Gate – without realizing that these are largely available and affordable due to the poverty of children in West Africa or Brazil, of exploited pickers in Andalusian or Californian fields and orchards.”

Chat-controls: EU plans to abolish privacy for digital communications by Moritz Strohm (WSWS)

“A clear indicator that the combatting of child abuse is a pretext can be seen in cases in Germany where those spreading and producing child abuse images are investigated and arrested, but no efforts are made to delete these images from the internet, although they could be quickly taken off-line. Systems to search for such imagery can easily be re-purposed for alternative uses. They can also detect other content, as the technology does not differentiate between offending images or ones that are merely politically inconvenient.”

Getting the surveillance state in place. Europe is going off the rails.

“[…] in cases of child abuse images, mere suspicion is sufficient to destroy the reputation and life of the accused. Add to this the capabilities of the security agencies with sufficient powers to plant material on a target device or manipulate a harmless image so that it triggers a flag when sent.”

“These plans toward chat-controls show the hypocrisy of the ruling class: It uses child abuse as a pretext to establish a surveillance and censorship infrastructure, abusing victims of abuse a second time.”

“[…] these plans reveal the true character of the EU. These institutions, whose goals, according to official propaganda, are the unification of the continent under the umbrella of freedom and democracy, is developing the surveillance methods of a dictatorship. The EU is a capitalist state alliance, which helps its members impose anti-democratic measures.”

Roaming Charges: Nuclear Midnight’s Children by Jeffrey St. Clair (CounterPunch)

“One out of every five people in Kentucky live beneath the poverty line, but Mitch McConnell said this weekend that “the single most important thing going on in the world right now is to beat the Russians in Ukraine.””

“Marco Rubio: “We don’t need a military focused on the proper use of pronouns. We need a military focused on blowing up Chinese aircraft carriers.””

“There are some big lingering questions over Biden’s announcement proclaiming the assassination of Ayman al-Zawahiri, like what airspace did the killer drones traverse? It had to be Pakistan’s, which denies it. Yet Zawahiri’s killing took place shortly after Gen. Qamar Bajwa, chief of the Pakistan Army, asked the Biden administration for help in securing an urgently needed IMF loan to shore up his country’s cratering economy. But the biggest question of all is: where are Zawahiri’s remains? The Taliban says it’s found no evidence of Zawahiri’s body at the bomb site, where a missile supposedly killed him as he stood on the balcony of his hideout in a Kabul neighborhood. How long before the US drones Zawahiri again?”

“The US is still bombing Syria, which it justifies as a retaliation for Shia militia attacks against US troops, which are, for some reason, still in Syria!”

“The US represents 4% of the world’s population, 25% of global Covid deaths, 23% of Covid cases and 35% of all Monkeypox cases.”

“Jacob Silverman: “Someday we’re going to look back on this whole Covid disaster and laugh because we’ll all have 40% of our original brain matter and can barely process reality.””

“Nearly one-third of the rise in global temperatures can be attributed to methane. Atmospheric methane had its highest growth rate yet recorded by modern instruments in 2020. That record was broken again in 2021.”

11,000 Federal Inmates Were Sent Home During the Pandemic. Only 17 Were Arrested for New Crimes. by C.J. Ciamarella (Reason)

“Of the more than 11,000 federal inmates who were released to home confinement during the COVID-19 pandemic, 17 were returned to prison for committing new crimes, according to the Bureau of Prisons (BOP).

“In response to a query from Keri Blakinger, a reporter for The Marshall Project, the Bureau of Prisons said that of the 17, 10 committed drug crimes, while the rest of the charges included smuggling non-citizens, nonviolent domestic disturbance, theft, aggravated assault, and DUI.”

Short Take: A Meritorious Defense by Scott H. Greenfield (Simple Justice)

“Merit should never have become a battlefront in the culture wars. I understand the impulse to declare the system rigged when so many children, particularly Black and Hispanic children, have fallen behind academically. But the answer to racial disparities in math and reading scores and advanced academic enrollment is not to blame the game and re-rig it to favor outcomes that please certain political constituencies but do little to make life better for struggling children. The solution is to channel more resources into disenfranchised communities — from the Black urban poor to the white rural poor in my native West Virginia. The solution is not to give up on merit.”

The problem is one of money. There is precious little money in education relative to other programs (e.g. the military). Public education is not a priority because no-one profits directly from it.

They’ve now managed to add so much administration that the right elites are profiting from it—draining public coffers for oversized salaries and benefits while the actual teachers get little—but also to acknowledge and respond to the catastrophe that is the American educational landscape by shouting “charter schools” from every hilltop, which is just another way of saying “make the profit motive paramount in determining how education works”, which has never, ever gone awry any other time.

The incentives are false and education won’t get any better, but at least the right people will be benefitting from it, so certain constituencies will stop complaining loudly and those that continue to be harmed by poor education everywhere aren’t heard in either case, so we’ll comfort ourselves that the problem is solved because it’s “gotten quieter”.

Meanwhile people still aren’t being educated and the society has a dwindling supply of people around who know how to do useful things, but that’s a problem for another day because there’s mad cash to be made pretending to educate kids at charter schools.

What fascinates me is that charter schools basically function like universities, which are bloated and top-heavy with administration and endowments and fund-management that has nothing to do with education and every to do with making the right elites a lot of money but, somehow, people who hate universities as bastions of liberal and woke thought and constantly slam them for their inefficiencies and waste love charter schools with all their hearts and don’t waste a second wondering whether the same fate could befall them as befell the universities because they’re all based on the profit motive as incentive rather than the “build useful people so society doesn’t fall” motive.

The Cost of “Defund” by Scott H. Greenfield (Simple Justice)

“If I sound a bit, oh, miffed about all this, it’s because I am. The past few years have seen an opportunity that comes around once in a lifetime to make fundamental reforms to the legal system that were never possible while we were in the throes of fear of crime or hero worship of police. And we blew it. And we blew it for all the wrong reasons.

“Black Lives Matter could have been so amazingly useful in changing one of the worst transgressions of police culture, the assumption that all black people were prone to commit crime and be violent, and that treating black people as less than human was acceptable.”

Surging prices in Europe: The ruling class makes workers pay for the capitalist crisis by Peter Schwarz (WSWS)

“In Germany, the price of food has risen by 12.7 percent and that of energy by 38 percent since a year ago. Next month, prices are expected to surge further as several government relief measures expire on October 1 and the gas surcharge comes into effect, a kind of special tax on all end users to compensate energy companies for the loss of Russian energy supplies. In addition, the high world market prices for electricity and gas are beginning to be passed through to household bills. The price of electricity on the European Energy Exchange (EEX) has risen twenty-fold in some cases. The Bundesbank therefore expects the inflation rate to be well above 10 percent this winter.”

“In reality, the attack on the living standards of broad sections of the population is a continuation of the class war that the financial oligarchy has been waging with growing intensity against the working class since the 1980s.

“This is shown by the very fact that profits continue to grow—as they did during the financial crisis and the pandemic—while wages collapse.”

Some might argue that wages have risen, but that’s all been eaten up by a loss in buying power for most people. It’s astonishing to observe how high prices are here, in Central NY, relative to what they were four years ago.

Roaming Charges: Losing It by Jeffrey St. Clair (CounterPunch)

“A third of an entire country–a big country, a country the size of Turkey and Venezuela–lies underwater, inundated by fierce floods from all directions.”

“Then came the rains. Rains like few other regions on earth have ever experienced. Rains that swelled the ancient Indus River over its banks and beyond its floodplains, creating a giant lake 100 kilometers wide almost overnight, which remains visible from space. A lake which can’t be drained, because there’s no place to pump the water to.

“The rains that drenched Sindh were 784% above the average for August. The rains that flooded Balochistan were 500% above normal. As much as 40 inches more than normal. Numbers so high they don’t really have a meaning.”

“In the midst of a mega-drought whose severity has not been seen in 1200 years, the vanishing waters of the West are being gobbled up by 31 coal plants in the region, which consume 156 million gallons a day to power the very plants whose emissions are driving the drought. That pales compared to nuclear plants, which can suck up Nuclear plants can suck up a billion gallons every day…”

“Germany’s 3-month experiment with super-cheap public transport reduced carbon dioxide emissions by 1.8 million tons–equivalent to powering about 350,000 homes for a year.”

And … it’s over. No money for it because Baerbock doesn’t see the point if the Ukrainian people don’t benefit directly. She’s insane.

On the other hand,

“China’s carbon emissions fell nearly 8 percent in the 2nd quarter compared with the same period last year, the sharpest decline in the past decade.”

Of course, no-one knows about this and people continue to point the finger of blame at China for the world’s ills, using it as justification for changing absolutely nothing about their own lifestyles, while their media sources soothe them with kind words.

Science & Nature

Things I Won’t Work With: Azidoazide Azides, More Or Less by Derek Lowe (Science)

“the X-ray crystal structure shows some rather strange bond distances, which indicate that there’s a lot of charge separation − the azides are somewhat positive, and the tetrazole ring somewhat negative, which is a further sign that the whole thing is trembling on the verge of not existing at all.”

“We’re talking high-nitrogen compounds here (a specialty of Klapötke’s group), and the question is not whether such things are going to be explosive hazards. (That’s been settled by their empirical formulas, which generally look like typographical errors). The question is whether you’re going to be able to get a long enough look at the material before it realizes its dream of turning into an expanding cloud of hot nitrogen gas.”

“[…] only tiny amounts of this stuff have ever been made, or ever will be. If this is its last appearance in the chemical literature, I won’t be surprised. There are no conceivable uses for it − well, other than blowing up Raman spectrometers, which is a small market − and the number of research groups who would even contemplate a resynthesis can probably be counted on one well-armored hand.”

Heatwave in China is the most severe ever recorded in the world by Michael Le Page (New Scientist)

“On 18 August, the temperature in Chongqing in Sichuan province reached 45°C (113°F), the highest ever recorded in China outside the desert-dominated region of Xinjiang. On 20 August, the temperature in the city didn’t fall below 34.9°C (94.8°F), the highest minimum temperature ever recorded in China in August. The maximum temperature was 43.7°C (110.7°F).”

“Together with the extreme heat, low rainfall in parts of China has led to rivers falling to low levels, with 66 drying up completely. In parts of the Yangtze, water levels are the lowest since records began in 1865. In a few places, local water supplies have run out and drinking water has had to be trucked in. On 19 August, China announced a national drought alert for the first time in nine years.”

Roaming Charges: Nuclear Midnight’s Children by Jeffrey St. Clair (CounterPunch)

“After cowering under the nuclear menace for nearly eight decades, after Trinity, Hiroshima and Nagasaki, after the big blasts at Novaya Zemlya, Amchitka and the Marshall Islands, after the radioactive disasters at Church Rock, Three Mile Island, Rocky Flats, Chernobyl, Hanford, and Fukushima? How can a demonic technology that has left only death, destruction, environmental ruin, cancer, sterility and genetic mutation as its legacy be treated so cavalierly by so many? We’ve reached the point where even Oliver Stone is pushing the virtues of nuclear power, despite its inextricable ties with the military-industrial complex he’s assailed most of his career.”

That is all true, in a way. It’s also hyperbolic, in a way. The histrionics apply jus as well to fossil fuels—even more so, in fact. So, what to do? Are we going to use less energy? Not likely.

“More crucially, in 1957 at speech before the American Chemical Society Teller, who later helped the Israelis develop their nuclear weapons program, became the first scientist to posit that the burning of fossil fuels would inevitably yield a climate-altering greenhouse effect, which would feature mega-storms, prolonged droughts and melting ice-caps. His solution? Replace the energy created by coal and gas-fired plants with a global network of nuclear power plants.

“Edward Teller’s deranged ideas of yesteryear have now been dusted off and remarketed by the Nuclear Greens, including James Lovelock, the originator of the Gaia Hypothesis, with no credit given to their heinous progenitor.”

Here, again, St. Clair’s hyperbole seems completely unfounded. Why is Teller “heinous”? Because he wants to use nuclear power? I suppose we should examine the degree to which extracting uranium causes climate change or environmental destruction (or regime-change, in the case of the Congo and Lumumba), but what’s the actual problem? The waste? Our current energy system generates a tremendous amount of waste, some of it quite radioactive in its own right. Is anyone proposing we stop using energy at this level entirely?

As I was flying on business for the first time in years, I was thinking that mankind is actually quite capable of putting together systems that remain safe over decades, prevailing against political erosion despite all odds. There are very, very few accidents with planes.

Art & Literature

Samaritan by Odie Henderson (RogerEbert.com)

“Until I’m proven wrong, I’m going to keep writing that the majority of these straight-to-streaming movies are not meant to be watched with any semblance of attention being paid. I’m a damn fool for trying to follow this movie, because there are no characters to care about and no follow throughs on the world building it attempts. It even has a twist that you should be able to predict during the opening credits, and the film doesn’t even do anything useful with that potentially interesting development. “Samaritan” proves, to paraphrase Tina Turner, that we don’t need another superhero.”

In Emily the Criminal, Crime Pays When Nothing Else Will by Eileen Jones (Jacobin)

“Once upon a time, decades back, it was possible to make the case that young underclass people could take out loans, get an education, and be reasonably sure of a career, or at least a stable job with a decent enough salary to pay back the loans and still afford food, housing, and transportation. It was never true for everyone in that situation, God only knows, and maybe not even a majority of those who tried. But enough young people could opt for that route and do okay that it was possible for people to believe, and politicians to campaign, on the idea that the system sort of worked.

“But now? Who can possibly be kidding themselves at this point except clueless rich people and raving mad ideologues?

“How many grad students doomed to joblessness and mountainous debt have to be roaming the plains, how many colleges and humanities programs have to shut down due to low enrollments, how many kids who bought the “STEM education guarantees good jobs” lie have to be on unemployment, how many young people fresh out of the university have to wind up working multiple jobs in the “gig economy” and yet find themselves still unable to afford the rent on a decent apartment, before we can all finally say, “We get it — it’s not working”?”

This is, essentially, the same argument we’ve been making for years, but this time, it’s about white people. That is, people that have historically been protected from the ravages of society are now being ravaged by it—just like the detritus we’d all agreed to sacrifice in years before. This movie has been made countless times, but it’s significant because it finally rings true of a middle-class, white woman rather than of a lower- or working-class, black man.

Philosophy & Sociology

When A Good Guy Refuses by Scott H. Greenfield (Simple Justice)

“As noted long ago, police rarely grasp that the reaction to their commands tends to differ wildly when the person with whom they’re interacting is a good guy. Bad dudes know why they’re being stopped. Good guys have no clue. Bad dudes know not to make a bad situation worse. Good guys are outraged that they are being treated so shabbily by police. And police tend to be oblivious to any of this.

“But isn’t the cluelessness a two-way street, where both the good guy and the cop are mistaken in their lack of appreciation of what the other is doing and why? Well sure, and the good guy would save himself a great deal of aggravation by cooperating rather than asserting what he believes to be his rights. As the old mantra goes, comply now, grieve later. On the other hand, the police are trained and equipped to handle various interactions with the public, whereas the public isn’t trained to appreciate the unknowns involved in their interaction with police. Let the trained public servant carry the weight of accommodation, just as he gets to carry the gun and shield. But it would still be wiser to comply than be on the television the next day, hopefully still alive.”

Technology

When 95% Isn’t Good Enough by Scott H. Greenfield (Simple Justice)

“[…] it’s the fact that this guy smugly tells us how to completely change our lives to suit his EV prioritization long before they’re ready for prime time with a great many tech and feasibility issues far from resolved which can be somewhat overcome if we just sacrifice our world so he can have his that makes his method of argumentation such a failure. All he wants is your money and fantasy logistics, and your life can be misery so he can feel like a savior. Does that do it for you? Do you feel a sudden itch to rush out and buy a Tesla?”

It’s a funny, “hot take” formulation, but I think it’s a bit unfair. The original author is excited about having possibly found a solution that might help address climate change. The author is wrong, I think, but that doesn’t make him smug. That he’s oblivious to how his solution will appear to the rest of the country that isn’t the primary target of his newspaper isn’t new and can’t really be interpreted as smug. He’s just ignorant.

Programming

Performance Improvements in .NET 7 by Stephen Toub (.NET Blog)

“I want everyone interested to walk away from this post with an upleveled understanding of how .NET is implemented, why various decisions were made, tradeoffs that were evaluated, techniques that were employed, algorithms that were considered, and valuable tools and approaches that were utilized to make .NET even faster than it was previously. I want developers to learn from our own learnings and find ways to apply this new-found knowledge to their own codebases, thereby further increasing the overall performance of code in the ecosystem. I want developers to take an extra beat, think about reaching for a profiler the next time they’re working on a gnarly problem, think about looking at the source for the component they’re using in order to better understand how to work with it, and think about revisiting previous assumptions and decisions to determine whether they’re still accurate and appropriate.”

“This is immeasurably helpful for performance analysis and tuning, even for questions as simple as “did my function get inlined” or “is this code I expected to be optimized away actually getting optimized away.” Throughout the rest of this post, I’ll include assembly snippets generated by one of these two mechanisms, in order to help exemplify concepts.”

Hardcore.

“Tiered compilation enables the JIT to have its proverbial cake and eat it, too. The idea is simple: allow the JIT to compile the same code multiple times. The first time, the JIT can use as a few optimizations as make sense (a handful of optimizations can actually make the JIT’s own throughput faster, so those still make sense to apply), producing fairly unoptimized assembly code but doing so really quickly. And when it does so, it can add some instrumentation into the assembly to track how often the methods are called. As it turns out, many functions used on a startup path are invoked once or maybe only a handful of times, and it would take more time to optimize them than it does to just execute them unoptimized. Then, when the method’s instrumentation triggers some threshold, for example a method having been executed 30 times, a work item gets queued to recompile that method, but this time with all the optimizations the JIT can throw at it. This is lovingly referred to as “tiering up.” Once that recompilation has completed, call sites to the method are patched with the address of the newly highly optimized assembly code, and future invocations will then take the fast path. So, we get faster startup and faster sustained throughput.”

This is very much like what WebKit has been doing with JavaScript Core for a long time. I first read about this technique in 2014; see Optimizing compilation and execution for dynamic languages. I recently read about a similar technique in Swift as well; see Links and Notes for March 19th, 2021 (search for “webkit”).

“Great, so now in .NET 7, we can largely avoid the tradeoffs between startup and throughput, as OSR enables tiered compilation to apply to all methods, even those that are long-running.”

“That all existed in .NET 6, so why are we talking about it now? Several things have improved. First, PGO now works with OSR, thanks to improvements like dotnet/runtime#61453. That’s a big deal, as it means hot long-running methods that do this kind of interface dispatch (which are fairly common) can get these kinds of devirtualization/inlining optimizations. Second, while PGO isn’t currently enabled by default, we’ve made it much easier to turn on.”

“PGO leads to a significant increase in the number of type checks, since call sites that specialize for a given type need to compare against that type. However, common subexpression elimination (CSE) hasn’t historically worked for such type handles (CSE is a compiler optimization where duplicate expressions are eliminated by computing the result once and then storing it for subsequent use rather than recomputing it each time). dotnet/runtime#70580 fixes this by enabling CSE for such constant handles.”

“In some cases, optimizations can do better when they’re exposed to more of the program, in other words when the tree they’re operating on is larger and contains more to be analyzed. However, various operations can break up these trees into smaller, individual ones, such as with temporary variables created as part of inlining, and in doing so can inhibit these operations. Something is needed in order to effectively stitch these trees back together, and that’s forward substitution. You can think of forward substitution almost like an inverse of CSE; rather than trying to find duplicate expressions and eliminate them by computing the value once and storing it into a temporary, forward substitution eliminates that temporary and effectively moves the expression tree into its use site.”

“A developer who wants or needs to go beyond what the high-level Vector<T> offers can choose to target one or more of these two types. Typically this would amount to a developer writing one code path based on Vector128<T>, as that has the broadest reach and achieves a significant amount of the gains from vectorization, and then if is motivated to do so can add a second path for Vector256<T> in order to potentially double throughput further on platforms that have 256-bit width vectors. Think of these types and methods as a platform-abstraction layer: you code to these methods, and then the JIT translates them into the most appropriate instructions for the underlying platform.”

“To many people, the word “performance” in the context of software is about throughput. How fast does something execute? How much data per second can it process? How many requests per second can it process? And so on. But there are many other facets to performance. How much memory does it consume? How fast does it start up and get to the point of doing something useful? How much space does it consume on disk? How long would it take to download? And then there are related concerns. In order to achieve these goals, what dependencies are required? What kinds of operations does it need to perform to achieve these goals, and are all of those operations permitted in the target environment? If any of this paragraph resonates with you, you are the target audience for the Native AOT support now shipping in .NET 7.”

“Native AOT is different. It’s an evolution of CoreRT, which itself was an evolution of .NET Native, and it’s entirely free of a JIT. The binary that results from publishing a build is a completely standalone executable in the target platform’s platform-specific file format (e.g. COFF on Windows, ELF on Linux, Mach-O on macOS) with no external dependencies other than ones standard to that platform (e.g. libc). And it’s entirely native: no IL in sight, no JIT, no nothing. All required code is compiled and/or linked in to the executable, including the same GC that’s used with standard .NET apps and services, and a minimal runtime that provides services around threading and the like.”

“Years ago, coreclr and mono had their own entire library stack built on top of them. Over time, as .NET was open sourced, portions of mono’s stack got replaced by shared components, bit by bit. Fast forward to today, all of the core .NET libraries above System.Private.CoreLib are the same regardless of which runtime is being employed. In fact, the source for CoreLib itself is almost entirely shared, with ~95% of the source files being compiled into the CoreLib that’s built for each runtime, and just a few percent of the source specialized for each (these statements means that the vast majority of the performance improvements discussed in the rest of this post apply equally whether running on mono and coreclr).”

“In .NET 6, there were almost 3000 [DllImport] uses throughout the core .NET libraries. As of my writing this, in .NET 7 there are… let me search… wait for it… 7 (I was hoping I could say 0, but there are just a few stragglers, mostly related to COM interop, still remaining). That’s not a transformation that happens over night.”

“[…] the pièce de résistance around primitive types in this release is “generic math,” which impacts almost every primitive type in .NET. There are significant improvements here, some which have been in the making for literally over a decade.”

“Speaking of call sites, one of the great things about having highly optimized IndexOf methods is using them in all the places that can benefit, removing the maintenance impact of open-coded replacements while also reaping the perf wins. dotnet/runtime#63913 used IndexOf inside of StringBuilder.Replace to speed up the search for the next character to be replaced:”

“ Finally on the IndexOf front, as noted, a lot of time and energy over the years has gone into optimizing these methods. In previous releases, some of that energy has been in the form of using hardware intrinsics directly, e.g. having an SSE2 code path and an AVX2 code path and an AdvSimd code path. Now that we have Vector128<T> and Vector256<T>, many such uses can be simplified (e.g. avoiding the duplication between an SSE2 implementation and an AdvSimd implementation) while still maintaining as good or even better performance and while automatically supporting vectorization on other platforms with their own intrinsics, like WebAssembly.”

“Arguably the biggest improvement around UTF8 in .NET 7 is the new C# 11 support for UTF8 literals. […] UTF8 literals enables the compiler to perform the UTF8 encoding into bytes at compile-time. Rather than writing a normal string, e.g. “hello”, a developer simply appends the new u8 suffix onto the string literal, e.g. “hello”u8. At that point, this is no longer a string. Rather, the natural type of this expression is a ReadOnlySpan<byte>.”

“One of the great things about improving things low in the stack is they have a multiplicative effect; they not only help improve the performance of user code that directly relies on the improved functionality, they can also help improve the performance of other code in the core libraries, which then further helps dependent apps and services.”

“These facilities are more advanced, but they’re used liberally throughout higher-performance code bases, and many of the optimizations in .NET in recent years are possible in large part due to these ref-related capabilities.”

“Enter scoped. The new C# keyword does exactly what we just wished for: put it on a ref or ref struct parameter, and the compiler both will guarantee (short of using unsafe code) that the method can’t stash away the argument and will then enable the caller to write code that relies on that guarantee.”

“This implementation is based on the notion of regular expression derivatives, a concept that’s been around for decades (the term was originally coined in a paper by Janusz Brzozowski in the 1960s) and which has been significantly advanced for this implementation. Regex derivatives form the basis for how the automata (think “graph”) used to process input are constructed. The idea at its core is fairly simple: take a regex and process a single character… what is the new regex you get to describe what remains after processing that one character? That’s the derivative.”

“For every regex construct (concatenations, alternations, loops, etc.) the engine knows how to derive the next regex based on the character being evaluated. This application is done lazily, so we have an initial starting state (the original pattern), and then when we evaluate the next character in the input, it looks to see whether there’s already a derivative available for that transition: if there is, it follows it, and if there isn’t, it dynamically/lazily derives the next node in the graph. At its core, that’s how it works.”

“The main benefit of the non-backtracking implementation is predictability: because of the linear processing guarantee, once you’ve constructed the regex, you don’t need to worry about malicious inputs causing worst-case behavior in the processing of your potentially susceptible expressions. This doesn’t mean RegexOptions.NonBacktracking is always the fastest; in fact, it’s frequently not. In exchange for reduced best-case performance, it provides the best worst-case performance, and for some kinds of applications, that’s a really worthwhile and valuable tradeoff.”

The INSANE performance boost of LINQ in .NET 7 by Nick Chapsas (YouTube)

This video illustrates very nicely how well-planned the changes to .NET in versions 5, 6, and 7 are. People wonder why .NET extends the language to include static methods on interfaces, increased usage of ref, support for vectors, SIMD, and so on. These changes culminate in something like .NET being able to improve performance of LINQ with numeric elements by 48x, all while using managed code and providing the same opportunities for performance increases to anyone using .NET.

Notes on the SQLite DuckDB paper by Simon Willison

“A key change is made to the join processing, which is to probe the Bloom filters before carrying out the rest of the join. Applying the Bloom filters early in the join pipeline dramatically reduces the number of tuples that flow through the join pipeline, and thus improves performance.”

“Although SQLite’s OLAP performance could be further improved in future work, there are several constraints that potential modifications to SQLite must satisfy.

“First, modifications should cause no significant performance regression across the broad range of workloads served by SQLite. Second, the benefit of an optimization must be weighed against its impact on the size of the source code and the compiled library. Finally, modifications should not break SQLite’s backwards compatibility with previous versions and cross-compatibility with different machine architectures.”

Optimizing Corax: Optimizing tree operations by Oren Eini

“The idea is that we do the deletion of terms in two stages. First, we gather all the ids we need to delete for all the terms from all the entries that are being deleted. Then we sort those values, and then we invoke a batch delete method.

“Unlike the individual RemoveValue() calls, we can now take advantage of the structure of the tree. In the case of wanting to remove [15,20], we can scan the tree (25,14, 19, 16, 15) to get to the first item that we remove. Then we proceed using the tree’s own structure. So deleting 20 means comparing (16, 19, 22, 20). In this case, we saved one operation, which isn’t that meaningful. But B+Tree’s most beautiful property is that they are dense. In this case, we are removing values from posting lists, which may contains millions of entries, and it isn’t uncommon to be able to pack thousands of entries per page. That means that the savings are huge.”