4 days Ago


Mixing your own SQL into Quino queries: part 1 of 2

Published by marco on in Programming

The Quino ORM[1] manages all CrUD—Create, Update, Delete—operations for your application. This basic behavior is generally more than enough for standard user interfaces. When a user works with a single object in a window and saves it, there really isn’t that much to optimize.

Modeled methods

A more complex editing process may include several objects at once and perhaps trigger events that create additional auditing objects. Even in these cases, there are still only a handful of save operations to execute. To keep the architecture clean, an application is encouraged to model these higher-level operations with methods in the metadata (modeled methods).

The advantage to using modeled methods is that they can be executed in an application server as well as locally in the client. When an application uses a remote application server rather than a direct connection to a database, modeled methods are executed in the service layer and therefore have much less latency to the database.

When Quino’s query language isn’t enough

If an application needs even more optimization, then it may be necessary to write custom SQL—or even to use stored procedures to move the query into the database. Mixing SQL with an ORM can be a tricky business. It’s even more of a challenge with an ORM like that in Quino, which generates the database schema and shields the user from tables, fields and SQL syntax almost entirely.

What are the potential pitfalls when using custom query text (e.g. SQL) with Quino?

  • Schema element names: An application needs to figure out the names of database objects like table and columns. It would be best not to hard-code them so that when the model changes, the custom code will be automatically updated.
    • If the query is in a stored procedure, then the database may ensure that the code is updated or at least checked when the schema changes.[2]
    • If the query is in application code, then care can be taken to keep that query in-sync with the model
  • Materialization: In particular, the selected fields in a projection must match the expectations of the ORM exactly so that it can materialize the objects properly. We’ll see how to ensure this in examples below.

There are two approaches to executing custom code:

  • ADO: Get a reference to the underlying ADO infrastructure to execute queries directly without using Quino at all. With this approach, Quino can still help an application retrieve properly configured connections and commands.
  • CustomCommandText: An application commonly adds restrictions and sorts to the IQuery object using expressions, but can also add text directly to enhance or replace sections of the generated query.

All of the examples below are taken directly from the Quino test suite. Some variables—like DefaultDatabase—are provided by the Quino base testing classes but their purpose, types and implementation should be relatively obvious.

Using ADO directly

You can use the AdoDataConnectionTools to get the underlying ADO connection for a given Session so that any commands you execute are guaranteed to be executed in the same transactions as are already active on that session. If you use these tools, your ADO code will also automatically use the same connection parameters as the rest of your application without having to use hard-coded connection strings.

The first example shows a test from the Quino framework that shows how easy it is to combine results returned from another method into a standard Quino query.

public virtual void TestExecuteAdoDirectly()
  var ids = GetIds().ToList();
  var people = Session.GetList<Person>();

  people.Query.Where(Person.MetaProperties.Id, ExpressionOperator.In, ids);

  Assert.That(people.Count, Is.EqualTo(9));

The ADO-access code is hidden inside the call to GetIds(), the implementation for which is shown below. Your application can get the connection for a session as described above and then create commands using the same helper class. If you call CreateCommand() directly on the ADO connection, you’ll have a problem when running inside a transaction on SQL Server. The SQL Server ADO implementation requires that you assign the active transaction object to each command. Quino takes care of this bookkeeping for you if you use the helper method.

private IEnumerable<int> GetIds()
  using (var helper = AdoDataConnectionTools.GetAdoConnection(Session, "Name"))
    using (var command = helper.CreateCommand())
      command.AdoCommand.CommandText = 
        @"SELECT id FROM punchclock__person WHERE lastname = 'Rogers'";

      using (var reader = command.AdoCommand.ExecuteReader())
        while (reader.Read())
          yield return reader.GetInt32(0);

There are a few drawbacks to this approach:

  • Your application will make two queries instead of one.
  • The hard-coded SQL will break if you make model changes that affect those tables and fields.
  • The ADO approach only works if the application has a direct connection to the database. An application that uses ADO will not be able to switch to an application-server driver without modification.

In the second part, we will improve on this approach by using the CustomCommandText property of a Quino query. This will allow us to use only a single query. We will also improve maintainability by reducing the amount of code that isn’t checked by the compiler (e.g. the SQL text above).

Stay tuned for part 2, coming soon!

Many thanks to Urs for proofreading and suggestions on overall structure.

[1] This article uses features of Quino that will only become available in version 1.12. Almost all of the examples will also work in earlier versions but the AdoDataConnectionTools is not available until 1.12. The functionality of this class can, however, be back-ported if necessary.
[2] More likely, though, is that the Quino schema migration will be prevented from applying updates if there are custom stored procedures that use tables and columns that need to be changed.

Mixing your own SQL into Quino queries: part 1 of 2


Capsule Movie Reviews Vol.2014.3

Published by marco on in Art, Film & Literature
Gravity (2013)

I can only say what I thought of this movie based on the way that I saw it: in HD on a conventional screen at home. I can imagine that the experience was very different in 3D and on a giant screen with a kick-ass sound system. The only downside I can think of is that if the sound-leveling was the same in the theater, it would have been an ear-blistering experience. If you set the volume high enough to hear the occasional radio whispers, many other parts of the movie nearly blew you out of your chair—or caused the neighbors to call to yell at you that their kids can’t sleep.

Gravity stars George Clooney and Sandra Bullock in an unlikely in-space scenario. The inconsistencies abound in a movie that purports to make an effort to get things right. It’s ludicrous because space is big. Neil DeGrasse Tyson did a masterful job of listing plot holes on his Twitter account. Just to sum up the ones I noticed:

  • Once you’re locked to something in space, you will not “drift away”. Once the tether stopped Bullock and Clooney, there was no force causing them to continue to drift from the space station. None. The station was not rotating so centripetal force did not come into play.
  • It sure was convenient that the space in which they found themselves was so inordinately populated with other stuff: ISS, the shuttle and the Chinese station were all within a couple of hundred miles of each other and in sight lines.
  • An utterly untrained and self-admittedly terrible pilot uses landing thrusters to hit a target in space and match speed with it? With minor adjustments made by a fire extinguisher? Sure, why not.
  • Why doesn’t the fire extinguisher come soaring down on her head during one of the many, sudden momentum changes when she’s in the capsule?
  • Why is nothing tethered? And why is there literally no instinct to tether anything on her part? Especially when she’s so absolutely amazing at navigating the tight tunnels of the station at high speed without so much as nicking a knee or elbow?
  • Why in God’s name was a medical doctor doing a spacewalk? This is not in any way explained. Armageddon did a better job of explaining why the utterly unqualified were suited up.
  • I did not notice this one, but I love DeGrasse Tyson for noticing it: “Nearly all satellites orbit Earth west to east yet all satellite debris portrayed orbited east to west.”

It was an action movie, but I didn’t really get into Bullock as an action actress. I could not have cared less about her character because there was almost zero character development. Having her character tell me that she lost a child does not count as developing her character. A movie has to have a character that you root for and I honestly could not have cared less if she lived or died. I was actually pleasantly surprised to think that the movie would end with her turning off the oxygen in the Russian capsule (which Clooney kept calling the “Soyez). This would have been a delightfully an realistic existentialist ending. See Magic Mike below for how to end a movie.

Alas, she pulled herself up by her bootstraps, performed some utterly unbelievable miracles, forgave herself and learned to walk again. Yay for happy endings that confirm the ability of humans to overcome anything. Meh. I’m not leaving off a recommendation because the science was wrong, I’m leaving it off because I didn’t like the schmaltzy plot and I don’t have a giant 3D screen at home.

Real Steel (2011)
A film about the robot-boxing world of the future. Hugh Jackman plays a down-on-his-luck robot-boxer manager who was a strong, skilled and hard-headed boxer. In 2027, men no longer box; robots do. Jackman’s failures as a robot-boxer driver are only exceeded by his failures at gambling. Long story short, this is a Disney movie about a little robot fighting against a giant robot owned by a steely-eyed Russian lady—it’s like Rocky IV all over again. Hugh Jackman is good, as usual: he’s charming even though he’s an utterly useless idiot for much of the film, seemingly intent on self-destruction for reasons that are unclear. Evangeline Lilly plays a plucky boxing-manager’s daughter—the same who managed Jackman’s former career. After Jackman inexplicably and almost deliberately wastefully burns through a couple of expensive robots, his long-lost son joins him for the summer and discovers a long-buried, early-model robot at a junkyard. The little robot turns out to be plucky and trainable and hard-headed and ready to bite off a lot more than it can chew. Yadda yadda yadda. It was entertaining and well-made—and watching mechanical robots pound each other in the brainpan without any perceivable form of defense is much preferred over watching the same with humans. The boxing scenes are well-done and quite exciting. recommended.
Star Trek: Insurrection (1998)
The crew of Star Trek: The Next Generation end up in the middle of a dispute between a new race whose longevity is waning and the simple residents of a planet whose radiation imparts rejuvenatory effects on its few inhabitants. Thanks to corruption and misguided notions of charity, Star Fleet stands solidly behind the dying, but invading, race and feels that the few hundred inhabitants of the planet have no right to sit on a resource that has the potential to prolong millions if not billions of lives. They want to oust the inhabitants and let the other race in to research and develop the energy that is their Fountain of Youth. Picard and crew quite rightly see the inherent injustice of this and intervene on behalf of the residents, whom they’ve in the meantime befriended. Cue heroics and Star Trek-style badassery in which our favorite crew triumphs and simultaneously proves that Star Fleet and the “ancient” race never truly had a moral leg to stand on. Slow-paced as you would expect—and with battle scenes that are laughable by today’s sci-fi standards—but also rife with the expected philosophical and political discussions, into which parallels to modern-day issues and situations can easily be read, but which would in all likelihood be denied by the makers of the film, albeit with perhaps a sly smile and a wink. Recommended for fans of the genre.
Lolita (1997)

This film is lovingly narrated by Jeremy Irons, who also has the lead role. The film shows his character moving in with Lolita and her mother (played by Melanie Griffith) and slowing being pulled into Lolita’s orbit. Or rather, he is immediately smitten and she slowly pretends to seduce him. She is aware of her power over him, but toys with it casually, not even letting it take precedence over being a teenager. It’s lovingly filmed with a focus on the nubile young Lolita from the eye of the narrator. And Jeremy Irons is a wonderful narrator.

Lolita is young and obnoxious but the bloom only slowly comes off the rose for Humbert, as long as she’s banging him. The interview at the college—which turns out to be a prep school for débutantes—was quite funny and featured a zeugma, “Here at Beardsley Prep, we’re less concerned with Medieval dates than weekend ones.” Slowly, Lolita comes to be in total control, twisting him around by his predilections and his guilt about them. She irritates him deliberately and is deliberately obnoxious, knowing that her sexual favors allow her everything. When Humbert says, “You’re very young and I know it’s hard to imagine that people will try to take advantage of you,” it’s quite hard to keep a straight face.

The movie is a PSA for “do not date too young or too crazy and definitely not both”. He is her slave; he is in love. Whereas he does not try to break her at all, she definitely breaks him. Being an ephebophile is his only societal flaw; he is otherwise not capable of the brutality—psychological and otherwise—required to keep her under control. Spoiler alert: he can’t do so and she ends up running away with another “lover of nymphets”, with whom she comes to an unhappy end three years later. In the end, he has broken her and she’s only concerned with money and thinks nothing of performing for it. He has broken her because she is the only thing he ever loved and his touch twisted her into something base and stupid and unlovely. And still he loves her.

The power that Lolita acquired in her youth rewarded her, but it was a cheap substitute for what perhaps could have been. It is difficult to judge the potential of such a young creature: was her precocity indicative of an intelligence that would find other channels of expression later? Or was it the pinnacle of her cleverness, manipulating men bedazzled by her nubility? Nabakov argues that we will never know—because Humbert imposed himself into the situation, collapsing the quantum waveform, and dooming her to a life of dimmed prospects, where her imagination cannot reach farther than to think of which sugar daddy she will grace with her wiles—but not whether life could be lived without one.

Rien à Declarer (2011)

I saw this movie before, on a plane, in French with English subtitles. This time I watched the first part in French with German subtitles, but my viewing partner doesn’t understand much (any) French and the dialogue comes so quickly that she was reading the whole time. It’s still good in German but it loses something, I think. It’s an absolutely fantastic French comedy, an exemplar of the genre. My favorite joke:

“Q: Why does the Frenchman laugh 3 times when he hears a joke about Belgians?

“A: Once for when he hears it, once for when someone explains it to him and once again when he finally understands it.”

See the previous review for a short synopsis. Highly recommended.

Nighthawks (1981)
Instead of Carl Weathers, Sylvester Stallone teams up with Billy Dee Williams as New York City cops hunting terrorist Rutger Hauer. Stallone looks awesome and young in his beard, leather jacket and 70s-era shooter glasses. And Hauer, even so early[1], plays the perfect Euro-terrorist. When he’s finally cornered with his hostages on the Roosevelt Island gondola, one of the ladies says to him, “Please leave us alone; we’ve done nothing,” he haughtily responds with his characteristic smirk, “You must be very proud.” Wicked burn. Minutes later, he wastes her in front of Stallone to set an example—definitely not trying for the PG-rating. Although the film is far less gory than it would have been were it shot today, it has a more brutal sensibility than is common for action films these days. Stallone and Hauer spend a lot of time squinting menacingly into each other’s eyes, but it kind of works. Also, the pacing is more deliberate, the shots are far longer and there is no shaky cam. I’m kind of a sucker for this kind of action film, I guess. Recommended.
Blue Jasmine (2013)

This film is the 2013 installment of the long-running streak of yearly films by Woody Allen. Though there are flashes of Allen in Jasmine’s dialogues, this is a very thematically and artistically different film than many of his others. If you hadn’t told me it was a Woody Allen movie, I may never have guessed (whereas To Rome with Love, for example, was unmistakably Allen).

It stars Cate Blanchett as a former socialite-on-top-of-the-world whose husband’s crookedness she’d steadfastly ignored, all the while pretending that all she had was somehow deserved of someone of her talents, intellect and sensibilities. She moves in with her sister—both girls were adopted by the same parents, but from different families—and tries to put her life back together. In this, she does much better than expected, getting a menial job and persevering for more than a day. She continued to inhale pills (provenance and type unknown) as well as nearly limitless amounts of Stoli vodka.

In the end, she is unrepentant and bitter, convinced that the world is at fault for her downfall. Her husband was a criminal and a philanderer and an all-around immoral person. When she turns him in to the FBI out of spite, her son hates the mom rather than the dad, whose criminality is at the root of all of the family’s wealth but also its problems. The film is much, much, much darker than other Woody Allen movies, with no one really coming out on top in the end. Recommended.

Straw Dogs (2011)
The remake of the 1970s classic that starred Dustin Hoffman[2], but this time starring Kate Bosworth, James Marsden and Alexander Skarsgård. I watched it because of Skarsgård, who was so good in Generation Kill but he didn’t have a lot to work with in the role of the leader of a group of not-always-vaguely rapey misanthropes. The story is of Bosworth moving back to her hometown with her author-husband. Dominic Purcell stars as a mentally handicapped man who’s put upon by the town, especially the extremely alcoholic former coach, played well by James Woods. The town has a distinctly menacing and anti-intellectual and highly church-y vibe, to which the husband is all-but-oblivious. He was never destined to mix in well with the people of town but the coming disaster is hastened by his superiority. His wife doesn’t do nearly enough to fight of the attentions of the local XY-carriers, choosing instead to at-times revel in their attention. The film does more than play with the idea of a woman getting’ what’s comin’ to her. This will, of course, not end well. The actors are decent, but the plot is a bit too manipulative and undernourished for my taste. Hopefully, the original is better. Saw it in German. Not recommended.
Election (1999)
This is an absolute classic about a deceptively sociopathic and egotistical high-school student named Tracy Flick, played by Reese Witherspoon. Matthew Broderick plays a sad-sack teacher at her school named Jim McAllister. McAllister and Flick narrate much of the film along with Paul and Tammy Metzler, who run against Flick for the student-council presidency, all for their own reasons. McAllister’s life circles the drain with a pathetic attempt at an affair with his wife’s best friend (who also happens to be the wife of his own best friend, with whom he used to teach but who was thrown out of both the school and his own home when he was caught having an affair with Tracy). Witherspoon is penetratingly obnoxious and terrifying. Broderick is great as a loser who was happy with what he had, teaching ethics and morals and having none of either. Who will end up winning? Well, the one who wants it most—and understands the least of ethics. Will McAllister give up the last of his tenets in order to stop her? Will it be worth it? There are no good guys in the movie, but you’ll still feel that the wrong people won.
Black Snake Moan (2006)

Samuel L. Jackson plays Lazarus, a God-fearing full-time vegetable farmer and part-time blues guitarist whose wife has left him, Christina Ricci plays Rae, a caricature of the town slut whose reputation from high school follows, defines and leads her well into her twenties. She is psychologically unstable, at best, with a thirst for men—to be more precise, a very specific part of men—that is depicted as medically uncontrollable. Not that she doesn’t try to self-medicate: no pill or drink goes unconsumed in her presence. Justin Timberlake plays her boyfriend, who knows of her past and predilections but thinks that they are in the past and under control. No sooner does he set foot on a bus, headed forArmy boot camp, than Rae hops into bed with a former lover or three. It is made clear that these actions are out of her control and are to be considered fallout from the psychological trauma of having been regularly abused by her father (or step-father?) as a teenager.

Long story short, Lazarus takes up the Herculean task of trying to cure her of her smutty desires. It’s hard to tell how serious the movie takes itself—it seems to think it’s something more than just an excuse to show Ricci’s pretty little self be used and abused in various stages of dishabille. If the dishabille doesn’t sell you, then perhaps Jackson’s musical number near the middle of the film will make it worth your while. It’s quite haunting and well worth the ride. Timberlake returns at some point with his own bushel of psychological problems and mixes things up a bit. Saw it in German. Hard to recommend but it wasn’t as terrible as it may sound.

The Hobbit: The Desolation of Smaug (2013)

This is part two of a three-part homage to a three-hundred–page book. The last time I read it, I would definitely have called it a “children’s” book when compared with the sweeping mythos and breadth of the Fellowship of the Ring. The story of The Hobbit is of a decidedly non-adventurous member of a non-adventurous and nondescript race of miniature beings who live under hills, play in the sun and snack all day long. They are human-shaped rabbits, in other words.

The cast of the first film returns, joined by Evangeline Lilly as a pretty elf—not much of a stretch there—and Orlando Bloom as Legolas. Many arrows are loosed and much elvish fighting skill is on display as orc after orc after orc is dispatched by these two in their attempt to help the dwarves on their quest. There is a bit of confusion on that point, but the upshot is that that is what they end up doing. Gandalf is also back, especially good in a scene that reveals the Necromancer for what he truly is. Benedict Cumberbatch is almost unrecognizable as the voice of Smaug, a gigantic dragon who sits on a gigantic hoard and who is possessed of a gigantic ego.

The storyline of the book is enhanced by an escapade that traps the arrogant Smaug, if only temporarily. The smith-works of the dwarves below Erebor—the Lonely Mountain—are beautiful and of an imposing scale that beggars belief. Truly impressive visuals but the story, as with the first installment, is a bit threadbare in places, failing to cover up the fact that it’s been stretched over three films. Recommended for fans of the books or fans of big-budget action films, of which this is a more than passable exemplar.

Magic Mike (2012)
Channing Tatum stars as the eponymous hero, a self-styled entrepreneur who runs a car- and truck-detailing business as well as a roofing/contracting firm and playing the lead role in a male revue, stripping at night. His real passion is building one-of-a-kind furniture from found objects, but he barely finds time for that. He does find time for a decent amount of harmless partying and fun, usually with two or more companions at once, one of whom is an adventurous Olivia Munn. He meets and takes pity on a sad-sack named Adam, taking him under his wing and introducing him to the world of male revue. Adam’s sister—played by the sloe-eyed and quite pretty newcomer Cody Horn—is of a more sober bent. She hardly cracks a smile once throughout the movie although she is not immune to Tatum’s infectious humor and inestimable charm (like when he sees that she’s clearly irritated by Dallas’s drivel about his lifestyle and how people should raise kids, he follows her and asks if she wants him to get her Dallas’s number because he’s starting a life-coaching business and he can tell that she’d be interested). Dallas is played by Matthew McConaughey, in a role he was born to play. He comes full circle with the beginning of his career, often repeating “all right, all right, all right”—which he first uttered as David Wooderson in Richard Linklater’s Dazed and Confused. The role of Dallas is anyone’s best guess at what Wooderson would look like as a grown-up. Stephen Soderbergh did a great job and treated the material quite seriously. It was a funny, well-made movie with an absolutely perfect ending. While McConaughey is good, it’s Tatum who holds the film together. Recommended.

[1] In IMDb, it looks like this was Hauer’s first American movie—everything else before that was Dutch.
[2] I have that one in my list of thrillers to watch, but this one came on TV instead.

2 weeks Ago


Criminal Justice in the U.S.

Published by marco on in Public Policy & Politics

The article Theater of Justice by Molly Crabapple (VICE) is an article by an artist who also occasionally does courtroom sketches.

She tells of Cecily McMillan, who was beaten into a seizure by police offers and who two years later stands trial for assaulting a police officer, facing seven years in prison. The officer’s record of having beaten other suspects was deemed inadmissable.

Or there is the other recent case of a black woman who tried to stand her ground, as others have successfully done. She fired a warning shot into the air, killing no one, not even wounding anyone. These were the actions of “Marissa Alexander, a PhD and mom who [wanted to] stop her husband from beating her,” That’s not a good reason, is it? Are we even sure that her husband isn’t allowed to beat her in that state? And that’s not nearly as good a reason as the guy had who killed a boy in the back-seat of his SUV for playing music too loud. Not guilty! But Marissa’s going to go away for a long time for her transgression.

“[She] was offered three years as a plea deal for aggravated assault with a deadly weapon. She refused, knowing herself innocent. The judge sentenced her to 20 years. Now, she’s appealing. If she loses, the prosecutor wants to lock her up for 60.

“This is a “trial tax” you pay if you annoy the courts by insisting you are innocent.”

Who does this uppity woman think she is? Does she think she’s white? Rich? A citizen? A human being? Do not speak of justice in a system that produces hypocrisy on this scale. And the system does everything it can to make being poor or disadvantaged increase chances of prosecution dramatically.

We still have jury trials in the States; this means that non-professional, easily misled and nigh-constantly deluded undereducated head-cases are deciding your fate. Those are your peers. They can’t string two logical sentences together; what are the odds that they can wend their way through the facts of the case to come to a just conclusion? Nearly zero. What are the odds that they will decide your fate based on how you dress or act rather than evidence? Nearly certain.

“The poor, the brown, the trans – to juries, they’re guilty unless proven otherwise. Innocence is the absence of guilt. It is near impossible to prove a negative. […] If you’re too poor to afford bail, you arrive in court in chains. If you have no family to bring you a suit, you wear your prison jumpsuit.”

And people who haven’t yet been convicted are made to suffer beforehand. The unconvicted are left to stew behind bars because they can’t afford ridiculous bail. The homeless guy who was recently broiled to death in Riker’s Island because he couldn’t pay $2500 for bail on his charge of loitering was in jail for this reason. He was luckier than the homeless guy in the SouthWest U.S. who was executed by police officers for the same crime. Sure, those are anecdotes, but that doesn’t change the fact that “[…] the average defendant [is] a person of colour charged with a drug crime.” And more and more prisoners are going away for longer sentences; more and more people are taking years before they get their trial.

“Because the entire system would implode if everyone demanded a trial, prosecutors push plea bargains like restaurants hawking early bird specials. But instead of money, they’re haggling over life. If you’re too poor for a lawyer or have already spent months in jail because you can’t make bail, plea bargains can be irresistible. They account for 95 percent of felony convictions.”

As mentioned above, 95% just take the plea bargain in order to get some form of a life back. This is a life with a felony record and drastically reduced chances of making anything of yourself in a society that hates its ex-cons.

“Most trials resemble not grand dramas but factory farms. The raw material is a person. The product is a prisoner. Trials are deliberately dull. They move glacially, on state time rather than human time. If you hire your own lawyers – a necessity to have a chance of winning – you’ll blow through your life savings. As the cop cliché goes, “You can beat the rap, but you can’t beat the ride.””

If you don’t plea out, you lose your life savings and may still go to jail. If you do plea out, you lose all chance of ever making decent money again. You see? In America, you still have the freedom to choose.


Big data ignores lessons learned

Published by marco on in Science & Nature

The article Big data: are we making a big mistake? (Financial Times) bursts the bubble of the wide-eyed, overconfident and underinformed techies who think that their giant piles of data will fix everything. The article contains many interesting examples, some of which are touched on in the conclusion, cited below:

“Uncanny accuracy is easy to overrate if we simply ignore false positives, […] The claim that causation has been “knocked off its pedestal” is fine if we are making predictions in a stable environment but not if the world is changing (as with Flu Trends) or if we ourselves hope to change it. The promise that “N = All”, and therefore that sampling bias does not matter, is simply not true in most cases that count. As for the idea that “with enough data, the numbers speak for themselves” – that seems hopelessly naive in data sets where spurious patterns vastly outnumber genuine discoveries.

““Big data” has arrived, but big insights have not. The challenge now is to solve new problems and gain new answers – without making the same old statistical mistakes on a grander scale than ever.”

The upshot is that you think your data is “big” but it is most likely not big enough. Whereas sampling bias is diminished compared to smaller datasets, the claims made based on the big data are correspondingly bigger, eradicating the increased confidence. Selectively filtering results to focus on the expected result is a pitfall not necessarily of bad statistics, but of bad scientists/engineers as well.

The article is a good read for those who can get behind the FT paywall or who haven’t used up all of their “free views” for the year.


Russophobia: the Lunatics are at the Helm

Published by marco on in Public Policy & Politics

I am so tired of hearing of scintillatingly smart people who can’t seem to ever say anything that is even tangentially well-informed. We knew that the Bush administration was a booby-hatch full of cantankerous old farts who hadn’t been right about anything or even had an original thought since before it became illegal to beat your wife and black people, not necessarily in that order. That doesn’t excuse them in any way at all, but they didn’t even really have a veneer of intelligentsia to them.

And now we have a new administration full of supposed young guns, ready to take on the 21st century. Not only is the Obama administration a moral and ethical failure throughout the whole spectrum but this supposedly technically savvy and hyper-informed and educated pile of Rhodes and Constitutional scholars can’t even seem to grasp the basics of human interaction beyond that which you would find in any neighborhood sandbox. They are a bunch of kindergartners who don’t know enough to shut up and let the grownups handle things.

They are so seduced by what they continue to cling to as U.S. hegemony and power that they coast along, not even bothering to make up a story that even halfway jibes with reality. We end up with policy that is not only criminally stupid and dishonest and offensive to anyone with half a brain and half an education who’s read half a history book or even half-paid attention to current events, but it will drag the hubris-laden vessel of the U.S.—and likely a lot of the rest of world with it—to very murky depths before they’re through.

There is no need for diplomacy when you can just stamp your foot and scream and make up all of your own history and facts and information and have the sails of your stupidity belled out by the hot air blown in vast and steaming amounts by a slavish corporate media intent on selling lies that will buoy their bottom lines for the next quarter. And to hell with the rest of it. I got mine, jack.

Obama stood in front of an assembly in Bruges and dribbled out the most spectacularly uninformed and nuance-free drivel you could imagine, all but starting World War III with seemingly nary a thought that others on the planet may not have the global domination of the U.S. as a core guiding principle.

His advisors and helpmeets are no better, with the bevy of women he’s appointed to relatively high office doing absolutely nothing to provide evidence to support the theory that if women ruled the world, we’d have less violence. To the contrary, Samantha “boom-boom” Power nearly fell on her Russian counterpart with savage blows before she got herself under control.

There is a severe problem when the lunatics run the asylum. It is even worse when they forget where reality ends and their own propaganda begins.

John Kerry is another such laughable idiot—a buffoon if there ever was one—who doesn’t waste a single second trying to convince anyone of anything—instead averring with such a self-assured knowledge that he is telling the gospel truth that it is hard to believe that he could even tell the difference anymore.

All of these fools steamroll right past the ironies, hypocrisies and shocking double-standards that abound in all of their argumentation, assuming that we are so much stupider than they. It may well be that most people will simply go along with what these scholars say because they feel that such smart people could not possibly be deluded on so grandiose a scale.

Those believers are sadly mistaken. This is not the first time that the lunatics are at the helm. It is arguable that it has always been thus. That does not in any way make it more palatable.

Perhaps these leading lights of the U.S. State Department are justified in their cynicism. But I simply want it noted that people who are purportedly intelligent but spend all of their time grubbing for power and saying the most mind-bogglingly inane and provably false things should not be heeded. Their original intelligence does not matter because they are not employing it. At all. That they know better but are cynically manipulating the world for their personal gain is not even cold comfort. Far better if we could just get them to leave us alone.

And don’t think that this vapidity is confined to the administration and its hangers-on. The Republican love of Putin is a pure knee-jerk reaction against the Obama administration, nothing more. They are not handling this any better nor are they exhibiting any greater level of intelligence than does a plant when it leans toward the sun.

Putin is not a grand guy but he is rational, he can be eminently reasonable and he seems to have the best interests of his country at heart. He has also exhibited absolutely no designs on taking over other countries. Crimea was not a takeover. The situation was forced on him by any logical reading of the events. That the U.S. paints this in any other way is so disingenuous and cynical that it’s nearly warping the space-time continuum. Be that as it may, all of the things listed above could be used as levers by diplomats worth their name in order to come to an agreement with Russia. But an agreement is not what is sought.

What is sought instead by the U.S. is utter domination and degradation and capitulation on the part of any other country that has a whit of power remaining. Countries like Russia and Chine will not bow so easily—nor do they have any reason to, when one looks at the facts and the reality rather than the mythic world of American exceptionalism. The Idiocracy that is America is shooting itself in the foot time and again, thinking that it can create history and bend reality and facts to its whims. The rest of the world tires of these spoiled-child antics and just wishes the U.S. would go away.

Kerry leaps to support the rebels in Venezuela—but it is a rebellion of the rich against the poor, throwing a temper tantrum because they want their country back in the hands of a few oligarchic families. But the OAS was recently asked by the U.S. to vote for intervention and overthrow of Maduro in Venezuela. The U.S. received a resounding NO, with only Canada and Panama voting in favor. Everyone else in the OAS told the U.S. (and Canada) to go f&%*k themselves. And rightly so. Because only an immoral jackass would support a “revolution” promulgated by the upper class against the poor. When put that way, it makes sense that Kerry was for it.

When Honduras was overthrown by an extreme right-wing party, the Obama administration fell all over itself to validate the revolution and welcome the new rulers into the international community—constitutionality be damned. It was the same in Venezuela during the 24-hour coup of Chavez. It was the same in Libya. It was the same in Kosovo. It was the same in Ukraine, where the U.S. once again cheerfully supports a fascist government—as long as they accept U.S. (NATO) missiles on their border with Russia. Shooting distance to Iran is also not a bad consolation prize.

The lunatics are running the asylum and we’re all along for the ride. The media doesn’t care. They love the simple story told by the administration. Or is it that the administration loves the simple story told by the media? It’s so hard to tell who’s the dog and who’s the tail and who’s doing the wagging.

The danger that the rest of the world sees is that these kindergarten-level administrators and diplomats and officials have a ton of power and weapons and influence still to burn. And they have a craven and willing media at their back, which is eager to sell a ton of advertising for their Cold War Redux coverage.

We can only hope that Putin remains reasonable and picks his battles and doesn’t get drawn into the idiocy. He has been reassuringly stable and grounded so far. It is an utter shame that we have to hope for this cypher of a man to prevent the gaggle of idiots at the helms of other countries (Angela Merkel has toned down the rhetoric considerably of late, in fairness) from plunging us all into a nuclear winter.

If you’ve got so much time on your hands to poke the Russian bear, why don’t you expend some energy on doing something about climate change? You know, instead of just pretending the problem doesn’t exist because it’s politically difficult in an election year. Every other year in the States is an election year. It’s an excuse to never have to engage your giant brain and actually do something. And your giant brain has deluded you into thinking that no one else could possibly be as smart as you and therefore you should get all the toys and cupcakes. This is laziness and intellectual dishonesty at its core. Talk about entitlement.

And pro tip: Just because someone speaks English with an accent doesn’t make them stupid. Nor does it make them smart—I’m looking at you, Henry Kissinger. You are the original proponent of the take-all-the-toys-stick-your-fingers-in-your-ears-and-coast-on-your-reputation-for-smarts strategy. You and Bob Mcnamara, who admitted his oopsie only after millions of Vietnamese had died. Good timing. Lots of other people smarter than you knew that what you were doing was wrong before you even started doing it. Nobody listened to them. Just like no one is listening now, instead leaving the reins in the hands of the utter children currently in charge of the U.S.

In conclusion, stop listening to people who you’ve been told are smart but who never seem to say anything smart or reasonable or well-informed. They are going to lead us down the primrose path of destruction and make life a lot worse for everyone.


I didn’t include any citations or references in the main text although there are many good ones and they all pretty much say the same thing: the mainstream media/U.S. version of events is pure mendacity. This example from the article Obama’s Sleepwalk Toward War by Paul Craig Roberts (CounterPunch) is as good an example as any.

“The extraordinary transparent lie that Russia sent an army into Ukraine and annexed Crimea is now accepted as fact everywhere in the West, even among critics of US policy toward Russia.

“Obama, whose government overthrew the democratically elected government in Ukraine and appointed a stooge government that has threatened the Russian provinces of Ukraine, falsely accuses Putin of “invading and annexing” Crimea.”

Paul Craig Roberts continues to report well and honestly. Tariq Ali also described the hypocrisy as breathtaking, Pepe Escobar is always informative. Diane Johnstone also wrote eloquently and well on the topic and drew parallels to NATO and the U.S. in Yugoslavia. Israel Shamir continues to provide good background. And not all members of the EU are equally deluded: Gregor Gysi gave a good and impassioned speech to the German Bundestag. James Howard Kunstler notes that “In [his] lifetime, there has never been a more pointless and unnecessary international crisis than the current rumble over Ukraine, and it’s pretty much all our doing” before returning to his reporting on the ongoing global financial collapse, suggesting that this might be more important (I chose climate change above).

Even the text of Vladimir Putin’s speech to the Kremlin after accepting Crimea back into the fold, is very good and historically informative reading (and is not without humor; see emphasis).

“First, we had to help create conditions so that the residents of Crimea for the first time in history were able to peacefully express their free will regarding their own future. However, what do we hear from our colleagues in Western Europe and North America? They say we are violating norms of international law. Firstly, it’s a good thing that they at least remember that there exists such a thing as international law – better late than never. (Emphasis added)”