The GardenA blog by yours truly

Posts tagged as “machine learning”

Dear Tate Modern, i have a very cool idea for an art exhibit that i think you would really like. P.S. I am not a crackpot.

A man looking at two versions of Caspar David Friedrich’s “Wanderer Above the Sea of Fog” — the original, and a noticeably less good AI version

There’s been a lot of kerfuffle in the art world as of late about the ethics and capabilities of AI art (previously), and as Britain’s leading institution for contemporary art, you seem like just the right people to bring it to the public. My proposal is simple, but effective — let man and machine compete on equal footing.

Eight or so talented human artists will be given a prompt to work from. At the same time, the same prompt will be given to a state-of-the-art machine-learning algorithm, like Midjourney or Stable Diffusion. In the gallery, the two works — one made by metal, one made by flesh — will be hung side by side, and the audience will not be told which is which.

Next to each diptych will sit two bins where visitors can dispense plastic tokens (like the ones they have at Asda) to vote on which painting is their favourite. At the end of the exhibition’s run (or perhaps updating live; your call), the votes will be tallied up, and we’ll finally find out whether us or our creations are the better artists.

If you really wanted to provoke, you could ask the humans to provide you with a list of every painting they’ve ever seen, every photo they’ve ever taken, every film they’ve ever watched, and every song they’ve ever heard. Then you put that big list up on the wall, tell the visitors that Advanced Biological Neural Learning Algorithms have taken quote-unquote “inspiration” from all of these copyrighted works, and put to vote whether you should contact the rightsholders and ask them to sue. It would be only fair.

Chære and regards, Xanthe. P.S. — I am not a crackpot.

What does AI make of the Gods?

I recently bought 1000 images’ worth of credits on DreamStudio — a machine-learningα-powered art generator — on a whim and, after the requisite “Boris Johnson taking a bath of baked beans” joke entries, i thought it would be an interesting test to get it to generate some images for my shrines (on- and offline).

Four images of god. From left to right: a stone carving resembling Zeus, a tapestry with a four-armed figure draped in green robes, fire reaching into the sky with a nebula in the centre, and a stone carving by the seaside resembling the face of Jesus of Nazareth.
Just typing in “God” brought a fascinating cavalcade of interpretations — some clearly Hellenic, some Christian, some taking more inspiration from the dharmic faiths, and the occasional completely abstract depiction.

My motivations were twofold: first, due to copyright constraints, all of the icons adorning these shrines were either old baroque paintings or freely-licenced photos of even older marble statues, which didn’t necessarily represent my mental image of the Gods’ appearances — a topic which, of course, will vary massively from artist to artist and culture to culture. Second, i thought it would be a fascinating experiment to see how this machine learning algorithm, which has taken in hundreds upon thousands (perhaps millions; i’ve not checked) of images, views the Gods in its latent space. Just as it has a prototypical idea of a “dog” and a “cat”, surely it also has one for “God” and “Dionysos”.

Hestia in a toga leaning on a pillar near a hearth, with a halo-like glow emanating from Her head

As is tradition, we begin this article with Hestia (although Her portrait was actually the final one to be generated). On the broad strokes, my computer collaborator knocked it out of the park — but a closer look reveals some glaring imperfections in the face and hands, a theme which we’ll be seeing a lot of (and which i sometimes managed to harness to my advantage).

Apollon — a lithe, youthful, Caucasian man with waving blonde locks of hair — reaches up towards the heavens in front of a hilly valley

I should note that i’m not just feeding it theonyms with no added context: the programme works best if you help it along to your goal with a heaping of adjectives and descriptors, say, to tell it that this is indeed meant to be an artwork (“4K ultra HD”, “trending on ArtStation”), the details of the pose and background you want (“blonde hair”, “raising His hand to the sky”), or the style and artists you want it to take from (“baroque painting by Thomas Cole”, a prominent painter of beautiful, well-lit landscapes). If you calibrate it just right, it can make some genuinely beautiful stuff, like the above picture of Apollon (which i did, admittedly, have to manually touch up to get rid of a prominent Habsburg chin).

Gæa, framed by Celtic knotwork, as a green-skinned, bare-chested woman with leaves for hair

It may be an immensely powerful tool, but DreamStudio can also be rather prudish.β It blurs out any images it thinks might contain the utterly offensive sight of the genitalia with which we are all born, which can be a real problem if the relevant pictures it’s learnt from are all Greek and Roman statues — not exactly works known for their nether modesty. The detection software isn’t perfect, though, and sometimes, like in this portrait of Gæa, it lets a few slip past (perhaps because of the greenish tone with which i instructed itγ to portray Her skin).

Hermes as a swarthy young man wearing a three-feathered crown, fleshy wings emanating from His sides

The algorithm sometimes has issues with more complex prompts, for it is just a machine, and doesn’t actually understand that “ball on top of a red box” means that the ball indeed should be on top of the box, as opposed to by its side, beneath it, or fused together in a horrific amalgam. These troubles somewhat manifested themselves in the above portrait of Hermes; the winged cap He is traditionally depicted with has transformed itself into both a crown and a hulking pair of soaring, fleshy wings emanating from His shoulders, and the recognisable caduceus has been reduced to a bamboo stick by His side.

Perhaps it’s just the style i instructed it to paint in — sixteenth-century European paintings aren’t renowned for their diversity — but DreamStudio also has some real trouble with darker skin tones. You can cry “dark skin, dark bronze skin, dark skin, dark skin, dark skin, black” all you want, but the only thing that can consistently get it to generate anything a shade below the average Spaniard is “African American”, which tends to bring along a heap of other associated physical changes besides just skin tone. (I have to say, i don’t particularly envision Hermes as the eponymous Futurama character in my head.)

Hermaphroditus as an androgynous, twinkish fellow with three arms, laying down by a pond in a bed of leaves

It also has quite some trouble with arms and legs. Originally, i thought of its odd morphings and multiplications as a bug to be stamped out, but i came to see them as a feature, representing the manifold, varied aspects of the Gods, their omnipresence, transcending the limits of human form. (This is also why the Hindus do it, if i recall correctly.)

I would have rather the above portrait of Hermaphroditos been slightly more, ah, gynomorphic around the chest, so to speak, but i’d been trying to get a decent pose for what felt like an hour and i didn’t feel like fighting the blur anymore.

Dionysos as a three-legged and -armed bearded man, overweight but muscled, covered in flowers

So then — it’s a bit off in places, and lacks the leopard-skin toga i would have liked, and lord knows what the objects He’s holding are meant to be, and it turned out the computer really, really, struggled with the basic concept of a faun or satyr’s legs, but we end this post with DreamStudio’s interpretation of an icon of Dionysos, framed by some beautiful landscape.

Navigating through the neural net’s knowledge and limitations has been a fascinating, illuminating exercise, which has left no doubt in my mind that “AI art” is, indeed, just that: art. It seems to me much more comparable to something like photography than painting: rather than doing the hard work by hirself with brush strokes and pencil lines, the artist guides hir computer collaborator through latent space, pressing “click” when sie finds something appealing. One can only hope the Muses would approve.