Last night, a Reddit user by the name of frigis9 posted a series of six images that feature detailed graphical upgrades to classic MS-DOS computer games such as Commander Keen 6 and The Secret of Monkey Island. The most interesting part is how they did it: by using an image synthesis technique called "img2img" (image to image), which takes an input image, applies a written text prompt, and generates a similar output image as a result. It's a feature of the Stable Diffusion image synthesis model released last week.
[...] Art quality in image synthesis currently requires much trial and error with prompts and cherry-picking to achieve the kinds of results frigis9 posted—likely hours of work. But with some incremental advances in image synthesis techniques and GPU power, we could imagine an emulator upgrading vintage game graphics in real time within a few years.
Related Stories
US firm Getty Images on Tuesday threatened to sue a tech company it accuses of illegally copying millions of photos for use in an artificial intelligence (AI) art tool:
Getty, which distributes stock images and news photos including those of AFP, accused Stability AI of profiting from its pictures and those of its partners. Stability AI runs a tool called Stable Diffusion that allows users to generate mash-up images from a few words of text, but the firm uses material it scrapes from the web often without permission.
The question of copyright is still in dispute, with creators and artists arguing that the tools infringe their intellectual property and AI firms claiming they are protected under "fair use" rules.
Tools like Stable Diffusion and Dall-E 2 exploded in popularity last year, quickly becoming a global sensation with absurd images in the style of famous artists flooding social media.
Related:
- Adobe Stock Begins Selling AI-Generated Artwork
- Pixel Art Comes to Life: Fan Upgrades Classic MS-DOS Games With AI
- NVIDIA Research's GauGAN AI Art Demo Responds to Words
- Alien Dreams: An Emerging Art Scene
Things are moving at lightning speed in AI Land. On Friday, a software developer named Georgi Gerganov created a tool called "llama.cpp" that can run Meta's new GPT-3-class AI large language model, LLaMA, locally on a Mac laptop. Soon thereafter, people worked out how to run LLaMA on Windows as well. Then someone showed it running on a Pixel 6 phone, and next came a Raspberry Pi (albeit running very slowly).
If this keeps up, we may be looking at a pocket-sized ChatGPT competitor before we know it.
[...]
For example, here's a list of notable LLaMA-related events based on a timeline Willison laid out in a Hacker News comment:
- February 24, 2023: Meta AI announces LLaMA.
- March 2, 2023: Someone leaks the LLaMA models via BitTorrent.
- March 10, 2023: Georgi Gerganov creates llama.cpp, which can run on an M1 Mac.
- March 11, 2023: Artem Andreenko runs LLaMA 7B (slowly) on a Raspberry Pi 4, 4GB RAM, 10 sec/token.
- March 12, 2023: LLaMA 7B running on NPX, a node.js execution tool.
- March 13, 2023: Someone gets llama.cpp running on a Pixel 6 phone, also very slowly.
- March 13, 2023, 2023: Stanford releases Alpaca 7B, an instruction-tuned version of LLaMA 7B that "behaves similarly to OpenAI's "text-davinci-003" but runs on much less powerful hardware.
Related:
DuckDuckGo's New Wikipedia Summary Bot: "We Fully Expect It to Make Mistakes"
Robots Let ChatGPT Touch the Real World Thanks to Microsoft (Article has a bunch of other SoylentNews related links as well.)
Netflix Stirs Fears by Using AI-Assisted Background Art in Short Anime Film
Paper: Stable Diffusion "Memorizes" Some Images, Sparking Privacy Concerns
The EU's AI Act Could Have a Chilling Effect on Open Source Efforts, Experts Warn
Pixel Art Comes to Life: Fan Upgrades Classic MS-DOS Games With AI
The AI software Stable Diffusion has a remarkable ability to turn text into images. When I asked the software to draw "Mickey Mouse in front of a McDonald's sign," for example, it generated the picture you see above.
Stable Diffusion can do this because it was trained on hundreds of millions of example images harvested from across the web. Some of these images were in the public domain or had been published under permissive licenses such as Creative Commons. Many others were not—and the world's artists and photographers aren't happy about it.
In January, three visual artists filed a class-action copyright lawsuit against Stability AI, the startup that created Stable Diffusion. In February, the image-licensing giant Getty filed a lawsuit of its own.
[...]
The plaintiffs in the class-action lawsuit describe Stable Diffusion as a "complex collage tool" that contains "compressed copies" of its training images. If this were true, the case would be a slam dunk for the plaintiffs.But experts say it's not true. Erik Wallace, a computer scientist at the University of California, Berkeley, told me in a phone interview that the lawsuit had "technical inaccuracies" and was "stretching the truth a lot." Wallace pointed out that Stable Diffusion is only a few gigabytes in size—far too small to contain compressed copies of all or even very many of its training images.
Related:
Ethical AI art generation? Adobe Firefly may be the answer. (20230324)
Paper: Stable Diffusion "Memorizes" Some Images, Sparking Privacy Concerns (20230206)
Getty Images Targets AI Firm For 'Copying' Photos (20230117)
Pixel Art Comes to Life: Fan Upgrades Classic MS-DOS Games With AI (20220904)
A Startup Wants to Democratize the Tech Behind DALL-E 2, Consequences be Damned (20220817)
(Score: 5, Insightful) by looorg on Monday September 05 2022, @04:30PM (13 children)
Looking at the image I'm not sure I like this at all. I don't really think there was much of an upgrade in that regard in going from the "pixelart" to the other more "life like" art. The composition and content changed. They are not really remakes but more re-imagining. Not what I would consider to be upgrades.
Also why was the poses changed by the AI? Duke lost his white t-shirt he had the ripped shirt or whatnot. The color probably comes from the very limited palette of MS DOS cause I don't think Duke would have been a fan of pink and purple shirts.
The monkey island image where she is now a lot closer to Guy Brush and is tilting her head on his shoulder. She is no longer standing behind him but in front of him tilting backwards, hair changed, no earrings, also Guy Brush got a new shirt, non of that old pirate look anymore.
Kingsquest guy got less hair.
The woman/alien in the last image apparently got larger breasts (typical AI) and lost the metallic headband ...
Is this was companies will use now to do "remakes" of all their classic games to try and resell them again? No more abandonware, just filter the graphics over and over again to always keep the old stuff in circulation. Or as an emulator feature. If you want to play old games via emulator do you really want them to look new and modern? Is that really an upgrade? I don't think so.
(Score: 0) by Anonymous Coward on Monday September 05 2022, @04:46PM (4 children)
I'm sure we're going to be seeing Stable Diffusion EVERYWHERE in the short term, at least until the novelty wears off and it settles out with people just using it for nefarious purposes.
(Score: 4, Informative) by looorg on Monday September 05 2022, @05:11PM (2 children)
I hope not. But there is I guess a segment for that. People that will like it. It's new and cool or whatever.
That said I find that it breaks a lot of the esthetics of pixel art. How you worked with the limitations of the system. In the case of the image in the article a lot of it is ms-dos palette issues that had to be worked around. There was only 8 colours to work with, or if you like 16 colours as there is a dark and a light version of each color. But then not all implementations or graphics cards decided to show them the exact same so there was slight differences there to. So choices had to be made of contrast and such.
As noted I think that Duke's pink shirt is a good example; you have the light pink or purple vs the dark pink and some black but the altered new version is just that. More colours, it takes away some of the pixel aspect of the creation. The choices that had to be made.
The other thing, I guess the main thing, that I do find weird in the examples is how many things change between the old and new version. Does the AI have a problem identifying details or did the human in charge feel like changing things, and if so why?
Duke3d
white t-shirt gone
armband (right) gone
belt is gone or became white
head shrunk in proportion to the body
new gun that looks nothing like the old
the hands that are supposed to hold it look all weird and doesn't fit the rest of the body, they certainly are not holding the gun.
Monkey island
girl moves forward
leans head on shoulders
more hair, different hair cut
earrings gone
scarf got bigger
guy brush lost his pirate shirt
kingsquest
head tilted differently
looks like he has less hair
shoulders was broader before and didn't drop
feather changed colour
hat got pointier
StarControllaliengirl
Bigger breasts, i guess this is part of her filling out in general; no longer have that extreme hourglass figure.
headband gone
armbands gone on the upper arm
not as much clown makeup
These are just some of the things I notice at first (or second glance) I'm sure I could find more issues between them if I spent more time looking at it. That said I guess it is supposed to be more of a "wow neat stuff that can be done with AI" and not pointing out the flaws or weird things with the result.
(Score: 5, Insightful) by JoeMerchant on Monday September 05 2022, @09:19PM
I think most of the changes can be summed up as: features have been steered towards the stock photos that the AI trained on.
🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Tuesday September 06 2022, @07:44AM
girl? That's Governor Marley to you!
(Score: 0) by Anonymous Coward on Monday September 05 2022, @06:24PM
The novelty will not wear off. It's already at a point of being pretty good with fascinating results, usually needing some cleanup. People can make careers off of mastering it and secondary tools right now. If these algorithms get even better in the coming years, they will spread everywhere.
Developers are crippling algorithms and datasets to stop pr0n, and more control is needed to use them effectively for nefarious purposes. That will be a slow burn but build up as mainstream uses become popular.
(Score: 5, Insightful) by JoeMerchant on Monday September 05 2022, @05:17PM (2 children)
I've only taken the 30,000 ft flyover of the tech, but from there it looks like most of it is developing on "text based" image generation, which is impressive on its own, but it means that the details in the images do get overlooked in the more coarse text descriptions of the images given. Major things like head-bands, etc. get lost - I assume because they're not in the text and not in the AI's experience for generation. It's making a lot of generic stuff based on the (presumably millions, even billions of) images that it has "trained on" - and has limited morphing capability from source pixel-art.
The art world is tweaked over someone entering an AI generated image in a contest and the judges awarding it first prize... well, good for them, AI is just a fancy tool for creation, not everything called art has to be drawn freehand with burnt sticks and ochre powder on rough hewn stone.
🌻🌻 [google.com]
(Score: 2) by looorg on Monday September 05 2022, @06:00PM (1 child)
I saw that one in the news to. I found it weird. It was a digital art contest. So I don't see what the issue was really. I didn't look that much into it. After all people still chose that one as the best in the competition. How it was created I guess is secondary unless they had rules specifically telling how the images or painting should or could have been created. That an AI did it or if he sat drawing pixels or splines or whatever method he used as long as it then was digital it should all be good?
In this case I think there is a slight difference and I'm not a fan of this AI enhancement of old computer games.
(Score: 2) by JoeMerchant on Monday September 05 2022, @06:20PM
Well, the game is hardly enhanced yet... as a proof of concept you can see where it might lead in 5-10 years, and I'd be all for an AI enhanced and slightly reworked gameplay re-releases of the old Atari 400/800 games like Star Raiders, Seven Cities of Gold, M.U.L.E. etc. For extra cool factor, put a slider control somewhere that you can morph between the original 8-bit graphics up through 8K HD, and another for original sound up through full-synth orchestral.
🌻🌻 [google.com]
(Score: 2) by Ingar on Monday September 05 2022, @06:10PM (2 children)
To quote the author, frigis9
It would be a rather tedious job to convert all the sprites of an old game.
(Score: 2) by JoeMerchant on Monday September 05 2022, @08:41PM
The potential here is that once you tune the AI to be able to do a passable transform of one sprite, that same tune should do a passable job transforming a bunch of other sprite perspectives...
🌻🌻 [google.com]
(Score: 2) by looorg on Monday September 05 2022, @09:14PM
Considering that most sprites in old games are not very large, nor are there all that many of them I don't think it will be to hard or take to long. After all most of the machines doesn't have the ability to display all to many sprites at the same time, without a lot of tricks and shenanigans. Nor do they have memory or storage to store them. So you better hand optimize like a God if you want to squeeze in in there.
Another aspect I guess is that part of the transformation might be that you can't change to much with it without also changing the code that uses them. They are a bit limited in that regard. They also tend to be precision pixels involved so that parts fit with each other to form a whole so if the AI overmind start to change to what looks good that could be very very weird and wonky.
But basically you kind of have to reprogram a lot of it then if you are just going to change or scale the graphics then you might as well just rewrite the whole thing so why bother with scaling up graphics and such when you can just make new once? So unless this is just going to be some kind of emulation overlay that swaps things in near real time this seems a bit odd. After all if you are just doing a remake you are doing that and then this seems a bit over the top. But perhaps that is the next step, once the graphics are down the next AI will fix the sound beeps (or turn them into a modern quality soundtrack) and the AI after that will translate the code and adapt to something new. Not holding my breath really. I'll keep enjoying my 8- and 16-bit classics the way it was intended.
(Score: 4, Informative) by stormreaver on Tuesday September 06 2022, @12:33AM (1 child)
Don't worry too much about this, as it appears to be largely snake-oil clickbait. I read the description of the program, and it's rather unimpressive. It scans a large database of tagged images, and composites the results together. That's why it's so important to get the description right: the description has to match the image tags, or you'll get poor results. You don't even have to feed it the source images, as they aren't used anywhere in the process.
It's not upscaling, image synthesis (in any meaningful way), or anything even remotely close to an AI drawing an image based on a human description. It's just image searching. There have to be a existing images that match your description, or nothing will happen. It's just a database search followed by compositing.
(Score: 3, Interesting) by acid andy on Tuesday September 06 2022, @12:32PM
DALL-E2 [wikipedia.org] seems to work better than stable diffusion [medium.com]. DALL-E / DALL-E2 use a natural language parser so unlike the tech you're describing that's not only matching words directly to image tags. I assume it still makes its images by breaking up ones from a training set (I can't find a full description of that part of the tech), although I suspect even the human brain does this to a degree to fill in details in vision.
I certainly get the feeling a lot of artists are going to be out of a job in a few years, at least where generic filler artwork is needed.
As for TFA, I don't like what they're doing to the game artwork either. They should've at the very least generated cartoon characters. I've been a lot more impressed with the use of AI texture upscaling to add detail to old games without compromising the original artwork.
If a cat has kittens, does a rat have rittens, a bat bittens and a mat mittens?
(Score: 3, Interesting) by JoeMerchant on Monday September 05 2022, @08:43PM (5 children)
A morph of the space invaders sprites with HR Geiger's art library should make a cool result....
🌻🌻 [google.com]
(Score: 2) by looorg on Monday September 05 2022, @09:08PM (1 child)
aka 8-bit nightmare fuel.
(Score: 2) by JoeMerchant on Monday September 05 2022, @09:11PM
Space Invaders was trying hard to be nightmare fuel, but 8 bit sprites can only go so far...
🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Tuesday September 06 2022, @01:43AM
I'd like to see what it would do with the guys from my old Intellivision games [wikipedia.org].
(Score: 2) by hendrikboom on Saturday September 10 2022, @02:40PM (1 child)
You mean Giger?
(Score: 2) by JoeMerchant on Saturday September 10 2022, @05:48PM
Most likely, I have a mental block around those names for some reason. This stuff, anyway:
https://www.wikiart.org/en/h-r-giger/no-250-li-i-detail [wikiart.org]
🌻🌻 [google.com]
(Score: 2) by SomeGuy on Tuesday September 06 2022, @12:20PM (2 children)
Yeish, Keen's hand looks all wrong. The entire thing looks creepy and wrong.
Welcome to the future where all movies, music, games, product art, articles, web pages, and so on is generated by an "AI". Everything you see or hear will be this surreal slurry of creepy garbage. (And every other word will be "fuck" because that is what the AI algorithms determined idiots want to hear). In the off chance you find any piece of originality, anyone with a brain would realize it was someone else's work pulled from a large database and distorted. But since nobody has a brain anymore, everyone will love it.
(Score: 2) by acid andy on Tuesday September 06 2022, @12:35PM (1 child)
But how would that be any different to what's being produced today? Hollywood, I'm looking at you...
If a cat has kittens, does a rat have rittens, a bat bittens and a mat mittens?
(Score: 2) by Freeman on Tuesday September 06 2022, @03:47PM
To a certain extent that's true, but copyright tends to ensure a nominal percentage of original content. I.E. Blatantly ripping things off, doesn't fly.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 3, Interesting) by Freeman on Tuesday September 06 2022, @03:55PM
I dislike all of the AI generated images. Sure, I grew up with computers and low detail graphics aren't an immediate revulsion to me. Unlike, with what seems fairly standard nowadays. Still, there are exceptions to that. Terraria is pretty big on the pixel art style, but is a hugely successful independent game. "Pretty" graphics, don't make a game. They don't necessarily make "good art" either. You can generally tell, if a game's art was well done or was just another task on a checklist.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"