A machine learning algorithm has created tiny (64×64 pixels) 32-frame videos based on text descriptions:
The researchers trained the algorithm on 10 types of scenes, including "playing golf on grass," and "kitesurfing on the sea," which it then roughly reproduced. Picture grainy VHS footage. Nevertheless, a simple classification algorithm correctly guessed the intended action among six choices about half the time. (Sailing and kitesurfing were often mistaken for each other.) What's more, the network could also generate videos for nonsensical actions, such as "sailing on snow," and "playing golf at swimming pool," the team reported this month at a meeting of the Association for the Advancement of Artificial Intelligence in New Orleans, Louisiana.
[...] Currently, the videos are only 32 frames long—lasting about 1 second—and the size of a U.S. postage stamp, 64 by 64 pixels. Anything larger reduces accuracy, says Yitong Li, a computer scientist at Duke University in Durham, North Carolina, and the paper's first author. Because people often appear as distorted figures, a next step, he says, is using human skeletal models to improve movement.
Tuytelaars also sees applications beyond Hollywood. Video generation could lead to better compression if a movie can be stored as nothing but a brief description. It could also generate training data for other machine learning algorithms. For example, realistic video clips might help autonomous cars prepare for dangerous situations they would not frequently encounter. And programs that deeply understand the visual world could spin off useful applications in everything from refereeing to surveillance. They could help a self-driving car predict where a motorbike will go, for example, or train a household robot to open a fridge, Pirsiavash says.
An AI-generated Hollywood blockbuster may still be beyond the horizon, but in the meantime, we finally know what "kitesurfing on grass" looks like.
Related Stories
There has been some controversy over Deepfakes, a process of substituting faces in video. Almost immediately, it was used for pornography. While celebrities were generally unamused, porn stars were alarmed by the further commodification of their rôle. The algorithm is widely available and several web sites removed objectionable examples. You know something is controversial when porn sites remove it. Reddit was central for Deepfakes/FakeApp tech support and took drastic action to remove discussion after it started to become synonymous with fictitious revenge porn and other variants of anti-social practices.
I found a good description of the deepfakes algorithm. It runs via a standard neural network library but requires considerable processing power on specific GPUs. I will describe the video input (with face to be removed) as the source and the face to be replaced as the target. The neural network is trained with the target face only. The source is distorted and the neural network is trained to approximate reference images of the target. When the neural network is given the source, it has been trained to "undistort" the source to target.
[Continues...]
(Score: 3, Insightful) by Anonymous Coward on Saturday February 24 2018, @09:06PM
And the Records Department, after all, was itself only a single branch of the Ministry of Truth, whose primary job was not to reconstruct the past but to supply the citizens of Oceania with newspapers, films, textbooks, telescreen programmes, plays, novels — with every conceivable kind of information, instruction, or entertainment, from a statue to a slogan, from a lyric poem to a biological treatise, and from a child’s spelling-book to a Newspeak dictionary. And the Ministry had not only to supply the multifarious needs of the party, but also to repeat the whole operation at a lower level for the benefit of the proletariat. There was a whole chain of separate departments dealing with proletarian literature, music, drama, and entertainment generally. Here were produced rubbishy newspapers containing almost nothing except sport, crime and astrology, sensational five-cent novelettes, films oozing with sex, and sentimental songs which were composed entirely by mechanical means on a special kind of kaleidoscope known as a versificator. There was even a whole sub-section — Pornosec, it was called in Newspeak — engaged in producing the lowest kind of pornography, which was sent out in sealed packets and which no Party member, other than those who worked on it, was permitted to look at.
(Score: 2) by looorg on Saturday February 24 2018, @10:07PM (3 children)
"An AI-generated Hollywood blockbuster may still be beyond the horizon, but in the meantime, we finally know what "kitesurfing on grass" looks like."
Well .... A lot of hollyweird scripts and movies are really shit so it might not be noticeable that it is written by an AI. I'm sure some critic will proclaim it to be really deep and profound. Considering that some movies appear to have stories that fit on a napkin and / or be described in flowcharts or is just one scene of violence after another it might not take that much to get one going. If academics can write shit papers that gets published an AI script might not be far fetched -- they could even write an article and paper about it.
(Score: 2) by maxwell demon on Sunday February 25 2018, @07:55AM (1 child)
But Hollywood blockbusters typically need more than 64x64 pixels.
The Tao of math: The numbers you can count are not the real numbers.
(Score: 2) by c0lo on Sunday February 25 2018, @10:07PM
Unfortunately. What a waste!
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by VLM on Sunday February 25 2018, @02:57PM
Its generally believed that pr0n leads technology, but there is a side path where about a decade ago the technology to turn free text into rap music famously launched the career of moonman on 4chan /pol/ (Um, don't google that at work NSFW).
Likewise I would suspect the most creative users of this technology will be 4chan /pol/ asking the video oracle for stuff like "gimmie birth of a nation but islamic themed" or "how about 1970s blaxploitation recast with soyboy hipster theme". At college we had to watch "Uptown Saturday Night", and I don't remember why, possibly as lame of a reason as the professor thought it was funny, that was a tolerable comedy in itself, but imagine that recast with the funniest stereotypes of modern hipsters. Actually, that might sell pretty well as a formulaic teen movie.
The other thing you'll see a heck of a lot of is "Modify the Zapruder film such that the secret service agent clearly shoots JFK instead of blurrily shoots him on the original" or whatever. Honestly though I think there was a Cuban hit team on the grassy knoll and to avoid global thermonuclear war in response, clear heads prevailed and the whole thing was covered up. It fits a lot of peculiarities in Cuban/USA (and Cuban/USSR) relations since 1960 or so. I admit it suffers a little from the rationalization argument that it makes far too much sense and matches evidence too well such that its a little unrealistic, I know from personal experience that real world military operations (admittedly not hard core stuff like political leader assassinations) are always a little screwed up in the details and nothing ever goes precisely according to plan.
(Score: 2) by SomeGuy on Saturday February 24 2018, @10:15PM (1 child)
We could do something similar with music too. Perhaps we could call it MIDI?
Been there done that.
DOOME1M1.MID. (You can hear it in your head now can't you? :P )
(Score: 3, Interesting) by Ethanol-fueled on Saturday February 24 2018, @10:51PM
MIDI was a godsend for getting files of songs and opening them in the score editor to learn how to play them. Considering that the official books were over 20 bucks an album (more for larger or more rare ones, my then-girlfriend paid 40 for an official Tori Amos book which wasn't even accurate) the extra effort to download the midis was worth it, and anyway good luck finding the sheet music for Tarkus or Trilogy back then.
But the reason why this is on-topic is because similar approaches have been used to generate music, though with comparatively much better results than what is described in the article. If anybody can create their own "deep fakes" now, just imagine the capabilities of government agencies. Some of them probably have lots of very good blackmail material on very important people, and muh deep fake calls truth into question and gives falsehoods credibility.
(Score: 4, Touché) by requerdanos on Sunday February 25 2018, @02:03AM (2 children)
A pixel is not a measure of width, depth, length, or any other "size" measurement. It indicates that however large a given image--the size of a hydrogen atom, or the size of the known universe--that image has been divided into elements of usually equal size.
Thus, the videos are either the size of a hydrogen atom or smaller, the size of the known universe or larger, or somewhere in between. Thanks.
For a better comparison, they are about the size of an elephant: link to 64x64 picture of elephant [freworld.info]. Photo license CC BY-SA 4.0, attribution "Andrew Shiva"
They are about the size of a school bus: link to 64x64 picture of school bus [freworld.info]. Photo license Creative Commons Attribution-Share Alike 3.0 Unported, attribution "Marcin Szala"
They are about the size of the Eiffel Tower: link to 64x64 picture of Eiffel Tower [freworld.info]. Photo license Creative Commons Attribution-Share Alike 3.0 Unported, attribution "Julie Anne Workman".
They are about the size of--wait for it--The Library of Congress: link to 64x64 picture of LoC as seen from the North [freworld.info]. Photo license Creative Commons Attribution-Share Alike 3.0 Unported, attribution, some Wikipedia user named Hugaholic (I didn't ask).
Note: Because the films are 32 frames, you must look at any of the above photos about 32 times to best get the best feel for the "size".
(Score: 2) by VLM on Sunday February 25 2018, @02:35PM (1 child)
Its also worth pointing out no one would collect postage stamps if they were that cruddy looking. 200 to well over 600 dpi equivalent is more like it. I remember my father collected stamps for awhile and when I was a kid he pointed out a stamp tangentially showing the instrument panel of an aircraft and using a magnifying glass you could read the needles on the visible instruments, and I was impressed. Not sure if thats normal or unusual. Looking over his collection together they all looked pretty high res in general. It seems unusual to issue stamps that are intentionally ugly, also. In like the ancient boomer era, stamps were kind of a nation-level 4chan meme factory so if you wondered what meme the nation of Brunei found interesting, my dad probably had one of their stamps. Ebay was very hot for stamp trading around the turn of the century, he sold them all.
An even worse would be a comparison to coin currency. If you have really good vision or a camera macro lens or a triplet, the oldest wheat pennies have the artist's initials along the bottom border, the lincoln memorial obverse side (up till '09 or so) if you looked real close you could see lincoln's statue inside his memorial. IIRC you can count 34 individual kernels of wheat on the obverse of a wheat penny. In the old days before smart phones everywhere, if you were bored enough you could entertain yourself for awhile with a pocket of change and a magnifying glass or geologists triplet, although a guidebook to coin collecting helped.
I find the mint's behavior over the last decade or two to be pretty annoying and money grubbing. It was cool having the same currency for decades and you got to know its little details pretty well, now it seems every year we have all new paper and/or coinage. Perhaps trying to cover up massive nation-state level counterfeiting rings. "Oh you went to the expense to make indistinguishable copies of our $20? F you we'll release a new one seemingly every year just to F with you" You can't win that battle but you can put up a fight, perhaps that is what our mint is doing. It is kinda obvious, though, last century they messed with at most one individual major currency item per decade, lately they mess with every piece of currency every decade.
(Score: 2) by requerdanos on Sunday February 25 2018, @09:49PM
Makes me wonder, in an idle sort of way, what area of a postage stamp 64x64px would be able to reproduce faithfully. A mm2, maybe?
Every $20 ever issued is still valid currency. Don't counterfeit them, but if you do--counterfeit whichever one(s) of them you want. They're all valid.
(Score: 3, Funny) by Anonymous Coward on Sunday February 25 2018, @02:29AM (3 children)
Teenager in slightly dystopian, but totally self-inconsistent world, seeks to find their individuality, while overthrowing a comically inept dictator.
Split into several movies with the last part split into part 1 and 2.
*Generate*
(Score: 0) by Anonymous Coward on Sunday February 25 2018, @02:59AM (1 child)
> ...nonsensical actions, such as "sailing on snow...
Sounds like ice boating to me, not "nonsensical" at all.
(Score: 0) by Anonymous Coward on Sunday February 25 2018, @03:19AM
Ou Est Le Swimming Pool [wikipedia.org] ?
(Score: 0) by Anonymous Coward on Sunday February 25 2018, @03:07AM
Sigh. And just after The Hunger Games novels convinced teens that it was cool to read books in public, the movies happened.
(Score: 2) by VLM on Sunday February 25 2018, @02:42PM
Oh, old timers already know.
There's a whole genre of crappy TV and movies from the 70s where everyone from the actors to cameramen to directors were high on something, and oh boy does it show. One tiny example is laugh tracks, some weed smokers get the giggles so some 70s shitcoms are little more than stoner giggle-fests. So "kitesurfing on grass" well I think I saw video footage of pretty much everything "on grass". Ironically I don't smoke, weed gets me headaches when I was young and now its an association (kinda like you can drink vodka like water until the first time you puke it back up, at which point you can't drink vodka anymore). But given that, I feel a kinda of contact high after watching "XYZ on grass" 70s hollywood productions.
The stuff thats re-broadcast in 2018 is kinda tame compared to the crap that was broadcast live in the 70s. Also the first season of something like "all in the family" is rebroadcast over and over and they pretend the last seasons never happened. Kinda like Battlestar Galactica was "OK" but we'll just pretend "Battlestar 1980" or whatever it was called never happened.