Bored in Work 1039 (1 Viewer)

  • Thread starter pete
  • Start date
  • Replies 10
  • Views 969
  • Watchers 1

pete

chronic procrastinator
Staff member
Since 1999
Joined
Nov 14, 1999
Messages
62,900
Solutions
3
Location
iPanopticon
Website
thumped.com
To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.


In this video, I utilized artificial intelligence to generate an animated music video for the song Canvas by Resonate. This tool allows anyone to generate beautiful images using only text as the input. My question was, what if I used song lyrics as input to the AI, can I make perfect music synchronized videos automatically with the push of a button? Let me know how you think the AI did in this visual interpretation of the song.

After getting caught up in the excitement around DALL·E2 (latest and greatest AI system, it's INSANE), I searched for any way I could use similar image generation for music synchronization. Since DALL·E2 is not available to the public yet, my search led me to VQGAN + CLIP (Vector Quantized Generative Adversarial Network and Contrastive Language–Image Pre-training), before settling more specifically on Disco Diffusion V5.2 Turbo. If you don't know what any of these words or acronyms mean, don't worry, I was just as confused when I first started learning about this technology. I believe we're reaching a turning point where entire industries are about to shift in reaction to this new process (which is essentially magic!).

Important note:
While this AI is impressive, it still required additional input beyond just the song lyrics to achieve the music video I was looking for. For example, I added keyframes for camera motion throughout the generated world. These keyframes were manually synchronized to the beat by me. I also specified changes to the art style at different moments of the song. Since many of the lyrics are quite non-specific, even a human illustrator would have a hard time making visual representations. To make the lyrics more digestible by the AI, I sometimes modified the phrase to be more coherent, such as specifying a setting or atmosphere.

This was my first time working with DDV5, and I'm very happy with the results! There were many times where my jaw dropped upon seeing what the AI came up with. I haven't felt this sense of wonder from technology since I first experienced a HD videogame as a child.

If you would like to learn more about how this video was made, try this yourself, or ask me any questions, I'll post a more detailed explanation of how to get started on Patreon (link below). The post is free to the public, no need to pay. If you do want to support me and become a member that would be much appreciated, you'll also automatically be entered into the end screen minigames where you earn points on each video and move up the leaderboard!

Join on Patreon to automatically have your name included in the next video: DoodleChaos is creating Music Visualizations and Kinetic Contraptions | Patreon
 
what the actual fuck

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.
 
I was reading the quote tweet comments for the lols and there were a few 'why wasn't this in the media' ones as per usual. I suspect those people watch fox instead of cnn
 
To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

practical special effects > visual effects
 
i cannot fathom how it was done, from about 1:50 onwards the camera pans back to face where it had been coming from with no indication of how it was carried, and it just takes off from there.
 
Shocking for an American movie, the movie’s biggest set piece is the French assualt on the Ouistreham casino. This features a magnificent helicopter tracking shot of the French commandos charging through the streets.

 

Users who are viewing this thread

Activity
So far there's no one here
Old Thread: Hello . There have been no replies in this thread for 365 days.
Content in this thread may no longer be relevant.
Perhaps it would be better to start a new thread instead.

21 Day Calendar

Mohammad Syfkhan 'I Am Kurdish' Dublin Album Launch
Bello Bar
1 Portobello Harbour, Saint Kevin's, Dublin, Ireland
Mohammad Syfkhan 'I Am Kurdish' Dublin Album Launch
Bello Bar
1 Portobello Harbour, Saint Kevin's, Dublin, Ireland
Bloody Head, Hubert Selby Jr Infants, Creepy Future - Dublin
Anseo
18 Camden Street Lower, Saint Kevin's, Dublin, Ireland

Support thumped.com

Support thumped.com and upgrade your account

Upgrade your account now to disable all ads... If we had any... Which we don't right now.

Upgrade now

Latest threads

Latest Activity

Loading…
Back
Top