This video is a rough demonstration of how a video might be generated from Music Lyric prompts using various techniques of AI coercion. It shows some of the promise, but also some of the challenges in attempting such a feat.
- Quickly generate thousands of original works of art from mere text prompts
- undesirable images… human moderation may always be required
- dirty lyrics will generate pictures that are not always family friendly
- sometimes lyrics are interpreted too literally
- camera motion is simulated
- rate of frame change if often unnatural, but this is something I’m working on balancing through algorithms
Now here’s a AI interpretation of a book passage that is considerably cleaner. This video has a higher framerate and constant motion and doesn’t suffer from as many visual glitches as the previous video. A goal of this project is to achieve smooth dreamy motion influenced by mere words and music.