Poetic Fusion is a project exploring the translation of poetry to a visual medium. The project uses the ModalArt TextKeywordImage tool to query images based on text, which are then combined to create the resulting image. All poems are the original work of the authors cited, and the corresponding images are released in the public domain without license. For more information on project details, email loudunwirings AT gmail DOT com.
Art Between Modalities
Author: Bradley Baker
Currently in development.
0.0.1 - Initial Development Version 0.0.2 - Added more flexible support for scales, and support of spectrogram img-encoding 0.0.3 - Moved to Python3, Added Text to Image Converter, additional Spectrogram encoding, and updates to EdgeMidi
This package strives to foster the creation of art by conversion of data modalities, facilitating new perspectives on the art-creation process and the art-encounter via technology.
Current functionality targets conversion between RGB images, audio, and text; however, conversion between other kinds of data and art modalities will be explored in the future.
Feedback is appreciated.
Midi works most simply in two dimensions - the Pitch (Y) dimension, and the Time (X) dimension. There are additional dimensions given in the intensity of the strike on the virtual keyboard and in the channel chosen, etc; however, the simpleest mapping imagines these values as pre-determined constants, such that the MIDI functionality involves some mapping of pitches over time.
Similarly images at their most basic have two spatial dimensions, an X and Y spatial axis. The idea of this program is thus spatial information given in an image to create a mapping from the image into this basic MIDI-space. There are many potential mappings from the image-space into MIDI-basic-space and into other MIDI spaces; however, one simple mapping uses a binary-spatial-indicator to detect whether or not an image is present at a given spatial coordinate in an image, and if so, adds a positive binary indicator to the same coordinate in the MIDI-basic-space.
To put it simply, we imagine that the MIDI-basic-space of pitches over time is placed as a grid over an image. Then, wherever we detect an edge in the image at a coordinate (x,y), we indicate that pitch (y) should be played at time (x).
This software performs 2 basic functions: 1) Edge Detection 2) Edge -> MIDI mapping
Beyond these basic functions, there is the included functionality of mapping images to particular Musical Keys.
Converts image data into a spectrogram which can be used to generate .wav files. Alexodan's img-encode library is included, and additional "Squashed" analysis provides quantization of the spectrogram to a particular choice of scale.
All scales in scales.csv are currently supported.
With input text, this module queries google images for images with related keywords. Queries are currently made naively, line by line. The library outputs a generated image created by combining the queried images.
For example, we take the text of "Ten Maps of Sardonic Wit" by Christian Bok:
Ten Maps of Sardonic Wit atoms in space now drift on a swift and epic storm soft wind can stir a poem snow fits an optic dream into a scant prism of dew words spin a faint comet some words in fact paint two stars of an epic mind manic words spit on fate
and output the following image:
I am looking for active co-developers. Contact me at bbradt.com if you would like to contribute.
EdgeMidi MIDI mapping and output for Images already Edge-Detected All MAJOR scales All MINOR scales GNU-style Command-Line arguments for: Adjusting resolution of grid Input/Output Turning debugging on/off more Pitch-Grid Debugging Outputs Customizeable Config File Spectro Squashed Spectrogram Analysis with Quantization Additional ImgEncode functionality based on [alexodan's img-encode library](https://github.com/alexadam/img-encode) TextKeywordImage Generate an Image based on text
GUI Unit Tests Image-Format Tests Edgemidi
More command line arguments: Adjust MIDI Tempo Adjust MIDI Volume Alternative Octave-Grid Wrapping Choices Dynamic Image Transformation : Auto-Playback Generation of Video including time-marker showing pitch-playback Auto-Dependency Check/Installation More Scales Full Harmonic Scales All augmented/diminished keys All modes for major/minor keys Blues scales Jazz scales ...
Usage: Required Packages: See requirements.txt
The tool is used only via the command line currently. A GUI is on the TODO list. For now, the tool can be used via the command: $python EdgeMidi.py -i <inputfile> $python TextKeywordImage.py -i <inputfile> $python SquashSpectro.py -i <inputfile> For additional flags and arguments via the command line, type python EdgeMidi.py -h python TextKeywordImage.py -h python SquashSpectro.py -h