Back in 2002 and 2003 I experimented with a few approaches towards sketch - and gesture-driven art generation. One idea was that users would, with a few strokes and gestures, be able to create expressive graphical animations. The resulting engine takes in user-generated strokes and, depending on gestures made with the mouse, generates animated variations. See here an
example animation.
The prototype interface is very simple with two sketchpads, below the generated animated graphics:

Try the basic functions for yourself with this online
version which I called Sumi-e botany for its graphic effect. Sketch something into the top and bottom sketchpads (stem and leaf), move the slider at the bottom (environment temperature) and hit "create plant" -
voilą! The mouse gesture input for controlling the animation movement is here a slider located below the resulting artwork.