Trained using Flux LoRA with a customised dataset — transforming Chinese calligraphy into an ink-snake typestyle to celebrate the Year of the Snake.
A self-set challenge: turn a lifelong fear of snakes into a culturally resonant typestyle, where snake forms adapt to the structure of Chinese characters.
I've always been terrified of snakes, so when the Year of Snake approached, I challenged myself to create art that would transform this fear into something meaningful. While researching Chinese snake paintings online, I had an epiphany — the fluid movement of snakes somewhat mirrors the flow of Chinese characters, and Chinese calligraphy's abstract nature opened up possibilities: what if snakes could become a typestyle for Chinese characters?
I then coined this concept "Chinese Snake Calligraphy," where snake forms would adapt to character structures. By focusing on the graceful and abstract flow of Chinese brush ink rather than realistic snake features, I could create something less intimidating yet culturally resonant. My search results revealed stunning examples of this harmony between snake forms and Chinese calligraphic strokes:
My goal was straightforward: train a model to transform Chinese calligraphy into ink-snake style. Approaches like pix2pix weren't feasible due to limited examples of snake-styled characters. While ControlNet Line Detection can offer precise stroke mapping, it would restrict the artistic freedom needed for abstract interpretations. Therefore, I turned to Flux LoRA training after consulting with an AI expert who introduced me to fal.ai. Then, studying others' creative processes helped me understand the tool's potential and guided my first steps.
Flux LoRA combines Flux (an AI image-generation model) with LoRA (Low-Rank Adaptation), a technique for fine-tuning AI models — offering a more efficient and accessible way to adapt powerful AI models to create custom artistic styles.
Dataset preparation was undoubtedly the most crucial step. "Filtering" and "Categorizing" became my guiding principles. I batch-collected ink-snake images with white backgrounds from various sources (Shutterstock, iStock, Pinterest), then conducted training iterations while continuously refining the dataset:
Testing was equally iterative, involving multiple inference tests and parameter adjustments (prompts, scale, seed numbers). To streamline the process, I prepared a testing dataset of 5 Chinese characters on white background. After numerous inference runs, I cataloged the most satisfying outputs. This systematic approach led to a model that achieved 90% of my artistic expectation.
For Flux LoRA training, quality beats quantity — a small batch of cohesive-styled images outperforms a large pool of inconsistent ones.
Moving to animation, I chose Runway Gen-3 Alpha Turbo to bring the snakes to life. Using the Flux outputs as final frames, I prompted "snakes moving around in fluid movement in the center" to mimic the gradual stroke progression of Chinese calligraphy. I used consistent seed numbers to maintain motion coherence.
Interestingly, characters with more strokes produced better results than simpler ones like "大 (big)." I'm guessing this is because simpler characters have more white space, giving the model too much freedom in motion generation and making it harder to maintain structural integrity. Complex characters provide more visual constraints and anchor points, which help the model generate more controlled animations that preserve the character's essence. After multiple iterations, I selected the most successful outputs for final implementation.
"蛇年大吉 (shé nián dà jí)" — a Chinese New Year blessing meaning "Wish you prosperity in the Year of the Snake" — rendered in the trained ink-snake style, along with its animated version.
I've also brought "Fortune (福)" into the world through AR — at Chinatown and St. Paul's Cathedral.