thread about my palette cycler app
#mastodon #threads
related
here’s today’s mental puzzle: is normal mapping possible in. 256 indexed color mode from a single texture image?
trivial answer is that it’s straightforward to use standard median cut and dithering techniques on the rgb normal map. the different “channels” can be read for each pixel by finding the rgb color for its index and just, using the r g and b values from the palette to do the light calculation. but can this be pushed further? what if you do the light calculation for each palette color, and that is now a palette you can switch to?
further, can you combine normal map and rgb diffuse texture, and apply lighting with a palette swap?
how far can you push it? how far can you take crossfading multiple palettes?
I realised soon after watching this video from Mark Fararri years ago, that it was theoretically possible to encode any arbitrary animation as a color cycling animation. While Mark uses various tricks, it occurred to me that you could just use the median cut algorithm in a weird way.
median cut is the algorithm tools usually use to reduce 24bit images to indexed palatted images. but it turns out there’s nothing in median cut that limits it to 3 input channels
youtube.com/watch?v=aMcJ1Jvtef
and so, since most tools that do median cut don’t take more than 3 channels as input, i’ll need to writr custom software to test the theory. but in essence it’d work by chucking each frame of animation on as an additional 3 channels for median cut. in principle, similar areas of the images should merge over time into the same palette entries. i am interested in what feeding cyclegan results into this would do
essentially the palette would become not just a list of rgb colors, but a list of R1G1B1, R2G2B2… RnGnBn colors.
but surely this isn’t limitless! what will be interesting is finding out where and how this scheme breaks.
what would also be interesting is trying out wweeeeird shit, like more than 256 palette entries to hold more animation frames before breaking.
at one point you were mentioning something about a video codec based on retro video concepts?
at the moment i am sorta stuck in a mental loop. and also on implementing a simple fft. i realised that individual pixels with their colors plotted over time tend to have periodic patterns, and that i should sort/merge colors on their wave properties over and above their colors.
when waves of slightly different phase get merged together, their image regions merge into .. a motion blur!
and so there’s an uncanny relationship between the image and region space and the palette wave space sorta thing that i am trying to wrap my brain around hoping something magical happens, and trying to work out where the motion compensation blocks fit in. if i wanted to go old school i could put some hard limit ot the motion compensation blocks similar to an nes or something, and integrate it all with my slitscan compression scheme, i just don’t know how to fit all the peices together yet
How much fidelity are you going for? You mentioned something about exploring compression artifacts as an aesthetic, which I think would be pretty neat.
—
i aim for the capability of being lossless or close to lossless, with dials for lossiness.
—
Wish I could help with the mental block, but I can barely keep things straight with 1d FFTs in the audio domain.
If these are tracking changes of pixels or a region of pixels over a period of time, how would you determine the length of the FFT vector?
You did mention GIFs, which are naturally periodic.
—
the idea, and the reason I’m not just picking an fft thing off the shelf, is I am only interested in the one or two dominant frequencies, not to use to reproduce the color later, but as a sorting key.
see the median cut algorithm cut algorithm I started with assumes that the cloud of vectors you’re working with are diverse, but some of my input images are 2 or 3 colors total black and white, so when you plot the vectors out, even if you incude each frame as a new dimension, are really just sitting on a couple planes- cutting the median of this space doesn’t produce good results.
so what I really want to be doing is sorting the animated palette colors according to wavelength and phase, and only THEN by color. then natural cycling groups would emerge from that sort.
frequency domain is a nice space to cut a median in.
Zen @zensaiyuki list of videos about color cycling animations
youtube.com/watch?v=LUbrzg21X9
youtube.com/watch?v=Z-VO-hxLsE
youtube.com/watch?v=aMcJ1Jvtef
https://mastodon.social/@jplebreton/103354763788884848
Current color palettes used: Deluxe Paint ][, Doom, Quake. I’ll probably add Heretic and the Atari 8-bit micros. 256 color palettes with smooth ramps work nicely, I added special code to define these and interpolate along them.
