As a designer, I've built variants of this several times throughout my career.

The author's approach is really good, and he hits on pretty much all the problems that arise from more naive approaches. In particular, using a perceptual colorspace, and how the most representative colour may not be the one that appears the most.

However, image processing makes my neck tingle because there are a lot of footguns. PNG bombs, anyone? I feel like any library needs to either be defensively programmed or explicit in its documentation.

The README says "Finding main colors of a reasonably sized image takes about 100ms" -- that's way too slow. I bet the operation takes a few hundred MB of RAM too.

For anyone that uses this, scale down your images substantially first, or only sample every N pixels. Avoid loading the whole thing into memory if possible, unless this handled serially by a job queue of some sort.

You can operate this kind of algorithm much faster and with less RAM usage on a small thumbnail than you would on a large input image. This makes performance concerns less of an issue. And prevents a whole class of OOM DoS vulnerabilities!

As a defensive step, I'd add something like this https://github.com/iamcalledrob/saferimg/blob/master/asset/p... to your test suite and see what happens.

I've wanted something like this for level of detail processing.

This is a render from Second Life, in which all the texture images were shrunk down to one pixel, the lowest possible level of detail, producing a monocolor image. For distant objects, or for objects where the texture is still coming in from the net, there needs to be some default color. The existing system used grey for everything. I tried using an average of all the pixels, and, as the original poster points out, the result looks murky.[1] This new approach has real promise for big-world rendering.

[1] https://media.invisioncic.com/Mseclife/monthly_2023_05/monoc...

OKPalette by David Aerne is my favorite tool for this, it chooses points sensibly but then also lets you drag around or change the number of colors you want: https://okpalette.color.pizza/

I've been doing something similar! I've got a Home Assistant dashboard on my desk and wanted the media controls to match the current album art. I need three colors: background, foreground, and something vibrant to set my desk lamp to [1].

The SpotifyPlus HA integration [2] was near at hand and does a reasonably good job clustering with a version of ColorThief [3] under the hood. It has the same two problems you started with though: muddying when there's lots of gradation, even within a cluster; and no semantic understanding when the cover has something resembling a frame. A bit swapped from okmain's goal, but I can invert with the best of them and will give it a shot next time I fiddle. Thanks for posting!

[1] https://gist.github.com/kristjan/b305b83b0eb4455ee8455be108a... [2] https://github.com/thlucas1/homeassistantcomponent_spotifypl... [3] https://github.com/thlucas1/SpotifyWebApiPython/blob/master/...

It reminds me a bit of this post from the Facebook engineering blog (2015) [1] where they discuss embedding a very tiny preview of images into the html itself so they show immediately while loading the page, especially with very slow connections.

[1] https://engineering.fb.com/2015/08/06/android/the-technology...

I’m surprised the baseline to compare against is shrinking the image to one pixel, that seems extremely hacky and very dependent on what your image editor happens to do (and also seems quite wasteful… the rescaling operation must be doing a lot of extra pointless work keeping track of the position of pixels that are all ultimately going to be collapsed to one point).

So, making a library that provides an alternative is a great service to the world, haha.

An additional feature that might be nice: the most prominent colors seem like they might be a bad pick in some cases, if you want the important part of the image to stand out. Maybe a color that is the close (in the color space) to the edges of your image, but far away (in the color space) from the center of your image could be interesting?

This is nice! I looked into this quite a lot some years back when I was trying to summarize IKEA catalogs using color and eventually wrote an R package if you want to look into an alternative to e.g. k-means: https://github.com/lemonad/colorhull (download https://github.com/lemonad/ikea-colors-through-time/blob/mas... for more details on how it works)

Really interesting read. Thanks for sharing. Is the performance bottleneck around the resizing to 250k pixels? Would it still work if you sampled 15,625 4x4 patches evenly around the image to gather those pixels instead of resizing?

In the past when i tried just using image magick's built in -kmeans for this, i found chosing the second most prominent colour often looked really good. The primary was too much of the same thing.

I’d be interested in trying this out as a command-line tool. It would be useful on its own and the fastest way to evaluate results.

> simple 1x1 resize

How is it "simple"? There are like a ton of different downscaling algorithms and each of them might produce a different result.

Cool article otherwise.

I really like this approach. I worked on this problem (create a nice background for an image) for a couple weeks many years ago while organizing my desktop wallpaper collection, and never came up with a good answer. Unfortunately, I think that it's been "solved" in the tiktok era; an enlarged and blurred version of the image is used to fill the background space.

The blurred mirror is inoffensive to almost everyone, and yet it always strikes me as gauche. Easy to ignore and yet I feel that it adds a lot of useless visual noise.

See also https://github.com/material-foundation/material-color-utilit...

[flagged]