Skip to content

Bug: Many tests being skipped for dominant-color-images plugin #2521

Closed

Description

@nickchomey

Bug Description

Steps to reproduce

  1. Run npm install to install the Node.js dependencies.
  2. Run composer install to install the PHP dependencies.
  3. Run npm run build to do an initial build of the assets.
  4. Run npm run wp-env-test start
  5. run npm run test-php:dominant-color-images

it skips 24 tests, all of which relate to mime_type support. I dont know enough about wp-env, nor how the tests are built/run, so I dont have much of an idea about what is going on.

Additional Context

npm run test-php:dominant-color-images

> test-php:dominant-color-images
> wp-env --config=.wp-env.test.json run cli --env-cwd=/var/www/html/wp-content/plugins/performance composer test:dominant-color-images

ℹ Starting 'composer test:dominant-color-images' on the cli container. 

Composer could not detect the root package (wordpress/performance) version, defaulting to '1.0.0'. See https://getcomposer.org/root-version
Installing...
Running as single site... To run multisite, use -c tests/phpunit/multisite.xml
Not running ajax tests. To execute these, use --group ajax.
Not running ms-files tests. To execute these, use --group ms-files.
Not running external-http tests. To execute these, use --group external-http.
PHPUnit 8.5.42 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.31
Configuration: /var/www/html/wp-content/plugins/performance/phpunit.xml.dist

  Warning - The configuration file did not pass validation!
  The following problems have been detected:

  Line 47:
  - Element 'coverage': This element is not expected.

  Test results may not be as expected.


..................SS...........SSSSSSSSSSSSSSSSSSSS............  63 / 193 ( 32%)
............................................................... 126 / 193 ( 65%)
..............................................................S 189 / 193 ( 97%)
S...                                                            193 / 193 (100%)

Time: 4.64 seconds, Memory: 59.88 MB

There were 24 skipped tests:

1) Test_Dominant_Color_Image_Editor_GD::test_get_dominant_color_invalid with data set "tiff" ('/var/www/html/wp-content/plug...e.tiff')
Mime type image/tiff is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

2) Test_Dominant_Color_Image_Editor_GD::test_get_dominant_color_invalid with data set "bmp" ('/var/www/html/wp-content/plug...ge.bmp')
Mime type image/bmp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

3) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "animated_gif" ('/var/www/html/wp-content/plug...ed.gif', array('874e4e', '864e4e', 'df7f7f'), true)
Mime type image/gif is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

4) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "red_jpg" ('/var/www/html/wp-content/plug...ed.jpg', array('ff0000', 'fe0000'), false)
Mime type image/jpeg is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

5) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "green_jpg" ('/var/www/html/wp-content/plug...en.jpg', array('00ff00', '00ff01', '02ff01'), false)
Mime type image/jpeg is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

6) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "white_jpg" ('/var/www/html/wp-content/plug...te.jpg', array('ffffff'), false)
Mime type image/jpeg is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

7) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "red_gif" ('/var/www/html/wp-content/plug...ed.gif', array('ff0000'), false)
Mime type image/gif is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

8) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "green_gif" ('/var/www/html/wp-content/plug...en.gif', array('00ff00'), false)
Mime type image/gif is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

9) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "white_gif" ('/var/www/html/wp-content/plug...te.gif', array('ffffff'), false)
Mime type image/gif is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

10) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "trans_gif" ('/var/www/html/wp-content/plug...ns.gif', array('5a5a5a', '020202'), true)
Mime type image/gif is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

11) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "red_png" ('/var/www/html/wp-content/plug...ed.png', array('ff0000'), false)
Mime type image/png is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

12) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "green_png" ('/var/www/html/wp-content/plug...en.png', array('00ff00'), false)
Mime type image/png is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

13) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "white_png" ('/var/www/html/wp-content/plug...te.png', array('ffffff'), false)
Mime type image/png is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

14) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "trans_png" ('/var/www/html/wp-content/plug...ns.png', array('000000'), true)
Mime type image/png is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

15) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "red_webp" ('/var/www/html/wp-content/plug...d.webp', array('ff0000'), false)
Mime type image/webp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

16) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "green_webp" ('/var/www/html/wp-content/plug...n.webp', array('00ff00'), false)
Mime type image/webp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

17) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "white_webp" ('/var/www/html/wp-content/plug...e.webp', array('ffffff'), false)
Mime type image/webp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

18) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "trans_webp" ('/var/www/html/wp-content/plug...s.webp', array('000000'), true)
Mime type image/webp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

19) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "balloons_webp" ('/var/www/html/wp-content/plug...s.webp', array('c1bbb9', 'c0bbb9', 'c0bab8', 'c3bdbd', 'bfbab8'), false)
Mime type image/webp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

20) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_valid with data set "half_opaque" ('/var/www/html/wp-content/plug...ue.png', array('7e7e7e'), true)
Mime type image/png is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:181

21) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_invalid with data set "tiff" ('/var/www/html/wp-content/plug...e.tiff')
Mime type image/tiff is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

22) Test_Dominant_Color_Image_Editor_Imagick::test_get_dominant_color_invalid with data set "bmp" ('/var/www/html/wp-content/plug...ge.bmp')
Mime type image/bmp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

23) Test_Dominant_Color::test_get_dominant_color_invalid with data set "tiff" ('/var/www/html/wp-content/plug...e.tiff')
Mime type image/tiff is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

24) Test_Dominant_Color::test_get_dominant_color_invalid with data set "bmp" ('/var/www/html/wp-content/plug...ge.bmp')
Mime type image/bmp is not supported.

/var/www/html/wp-content/plugins/performance/plugins/dominant-color-images/tests/data/class-testcase.php:209

OK, but incomplete, skipped, or risky tests!
Tests: 193, Assertions: 472, Skipped: 24.
✔ Ran `composer test:dominant-color-images` in 'cli'. (in 5s 172ms)

Not sure what to fill in here - its all just wp-env. Here's some output from Site Health

Server architecture	Linux 6.18.7-76061807-generic x86_64
Web server	Apache/2.4.67 (Debian)
PHP version	8.3.31 (Supports 64bit values)


Active editor	WP_Image_Editor_Imagick
ImageMagick version number	1809
ImageMagick version string	ImageMagick 7.1.1-43 Q16 x86_64 22550 https://imagemagick.org
Imagick version	3.8.1
File uploads	Enabled
Max size of post data allowed	1G
Max size of an uploaded file	1G
Max effective file size	1 GB
Max simultaneous file uploads	20
Imagick Resource Limits	
area: 244 MB
disk: 2147483648
file: 768
map: 2 GB
memory: 1 GB
thread: 1
time: 0
ImageMagick supported file formats	3FR, 3G2, 3GP, AAI, AI, APNG, ART, ARW, ASHLAR, AVCI, AVI, AVIF, AVS, BAYER, BAYERA, BGR, BGRA, BGRO, BIE, BMP, BMP2, BMP3, BRF, CAL, CALS, CANVAS, CAPTION, CIN, CIP, CLIP, CMYK, CMYKA, CR2, CR3, CRW, CUBE, CUR, CUT, DATA, DCM, DCR, DCRAW, DCX, DDS, DFONT, DNG, DPX, DXT1, DXT5, EPDF, EPI, EPS, EPS2, EPS3, EPSF, EPSI, EPT, EPT2, EPT3, ERF, FARBFELD, FAX, FF, FFF, FILE, FITS, FL32, FLV, FRACTAL, FTP, FTS, FTXT, G3, G4, GIF, GIF87, GRADIENT, GRAY, GRAYA, GROUP4, HALD, HDR, HEIC, HEIF, HISTOGRAM, HRZ, HTM, HTML, HTTP, HTTPS, ICB, ICO, ICON, IIQ, INFO, INLINE, IPL, ISOBRL, ISOBRL6, J2C, J2K, JBG, JBIG, JNG, JNX, JP2, JPC, JPE, JPEG, JPG, JPM, JPS, JPT, JSON, K25, KDC, KERNEL, LABEL, M2V, M4V, MAC, MAP, MASK, MAT, MATTE, MDC, MEF, MIFF, MKV, MNG, MONO, MOS, MOV, MP4, MPC, MPEG, MPG, MPO, MRW, MSL, MTV, MVG, NEF, NRW, NULL, ORA, ORF, OTB, OTF, PAL, PALM, PAM, PATTERN, PBM, PCD, PCDS, PCL, PCT, PCX, PDB, PDF, PDFA, PEF, PES, PFA, PFB, PFM, PGM, PGX, PHM, PICON, PICT, PIX, PJPEG, PLASMA, PNG, PNG00, PNG24, PNG32, PNG48, PNG64, PNG8, PNM, POCKETMOD, PPM, PS, PS2, PS3, PSB, PSD, PTIF, PWP, QOI, RADIAL-GRADIENT, RAF, RAS, RAW, RGB, RGB565, RGBA, RGBO, RGF, RLA, RLE, RMF, RW2, RWL, SCR, SCT, SFW, SGI, SHTML, SIX, SIXEL, SPARSE-COLOR, SR2, SRF, SRW, STEGANO, STI, STRIMG, SUN, TEXT, TGA, THUMBNAIL, TIFF, TIFF64, TILE, TIM, TM2, TTC, TTF, TXT, UBRL, UBRL6, UIL, UYVY, VDA, VICAR, VID, VIFF, VIPS, VST, WBMP, WEBM, WEBP, WMV, WPG, X, X3F, XBM, XC, XCF, XPM, XPS, XV, XWD, YAML, YCBCR, YCBCRA, YUV
Image format transforms	image/heic → image/jpeg, image/heif → image/jpeg, image/heic-sequence → image/jpeg, image/heif-sequence → image/jpeg
GD version	bundled (2.1.0 compatible)
GD supported file formats	GIF, JPEG, PNG, WebP, BMP, AVIF
Ghostscript version	10.05.1

Activity

  1. moved this to Not Started/Backlog 📆 in WP Performance Ongoingon Jun 6, 2026
  2. westonruter commented on Jun 6, 2026

    @westonruter
    Member

    @nickchomey Thanks for reporting. How does #2522 work for you?

  3. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    Again, I'm brand new to wp-env and all of this testing stuff, but I can't help but think that copilot has misunderstood the issue and is trying to create a slop hack to just not even run the skipped tests, rather than actually fix the issue...

    My interpretation of the situation is that imagick and gd are installed in wp-env (or does it go through to check if they're in my host os?) and as site health shows, they SUPPORT those formats (and many more). So, the issue seems to be that the testing mechanism is not detecting that those formats are supported, rather than that they aren't actually supported at all.

    Also, it seems to be implementing its own mime type check, when the image editor classes already have such a function and it is called by _wp_image_editor_choose

    https://github.com/WordPress/wordpress-develop/blob/f36e443c93027a827eb2882096eadc179fd94fe6/src/wp-includes/media.php#L4377

    Though, I don't know if that path is actually used in the test harness.

    Worse, it seems like that proposed changed would just prevent imagick editor from being loaded at all - wouldn't that prevent other tests from running successfully...?

    Moreover, it ignored the fact that the first two skipped tests were for GD

    Whatever the case, the fix seems to be needed more at that wp-env-test level rather than the specific plugin's test script.

    Am I misunderstanding something?

  4. westonruter commented on Jun 6, 2026

    @westonruter
    Member

    @nickchomey Well, I'm comparing a unit test run before the change and after:

    For 169 tests, tt reduces the skipped test count down from 24 to 6. So it seems like an improvement. I'm at WordCamp Europe right now so I can't currently dig in too deep to this. I recall @thelovekesh or @b1ink0 having previously having identified issues with the testing environment not having the image libs available, so either of them (or maybe someone else I'm forgetting), would be more helpful.

  5. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    Something just feels very off about it all.

    I'll see if I can figure out how to get xdebug working in wp-env-test - if so, I'll step debug and surely get clarity about what's going on.

    If you know of any resources that might point me in the right direction with that, it would be appreciated. I normally use DDEV and it's all just seamless.

  6. thelovekesh commented on Jun 6, 2026

    @thelovekesh
    Member

    I think I proposed #1085 to fix this. Test setup should be deterministic and correctly setup which is guaranteed with proposed solution. wp-env can hide many things and we will never have full visibility since the whole system is abstracted by nodejs scripts.

  7. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    Browsing that issue, it seems relevant here. I'm also enormously skeptical about wp-env - at the moment it just seems like a very convoluted black box, which seemingly doesn't create a normal WordPress environment, perhaps especially for testing.

    As mentioned, Ive been developing with DDEV, which also uses docker but is really just creating an actual environment as you'd have on your hosting server.

    If I could come to understand wp-env better and, more importantly, step debug the tests, perhaps I'd feel more confident about it. But I don't know if I can even be bothered to look further - which goes against the argument that was made in #1085 by various people, that wp-env makes everything easier and more approachable for newcomers...

  8. b1ink0 commented on Jun 6, 2026

    @b1ink0
    Contributor

    For 169 tests, tt reduces the skipped test count down from 24 to 6. So it seems like an improvement. I'm at WordCamp Europe right now so I can't currently dig in too deep to this. I recall @thelovekesh or @b1ink0 having previously having identified issues with the testing environment not having the image libs available, so either of them (or maybe someone else I'm forgetting), would be more helpful.

    The main issue is not wp-env, but the Docker images it uses. The cli image is a very lightweight Alpine-based image, and it does not appear to support the full set of ImageMagick formats.

    I don't think this is intentional. We should probably raise an issue with the upstream Docker image. I previously reported an issue with the WordPress image not supporting AVIF, and it was fixed very quickly: #2245 (comment)

    If you run the following snippet in the wp-env cli container:

    php -r 'print_r(Imagick::queryFormats());'

    you get:

    /var/www/html $ php -r 'print_r(Imagick::queryFormats());'
    Array
    (
    )

    But in the wp-env wordpress container:

    root@fe0aea7735f2:/var/www/html# php -r 'print_r(Imagick::queryFormats());'
    Array
    (
        [0] => 3FR
        [1] => 3G2
        [2] => 3GP
        [3] => AAI
        [4] => AI
        [5] => APNG
        [6] => ART
        [7] => ARW
        [8] => ASHLAR
        [9] => AVCI
        [10] => AVI
        [11] => AVIF
        ...
        [260] => YUV
    )

    So there are two possible paths:

    1. Investigate whether the cli image can be updated to include full ImageMagick format support. If so, no changes would be needed on our side.
    2. If that is not feasible, use the wordpress image for image-related plugins and test suites Avoid forcing Imagick-only editor when Imagick lacks core format support #2522 (comment).

    I haven't dug deeply into the cli image yet, but I'll do some investigation and raise an issue with the upstream Docker library if needed.

    cc: @nickchomey, @westonruter

  9. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    OK, so you've confirmed what my initial suspicion was - that the copilot PR is not addressing the real issue, but instead likely lazily and silently masking deeper issues in wp-env cli test environment. Any effort that doesn't address that seems misguided. Id be in favor of changing/fixing the image, if that's the root of it.

  10. b1ink0 commented on Jun 6, 2026

    @b1ink0
    Contributor

    I haven't dug deeply into the cli image yet, but I'll do some investigation and raise an issue with the upstream Docker library if needed.

    So I did some digging and found that the cli image will not support ImageMagick at all due to Alpine's limitations and the way the image is configured. There is a comment in the Docker image itself that explains why it is not expected to work:

    https://github.com/docker-library/wordpress/blob/862322ddbba563563ae63e1dcee31d477626eb8e/cli/php8.2/alpine/Dockerfile#L50-L55

    # WARNING: imagick is likely not supported on Alpine: https://github.com/Imagick/imagick/issues/328
    # https://pecl.php.net/package/imagick
    	pecl install imagick-3.8.1; \
    	docker-php-ext-enable imagick; \
    	rm -r /tmp/pear; \
    	\

    @westonruter Given this, I think we should just switch to the wordpress image for image-related plugins and test suites.

  11. thelovekesh commented on Jun 6, 2026

    @thelovekesh
    Member

    wp-env generates the dockerfile, build and run it, so it's wp-env specific. also, tests should run in tests wordpress image and not the cli image.

  12. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    It sounds like you folks are knowledgeable and motivated about this issue and I don't think I have much left to add, so I'll just observe from the sidelines. Hopefully the cli/tests image can be changed to resolve the various problems that have been noted in this and related issues/prs.

    Though, I suspect that appropriately resolving this will expose other silent failures across the performance repo (and perhaps even gutenberg etc as well), as alpine/imagick seems like a sort of surface issue rather than an edge case...

  13. nickchomey commented on Jun 6, 2026

    @nickchomey
    ContributorAuthor

    I just got xdebug working in wp-env wordpress and cli. Was surprisingly easy. For others' reference:

    1. npm run wp-env-test start -- --xdebug
    2. configure the pathMapping in your vscode xdebug launch.json profile to something like /var/www/html/wp-content/plugins/performance/": "${workspaceFolder}/../pathtorepo/../performance/". Then it works as normal.

    (also documented in more detail here https://github.com/WordPress/gutenberg/tree/HEAD/packages/env#using-xdebug)

    Was able to confirm that the copilot PR was completely misguided - the $missing_formats variable contains all formats and so the add_filter never runs.

    Anyway, I see that #2523 provides a fix. Thanks for the quick collaboration on this! I'll rebase my work for #2519 on top of that and carry on

  14. moved this from Not Started/Backlog 📆 to Done 😃 in WP Performance Ongoingon Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

[Plugin] Image PlaceholdersFormerly "Dominant Color Images" [slug: dominant-color-images][Type] BugAn existing feature is broken

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions