Mark AVX2/AVX3 as non-broken on 32-bit x86

[johnplatts]

Here is the how HWY_BROKEN_32BIT is currently defined in hwy/detect_targets.h:

highway/hwy/detect_targets.h

Lines 155 to 159 in ceb1f0f

	#if HWY_ARCH_X86_32
	#define HWY_BROKEN_32BIT (HWY_AVX2 | HWY_AVX3 | HWY_AVX3_DL | HWY_AVX3_ZEN4)
	#else
	#define HWY_BROKEN_32BIT 0
	#endif

I was able to get HWY_AVX2/HWY_AVX3/HWY_AVX3_DL compiled successfully for 32-bit x86 on GCC 9, GCC 12, Clang 13, and Clang 15 on Ubuntu 22.04, and all of the Highway unit tests pass for HWY_AVX2/HWY_AVX3/HWY_AVX3_DL on 32-bit x86 with GCC 9, GCC 12, Clang 13, and Clang 15.

I also was able to get HWY_AVX2 compiled successfully for 32-bit x86 with MSVC 2019 (19.29), and all of the Highway unit tests pass for the HWY_AVX2 target on 32-bit x86 when compiled with MSVC 19.29.

HWY_BROKEN_32BIT should be changed to 0 for 32-bit x86 when compiled with GCC 9 and later or Clang 13 and later, and HWY_BROKEN_32BIT should be changed to (HWY_AVX3 | HWY_AVX3_DL | HWY_AVX3_ZEN4) for MSVC 19.29 or later.

Here is an updated version of the HWY_BROKEN_32BIT macro that will mark AVX2 as non-broken on 32-bit x86 with MSVC 19.29 or later, GCC 9.0 or later, or Clang 13.0 or later and that will mark AVX3 as non-broken on 32-bit x86 with GCC 9.0 or later or Clang 13.0 or later:

#if HWY_ARCH_X86_32
#if ((HWY_COMPILER_GCC_ACTUAL && HWY_COMPILER_GCC_ACTUAL < 900) || \
     (HWY_COMPILER_CLANG && HWY_COMPILER_CLANG < 1300))
 #define HWY_BROKEN_32BIT 0
#elif HWY_COMPILER_MSVC && HWY_COMPILER_MSVC < 1929
 #define HWY_BROKEN_32BIT (HWY_AVX3 | HWY_AVX3_DL | HWY_AVX3_ZEN4) 
#else
 #define HWY_BROKEN_32BIT (HWY_AVX2 | HWY_AVX3 | HWY_AVX3_DL | HWY_AVX3_ZEN4) 
#endif
#else  // x86_64 or non-x86
 #define HWY_BROKEN_32BIT 0 
#endif  // HWY_ARCH_X86_32

[jan-wassenberg]

Thanks for testing on the various compilers. If I understand correctly, we mean

#if !HWY_ARCH_X86_32 || (HWY_COMPILER_GCC_ACTUAL >= 900 || HWY_COMPILER_CLANG >= 1300)
#define HWY_BROKEN_32BIT 0
#else
#define HWY_BROKEN_32BIT (HWY_AVX2 | HWY_AVX3 | HWY_AVX3_DL | HWY_AVX3_ZEN4) 

To simplify things, let's keep HWY_BROKEN_MSVC separate - it is easy to OR together as many HWY_BROKEN as we want.
For MSVC I think you are saying that >= 19.29 disables AVX3+ and < 19.29 disables AVX2?

This sounds good to me, I'd be happy to give that a go when back next week, or would you like to send a pull request instead?

(Note that we want JPEG XL to also work if Highway considers the target non-broken, and that is a higher bar to pass than just the Highway tests, so it is possible we might require a newer MSVC version for that.)

[johnplatts]

To simplify things, let's keep HWY_BROKEN_MSVC separate - it is easy to OR together as many HWY_BROKEN as we want. For MSVC I think you are saying that >= 19.29 disables AVX3+ and < 19.29 disables AVX2?

Yes

[johnplatts]

I did some further testing, and I was able to get the HWY_AVX2 target compiled successfully for 32-bit x86 with GCC 7 and GCC 8 using an Ubuntu 18.04 Docker container. The Highway unit tests for the HWY_AVX2 target all pass on 32-bit x86 when compiled with GCC 7 or GCC 8.

I also was able to get the HWY_AVX2 target compiled successfully for 32-bit x86 with Clang 7, Clang 8, Clang 9, Clang 10, Clang 11, and Clang 12, but I have to pass additional C++ compiler flags to clang++ to get Highway to compile successfully for 32-bit x86 on x86_64 with Clang 12 and earlier on Ubuntu. The Highway unit tests for the HWY_AVX2 target all pass on 32-bit x86 when compiled with Clang 7 or later.

Here are the additional compiler flags that need to be passed into Clang 11/Clang 12 on x86_64 Ubuntu 22.04 to get Google Highway to compile successfully for 32-bit x86:
-m32 -isystem /usr/i686-linux-gnu/include

Here are the additional compiler flags that need to be passed into Clang 7/8/9/10 on x86_64 Ubuntu 18.04 to get Google Highway to compile successfully for 32-bit x86:
-m32 -isystem /usr/i686-linux-gnu/include -isystem /usr/i686-linux-gnu/include/c++/12/i686-linux-gnu

I was able to get the HWY_AVX3 target compiled successfully for 32-bit x86 using GCC 7, GCC 8, Clang 7, Clang 8, Clang 11, and Clang 12 with the fix in pull request #1281. All of the Highway unit tests for the HWY_AVX3 target pass on 32-bit x86 when compiled with GCC 7, GCC 8, Clang 8, Clang 11, and Clang 12.

There is a compiler bug in Clang 9 and Clang 10 that causes the HWY_AVX3 target to fail to compile for 32-bit x86.

[johnplatts]

(Note that we want JPEG XL to also work if Highway considers the target non-broken, and that is a higher bar to pass than just the Highway tests, so it is possible we might require a newer MSVC version for that.)

I did test compile JPEG XL for the HWY_AVX2/HWY_AVX3 targets on 32-bit x86, but some of the tests for JPEG XL were failing for the HWY_AVX2/HWY_AVX3 targets on 32-bit x86 but were passing on 64-bit x86.

[jan-wassenberg]

Sorry to hear that JPEG XL tests still fail for AVX2/3 on 32-bit. @johnplatts do they fail even with recent GCC and Clang?
If so, then I think we had best leave AVX2/3 disabled on 32-bit :/

One thing we can do is add your compiler flag comment to the readme, this looks helpful for anyone still using 32-bit.

[jan-wassenberg]

Following up: if JPEG XL indeed still fails AVX2/3 for 32-bit, I think we'll need to keep them marked as broken - but with the option of overriding HWY_BROKEN_TARGETS for anyone who wants.
Closing then, but please feel free to reopen for further discussion :)