Thanks to Sebastien and Julian for the hints. There seem to have been some performance variation in upstream around 6.9.10...6.9.11, but never quite as bad as the Ubuntu package. Steps taken: $ sudo apt-get build-dep imagemagick-6.q16 $ git clone https://github.com/ImageMagick/ImageMagick6.git $ cd ImageMagick6 $ git checkout 6.9.11-60 $ ./configure --prefix=/usr/local/stow/ImageMagick6 --with-modules --with-gs-font-dir=/usr/share/fonts/type1/gsfonts -with-magick-plus-plus --with-djvu --with-heic --with-openjp2 --with-webp --with-wmf --without-gvc --without-dps --without-fpx --with-perl --without-rsvg --with-fftw $ make $ sudo make install $ cd /usr/local/stow $ sudo stow ImageMagick6 $ sudo ldconfig $ cd ~ $ time /usr/local/bin/convert -version scan.pnm -compress Fax scan-upstream.pdf Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-24 https://imagemagick.org Copyright: (C) 1999-2021 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC Modules OpenMP(4.5) Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib real 0m10.906s user 0m11.559s sys 0m0.829s This is slower than I had on Focal, I believe it was around 5 seconds but I don't have measurements. Unstow it and I'm back at 4m44s: $ cd /usr/local/stow $ sudo stow -D ImageMagick6 $ sudo ldconfig $ cd ~ $ time /usr/bin/convert -version scan.pnm -compress Fax scan-ubuntu.pdf Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org Copyright: (C) 1999-2021 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC Modules OpenMP(4.5) Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib real 4m43.623s user 2m6.014s sys 2m38.471s $ ls -la scan-*.pdf -rw-rw-r-- 1 i i 127070 Okt 15 16:35 scan-ubuntu.pdf -rw-rw-r-- 1 i i 127046 Okt 15 16:29 scan-upstream.pdf Configuring the package resulted in the following: ============================================================================== ImageMagick 6.9.11-60 is configured as follows. Please verify that this configuration matches your expectations. Host system type: x86_64-pc-linux-gnu Build system type: x86_64-pc-linux-gnu Option Value ------------------------------------------------------------------------------ Shared libraries --enable-shared=yes yes Static libraries --enable-static=yes yes Module support --with-modules=yes yes GNU ld --with-gnu-ld=yes yes Quantum depth --with-quantum-depth=16 16 High Dynamic Range Imagery --enable-hdri=no no Install documentation: yes Memory allocation library: JEMalloc --with-jemalloc=no no TCMalloc --with-tcmalloc=no no UMem --with-umem=no no Delegate library configuration: BZLIB --with-bzlib=yes yes Autotrace --with-autotrace=no no DJVU --with-djvu=yes yes DPS --with-dps=no no FFTW --with-fftw=yes yes FLIF --with-flif=yes no FlashPIX --with-fpx=no no FontConfig --with-fontconfig=yes yes FreeType --with-freetype=yes yes Ghostscript lib --with-gslib=no no Graphviz --with-gvc=no HEIC --with-heic=yes yes JBIG --with-jbig=yes yes JPEG v1 --with-jpeg=yes yes JPEG XL --with-jxl=yes no LCMS --with-lcms=yes yes LQR --with-lqr=yes yes LTDL --with-ltdl=yes yes LZMA --with-lzma=yes yes Magick++ --with-magick-plus-plus=yes yes OpenEXR --with-openexr=yes yes OpenJP2 --with-openjp2=yes yes PANGO --with-pango=yes yes PERL --with-perl=yes /usr/bin/perl PNG --with-png=yes yes RAQM --with-raqm=yes no RAW --with-raw=yes no RSVG --with-rsvg=no no TIFF --with-tiff=yes yes WEBP --with-webp=yes yes WMF --with-wmf=yes yes X11 --with-x= yes XML --with-xml=yes yes ZLIB --with-zlib=yes yes ZSTD --with-zstd=yes no Delegate program configuration: GhostPCL None pcl6 (unknown) GhostXPS None gxps (unknown) Ghostscript None gs (9.55.0) Font configuration: Apple fonts --with-apple-font-dir=default Dejavu fonts --with-dejavu-font-dir=default none Ghostscript fonts --with-gs-font-dir=/usr/share/fonts/type1/gsfonts /usr/share/fonts/type1/gsfonts/ URW-base35 fonts --with-urw-base35-font-dir=default /usr/share/fonts/type1/urw-base35/ Windows fonts --with-windows-font-dir=default /usr/share/fonts/truetype/msttcorefonts/ X11 configuration: X_CFLAGS = X_PRE_LIBS = -lSM -lICE X_LIBS = X_EXTRA_LIBS = Options used to compile and link: PREFIX = /usr/local/stow/ImageMagick6 EXEC-PREFIX = /usr/local/stow/ImageMagick6 VERSION = 6.9.11-60 CC = gcc CFLAGS = -I/usr/include/libxml2 -I/usr/include/libpng16 -pthread -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/fribidi -I/usr/include/harfbuzz -I/usr/include/cairo -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/OpenEXR -I/usr/include/lqr-1 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/openjpeg-2.4 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -pthread -fopenmp -Wall -g -O2 -mtune=core2 -fexceptions -pthread -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 CPPFLAGS = -DMAGICKCORE_HDRI_ENABLE=0 -DMAGICKCORE_QUANTUM_DEPTH=16 PCFLAGS = DEFS = -DHAVE_CONFIG_H LDFLAGS = LIBS = -llcms2 -lfreetype -llqr-1 -lglib-2.0 -lfftw3 -lxml2 -lfontconfig -lfreetype -lXext -lSM -lICE -lX11 -lXt -llzma -lbz2 -lz -lltdl -lm -lpthread CXX = g++ CXXFLAGS = -pthread FEATURES = DPC Cipher OpenMP Modules DELEGATES = bzlib djvu fftw fontconfig freetype heic jbig jng jpeg lcms lqr lzma openexr openjp2 pango png ps tiff video webp wmf x xml zlib ==============================================================================