How we test

Last updated: Oct 20, 2026

The reviews, benchmarks and screenshots on this site come from our own testing. This page explains the machine we use, which versions we test, which files, how we time things and how we correct mistakes, so you can judge whether our results apply to you and repeat them yourself.

The test machine

The machine has no graphics card, so every app runs with software rendering. That exposes what happens without a usable GPU (common in virtual machines, remote desktops and on older PCs), but it also means interface drawing and video or motion previews are slower than on a PC with graphics. We point this out wherever a conclusion depends on it.

The versions we test

We only test official builds: the Linux AppImage of each app downloaded from its GitHub Releases, never a mirror or a repackaged version. ArtCraft has no Linux installer, so we built it from source following its official development guide. Versions in the current round (2026-10-20):

• ArtCraft 0.42.0 (source, commit 7a2935c)
• PhotoCraft 0.9.0
• VectorCraft 0.13.1
• FilmCraft 0.9.0
• LightCraft 0.9.1
• PdfCraft 0.9.0
• EffectCraft 0.11.1
• DesignCraft 0.9.0
• WordCraft 0.8.0
• GridCraft 0.8.0
• DeckCraft 0.8.0
• CADCraft 0.8.0
• SoundCraft 0.8.0
• FigCraft 0.7.0
• ArtCraft Launcher 0.5.1

Every review states the version and date at the top. When a new version ships, the version number and download links on each page update automatically, but the review text only changes after we test again.

The test files

We only use public files with clear licences, so anyone can download them and repeat our tests:

• PSD: the psd-tools test suite (316 PSD/PSB files, MIT licence).
• PDF: the pypdf sample collection (35 files, CC BY-SA 4.0).
• Photos: public-domain NASA images, CC0 photos from scikit-image and the Kodak test image set.
• Office documents, drawings, audio and video: .docx, .xlsx, .pptx, .dxf, .mp4, .wav and SVG files we generated ourselves with open-source libraries (python-docx, openpyxl, python-pptx, ezdxf, FFmpeg).

Files whose licence is unclear (some camera raw samples, for example) are used for testing only and their images are not shown in screenshots.

How we measure

• Speed: wherever possible we use the command line for the same job in every tool, to rule out mouse timing. Each task runs at least three times and we report the median, with nothing else running.
• Baselines: every test is compared with established open-source tools (GIMP, ImageMagick, MuPDF, Poppler, qpdf, Ghostscript, Tesseract), with their versions and settings. Where settings don’t map one-to-one (a blur radius, say) we say so and check the outputs by eye.
• Compatibility: for PSD we compare the app’s render pixel by pixel with the composite Photoshop stored in the file (root-mean-square difference); for PDF we cross-check the app’s own checker against MuPDF and Poppler.
• Memory: resident memory (RSS) once the window has settled, summed over all processes for multi-process apps.
• Start-up time: from the launch command to the window appearing. If a baseline tool behaves abnormally on our machine for environmental reasons, we leave its number out and say why.

Screenshots and recordings

Images marked “Screenshot: CraftSuite” in reviews and first-run notes were taken by us on the test machine while doing what the caption describes. They are not retouched or composited (side-by-side comparisons are labelled as such). Recordings show real operation at normal speed. A few images come from official demo material, for example ArtCraft features we couldn’t use without an account, and are credited under the image. The product image at the top of each app page comes from the app’s official repository and is credited with its licence.

What we can’t test

Reviews say plainly what we didn’t test. Current limits: no hands-on testing on Windows or macOS (the install guides are based on official documentation and reader reports); no performance testing with a dedicated GPU; and no ArtCraft account or credits, so no review of AI generation quality.

Independence

CraftSuite has no relationship with the ArtCraft team, Adobe, Microsoft or any other vendor. We receive no payment, free licences or early builds, and we don’t send reviews to developers for approval. The site is funded by Google AdSense ads; advertisers have no say in our conclusions and we don’t accept paid reviews. See About and the Disclaimer.

Corrections

If you spot a wrong number, an outdated conclusion, or get a different result on your own computer, please email us (contact@craftsuite.net), ideally with the app version, your OS and the steps to reproduce. Once verified we fix the page promptly and update its date; significant corrections are noted in the relevant section.