The matrix calibrator script

By: Frank Rysanek of FCC prumyslove systemy s.r.o. [rysanek (AT) fccps.cz]

Download

This leading chapter is a shortcut.
Download right here.
Apart from this Perl script, you'll need the original xinput_calibrator (such as the Debian 9 standard package xinput-calibrator) and some common X cmdline tools, namely xinput (from xinput.deb) and xrandr (from x11-xserver-utils.deb).

Intro

As many of you out there may be aware, touchscreen calibration in X-Windows has always been a neglected child. During the last decade or so, I seem to recall three or four different generic calibration tools for X-Windows:

All of them from 3rd parties rather than from the X.org / FreeDesktop.org core team. And, understandably none of them very vibrant with maintenance.
I'm afraid to say that none of them has worked for me this time around, in Debian 9 as of June 2018.

It's making me wonder. X.org is such a hugely successful project. It has defined internal standard layers and interfaces for input device coordinate processing and calibration. And, the X.org team do not provide a generic calibrator at the same time?
I mean - it's understandable that there's progress in the internals, and the interfaces inevitably develop in time, but what's the point if there are no calibration tools in the same box?
Kudos to the independent authors of the three (four) projects.

An obvious response might be "Oh, calibration? Where have you been during the last decade? Projected Capacitive touchscreens with the HID interface are all the rage now, and they come calibrated for the native display resolution. Oh you have some odd industrial touchscreen that needs to be calibrated? Come on, that's ghetto hardware you have there. Get something decent."
And that's a fine argument :-) except that in my "industrial" niche, the vendors tend to sell all sorts of different TS sensors and controllers, given by the fact that PCap unfortunately doesn't fit all the uses. $DEITY bless Linux for having vanilla drivers for all the TS controller brands out there - but, the one missing bit is a reliable and up to date vanilla calibrator in X. I need a calibrator in Linux maybe once in two years and it's always a different buggy story.

xtcal - promising but biased

Okay I'm getting to the point now :-)

As of the time of this writing (early June 2018), the xtcal seems pretty promising.
It purports to use an algorithm based on "least squares" to derive the "best fit model", which it then turns into the desired "coordinate transformation matrix".

Except that the result wasn't flawless on my hardware. Xtcal compiled just fine on the build machine, I turned it into a quick and dirty .deb package (amd64) for the target box, it installed without a hitch, it ran just fine...
But the result is off. No matter how hard I try to touch the precise center of the crosshair marks, and avoid bias on my part due to angle of view, the resulting alignment (matrix) seems to suffer from a small rotational error - like 5 degrees up on the right side.
Based on what xinput_calibrator says, my X axis is mirrored, but that shouldn't matter to a generic calibration algorithm.
This might be a minor bug in the xtcal. I haven't reported the bug yet - I will revise this text if xtcal gets corrected.

The math is a little over my head... See the references chapter at the end of this text. And there's hardly any formal documentation of the "matrix" from X.org.
I generally understand that my clicks on the crosshair marks, presented by the calibrator app, provide an approximate mapping between TS sensor coordinates and scren pixel coordinates. These have to be "fitted" by some algorithm to a shape that in theory best describes a possible "composite affine transformation" = a combination of the elementary affine transformations allowed. Note that perspective transformation is not among them (not in 2D + 1). Once the measured data makes a plausible "source image", the rest is down to some matrix math - multiplications and gaussian eliminations.
Perhaps see the math.stackexchange article for the best explanation of the algorithm.

Perl with crude ballparking to the rescue

In need of a calibration tool, I ended up writing a simple wrapper script in Perl, a wrapper around the venerable xinput_calibrator. The script uses the calibrator's "debug" output, which lists the actual (miscalibrated) touch coordinates coming from the kernel's input layer. It also finds out the screen resolution and leverages inside knowledge of the pixel positions of the crosshair marks in the xinput_calibrator's window.

The script uses a set of formulas taken from the Arch Linux Wiki "Talk" on matrix calibration (see the references down below) to turn the axis-aligned "rectangular" four point calibration readings into a plausible matrix.
The four points used as input are preprocessed in the script for an "average fit" axis-aligned rectangle. Thus, the matrix is based on "optimal" and sanitized four points, theory-adherent in the geometric sense, which should result in a clean calibration (no foul residues from the transform).

Example output of my calibrator script in the text console:

root@debian:~# matrix.pl 
Using 'xinput list' to find some pointer devices, to give you a choice:
1.) Virtual core XTEST pointer
2.) Genius Optical Mouse
3.) DIALOGUE INC PenMount USB
Which device do you want to calibrate? Type a number and press <enter>
3
You chose 3 = device "DIALOGUE INC PenMount USB"
Screen resolution: 1024 x 768
We'd better cancel previous calibration, by inserting a unity matrix.
Launching the xinput_calibrator.
Click 0 : X=818, Y=154
Click 1 : X=203, Y=149
Click 2 : X=818, Y=617
Click 3 : X=204, Y=604
Matrix = -1.24979658258747, 0, 1.12337266069976, 0, 1.25490196078431, -0.122549019607843, 0, 0, 1
Will try to apply the matrix immediately. This is still just a runtime setting.
Matrix update cmd: xinput set-prop "DIALOGUE INC PenMount USB" "Coordinate Transformation Matrix" -1.24979658258747, 0, 1.12337266069976, 0, 1.25490196078431, -0.122549019607843, 0, 0, 1
By now, the input device might as well be calibrated.
To check, try     xinput list-props    .

To make the changes permanent, create or edit a file called
    /etc/X11/xorg.conf.d/99-calibration.conf
and make it look like this (feel free to copy+paste):

Section "InputClass"
        Identifier      "calibration"
        MatchProduct    "DIALOGUE INC PenMount USB"
        Option          "TransformationMatrix" "-1.24979658258747 0 1.12337266069976 0 1.25490196078431 -0.122549019607843 0 0 1"
EndSection

Lo and behold... the calibration seems spot on.

Remaining unresolved issues

The matrix.pl script in its current incarnation doesn't support rotation.
Not even "integer multiples of 90 degrees".

I may return to the affine transforms someday later. I mean I should probably stop whining and do something constructive. Maybe I could leverage the Matrix PDL library to help me with all the matrix math - it's got matrix stuff and even specifically the projective transforms. Apparently you can specify a tranform by providing just the two sets of points (source and destination) :-)

It's making me wonder if there's any point in supporting rotation other than 90 degrees and multiples. In most cases, the actual rotational mis-alignment of the touchscreen will be less than the inaccuracy of a human finger doing the calibration on an otherwise pretty "X/Y axis aligned" sensor. Except for sensors that are clearly defective (appearing in our RMA dept.) and those aren't worth calibrating anyway (not affine-compatible anymore).

So perhaps overfitting to an apparent tiny angular error (rotation) is less practical than just averaging for a perfectly "axis-aligned" rectangle.
Note that if the full matrix transform is used, this small distinction can be implemented as a choice of the "calibration input preprocessing algorithm":

Credits / prior art / further reading