Inconsistent mouse events for Acer T231H multitouch monitor

I had hoped that backing http://cgit.freedesktop.org/xorg/xserver/commit/?id=634b0da9a83076d0e9e0fc44dc5dc77b0c368bc1 out of the xorg server core code base might be enough to solve this, but if I do so, core events never have any buttons set in their state, so that is not a solution. Nevertheless, it might well be that this problem lies somewhere inside the xorg server itself, and not the evdev driver module. Although I haven't yet pinpointed the location in the sources where that state is managed, it seems to me as if the evdev code were pretty stateless, submitting only button press and release events as appropriate. And as those apparently arrive correctly at xev, the state management appears to be wrong for some other reason.

Searching for ways to obtain more information, I just ran "xinput test-xi2 2" where 2 is the id of my virtual core pointer. When I drag my finger inside the test window, here is what I see in the console (after enlarging my scrollback appropriately):

 - DeviceChanged due to SlaveSwitch
 - RawTouchBegin
 - Enter, buttons: (empty)
 - Motion (emulated), buttons: 1
 - ButtonPress (emulated), buttons: 1
 - Motion (emulated), buttons: 1, repeated while dragging
 - ButtonRelease (emulated), buttons: 1
 - TouchBegin, buttons: (empty)
 - TouchUpdate: buttons: (empty), repeated to mimic the original drag

There are a number of things very strange about this:
 - The whole TouchBegin/TouchUpdate sequence only gets emitted after I release my finger.
 - The TouchBegin does not have a matching TouchEnd. Very likely this causes the error in core event state.
 - After I release my finger, the mouse cursor jumps to the location where I began my drag.
   This behaviour is only within the xinput test window, outside the cursor will stay at the end of a drag.

Repeating the test for input device 9, which is the touch screen, I see this:
 - RawTouchBegin
 - Motion (emulated), button: 1
 - TouchBegin, buttons: (empty)
 - ButtonPress (emulated), buttons: 1
 - RawTouchUpdate
 - TouchUpdate, buttons: (empty)
 - Motion (emulated), buttons: 1
 - The preceeding three events repeated in this order for the duration of the drag
 - RawTouchEnd
 - TouchEnd, buttons: (empty)
 - ButtonRelease (emulated), buttons: 1

So here the Touch events are delivered in sync with the Motion events, there is a TouchEnd where one would expect it to be. The jumping of the mouse cursor to the position where the drag began still remains, though.

To me this looks a lot as if the events made it into the dix layer all right, but the emulation of a core pointer based on these events is somehow broken. For this reason, I now believe the problem to lie within the xserver-xorg-core binary package.

"xinput test 2" does not work at all ("unable to find device 2"), whereas "xinput test 9" shows pretty sane behaviour: a single motion followed by a button press 1 at the beginning, a sequence of motion events during the drag, and a button release 1 at the end. When launched with the -proximity flag, it complains of a bad request and terminates immediately. Makes sense, as this device doesn't provide any proximity information.

Status changed to 'Confirmed' because the bug affects multiple users.

The same behavior on another hardware is reported in this question:
https://answers.launchpad.net/ubuntu/+source/xserver-xorg-input-evdev/+question/201072
concerning the wetab tablet with egalax touchscreen

Similar behaviour, but not entirely the same: I do receive press events. The incorrect status is the same, though.

Does `xdotool mousedown/mouseup/click/etc` or similar commands using `xte` change the status back to 0x000?

They do not help for the egalax touchscreen.

OK, I bvelieve I've identified the main problem here:

http://cgit.freedesktop.org/xorg/xserver/commit/?id=a986f2f30cbe2a00e72ded7315c4951d7703e549
http://anonscm.debian.org/gitweb/?p=pkg-xorg/xserver/xorg-server.git;a=commitdiff;h=96d8df5bc9d400d55830b23afe5525b222f8dfc7

Due to that commit, the ET_TouchEnd branch of UpdateDeviceState never receives the TOUCH_END flag it expects, so it never lifts the button. In fact, there are currently only two references to the TOUCH_END macro in the whole upstream xorg git tree: one its definition in include/input.h, the other this check in Xi/exevents.c. There is noone left ever setting this flag.
http://cgit.freedesktop.org/xorg/xserver/tree/include/input.h?id=a986f2f30cbe2a00e72ded7315c4951d7703e549#n80
http://cgit.freedesktop.org/xorg/xserver/tree/Xi/exevents.c?id=a986f2f30cbe2a00e72ded7315c4951d7703e549#n974

The reodering of events which the commit does will also break the state reported for the release event. As far as I understand things (and as it behaves for my regular mouse), the state should reflect the state of buttons BEFORE the event happened. There is a comment to that effect somewhere in the X sources. Reordering the calls in ProcessTouchEvent will cause the state to be already 0 for the release event, where it should be 0x100 still.

As the original commit originated from Chase Douglas who is a member of the ubuntu-x-swat team, I suppose he will read this message here and might provide additional information as to how his change was intended to work, and whether backing it out of the code will likely break things for other users. If he doesn't usually follow bug mail, someone please make him aware of this issue here.

The only remaining problem is the fact that the ButtonPress event has state 0x100 already, where it should have state 0x000 to match the lack of pressed buttons preceeding that event. I'll see whether I can pinpoint that problem as well. If I can, I will attach a patch including that fix as well. Otherwise a simple reverse application of the commit I pointed out above will be enough.

@Stefan Krastanov, please see if this fix works for you, otherwise please open a new bug report for your issue.

OK, I believe I now have a good idea as to why the first ButtonPress in xev has state 0x100. This appears to be due to an TouchOwnership event. If ownership changes, such an event is inserted into the queue, so it will always be processed after the BeginTouch event. I wonder how much later it might get processed, but as there appears to be some history, I simply assume everything is all right there. So the TouchBegin arrives, sets state = 0x100, enqueues an Ownership event, and when that arrives, ProcessTouchOwnershipEvent will first run TouchRejected. This in turn will call EmitTouchEnd, but will not update state. Then TouchPuntToNextOwner will replay the history. The ButtonPress core event corresponding to the initial TouchBegin is now delivered with the CURRENT device state, i.e. 0x100.

For source code of TouchRejected, see
http://cgit.freedesktop.org/xorg/xserver/tree/Xi/exevents.c?id=a986f2f30cbe2a00e72ded7315c4951d7703e549#n1210

I haven't come up with a fix yet, as I haven't fully understood this whole ownership and history/replay business here. But I assume that a proper fix would be restoring the state to what it was at the beginning of the history before replaying said history. Input from people more fluent in that code would be greatly appreciated.

I have a patch for the second issue of the mouse state during replay. Will attach it here shortly. I'm currently trying to get this to my PPA, for quantal first and for precise afterwards.

Editing the change log, I found that the removal of the TOUCH_END is not part of the orig tarball, but comes from an ubuntu patch, 507_touchscreen_fixes.patch. The changelog entry by Chase Douglas quotes bug #974887 as a reference on those modifications.

OK, here are the two patches which I currently use to get a working core pointer from my touch screen. I've included them in packages available from https://launchpad.net/~gagern/+archive/ppa for precise and quantal, so feel free to use those to give things a try.

The attachment "Reverse commit which removed TOUCH_END" of this bug report has been identified as being a patch. The ubuntu-reviewers team has been subscribed to the bug report so that they can review the patch. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-reviewers team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

As the core pointer button state appears to be exclusively maintained inside the x server core, the evdev driver is blameless.

Also affected: Atmel maXtouch (see https://answers.launchpad.net/utouch/+question/183765)

I applied the patches in #11 and #12 to the current precise xserver-xorg-core source (1.11.4-0ubuntu10.2) and unfortunately, it does not solve the atmel maXtouch "loss of single click" (I can reproduce that behavior systematically by calling Compiz's Expo plugin and double-clicking one of the viewport; after that single-click is lost).
Worse, drag'n drop does not work anylonger (e.g. to move a window by cliking-holding-moving its title bar).
Sorry for the negative report.

Hi Martin,

Thanks for finding this issue. Good work on making some patches, but I don't think they are quite right. We need to set the internal button state of the device when the touch begin comes in, otherwise we will not get the right result when we call {X,XI}QueryPointer().

A simpler solution would be to modify DeliverTouchEmulatedEvent(). There is the following line:

ptrev->device_event.corestate = event_get_corestate(dev, kbd);

We need to modify it so that it also passes in a boolean to event_get_corestate() to determine whether to mix the touch emulated button state in with the normal button state. Then, we tell the function to mix in the touch emulated state only on touch update and end events. We can reason that this is correct because only one touch may be emulated at a time per device, and because only one window may receive an emulated button press for the touch begin at a time.

I will attempt to make a test and a patch for this tomorrow.

Thanks!

Upstream patch has been posted here:

http://lists.x.org/archives/xorg-devel/2012-July/032280.html

Hello Chase,
Thank you for looking into this.
When not subscribed to the X.org mailing list, obtaining the patch is kind of a hassle (I couldn't find a download option and I'd rather avoid copy/paste). Can you post it here too, so we can download/test it? Or will it be included in the next xserver-xorg-core package update?
Thanks again,
Cédric

Hi Chase,

thank you for looking into this, and working towards a solution.

Is the patch you referenced intended as a replacement for both my patches? Working with the touch screen alone, things work out fairly well. But if I also use a mouse, then the lack of a TOUCH_END event still causes the mouse moves to register as drags, i.e. with state 0x100. So I feel like your patch might be a suitable replacement for the one in comment #12, but the one in comment #11 is still required.

I guess you head a good reason to write the changes I mentioned in comment #8, the ones which #11 backs out. I must confess I still haven't fully understood the rationale behind that commit, mostly because I still don't know about that listener concept. So perhaps you want to improve that change instead of reverting it. A proper proof should probably get rid of every occurrence of the TOUCH_END macro in the code. It should be tested on a setup with both a touch and a classical mouse device.

With that patch by Chase, I also wonder what would happen if two touch devices were used simultaneously.

When I both touch my screen and click my conventional mouse button, I get two button events, but the state is the bitwise or of both, so the second click will be "pressed button 1 while button 1 was pressed". Makes sense, in a certain way, although it might well confuse some applications.

But what if I had two touch devices controlling the core pointer? I believe it is likely with the current setup that in this case, the second device would generate a ButtonPress with state 0x000. Or is it impossible for more than one touch device to control the core pointer at a given time? I don't have more than one device, so I can't test this.

I'll agree that given current technology, this seems a rather rare corner case, and probably not worth the effort to deal with properly. But any shortcomings in that respect should perhaps at least be documented somewhere.

Cédric, I attached the patch, after copy & paste from gmane. But it applies all right, at least to the xorg-server-1.12.1.902-1ubuntu1 currently in quantal. For precise, some adjustments are required.

I updated https://launchpad.net/~gagern/+archive/ppa to provide fixed versions of the latest packages, and they use the patches from comment #11 and comment #19.

Hello,

Thanks Martin for your PPA and the precise packages; unfortunately, I run amd64 :-/

I first applied only #19 patch (yes, it did require some adjustments) to the latest precise X.org packages (1.11.4-0ubuntu10.3): behavior remains the same (at least as far as Compiz plugins are concerned: "Expo" triggers the issue systematically, "Shift Switcher" one time out of two).

Then I saw Martin's comment and applied patches #11 and #19: behavior remains the same (as far as Compiz plugins...) AND windows moving (clicking-holding-dragging) is lost right from the start.

I think we can conclude that #11 triggers the "windows moving (clicking-holding-dragging) is lost" issue (since I also bumped into it #17)

Now, I'm wondering whether Compiz might have some problem on its own. Example given: I loose single/double click as soon as I launch the "Expo" plugin and select (double-click on) a viewport. After, no amount of single/double-clicking does anything (on desktop icons, windows minimize/maximize/close buttons, panel applets, etc.). BUT, single/double-clicking still work in Compiz plugin (if I launch the "Expo" or "Shift Switcher", single/double-clicking works).

I also gave #19 patch (alone) a try without launching Compiz (use metacity alone instead) or multi-touch (ginn) stuff. Before, even in that (no-compiz/no-ginn) scenario, single/double-clicking got lost rather rapidly. Here, I've been clicking and double-clicking around and using applications for several minutes without any problem. As soon as I launch ginn again, I start to have some problems with the applets icons (most of the time, they don't react to single click, though they sometime - rarely - still do). Other single/double-clicking still works.
Launching Compiz alone (without ginn), and all problems are back.

Conclusion using patch #19, without patch #11 (cf. "windows moving (clicking-holding-dragging) is lost" issue), without Compiz (cf. "single/double-clicking is lost" issue) and without ginn (cf. "applets react only whimsically to single-click" issue), single/double-clicking at least seems to work reliably enough to allow the touchscreen to be used (before, it was a "no go"), even if sometime, one must (single) click two time on an icon for the expected behavior to happen.

About that "sometime, one must (single) click two time on an icon for the expected behavior to happen": I can reproduce the behavior systematically by: 1. clicking on a network-manager icon (menu opens), 2. clicking on the desktop to close the menu (menu closes), 3. clicking on a network-manager icon again (menu does not open; instead, a selection rectangle is drawn), 4. clicking on a network-manager icon again (this time, menu opens again)

Let me know if I can help any further.

Hi Cédric,

Although this took a little while longer, the amd packages have been built successfully by now, and are available from my ppa.

I have the impression that your problems manifest in cases when some application is listening to the higher level touch interfaces, presumably using XInput. My problems on the other hand occur in a bare bones X where applications only use the core pointer. Those appear to be two sides of a coin: the commit reverted by comment #11 was intended to deal with a changing set of listeners, which to me sounds like some gesture-interpreting software. So it seems you need that fix, but I can't have it, as it breaks the core pointer. Let's hope someone finds a fix which will work with multiple listeners and still maintain proper core pointer state. As things work all right for me at the moment, I have little incentive to work on a part of this issue that doesn't affect me. Particularly as I haven't got all this gesture stuff set up to work for me, so I cannot test things yet.

Hello Martin,
Yes indeed, it seems we're bumping is several different issues. Yours - and its fix - helped improve the global picture. I have absolutely no understanding of how all that X and multi-touch works, so I don't know where to start from (in respect with where/how to report the other issues). All I can do is give as detailed a report as possible, so those who know can extract what hopefully useful information they can (and maybe point some directions for further bug reporting and/or testing).
Thanks for keeping up with me so far ;-)
Cheers

On 07/04/2012 03:30 AM, Martin von Gagern wrote:
> Is the patch you referenced intended as a replacement for both my
> patches? Working with the touch screen alone, things work out fairly
> well. But if I also use a mouse, then the lack of a TOUCH_END event
> still causes the mouse moves to register as drags, i.e. with state
> 0x100. So I feel like your patch might be a suitable replacement for the
> one in comment #12, but the one in comment #11 is still required.

If you aren't receiving a TouchEnd event, then there is something wrong.
The patch you are reverting is needed to fix a different bug. If you can
reproduce the issue easily, please describe how. Then we can look to
resolve it.

Thanks!

On 07/04/2012 03:37 AM, Martin von Gagern wrote:
> With that patch by Chase, I also wonder what would happen if two touch
> devices were used simultaneously.
>
> When I both touch my screen and click my conventional mouse button, I
> get two button events, but the state is the bitwise or of both, so the
> second click will be "pressed button 1 while button 1 was pressed".
> Makes sense, in a certain way, although it might well confuse some
> applications.
>
> But what if I had two touch devices controlling the core pointer? I
> believe it is likely with the current setup that in this case, the
> second device would generate a ButtonPress with state 0x000. Or is it
> impossible for more than one touch device to control the core pointer at
> a given time? I don't have more than one device, so I can't test this.
>
> I'll agree that given current technology, this seems a rather rare
> corner case, and probably not worth the effort to deal with properly.
> But any shortcomings in that respect should perhaps at least be
> documented somewhere.

Great point here :). I had to look at the code to determine the correct
answer. I worried that, like you said, it would be buggy if you had two
touchscreens. However, this should not be a problem.

Input devices have a two-layer hierarchy. There are slave devices and
master devices. Slave devices represent a physical device, like a
touchscreen. Master devices represent a group of slave devices. Most
people only have one master device configured, and all the slave devices
are attached to it. If you have two mice, you can create a second master
and attach one slave mouse to each. Then you'll have two independent
pointers on screen :).

A client can listen to events from master and/or slave devices. The
events look and behave almost exactly the same. When you press your
mouse button, a button press event is generated for the slave device and
the attached master device.

If you have two touchscreens with one master, then each slave device can
emulate a button press event. However, there can be only one emulated
touch per device (no matter whether it's a slave or a master), so the
first touch is emulated for the master, and the second touch is not.

Thanks!

On 05.07.2012 23:35, Chase Douglas wrote:
> If you aren't receiving a TouchEnd event, then there is something wrong.

I am receiving the TouchEnd, but it doesn't update the core state the
way it should.

> The patch you are reverting is needed to fix a different bug. If you can
> reproduce the issue easily, please describe how. Then we can look to
> resolve it.

Please re-read comment #8.

Steps to reproduce:
 1. Start xev
 2. Touch screen and release
 3. Move regular mouse

Expected behaviour:
 2. Release event with state 0x100
 3. Mouse move with state 0x000

Actual behaviour:
 3. Mouse move with state 0x100

Behaviour if you simply disable the check for the TOUCH_END flag in
UpdateDeviceState:
 2. Release event already with state 0x000

I see this same behavior on a Samsung Slate 7, which has an Atmel maxtouch multitouch touchscreen. Running Precise, with xserver-xorg-input-evdev 1:2.7.0-0ubuntu1.2 and xserver-xorg-core 2:1.11.4-0ubuntu10.8.

If I don't touch the touchscreen since the X server has started, xev reports the state of events as expected when using a mouse. 0x000 for the ButtonPress, 0x100 for motion with the button down and the ButtonRelease, then 0x000 after. However, after the touchscreen is touched once, the state becomes 0x100 for the first ButtonPress from the touchscreen and stays at 0x100 thereafter, even after the ButtonRelease. Keyboard keys, mouse clicks, mouse motion, etc. are all state 0x100 until the X server is restarted.

Found out something new. I compiled the git version of evdev, xf86-input-evdev-2.7.0-20-g5af11b6, and when multitouch isn't enabled, the core state problem is gone. The autoconf script doesn't detect XI22 on Precise because the X.org version is too old, so MT isn't enabled by default.

Once I modified configure.ac to turn MULTITOUCH back on, then the problem was back as well and state is locked to 0x100 after the touchscreen is used.

I've also found that I get far more of the problems Cédric mentions when ginn is running. It seems like what happens is the touchscreen stops generating ButtonPress events, but it still does motion events and state is still locked at 0x100. So the effect is that one is moving the pointer around with the button held down, but never pushing the button. This usually seems like left-click has stopped working. But I imagine the state problem and the resulting "drags with no click" can explain in some strange behavior seen in certain apps.

Could you recheck on raring? It seems there have been some touch related fixes in xorg-server since the quantal xserver release.

note recent updates for the xorg bug here:
https://bugs.freedesktop.org/show_bug.cgi?id=56578#c17

I had similiar problems with ubuntu 13.04 on an Acer Iconia Tab W500.
The patches in https://bugs.freedesktop.org/show_bug.cgi?id=56578#c17 solved the problem for me.

I have partial (full) success (on the Lenovo Thinkpad Twist, an Intel-based convertible, see also bug 1068994):

I have rebuilt the current Raring package of xorg-server (1.13.3-0ubuntu5) with the following two patches:

1. http://cgit.freedesktop.org/~whot/xserver/commit/?h=touch-grab-race-condition-56578-v2&id=0498a4f0e0b90a850df7022a3356f10adabff855

(found via https://bugs.freedesktop.org/show_bug.cgi?id=56578#c17)

2. http://lists.x.org/archives/xorg-devel/2013-April/035878.html

and after that clicking via touch screen on the Lenovo Thinkpad Twist works reliably. Only remaining problems are (but the touch click ability does not get lost by them):

a. In Chromium when you create a new tab, the new tab contains icons for web apps (at least the app store and perhaps some examples). These icons cannot be clicked by touch, only with a mouse. All the rest in Chromium is clickable by touch.

b. Touch clicks do not work in XBMC, but after using and leaving XBMC with an external mouse on the normal desktop touch-clicking works again.

These are probably separate bugs which got revealed by the now working touch click.

Complete patch for xorg-server is attached.

This bug is perhaps duplicate of bug 1099289 or bug 1068994. I have attached a patch (for xorg-server) to that bugs which solves the problem on the Lenovo Thinkpad Twist.

Sorry, previous comment was meant for another bug.

Possibly bug 1099289 or bug 1068994 are duplicates of this one.

Sorry, patch is not complete. Here is the correct one.

Upstream marked this as fixed. So this should be fixed in saucy now. Backporting this is non-trivial, and it might be easier to test if using a newer version of onboard fixes this problem. Newer versions of onboard workaround it by using xi2 events directly.

For folks using Ubuntu 12.04LTS, and can't use the quantal/raring/newer X server (due to whatever reasons, like maybe a binary-only video driver for a "legacy" video card), this patch may work for you.

It's a first attempt at backporting Peter Hutterer's touch-grab-race-condition-56578-v3 branch. Some sections aren't pretty due to all the indentation changes, but it works for me. :) The patch was done mostly by hand.

In order to use the patch you have to comment out the following from the series file:
505_query_pointer_touchscreen.patch
506_touchscreen_pointer_emulation_checks.patch
507_touchscreen_fixes.patch

Let me re-state, this patch is for xserver-xorg-core_1.11.4-0ubuntu10.*.deb Specifically I based it off of xserver-xorg-core_1.11.4-0ubuntu10.14_i386.deb

Please let me know what you think. Thanks!
Chris

Can someone sum up what we have to do in 13.04 to get rid of the stuck click problem? I am new to ubuntu, I have installed ubuntu 13.04 on my nexus 7. When I dist-upgrade from ppa:canonical-x/x-staging, it just boot up to a black screen. I get notification or error and I have anotification that I am connected to my network but desktop is black. What do I do?

Hi all,

The patch I posted previously had a bug in xf86UnrealizeCursor(). I had accidentally used dixLookupScreenPrivate(), when I should have used dixLookupPrivate(). The attached patch is an update with the fix.

Chris

This is quite a severe bug. If you have a touchscreen connected, and you have touched it at some point:

1) start gitk or tkinfo or a recent Qt Creator (or probably many other Qt 5 programs)
2) scroll some long text with the mouse wheel
3) move the mouse

It selects text as if the left mouse button was being pressed. So in other words if you have a touchscreen your mouse wheel isn't very useful anymore in certain applications.

I think it should be fixed in 12.04 too, not just in newer releases.

It should be fixable on precise by installing xserver-xorg-lts-saucy.

The Precise Pangolin has reached end of life, so this bug will not be fixed for that release