[Bug 1672465] Re: Onboard doesn't work in Wayland

[Jeremy Bicha]

** Also affects: onboard (Ubuntu)
Importance: Undecided
Status: New

** Tags added: wayland

** Changed in: onboard (Ubuntu)
Importance: Undecided => High

** Changed in: onboard (Ubuntu)
Status: New => Confirmed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[Francesco Fumanti]

** Changed in: onboard
Status: Confirmed => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[Sebastien Bacher]

Could somebody from the onboard team gives a status update there of how
onboard work with gnome-shell and under wayland?

Did anyone try to replace caribou with onboard in gnome-shell, is that
doable/easy? How do they compare usability/feature wise?

With the switch from Unity to GNOME we need to have a look at what osk
we can/should use, the onboard team has been great over the years and it
would be nice if we could keep working together. That discussion might
be a bit off topic for this bug report though so let me know if you
would prefer to move it to another report, a mailing list, direct email,
...
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[Francesco Fumanti]

Hi Sebastien,

First of all, thanks for you appreciating our team and we also hope to
continue the cooperation with you.

As Onboard has more features than Caribou, there already has been some demand for its integration into GNOME and people have also done some work on it; for example:
https://stackoverflow.com/questions/33065735/replacing-gnomes-virtual-keyboard-with-onboard-hide-show-via-dbus-doesnt-wor

In the meantime, Onboard is shipping an own extension for its integration into GNOME shell.
https://launchpad.net/ubuntu/xenial/amd64/gnome-shell-extension-onboard
https://bugs.launchpad.net/onboard/+bug/1538627

The release for zesty does not fully support GNOME shell yet, but since
Ubuntu has decided to switch to GNOME Shell and Wayland, these two items
have become high priority for us and marmuta is currently actively
working on the port. He told me that he made some progress, but he is
better placed than me to give you details on it, so I leave it up to
him.

As far as I know, Lightdm is also being replaced by GDM, which will also
be relevant for Onboard...

Thanks again for contacting us.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[marmuta]

Did anyone try to replace caribou with onboard in gnome-shell, is that doable/easy?

I did, and came to the conclusion that fully replacing caribou with the existing Onboard is close to impossible. That's mainly because applications don't have access to the activities and authentication views. Can't search applications or enter passwords with Onboard. In Wayland there was the additional problem of gdk_wayland_window_set_accept_focus not being implemented and Onboard uselessly sending key-strokes to itself.

There is an alternative approach, though, which I tried to outline in #4. The idea was to embed Onboard directly into gnome-shell via shell extension and custom clutter actors.
The good news is, that yes, it works. See attached screen-shot with the new Onboard in the Activities view. It handles similar to the stand-alone application, can be docked to screen-edge or left floating, resized and moved, etc. Focusing is not an issue and it can be shown anywhere at any time.

There's plenty of work left to do, though. My goal has been to get
something usable in X and Wayland before Artful feature freeze. It is
already starting to get there in X, but problems remain in Wayland.

Wayland's input separation makes it hard to find replacements for anything that generates input events, key-strokes, clicks or wants to receive global clicks and key-presses. Accessibility features will probably take a hit. Mousetweaks isn't getting ported to Wayland for the same reasons, so no hover-click support for now. It's unclear if we will find replacements for the click mapping buttons (right-, middle-, double-, drag-click).
Write access to /deb/uinput would help a lot in that regard, but this undermines Wayland's security.

Other than that, we'll probably get most of the old Onboard's features
ported, including the new emoji palette, which hadn't been released yet.

Enabling Onboard in gnome-shell will mean having the Onboard_Indicator
extension installed and enabled. If disaster strikes, all that's needed
to return to the shell's built-in keyboard is to disable the extension.


** Attachment added: "The new onboard in gnome-shell, Wayland."
Loading Image...
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[Jeremy Bicha]

Personally, I would vote for Onboard to be integrated into GNOME
upstream instead of Caribou.

Caribou is very deficient compared to Onboard and has not really been
actively developed much. (There was a GSOC proposal to do some things
but I am guessing the student could probably end up working just as
easily on Onboard in GNOME instead.)

Two prominent examples: I believe Caribou only supports a very small
number of hard-coded keyboard layouts. Caribou is a fairly simple
onscreen keyboard that does not have as many accessibility features as
Onboard.

So Caribou can work if you just want a simple English onscreen keyboard,
but if you need a different keyboard layout or more accessibility
options, it's not very helpful at all.

Therefore, I recommend that the Onboard team open a discussion with the
GNOME Shell developers about using Onboard's core to replace Caribou.
(That could help with the Wayland difficulties since the necessary
integration could be built directly into GNOME Shell.)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[marmuta]

Let me give you an update on the current state of Onboard in gnome-
shell.

First, things changed - a lot. The work is happening in a new independent project OnboardOSK at
https://launchpad.net/onboardosk
This is basically a fork of Onboard, written mainly in C++. There's no Python involved apart from supporting tools.

At the project's core is the (largely) toolkit-independent shared
library libonboardosk.so. It contains an onscreen keyboard closely
resembling the original Onboard in appearance and, well eventually,
features.

A second shared library libonboardoskgnomeshell.so implements the
toolkit-dependent parts, here by integrating with Clutter and GObject
introspection.

A new GNOME Shell extension then embeds OnboardOSK into the shell and
provides the shell's services to the shared libraries below.

The end result is an onscreen keyboard that is already quite
indistinguishable from the old standalone Onboard, but, and that's a big
but, it's not done and realistically won't be before Artful feature
freeze.

So what works, what's missing? The majority of the old Onboard's
features are ported by now, e.g., key logic, word suggestions, emoji
palette, themes, color schemes, etc. are there. "window" (actor)
handling has become more complete and smoother. Hover-click does
something even without mousetweaks (for Wayland), but is limited to the
keyboard itself.

The main missing things are
- long-press popups (in progress currently)
- click-mapping buttons (right-, middle-, double-, drag-click)
- modifier+click for accessibility
- keyboard scanning
- a bunch of dialogs: snippets, language model error recovery

Most worryingly, however, Wayland support is still lagging behind X.
Developing in Wayland is a bit of a pain because there is no easy way to
restart GNOME Shell (that I'm aware of). Reloading extensions is
insufficient as this won't reload shared libraries. Therefore my
strategy has been to get as much as possible done outside of GNOME Shell
in Wayland, replacing with GNOME Shell in X and test_stage.js in Wayland
and switch back later. Well, that's going to be soon, but the time is
up.

So, apologies, I tried to make it happen for 17.10, but it'll take more
time. We'll very likely make the 18.10 release, however, as default or
not.

FWIW, this message was typed with with OnboardOSK in GNOME Shell (X).
What is there seems to work alright. No crashes lately, no noticeable
leaking (unlike Python Onboard).
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[marmuta]

** Also affects: onboardosk
Importance: Undecided
Status: New

** Changed in: onboardosk
Status: New => Confirmed

** Changed in: onboardosk
Importance: Undecided => High

** Changed in: onboard
Status: In Progress => Won't Fix
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions

[marmuta]

Post by marmuta
We'll very likely make the 18.10 release

I meant to write:
We'll very likely make the 18.04 release
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1672465

Title:
Onboard doesn't work in Wayland

To manage notifications about this bug go to:
https://bugs.launchpad.net/onboard/+bug/1672465/+subscriptions