App input options - press and hold element

[smarter-phone]

Apps with elements that aren't supported by accessibility services can't be controlled with autoinput (field cannot be clicked)

Is there any way of interacting?
Assume also no intents; also this app has kb shortcuts AND midi mapping... but not for a required button...

ideally I'd control it by identifying element, not just location, to support multi-app / split-screen / float, but if possible only using xy coordinates I'll take that!

Rooted S20 android 13

Many thanks!

 

[MacroDroidDev]

I'm not entirely clear what you mean by "Apps with elements that aren't supported by accessibility services", can you provide an example?

In general the UI Interaction gesture can work based on X,Y locations without needing to identify the individual UI controls.

 

[smarter-phone]

That's what i read online, after reading autoinput can can do anything, and may explain which apps can be controlled with that action...

I've just found and tried actionv2 which works well , with the xy bit... and it seemed to find the related class (but that may not affect usage, eg still requires full screen with element at expected position)... after providing the secure setting permission using adb (locally! : ) the action succeeded, and a quick test whilst the app was not fullscreen also worked