UI Interaction not working for specific apps

[chaoscreater]

I've been using UI Interaction for unlocking my phone screen and it works fine.

However, I was trying to set this up for unlocking Teams and it doesn't work. At my company, we use Teams and it enforces InTune policy on your device, for example it would require that you configure a PIN for Teams. When you open Teams, it would force you to type in the PIN, kind of like what banking apps do. For some reason, the numbers on the PIN popup aren't able to be interacted with using UI Interaction. It doesn't find the controls at all. I've tried various different options and it just wouldn't click into anywhere in the PIN popup.

I've tried to use the Paste option to paste the PIN unlock digits and it doesn't work either.

I've also tried using a shell script to input keyevent (to enter the PIN number) and it doesn't work.

Does anyone have any ideas?

 

[Qarboz]

I've been using UI Interaction for unlocking my phone screen and it works fine.

However, I was trying to set this up for unlocking Teams and it doesn't work. At my company, we use Teams and it enforces InTune policy on your device, for example it would require that you configure a PIN for Teams. When you open Teams, it would force you to type in the PIN, kind of like what banking apps do. For some reason, the numbers on the PIN popup aren't able to be interacted with using UI Interaction. It doesn't find the controls at all. I've tried various different options and it just wouldn't click into anywhere in the PIN popup.

I've tried to use the Paste option to paste the PIN unlock digits and it doesn't work either.

I've also tried using a shell script to input keypress event (to enter the PIN number) and it doesn't work.

Does anyone have any ideas?

Have you tried using XY coordinates?

 

[chaoscreater]

Have you tried using XY coordinates?

Yes. Tried all the options. X,Y location works fine for other apps but just not Teams.

I think Teams is a bit special, because it uses another app (Company Portal) to detect if your phone is rooted or not. My phone is rooted, but I've also hide root from Teams and so I can use it just fine. But because of the extra security measures, I'm wondering if the PIN popup blocks any other app from interacting with it, including blocking screenshot, etc.

 

[Endercraft]

Because of the extra security measures, I'm wondering if the PIN popup blocks any other app from interacting with it, including blocking screenshot, etc.

If that's the case then it may be made on a canvas just for that purpose.

 

[chaoscreater]

If that's the case then it may be made on a canvas just for that purpose.

But surely MacroDroid with root should be able to UI interact with that canvas. Just because apps have extra security measures, such as being able to detect whether your device is rooted, doesn't mean they can't be bypassed. Plenty of banking apps detect root and they can all be bypassed. I even bypassed Microsoft's Company Portal from detecting root and spoofed location for various apps that can detect root + mock location settings. All of this is possible.

 

[Endercraft]

I'm pretty sure some apps can do that without root. But yes it's possible to fake a press using root. I think it's possible with adb too.

 

[chaoscreater]

Found my solution:

Clickmate - Auto Clicker Macro - Apps on Google Play

Auto Clicker that allows to record and repeat your touch and taps!

play.google.com

It's a very basic tool and easy to use. It supports plugin for MacroDroid and Tasker. You can just record your clicks/taps on the screen and then save it, then invoke that from MacroDroid. Works fine with the Teams pin code popup.

The lifetime license for this is rather pricey, at $30. Ended up finding a "free" copy online if you know what I mean. I'd rather pay $30 to MacroDroid or Tasker devs and I have actually, own a couple of licenses just to show my support.

Anyway, I tried to use the TouchTask plugin to get the UI elements on the screen. However, you have to tap on the TouchTask notification in order to trigger it, which means you need to swipe down from the top of your phone. When Teams PIN popup prompt is shown, it prevents you showing the notifications. Swiping down doesn't do anything. So I had to use Read Screen Contents action in MacroDroid. Once I got the UI element, it's just a simple of if condition to trigger the PIN unlock with ClickMate. Easy.

 

[MacroDroidDev]

It's on my long term todo list to better improve handling of UI Interaction so it can compete with stuff like clickmate but it's a biggish task so no timescales on this I'm afraid.