Cannot find element with RPA.Desktop

Troubleshooting
#1

Hi, experts,

I’m new with RPA. I’m trying write my own automation test case with an outlook plugin. The process should be:

  1. Launch Outlook.
  2. Click plugin icon in Outlook Ribbon to log in. The text for the icon is “Log In”.

Below is my python script:
import time
from RPA.Desktop import Desktop
desktop = Desktop()

def lanuch():
    desktop.open_application("my application path and executive file here")
    time.sleep(10)
    desktop.click('ocr:"Log In"')
    time.sleep(10)

But when I run with robotframework, it shows:

TimeoutException: No matches found for: ocr:“Log In”

Troubleshooted:
1.Verifyed the application window was opened and the “Log In” icon also displayed on windows.
2. Tried “desktop.wait_for_element(‘ocr:“Log In”’)”, also cannot find this icon.

Ask for help:
Are there any steps I’m missing here? How to implement the python script find element via control’s text?

Thanks very much!

#2

I still stuck on click through control text, can someone help to check this? Thanks in advanced.

#3

Hi, @weiwei.cai3! Can you share your conda.yaml configuration file? I would like to see what dependencies you have there.

I’m thinking if any of these might affect the OCR:

Advanced computer-vision features such as image template matching and OCR require an additional library called rpaframework-recognition.

The dependency can be either added separately or through additional extras with rpaframework[cv]. If installing recognition through pip instead of conda, the OCR feature also requires tesseract.

I don’t have access to my Windows computer and Outlook, so can not test this myself right now.

#4

Thanks for your reply.
Where can I find the conda.yaml configuration file? I create my sample test case with Robot Framework and edit the python file and .robot file with notepad. There is no conda.yaml configuration file in my test suit folder.
rpaframework-recognition and tesseract libraries are installed through pip on my machine.

image
image509×626 11.4 KB

In addition, I tried use ‘Click ocr:“Archive”’ to click one of my inbox folder, it works fine. But the ocr:“Log In” for my outlook plugin doesn’t work.

Best Regards,
Weiwei

#5

Below is the screenshot of my plugin location in Outlook desktop application.

image
image1356×144 18.8 KB

#6

Hi

OCR locators are unreliable especially with buttons. In this case the OCR might not work, because text if divided into two lines. Could you use Windows locators or images instead?

There is also a dedicated library for handling Outlook tasks in Windows: RPA.Outlook.Application. I don’t know if it would help you.

#7

Just quick comment, could you try to put newline into string you are finding ocr:"Log\nIn" because texts are in separate lines ?

#8

Hi Teppo,

Thanks for your suggestion, I checked the RPA.Outlook.Application library, I don’t found click button related keywords.

#9

Hi mika,

I tried ocr:“Log\nIn”, but still no luck. :frowning:
Thanks.

#10

I don’t think OCR can work with this. You will have to fallback to windows locators or image locators.

#11

@weiwei.cai3
In addition to what @Teppo mentioned. Check if application has shortcut for the action you want perform. They are usually the easiest way to navigate places in the app.

https://support.microsoft.com/en-us/topic/keyboard-shortcuts-for-outlook-3cdeb221-7ae5-4c1d-8c1d-9e63216c1efd?ui=en-us&rs=en-us&ad=us

#12

As that “Log In” is in the Home “ribbon”. Maybe pressing keys ALT+H and then L might work ?

#13

Actually… press that ALT+H while in the Outlook application and it should show you the shortcut key what to press next.

Capture
Capture1709×116 11 KB

#14

Hi mika,

Thanks for your suggestion. I can use shortcut key to navigate to the UI. Since I’m new for RPA.Desktop, I’m not sure if I use the correct syntax or if I miss something. But now I can confirm the syntax and usage is correct. :slight_smile:

Thanks for all guys to help me check this issue. And I have reported this issue on github, before this issue resolved, I will use screenshot or shortcut to navigate to UI: ocr:"Log In" doesn't work if the Log in text in two lines · Issue #141 · robocorp/rpaframework · GitHub

Thanks again.