File name detection using pytesseract and pyautogui in Python

Troubleshooting
1

import os
import time
from PIL import ImageGrab
import pytesseract
import pyautogui
from pywinauto.application import Application
import pywinauto

desktop_link_path =“C:\App\Appexe”

Wait for the path to load and take a screenshot

time.sleep(2)
os.startfile(desktop_link_path)

Wait for the window to open

time.sleep(2)

folder_path = “C:\poc\poc\pictures”
screenshot_filename = “screenshot.png”
screenshot_path = os.path.join(folder_path, screenshot_filename)

screenshot = ImageGrab.grab()
screenshot.save(screenshot_path)

Set the coordinates of the area to search for the file name

search_area_coordinates = (0, 0, screenshot.width, screenshot.height)

Define the file name to search for

filename_to_search = “File_name_1”
search_area_screenshot = screenshot.crop(search_area_coordinates)
search_area_text = pytesseract.image_to_string(search_area_screenshot)
print(search_area_text)

While applying ocr the text detection is not always the same and i got the file name missing the first _ that’s why I added this code while searching for the file name

if “File” in search_area_text and “name” in search_area_text and “1” in search_area_text:
filename_to_search = “File_” + search_area_text.split(“File”)[1].split(“name”)[0] + “_name_1.exe”
file_name_location = pyautogui.locateOnScreen(filename_to_search)
if file_name_location is not None:
# If the file name is found on the screen, click on it
file_name_center = pyautogui.center(file_name_location)
pyautogui.click(file_name_center)

But still it doesn’t work. For those who have worked with pyautogui and pytesseract what seems to be the problem in this code ?

2

Hello,
Have you considered using Robocorp tooling for your automation project? There are already created libraries, examples and guides for automating desktops, for example:

3

Hello raivo, thank you for your answer.

I wanted to use python as the main language for my project. I have changed the code above to a simpler one to be able to run the task. However, I am considering the use Robocorp tooling for the next steps in addition to python.