Web scapper tutorial error


I was trying to run the tutorial, but I end up getting this message, does any know how to solve this?

4 Likes

Hi, kenglimzero!

Thank you for the question! The issue here was the robot was very naive and expected the cookie notice to always appear.

However, depending on browser settings (and possibly plugins), the cookie notice might not always appear.

I updated the tutorial to handle this scenario better. The change was small but significant. This call:

Accept cookie notice

…was changed like this:

Run Keyword And Ignore Error    Accept cookie notice

Your feedback improved the robot quite a bit, thank you! :slight_smile:

4 Likes

I am not getting this error, but another one with the same tutorial. When I run this robot on cloud, the browser is not loading up the page properly hence the step where tweets are captured fails. This works fine on my local machine. Any help would be appreciated. Thanks!

Hi, @oberoi.harpreet!

The reason for the failure is that the browser in the cloud seems to run in a small resolution, causing Twitter to render in mobile layout, where things are a bit different than on a larger resolution.

EDIT: Actually this might be caused by something else (although the sizing issue exists), but investigating anyway :sweat_smile:

The Maximize Browser Window keyword should fix this in theory, but it does not seem be doing anything in the cloud container. We try to see if this can be fixed.

Thanks for reporting this!

// Jani

Update: Unfortunately it seems that the robot will never work in this form from the cloud, since Twitter blocks the requests coming from there. So this is something you run only on a local computer.

Decided to improve the good old bot a bit. Now it can get rid of unnecessary UI elements to get better screenshots, and scroll down the page to load the dynamic content. Also simplified the code somewhat.