Full memory consumption by python

Hi There,

I have a robot that performs quite a simple process with RPA.Browser.Selenium
But I notice that it consumes 38 GB of memory. (found out after release, because my development pc has 64GB)

I found out that it is because I was using a FOR ${counter} IN RANGE 1 9999999999 loop.
And I now changed it to WHILE ${True}

This post might help someone who is facing the same problem.
(and my now be closed)

Hi, thanks for the post @brian!

Worth to take a look at our docs on WHILE:

…it has some built-in stuff that you might not expect.

Also when creating infinite loops one always needs to be super careful the the “fire-exits are clear of any obstacles” :wink:

In Robot Framework the log building will eventually use up the memory if you do not stop at some point or control what you log. In some cases implementing the long-running loops in Python that is encapsulated by a keyword is the best solution, both for performance and logging.

BR, Kari

Hi @Kari,

How would you recommend to control what is being logged when using a loop in Robot Framework? I can think of setting the log level to WARN, but then I would lose the ability to easily log important steps as Info. What would be the best practices here?

Hi @brian,

If I understood correctly, changing from the FOR loop to WHILE solved your problem with the memory consumption, is that it? How much is it using now?

I have faced a memory leaks of sorts, but it is related to Chrome not dealing very well with an almost infinite scroll page. When I modified the robot to use Firefox, the browser stopped using so much RAM.

Hi @Vinicius,

One way of controlling the logging of Robot Framework is using:

…this enables marking specific keywords so that they are not logged.

I think the main point behind the “Big loops” topic is the simple question of Why?

We have seen multiple implementations that have these big loops just because the previous tool used to have them. So taking a step back and looking at what the bot needs to do, then splitting it up into understandable components, figuring out better triggering methods (e.g. idle polling is huge resource waste if you can avoid it) usually gets rid of these problems and yields a much faster and resilient bot.

BR, Kari

2 Likes

Hi! More options for controlling logging:

Flatten keywords:

Flattening keywords is done already when the output file is parsed initially. This can save a significant amount of memory especially with deeply nested keyword structures.

Remove keywords:

Removing keywords is done after parsing the output file and generating an internal model based on it. Thus it does not reduce memory usage as much as flattening keywords.

1 Like

Thank you so much for your input, @jani. Since in this particular case a big loop can’t be avoided, Flatten keywords and Remove keywords are exactly what I needed.

1 Like